From patchwork Fri Jul 8 14:54:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 62CF6C43334 for ; Fri, 8 Jul 2022 14:55:35 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363468.594008 (Exim 4.92) (envelope-from ) id 1o9pO0-0006vQ-7j; Fri, 08 Jul 2022 14:55:16 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363468.594008; Fri, 08 Jul 2022 14:55:16 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pO0-0006vJ-4u; Fri, 08 Jul 2022 14:55:16 +0000 Received: by outflank-mailman (input) for mailman id 363468; Fri, 08 Jul 2022 14:55:14 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pNy-0006vD-GK for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:14 +0000 Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150047.outbound.protection.outlook.com [40.107.15.47]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id f78601f9-fecd-11ec-924f-1f966e50362f; Fri, 08 Jul 2022 16:55:12 +0200 (CEST) Received: from AM5PR0402CA0013.eurprd04.prod.outlook.com (2603:10a6:203:90::23) by PR2PR08MB4713.eurprd08.prod.outlook.com (2603:10a6:101:22::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.17; Fri, 8 Jul 2022 14:55:10 +0000 Received: from AM5EUR03FT044.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:90:cafe::88) by AM5PR0402CA0013.outlook.office365.com (2603:10a6:203:90::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20 via Frontend Transport; Fri, 8 Jul 2022 14:55:10 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT044.mail.protection.outlook.com (10.152.17.56) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:09 +0000 Received: ("Tessian outbound 8e3d5168572a:v122"); Fri, 08 Jul 2022 14:55:09 +0000 Received: from 86cda3d7a080.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 17BC7FDF-488F-4B5D-8CA7-89AFB57836DB.1; Fri, 08 Jul 2022 14:55:03 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 86cda3d7a080.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:03 +0000 Received: from DB6PR0501CA0004.eurprd05.prod.outlook.com (2603:10a6:4:8f::14) by AM6PR08MB5287.eurprd08.prod.outlook.com (2603:10a6:20b:b0::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.14; Fri, 8 Jul 2022 14:54:59 +0000 Received: from DBAEUR03FT038.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:8f:cafe::e0) by DB6PR0501CA0004.outlook.office365.com (2603:10a6:4:8f::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.17 via Frontend Transport; Fri, 8 Jul 2022 14:54:58 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT038.mail.protection.outlook.com (100.127.143.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:54:58 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:54:57 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:54:55 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f78601f9-fecd-11ec-924f-1f966e50362f ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=dj1O6EmXJOY+M8DnRPih7/dBxbFEv5B3XYg7ujizs2fjCIqtb3LnzdsSVemnfBzHL6Yi9CNL0niHVaEiQ+zA0C12VO9rQuHFxSPA/ZuVGtyVuXp2mRxW9Ju8q54UcZspvolWq0128s3FTHI8h9SMHc2Nx4gUvcbIVfj/z4B9cqO8gpt+zZeXt87qPsEnwU4UjbDpc1MCg+t8rm7VGZdApPaRokMII/U1TWvck73DoZNckrbYotVa8tdNbrq7Aukg3Gd83cCClLuCY8bq3dBQEUpldNWr7k2rgwY6Yjf2/iUnaRazm5GOnv0n8YcbSImhwdqrJ9pW/k2SKS7h3UM3pg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Aj9rrheaoXEduUj4KYU6olJZRmwgTRCWkOezbyAQ/9k=; b=CdX08dIF1t+OzPMGHOfbdHfGyX/WB7XI0H7VPs4LGHzao3EgfdIgDO6GhyIN2Gx+f9HPOmwWhTaGCsgnpZFGcmI4/1N51NXElhNFFyVHqJWu6xWL285IFEbvy2AdD4oD1rj9bDdn+YjhHGon6nVMmwppwldtJwmOSOXX0THTAZtx/hJAtg51P9FD87xXeHb1ZCP99npRwUCLSwPE5PNf9es0jFIGq1AM2OhXNemFEtS1S/ga/c0FHIIfpgS6+M++BAOgzsmuljOdqGRSb8ElRnQ1rmyw/5TV++tfbgj7bTA4227KQAYQH+3s9BKdEg4pUGP4wLtFrXTrm7yIAHVkDw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Aj9rrheaoXEduUj4KYU6olJZRmwgTRCWkOezbyAQ/9k=; b=NzhRyH8M8/Vk6H1saVsVIb3/LqOXl/CumRIz7VuSLwYeVM8hr4J8MgOUKPsHWpRImRbtVxMFcQ/aOTZuuK0ML38lcRO87fA2flpfW8+u4NWq6kOSXrm0GOUJHYoXjQpHIY9GOhpiGxaQl5duugOhvFeolgFPXcCjxSVW5xOgYnU= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: d9e61ec64daefca6 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=i91Ow+/rBOw/ykikzC2hBDM1oCRvJLjx8E75C5oujeboLGSPZgbBDME85Bf5+FWUNy5ZhStPmt/dkaboWmymo7kbXycsUyZrQspZJkHch4r2vjKtpls0qwgmDgLIEOyX2U64r8ojQCuhFiL5L/fITZbLyfSo5asO4Ts1qThQKm5omcq8ur9GJJzpgWjul0MsRebxRa3kIigs7iQAfCefrw4/BA3b9UDZpgjTUkjpt1h9AJc4CfyLk7cx9KWvRJ8hS7h0gM5R+BDFbzC+UBvOYbBqEphPg0IIGtcjkHjkH4ORnlbTvRdsZesPV0zWoljb3I7zWN531iPG9j0iZ0uBrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Aj9rrheaoXEduUj4KYU6olJZRmwgTRCWkOezbyAQ/9k=; b=cJkgh39BvCfHn8c5CfEkcNtG850YgYxJZ2tkMVEX8ZaTeA+CHiNEi3lV16c7aVV2EaW1BdjnFpiCzAJ/EC3TIZ5DJYim3vNXO/QEW/195eq9KTPusszT+igTLZPiAji9XBDYYuvUkb7EvN7fgApXfInFShY1Sxq2vbAg2w5HQU8R9olgdPqq5vqTiZSDbUsNmk4B/vvwtjFTN7iS97/93nrp/NgCP9+tLCNKjH5QBKcE9WKogr+yNSQx3t2MIUgx7MY+hAgnbZiGTol6y/QrfIl2lHanG1/y11HQl+0Xh4uYjVbibMD1DgDryFcwHnlg4qXu9Omc8J4WwqnswoRygA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Aj9rrheaoXEduUj4KYU6olJZRmwgTRCWkOezbyAQ/9k=; b=NzhRyH8M8/Vk6H1saVsVIb3/LqOXl/CumRIz7VuSLwYeVM8hr4J8MgOUKPsHWpRImRbtVxMFcQ/aOTZuuK0ML38lcRO87fA2flpfW8+u4NWq6kOSXrm0GOUJHYoXjQpHIY9GOhpiGxaQl5duugOhvFeolgFPXcCjxSVW5xOgYnU= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v2 1/9] xen/x86: introduce a helper to update memory hotplug end Date: Fri, 8 Jul 2022 22:54:16 +0800 Message-ID: <20220708145424.1848572-2-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: cf32518d-7d79-4408-aa33-08da60f1da9f X-MS-TrafficTypeDiagnostic: AM6PR08MB5287:EE_|AM5EUR03FT044:EE_|PR2PR08MB4713:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: hveMIibSlmdUxpZQdFdSN1ID2n8hbNQCFqZxX7sIfjxj2bHHTlTYoVKCNOeHDAxxJhhvT7Sm74hjBBpPgVmNoY6klyq1Qy1pKsuYqYyEsFVvPsPyBr9bKtZiURzXC4nw1NNQH5C5QKGOQMNAZ4rH1LqcTrofYJbRjBInDqGN7guetAMSiQutWO5/ATu7ffZgIqZ6zmt4vHtSr60zQJx74a2DqhdKe/ML9rNg1tR+o3bGlbgR8P7Pob93nT8Se+Onib1hs42F62yf/plBIUJf5uO0NbkKuqFu0Un6WDSoJOw+AMemXIpuZCoGz5qMxLJKmok07+hal5P8WR5MJhaOM9h2VaPh4R9MTmfPXgFGDOGmytRL5N72ZleMUBGM4JgiyedXIOIjnNejGsopd+LOUKZy7ONenIHyN476ItB/vOALkHKOKo5GSQ8RIMz2vp9yv8NwHAGH5yV7+wcXepSZ/PwVqYw5YjupHZabwgSIAVO9PUgIfe7ohO61k+m1FSTDbwWOKHUeOBdyJN9ekxY7tb5cFMOzWvCXFSzAzDM6cdbTfUloVSwGXyqOhu2R+NrfCCDNliieRCfH3K+21EsfJVhYrouFTGycUSUHAAA3Kz8BYsdQ7NOM3+UVXnvBPncmfgYkyxAku5L0PC8BbtmPunoEJIBxBRPY2IJiLykrrUekUOzawLoPyj/Z0rCcP0j5EsCXqT0p1+4Jre3hAcyd2MPNRmR8YBsDO6tjkFE4fpt3+WoaJ1eY8mULarMz2p3eCDTxqwOXrqbE6BD8VDOgTzvKJVvw/dLPXc/9/Y+EKpODAyeOp6IQGLUzemh2LVdNU5FuaMER1MDgRq0Tf2WTXF1ih5dRce1nTIek2NUGaUQ= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(396003)(136003)(376002)(39860400002)(46966006)(36840700001)(40470700004)(36860700001)(26005)(86362001)(81166007)(40460700003)(4326008)(70586007)(70206006)(356005)(54906003)(6916009)(40480700001)(316002)(82740400003)(2906002)(8676002)(47076005)(44832011)(426003)(186003)(1076003)(83380400001)(2616005)(15650500001)(336012)(478600001)(7696005)(82310400005)(41300700001)(6666004)(8936002)(36756003)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5287 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT044.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 01af82af-140f-44be-6046-08da60f1d3ea X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7BFApOGkIxE6e4bK+H+FFgbS8NQCW8tQma9WkcikPtffSLzpgQoHbSty+nzzXPhE0HpyUzDNqIkTrT9N6B8thExrl+b3giQ37JZwK5IPwv4vabx/xrcSuBUmiqWd2nmzpLG24chuFbqIa0EgNMv8KUy5W7EqMEAPnviyWfc80pwa4ONN0UIAhjyCF5LWqNAbmu1i+z63W9ZDAx7Z2AKh7RwkWxsSE9sR1l/e+0x9838BRQouFsmMbrnS27XY6YdXvQUMCr8vTg1+3ExhsuDFGVT1JbLSZCDT1Nd9zmmpGXoVrfad36u+g6K2w8azCjyulAoK7zhmFojB3KUYVLxYmWUkl9sDM53gjCp97NutaQ0Ny92HIDcQfinZFpvfpPbZk7PhlUA1VU2xmKVKz6I6tiCYPgA/el3qi28PudjVv7jP4sUIiq/1uw1U1GCdACxLiRGPFNSbovvwCtqStCLltXDIaM0eLYzmn7ITdAuGfp8y3k8v4ftnAD+w2VniJLxzDW8/EmXblKSPStTYMF3OLJBCVxsxDw8Zb23lDSOdAqUWBxVOMh9eBHcQGOrvzMrlaitZBRzHqhci7V6cR584QtxV4He6iWeOjBcYIXyCv4PsC0Cf+FUtl7xu1knr4woCd73tVAweQwXTTSMRHiSLUgXyBlJ4n7wwmQJ1dR0KOP4n5OX6MfAoCwbrkwzykpJq2Y6iJ4C1hpX5y1EHRnc3fq9sjeyxNAgQp5jsf2eOyVQVxe9PEgW0rwolL+K6lF/kyQr/DavUk65mssngViy3GvE3vOfDj1GcNYkMXQza/4BZQwaC/EVvzSpg6E9sjvZm X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(346002)(376002)(136003)(396003)(46966006)(36840700001)(40470700004)(54906003)(2616005)(83380400001)(186003)(1076003)(47076005)(40480700001)(316002)(6916009)(426003)(8676002)(70206006)(336012)(36756003)(4326008)(70586007)(86362001)(82310400005)(478600001)(81166007)(44832011)(40460700003)(7696005)(8936002)(82740400003)(5660300002)(2906002)(6666004)(15650500001)(41300700001)(36860700001)(26005);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:09.6385 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cf32518d-7d79-4408-aa33-08da60f1da9f X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT044.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR2PR08MB4713 x86 provides a mem_hotplug variable to maintain the memory hotplug end address. We want to move some codes from x86 to common, so that it can be reused by other architectures. But not all architectures have supported memory hotplug. So in this patch, we introduce this helper to replace mem_hotplug direct access in these codes. This will give the ability of stubbing this helper to those architectures without memory hotplug support. Signed-off-by: Wei Chen --- v1 -> v2: 1. Refine the commit message. 2. Merge v1 patch#9,10 into one patch. Introduce the new functions in the same patch that this patch will be used first time. 3. Fold if ( end > mem_hotplug ) to mem_hotplug_update_boundary, in this case, we can drop mem_hotplug_boundary. --- xen/arch/x86/include/asm/mm.h | 6 ++++++ xen/arch/x86/srat.c | 3 +-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h index 07b59c982b..b3dfbdb52b 100644 --- a/xen/arch/x86/include/asm/mm.h +++ b/xen/arch/x86/include/asm/mm.h @@ -476,6 +476,12 @@ static inline int get_page_and_type(struct page_info *page, extern paddr_t mem_hotplug; +static inline void mem_hotplug_update_boundary(paddr_t end) +{ + if ( end > mem_hotplug ) + mem_hotplug = end; +} + /****************************************************************************** * With shadow pagetables, the different kinds of address start * to get get confusing. diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index b62a152911..f53431f5e8 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -418,8 +418,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) memblk_nodeid[num_node_memblks] = node; if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) { __set_bit(num_node_memblks, memblk_hotplug); - if (end > mem_hotplug) - mem_hotplug = end; + mem_hotplug_update_boundary(end); } num_node_memblks++; } From patchwork Fri Jul 8 14:54:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B77DCCA47F for ; Fri, 8 Jul 2022 14:55:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363469.594019 (Exim 4.92) (envelope-from ) id 1o9pO4-0007BR-Gu; Fri, 08 Jul 2022 14:55:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363469.594019; Fri, 08 Jul 2022 14:55:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pO4-0007BK-D0; Fri, 08 Jul 2022 14:55:20 +0000 Received: by outflank-mailman (input) for mailman id 363469; Fri, 08 Jul 2022 14:55:19 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pO3-0007Ai-70 for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:19 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80070.outbound.protection.outlook.com [40.107.8.70]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id faf61f11-fecd-11ec-bd2d-47488cf2e6aa; Fri, 08 Jul 2022 16:55:17 +0200 (CEST) Received: from FR3P281CA0071.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::7) by PAXPR08MB7550.eurprd08.prod.outlook.com (2603:10a6:102:24d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:15 +0000 Received: from VE1EUR03FT033.eop-EUR03.prod.protection.outlook.com (2603:10a6:d10:4b:cafe::3e) by FR3P281CA0071.outlook.office365.com (2603:10a6:d10:4b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.13 via Frontend Transport; Fri, 8 Jul 2022 14:55:15 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT033.mail.protection.outlook.com (10.152.18.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:14 +0000 Received: ("Tessian outbound 6f9e7ef31fa8:v122"); Fri, 08 Jul 2022 14:55:14 +0000 Received: from e4df9d7cd0e0.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 0D4ED391-D037-4F64-8AA7-E34F472A8656.1; Fri, 08 Jul 2022 14:55:07 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e4df9d7cd0e0.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:07 +0000 Received: from DB6PR0501CA0008.eurprd05.prod.outlook.com (2603:10a6:4:8f::18) by AM9PR08MB7029.eurprd08.prod.outlook.com (2603:10a6:20b:41b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15; Fri, 8 Jul 2022 14:55:05 +0000 Received: from DBAEUR03FT038.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:8f:cafe::a7) by DB6PR0501CA0008.outlook.office365.com (2603:10a6:4:8f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20 via Frontend Transport; Fri, 8 Jul 2022 14:55:04 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT038.mail.protection.outlook.com (100.127.143.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:04 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:00 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:54:58 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: faf61f11-fecd-11ec-bd2d-47488cf2e6aa ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=Y8rlz8VKvCJ+pb3wMHP1qpBkwPNuhFHtB0PdNP3YEn5chgAM9ryASwJgPHIuprT3oi//6PdAQbQVJXsdZCY5c+PMZkC+ZDcJPgTZ2hQ8pNxF9cA2XSzA5FjLiIb+F4DlAmu0fkxqa8X+Vy78NJhTKUwWsGA+hasxXuOJwsyIeCGbpF1+Uq1t3IbeqyTJYJNHMt7sjUMHWx/XWz12sbxT7FHo94/MnTECFODCoib5Zb4fhWk1NJjB4pN3TtDtDdpIwnFHqem1BXoINgv2olAjTWuwHQ1ltLdwddcy/N/Wa2uAGJxhOvOrj1jxniz+WkQl6tbvwLQpFAR0uKntv25XDg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1ho4BETRsxA2qUWrMdXER8nJ1DBBHm4uFN3lKRvF6is=; b=NDruzvjU1gpf2YANfcp81r7JhVKYs9/aag8ZsZxYfjHy1SCwhEqhgbqkVaHrELsbQ41OKZm5DaETIJd0EHzRKN6gUCj5W6Yz0lG6koUP9fOdZ/ofdeLMVsrY03jgO1iMK8YXdxivDFaqosbVZOJ84UENk2xhobfZBMMdc2C5Jvg5f5BLjwmWXHIFpOmG3IzuJh/Lc5jpSAh/bz5yBX59nspmAVotIiJ/l7t1SwzEnT2RAHaLj9VqvG+klWhWe3Ma8LyX0VZK+xUiu2Jpve+XRb4RmQLy78V9ZEm3POptNtVYqqvNB9JIxmtbuGMrDbTZ5tsQwm+MTO0bNnXDArTdcA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1ho4BETRsxA2qUWrMdXER8nJ1DBBHm4uFN3lKRvF6is=; b=JGRpIbcUDYc8VOZ45VEnA5jLosCAjxYj5oIRqghV11XlYoe9L8v7ZD9nT/rdAV7xlY5EHn5HqGNdPtgzQINAva4Or1WwS7T8wEpYsAmHZRCoLRWT/b+PJZ1Bbomtm9JLIEfMoa/dz92th6BuxC1sjnylGlZtzx1zhTVxOTglOOk= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: acad7d1847ba8541 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SW7SOapgsaHjy5V6+jxaS1v2fpGORrIv151bhleD8GlSdsJKBFWBYAO05H5auCvOy33B9xfJVrtQygf1cpnlgB66xCRvWbzpXKx2XQD8ZUUxPUIXdkUsQjF0Exll0TC4tWfYla4kJHPJXOI1bXwu3AiNqBUQeLSFjnput2EcMM3MAMEweCq4lpb1aMezjJ5LwdY3q7ktUWPd1Kgg/gLtBP99H6mIFOp2wXqRp++q928VHCtGSgfKQtU3R6mZ0tPuRVv+K4sJ11YQbaKdZ505vp33ZP4blzgTH6bt9AM8I4hm8ExS1PVJqFlRFCPuNxXvTBJxlh6+AxA3fNgH4hsLzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1ho4BETRsxA2qUWrMdXER8nJ1DBBHm4uFN3lKRvF6is=; b=Q4qjvS+JVog5XPlhW15T5rKgeJZS4cqEP74uaEbX7FJwLaRdP0LvYnKJMoz6jK6R463Ic22wspdCcSEa6gfe6NKQcH15aRkshSSjlVIH+EHanB7pAmRMVU6qkX8jsYtoruGl4sOvIl9L9ups+2KX2JigxOAbqQbs6uRwAbvlpAzrBlypD0uB6DOfbyGG31VYugc3hfRdt5+1kMIVH3WSmY//Y+cq3qjswbNaBEQ2khRExSLEHSY10MbRf28RSN7SyP9tBOEveeerXXYdN1aJkpTPVyejYcm1rOQ4yjqfzbuQeirrD+OTSgcCA7thhJY3UsZiBLeb6gRps6MnRE32gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1ho4BETRsxA2qUWrMdXER8nJ1DBBHm4uFN3lKRvF6is=; b=JGRpIbcUDYc8VOZ45VEnA5jLosCAjxYj5oIRqghV11XlYoe9L8v7ZD9nT/rdAV7xlY5EHn5HqGNdPtgzQINAva4Or1WwS7T8wEpYsAmHZRCoLRWT/b+PJZ1Bbomtm9JLIEfMoa/dz92th6BuxC1sjnylGlZtzx1zhTVxOTglOOk= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu Subject: [PATCH v2 2/9] xen/x86: Use enumerations to indicate NUMA status Date: Fri, 8 Jul 2022 22:54:17 +0800 Message-ID: <20220708145424.1848572-3-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: 7e826c80-2867-4879-d9c9-08da60f1dda5 X-MS-TrafficTypeDiagnostic: AM9PR08MB7029:EE_|VE1EUR03FT033:EE_|PAXPR08MB7550:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: AJdPe4WqzibUEo4iL+zKkkA4luh8/sgCE6g9v4kfNCaN11LUAlKMskmnRYFFyOmKkWv2gVFcear9ZuiYsmeIV4nc+RV1tEEtmtxsq+U+9kmzPkkSVJl76vZZl8zXBtFn9A98EHGEBtX1FUErlwvnWvymp4s5Zs6CH17aMVutUSDiDqBmJPi4H2oBCRhTWZGPTzvlcQh64NsonbTLHCCQfl70BxwIeF/mfSVFP5Pm4QWz5NZlhQ38Y23QI3yHXOsPdhBQhsQW+ybxu7wv3s5Whbolmsu3wBCACJkF5pEvoWdt0ZsSkhhpLzTGjMk+J8IyrDJ5htVFLSQJpR5GDHe2WtvefqFUf72EV1HwMTTG+svAQrmSpds1DD6SKMkBydKLFVPOW14h1/lRlY1DJm2n6+rDXp7Mw/JhtFGwuNBk/3fBwQXVjpAnOihV/mmhz6n3aBZq0Zn2jgD4Tk0ha4aY2+0A2e02caKc+b436Yq2mz2Bq/SXAG5hpkH03sbJs0bKm1doX+2x5sqXsbd2Vctao7ox2MoJTIyc5bzRQWA588aNDKnGEIVCZgS9AFBsBTi64zBtTsUkz3+ZI7Hf2WS7vkSUlqIFOxxRqHkCKFQTe76pACnOyy7FOpKlSIUfKhe10ZE/4BjGZp+mNa3YvRw1rQXstuAC+TyhburzWnXt9USJnlJMC9hHEYyl2r+8dg+1i33ccT3XHYDDmABcvpxTuND2j+HXKnZKZQPeH1v8ZiySGI1GtvfUOxHD1LkTHLVpxy2JzfaBe7r1BpVMrUaTvWxs4R5X4YEErkV4fr0FhW7iKoyQ23x7BcnZlMg3T8pV+8mMpJky/Ax5vPMJzsLGFBolJPJVnfIAydBTw24hLdw= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(136003)(39860400002)(396003)(376002)(36840700001)(40470700004)(46966006)(82310400005)(44832011)(8936002)(40480700001)(5660300002)(81166007)(356005)(6916009)(1076003)(54906003)(82740400003)(7696005)(186003)(2616005)(478600001)(316002)(41300700001)(26005)(40460700003)(36860700001)(47076005)(336012)(6666004)(8676002)(4326008)(426003)(36756003)(70206006)(70586007)(83380400001)(2906002)(86362001)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB7029 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT033.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 6f8a719d-dc85-41a2-0ac2-08da60f1d7b7 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HdbZJfUTAZfylp7ukb+DMc25W7zMCIASwTQ8v7Lc3WMlMF7uu4zI+56geyFk4pcF+cEnFtdf4k6tKC5yx0SIRCJNWjwfizvdLZ+CPPEoybNuNDA3EjpbqrIZJkQzeNZAq6Hspec1+WlhWONdmjq9/DeCin+/rpKNf4pAMuXocfQUycURVwaPDoLPFiWHfCzHGwxAUEFofH/nGj+koYq5QIJpVpvxPpQdVQU2nHRfmC9C2TTwqMabpFF40p6lYAR21Ak4+hEO8M3gxWfEBQXVRfOQB956mRaPl8rv99Jq5amsvBKp4DH1QHak/1RuYnaVNIemdsVVmzFYBpTkQnYgb8sNPi0sqqddfFk/+Ksx51FPs2kaXPzmFh3OvFKUxReqNF2CZ0RwnGnmgBagV3steYEou6dyZLv2wEIEadxFTZ46tknMPpTo4WprNNava0cN6zv/guEi/jMeuNVJVOCLChEVbdVhcvcNbKHmag/+SQWIREDmQV+bQJBgoVFRDuvPO44M+3qE1NcH5f7H8NyVZP/HX5mTXs2gy0V2rh4g4kGyaGukaakiHtbZYfgdVzglF+suY88makgz37v1gsUoJ1d67Tfx27qs4OUlE14wxDX9BiAzEFihugqSVJrTELVANAZy+yz50XWWYMSyeK0AO2drVezkaovIaf9mNfuxVOXBzyy45OuuPuRHIvosNrbkfqr13kcsLJZJCDgFyhxPTngF5WJruQ3xcQgEbUfjMNFYI856AwnX4b/xf6HvLIxN5iHhgNUwMB1ykna4ntiJsYuo867xQ/5Jox6MBi9sBR9OQZZJk+KxGMQiJFkjYYsH X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(346002)(376002)(396003)(136003)(40470700004)(46966006)(36840700001)(83380400001)(1076003)(426003)(82740400003)(186003)(5660300002)(2616005)(2906002)(47076005)(36860700001)(44832011)(40480700001)(36756003)(8936002)(336012)(54906003)(478600001)(41300700001)(7696005)(6666004)(26005)(4326008)(86362001)(82310400005)(40460700003)(81166007)(8676002)(70586007)(316002)(6916009)(70206006);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:14.6959 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7e826c80-2867-4879-d9c9-08da60f1dda5 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT033.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB7550 In current code, x86 is using two variables, numa_off and acpi_numa, to indicate the NUMA status. This is because NUMA is not coupled with ACPI, and ACPI still can work without NUMA on x86. With these two variables' combinations, x86 can have several NUMA status: NUMA swith on, NUMA swith off, NUMA swith on with NUMA emulation, NUMA swith on with no-ACPI, NUMA swith on with ACPI. In this case, we introduce an enumeration numa_mode in this patch to indicate above NUMA status, except NUMA on with emulation. Because NUMA emulation has another variable, numa_fake, to indicate the number of nodes for emulation. We can't use the enumeration to replace it at the same time. But it still can be indicated by numa_on and numa_fake as what it has been indicated. Based on the enumeration we introduce numa_enabled_with_firmware for callers to check NUMA status is enabled + ACPI. Using this helper is because some NUMA implementation will use other firmware, this helper will be easy to them to check enabled + others. As we have touched srat_disabled, we have corrected its return value from int to bool. Signed-off-by: Wei Chen --- v1 -> v2: 1. Remove fw_numa. 2. Use enumeration to replace numa_off and acpi_numa. 3. Correct return value of srat_disabled. 4. Introduce numa_enabled_with_firmware. --- xen/arch/x86/include/asm/acpi.h | 1 - xen/arch/x86/include/asm/numa.h | 16 +++++++++++++--- xen/arch/x86/numa.c | 28 +++++++++++++++------------- xen/arch/x86/setup.c | 3 ++- xen/arch/x86/srat.c | 13 +++++++------ 5 files changed, 37 insertions(+), 24 deletions(-) diff --git a/xen/arch/x86/include/asm/acpi.h b/xen/arch/x86/include/asm/acpi.h index 9a9cc4c240..ab0d56dd70 100644 --- a/xen/arch/x86/include/asm/acpi.h +++ b/xen/arch/x86/include/asm/acpi.h @@ -101,7 +101,6 @@ extern unsigned long acpi_wakeup_address; #define ARCH_HAS_POWER_INIT 1 -extern s8 acpi_numa; extern int acpi_scan_nodes(u64 start, u64 end); #define NR_NODE_MEMBLKS (MAX_NUMNODES*2) diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/numa.h index c32ccffde3..ee8262d969 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -28,12 +28,22 @@ extern nodeid_t pxm_to_node(unsigned int pxm); #define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) #define VIRTUAL_BUG_ON(x) +/* Enumerations for NUMA status. */ +enum numa_mode { + numa_on = 0, + numa_off, + /* NUMA turns on, but ACPI table is bad or disabled. */ + numa_no_acpi, + /* NUMA turns on, and ACPI table works well. */ + numa_acpi, +}; + extern void numa_add_cpu(int cpu); extern void numa_init_array(void); -extern bool numa_off; - +extern bool numa_enabled_with_firmware(void); +extern enum numa_mode numa_status; -extern int srat_disabled(void); +extern bool srat_disabled(void); extern void numa_set_node(int cpu, nodeid_t node); extern nodeid_t setup_node(unsigned int pxm); extern void srat_detect_node(int cpu); diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index 627ae8aa95..0777a7518d 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -47,12 +47,16 @@ cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly; nodemask_t __read_mostly node_online_map = { { [0] = 1UL } }; -bool numa_off; -s8 acpi_numa = 0; +enum numa_mode numa_status; -int srat_disabled(void) +bool srat_disabled(void) { - return numa_off || acpi_numa < 0; + return numa_status == numa_off || numa_status == numa_no_acpi; +} + +bool __init numa_enabled_with_firmware(void) +{ + return numa_status == numa_acpi; } /* @@ -254,12 +258,13 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn) #endif #ifdef CONFIG_ACPI_NUMA - if ( !numa_off && !acpi_scan_nodes(start, end) ) + if ( numa_status != numa_off && !acpi_scan_nodes(start, end) ) return; #endif printk(KERN_INFO "%s\n", - numa_off ? "NUMA turned off" : "No NUMA configuration found"); + numa_status == numa_off ? "NUMA turned off" + : "No NUMA configuration found"); printk(KERN_INFO "Faking a node at %"PRIpaddr"-%"PRIpaddr"\n", start, end); @@ -292,13 +297,13 @@ void numa_set_node(int cpu, nodeid_t node) static int __init cf_check numa_setup(const char *opt) { if ( !strncmp(opt,"off",3) ) - numa_off = true; + numa_status = numa_off; else if ( !strncmp(opt,"on",2) ) - numa_off = false; + numa_status = numa_on; #ifdef CONFIG_NUMA_EMU else if ( !strncmp(opt, "fake=", 5) ) { - numa_off = false; + numa_status = numa_on; numa_fake = simple_strtoul(opt+5,NULL,0); if ( numa_fake >= MAX_NUMNODES ) numa_fake = MAX_NUMNODES; @@ -306,10 +311,7 @@ static int __init cf_check numa_setup(const char *opt) #endif #ifdef CONFIG_ACPI_NUMA else if ( !strncmp(opt,"noacpi",6) ) - { - numa_off = false; - acpi_numa = -1; - } + numa_status = numa_no_acpi; #endif else return -EINVAL; diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index f08b07b8de..4841af5926 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -329,7 +329,8 @@ void srat_detect_node(int cpu) node_set_online(node); numa_set_node(cpu, node); - if ( opt_cpu_info && acpi_numa > 0 ) + /* Print CPU info when NUMA is enabled with ACPI. */ + if ( opt_cpu_info && numa_enabled_with_firmware() ) printk("CPU %d APIC %d -> Node %d\n", cpu, apicid, node); } diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index f53431f5e8..422e4c73e3 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -185,7 +185,7 @@ static __init void bad_srat(void) { int i; printk(KERN_ERR "SRAT: SRAT not used.\n"); - acpi_numa = -1; + numa_status = numa_no_acpi; for (i = 0; i < MAX_LOCAL_APIC; i++) apicid_to_node[i] = NUMA_NO_NODE; for (i = 0; i < ARRAY_SIZE(pxm2node); i++) @@ -260,7 +260,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa) apicid_to_node[pa->apic_id] = node; node_set(node, processor_nodes_parsed); - acpi_numa = 1; + numa_status = numa_acpi; if (opt_acpi_verbose) printk(KERN_INFO "SRAT: PXM %u -> APIC %08x -> Node %u\n", @@ -295,7 +295,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa) } apicid_to_node[pa->apic_id] = node; node_set(node, processor_nodes_parsed); - acpi_numa = 1; + numa_status = numa_acpi; if (opt_acpi_verbose) printk(KERN_INFO "SRAT: PXM %u -> APIC %02x -> Node %u\n", @@ -484,7 +484,7 @@ static int __init cf_check srat_parse_region( (ma->flags & ACPI_SRAT_MEM_NON_VOLATILE)) return 0; - if (numa_off) + if (numa_status == numa_off) printk(KERN_INFO "SRAT: %013"PRIx64"-%013"PRIx64"\n", ma->base_address, ma->base_address + ma->length - 1); @@ -499,7 +499,7 @@ void __init srat_parse_regions(paddr_t addr) u64 mask; unsigned int i; - if (acpi_disabled || acpi_numa < 0 || + if (acpi_disabled || numa_status == numa_no_acpi || acpi_table_parse(ACPI_SIG_SRAT, acpi_parse_srat)) return; @@ -528,7 +528,8 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end) for (i = 0; i < MAX_NUMNODES; i++) cutoff_node(i, start, end); - if (acpi_numa <= 0) + /* Only when numa_on with good firmware, we can do numa scan nodes. */ + if (!numa_enabled_with_firmware()) return -1; if (!nodes_cover_memory()) { From patchwork Fri Jul 8 14:54:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1B01C43334 for ; Fri, 8 Jul 2022 14:55:44 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363471.594041 (Exim 4.92) (envelope-from ) id 1o9pOH-0007vV-8G; Fri, 08 Jul 2022 14:55:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363471.594041; Fri, 08 Jul 2022 14:55:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOH-0007vF-49; Fri, 08 Jul 2022 14:55:33 +0000 Received: by outflank-mailman (input) for mailman id 363471; Fri, 08 Jul 2022 14:55:31 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOF-0007Ai-CM for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:31 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2089.outbound.protection.outlook.com [40.107.22.89]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 01da2c41-fece-11ec-bd2d-47488cf2e6aa; Fri, 08 Jul 2022 16:55:29 +0200 (CEST) Received: from FR3P281CA0070.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:4b::22) by DB9PR08MB7179.eurprd08.prod.outlook.com (2603:10a6:10:2cc::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:26 +0000 Received: from VE1EUR03FT033.eop-EUR03.prod.protection.outlook.com (2603:10a6:d10:4b:cafe::3c) by FR3P281CA0070.outlook.office365.com (2603:10a6:d10:4b::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.13 via Frontend Transport; Fri, 8 Jul 2022 14:55:26 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT033.mail.protection.outlook.com (10.152.18.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:25 +0000 Received: ("Tessian outbound 13cb25bfb745:v122"); Fri, 08 Jul 2022 14:55:25 +0000 Received: from 31d9462c680e.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 6A78CC50-9495-420C-98C8-2FB08507F2EA.1; Fri, 08 Jul 2022 14:55:18 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 31d9462c680e.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:18 +0000 Received: from DB9PR05CA0015.eurprd05.prod.outlook.com (2603:10a6:10:1da::20) by VI1PR08MB4511.eurprd08.prod.outlook.com (2603:10a6:803:f9::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.15; Fri, 8 Jul 2022 14:55:06 +0000 Received: from DBAEUR03FT009.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:1da:cafe::46) by DB9PR05CA0015.outlook.office365.com (2603:10a6:10:1da::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20 via Frontend Transport; Fri, 8 Jul 2022 14:55:06 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT009.mail.protection.outlook.com (100.127.143.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:06 +0000 Received: from AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:05 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX01.Emea.Arm.com (10.251.26.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:03 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:55:01 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 01da2c41-fece-11ec-bd2d-47488cf2e6aa ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=Nwcj3sou0jYGFwTRilG6ZVxUBERKYDYiUsijrH3EKwp5hlgtX6gvhf/Sya1dxPkz+yL1xIOmmzYwml47SR7rER1FTFqH2wumJsUtXM3p03iLTF3mjmwGGRHVegbeE4PcbJ6cgjO7LFXEdIcekAmgmtCdr3c3LfDXt+MFmgrdAem/uW0iM8xu0ZaM8fo2C7I7jtt1wiQ5gyv2FuwNMmGTJseWXzKoXNS+oHOtKmZj8a7Ar8fRs8PGPPJSfftYaYygXa5bwrti4GyEnSi5jFF7Ujy6THXfbMc0VQPeigbFgtx3CoJ8LKMH/vSKpoZW5wpA3IA6SwbOoFi7hyQc/mvV6w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9rp+JDYzWrvvbuy8ZPNrJYoe9iSuHHdiEXdj+bJMmS4=; b=VOj0Buys+rflBeHYNVCjdrFf9RBsk9BH7M2W+pBMsDZKeyMrJL6KqxBtlKsPD++Tg53GrpF+Q6A8JjeJ8pGG0yrTjlbkNGS1QEhkXKn/sNWacntV4CbGAJM3jh5mMLEOWLEZpVRURPt1saKqla+L+1vaauy3pnhrjBrfXcv+H7Ib8yWWO/PFJD+tYP30KfZG2CNo+n01LkdJ0gei6tt+9CZOSGW9zG1+RNEBRvbWBwGLgANdKL6FB5XsRvK8afAqB2XZA1e3fKhbEBUcXlgZBQDVHnSGv6UMdwIpmzPYOfZuqWDfRHq6kIuThw28TXKCsfJAHFsZVpxnHbNsuS9Nkw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9rp+JDYzWrvvbuy8ZPNrJYoe9iSuHHdiEXdj+bJMmS4=; b=NuzlkyGE5WN5hNskc657a6o1ZBVeO3f9DbA/+Oi/2cHzv/sC3FivNGHhPavH17y8CJYsupRnec2Sjz1FfIw/KgPq1qd33RUG/4r65Rz+QJzH7kIRQDfSU301ZxxT1BNsVL+n5dDUlxQHiL9eRTX6VQ+Bzm2U3oQs0DVZbI9D3H0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 06c9b3aeb875b9c1 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DHakffQVIBqwEqU/rcqsZkv3OfKzeTVRJv8lNDqXsht+LDyfETVM2mrj45TRm0hqp9mXbNLIEWmjZEJ74dU360KQMfkE22aT0DpQOrH0ATZ5YNdk/uZ+paHvp2sWHRS3nabyqeAawRrMwsk3aqvHKWDBenSD3UgK+vXw+BYYZCM/NZgAOU4R5bFc7JPaEFLD5NoRAzzxddYVZSjJkgT20WRGKZnfNz5VASdHkV4tcisXDkCo2sWNGbcIP2uzBXB6vfojHMXzjT8M4VnmW4EQAcP1GyFQ7urkBqmrLg23ZjNQ+FzsXSyFeLiaAiEsDGSTOo3PKmOXpM/4p2wwc9MRTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9rp+JDYzWrvvbuy8ZPNrJYoe9iSuHHdiEXdj+bJMmS4=; b=XDDHBLczuF5YjM+by/18qz9RC+vlYB5pBh6p+M3Dv5Bv2Abj87WjP04g7Wwm65/29lbxmczOddzA89RvB9f8P6ymQXR4SzDwL58IWU15C+G995J/1NRrGppYKU1D07IMWVz7XBnCwlfxw212ZEpOOKc1765MulMKXmo9picKR4ZDOn4TzMZf0Vnyj4H62PFBEs29tn31kLPXYY/2pbPfeyNgCOLTB8rnkRACHVDw2iH8tfie84D6XizLtOGT2MkVOYLvBRyDhT4cY9L/CtkMJVHepKiEJn+Osav3BptywzZKoNvb9I/3EkLuCYaIJvyD60Gc9gU/AM4h3pGX0uIShg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9rp+JDYzWrvvbuy8ZPNrJYoe9iSuHHdiEXdj+bJMmS4=; b=NuzlkyGE5WN5hNskc657a6o1ZBVeO3f9DbA/+Oi/2cHzv/sC3FivNGHhPavH17y8CJYsupRnec2Sjz1FfIw/KgPq1qd33RUG/4r65Rz+QJzH7kIRQDfSU301ZxxT1BNsVL+n5dDUlxQHiL9eRTX6VQ+Bzm2U3oQs0DVZbI9D3H0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v2 3/9] xen/x86: move generically usable NUMA code from x86 to common Date: Fri, 8 Jul 2022 22:54:18 +0800 Message-ID: <20220708145424.1848572-4-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: 49745b58-c02d-4413-75e0-08da60f1e43b X-MS-TrafficTypeDiagnostic: VI1PR08MB4511:EE_|VE1EUR03FT033:EE_|DB9PR08MB7179:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: RYfULEEnsTM3mRQbRcX7hbjDdR0ym+QHsG2fxbH8o3XDHC/psvaQ2wA7UPPgXBxYFRXbCCKN6+10vpmF5H6BIObZ6AmMmsKLFm2FIwc4Gb3BqoM4gt5BBXNGOvwh7C9D9A49LV29OK50IxYDTceu0wRCrdVVPO1lOZmnlRH28ag0sgb8PGf9o2J09D8UX82qmwgyd44oH2EE7kO1FuVsnSk2NG3zYo8ad9JOzUGgrSEWnZLBOzoHzo4tyu67wvNBQSBjAm/5l05c0XJBB9y7QlewqHH/TtT9mTBwNhhRTi45/gEUA+xVJAn5RFYRv4HIth42f8BVp1qi+xu5O0GVs0R2lwpdYz81sohcosMIkR0a1xLn36yKCnkJG4lmEAGbw4EabA8HXIIl8rAA9PH2RF146ibgJVGeIe5lQdNLPgmBUAmrIJAwjvc4yaB+361VhrmKsV7VEdcynRstI2AswASJlzghoE47dEkxpoNnYaqFkzRNQy4xJo/F0Xmb25s4DbZ/rxPNmWXgc51CWTEds9RFTJxrU1hB/YbWzpdnYb+OQZ3hdzcDQJ1DzLuhPM8JCLcmcHLGXQ6RSs6T17uaDBDLnLKBgLdgSKX70NIxSQKTEVEMZu4VFP+kTGh0dgqAtfXgOwJQnk+fFPUWEe3H+a13TqzYZzun6M8QIdcr6pgO35edNCIRjQBX7FoowzufRD97urRUtQRLvyAbXjbgR5K91PTToWI1Cj76Y/+xSFDPgPbr19O/fenwu+3XNPxBGhMIqEkY6yNn2EOgojvEMYsutSvdOgqarMipsmOTPFsxttjAojZLhzKMM8/jIhiHEhuxG0FtQwbvz+cCN1lknF7TH07FBwZh7U00EwRVmD8= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(396003)(376002)(136003)(346002)(40470700004)(46966006)(36840700001)(82310400005)(70206006)(6916009)(83380400001)(4326008)(81166007)(70586007)(316002)(426003)(1076003)(186003)(2616005)(8676002)(41300700001)(356005)(26005)(478600001)(82740400003)(54906003)(47076005)(6666004)(86362001)(30864003)(40480700001)(2906002)(40460700003)(36860700001)(7696005)(44832011)(5660300002)(336012)(36756003)(8936002)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4511 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT033.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: c29c1cd3-efcf-4b6c-7b36-08da60f1d88b X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8trM9A6Nu5O9y6dvk0Ca8Wh6MbhqUE/kcoJ5j8pAN8TONw9lZ9FiGtCWXfMIEgUgpHGzmwtlpSYjm2AYwxlcpzCQov2n21wByu4DiG1DwHT++OeLDdwJbMxeVL3Fy4XJCHeiNaWMsN5VHu13dlXzDBHqkMbLUrw3uTwsviU+FfNMvvBGXtDcYyFGZp9FclPkQgSV9VVm0ZLmb6FVrSRSbChtnCq7AaSCzN0MmtmtcTWx56aAp5pgJFP+EljoprDEE3okmd8Sxg9kroTxa4QMVRPdHWN5FHzpzlDDTlYQgvKhdederetOfVn6mSWQ/YLZNewFvhRcevUb0TeMJiyMzO+5llrxY/tZ/kRdi8oy6jHF+63NUmclRL52KJZ7/f3g00vt22giPrBXPTbFYV424F46AQUpCWRBYqT3IghhMAFCa6ikRHvDgtG7kC6/BQ7iWElsFjXEb29GFMpG4EtuKnVSBLpY6aehXLrDEsRTNnX13LKc3xYKH6szs3GN1R/TodMrD/HVLCFRD+hOZtlDUoLlXzwnm0KRyY8lfaOEB+hnHCrA620wGqfz8PiJ/omYTvVCV8Fj24hDisJdQ1hClV8+Hq+89LaXRDaxaKHCLj3R/0XXjoXObUGTEJLJgT/2KydplLMu+tLAeuizJSa3FkO+k8pNI1NDQc63FwYOcrNTLOCygDu3kghVDPzmVmKf6pW0ZrFrZ8QUN13rLeiwGt4OWGJY1hYCOjO5qrc+8H+Na5KVRPm+JHtKbCCF+uueeZbIXnqCySmpA1CXL3JBWUrXcug5BbPFOLGaoGT4rZ9PjgdJtr8bRYKOG/8N26Sq X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(346002)(376002)(396003)(136003)(39860400002)(46966006)(36840700001)(40470700004)(107886003)(316002)(1076003)(36756003)(40480700001)(30864003)(82740400003)(6666004)(5660300002)(86362001)(6916009)(44832011)(2616005)(83380400001)(54906003)(8936002)(478600001)(47076005)(336012)(4326008)(8676002)(26005)(36860700001)(70586007)(7696005)(186003)(81166007)(426003)(82310400005)(41300700001)(2906002)(70206006)(40460700003);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:25.7427 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 49745b58-c02d-4413-75e0-08da60f1e43b X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT033.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB7179 There are some codes in x86/numa.c can be shared by architectures to implement NUMA support. Just like some variables and functions to check and store NUMA memory map. And some variables and functions to do NUMA initialization. In this patch, we move them to common/numa.c and xen/numa.h and use the CONFIG_NUMA to gate them for non-NUMA supported architectures. As the target header file is Xen-style, so we trim some spaces and replace tabs for the codes that has been moved to xen/numa.h at the same time. As we have moved phy_to_nid to xen/numa.h, we don't need a separate MAX_NUMNODES in asm/numa.h. Now all architectures can use the same MAX_NUMNODES in xen/numa.h. The conditional macro can be removed. Signed-off-by: Wei Chen --- v1 -> v2: 1. New patch in v2. --- xen/arch/x86/include/asm/numa.h | 64 ----- xen/arch/x86/include/asm/setup.h | 1 - xen/arch/x86/numa.c | 431 +----------------------------- xen/common/Makefile | 1 + xen/common/numa.c | 439 +++++++++++++++++++++++++++++++ xen/include/xen/numa.h | 74 ++++++ 6 files changed, 515 insertions(+), 495 deletions(-) create mode 100644 xen/common/numa.c diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/numa.h index ee8262d969..d8960743d4 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -9,81 +9,17 @@ typedef u8 nodeid_t; extern int srat_rev; -extern nodeid_t cpu_to_node[NR_CPUS]; -extern cpumask_t node_to_cpumask[]; - -#define cpu_to_node(cpu) (cpu_to_node[cpu]) -#define parent_node(node) (node) -#define node_to_first_cpu(node) (__ffs(node_to_cpumask[node])) -#define node_to_cpumask(node) (node_to_cpumask[node]) - -struct node { - paddr_t start, end; -}; - -extern int compute_hash_shift(struct node *nodes, int numnodes, - nodeid_t *nodeids); extern nodeid_t pxm_to_node(unsigned int pxm); #define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) -#define VIRTUAL_BUG_ON(x) - -/* Enumerations for NUMA status. */ -enum numa_mode { - numa_on = 0, - numa_off, - /* NUMA turns on, but ACPI table is bad or disabled. */ - numa_no_acpi, - /* NUMA turns on, and ACPI table works well. */ - numa_acpi, -}; - -extern void numa_add_cpu(int cpu); -extern void numa_init_array(void); -extern bool numa_enabled_with_firmware(void); -extern enum numa_mode numa_status; extern bool srat_disabled(void); -extern void numa_set_node(int cpu, nodeid_t node); extern nodeid_t setup_node(unsigned int pxm); extern void srat_detect_node(int cpu); -extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end); extern nodeid_t apicid_to_node[]; extern void init_cpu_to_node(void); -static inline void clear_node_cpumask(int cpu) -{ - cpumask_clear_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); -} - -/* Simple perfect hash to map pdx to node numbers */ -extern int memnode_shift; -extern unsigned long memnodemapsize; -extern u8 *memnodemap; - -struct node_data { - unsigned long node_start_pfn; - unsigned long node_spanned_pages; -}; - -extern struct node_data node_data[]; - -static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr) -{ - nodeid_t nid; - VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >= memnodemapsize); - nid = memnodemap[paddr_to_pdx(addr) >> memnode_shift]; - VIRTUAL_BUG_ON(nid >= MAX_NUMNODES || !node_data[nid]); - return nid; -} - -#define NODE_DATA(nid) (&(node_data[nid])) - -#define node_start_pfn(nid) (NODE_DATA(nid)->node_start_pfn) -#define node_spanned_pages(nid) (NODE_DATA(nid)->node_spanned_pages) -#define node_end_pfn(nid) (NODE_DATA(nid)->node_start_pfn + \ - NODE_DATA(nid)->node_spanned_pages) #define arch_want_default_dmazone() (num_online_nodes() > 1) extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node); diff --git a/xen/arch/x86/include/asm/setup.h b/xen/arch/x86/include/asm/setup.h index 21037b7f31..ae470ea12f 100644 --- a/xen/arch/x86/include/asm/setup.h +++ b/xen/arch/x86/include/asm/setup.h @@ -20,7 +20,6 @@ void early_time_init(void); void set_nr_cpu_ids(unsigned int max_cpus); -void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn); void arch_init_memory(void); void subarch_init_memory(void); diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index 0777a7518d..193314dbd9 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -4,20 +4,11 @@ * Adapted for Xen: Ryan Harper */ -#include -#include #include -#include +#include #include #include -#include -#include -#include -#include -#include #include -#include -#include #ifndef Dprintk #define Dprintk(x...) @@ -26,28 +17,12 @@ /* from proto.h */ #define round_up(x,y) ((((x)+(y))-1) & (~((y)-1))) -struct node_data node_data[MAX_NUMNODES]; - -/* Mapping from pdx to node id */ -int memnode_shift; -static typeof(*memnodemap) _memnodemap[64]; -unsigned long memnodemapsize; -u8 *memnodemap; - -nodeid_t cpu_to_node[NR_CPUS] __read_mostly = { - [0 ... NR_CPUS-1] = NUMA_NO_NODE -}; /* * Keep BIOS's CPU2node information, should not be used for memory allocaion */ nodeid_t apicid_to_node[MAX_LOCAL_APIC] = { [0 ... MAX_LOCAL_APIC-1] = NUMA_NO_NODE }; -cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly; - -nodemask_t __read_mostly node_online_map = { { [0] = 1UL } }; - -enum numa_mode numa_status; bool srat_disabled(void) { @@ -59,267 +34,6 @@ bool __init numa_enabled_with_firmware(void) return numa_status == numa_acpi; } -/* - * Given a shift value, try to populate memnodemap[] - * Returns : - * 1 if OK - * 0 if memnodmap[] too small (of shift too small) - * -1 if node overlap or lost ram (shift too big) - */ -static int __init populate_memnodemap(const struct node *nodes, - int numnodes, int shift, nodeid_t *nodeids) -{ - unsigned long spdx, epdx; - int i, res = -1; - - memset(memnodemap, NUMA_NO_NODE, memnodemapsize * sizeof(*memnodemap)); - for ( i = 0; i < numnodes; i++ ) - { - spdx = paddr_to_pdx(nodes[i].start); - epdx = paddr_to_pdx(nodes[i].end - 1) + 1; - if ( spdx >= epdx ) - continue; - if ( (epdx >> shift) >= memnodemapsize ) - return 0; - do { - if ( memnodemap[spdx >> shift] != NUMA_NO_NODE ) - return -1; - - if ( !nodeids ) - memnodemap[spdx >> shift] = i; - else - memnodemap[spdx >> shift] = nodeids[i]; - - spdx += (1UL << shift); - } while ( spdx < epdx ); - res = 1; - } - - return res; -} - -static int __init allocate_cachealigned_memnodemap(void) -{ - unsigned long size = PFN_UP(memnodemapsize * sizeof(*memnodemap)); - unsigned long mfn = mfn_x(alloc_boot_pages(size, 1)); - - memnodemap = mfn_to_virt(mfn); - mfn <<= PAGE_SHIFT; - size <<= PAGE_SHIFT; - printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n", - mfn, mfn + size); - memnodemapsize = size / sizeof(*memnodemap); - - return 0; -} - -/* - * The LSB of all start and end addresses in the node map is the value of the - * maximum possible shift. - */ -static int __init extract_lsb_from_nodes(const struct node *nodes, - int numnodes) -{ - int i, nodes_used = 0; - unsigned long spdx, epdx; - unsigned long bitfield = 0, memtop = 0; - - for ( i = 0; i < numnodes; i++ ) - { - spdx = paddr_to_pdx(nodes[i].start); - epdx = paddr_to_pdx(nodes[i].end - 1) + 1; - if ( spdx >= epdx ) - continue; - bitfield |= spdx; - nodes_used++; - if ( epdx > memtop ) - memtop = epdx; - } - if ( nodes_used <= 1 ) - i = BITS_PER_LONG - 1; - else - i = find_first_bit(&bitfield, sizeof(unsigned long)*8); - memnodemapsize = (memtop >> i) + 1; - return i; -} - -int __init compute_hash_shift(struct node *nodes, int numnodes, - nodeid_t *nodeids) -{ - int shift; - - shift = extract_lsb_from_nodes(nodes, numnodes); - if ( memnodemapsize <= ARRAY_SIZE(_memnodemap) ) - memnodemap = _memnodemap; - else if ( allocate_cachealigned_memnodemap() ) - return -1; - printk(KERN_DEBUG "NUMA: Using %d for the hash shift.\n", shift); - - if ( populate_memnodemap(nodes, numnodes, shift, nodeids) != 1 ) - { - printk(KERN_INFO "Your memory is not aligned you need to " - "rebuild your hypervisor with a bigger NODEMAPSIZE " - "shift=%d\n", shift); - return -1; - } - - return shift; -} -/* initialize NODE_DATA given nodeid and start/end */ -void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end) -{ - unsigned long start_pfn = paddr_to_pfn(start); - unsigned long end_pfn = paddr_to_pfn(end); - - NODE_DATA(nodeid)->node_start_pfn = start_pfn; - NODE_DATA(nodeid)->node_spanned_pages = end_pfn - start_pfn; - - node_set_online(nodeid); -} - -void __init numa_init_array(void) -{ - int rr, i; - - /* There are unfortunately some poorly designed mainboards around - that only connect memory to a single CPU. This breaks the 1:1 cpu->node - mapping. To avoid this fill in the mapping for all possible - CPUs, as the number of CPUs is not known yet. - We round robin the existing nodes. */ - rr = first_node(node_online_map); - for ( i = 0; i < nr_cpu_ids; i++ ) - { - if ( cpu_to_node[i] != NUMA_NO_NODE ) - continue; - numa_set_node(i, rr); - rr = cycle_node(rr, node_online_map); - } -} - -#ifdef CONFIG_NUMA_EMU -static int numa_fake __initdata = 0; - -/* Numa emulation */ -static int __init numa_emulation(unsigned long start_pfn, - unsigned long end_pfn) -{ - int i; - struct node nodes[MAX_NUMNODES]; - uint64_t sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake; - - /* Kludge needed for the hash function */ - if ( hweight64(sz) > 1 ) - { - u64 x = 1; - while ( (x << 1) < sz ) - x <<= 1; - if ( x < sz/2 ) - printk(KERN_ERR "Numa emulation unbalanced. Complain to maintainer\n"); - sz = x; - } - - memset(&nodes,0,sizeof(nodes)); - for ( i = 0; i < numa_fake; i++ ) - { - nodes[i].start = pfn_to_paddr(start_pfn) + i * sz; - if ( i == numa_fake - 1 ) - sz = pfn_to_paddr(end_pfn) - nodes[i].start; - nodes[i].end = nodes[i].start + sz; - printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"MB)\n", - i, - nodes[i].start, nodes[i].end, - (nodes[i].end - nodes[i].start) >> 20); - node_set_online(i); - } - memnode_shift = compute_hash_shift(nodes, numa_fake, NULL); - if ( memnode_shift < 0 ) - { - memnode_shift = 0; - printk(KERN_ERR "No NUMA hash function found. Emulation disabled.\n"); - return -1; - } - for_each_online_node ( i ) - setup_node_bootmem(i, nodes[i].start, nodes[i].end); - numa_init_array(); - - return 0; -} -#endif - -void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn) -{ - int i; - paddr_t start = pfn_to_paddr(start_pfn); - paddr_t end = pfn_to_paddr(end_pfn); - -#ifdef CONFIG_NUMA_EMU - if ( numa_fake && !numa_emulation(start_pfn, end_pfn) ) - return; -#endif - -#ifdef CONFIG_ACPI_NUMA - if ( numa_status != numa_off && !acpi_scan_nodes(start, end) ) - return; -#endif - - printk(KERN_INFO "%s\n", - numa_status == numa_off ? "NUMA turned off" - : "No NUMA configuration found"); - - printk(KERN_INFO "Faking a node at %"PRIpaddr"-%"PRIpaddr"\n", - start, end); - /* setup dummy node covering all memory */ - memnode_shift = BITS_PER_LONG - 1; - memnodemap = _memnodemap; - /* Dummy node only uses 1 slot in reality */ - memnodemap[0] = 0; - memnodemapsize = 1; - - nodes_clear(node_online_map); - node_set_online(0); - for ( i = 0; i < nr_cpu_ids; i++ ) - numa_set_node(i, 0); - cpumask_copy(&node_to_cpumask[0], cpumask_of(0)); - setup_node_bootmem(0, start, end); -} - -void numa_add_cpu(int cpu) -{ - cpumask_set_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); -} - -void numa_set_node(int cpu, nodeid_t node) -{ - cpu_to_node[cpu] = node; -} - -/* [numa=off] */ -static int __init cf_check numa_setup(const char *opt) -{ - if ( !strncmp(opt,"off",3) ) - numa_status = numa_off; - else if ( !strncmp(opt,"on",2) ) - numa_status = numa_on; -#ifdef CONFIG_NUMA_EMU - else if ( !strncmp(opt, "fake=", 5) ) - { - numa_status = numa_on; - numa_fake = simple_strtoul(opt+5,NULL,0); - if ( numa_fake >= MAX_NUMNODES ) - numa_fake = MAX_NUMNODES; - } -#endif -#ifdef CONFIG_ACPI_NUMA - else if ( !strncmp(opt,"noacpi",6) ) - numa_status = numa_no_acpi; -#endif - else - return -EINVAL; - - return 0; -} -custom_param("numa", numa_setup); - /* * Setup early cpu_to_node. * @@ -368,146 +82,3 @@ unsigned int __init arch_get_dma_bitsize(void) flsl(node_start_pfn(node) + node_spanned_pages(node) / 4 - 1) + PAGE_SHIFT, 32); } - -static void cf_check dump_numa(unsigned char key) -{ - s_time_t now = NOW(); - unsigned int i, j, n; - struct domain *d; - struct page_info *page; - unsigned int page_num_node[MAX_NUMNODES]; - const struct vnuma_info *vnuma; - - printk("'%c' pressed -> dumping numa info (now = %"PRI_stime")\n", key, - now); - - for_each_online_node ( i ) - { - paddr_t pa = pfn_to_paddr(node_start_pfn(i) + 1); - - printk("NODE%u start->%lu size->%lu free->%lu\n", - i, node_start_pfn(i), node_spanned_pages(i), - avail_node_heap_pages(i)); - /* sanity check phys_to_nid() */ - if ( phys_to_nid(pa) != i ) - printk("phys_to_nid(%"PRIpaddr") -> %d should be %u\n", - pa, phys_to_nid(pa), i); - } - - j = cpumask_first(&cpu_online_map); - n = 0; - for_each_online_cpu ( i ) - { - if ( i != j + n || cpu_to_node[j] != cpu_to_node[i] ) - { - if ( n > 1 ) - printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]); - else - printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); - j = i; - n = 1; - } - else - ++n; - } - if ( n > 1 ) - printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]); - else - printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); - - rcu_read_lock(&domlist_read_lock); - - printk("Memory location of each domain:\n"); - for_each_domain ( d ) - { - process_pending_softirqs(); - - printk("Domain %u (total: %u):\n", d->domain_id, domain_tot_pages(d)); - - for_each_online_node ( i ) - page_num_node[i] = 0; - - spin_lock(&d->page_alloc_lock); - page_list_for_each(page, &d->page_list) - { - i = phys_to_nid(page_to_maddr(page)); - page_num_node[i]++; - } - spin_unlock(&d->page_alloc_lock); - - for_each_online_node ( i ) - printk(" Node %u: %u\n", i, page_num_node[i]); - - if ( !read_trylock(&d->vnuma_rwlock) ) - continue; - - if ( !d->vnuma ) - { - read_unlock(&d->vnuma_rwlock); - continue; - } - - vnuma = d->vnuma; - printk(" %u vnodes, %u vcpus, guest physical layout:\n", - vnuma->nr_vnodes, d->max_vcpus); - for ( i = 0; i < vnuma->nr_vnodes; i++ ) - { - unsigned int start_cpu = ~0U; - - if ( vnuma->vnode_to_pnode[i] == NUMA_NO_NODE ) - printk(" %3u: pnode ???,", i); - else - printk(" %3u: pnode %3u,", i, vnuma->vnode_to_pnode[i]); - - printk(" vcpus "); - - for ( j = 0; j < d->max_vcpus; j++ ) - { - if ( !(j & 0x3f) ) - process_pending_softirqs(); - - if ( vnuma->vcpu_to_vnode[j] == i ) - { - if ( start_cpu == ~0U ) - { - printk("%d", j); - start_cpu = j; - } - } - else if ( start_cpu != ~0U ) - { - if ( j - 1 != start_cpu ) - printk("-%d ", j - 1); - else - printk(" "); - start_cpu = ~0U; - } - } - - if ( start_cpu != ~0U && start_cpu != j - 1 ) - printk("-%d", j - 1); - - printk("\n"); - - for ( j = 0; j < vnuma->nr_vmemranges; j++ ) - { - if ( vnuma->vmemrange[j].nid == i ) - printk(" %016"PRIx64" - %016"PRIx64"\n", - vnuma->vmemrange[j].start, - vnuma->vmemrange[j].end); - } - } - - read_unlock(&d->vnuma_rwlock); - } - - rcu_read_unlock(&domlist_read_lock); -} - -static int __init cf_check register_numa_trigger(void) -{ - register_keyhandler('u', dump_numa, "dump NUMA info", 1); - return 0; -} -__initcall(register_numa_trigger); - diff --git a/xen/common/Makefile b/xen/common/Makefile index 3baf83d527..9a3a12b12d 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -26,6 +26,7 @@ obj-$(CONFIG_MEM_ACCESS) += mem_access.o obj-y += memory.o obj-y += multicall.o obj-y += notifier.o +obj-$(CONFIG_NUMA) += numa.o obj-y += page_alloc.o obj-$(CONFIG_HAS_PDX) += pdx.o obj-$(CONFIG_PERF_COUNTERS) += perfc.o diff --git a/xen/common/numa.c b/xen/common/numa.c new file mode 100644 index 0000000000..bc6a2ded14 --- /dev/null +++ b/xen/common/numa.c @@ -0,0 +1,439 @@ +/* + * Generic VM initialization for NUMA setups. + * Copyright 2002,2003 Andi Kleen, SuSE Labs. + * Adapted for Xen: Ryan Harper + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct node_data node_data[MAX_NUMNODES]; + +/* Mapping from pdx to node id */ +int memnode_shift; +static typeof(*memnodemap) _memnodemap[64]; +unsigned long memnodemapsize; +u8 *memnodemap; + +nodeid_t cpu_to_node[NR_CPUS] __read_mostly = { + [0 ... NR_CPUS-1] = NUMA_NO_NODE +}; + +cpumask_t node_to_cpumask[MAX_NUMNODES] __read_mostly; + +nodemask_t __read_mostly node_online_map = { { [0] = 1UL } }; + +enum numa_mode numa_status; + +/* + * Given a shift value, try to populate memnodemap[] + * Returns : + * 1 if OK + * 0 if memnodmap[] too small (of shift too small) + * -1 if node overlap or lost ram (shift too big) + */ +static int __init populate_memnodemap(const struct node *nodes, + int numnodes, int shift, nodeid_t *nodeids) +{ + unsigned long spdx, epdx; + int i, res = -1; + + memset(memnodemap, NUMA_NO_NODE, memnodemapsize * sizeof(*memnodemap)); + for ( i = 0; i < numnodes; i++ ) + { + spdx = paddr_to_pdx(nodes[i].start); + epdx = paddr_to_pdx(nodes[i].end - 1) + 1; + if ( spdx >= epdx ) + continue; + if ( (epdx >> shift) >= memnodemapsize ) + return 0; + do { + if ( memnodemap[spdx >> shift] != NUMA_NO_NODE ) + return -1; + + if ( !nodeids ) + memnodemap[spdx >> shift] = i; + else + memnodemap[spdx >> shift] = nodeids[i]; + + spdx += (1UL << shift); + } while ( spdx < epdx ); + res = 1; + } + + return res; +} + +static int __init allocate_cachealigned_memnodemap(void) +{ + unsigned long size = PFN_UP(memnodemapsize * sizeof(*memnodemap)); + unsigned long mfn = mfn_x(alloc_boot_pages(size, 1)); + + memnodemap = mfn_to_virt(mfn); + mfn <<= PAGE_SHIFT; + size <<= PAGE_SHIFT; + printk(KERN_DEBUG "NUMA: Allocated memnodemap from %lx - %lx\n", + mfn, mfn + size); + memnodemapsize = size / sizeof(*memnodemap); + + return 0; +} + +/* + * The LSB of all start and end addresses in the node map is the value of the + * maximum possible shift. + */ +static int __init extract_lsb_from_nodes(const struct node *nodes, + int numnodes) +{ + int i, nodes_used = 0; + unsigned long spdx, epdx; + unsigned long bitfield = 0, memtop = 0; + + for ( i = 0; i < numnodes; i++ ) + { + spdx = paddr_to_pdx(nodes[i].start); + epdx = paddr_to_pdx(nodes[i].end - 1) + 1; + if ( spdx >= epdx ) + continue; + bitfield |= spdx; + nodes_used++; + if ( epdx > memtop ) + memtop = epdx; + } + if ( nodes_used <= 1 ) + i = BITS_PER_LONG - 1; + else + i = find_first_bit(&bitfield, sizeof(unsigned long)*8); + memnodemapsize = (memtop >> i) + 1; + return i; +} + +int __init compute_hash_shift(struct node *nodes, int numnodes, + nodeid_t *nodeids) +{ + int shift; + + shift = extract_lsb_from_nodes(nodes, numnodes); + if ( memnodemapsize <= ARRAY_SIZE(_memnodemap) ) + memnodemap = _memnodemap; + else if ( allocate_cachealigned_memnodemap() ) + return -1; + printk(KERN_DEBUG "NUMA: Using %d for the hash shift.\n", shift); + + if ( populate_memnodemap(nodes, numnodes, shift, nodeids) != 1 ) + { + printk(KERN_INFO "Your memory is not aligned you need to " + "rebuild your hypervisor with a bigger NODEMAPSIZE " + "shift=%d\n", shift); + return -1; + } + + return shift; +} + +/* initialize NODE_DATA given nodeid and start/end */ +void __init setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end) +{ + unsigned long start_pfn = paddr_to_pfn(start); + unsigned long end_pfn = paddr_to_pfn(end); + + NODE_DATA(nodeid)->node_start_pfn = start_pfn; + NODE_DATA(nodeid)->node_spanned_pages = end_pfn - start_pfn; + + node_set_online(nodeid); +} + +void __init numa_init_array(void) +{ + int rr, i; + + /* + * There are unfortunately some poorly designed mainboards around + * that only connect memory to a single CPU. This breaks the 1:1 cpu->node + * mapping. To avoid this fill in the mapping for all possible + * CPUs, as the number of CPUs is not known yet. + * We round robin the existing nodes. + */ + rr = first_node(node_online_map); + for ( i = 0; i < nr_cpu_ids; i++ ) + { + if ( cpu_to_node[i] != NUMA_NO_NODE ) + continue; + numa_set_node(i, rr); + rr = cycle_node(rr, node_online_map); + } +} + +#ifdef CONFIG_NUMA_EMU +static int numa_fake __initdata = 0; + +/* Numa emulation */ +static int __init numa_emulation(unsigned long start_pfn, + unsigned long end_pfn) +{ + int i; + struct node nodes[MAX_NUMNODES]; + uint64_t sz = pfn_to_paddr(end_pfn - start_pfn) / numa_fake; + + /* Kludge needed for the hash function */ + if ( hweight64(sz) > 1 ) + { + u64 x = 1; + while ( (x << 1) < sz ) + x <<= 1; + if ( x < sz/2 ) + printk(KERN_ERR "Numa emulation unbalanced. Complain to maintainer\n"); + sz = x; + } + + memset(&nodes,0,sizeof(nodes)); + for ( i = 0; i < numa_fake; i++ ) + { + nodes[i].start = pfn_to_paddr(start_pfn) + i * sz; + if ( i == numa_fake - 1 ) + sz = pfn_to_paddr(end_pfn) - nodes[i].start; + nodes[i].end = nodes[i].start + sz; + printk(KERN_INFO "Faking node %d at %"PRIx64"-%"PRIx64" (%"PRIu64"MB)\n", + i, + nodes[i].start, nodes[i].end, + (nodes[i].end - nodes[i].start) >> 20); + node_set_online(i); + } + memnode_shift = compute_hash_shift(nodes, numa_fake, NULL); + if ( memnode_shift < 0 ) + { + memnode_shift = 0; + printk(KERN_ERR "No NUMA hash function found. Emulation disabled.\n"); + return -1; + } + for_each_online_node ( i ) + setup_node_bootmem(i, nodes[i].start, nodes[i].end); + numa_init_array(); + + return 0; +} +#endif + +void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn) +{ + int i; + paddr_t start = pfn_to_paddr(start_pfn); + paddr_t end = pfn_to_paddr(end_pfn); + +#ifdef CONFIG_NUMA_EMU + if ( numa_fake && !numa_emulation(start_pfn, end_pfn) ) + return; +#endif + +#ifdef CONFIG_ACPI_NUMA + if ( numa_status != numa_off && !acpi_scan_nodes(start, end) ) + return; +#endif + + printk(KERN_INFO "%s\n", + numa_status == numa_off ? "NUMA turned off" + : "No NUMA configuration found"); + + printk(KERN_INFO "Faking a node at %"PRIpaddr"-%"PRIpaddr"\n", + start, end); + /* setup dummy node covering all memory */ + memnode_shift = BITS_PER_LONG - 1; + memnodemap = _memnodemap; + /* Dummy node only uses 1 slot in reality */ + memnodemap[0] = 0; + memnodemapsize = 1; + + nodes_clear(node_online_map); + node_set_online(0); + for ( i = 0; i < nr_cpu_ids; i++ ) + numa_set_node(i, 0); + cpumask_copy(&node_to_cpumask[0], cpumask_of(0)); + setup_node_bootmem(0, start, end); +} + +void numa_add_cpu(int cpu) +{ + cpumask_set_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); +} + +void numa_set_node(int cpu, nodeid_t node) +{ + cpu_to_node[cpu] = node; +} + +/* [numa=off] */ +static int __init cf_check numa_setup(const char *opt) +{ + if ( !strncmp(opt,"off",3) ) + numa_status = numa_off; + else if ( !strncmp(opt,"on",2) ) + numa_status = numa_on; +#ifdef CONFIG_NUMA_EMU + else if ( !strncmp(opt, "fake=", 5) ) + { + numa_status = numa_on; + numa_fake = simple_strtoul(opt+5,NULL,0); + if ( numa_fake >= MAX_NUMNODES ) + numa_fake = MAX_NUMNODES; + } +#endif +#ifdef CONFIG_ACPI_NUMA + else if ( !strncmp(opt,"noacpi",6) ) + numa_status = numa_no_acpi; +#endif + else + return -EINVAL; + + return 0; +} +custom_param("numa", numa_setup); + +static void cf_check dump_numa(unsigned char key) +{ + s_time_t now = NOW(); + unsigned int i, j, n; + struct domain *d; + struct page_info *page; + unsigned int page_num_node[MAX_NUMNODES]; + const struct vnuma_info *vnuma; + + printk("'%c' pressed -> dumping numa info (now = %"PRI_stime")\n", key, + now); + + for_each_online_node ( i ) + { + paddr_t pa = pfn_to_paddr(node_start_pfn(i) + 1); + + printk("NODE%u start->%lu size->%lu free->%lu\n", + i, node_start_pfn(i), node_spanned_pages(i), + avail_node_heap_pages(i)); + /* sanity check phys_to_nid() */ + if ( phys_to_nid(pa) != i ) + printk("phys_to_nid(%"PRIpaddr") -> %d should be %u\n", + pa, phys_to_nid(pa), i); + } + + j = cpumask_first(&cpu_online_map); + n = 0; + for_each_online_cpu ( i ) + { + if ( i != j + n || cpu_to_node[j] != cpu_to_node[i] ) + { + if ( n > 1 ) + printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]); + else + printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); + j = i; + n = 1; + } + else + ++n; + } + if ( n > 1 ) + printk("CPU%u...%u -> NODE%d\n", j, j + n - 1, cpu_to_node[j]); + else + printk("CPU%u -> NODE%d\n", j, cpu_to_node[j]); + + rcu_read_lock(&domlist_read_lock); + + printk("Memory location of each domain:\n"); + for_each_domain ( d ) + { + process_pending_softirqs(); + + printk("Domain %u (total: %u):\n", d->domain_id, domain_tot_pages(d)); + + for_each_online_node ( i ) + page_num_node[i] = 0; + + spin_lock(&d->page_alloc_lock); + page_list_for_each(page, &d->page_list) + { + i = phys_to_nid(page_to_maddr(page)); + page_num_node[i]++; + } + spin_unlock(&d->page_alloc_lock); + + for_each_online_node ( i ) + printk(" Node %u: %u\n", i, page_num_node[i]); + + if ( !read_trylock(&d->vnuma_rwlock) ) + continue; + + if ( !d->vnuma ) + { + read_unlock(&d->vnuma_rwlock); + continue; + } + + vnuma = d->vnuma; + printk(" %u vnodes, %u vcpus, guest physical layout:\n", + vnuma->nr_vnodes, d->max_vcpus); + for ( i = 0; i < vnuma->nr_vnodes; i++ ) + { + unsigned int start_cpu = ~0U; + + if ( vnuma->vnode_to_pnode[i] == NUMA_NO_NODE ) + printk(" %3u: pnode ???,", i); + else + printk(" %3u: pnode %3u,", i, vnuma->vnode_to_pnode[i]); + + printk(" vcpus "); + + for ( j = 0; j < d->max_vcpus; j++ ) + { + if ( !(j & 0x3f) ) + process_pending_softirqs(); + + if ( vnuma->vcpu_to_vnode[j] == i ) + { + if ( start_cpu == ~0U ) + { + printk("%d", j); + start_cpu = j; + } + } + else if ( start_cpu != ~0U ) + { + if ( j - 1 != start_cpu ) + printk("-%d ", j - 1); + else + printk(" "); + start_cpu = ~0U; + } + } + + if ( start_cpu != ~0U && start_cpu != j - 1 ) + printk("-%d", j - 1); + + printk("\n"); + + for ( j = 0; j < vnuma->nr_vmemranges; j++ ) + { + if ( vnuma->vmemrange[j].nid == i ) + printk(" %016"PRIx64" - %016"PRIx64"\n", + vnuma->vmemrange[j].start, + vnuma->vmemrange[j].end); + } + } + + read_unlock(&d->vnuma_rwlock); + } + + rcu_read_unlock(&domlist_read_lock); +} + +static int __init cf_check register_numa_trigger(void) +{ + register_keyhandler('u', dump_numa, "dump NUMA info", 1); + return 0; +} +__initcall(register_numa_trigger); diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 7aef1a88dc..db0e524a0e 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -18,4 +18,78 @@ (((d)->vcpu != NULL && (d)->vcpu[0] != NULL) \ ? vcpu_to_node((d)->vcpu[0]) : NUMA_NO_NODE) +/* The following content can be used when NUMA feature is enabled */ +#ifdef CONFIG_NUMA + +extern nodeid_t cpu_to_node[NR_CPUS]; +extern cpumask_t node_to_cpumask[]; + +#define cpu_to_node(cpu) (cpu_to_node[cpu]) +#define parent_node(node) (node) +#define node_to_first_cpu(node) (__ffs(node_to_cpumask[node])) +#define node_to_cpumask(node) (node_to_cpumask[node]) + +struct node { + paddr_t start, end; +}; + +extern int compute_hash_shift(struct node *nodes, int numnodes, + nodeid_t *nodeids); + +#define VIRTUAL_BUG_ON(x) + +/* Enumerations for NUMA status. */ +enum numa_mode { + numa_on = 0, + numa_off, + /* NUMA turns on, but ACPI table is bad or disabled. */ + numa_no_acpi, + /* NUMA turns on, and ACPI table works well. */ + numa_acpi, +}; + +extern void numa_add_cpu(int cpu); +extern void numa_init_array(void); +extern void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn); +extern bool numa_enabled_with_firmware(void); +extern enum numa_mode numa_status; + +extern void numa_set_node(int cpu, nodeid_t node); +extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end); + +static inline void clear_node_cpumask(int cpu) +{ + cpumask_clear_cpu(cpu, &node_to_cpumask[cpu_to_node(cpu)]); +} + +/* Simple perfect hash to map pdx to node numbers */ +extern int memnode_shift; +extern unsigned long memnodemapsize; +extern u8 *memnodemap; + +struct node_data { + unsigned long node_start_pfn; + unsigned long node_spanned_pages; +}; + +extern struct node_data node_data[]; + +static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr) +{ + nodeid_t nid; + VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >= memnodemapsize); + nid = memnodemap[paddr_to_pdx(addr) >> memnode_shift]; + VIRTUAL_BUG_ON(nid >= MAX_NUMNODES || !node_data[nid]); + return nid; +} + +#define NODE_DATA(nid) (&(node_data[nid])) + +#define node_start_pfn(nid) (NODE_DATA(nid)->node_start_pfn) +#define node_spanned_pages(nid) (NODE_DATA(nid)->node_spanned_pages) +#define node_end_pfn(nid) (NODE_DATA(nid)->node_start_pfn + \ + NODE_DATA(nid)->node_spanned_pages) + +#endif + #endif /* _XEN_NUMA_H */ From patchwork Fri Jul 8 14:54:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0E964C43334 for ; Fri, 8 Jul 2022 14:55:39 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363470.594030 (Exim 4.92) (envelope-from ) id 1o9pOB-0007Wi-Td; Fri, 08 Jul 2022 14:55:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363470.594030; Fri, 08 Jul 2022 14:55:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOB-0007Wb-QW; Fri, 08 Jul 2022 14:55:27 +0000 Received: by outflank-mailman (input) for mailman id 363470; Fri, 08 Jul 2022 14:55:27 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOA-0006vD-Pd for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:27 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2061.outbound.protection.outlook.com [40.107.20.61]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ffd1eb02-fecd-11ec-924f-1f966e50362f; Fri, 08 Jul 2022 16:55:25 +0200 (CEST) Received: from AM5P194CA0006.EURP194.PROD.OUTLOOK.COM (2603:10a6:203:8f::16) by PAXPR08MB6912.eurprd08.prod.outlook.com (2603:10a6:102:137::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:24 +0000 Received: from AM5EUR03FT064.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:8f:cafe::34) by AM5P194CA0006.outlook.office365.com (2603:10a6:203:8f::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16 via Frontend Transport; Fri, 8 Jul 2022 14:55:24 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT064.mail.protection.outlook.com (10.152.17.53) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16 via Frontend Transport; Fri, 8 Jul 2022 14:55:23 +0000 Received: ("Tessian outbound 6f9e7ef31fa8:v122"); Fri, 08 Jul 2022 14:55:23 +0000 Received: from 32f413fbdd89.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id C20FECCA-9427-4FEF-822D-2A857CD2E588.1; Fri, 08 Jul 2022 14:55:17 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 32f413fbdd89.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:17 +0000 Received: from DB6PR0202CA0044.eurprd02.prod.outlook.com (2603:10a6:4:a5::30) by DB9PR08MB6649.eurprd08.prod.outlook.com (2603:10a6:10:26c::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:14 +0000 Received: from DBAEUR03FT058.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:a5:cafe::6a) by DB6PR0202CA0044.outlook.office365.com (2603:10a6:4:a5::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20 via Frontend Transport; Fri, 8 Jul 2022 14:55:14 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT058.mail.protection.outlook.com (100.127.142.120) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:14 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:07 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:55:04 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ffd1eb02-fecd-11ec-924f-1f966e50362f ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=Wmw9uhcQvlqoQfsqQKnCb6GYRbv0nfzcevcMi9oDjhtet8V49Jka6Cn7iN71MV7YJs4OmYjZcTRvKhk6AvO6do8jtTUHb2XG2009Ml0rwsDSXVpEkE7qEzB60+tb4cGmEtKKoG3n41GRLMJiaME6pIwFtbvkplNSaTU2lcMopYt7yJEqRBK5hD+LOtBUB3OltG85t2pwWQ5HH4ZOrHv0It7yhM2ozas+/IJBNjlNNRrqTwvKHr8YLatgFN1BRgEQPSg2OmP9zO0AlYoIbAmdgODvYkXO3vEKo/LUIsjoNMuJ685k89nBUYPf5vCWgYkMa3HIKuoOAMxCETG7J7BROA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6hrldKnD93EUpTTS3t0Fp32LcqOWPG36yIxeFRBs9ew=; b=U1D0CzkuG/bS9NDAKPHyY3uDiuz20Eavvyfo8M0Ld/kL7u7MUffbRK8hcSFXUuo0WbEOo2dXfFm/EFEmvmzaBiUAbDrAEiPRzs4XNeU6plEDi+X8CErEZhJ8qZM1Uk0fzqfm/MIgzf8F2AvvrUbNPx4IKi1mH3DM+qbGoqI3CylWi3zKgEG+fbsnr2s/cPEvXNZIxs2+Uy7Tg5OKRNgSQh6OTQCj65Ol9iWW47/PJ5ROu72qxHqnxvlcUkxuoZry63dzcIzclpmpUOFA6q8JXxiUmXdILcEwTSYzfcw+e70juzaLOHKUSWY18c4DNXx2f82eNAaWe7416LrslsJ7Ng== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6hrldKnD93EUpTTS3t0Fp32LcqOWPG36yIxeFRBs9ew=; b=W52zosK23K2Dbskpj5kcMvByi2K0X6WLHttkygQDXQZvtxBCh8XLD+LHhyIGxyuSMSp2IzB+S8RQJA6n3p373HZQXwQb9B0JIW/YMcBoeQj5XDiWILkZw8uzBoQrfKmqc4DMy49thp1uNd5boMfLRmGhlp305LGYFvw9lAm2fyM= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 1e8f0e31302abb90 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DIHrdJvFu0vMjmIyUBGHBwFFYq4cd+uNR0WGKg+FeGnM3E1Ro3ofkprkX0OGeJCnj11SucXA2RXUGHbQS7Xvk06DCyrWY22zLbeHKarJbcs8SRmNNz1XoiU6r7Nt1b7cACMAL0tspsqeIoH6A8t3bJFHoe9VX/kLnZirAv58TnLudgcm67+qVNNW4q/yqytHesY7vFBjUuYeqrZE8cLqhGIlojz2/QN3JBKvlwnRkxkOV+FVtgZim491wgG+N0AYI1lMi+qOkhfBOL6nX5WqYRmlz4LixSfFitigKpLaRYJ2SExb15uwJcaldq8Fc/2j2J4VrKvv5jLjfA180po3Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6hrldKnD93EUpTTS3t0Fp32LcqOWPG36yIxeFRBs9ew=; b=Ij8niYl0dRhV3OChGMOKPu08IIiWnM66A9e7957ZEBaQk3RDITL87wehcfZmj/kN7X5Xu8nTQIwD2m7wqDjTkGx4uwDXw9ifacl8rC4tO+Vrp3sXpK8Bo4AUneYGix4Q88TLb/M1Pk6MrgrZIP8f5gKWjhuvpcmhu1us6rwtAD/OgfdeG+Z85tnIzkrnb+x6S+frN5FeocAc5kZg5InTK9enykOekU0msmh08G1MvOc5wAvQYeYib1u3asu4UuV8MdkgAXNSrzB5gTj8Vg07WMkKG/quuc3/HK5J/7x3YHOBERoT3Q6uiZyNafa0gRgUS/xcMX5C5Cvf5PtzdwoIKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6hrldKnD93EUpTTS3t0Fp32LcqOWPG36yIxeFRBs9ew=; b=W52zosK23K2Dbskpj5kcMvByi2K0X6WLHttkygQDXQZvtxBCh8XLD+LHhyIGxyuSMSp2IzB+S8RQJA6n3p373HZQXwQb9B0JIW/YMcBoeQj5XDiWILkZw8uzBoQrfKmqc4DMy49thp1uNd5boMfLRmGhlp305LGYFvw9lAm2fyM= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Andrew Cooper , George Dunlap , "Jan Beulich" , Julien Grall , "Stefano Stabellini" , Wei Liu , Jiamei Xie Subject: [PATCH v2 4/9] xen/x86: Use ASSERT instead of VIRTUAL_BUG_ON for phys_to_nid Date: Fri, 8 Jul 2022 22:54:19 +0800 Message-ID: <20220708145424.1848572-5-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: 371f0f16-54cb-4d99-5546-08da60f1e2f3 X-MS-TrafficTypeDiagnostic: DB9PR08MB6649:EE_|AM5EUR03FT064:EE_|PAXPR08MB6912:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: C/TH2QUcBlatTcsOZce6fy1N79coYK1iS0Qql1xPWeWxV05ah19+w9cigN5uxOtvaCdvkndztLMYafHHdCW5WMgOQsOwwmOJFvsy/9AhRIILOLUo89Bt+av9NKsln+9XijMJQ+giOVD0R/UqyGZw9WSVfECn97813l1FCYkvt07f692KsPS4+v242jq5owAou+APQ70+rP340jEgHhHqX+kkLzz/vMWWtLif04vgJ5KczJxwZzFKeYp4CmYUltlobVxO7WCMZZ0poB683ppqGeLzPXx8vs6z39WNnCPd3QalSwWyfzdLhE3oFxlfIrv8ZABPyiyjvaZ8/g/Zx4QUe/J6wDM3WY9B1Ap6SYd41nSHZ6BPCVByTB5uGKyHHP5Dk00lyWRZHs8QzqmCx6dst/qmkZC55iXvkuHzW6Vexm/1CabCRK/RYxWj5fqIgG5GOTjCgWG1HVk1yTvhz1f0PVGB6sG8mHnbY8GQk5AcnuOfpX+TQdlay2mJgGwhSSEDr74GTa608htGBiqCCfffUCV1DXzhYMiqseKsoyvbsru4lrJRllXqwH3Y+hzpVKtwR7T95sdMlDNBELyxaXZnjf49QKMfoRRz4o6oU+OCDdgFEsSZkyivrO0e8u0j1un3fTDmjeU0XUIIjrMiA1JzyKl4Xmv7uSJWI4s6b6o7lJjb27GxVO/J37dpIV1SRxHGFyeq20x4o6nP1RErNefGf5+NPOBTN92wulr+Q5adiYMXcd9KjXQKtnOLPO65iVBWMUNCoHVXQCS8cTmcX6TkS7zsoZpvPdUoH3y7GoWu7LzyRYC+gC/7AN/hTeOSaaEtzoKe1JjuQ8I8XVKZyqwcLGsWB01dmc3SxYz/czNI+yk= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(396003)(39860400002)(136003)(376002)(46966006)(36840700001)(40470700004)(5660300002)(8936002)(44832011)(2906002)(40480700001)(40460700003)(82310400005)(83380400001)(36860700001)(82740400003)(86362001)(6666004)(81166007)(186003)(36756003)(6916009)(54906003)(7696005)(70206006)(70586007)(2616005)(4326008)(316002)(356005)(1076003)(26005)(47076005)(336012)(41300700001)(478600001)(426003)(8676002)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB6649 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT064.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 0cccf23a-e4a2-49e3-7d59-08da60f1dd49 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: t+y/a9yq0ij6V+9RnM/faSB+7E8mYS+EDOoqUYD5qi0SL/HTW5kjXZc6lMgbq5kXKILjhdnnRmsGwUJH7dCJU9UF6UeYF5B/77boXc5h6sdKhZ190ydLK0Q9LTXM2RYmna0AiFIkECtGynA7BpyxFW0QGawgEALuNfP5JGufkuwYrbl9efPtW2jSbPJ2N6ae6ncW/kSNdR4G9uudCpoAUnwfsUWqCK2p9rX+vZnQGedyihVWFvN5UXTvrq162UysYwcqYB+yMrnuLvGSis3BWOtNFWl4bSWM9J6KeNmEh3f3OowtlzeeFgYDhO9TAYoVi3/grBYAHq4bVfbkIg8torepf0pAPUvQnHLkOBTBTZX0pvnp6ugykDDn/HEZ/bBZHrMYLIrgKPD8Ks+f03XnPmLvyiiN/oggBmJ1gUFIDssSMbXVQl0zZgfjIuhO5fbnpL0aq+QG53htlB8g/rfaByM1wj7q73OSk95Mm2fSTt6hzPrF4YeVa8bAJ20yrnplbPTCevZQATK+CLMe5pjB1gabDYSw8PClZ24NDLA8WlKUal1NVX3j6JJBOrjvV0oCRdztfBotdmrwDVgDz4CJJ0wwn1R/h9E3OfKwhg1Q/q5T7yyYYIijrVLjZhQ5ik4GFjqUy6sqogjJO6190rgB4PaoBesIhA/mEJeOQIqs4n0mRSRj4mtbXuR0BZoAcKCPSPuv/zW2ga91M0NVjAwYGgSIMqMJTuHoXWpYbfFFA2S2QFyOfDQoKutnsQ7F3YFalxQscI/YWB8wDgEJg83czUJKlnZIa6l/JLk0qSSkGlOrSKs76kq5/yZhDXyDPMM8 X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(346002)(396003)(39860400002)(376002)(136003)(36840700001)(46966006)(40470700004)(336012)(41300700001)(40460700003)(36860700001)(6666004)(426003)(47076005)(1076003)(26005)(2616005)(316002)(7696005)(6916009)(83380400001)(54906003)(36756003)(2906002)(5660300002)(8936002)(81166007)(40480700001)(70206006)(4326008)(8676002)(44832011)(86362001)(70586007)(186003)(82740400003)(82310400005)(478600001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:23.6770 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 371f0f16-54cb-4d99-5546-08da60f1e2f3 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT064.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR08MB6912 VIRTUAL_BUG_ON is an empty macro used in phys_to_nid. This results in two lines of error-checking code in phys_to_nid that is not actually working and causing two compilation errors: 1. error: "MAX_NUMNODES" undeclared (first use in this function). This is because in the common header file, "MAX_NUMNODES" is defined after the common header file includes the ARCH header file, where phys_to_nid has attempted to use "MAX_NUMNODES". This error was resolved after we moved the phys_to_nid from x86 ARCH header file to common header file. 2. error: wrong type argument to unary exclamation mark. This is because, the error-checking code contains !node_data[nid]. But node_data is a data structure variable, it's not a pointer. So, in this patch, we use ASSERT instead of VIRTUAL_BUG_ON to enable the two lines of error-checking code. And fix the left compilation errors by replacing !node_data[nid] to !node_data[nid].node_spanned_pages. Although NUMA allows one node can only have CPUs but without any memory. And node with 0 bytes of memory might have an entry in memnodemap[] theoretically. But that doesn't mean phys_to_nid can find any valid address from a node with 0 bytes memory. Signed-off-by: Wei Chen Tested-by: Jiamei Xie Acked-by: Jan Beulich --- v1 -> v2: 1. Move from part#1 to part#2. (Comment from NUMA part1 series) 2. Refine the justification of using !node_data[nid].node_spanned_pages. (From NUMA part1 series) 3. Use ASSERT to replace VIRTUAL_BUG_ON in phys_to_nid. 4. Adjust the conditional express for ASSERT. 5. Move MAX_NUMNODES from xen/numa.h to asm/numa.h for x86. 6. Use conditional macro to gate MAX_NUMNODES for other architectures. --- xen/include/xen/numa.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index db0e524a0e..695ad51df0 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -36,8 +36,6 @@ struct node { extern int compute_hash_shift(struct node *nodes, int numnodes, nodeid_t *nodeids); -#define VIRTUAL_BUG_ON(x) - /* Enumerations for NUMA status. */ enum numa_mode { numa_on = 0, @@ -77,9 +75,9 @@ extern struct node_data node_data[]; static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr) { nodeid_t nid; - VIRTUAL_BUG_ON((paddr_to_pdx(addr) >> memnode_shift) >= memnodemapsize); + ASSERT((paddr_to_pdx(addr) >> memnode_shift) < memnodemapsize); nid = memnodemap[paddr_to_pdx(addr) >> memnode_shift]; - VIRTUAL_BUG_ON(nid >= MAX_NUMNODES || !node_data[nid]); + ASSERT(nid < MAX_NUMNODES && node_data[nid].node_spanned_pages); return nid; } From patchwork Fri Jul 8 14:54:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0FF02C43334 for ; Fri, 8 Jul 2022 14:55:50 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363473.594052 (Exim 4.92) (envelope-from ) id 1o9pOL-0008Ku-PJ; Fri, 08 Jul 2022 14:55:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363473.594052; Fri, 08 Jul 2022 14:55:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOL-0008Kh-LL; Fri, 08 Jul 2022 14:55:37 +0000 Received: by outflank-mailman (input) for mailman id 363473; Fri, 08 Jul 2022 14:55:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOK-0007Ai-Hf for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:36 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140089.outbound.protection.outlook.com [40.107.14.89]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 0594bba1-fece-11ec-bd2d-47488cf2e6aa; Fri, 08 Jul 2022 16:55:35 +0200 (CEST) Received: from AS9PR06CA0759.eurprd06.prod.outlook.com (2603:10a6:20b:484::13) by DB8PR08MB4956.eurprd08.prod.outlook.com (2603:10a6:10:e0::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.17; Fri, 8 Jul 2022 14:55:29 +0000 Received: from AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:484:cafe::97) by AS9PR06CA0759.outlook.office365.com (2603:10a6:20b:484::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.17 via Frontend Transport; Fri, 8 Jul 2022 14:55:29 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT015.mail.protection.outlook.com (10.152.16.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:27 +0000 Received: ("Tessian outbound afad550a20c6:v122"); Fri, 08 Jul 2022 14:55:27 +0000 Received: from ad85ffab2f30.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id DC83907C-7725-441D-8A51-276B809E94B0.1; Fri, 08 Jul 2022 14:55:21 +0000 Received: from EUR03-AM5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ad85ffab2f30.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:21 +0000 Received: from DB6PR0202CA0043.eurprd02.prod.outlook.com (2603:10a6:4:a5::29) by DB8PR08MB4170.eurprd08.prod.outlook.com (2603:10a6:10:a7::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:19 +0000 Received: from DBAEUR03FT058.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:a5:cafe::2) by DB6PR0202CA0043.outlook.office365.com (2603:10a6:4:a5::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20 via Frontend Transport; Fri, 8 Jul 2022 14:55:18 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT058.mail.protection.outlook.com (100.127.142.120) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:18 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:11 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:55:08 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0594bba1-fece-11ec-bd2d-47488cf2e6aa ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=bWeQtnwn7vpeYBK9GVTJ5pfyzM2KNiPqjJXFddUeVTs4QS4MXemfL4gH4ONmuKfP0BVQ38mPUqXl7KwEdncpFef5F9MTGLIEn2rq0LQ99vp5hWCDljzZx+DN+gY61S91hilUu6I8jteYGFO8oPNXoNdW6nH+nnVrgeHg/ez4Xg7JWO2LRox0mcs/xpHURGX/Hvxu3Ygk2I2LRu4qCyzec1ZiTKW0iNYVzXvLGyfgHKzkkco4I+r9AlGPY3nvHbrnV+fx21UzcbB4k/HfoiFVA9TrUe69LOP6QnQAV8m2hNJqGJf5GYkkzNB+LZ4sjstZD2O2DgfDREdL7xP/z1EgkA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=d9ZTzLCFsbwcnMhhUkqTjENjME/eU+MsQ1SV7aGwsNU=; b=SetGHbioTzqDcKYyNmDgcTJaek88+mTJjjYDMBG/JdpcLDrGPg2XBRIVsXgsjA7eCRve+bfKWuBbRi4o1FoZ5jT9nQk1sVA0uRlEPL203WlpPOtu3GU8uqucxmIlECwHkYxJW62EofqjdLvXTtTUzbTHAdsQj5lU2aeNLM8BP3+vNm7r0lJoW+rmkeO+Kw3lVyFiL8Qr/19wFf/hr064qGejtZ8Wv4PDpl8g7zxVqyXgfEpbl/S0s1wYpRDChKjqj5gUFsSdw5gTtIBZO0DKToG+Kng0ouVijhO1aB1z8EoHwUwEbqPWAo/ovmygvWzb1ilCp4wwhY+lWldVC4YcWw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d9ZTzLCFsbwcnMhhUkqTjENjME/eU+MsQ1SV7aGwsNU=; b=64/QokKx4zCtG6mPCpg/5Jmq3QyyEV2ffKC1gJIEIwstes/1RPXmmJiPtznFuW+lhe5FxP/V7pePQOVKtY2iw1YW3OPo/4VSx02A4X9isfojQwa2O+zqglMs29LFP4XD2nxFxUqcr+FXRG36j3o6SRNWkaMaIzkcIcLf3T+blRQ= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 48b2e135b2c418b6 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mC5YUSdxGVt+CAPSNB1J/xpfu4uuMgFdkcIudF1egaAqa3Ti2x+aqu1cwgPq1oAxen6KpiJtxntz7aZPII3o+EAYt0gPVPG/AGAgXDjM3D+Fx40Gk5eUk2TVydVCeJEyahVhEvHAEkj5jvXo1R1ApllvAhC3TrLdrBSXZU4Z33AVYHe1PrpqrWtYnb9y3D9f/R+V/htSVzW01p2jrRr27OoSnUtItMCx/D+8nDek+vZCqlK3Em81nZGStozct28Yq7HghkP9fEmLFAs3C3O8lZB/zRNjnWGndUL+wCQyIslIzowuILi/UAXb57c2EPZ558JJxn0Hz+qxIgArUJVBaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=d9ZTzLCFsbwcnMhhUkqTjENjME/eU+MsQ1SV7aGwsNU=; b=kjSzMF7GpcPkwtBF5OLqLwb4TbYyOvLUSjwvV6hzlGvWqvnfkdHlvbysR+g1o3AWESq2Pf6vPBRbsUhwN/KO8ZMNyMawRuYWKEffS4TqS5+wTMK6l52UayEkWBL9yC22w26ZGqUB+KWfyUG413szFTCTl/Z1eWFKxU6B70cMmB9XKMeJg3GR+V409FLqUCiwxNdhmwbYhzbwpCp86xP3ck0vbWDTQqxXfJm174wXmxG9mCPy55tNLM9A810k3VwGSd7C4RI5tiInjq3nwbdx2dT9tvUpLp18wnR6NTpdTmvmK4Ff5SH8/tZnd9EdryVJn9oMjdvebfoBGsY3l61zkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=d9ZTzLCFsbwcnMhhUkqTjENjME/eU+MsQ1SV7aGwsNU=; b=64/QokKx4zCtG6mPCpg/5Jmq3QyyEV2ffKC1gJIEIwstes/1RPXmmJiPtznFuW+lhe5FxP/V7pePQOVKtY2iw1YW3OPo/4VSx02A4X9isfojQwa2O+zqglMs29LFP4XD2nxFxUqcr+FXRG36j3o6SRNWkaMaIzkcIcLf3T+blRQ= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v2 5/9] xen/x86: use arch_get_memory_map to get information from E820 map Date: Fri, 8 Jul 2022 22:54:20 +0800 Message-ID: <20220708145424.1848572-6-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: 6870b44f-5bf1-47ad-edf3-08da60f1e566 X-MS-TrafficTypeDiagnostic: DB8PR08MB4170:EE_|AM5EUR03FT015:EE_|DB8PR08MB4956:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 0FdLpdkC/lCco0Buzm0eCtYPSfxKTy4mNh95uHebDev/JH057FQqBG1t0cN2kowQ4fcwD0i3eQynY4d8bGOjWgwaWOMJYaYVHVQZM9oD3ciXg+J7OT661ZGGXD7H5NOpMv5y3kV3D/oA7HG4LurcbSqJjLUjafwvV0bj/fsMCntJrZB8T1nq7yIegnax+KF3FpIQyb++jHB/dvotnLyYqxr9SVU3E1zqh7pnlfvHbzLNWn4YUaQBETHJCzEDet/6qbaHWHGudmjaLopnkUzQLxvSW6yokxuNJKKu2hIJMbpGfqNUkicEjiKwojizMaM54GsO8NbiWyw0gcZyQwG3TpO2fmvfJK8il93eYcAqnmEFqllKOj/m1rn8vUXKatyMe0Oz3Qo5Hej5S7lxme3vfotF/gUuVzdJ5vPQnYD7qzZgPYSotZWId4htUYPvO48ap5YSHdfXB4FmsXv63L6/EmdT4kobl02VpkuLYKaZbblaVymHgfPa/b1o/6bsoOBXkstdUfDLEF/M7Oh/B3jlDCrPnoneUI8cSinAIAu+jaYfDLb32bSvCP9TOIx3+v3XCq4xKmd+rcDZBkS5qi6Vo/ix/6g7Dj6CgaYUPbVtNmRL5KKSA8ZO61CBxLqUY/Y0jyEo6cVqWt0IT+9VzSDlNHdbCT7/btYiwj67CJk8geYyc4DaRgbsALvZjMlCRVt1T94qSCp3frSkoKasZ1z1g5guZDPFWXZTHKzYti23/cnikYyWg14CJAZT+fhirbPD3X43gXxXYyGOVboQjku2rApC/kcr37drYBIxjCenl6XqYVY11H1rCYtgIzrffKE4r/vNrCePO4JZP/YeoUGDqQ63WMO10aUDj+s/UvZdFpg= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(376002)(346002)(39860400002)(136003)(396003)(40470700004)(46966006)(36840700001)(356005)(70586007)(54906003)(8676002)(6916009)(478600001)(36860700001)(81166007)(36756003)(86362001)(316002)(47076005)(336012)(2616005)(26005)(426003)(41300700001)(7696005)(4326008)(186003)(1076003)(6666004)(82310400005)(40480700001)(70206006)(83380400001)(44832011)(5660300002)(8936002)(2906002)(82740400003)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4170 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 35509b3b-379b-4cef-b17a-08da60f1e014 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: o56Jvfuzhg14kFFTQPv9tnebEy4szIO4OOZUelv4tf/2XBnfAmj/c7a4jWhGA+aLRCGZ7AMZF9XkVBiu28ubhP+zUp6KYKG3LYeBlk4R685WwnOyxsRT90LJSrohmrTPpwq0sj0/bz4yWmpw400xgjPzmfh9KKb4I+jO8dFoT70QJJEz3/RpQVcjqifJaDn1gyg9XFRaa0xxEwx+C+iNZO5JEdg6c6IAAzXWnXBS+7QIhlh81neYpFVYc/7AKkn9VfpfT24pc0rTwJs2psYn9C4NIRMfUrSjivcOByhAVD/mhU598CpadovmnkVArFADEzuhbZbFIfR1Ty+BTPIcdYY6waLWZ1RbhVWdA+UlpurlznYurw9CNuBUKJKf8FR0/emaHzmV630ADvtkI83aFXhWMg+KYTO52aXIqDaOYX/LnPSdIR64FBP6KM6cYxswEB+zLRwI56+ARSgOrCKI9fneUka1gzNkmnVQnrJuKKt2/R2EY0wW0bSRHowJtdAdxWLF+A0+8KinjuKCJ2LD99s66o1JYANPaY7nuFzPGkOl/HoifOVI4oMyfX6Efn9otHUKcq84sA6unDCuY2kwJw3QDV5Bjf/nEQol6W4Qi1zgpmoksvplrXN1ZLxKfzxdrCIlDlDn+1QA3oGkVWFaDrrGSDLBwcORbo3w0gIi8UuxZXGMjVY/oETDDf1UInp/FIIRbR3ZYjrBnsSSKeDlJDL1qR/xX+Oxf1rcbKrMOkeusyOLYN0nNn7ogMtOO4lp87JkRxRy3oN283I3Ebivhn3VDRdIzyEjIYSMzjv68YyB6BCtJErPrK6u86ENe7c8 X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(136003)(346002)(396003)(376002)(39860400002)(46966006)(36840700001)(40470700004)(47076005)(82310400005)(107886003)(2616005)(86362001)(316002)(426003)(1076003)(54906003)(186003)(336012)(6916009)(81166007)(36756003)(36860700001)(70206006)(8676002)(4326008)(70586007)(40460700003)(44832011)(26005)(2906002)(5660300002)(82740400003)(478600001)(8936002)(41300700001)(7696005)(83380400001)(40480700001)(6666004);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:27.7834 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6870b44f-5bf1-47ad-edf3-08da60f1e566 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT015.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB4956 The sanity check performed by the nodes_cover_memory function is also necessary for other architectures that do not yet support NUMA. But now, the code of nodes_cover_memory is tied to the x86 E820. In this case, we introduce arch_get_memory_map to decouple architecture specific memory map from this function. This means, other architectures like Arm can also use it to check its bootmem info. Depending on arch_get_memory_map, we make nodes_cover_memory become architecture independent. We also use neutral words to replace SRAT and E820 in the print message of this function. This will make the message more common. As arch_get_memory_map use unsigned int for index, we also adjust the index in nodes_cover_memory from int to unsigned int. Signed-off-by: Wei Chen --- v1 -> v2: 1. Use arch_get_memory_map to replace arch_get_memory_bank_range and arch_get_memory_bank_number. 2. Remove the !start || !end check, because caller guarantee these two pointers will not be NULL. --- xen/arch/x86/numa.c | 23 +++++++++++++++++++++++ xen/arch/x86/srat.c | 18 +++++++++++------- xen/include/xen/numa.h | 3 +++ 3 files changed, 37 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c index 193314dbd9..fb235c07ec 100644 --- a/xen/arch/x86/numa.c +++ b/xen/arch/x86/numa.c @@ -9,6 +9,7 @@ #include #include #include +#include #ifndef Dprintk #define Dprintk(x...) @@ -82,3 +83,25 @@ unsigned int __init arch_get_dma_bitsize(void) flsl(node_start_pfn(node) + node_spanned_pages(node) / 4 - 1) + PAGE_SHIFT, 32); } + +/* + * This function provides the ability for caller to get one RAM entry + * from architectural memory map by index. + * + * This function will return zero if it can return a proper RAM entry. + * otherwise it will return -ENODEV for out of scope index, or return + * -EINVAL for non-RAM type memory entry. + */ +int __init arch_get_memory_map(unsigned int idx, paddr_t *start, paddr_t *end) +{ + if ( idx >= e820.nr_map ) + return -ENODEV; + + if ( e820.map[idx].type != E820_RAM ) + return -EINVAL; + + *start = e820.map[idx].addr; + *end = e820.map[idx].addr + e820.map[idx].size; + + return 0; +} diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 422e4c73e3..5bc9096a15 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -427,18 +427,22 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) Make sure the PXMs cover all memory. */ static int __init nodes_cover_memory(void) { - int i; + unsigned int i; - for (i = 0; i < e820.nr_map; i++) { + for (i = 0; ; i++) { int j, found; paddr_t start, end; - if (e820.map[i].type != E820_RAM) { + /* Try to loop memory map from index 0 to end. */ + found = arch_get_memory_map(i, &start, &end); + + /* Index relate entry is not RAM, skip it. */ + if (found == -EINVAL) continue; - } - start = e820.map[i].addr; - end = e820.map[i].addr + e820.map[i].size; + /* Reach the end of arch's memory map */ + if (found == -ENODEV) + break; do { found = 0; @@ -457,7 +461,7 @@ static int __init nodes_cover_memory(void) } while (found && start < end); if (start < end) { - printk(KERN_ERR "SRAT: No PXM for e820 range: " + printk(KERN_ERR "NUMA: No node for memory map range: " "[%"PRIpaddr", %"PRIpaddr"]\n", start, end - 1); return 0; } diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 695ad51df0..6d02204991 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -88,6 +88,9 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr) #define node_end_pfn(nid) (NODE_DATA(nid)->node_start_pfn + \ NODE_DATA(nid)->node_spanned_pages) +extern int arch_get_memory_map(unsigned int idx, + paddr_t *start, paddr_t *end); + #endif #endif /* _XEN_NUMA_H */ From patchwork Fri Jul 8 14:54:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7F231C43334 for ; Fri, 8 Jul 2022 14:56:01 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363482.594096 (Exim 4.92) (envelope-from ) id 1o9pOX-0001Yf-DI; Fri, 08 Jul 2022 14:55:49 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363482.594096; Fri, 08 Jul 2022 14:55:49 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOX-0001YP-A1; Fri, 08 Jul 2022 14:55:49 +0000 Received: by outflank-mailman (input) for mailman id 363482; Fri, 08 Jul 2022 14:55:47 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOV-0007Ai-BG for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:47 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2042.outbound.protection.outlook.com [40.107.20.42]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 0b8db247-fece-11ec-bd2d-47488cf2e6aa; Fri, 08 Jul 2022 16:55:45 +0200 (CEST) Received: from AS9PR06CA0351.eurprd06.prod.outlook.com (2603:10a6:20b:466::17) by HE1PR0801MB1785.eurprd08.prod.outlook.com (2603:10a6:3:81::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.15; Fri, 8 Jul 2022 14:55:34 +0000 Received: from AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:466:cafe::42) by AS9PR06CA0351.outlook.office365.com (2603:10a6:20b:466::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:34 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT004.mail.protection.outlook.com (10.152.16.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:33 +0000 Received: ("Tessian outbound 190453a6d737:v122"); Fri, 08 Jul 2022 14:55:33 +0000 Received: from 561225665f87.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 54305FD9-A266-49F2-A9C8-21E68CE62572.1; Fri, 08 Jul 2022 14:55:26 +0000 Received: from EUR02-VE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 561225665f87.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:26 +0000 Received: from DB6P192CA0010.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::20) by AM6PR08MB5096.eurprd08.prod.outlook.com (2603:10a6:20b:ee::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:23 +0000 Received: from DBAEUR03FT024.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:b8:cafe::12) by DB6P192CA0010.outlook.office365.com (2603:10a6:4:b8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:23 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT024.mail.protection.outlook.com (100.127.142.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:22 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:14 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:55:11 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0b8db247-fece-11ec-bd2d-47488cf2e6aa ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=nVNsbL1FSLwPNF9QeRA22t26kaw/nWGND0g4sNK9/4wR6D5pNDV2vt2cnKyoYweOanoHld04OFoCr8ncY5t1AvBj/ukmkg50RibvUv2ihJ+D7uDvsGTmPXlX0rMejj4EdKXk63a0zEEOumoSutZKDofyT8q/bZCwecVQOd5O7JNYKw+z0UPmgkVCnNuElWN8bPh7uO6Zlk/DFGsICtOUG5kKmOW7tN2eEIU2tokJC4Kh9caY2pv2ynNc9rnoEUCAAGTrQAjJj6PxzEKGFk/SV+SRMK64x1Y1HO6MTezwV/23KuQ0MKSih1+PWG6r9BQOm3j91+Owwoj5AzrwVJ978w== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uJcku4B3Sy8ii3n55n/z+fUNLo1qJfxiNfc00q729Dw=; b=l3r4HAsnK/PBLTht73V8h7d6IoFIYIKJ+hHdqrTYIXZ+6AnYOZryzZVyqeztEcitiVKsLHX9BBhEd/nrO2BuAMu3wfNm97vxDZj4JcUZuE98XaYoCA6FyV9mHziIFwrXNofwgpl/hYYOjpLGIPURP7AaFpGqoUPRVPR/pWCMyoIn7uDJAnUdkdaw0rFeGk+2gmo87PJd+AZDJ4I4LTiC4jOV9gUs7hHCBHT5dhuejKainkQhzk22WNRyvkEzusUmwMwj9LxmWPkIb4MhKI+nJ+kUskPbopb7KVVKjRStpB/A+AE61Xlut0/zDYNt6/wJPaNB6we7cQGmDUjkXfzpuA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uJcku4B3Sy8ii3n55n/z+fUNLo1qJfxiNfc00q729Dw=; b=Hb6SQmaVh++q7O/4Eelh2dLWPZV+gfha2bkBYSY9g8WOK5Oo+WjIixyjGTgvr2CAvyFU8OFef3MrRQHG48MDxH5ldZkzl+tkuiANFBcdYJbWx/4TvGUQRCoynP8vJsUDoxlkYz1CvEBcUhKYXbbK10LNNjL69mVLZ7wobJlrNI0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 0d1a122662ef9862 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Bc9Pkx3O0+GJjsdOW+pEqPl8Godm0dhrZK77vovOeG5Mtt2FQY5bQJeinEitlTqhpQoVGIv4eDKmnK0T0jgjl3Gt6zF/z8J/S67euF4v1jiBHgfIwiQbHQN6Aeo0p/wH34KMDOGM8NwHt0TAIr8y/dWVxu5yicfU/voTpyhBltIJ9xRrLdu/DbsCYBVws0kxtxg18tXI3+pLhc/86a4fidcdpeadPSTA0HU1V+I+6IgPQoKtaqz5Dmzw7+B2lYCnicbDkAP1TAEsR2Jp3LTDtN21YTR+oXB3d03th5KrFEYWztMVQ2fA4m4BBGy0UBKVrOf1p72N35nWYKlegj2DBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uJcku4B3Sy8ii3n55n/z+fUNLo1qJfxiNfc00q729Dw=; b=PGsB/YYMa/6bDgVLIUQfxjE4cIW+ERafdSyopYyX0Ga2+h5datoqhntYC046wJI54Z2wVcZBGHhd3iC0gE0dcpvIhjIFmz/IF0x/2Gt3YCJjK1+k966NLeeD/c9fPqlJysrvizf0qpNbWDEv1JLvJwWhi5sapvYhaDI2VGFBgvN9zqBy9clvTm5/Pw2bQcZ8otKwdWQIdp50t52smSFlLVmQUfoqN4dbYHHHDazVEnz41ZlMk+9+i+lvHeTRiwubqh9YndXB8A+ipgHxJ2H6+MeTmNDBm3/pxk0e78AGlBOMOTbtf4jVzC4yAPkrhaAk8aNX7BY7bC2NgsgyhCGOcg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uJcku4B3Sy8ii3n55n/z+fUNLo1qJfxiNfc00q729Dw=; b=Hb6SQmaVh++q7O/4Eelh2dLWPZV+gfha2bkBYSY9g8WOK5Oo+WjIixyjGTgvr2CAvyFU8OFef3MrRQHG48MDxH5ldZkzl+tkuiANFBcdYJbWx/4TvGUQRCoynP8vJsUDoxlkYz1CvEBcUhKYXbbK10LNNjL69mVLZ7wobJlrNI0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v2 6/9] xen/x86: move NUMA scan nodes codes from x86 to common Date: Fri, 8 Jul 2022 22:54:21 +0800 Message-ID: <20220708145424.1848572-7-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: 754677fb-9a43-4b30-5624-08da60f1e8f2 X-MS-TrafficTypeDiagnostic: AM6PR08MB5096:EE_|AM5EUR03FT004:EE_|HE1PR0801MB1785:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 6MrmC9oHO4NMqJXm0J1L06Ap6h8koLIkYn2AGzhbgJsGY09awhOvCMvsGU0MQdokll97Mux+hXwKi+QkX6UxuB5i7uc+3IDRV7Qequd+o/zRCjH6PgPNJKh7VVw7m5Zg71FRJeEGlawK31+C0P9LEBptydsjbCO6rtgvf5ssdZVEwHdHfIQ5Qm5lu7ilXnf41l+tHSU3q1dt3x3Og9/cBq7iIZ061y2szMluB8Iy+EFsWMUYeAVQlR0fWde+zkriZNBbMwhsyNdjSqjXvtiP1lsvpoXmW5T6SfRPlPWGCtJdzkyyymTSb/Zrj1Y7iRkN3GdhptO71ky/nZq77NAvgRm86c2rCYn8LLTSpG5xbAb1dGzOmKfB3hrRUi5c2HWExT6axzyO2JvuV5rmd5mRGL/og6M24Qz/sD3xxHfiG8vmorPysF+7LTWliCcW7cp8C0tcdmrnKhrZ/uwa97sU8DGpsOc4wcRdqZvooBNfquSa3WRuhIAMhM0kgC22JjWNM+N+2FOn3TxFpQ7RkmWzbUO2oIMnQozapd1voWKzlhOy9YsWS+SfQP664JMJH9R9SEOqsK6UrEWE7M2JaUwa6AMs/aT8of4xpRDKlkgJOQAUUxjyRV0YtBMeiXmezwA3tRoyeDCptYG9vZG5Cl+2ifY4ZwFpZrdMC78Nv1b5OWlCDkFGjoQbzgk7o//vFySZubgqMPeOm0AR76/T/9/uS1sek/bheei63OHsrgl/VheBF5+UfudteVzijFWkIkbDuNAMXHpz4qQAVsWsOQyPAK6KFMiZ8ompwYdK/+qSTWxy4nbvElKI9ECr0cFDK/15cTME9vW5OPKmV/R12Jc/fezE1QzAsSangR7c5f+OO7M= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(376002)(39860400002)(136003)(396003)(346002)(40470700004)(36840700001)(46966006)(186003)(40480700001)(70586007)(7696005)(70206006)(426003)(47076005)(336012)(2616005)(8676002)(6916009)(36756003)(1076003)(4326008)(54906003)(26005)(8936002)(5660300002)(83380400001)(40460700003)(316002)(86362001)(82310400005)(30864003)(2906002)(6666004)(44832011)(41300700001)(478600001)(36860700001)(356005)(82740400003)(81166007)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5096 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: e28d4ec3-8ebc-4fb1-5e84-08da60f1e27a X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZcZP1jXz6MdoKLNGA2Sy5cX3PzuDiWdCOB0Z9aGSRExDIgfkDDk3lPQ2zzGc5EHPPsGYFpD7J8bAah1AJ0916hjZ+FWN0wLu4lisDx1l27AULXQWnbM5VoD8/MWhmaFX9nOtDDUnRQiqDpDsF+XWCE40TKw4p5gsDvuOvSAVDxnkSB5m6MboDbdnNdlcX9rxPp+g5GBZiEBHZOJJlI4+2xXFgKOBkxfLSQvzDbMlY6DMoBy6m4AIVbl4fGtPwcbgvQWaqtcxmWda01LkWHB/ydUkFxsEaDPeuqqF4RfFaxUYmUog+7P03fd1fwLJjTO6YLkV3NTx132nmc3yJP3zragj3V8/tas5uVAOkzv1mUnUAH+DRaq0EBs0GH37OBtK1f5QchWCWY15sy214hNuuxV3nHezSgLR5k757zAI/Kd6W84fBwHFX6ifM/oj+86uulCxnP8+nShZXd//+VMk+9IHADLE00r/bDxEFKVhkepU5h10U3x3ghdAurS716WqcRejagMBAtA8CcIXC9V3L8tbaXmYLEPERKAPlq37y1Ul++nKryeRShUo0Fok/EcPyGrr8pdLZ1x5f0kc0umUYaqrwH4hJ/PmOIfdxXRFe74im13HW0Q/Mjcj9AaPxA/oEdOPEf+la4C1c07mJUwZMPyJ3HgpmtMIJ380qCaFpETTKGZehDcCTr3Mn1EqJO4mVMWu+uby+fk+EQ9bmEKpF6AC74D0m2/74DBRq5awJhSkvJ6sFH2lPMRLVu53LjKKRyupm8mIYKTRojsZ0hfFHZQQ6qpCL8Z8UCXABsPx3DXN/K77GdEd7ryo54WXS2eJ X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(346002)(376002)(39860400002)(136003)(396003)(40470700004)(46966006)(36840700001)(8936002)(36756003)(2906002)(82740400003)(41300700001)(426003)(70206006)(5660300002)(336012)(8676002)(44832011)(4326008)(186003)(47076005)(83380400001)(6916009)(86362001)(30864003)(2616005)(7696005)(316002)(478600001)(26005)(107886003)(40460700003)(70586007)(36860700001)(81166007)(82310400005)(6666004)(1076003)(54906003)(40480700001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:33.7146 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 754677fb-9a43-4b30-5624-08da60f1e8f2 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT004.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0801MB1785 x86 has implemented a set of codes to scan NUMA nodes. These codes will parse NUMA memory and processor information from ACPI SRAT table. But except some ACPI specific codes, most of the scan codes like memory blocks validation, node memory range updates and some sanity check can be reused by other NUMA implementation. So in this patch, we move some variables and related functions for NUMA memory and processor to common code. At the same time, numa_set_processor_nodes_parsed has been introduced for ACPI specific code to update processor parsing results. With this helper, we can move most of NUMA memory affinity init code from ACPI. And bad_srat and node_to_pxm functions have been exported for common code to do architectural fallback and node to proximity converting. For those NUMA implementations haven't supported proximity domain to node map. A simple 1-1 mapping helper can help us to avoid the modification of some very valuable print messages. Signed-off-by: Wei Chen --- v1 -> v2: 1. Add code comment for numa_update_node_memblks to explain: Assumes all memory regions belonging to a single node are in one chunk. Holes between them will be included in the node. 2. Merge this single patch instead of serval patches to move x86 SRAT code to common. 3. Export node_to_pxm to keep pxm information in NUMA scan nodes error messages. 4. Change the code style to target file's Xen code-style. 5. Adjust some __init and __initdata for some functions and variables. --- xen/arch/x86/include/asm/numa.h | 3 +- xen/arch/x86/srat.c | 288 ++---------------------------- xen/common/numa.c | 307 ++++++++++++++++++++++++++++++++ xen/include/xen/numa.h | 7 + 4 files changed, 328 insertions(+), 277 deletions(-) diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/numa.h index d8960743d4..eeb431cdb7 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -10,6 +10,7 @@ typedef u8 nodeid_t; extern int srat_rev; extern nodeid_t pxm_to_node(unsigned int pxm); +extern unsigned node_to_pxm(nodeid_t n); #define ZONE_ALIGN (1UL << (MAX_ORDER+PAGE_SHIFT)) @@ -22,8 +23,6 @@ extern void init_cpu_to_node(void); #define arch_want_default_dmazone() (num_online_nodes() > 1) -extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node); - void srat_parse_regions(paddr_t addr); extern u8 __node_distance(nodeid_t a, nodeid_t b); unsigned int arch_get_dma_bitsize(void); diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 5bc9096a15..9ae81afdff 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -24,10 +24,6 @@ static struct acpi_table_slit *__read_mostly acpi_slit; -static nodemask_t memory_nodes_parsed __initdata; -static nodemask_t processor_nodes_parsed __initdata; -static struct node nodes[MAX_NUMNODES] __initdata; - struct pxm2node { unsigned pxm; nodeid_t node; @@ -35,19 +31,6 @@ struct pxm2node { static struct pxm2node __read_mostly pxm2node[MAX_NUMNODES] = { [0 ... MAX_NUMNODES - 1] = {.node = NUMA_NO_NODE} }; -static unsigned node_to_pxm(nodeid_t n); - -static int num_node_memblks; -static struct node node_memblk_range[NR_NODE_MEMBLKS]; -static nodeid_t memblk_nodeid[NR_NODE_MEMBLKS]; -static __initdata DECLARE_BITMAP(memblk_hotplug, NR_NODE_MEMBLKS); - -enum conflicts { - NO_CONFLICT, - OVERLAP, - INTERLEAVE, -}; - static inline bool node_found(unsigned idx, unsigned pxm) { return ((pxm2node[idx].pxm == pxm) && @@ -110,78 +93,7 @@ nodeid_t setup_node(unsigned pxm) return node; } -int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node) -{ - int i; - - for (i = 0; i < num_node_memblks; i++) { - struct node *nd = &node_memblk_range[i]; - - if (nd->start <= start && nd->end >= end && - memblk_nodeid[i] == node) - return 1; - } - - return 0; -} - -static -enum conflicts __init conflicting_memblks(nodeid_t nid, paddr_t start, - paddr_t end, paddr_t nd_start, - paddr_t nd_end, unsigned int *mblkid) -{ - unsigned int i; - - /* - * Scan all recorded nodes' memory blocks to check conflicts: - * Overlap or interleave. - */ - for (i = 0; i < num_node_memblks; i++) { - struct node *nd = &node_memblk_range[i]; - - *mblkid = i; - - /* Skip 0 bytes node memory block. */ - if (nd->start == nd->end) - continue; - /* - * Use memblk range to check memblk overlaps, include the - * self-overlap case. As nd's range is non-empty, the special - * case "nd->end == end && nd->start == start" also can be covered. - */ - if (nd->end > start && nd->start < end) - return OVERLAP; - - /* - * Use node memory range to check whether new range contains - * memory from other nodes - interleave check. We just need - * to check full contains situation. Because overlaps have - * been checked above. - */ - if (nid != memblk_nodeid[i] && - nd->start >= nd_start && nd->end <= nd_end) - return INTERLEAVE; - } - - return NO_CONFLICT; -} - -static __init void cutoff_node(int i, paddr_t start, paddr_t end) -{ - struct node *nd = &nodes[i]; - if (nd->start < start) { - nd->start = start; - if (nd->end < nd->start) - nd->start = nd->end; - } - if (nd->end > end) { - nd->end = end; - if (nd->start > nd->end) - nd->start = nd->end; - } -} - -static __init void bad_srat(void) +void __init bad_srat(void) { int i; printk(KERN_ERR "SRAT: SRAT not used.\n"); @@ -259,7 +171,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa) } apicid_to_node[pa->apic_id] = node; - node_set(node, processor_nodes_parsed); + numa_set_processor_nodes_parsed(node); numa_status = numa_acpi; if (opt_acpi_verbose) @@ -294,7 +206,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa) return; } apicid_to_node[pa->apic_id] = node; - node_set(node, processor_nodes_parsed); + numa_set_processor_nodes_parsed(node); numa_status = numa_acpi; if (opt_acpi_verbose) @@ -306,12 +218,9 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa) void __init acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) { - struct node *nd; - paddr_t nd_start, nd_end; - paddr_t start, end; unsigned pxm; nodeid_t node; - unsigned int i; + int ret; if (srat_disabled()) return; @@ -322,15 +231,14 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) if (!(ma->flags & ACPI_SRAT_MEM_ENABLED)) return; - start = ma->base_address; - end = start + ma->length; /* Supplement the heuristics in l1tf_calculations(). */ - l1tf_safe_maddr = max(l1tf_safe_maddr, ROUNDUP(end, PAGE_SIZE)); + l1tf_safe_maddr = max(l1tf_safe_maddr, + ROUNDUP(ma->base_address + ma->length, + PAGE_SIZE)); - if (num_node_memblks >= NR_NODE_MEMBLKS) - { + if (!numa_memblks_available()) { dprintk(XENLOG_WARNING, - "Too many numa entry, try bigger NR_NODE_MEMBLKS \n"); + "Too many numa entries, try bigger NR_NODE_MEMBLKS!\n"); bad_srat(); return; } @@ -344,129 +252,10 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) return; } - /* - * For the node that already has some memory blocks, we will - * expand the node memory range temporarily to check memory - * interleaves with other nodes. We will not use this node - * temp memory range to check overlaps, because it will mask - * the overlaps in same node. - * - * Node with 0 bytes memory doesn't need this expandsion. - */ - nd_start = start; - nd_end = end; - nd = &nodes[node]; - if (nd->start != nd->end) { - if (nd_start > nd->start) - nd_start = nd->start; - - if (nd_end < nd->end) - nd_end = nd->end; - } - - /* It is fine to add this area to the nodes data it will be used later*/ - switch (conflicting_memblks(node, start, end, nd_start, nd_end, &i)) { - case OVERLAP: - if (memblk_nodeid[i] == node) { - bool mismatch = !(ma->flags & - ACPI_SRAT_MEM_HOT_PLUGGABLE) != - !test_bit(i, memblk_hotplug); - - printk("%sSRAT: PXM %u [%"PRIpaddr", %"PRIpaddr"] overlaps with itself [%"PRIpaddr", %"PRIpaddr"]\n", - mismatch ? KERN_ERR : KERN_WARNING, pxm, start, - end - 1, node_memblk_range[i].start, - node_memblk_range[i].end - 1); - if (mismatch) { - bad_srat(); - return; - } - break; - } - - printk(KERN_ERR - "SRAT: PXM %u [%"PRIpaddr", %"PRIpaddr"] overlaps with PXM %u [%"PRIpaddr", %"PRIpaddr"]\n", - pxm, start, end - 1, node_to_pxm(memblk_nodeid[i]), - node_memblk_range[i].start, - node_memblk_range[i].end - 1); + ret = numa_update_node_memblks(node, pxm, ma->base_address, ma->length, + ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE); + if (ret) bad_srat(); - return; - - case INTERLEAVE: - printk(KERN_ERR - "SRATļ¼š PXM %u: [%"PRIpaddr", %"PRIpaddr"] interleaves with PXM %u memblk [%"PRIpaddr", %"PRIpaddr"]\n", - pxm, nd_start, nd_end - 1, node_to_pxm(memblk_nodeid[i]), - node_memblk_range[i].start, node_memblk_range[i].end - 1); - bad_srat(); - return; - - case NO_CONFLICT: - break; - } - - if (!(ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE)) { - node_set(node, memory_nodes_parsed); - nd->start = nd_start; - nd->end = nd_end; - } - - printk(KERN_INFO "SRAT: Node %u PXM %u [%"PRIpaddr", %"PRIpaddr"]%s\n", - node, pxm, start, end - 1, - ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE ? " (hotplug)" : ""); - - node_memblk_range[num_node_memblks].start = start; - node_memblk_range[num_node_memblks].end = end; - memblk_nodeid[num_node_memblks] = node; - if (ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) { - __set_bit(num_node_memblks, memblk_hotplug); - mem_hotplug_update_boundary(end); - } - num_node_memblks++; -} - -/* Sanity check to catch more bad SRATs (they are amazingly common). - Make sure the PXMs cover all memory. */ -static int __init nodes_cover_memory(void) -{ - unsigned int i; - - for (i = 0; ; i++) { - int j, found; - paddr_t start, end; - - /* Try to loop memory map from index 0 to end. */ - found = arch_get_memory_map(i, &start, &end); - - /* Index relate entry is not RAM, skip it. */ - if (found == -EINVAL) - continue; - - /* Reach the end of arch's memory map */ - if (found == -ENODEV) - break; - - do { - found = 0; - for_each_node_mask(j, memory_nodes_parsed) - if (start < nodes[j].end - && end > nodes[j].start) { - if (start >= nodes[j].start) { - start = nodes[j].end; - found = 1; - } - if (end <= nodes[j].end) { - end = nodes[j].start; - found = 1; - } - } - } while (found && start < end); - - if (start < end) { - printk(KERN_ERR "NUMA: No node for memory map range: " - "[%"PRIpaddr", %"PRIpaddr"]\n", start, end - 1); - return 0; - } - } - return 1; } void __init acpi_numa_arch_fixup(void) {} @@ -522,58 +311,7 @@ void __init srat_parse_regions(paddr_t addr) pfn_pdx_hole_setup(mask >> PAGE_SHIFT); } -/* Use the information discovered above to actually set up the nodes. */ -int __init acpi_scan_nodes(paddr_t start, paddr_t end) -{ - int i; - nodemask_t all_nodes_parsed; - - /* First clean up the node list */ - for (i = 0; i < MAX_NUMNODES; i++) - cutoff_node(i, start, end); - - /* Only when numa_on with good firmware, we can do numa scan nodes. */ - if (!numa_enabled_with_firmware()) - return -1; - - if (!nodes_cover_memory()) { - bad_srat(); - return -1; - } - - memnode_shift = compute_hash_shift(node_memblk_range, num_node_memblks, - memblk_nodeid); - - if (memnode_shift < 0) { - printk(KERN_ERR - "SRAT: No NUMA node hash function found. Contact maintainer\n"); - bad_srat(); - return -1; - } - - nodes_or(all_nodes_parsed, memory_nodes_parsed, processor_nodes_parsed); - - /* Finally register nodes */ - for_each_node_mask(i, all_nodes_parsed) - { - uint64_t size = nodes[i].end - nodes[i].start; - - if ( size == 0 ) - printk(KERN_INFO "SRAT: node %u has no memory\n", i); - - setup_node_bootmem(i, nodes[i].start, nodes[i].end); - } - for (i = 0; i < nr_cpu_ids; i++) { - if (cpu_to_node[i] == NUMA_NO_NODE) - continue; - if (!nodemask_test(cpu_to_node[i], &processor_nodes_parsed)) - numa_set_node(i, NUMA_NO_NODE); - } - numa_init_array(); - return 0; -} - -static unsigned node_to_pxm(nodeid_t n) +unsigned node_to_pxm(nodeid_t n) { unsigned i; diff --git a/xen/common/numa.c b/xen/common/numa.c index bc6a2ded14..e3b66c54b5 100644 --- a/xen/common/numa.c +++ b/xen/common/numa.c @@ -14,6 +14,21 @@ #include #include +static nodemask_t __initdata processor_nodes_parsed; +static nodemask_t __initdata memory_nodes_parsed; +static struct node __initdata nodes[MAX_NUMNODES]; + +static int num_node_memblks; +static struct node node_memblk_range[NR_NODE_MEMBLKS]; +static nodeid_t memblk_nodeid[NR_NODE_MEMBLKS]; +static __initdata DECLARE_BITMAP(memblk_hotplug, NR_NODE_MEMBLKS); + +enum conflicts { + NO_CONFLICT, + OVERLAP, + INTERLEAVE, +}; + struct node_data node_data[MAX_NUMNODES]; /* Mapping from pdx to node id */ @@ -32,6 +47,298 @@ nodemask_t __read_mostly node_online_map = { { [0] = 1UL } }; enum numa_mode numa_status; +void __init numa_set_processor_nodes_parsed(nodeid_t node) +{ + node_set(node, processor_nodes_parsed); +} + +int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node) +{ + int i; + + for ( i = 0; i < num_node_memblks; i++ ) + { + struct node *nd = &node_memblk_range[i]; + + if ( nd->start <= start && nd->end >= end && + memblk_nodeid[i] == node ) + return 1; + } + + return 0; +} + +static +enum conflicts __init conflicting_memblks(nodeid_t nid, paddr_t start, + paddr_t end, paddr_t nd_start, + paddr_t nd_end, unsigned int *mblkid) +{ + unsigned int i; + + /* + * Scan all recorded nodes' memory blocks to check conflicts: + * Overlap or interleave. + */ + for ( i = 0; i < num_node_memblks; i++ ) + { + struct node *nd = &node_memblk_range[i]; + + *mblkid = i; + + /* Skip 0 bytes node memory block. */ + if ( nd->start == nd->end ) + continue; + /* + * Use memblk range to check memblk overlaps, include the + * self-overlap case. As nd's range is non-empty, the special + * case "nd->end == end && nd->start == start" also can be covered. + */ + if ( nd->end > start && nd->start < end ) + return OVERLAP; + + /* + * Use node memory range to check whether new range contains + * memory from other nodes - interleave check. We just need + * to check full contains situation. Because overlaps have + * been checked above. + */ + if ( nid != memblk_nodeid[i] && + nd->start >= nd_start && nd->end <= nd_end ) + return INTERLEAVE; + } + + return NO_CONFLICT; +} + +static void __init cutoff_node(int i, paddr_t start, paddr_t end) +{ + struct node *nd = &nodes[i]; + + if ( nd->start < start ) + { + nd->start = start; + if ( nd->end < nd->start ) + nd->start = nd->end; + } + + if ( nd->end > end ) + { + nd->end = end; + if ( nd->start > nd->end ) + nd->start = nd->end; + } +} + +bool __init numa_memblks_available(void) +{ + return num_node_memblks < NR_NODE_MEMBLKS; +} + +/* + * This function will be called by NUMA memory affinity initialization to + * update NUMA node's memory range. In this function, we assume all memory + * regions belonging to a single node are in one chunk. Holes (or MMIO + * ranges) between them will be included in the node. + * + * So in numa_update_node_memblks, if there are multiple banks for each + * node, start and end are stretched to cover the holes between them, and + * it works as long as memory banks of different NUMA nodes don't interleave. + */ +int __init numa_update_node_memblks(nodeid_t node, unsigned pxm, + paddr_t start, paddr_t size, + bool hotplug) +{ + unsigned int i; + paddr_t end = start + size; + paddr_t nd_start = start; + paddr_t nd_end = end; + struct node *nd = &nodes[node]; + + /* + * For the node that already has some memory blocks, we will + * expand the node memory range temporarily to check memory + * interleaves with other nodes. We will not use this node + * temp memory range to check overlaps, because it will mask + * the overlaps in same node. + * + * Node with 0 bytes memory doesn't need this expandsion. + */ + if ( nd->start != nd->end ) + { + if ( nd_start > nd->start ) + nd_start = nd->start; + + if ( nd_end < nd->end ) + nd_end = nd->end; + } + + /* It is fine to add this area to the nodes data it will be used later*/ + switch ( conflicting_memblks(node, start, end, nd_start, nd_end, &i) ) + { + case OVERLAP: + if ( memblk_nodeid[i] == node ) + { + bool mismatch = !(hotplug) != !test_bit(i, memblk_hotplug); + + printk("%sNUMA: PXM %u [%"PRIpaddr", %"PRIpaddr"] overlaps with itself [%"PRIpaddr", %"PRIpaddr"]\n", + mismatch ? KERN_ERR : KERN_WARNING, pxm, start, end - 1, + node_memblk_range[i].start, node_memblk_range[i].end - 1); + if ( mismatch ) + return -EINVAL; + break; + } + + printk(KERN_ERR + "NUMA: PXM %u [%"PRIpaddr", %"PRIpaddr"] overlaps with PXM %u [%"PRIpaddr", %"PRIpaddr"]\n", + pxm, start, end - 1, node_to_pxm(memblk_nodeid[i]), + node_memblk_range[i].start, node_memblk_range[i].end - 1); + return -EINVAL; + + + case INTERLEAVE: + printk(KERN_ERR + "NUMAļ¼š PXM %u: [%"PRIpaddr", %"PRIpaddr"] interleaves with PXM %u memblk [%"PRIpaddr", %"PRIpaddr"]\n", + pxm, nd_start, nd_end - 1, node_to_pxm(memblk_nodeid[i]), + node_memblk_range[i].start, node_memblk_range[i].end - 1); + return -EINVAL; + + case NO_CONFLICT: + break; + } + + if ( !hotplug ) + { + node_set(node, memory_nodes_parsed); + nd->start = nd_start; + nd->end = nd_end; + } + + printk(KERN_INFO "NUMA: Node %u PXM %u [%"PRIpaddr", %"PRIpaddr"]%s\n", + node, pxm, start, end - 1, + hotplug ? " (hotplug)" : ""); + + node_memblk_range[num_node_memblks].start = start; + node_memblk_range[num_node_memblks].end = end; + memblk_nodeid[num_node_memblks] = node; + if ( hotplug ) + { + __set_bit(num_node_memblks, memblk_hotplug); + mem_hotplug_update_boundary(end); + } + num_node_memblks++; + + return 0; +} + +/* + * Sanity check to catch more bad SRATs (they are amazingly common). + * Make sure the PXMs cover all memory. + */ +static int __init nodes_cover_memory(void) +{ + unsigned int i; + + for ( i = 0; ; i++ ) + { + int j, found; + paddr_t start, end; + + /* Try to loop memory map from index 0 to end. */ + found = arch_get_memory_map(i, &start, &end); + + /* Index relate entry is not RAM, skip it. */ + if ( found == -EINVAL ) + continue; + + /* Reach the end of arch's memory map */ + if ( found == -ENODEV ) + break; + + do { + found = 0; + for_each_node_mask( j, memory_nodes_parsed ) + if ( start < nodes[j].end + && end > nodes[j].start ) + { + if ( start >= nodes[j].start ) + { + start = nodes[j].end; + found = 1; + } + + if ( end <= nodes[j].end ) + { + end = nodes[j].start; + found = 1; + } + } + } while ( found && start < end ); + + if ( start < end ) + { + printk(KERN_ERR "NUMA: No node for memory map range: " + "[%"PRIpaddr", %"PRIpaddr"]\n", start, end - 1); + return 0; + } + } + return 1; +} + +/* Use the information discovered above to actually set up the nodes. */ +int __init acpi_scan_nodes(paddr_t start, paddr_t end) +{ + int i; + nodemask_t all_nodes_parsed; + + /* First clean up the node list */ + for ( i = 0; i < MAX_NUMNODES; i++ ) + cutoff_node(i, start, end); + + /* Only when numa_on with good firmware, we can do numa scan nodes. */ + if ( !numa_enabled_with_firmware() ) + return -1; + + if ( !nodes_cover_memory() ) + { + bad_srat(); + return -1; + } + + memnode_shift = compute_hash_shift(node_memblk_range, num_node_memblks, + memblk_nodeid); + + if ( memnode_shift < 0 ) + { + printk(KERN_ERR + "SRAT: No NUMA node hash function found. Contact maintainer\n"); + bad_srat(); + return -1; + } + + nodes_or(all_nodes_parsed, memory_nodes_parsed, processor_nodes_parsed); + + /* Finally register nodes */ + for_each_node_mask( i, all_nodes_parsed ) + { + paddr_t size = nodes[i].end - nodes[i].start; + + if ( size == 0 ) + printk(KERN_INFO "SRAT: node %u has no memory\n", i); + + setup_node_bootmem(i, nodes[i].start, nodes[i].end); + } + + for ( i = 0; i < nr_cpu_ids; i++ ) + { + if ( cpu_to_node[i] == NUMA_NO_NODE ) + continue; + if ( !nodemask_test(cpu_to_node[i], &processor_nodes_parsed) ) + numa_set_node(i, NUMA_NO_NODE); + } + numa_init_array(); + return 0; +} + /* * Given a shift value, try to populate memnodemap[] * Returns : diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 6d02204991..564add430c 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -52,6 +52,7 @@ extern void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn); extern bool numa_enabled_with_firmware(void); extern enum numa_mode numa_status; +extern void bad_srat(void); extern void numa_set_node(int cpu, nodeid_t node); extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end); @@ -90,6 +91,12 @@ static inline __attribute__((pure)) nodeid_t phys_to_nid(paddr_t addr) extern int arch_get_memory_map(unsigned int idx, paddr_t *start, paddr_t *end); +extern int valid_numa_range(paddr_t start, paddr_t end, nodeid_t node); +extern bool numa_memblks_available(void); +extern int numa_update_node_memblks(nodeid_t node, unsigned pxm, + paddr_t start, paddr_t size, + bool hotplug); +extern void numa_set_processor_nodes_parsed(nodeid_t node); #endif From patchwork Fri Jul 8 14:54:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911264 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 877AFC43334 for ; Fri, 8 Jul 2022 14:55:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363477.594074 (Exim 4.92) (envelope-from ) id 1o9pOQ-0000hA-IB; Fri, 08 Jul 2022 14:55:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363477.594074; Fri, 08 Jul 2022 14:55:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOQ-0000fi-Ai; Fri, 08 Jul 2022 14:55:42 +0000 Received: by outflank-mailman (input) for mailman id 363477; Fri, 08 Jul 2022 14:55:41 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOO-0006vD-S0 for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:41 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60049.outbound.protection.outlook.com [40.107.6.49]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 07e0f916-fece-11ec-924f-1f966e50362f; Fri, 08 Jul 2022 16:55:39 +0200 (CEST) Received: from DB6PR0601CA0012.eurprd06.prod.outlook.com (2603:10a6:4:7b::22) by AM0PR08MB3188.eurprd08.prod.outlook.com (2603:10a6:208:5e::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:35 +0000 Received: from DBAEUR03FT045.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:7b:cafe::18) by DB6PR0601CA0012.outlook.office365.com (2603:10a6:4:7b::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:35 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT045.mail.protection.outlook.com (100.127.142.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:34 +0000 Received: ("Tessian outbound 13cb25bfb745:v122"); Fri, 08 Jul 2022 14:55:34 +0000 Received: from 216719b84af4.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 8E07A32A-26C8-4653-A7AC-948110A42AB5.1; Fri, 08 Jul 2022 14:55:27 +0000 Received: from EUR01-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 216719b84af4.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:27 +0000 Received: from DB6P192CA0004.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::14) by AM0PR08MB5154.eurprd08.prod.outlook.com (2603:10a6:208:15d::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.20; Fri, 8 Jul 2022 14:55:25 +0000 Received: from DBAEUR03FT024.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:b8:cafe::3b) by DB6P192CA0004.outlook.office365.com (2603:10a6:4:b8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16 via Frontend Transport; Fri, 8 Jul 2022 14:55:24 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT024.mail.protection.outlook.com (100.127.142.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:24 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:17 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:55:15 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 07e0f916-fece-11ec-924f-1f966e50362f ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=DBlHUOywwnEnAxlQoRd65sHV4h+H3y3dTIk6cuDic5RgQCTEywUpA1pS0s2TBOld/ayDGgsQ5td6CEdUCYHWmO6caSiYhupcgoQ0ty9X7NvyKMb7sOsWk1OnZzgRmReW08B3cgQ7Vw1m0ZRouzGKMTj0Z4rOXgk28GYDF89EyXf0G4Zkp7x/9XgBE3hVThyg4vJbM7edodHbqIjUufOkrNAtcXge598mED60jCkBBJGLVLE/+dBwGmKPG/hhWP+adyxAxwfJmb628UVeHBb24FiI9USsL2hmIHLfQe0icmvC3gMJUwzcAD0hvwE/NGATao1mLWC/FjanuY0dlLToCw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Hk0IhxXN/zYIHE4+iVUsSa2FBDhvvWOeQyK22TgQPPM=; b=gjuRq4aBbaHp/4qK7TSyKWmNQwea3+v4LZTS0/lukFFieNlIR3+CZJUgCzMIwGjEtNut5ZlkGp2Dn1OBQoO1Ecwmw3Dki0g33wmfo7qTx3qO+GCodRhh1z7h/k8/8cdU7ruo5Ym9EONx2hhuzeQiC3ZM8EtjKS0njSA00wKVL/0rkdNcyg1dhCtScvtMOM/UTC853+2bqqWoUoFjaZgsFasOvD+Ab/sc3hu37WJNqngKOiAZj/wgDTQTybG6hHPc96c5IBiY3xnH/AqRpxoxs2d+PjlaJCBFdddc8Jjf2q3/GVD/P6LPGTsCiqmEL8KEZ9nqg7K61KuZEqapzkxWvA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Hk0IhxXN/zYIHE4+iVUsSa2FBDhvvWOeQyK22TgQPPM=; b=c86/JCq5BUh3HV5c59njQldpPEfVt2DOSe2hivj/Pb8zV4G91I3gXLrFglKW2iZKkZcWQRoplm6ShmpfCw1pyLKPghGpMm4pnxsQoC/iqxUl6yovuIVcRGq3HVSWAvYX7YIjnGwmprHuL+4VhAnM/17uOPSfftoM8HzvsjZE0Ik= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 1717f0940105f789 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bf8S3uppi2CODx1C2ONxHHSSp6FB5FBOJ15YJeDP5pyOA4DhFFFJtOyHDOwZm0GMdcQVMtdPQBR7LtwJUNY11sQ5e+amv+8I0ADqcasEf1Wz0/fCDngFDCdg+qT82ejCFEC01lg/ZiGN17aXqoPEz/rOBCPhxCFDwvLLRSyYWJVWLIGqgs9k6qIq/cMZZEFlwTzMWFsJELVW3uEuPUM+4POMiQMPn9EnfyDRW4VhjMogTsAhLMJRGjf+wiDg6rn1m1Zzd6O9Z9B+szHZJsfEwNYZXC5W8sxHLPaYuP4e7+qt6o2NhU9BELykQhYmoXBFkMjhDICd/QLlRyhAUeC/0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Hk0IhxXN/zYIHE4+iVUsSa2FBDhvvWOeQyK22TgQPPM=; b=KCsp2QiU8FpTvkNq03chMe9LEmBzRPh4MgrFpjItHenpzgdK5hRP+LivJlagZ3M6iSaK3eGXmCpVWhrDSuSAylYRbLGoeGJcA17viOzqRDHDIPDWuFCawkYR6fIXhctDBJf/EYjFdeoDcSxjgH3pYmaD1eNS+CK76UAZZlpo30NEr7N4XIW2VHSkSRAKFByYkcjEeAUvKfqKt1saNPqYKmLSumYFcCOmhLbBmzsvC7bYt1jA3z/DB8Cxmq/PteWrBTjbCsxzWU9nLwHZP8dkc7vTusXoLnfbixycso1Figjl4F3TwZCNyvj/wbJ6zjsKkON0la9DI48RF6BURVr2Gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Hk0IhxXN/zYIHE4+iVUsSa2FBDhvvWOeQyK22TgQPPM=; b=c86/JCq5BUh3HV5c59njQldpPEfVt2DOSe2hivj/Pb8zV4G91I3gXLrFglKW2iZKkZcWQRoplm6ShmpfCw1pyLKPghGpMm4pnxsQoC/iqxUl6yovuIVcRGq3HVSWAvYX7YIjnGwmprHuL+4VhAnM/17uOPSfftoM8HzvsjZE0Ik= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v2 7/9] xen/x86: rename bad_srat to numa_bad Date: Fri, 8 Jul 2022 22:54:22 +0800 Message-ID: <20220708145424.1848572-8-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: b50017ba-3fef-49bb-4551-08da60f1e990 X-MS-TrafficTypeDiagnostic: AM0PR08MB5154:EE_|DBAEUR03FT045:EE_|AM0PR08MB3188:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: LZiQA2NVb4dV6MKzZb22EPUEJSYe/N7aHI9PcAeFBAak9k4pzb1IRRJBqVKXMByjG8XZNHhHqAtYN26aXR7MSaGQiMdNNbu1OpVkMzuaoSYbRklxSJt0uRXC0IraB5muTRlzdr8f/AK//Rv6dVHkhWL5FnxOgVbEdjV4E0hAQJCeanSGRxzqw+CCCKYwr9nADkb2mh6f/d46NK9mKyjRUCEHxvTPec2qkhimqyG3GebSKZVmq+Iv6n4V86/W9UkT74VPH8J1XdK6tXAYljh8xeX6P31rdoUcLhTxc9A8GNVIYmRASA966QkwAUaegJdbf3p+xokLwvH3IN8GBubjm6NmhoF3lKsSM05ZHB24K+JAMXtN6VjWYuB5Ox0lorTHW5UBfBbP8u99TCXcDhhh1852d6hEm5w9++fDhBfnafB8EU9gZiZ21a6OXvwHDLrnKcfdy8bSiGdkjnpXJ+z0VcBI+XoH3iIebyGcOteu/xwgOvho+bSKd8zqC4Q0nhn8qszMNO0Wp7lmhGUsq2lpXBy690TZDPeL0669cTr+eMl0W1VQsJQJR/WM+J5WQGsEHiO9VYdecEulI+/yviEM4KMloEHmWcaBFw9diLUiFUQDYHhJgculG/UmcooFT90j52WmDMHLPr6Cb0R6lbH4gs3qoAV+rwkffxvKQOIvjzJglh6OMhNQlRB2Vf7Yh4Wl9JZM4j0y+zgJEJzzlE4FjrrL67tB2ejM0taC4NydBL1FxKNMoh2+sDFDRNmohotwHidor3hv9+/ID0S/qZBuip1+x2RwiUrFqtUeW9WJnsJ0/TNeDXTdnvtAqKFBS4T0bdr4/LWXjSi28iIGAJ07h+fWBQtiJKk45WqhjFoesFY= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(376002)(396003)(136003)(346002)(40470700004)(36840700001)(46966006)(82740400003)(426003)(7696005)(47076005)(86362001)(44832011)(40480700001)(336012)(82310400005)(6666004)(41300700001)(356005)(4326008)(2906002)(1076003)(2616005)(186003)(26005)(70586007)(70206006)(83380400001)(8676002)(6916009)(36756003)(40460700003)(8936002)(36860700001)(81166007)(478600001)(54906003)(316002)(5660300002)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5154 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 8b691d5c-4ac1-498f-234c-08da60f1e3a9 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EsZtFZanV3m/BYHgKsWNMoXVQj6y5Gm5MuJX66rcY6jZLz0pOY5jEM+iAEMvNHiGtU72zaezKXjAIfAvYwBS1nBMvurh5YyTZeHG/n4pn/SBMTLQNQ7WQdLZqgSFzpmsM6PIa2LPdiTGNlUsU9cB7nygiXuIkXLghWmsA74f1mFrqK7W3L/Ahe5HJS4bakFZqGLrR6ek3ZUUfr9Xeh+NTEQi7mv3aLbHH/DFhQ+i77zPm87+iW1B145GPOK0zfX90eq2XU803rCfGso2AI1Sdr5KCjw7wEBa1sYUA2ZAuvqmtz/KV4xiGt+9oSiQvpGsA2Bu3jDr8hlAtaqJbn3gyhXUEYnuwcM10iD2KV7IlRll7zC2TQXV9F0DlQNeUarRRD/TjsrtIND6jJ9hlyUDl6zmB1csGqEzQ7xifnJbCHqV5Kj65lv1E5ckErzdyD+3RWp4MeJesz3o1rX83kn9SkQF9kpqd4MF/1O+Yu20nwSM7wqTolf4FpRVUBMXN9uzhmVbO5szKgfqzKsJpMnGfXIqHEYZsC+BRH4gjckkCo5tTkxFXAdeEqVJ7mhXbwXdodypz3G9WztNiAL9osYhqLqJHK3S02o8ecelsRgqXrsJ2hCebyxmg8RCwfIJoSmr7tiMxYd7g7mHoxtU26vhmtC0PbJze6ZGlCWSmKaIc72oZMCZECOZsF6inab8XwXsDAM0lG9ZuV6r8MeX9Dmi5Fvb9lD47mBKtZs75ffEqVN4zQQ0E1rmmJssg6Q5+YkzkS9Jvh71tfTVgv+mWTpJAIJck0DbVIctbVU3HYV0H8zIDVJ6RRrtVf7bVhU3MoSs X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(396003)(39860400002)(136003)(376002)(346002)(46966006)(36840700001)(40470700004)(47076005)(82310400005)(83380400001)(40460700003)(41300700001)(81166007)(54906003)(36860700001)(6666004)(36756003)(426003)(44832011)(40480700001)(2906002)(6916009)(336012)(70206006)(1076003)(107886003)(5660300002)(8936002)(70586007)(316002)(4326008)(26005)(8676002)(186003)(82740400003)(7696005)(2616005)(86362001)(478600001);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:34.8277 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b50017ba-3fef-49bb-4551-08da60f1e990 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT045.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB3188 When NUMA initialization code is failed in scanning SRAT. It will call bad_srat to set disable NUMA and clear relate data. But this name is ACPI specific, we have moved generically usable NUMA codes to common, bad_srat has came with these codes to common code. But it's not reasonable for other NUMA implementations to implement a fall back function named bad_srat. So in this patch, we rename bad_srat to numa_bad. Signed-off-by: Wei Chen --- v1 -> v2: 1. New in v2. --- xen/arch/x86/srat.c | 18 +++++++++--------- xen/common/numa.c | 4 ++-- xen/include/xen/numa.h | 2 +- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/xen/arch/x86/srat.c b/xen/arch/x86/srat.c index 9ae81afdff..4afb37bf9f 100644 --- a/xen/arch/x86/srat.c +++ b/xen/arch/x86/srat.c @@ -93,7 +93,7 @@ nodeid_t setup_node(unsigned pxm) return node; } -void __init bad_srat(void) +void __init numa_bad(void) { int i; printk(KERN_ERR "SRAT: SRAT not used.\n"); @@ -153,7 +153,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa) if (srat_disabled()) return; if (pa->header.length < sizeof(struct acpi_srat_x2apic_cpu_affinity)) { - bad_srat(); + numa_bad(); return; } if (!(pa->flags & ACPI_SRAT_CPU_ENABLED)) @@ -166,7 +166,7 @@ acpi_numa_x2apic_affinity_init(const struct acpi_srat_x2apic_cpu_affinity *pa) pxm = pa->proximity_domain; node = setup_node(pxm); if (node == NUMA_NO_NODE) { - bad_srat(); + numa_bad(); return; } @@ -189,7 +189,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa) if (srat_disabled()) return; if (pa->header.length != sizeof(struct acpi_srat_cpu_affinity)) { - bad_srat(); + numa_bad(); return; } if (!(pa->flags & ACPI_SRAT_CPU_ENABLED)) @@ -202,7 +202,7 @@ acpi_numa_processor_affinity_init(const struct acpi_srat_cpu_affinity *pa) } node = setup_node(pxm); if (node == NUMA_NO_NODE) { - bad_srat(); + numa_bad(); return; } apicid_to_node[pa->apic_id] = node; @@ -225,7 +225,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) if (srat_disabled()) return; if (ma->header.length != sizeof(struct acpi_srat_mem_affinity)) { - bad_srat(); + numa_bad(); return; } if (!(ma->flags & ACPI_SRAT_MEM_ENABLED)) @@ -239,7 +239,7 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) if (!numa_memblks_available()) { dprintk(XENLOG_WARNING, "Too many numa entries, try bigger NR_NODE_MEMBLKS!\n"); - bad_srat(); + numa_bad(); return; } @@ -248,14 +248,14 @@ acpi_numa_memory_affinity_init(const struct acpi_srat_mem_affinity *ma) pxm &= 0xff; node = setup_node(pxm); if (node == NUMA_NO_NODE) { - bad_srat(); + numa_bad(); return; } ret = numa_update_node_memblks(node, pxm, ma->base_address, ma->length, ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE); if (ret) - bad_srat(); + numa_bad(); } void __init acpi_numa_arch_fixup(void) {} diff --git a/xen/common/numa.c b/xen/common/numa.c index e3b66c54b5..5ab061e991 100644 --- a/xen/common/numa.c +++ b/xen/common/numa.c @@ -300,7 +300,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end) if ( !nodes_cover_memory() ) { - bad_srat(); + numa_bad(); return -1; } @@ -311,7 +311,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end) { printk(KERN_ERR "SRAT: No NUMA node hash function found. Contact maintainer\n"); - bad_srat(); + numa_bad(); return -1; } diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 564add430c..4c4632ec27 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -52,7 +52,7 @@ extern void numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn); extern bool numa_enabled_with_firmware(void); extern enum numa_mode numa_status; -extern void bad_srat(void); +extern void numa_bad(void); extern void numa_set_node(int cpu, nodeid_t node); extern void setup_node_bootmem(nodeid_t nodeid, paddr_t start, paddr_t end); From patchwork Fri Jul 8 14:54:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E313CC433EF for ; Fri, 8 Jul 2022 14:55:50 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363476.594063 (Exim 4.92) (envelope-from ) id 1o9pOP-0000Kq-02; Fri, 08 Jul 2022 14:55:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363476.594063; Fri, 08 Jul 2022 14:55:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOO-0000Kb-T6; Fri, 08 Jul 2022 14:55:40 +0000 Received: by outflank-mailman (input) for mailman id 363476; Fri, 08 Jul 2022 14:55:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pON-0006vD-Rd for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:40 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2041.outbound.protection.outlook.com [40.107.104.41]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 07899fed-fece-11ec-924f-1f966e50362f; Fri, 08 Jul 2022 16:55:38 +0200 (CEST) Received: from DB6P192CA0016.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::26) by PA4PR08MB6032.eurprd08.prod.outlook.com (2603:10a6:102:e4::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15; Fri, 8 Jul 2022 14:55:36 +0000 Received: from DBAEUR03FT046.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:b8:cafe::2a) by DB6P192CA0016.outlook.office365.com (2603:10a6:4:b8::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:36 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT046.mail.protection.outlook.com (100.127.142.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:35 +0000 Received: ("Tessian outbound e5f49c5d38f9:v122"); Fri, 08 Jul 2022 14:55:35 +0000 Received: from e2cefff5c73e.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 6E361E81-15BB-4980-BDFE-118B4EC5B781.1; Fri, 08 Jul 2022 14:55:29 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e2cefff5c73e.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:29 +0000 Received: from DB6P192CA0004.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::14) by DB7PR08MB3178.eurprd08.prod.outlook.com (2603:10a6:5:24::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:26 +0000 Received: from DBAEUR03FT024.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:b8:cafe::ca) by DB6P192CA0004.outlook.office365.com (2603:10a6:4:b8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16 via Frontend Transport; Fri, 8 Jul 2022 14:55:26 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT024.mail.protection.outlook.com (100.127.142.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:26 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:21 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:55:18 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 07899fed-fece-11ec-924f-1f966e50362f ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=dUcjBxDOp0RiKR9LzrELyrmBSuLHdgOD6F0v3ODTD4xJ5TyiMM4oZDlBX0ksaBpwNzlNvF/9X5+MvrbbGxeWlu6g1wd0KPe7LQ/Q6Zl43tDLOD3L7cxbheRaFfHQfwpVGlMTbOSEHg49lDf1lsaQvErnFcogSKustXDZlBtD1RKeTHjz+ovCfFvTDwWZTPCEh7gn4jnsjVD3c0RiGxXr+olQntZdm1u1MNTCEi/9jhLo7Y5Dpx+6gsOeWxLSmAWbT1FyaU0zik2wA3DTR+KT68Y0pyBWHEO0ckHsje7HurXR0xrQfZmteXtCgtkFyvxGaw3E4WdCSPWE+X3mlM13zw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UQ5GWTIY4MWQGGyhLZ8eQoBIiDbRb+nQhneNRjZRrdA=; b=OMFoz5Tzop2xAvk8ORbRqKpTnwHvZdwrBLkvO2lbAGP7ZfMWoPRxPmjn0/8oPMgyYeNOLMBJfP3C6Bw2QWreuoZj/KZfL2YoBw4fq/qLVi+AuqVlOyN3JwnTP57dpNC1ZSGbnuq/8s2h8pxhEL6TI1KqPhcQcGCC2H5HmnG5L8HK7oTkWjGEP9InwPQc6UIwDImAMhZEeJEWa41m+xGyEv4crfC7QuFAWXmDmzfyQLycdINak+WJhnLD9p/+qiwaVx6okePa4Vpg5POLNKIU/Z6RxVDex4Ii5n9bTnNkfUUFYWzJbdUAVDTt4XlgOrc1f2o2UL8SOrxuNvnBbvIpaA== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UQ5GWTIY4MWQGGyhLZ8eQoBIiDbRb+nQhneNRjZRrdA=; b=iVdHgS6fU7FqzfL9hNkg4ZhXOhwgj2YvcTQi3xeoG90bN3wZf9zIi+gyC6wXCGZvRh2Cz//wDh2/i1wTfNv00L2nEXD9dNl5kcdjLsdjl5r1iK7Pqx7dR9okajC8C0Ymo33aY8wkGGKTWirCbDBrXFE5J12VBgxvcJX9G46KIrw= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 15e1c48a565aa983 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Sk+UwK2Hauzhbnx3shfUGBpacF2Qpvqatnk+geWuGsw0EKKsXwmBKDlceu67ezX4cV/NvzffR8r8TS0Kf+G67m5KYHqELXTr+duV+HpiU8Ud0dN+UzimnRiW51Kw2bwdFVonyhzYW75xX9GwS4INSrx/2MZ5UnyT8Wm3PPGFTMU+vhYuO61WiphtdVI/JyV5kXbk9Oi2EwdL30+Dd0dRlaQIv4IP92a/nGfW1FH0FMx1/qBYFh5loso2tmJP4IcbQEUZYlhBy6FlsRY8CJMpvAj64lIa/utSJ2keyP8d3OR9bY0VzuWo2lpK0HNRgWh6chTK1fkPqUVAx2f8VAD9Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UQ5GWTIY4MWQGGyhLZ8eQoBIiDbRb+nQhneNRjZRrdA=; b=PEQXCAxziGNx1u6AHJYJ+6hd8k734QBCCKZ4XadKZyVdsjClSHM3ia7GZcxutFlSMzCwhWsPHoHWhhs6cW6xnZTLwSMjuI+YzxhTrGYfZOfUyLLQQmaeJro6rs9czr6MU9yEjM8G2ROa+v0xjgUtWucs4s9iXsluXkaeWfbXKopEbl9lZkaKcPqvR6HaBfZ/zYr5ecPTabVs29gpKaeSaLxPc7NpCeY3UbGVnnbRd5Zn5mTqreQwTn7hTznYKH47nlugKKX2xYGny/7s/4HzyQrQZyLtQRYLLhCoClbgbehyNmc6hHTQKJ4PFmGo0QzXsEydMNejeMssYoOdNHZgxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UQ5GWTIY4MWQGGyhLZ8eQoBIiDbRb+nQhneNRjZRrdA=; b=iVdHgS6fU7FqzfL9hNkg4ZhXOhwgj2YvcTQi3xeoG90bN3wZf9zIi+gyC6wXCGZvRh2Cz//wDh2/i1wTfNv00L2nEXD9dNl5kcdjLsdjl5r1iK7Pqx7dR9okajC8C0Ymo33aY8wkGGKTWirCbDBrXFE5J12VBgxvcJX9G46KIrw= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v2 8/9] xen: rename acpi_scan_nodes to numa_scan_nodes Date: Fri, 8 Jul 2022 22:54:23 +0800 Message-ID: <20220708145424.1848572-9-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: 69039600-7f7e-4b18-43bb-08da60f1ea35 X-MS-TrafficTypeDiagnostic: DB7PR08MB3178:EE_|DBAEUR03FT046:EE_|PA4PR08MB6032:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: lA38u7ClFGoQyENw8XkQSPsE6AGjpZ1R/97AJp7/SCWYlvcohPAtsgUV9BIuiSzvEN3br38xCmgibvSmdm8QpSJdRJ/wiJ1N4yRGjIa5PFDL2ashpaM1yNIwWTfGJx5Tb3LXjXPvBjrv/tSVYX1xHU6rNUMA2dq8OQZoGD/l8Kk5Jx/a1yDeT/PwCYUSPQMYjgzDGNJcjYt/Ez5SoT9jSYu5Sa4BKXIPEmuEORE952UjdWN/QzhoG27w47DBWIw00KBkdTcrDnEiLqE+zPdLuj6QLtf2yrphGgvw49/+NfxdbqsCLOze/jT1CzxQPKoR2fbTGJbb3cZKeB+fjFCR/NOpUWhTmL9LNXYAxpH+Hk28yZlMqo2pmkBs9ayDdKXTAhhtkGa0yoX2AVqV2HkTesX7IfBRKDsMC66UR5h0hw3pAWywfawDvkLxdsEw1J9ke6/4vaUL38YSy86OHTkIhX9RXZYcOFZRlTG3/BpJI5lCf9pvYSvavOQvm+AZStWzLenS+AtC0WXzvjFyd2dLyuvHbcUXrsDdZbC9KmcYygQ2pWMxK4OuMmlIttvVO/U2DR/ZwwNnfZaEJBD2o6YmzEI1jS8jiZFhTzVidvT8it73EdlfOdQrVmMMpXX31kNZsuXiSKPrHMrA5l2uDCdeG68ZLNOaLUX9dyTGWxCUmiWg/ikKugjHoZFxI7/7iGXxdk1BNwYyyFrYs13ZxT9nEyVjWkZaqQAxqLO1AQ7mXq3/9VvNIk0gw35Ectk5Vc0vrh6PSfbYXmCoMDWPssnTNZOMYgTNH8TeaxiH9s0nU1HIzzjJ5ok22ikhyOY7Jl+xSQErBODocZ3dre+MTiizQcKzoLvtYQboqwgcrX3AfE4= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(396003)(376002)(346002)(39860400002)(136003)(40470700004)(36840700001)(46966006)(1076003)(8936002)(2616005)(478600001)(40480700001)(86362001)(83380400001)(6666004)(26005)(336012)(41300700001)(2906002)(47076005)(40460700003)(426003)(186003)(36756003)(82310400005)(44832011)(5660300002)(356005)(70586007)(4326008)(316002)(82740400003)(70206006)(8676002)(81166007)(54906003)(6916009)(7696005)(36860700001)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3178 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT046.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: af45ea87-b81d-4267-d5a6-08da60f1e4c4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Vt7KHSsxr9ifytSGnW7xWXyQxwgCTecBeyZt2MgShdrcVDHVlrHIhg6dL2enh88VbX59SAjP0cAtuXgLFDNglhclfRVOJNVLP3PyTSQJKFuJ+p8VnvaOGWKyw9UNInyRZxfJr85F1dVwv09d3QhXrz7/yGcy1sD8aHzTxMDlFR+DWR+2YSBGlbRgw9ujdgJu/1KJ4I3LcDfwHtR2whh38KWjZluS9tgBpWjWG/2UYGhY8i0CQfSoEsj0ZFqGYXT2QSuY17hfNWTayZDThgRBs7MfmDSoJe/5LqqzMtt5yJ+SFj3Z4m4CsXm0NIoo8BUl5/JyR4NhwS4TP+36A5T9w6TaBU85zFg2UEdW08CgvuV+l6mpCmVK80giINlFG9FUR9idNj6/E4l4CulMsGxxCm4zIfkC7hNUiJ7ABI9EypWaSSzvZp8x+/UQV2j2GWYMXueWDTdW26vHAMxpKEMYkCYN+AYzcDXBZ/32hxdI320YmV13NQvbfT2F+rStSFFeO7NC8vqU/wHTMPy3Eo12eazOSQU/cuWBhzFxkNW+KsOZYn6OjDcNeVk5Ly9JN1AQad5SgfI5XhHUDEg5/RFdeqIuN9ZxRyaFeWzHxTRV5d4e1aa0TimOzSvN7cvVYjDeVzk51qbkWBcHvUkqjU0hWKXiNHvSy6zqysJsw5DzJ9EiFtJbsOzmZcj0j5CkWrUziLifYluByi58FjdyK2sShwb0ElxGbthcsdRvEliH+9km2EpW2B9+Zr+ve7dJwoennAXuZ0de4VJkcR5f+lX4z9YtNbpZ8vi8CPJxoxtgNSkhwxJw4QrZMWlbJmyNgfO9 X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(396003)(346002)(136003)(376002)(39860400002)(40470700004)(46966006)(36840700001)(83380400001)(1076003)(426003)(86362001)(186003)(36756003)(2616005)(40460700003)(81166007)(47076005)(36860700001)(107886003)(336012)(8676002)(316002)(70586007)(478600001)(82740400003)(8936002)(44832011)(5660300002)(2906002)(54906003)(6916009)(82310400005)(4326008)(26005)(70206006)(41300700001)(7696005)(40480700001)(6666004);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:35.8934 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 69039600-7f7e-4b18-43bb-08da60f1ea35 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT046.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6032 We have moved acpi_scan_nodes from x86 to common. Because of of our previous work, this function no longer has many ACPI characteristics, except some "SRAT" words in print messages. So we rename acpi_scan_nodes to a more generic name numa_scan_nodes, and replace "SRAT" words in print messages. After doing this, it doesn't make sense to use CONFIG_ACPI_NUMA to gate numa_scan_nodes in numa_initmem_init. As CONFIG_ACPI_NUMA will be selected by CONFIG_NUMA for x86. So, we replace CONFIG_ACPI_NUMA by CONFIG_NUMA. We take this opportunity to make this function static, since it currently has no external callers. Signed-off-by: Wei Chen --- v1 -> v2: 1. Merge two patches into this patch: 1. replace CONFIG_ACPI_NUMA by CONFIG_NUMA. 2. replace "SRAT" texts. 2. Turn numa_scan_nodes to static. --- xen/arch/x86/include/asm/acpi.h | 1 - xen/common/numa.c | 10 +++++----- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/include/asm/acpi.h b/xen/arch/x86/include/asm/acpi.h index ab0d56dd70..f6ea3f1a9a 100644 --- a/xen/arch/x86/include/asm/acpi.h +++ b/xen/arch/x86/include/asm/acpi.h @@ -101,7 +101,6 @@ extern unsigned long acpi_wakeup_address; #define ARCH_HAS_POWER_INIT 1 -extern int acpi_scan_nodes(u64 start, u64 end); #define NR_NODE_MEMBLKS (MAX_NUMNODES*2) extern struct acpi_sleep_info acpi_sinfo; diff --git a/xen/common/numa.c b/xen/common/numa.c index 5ab061e991..0f62638e4c 100644 --- a/xen/common/numa.c +++ b/xen/common/numa.c @@ -285,7 +285,7 @@ static int __init nodes_cover_memory(void) } /* Use the information discovered above to actually set up the nodes. */ -int __init acpi_scan_nodes(paddr_t start, paddr_t end) +static int __init numa_scan_nodes(paddr_t start, paddr_t end) { int i; nodemask_t all_nodes_parsed; @@ -310,7 +310,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end) if ( memnode_shift < 0 ) { printk(KERN_ERR - "SRAT: No NUMA node hash function found. Contact maintainer\n"); + "NUMA: No NUMA node hash function found. Contact maintainer\n"); numa_bad(); return -1; } @@ -323,7 +323,7 @@ int __init acpi_scan_nodes(paddr_t start, paddr_t end) paddr_t size = nodes[i].end - nodes[i].start; if ( size == 0 ) - printk(KERN_INFO "SRAT: node %u has no memory\n", i); + printk(KERN_INFO "NUMA: node %u has no memory\n", i); setup_node_bootmem(i, nodes[i].start, nodes[i].end); } @@ -540,8 +540,8 @@ void __init numa_initmem_init(unsigned long start_pfn, unsigned long end_pfn) return; #endif -#ifdef CONFIG_ACPI_NUMA - if ( numa_status != numa_off && !acpi_scan_nodes(start, end) ) +#ifdef CONFIG_NUMA + if ( numa_status != numa_off && !numa_scan_nodes(start, end) ) return; #endif From patchwork Fri Jul 8 14:54:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Chen X-Patchwork-Id: 12911265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AE75BCCA47B for ; Fri, 8 Jul 2022 14:55:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.363478.594085 (Exim 4.92) (envelope-from ) id 1o9pOR-0000yu-R3; Fri, 08 Jul 2022 14:55:43 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 363478.594085; Fri, 08 Jul 2022 14:55:43 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOR-0000yO-Mz; Fri, 08 Jul 2022 14:55:43 +0000 Received: by outflank-mailman (input) for mailman id 363478; Fri, 08 Jul 2022 14:55:42 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1o9pOP-0006vD-Rl for xen-devel@lists.xenproject.org; Fri, 08 Jul 2022 14:55:42 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2077.outbound.protection.outlook.com [40.107.22.77]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 0889c9d2-fece-11ec-924f-1f966e50362f; Fri, 08 Jul 2022 16:55:40 +0200 (CEST) Received: from AS9PR06CA0729.eurprd06.prod.outlook.com (2603:10a6:20b:487::24) by AM6PR08MB5254.eurprd08.prod.outlook.com (2603:10a6:20b:d6::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16; Fri, 8 Jul 2022 14:55:37 +0000 Received: from AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:487:cafe::eb) by AS9PR06CA0729.outlook.office365.com (2603:10a6:20b:487::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16 via Frontend Transport; Fri, 8 Jul 2022 14:55:37 +0000 Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT024.mail.protection.outlook.com (10.152.16.175) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.16 via Frontend Transport; Fri, 8 Jul 2022 14:55:36 +0000 Received: ("Tessian outbound 6f9e7ef31fa8:v122"); Fri, 08 Jul 2022 14:55:35 +0000 Received: from 7752c8b1a1f5.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 21BF7C73-F39F-40A6-B26C-B6A8A57BAA62.1; Fri, 08 Jul 2022 14:55:29 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 7752c8b1a1f5.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 08 Jul 2022 14:55:29 +0000 Received: from DB6P192CA0005.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::15) by AM6PR08MB5046.eurprd08.prod.outlook.com (2603:10a6:20b:eb::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.18; Fri, 8 Jul 2022 14:55:27 +0000 Received: from DBAEUR03FT024.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:b8:cafe::bc) by DB6P192CA0005.outlook.office365.com (2603:10a6:4:b8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:27 +0000 Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT024.mail.protection.outlook.com (100.127.142.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5417.15 via Frontend Transport; Fri, 8 Jul 2022 14:55:27 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.9; Fri, 8 Jul 2022 14:55:24 +0000 Received: from ais-wip-ds.shanghai.arm.com (10.169.190.86) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.9 via Frontend Transport; Fri, 8 Jul 2022 14:55:21 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0889c9d2-fece-11ec-924f-1f966e50362f ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=IDTGzsRhwksJULtKqTjY6DjwNdhsnNqurtg+okPYa72msuUM/toiN8f1PRWhNI6gNxthYwrCMTzl3T07UM6Pc/LA1RAA/2gsG0a3GZ9lDSAGzc2DTCmvU9SriTtTevWI5qYnQbqpRDjGR8jjjR880icLtm7IEvWn4QjnRugTZ3P7FwmHAEaFtT0sNrgnLcMkZ/xW3E63bZckMblrldi+14UGAHUQu+DFXXLSrhd4EFt16ME5PDHBjogsYSiYsEeiQvO+5i8CgAphLkFDXji17NG8DXwYtnm8TWPfOefUiqWOlhmbeCeRW2KwD7t0Bpjnk/WHcWXDIfgVqjV7GXYS1A== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AqQSRNHq+Kh5K+Y/RMMNLbWNCA9xm34CxBmK5iY5sjc=; b=AfaHEjqEQ9V9viRakj2R4cKeA8zdehS1FtQEn74mzNDOLg+etnr0imuka4C2Z+bJzwbFpu/xXcoHo0bsvBexfGbbIWL5K89XvPqAgq+mB16IgP/z00hvyQzsKg2c05jLrwCXzId91AfG8+eJXTAsdk0ecdOMGC/LijDAx/i/uaz2g4/+A96cB/JUAb+lfUw0boJuqeN0cztx0Rv22RbR6CQI7TSneykEsICGY50lkqUYVspSArBbr2/A1AzYOkxWHwg+nuRMOGHK+XQx+M7LhBIK7kSn55TOIRkFsatjCRbB35F9u5HDSnpAKbBzIQ5Q5K0//dkdOwzsh+96MaODsg== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AqQSRNHq+Kh5K+Y/RMMNLbWNCA9xm34CxBmK5iY5sjc=; b=Yc4jdjb/IM4tHSoDvl2tvsQZ7aRMwSB+6yNDdg02G/WFQYh6JMByaDhv3yv0fmiTwugRRwSKpEDbh06FMU7B1oGPEWp4gncrMVKDJ0RJqcUnpXAI8JfiG5Ivgu5k62DBrFcjECybBvFEXFcnBE9cgjTf4yaXLvezTRBgZI8Otdg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C X-CheckRecipientChecked: true X-CR-MTA-CID: 7288404d879e14c7 X-CR-MTA-TID: 64aa7808 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G0JifeM/ZTVeOzaJd/rqJsYwn7ymazYlwX1IaRiHCQZXmTREvwpoWlPhp2oQQKngo1fHoKq18vZ2vpU+V2fVKU3xRuJX/b3IEO2yWzMnbwjdryuK6B1lWN789tLoLgcVcUZ9zA5AdtcP6sl5zj/5fbLzxNL9Zz9lS4klmqze8+OuOtScvHCSUgUazB/E239fjfwm+ZBttHKxPCUTB3oq+TYJvnnuE/Lil+bXJ8yRma+K3Q6btGEXM4YwxorcZfOPYXGvvaxiRG00vC/TrCpM/88qlIn1CDO9E0VCHmBlj4SnPQ6peu1d/m6JIO1VCmnbUH4x114o2UI9TYjERyNVvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AqQSRNHq+Kh5K+Y/RMMNLbWNCA9xm34CxBmK5iY5sjc=; b=Oey9p/ceIM+USjzcXwqHSC5QLXDOfn9XWku/cYDr4gJ4LkNRyFdV35/KbxI5b0CHQ6rbWBZc3JGdOjObKi2JOUqP2cDdHUjx9oi4CDwrKFBEVu62q47AfasenhqGu6I2PS1b26c2sQk9+EULuakylHOxK7msRNg16W7Sv3otSOyY5lXY2pdTJNBfHRaYkbja2hkabvGbLQyLG/Nmlc6O9UJXUovl8QsgzWRRf9M8NYnCskOj1oNTLdoIVCQe2mGfUOCOhPPuk28nJH9QscLNBo3vDQFHZ5iVYb4FUuURRTs/t+UlBQrFpRvplAClsSreOSENbqb7UXESUAOGu9CdhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AqQSRNHq+Kh5K+Y/RMMNLbWNCA9xm34CxBmK5iY5sjc=; b=Yc4jdjb/IM4tHSoDvl2tvsQZ7aRMwSB+6yNDdg02G/WFQYh6JMByaDhv3yv0fmiTwugRRwSKpEDbh06FMU7B1oGPEWp4gncrMVKDJ0RJqcUnpXAI8JfiG5Ivgu5k62DBrFcjECybBvFEXFcnBE9cgjTf4yaXLvezTRBgZI8Otdg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C From: Wei Chen To: CC: , Wei Chen , Andrew Cooper , George Dunlap , "Jan Beulich" , Julien Grall , "Stefano Stabellini" , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 9/9] xen: introduce a Kconfig option to configure NUMA nodes number Date: Fri, 8 Jul 2022 22:54:24 +0800 Message-ID: <20220708145424.1848572-10-wei.chen@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220708145424.1848572-1-wei.chen@arm.com> References: <20220708145424.1848572-1-wei.chen@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-Office365-Filtering-Correlation-Id: 34c1eade-cf35-4824-f0db-08da60f1ea50 X-MS-TrafficTypeDiagnostic: AM6PR08MB5046:EE_|AM5EUR03FT024:EE_|AM6PR08MB5254:EE_ x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 3Fd/XZcz0ZlvEEnQuXvVOhYKfLZCJSXaErlE9aLZxtzt/Xeg6NI5VLQ/iB9kv/13ceusoWyVWb3nJxYod3NMcmij2fFxGnPCsgQ/2U3VQqWaeKVu+qWseuaD76yyce/WqTyTynjKb9kLS8nt3pqE6HykCb+YxDvu0EHVGUeb/2wdNMZQKEGJjvOeER1KgqOI9XeV7QLpG5IntslBpRMNW/AbmVaf6Z+OKJlpFYjPOPUuOLymC0T4GAs0GJoOizS8x/FQ6Y3D7rZfib8mQQeVVQpaxsa3l4pJ1X5rU8mSsfaXMGTZC2zz2fPYX44K0JnKSqIs4nJ3z3uiGWuVEcSOZS/5u1xrjn3jcioThO1eGQJz4vmFp9196adB+ahQj/fVJ4/p5HFu9awacXRa2n3/SYvekzAyqwVSqCr+NNjb5WoaAWE/WcSpJmYHw+l/+N0QluHR/4g+/Bx0mpOp73dbTE+WZDVH9Tunm4KRBMQthHilTm/x03LjKC6e717yKLCsY3h/UivZFGf1PoYJzdPTYZBpXylivg3w44m+Vb7wGs5a+ISyx/KQjuFAOcnp1w6MAiMksmAIgpbfcDwyQInQxf+3//XQxfiRY6GRFCKKHZX/ZVQdKl7FNHhI1O+yKsckE6ZL/xY/xnfUUg0U+l6+W2K7vtP3alG/LlHxAgrHhWDoMbAGALAJPCE2aIhByve8puKvEObz9WmgUOnvJSgi2CcMtCayWNuyeghfUinxphAupqaXhjGh/a5Z+uneQKj28DnfbRuirFtr+wjVTdK9g+aj92XI2UKs0aIyQDEFY/8rkcD6ayeEAd8/nji/OckJKQlw7+Y+9AiVfpjBcgi6maLzX054H/iRw46vsqZ3CiE= X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:nebula.arm.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(136003)(39860400002)(396003)(346002)(376002)(36840700001)(46966006)(40470700004)(4326008)(316002)(82310400005)(40460700003)(2616005)(7696005)(186003)(54906003)(41300700001)(426003)(81166007)(36756003)(26005)(83380400001)(6916009)(336012)(356005)(47076005)(82740400003)(8936002)(8676002)(1076003)(2906002)(40480700001)(5660300002)(86362001)(70586007)(36860700001)(44832011)(478600001)(70206006)(36900700001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5046 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 3780fdb9-8ead-4cc7-2fe2-08da60f1e543 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1xzEf1u4GSKjmXRMhzNXl8P3FhPn7fOaqzSWV0tYk1P12sfM2YG3zN3nk4zRM0CpeQOfmWBIHjwdi/GlVUvDB8R843ChXluT7aXKObN3sdjogQ9vk2cGwEZr+XUCzg+C31DLhzdPpwgDJSm35lc8AKNTnKQFpc8bb8eMvpCOaV3gpYoRXrgBQLMzx2Lp+DBVIowftpRPD3drBT6mc9cxJ8GQwyHThRgsUOes2vRQWE6a1kO6NkBTJZ/qYsdt5S16/LGN3EUk1mAGI0l+OAUiOODnoO4fNsoqy9S/eazA5OQIvzPOiAVfRpcwOPx2Wo3J7tiqXqJoUuGLOBFfzSNyoDR2Zydc7dhrUA6UP7GhiMnhnVSgTd61faN9s238y/iay/jXEhCTgXq4vcKkc/RnwivfWtOirtCJgpDM651C4zD3+1dX/W+LEdc4GWYTshDYEBla8kRfq77b397s0HJ8UYVieG3LpZPxKrLCY10KrOVaWPrA54jH2worZ1E0iux4ayzIsDP10RTr9Ks4GStrgrh2yrP6thqCFE8geb7qVMm5kEXwhyvnJGv5WYbM+qrVrFFYYXVmxSlbpEzhY9MbfMiuTtFUKhtNAoIDPLIPXF4crew47bYYcIUYHahKyHB7FidrYRkPTr3McjlYQMgpgtMdRFCLnexDD66nQ6z4wgP2bsWx+ybOcMmg8/bGhQeiU41I+PrbRomCSvi/jNsKF475B66Ql64QYCag+GGXFGHuQjhCY0yIsVzL5W1iZKbCdcEe0a8SHLwDekSEfscmBwOIS9oiuRcHjrIhU+6lfRKKgs1xyiEA03kjXCEY8rUr X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(346002)(396003)(136003)(376002)(36840700001)(40470700004)(46966006)(36756003)(4326008)(2906002)(478600001)(8676002)(336012)(70586007)(82740400003)(40480700001)(54906003)(40460700003)(70206006)(82310400005)(107886003)(81166007)(47076005)(186003)(5660300002)(1076003)(2616005)(44832011)(41300700001)(6916009)(316002)(36860700001)(7696005)(83380400001)(86362001)(26005)(8936002)(426003);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2022 14:55:36.0435 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 34c1eade-cf35-4824-f0db-08da60f1ea50 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT024.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB5254 Current NUMA nodes number is a hardcode configuration. This configuration is difficult for an administrator to change unless changing the code. So in this patch, we introduce this new Kconfig option for administrators to change NUMA nodes number conveniently. Also considering that not all architectures support NUMA, this Kconfig option only can be visible on NUMA enabled architectures. Non-NUMA supported architectures can still use 1 as MAX_NUMNODES. As NODES_SHIFT is currently unused, we're taking this opportunity to remove it. Signed-off-by: Wei Chen --- v1 -> v2: 1. Add NODES_SHIFT remove message in commit log 2. Change NR_NUMA_NODES upper bound from 4095 to 255. --- xen/arch/Kconfig | 11 +++++++++++ xen/arch/x86/include/asm/numa.h | 2 -- xen/include/xen/numa.h | 10 +++++----- 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig index f16eb0df43..4fc16f83ac 100644 --- a/xen/arch/Kconfig +++ b/xen/arch/Kconfig @@ -17,3 +17,14 @@ config NR_CPUS For CPU cores which support Simultaneous Multi-Threading or similar technologies, this the number of logical threads which Xen will support. + +config NR_NUMA_NODES + int "Maximum number of NUMA nodes supported" + range 1 255 + default "64" + depends on NUMA + help + Controls the build-time size of various arrays and bitmaps + associated with multiple-nodes management. It is the upper bound of + the number of NUMA nodes that the scheduler, memory allocation and + other NUMA-aware components can handle. diff --git a/xen/arch/x86/include/asm/numa.h b/xen/arch/x86/include/asm/numa.h index eeb431cdb7..db76281c08 100644 --- a/xen/arch/x86/include/asm/numa.h +++ b/xen/arch/x86/include/asm/numa.h @@ -3,8 +3,6 @@ #include -#define NODES_SHIFT 6 - typedef u8 nodeid_t; extern int srat_rev; diff --git a/xen/include/xen/numa.h b/xen/include/xen/numa.h index 4c4632ec27..cac92d2688 100644 --- a/xen/include/xen/numa.h +++ b/xen/include/xen/numa.h @@ -3,14 +3,14 @@ #include -#ifndef NODES_SHIFT -#define NODES_SHIFT 0 -#endif - #define NUMA_NO_NODE 0xFF #define NUMA_NO_DISTANCE 0xFF -#define MAX_NUMNODES (1 << NODES_SHIFT) +#ifdef CONFIG_NR_NUMA_NODES +#define MAX_NUMNODES CONFIG_NR_NUMA_NODES +#else +#define MAX_NUMNODES 1 +#endif #define vcpu_to_node(v) (cpu_to_node((v)->processor))