From patchwork Thu Apr 7 13:27:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12805240 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D23CC433EF for ; Thu, 7 Apr 2022 13:27:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.300801.513170 (Exim 4.92) (envelope-from ) id 1ncSAV-00071a-M3; Thu, 07 Apr 2022 13:27:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 300801.513170; Thu, 07 Apr 2022 13:27:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ncSAV-00071T-Iz; Thu, 07 Apr 2022 13:27:23 +0000 Received: by outflank-mailman (input) for mailman id 300801; Thu, 07 Apr 2022 13:27:21 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1ncSAT-00071L-UF for xen-devel@lists.xenproject.org; Thu, 07 Apr 2022 13:27:21 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 739a1262-b676-11ec-a405-831a346695d4; Thu, 07 Apr 2022 15:27:20 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2171.outbound.protection.outlook.com [104.47.17.171]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-32-CHl0xR3IPmuFYhT2BynHxg-1; Thu, 07 Apr 2022 15:27:19 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM0PR04MB6321.eurprd04.prod.outlook.com (2603:10a6:208:143::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.22; Thu, 7 Apr 2022 13:27:17 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::914d:e08d:7798:8476]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::914d:e08d:7798:8476%5]) with mapi id 15.20.5144.022; Thu, 7 Apr 2022 13:27:17 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 739a1262-b676-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1649338040; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=BQOVcAGYC8ddiE2/MCQ8tM4DG8UY9B3n/ZS2T6mggZk=; b=b1kg0AmOV3JxpmQtAZdgM8eNGCDRxlD8GdA/xNFfqhHJD/koz82MjZVBf5g8ZAh5cuW+eP Ta0zoYAx9CiE/XEucvc/XGc14e4B8VYmrlDZ48y3rCfx0YDnVkLzlNUZTnjMKNHvHE23cE YDT82M5yjo2sdjxhBQPKTIVPy6Bi6UI= X-MC-Unique: CHl0xR3IPmuFYhT2BynHxg-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MVkMqapOCKKFj63UAk8SxUcqXarXyPm3/0vgizDxwoufv9Hp1bvRDPk94WadBLiBuVcIJZdsxwM7Ua67kuhdQdvMUiW4ESIQ6iN4AtegLqE/mx73tomjCpuVUfxF+U0lIBK4M1HqttUBnGqvzSz5EF5ODbgTrD+c25gsdC4DegoJ+TtlTX/tUUetXOArE8LP1v9P1jjgErnTFvbqlhQHACLenxHuqEsX7tP2E+BNsyjteRiEYOSa64/YeU50HYGffJEY4v+bUKqKcM9f59ttSj/Y9oZOLxe5GA1CDLvK2TeQfLq2988bDgH3WhKEQv5x5T2lnnltWTrE1D54+B+DUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BQOVcAGYC8ddiE2/MCQ8tM4DG8UY9B3n/ZS2T6mggZk=; b=WGgZwX9y05PnDclLpQ7T9/dJP5/OWOPtHGIeEuj4Q8CQGZJJUZ+NEVOZUn1zl2xu7S15GtZnWyeFbpvgtTHEjfldRQ2U6pslWdqqBNFQoorGBMN1BLEPJFOvX+HZ4mIPs1V2WAqmv87Sdl6dfiC0DcUMJz1VvKMNMBF8WwFSoB02kc/Uu8S6BPn5obGZjgGQG1Z/9qAf9QFSZn9o7fZ84C7BMp0M0HAk3vh2q1U/Jiil1Nh2vEAVwJGXwyQdGCHHQrEFAMyVVQ3JR0CIFCDuQY3CSqdbGhAw1jTRTItPVZEFgPpxPc1LvFJy4l9P2PPjcQDCJyXPFMb3yH9yOTCzeA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <1617b87a-640f-d235-701d-df2c57314b73@suse.com> Date: Thu, 7 Apr 2022 15:27:13 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Content-Language: en-US To: "xen-devel@lists.xenproject.org" Cc: George Dunlap , Dario Faggioli , Olaf Hering From: Jan Beulich Subject: [PATCH] x86: make "dom0_nodes=" work with credit2 X-ClientProxiedBy: FRYP281CA0008.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10::18) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fb084767-a481-492f-eb9d-08da189a55d8 X-MS-TrafficTypeDiagnostic: AM0PR04MB6321:EE_ X-LD-Processed: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba,ExtFwd X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iuRtJnqYNY6jQX0WBDRlvIFQuJXcAc/O44Bk96r7M16sjTA5CVAu6Q2yhogtQNQD23XyVALUbxCgl0NQ+AkeN2cNb5gqY15Ahjmo53VUnYe6aoZlbOHbQDnb0XQahV5GyMHc4bXBngIx3SXHXyD/5r+9SUsMCbUAUdU3uzWdm7v7v9thG1JsZUlOxacQLR4B59BA/uqSMJSLrTTPRhO6ohian0M79APP/zH5mM4eW9MMHsgtYayVX2uev56f51AaxrBas+445UgHoLN/0yfPVxuHHGKjKOfON7AF8cq6hra6VYUohob4AJLYrh2BkurFyVZSE1d8jafEyBsmoZ75wmKa3u3TE+9WAEXB7DcGRV0nVOJZqBiKI4/lfIJiBQ6BK4y5FGuMCIPShkZSOUUDN1i5QSIaxptKeOunxYM6P2WykkTzugRQDrfvNybYfqdQHSWn1ICyhovZ/+iVQ7NZGe7pN/XOsgL6UbuDlf2XjTVTxwocJOFoH+2vzQesz2L5+SkfY8jh+Z8iG3/nTzkiwVzIZHtnv/nEcFnho73i5pRm4ec4lbvzVNkBGucZujlcpZ35NaZ8Y32o+0T2Ze2vF3pR6HAZZPtPRYAesekjWJsDL08bmWWE/oDLfFlCup6g5H+yKrBqA4oF2KO9gCkOkoddx8r2FdAwhKtL339Mlus6ZOBXogjlHjxuRw9CfINI8HmtTT+Z5AXQDoTpeBRSYhJl4qP36gVPtm3t5f8WPPI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(4326008)(508600001)(8676002)(31686004)(36756003)(8936002)(6506007)(6486002)(38100700002)(66946007)(66476007)(66556008)(2616005)(6666004)(31696002)(26005)(6512007)(86362001)(54906003)(186003)(316002)(5660300002)(83380400001)(6916009)(2906002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?kkOu3xgISvQ8gLZHndXug9JsieQ+?= =?utf-8?q?lMeI/nH7fOD2PnOBUlPvulcV8B4TJuB19mCuX1+0Insgwqq5EE/Zofx5fPv1RMaY9?= =?utf-8?q?lZZbCHom0K6kTC2YC+rNE6QOA9sHw52G1eArIoQboAvsObSW8V2GEhh6bhi65JDnP?= =?utf-8?q?zMWk+b6qU3kH4AROpqXfkjU4kZeHEQbel8owLISIDhdRg4lRLlGrM7moVA6ToqITu?= =?utf-8?q?njeX9SOhJucGkbAO/g7jnNEDVLClRnpAkWIzc/L+le4gPdMOPQU18SZDeWhS9fQv2?= =?utf-8?q?tSq5RtJaBpEWu70TD0jzhdIw12SIw3fa3xeacUc7XwBohsFzbPb6pBsnBTUxKHE5u?= =?utf-8?q?vvju5u9x2oE3WrgkuEBwDNEUsEBOxORXcoZKMB6wfHOCIs0AopQGvkgACLXbUEzp4?= =?utf-8?q?8X56sWT6wX1bzPyqNoMA6Q6yH4/J9hiORA4CgY50gLv0qz5zL8lfH+eHTKsbXY9+x?= =?utf-8?q?2qubG1NLOBH+YwHON0mX1Juyj0L149UDAORVHx70rQ/G2nr38iGladh/SYUacuJqV?= =?utf-8?q?BvNG93PAn5y/D/ro0wNZOFvqjceoTgFPDfHPAVww7WaOIRPd2uQ9d7d/APyyzl1DR?= =?utf-8?q?aFb9dn80WXtertcCQiYkjI15AN3VabhVCUvff72iFYKdCS+a1VNNZ8I9AUnhsuARf?= =?utf-8?q?A9UGo8GKSsLhz3YzsO/ElJC+rBfjc1d/Tb73pS5/3EfAtDMSq8U+t8fckLpHeAorW?= =?utf-8?q?UrgnnvBAznIzM5AeOXZH/ZP4KYeznqX7mcEE2cu6adLxnHtUu3cKDTDExjT8tTJ4u?= =?utf-8?q?zN4DdNhzkU5sSDtIAxH19rYF6cXd3P+rdyFR7aAYPKvnQalbeg2JzcYdeQ71YvhQv?= =?utf-8?q?ShAibU1KfuRPMNavcXURMouh8RK/Jd9o7xH8AugRG4jnRfoGb40CObfZz0L+5tm0h?= =?utf-8?q?4KuYy07VQR4P0sD/OSv9WG+Gkt1p8muVV5ESs4/CCvlfUphCztwn02Cl3YBEQ8qXr?= =?utf-8?q?yzMcLX6k6ZbA9SHyTf5V/AjvxD3CNABX9gk3aMKJR1sN24zmH2WvqWfddEVGc02l+?= =?utf-8?q?K09RaJm85eSi08p0HC53bA/30P1ypqaXOmVrwA73FIVVR3uxrBKhhBCzKpj48Dp3g?= =?utf-8?q?N6ktt2MCIoSEy5Z8tuqaAAE/Kz8/3vrubT/0sqTORQNkOvVJuCIZgOdXYIbM5+g6q?= =?utf-8?q?PTnxNRShOkeVwIoWwU4/YCB6PC1tHpcLYBFAFSlUZqjGdDbXdFqb8PqeJ9JXjQrOx?= =?utf-8?q?pUKEXShb1YBRgNKhuhoe6E8ZUhBqu0lMxw5vZuJmm0ls+1WN5zg/8sjRL4f0NppWw?= =?utf-8?q?RzbcHxpKMOtH+f80b58phVLTwBtNs+gqe6CVg5ZPFc944pyXHrNo15AwYha8WjAfN?= =?utf-8?q?V1ZMCjwpp0+/KdZOBlKtextPx+SekUcuU3iBfeVTRyf8tfOSXoyGnkyeUY/BaDPct?= =?utf-8?q?AC7UWXBeIdgkXFqX55CJFVUaqJ3MuON+e7c7cmOXPyhe11wHM6gke43lNvNTKMERr?= =?utf-8?q?jf6GRg4ISPUhVuktkIZzI1te6BPsH4FAJ/uBF+dN5VJIQpYazHDG/FO9m3lJRR1Xu?= =?utf-8?q?lKrPwkSm8K5c1XNxDLLBx2xdUFbrqyk2rBVth3+loz1Fgas052TtiOCzmo3vStM8o?= =?utf-8?q?wdCFVyLmhABfbarvsDSclQ3CzcwNX0SY8oiXkASchVpvUYHEukyZjE8AS3ni4XF7e?= =?utf-8?q?59RGJgZAjo42fSzqbUgJ65zpsdTxF2WQ=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: fb084767-a481-492f-eb9d-08da189a55d8 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2022 13:27:17.1905 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: G0AJPc4/3UqgLNHiUKKerSeJZcay9F9sogIXyS29Nec0lfrzO9uAr2BSptPnie43YXKefzNu7Szoz6S6FZgFBw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB6321 Apart from setting the affinities, vCPU-s / units also need to be migrated off of CPUs which aren't part of the affinity. Otherwise, for a reason I haven't been able to determine, credit2 won't actually schedule them, resulting in the Dom0 boot process to hang (either right away or when bringing up "secondary" vCPU-s). Fixes: dafd936ddd ("Make credit2 the default scheduler") Reported-by: Olaf Hering Signed-off-by: Jan Beulich Signed-off-by: Dario Faggioli Signed-off-by: Dario Faggioli --- The Fixes: tag isn't very precise - it's rather the commit exposing the issue by default. I haven't been able to identify the actual commit which did introduce the problem; it may well be that it has always been there since the introduction of credit2. Credit2 moving the vCPU-s off of their initially assigned ones right away of course renders sched_select_initial_cpu()'s use of cpu_cycle() pretty useless. But I guess that's still useful for other schedulers. I wonder though whether sched_init_vcpu() shouldn't use the CPU map calculated by sched_select_initial_cpu() for its call to sched_set_affinity() in the non-pinned case, rather than setting "all". (I guess doing so might mask the issue at hand, but I think the change here would still be applicable at least from an abstract pov.) --- a/xen/common/sched/core.c +++ b/xen/common/sched/core.c @@ -3403,9 +3403,15 @@ void __init sched_setup_dom0_vcpus(struc { for_each_sched_unit ( d, unit ) { + spinlock_t *lock = unit_schedule_lock_irq(unit); + if ( !opt_dom0_vcpus_pin && !dom0_affinity_relaxed ) sched_set_affinity(unit, &dom0_cpus, NULL); sched_set_affinity(unit, NULL, &dom0_cpus); + + sched_unit_migrate_start(unit); + unit_schedule_unlock_irq(lock, unit); + sched_unit_migrate_finish(unit); } }