From patchwork Tue May 18 09:47:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D25BC433ED for ; Tue, 18 May 2021 09:52:05 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1616E610A1 for ; Tue, 18 May 2021 09:52:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1616E610A1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=w6mrUlN0y825EqF1kRSJUD79hCJ6on+gZxyje57auaI=; b=FbC6yeAQbUGXgxskBVSr+mcaa jsMHMO+UuQraSWtT2UKfa0VzJmuLL2Rr0RZdgVlD66z8s4jjirFKQ/vCQKDwrV9zjPBUXadgDtIJz Kusq7fOPddnOBXp8IX/DCxASBpwAaLslehDdVfmP7bDZtLCXsR8s7XbzRcX1YvpvZHG9bcok6uTlD Z9m7EnQDpy4aE/32GNs+yIRF59Ir1E0JhTCmw6v5/WaSBulaKNG0K4cvG83W2NZrYqmL1eUV6vNXH Msu/If4TRtURpw/gls0TH4EuaOyhY9A5UHsK/FzhB2i5lABsLwzhDytCaunmibor4sMdsME0yyQH/ fMBNyor/A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwMC-000EEz-9u; Tue, 18 May 2021 09:49:44 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKG-000Dld-PH for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:47:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=t/t2sA0lu1tV1j0vkkcTOro+7wRA5sxcwtiJVXFhYOs=; b=qi4NZbFhcw59eKUJlylekN3OVK TPCvXc9blLI8c0NPt/tlILzWXimdm2edlmWpnm3bnB8l/5DnV5Fyfe3pP5fDVCWvzBrr4nbT14h5g BZJcsQA4Z83Fao0/OVSF/jLdcDLFCYV9zBlHmKZp4wpVKgE2UgxSof4ECOfK6gdx9zsBoMkdnGDhR mbJZvxiTyWkmPWrjneMGhgtvVHltoorbX1joE4E4+wvnRHrZVbfyoG2P6XLENf6N6irdl4h/x0Eo2 U3SJUwJWeT5JR0ecYLK2rvXJEqi6OK3w4kvnvI1LDJDxNt/KEgdnya9Ec/lutgJu9t/hYnUxEwjJC BOMAg4MA==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKD-00EWZL-J7 for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:47:43 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 32F82613AF; Tue, 18 May 2021 09:47:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331261; bh=dCNPT7d4MXmE+nR7ZcFB90m9Ei2x4bTExWIoF4yD9Qs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cy31wkIT4MaetGLnyd6X5oYKOO6W5utH9aUBM6zP2nXpDlNAZCdzxyRzv3aARBcSA GvA+PAvJFZx59fLZ5rhqnb8m9xd8yanArmFS7vfNkss3+hBxtFhbnrf5OemviQCBlG e9VFeOuRjS9bN+Us8XYl0EnTl1PCLN12XJfaP12vEliroIHkTlrXQUEvtakDpH9f1a SGjzqFZCrZFyGhgs245iDLOR1dajoNFdcA/pqvnjC8vrpCOqMirNZ6x7HlDeiOytAp xEU/QbapGYA0zQOxh+TqUDlY7z1W0Hk5JIzUPB1LkUkXKZuDigASoIsHvI9ElYTi4t 4y7rw8o9sVHTA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 01/21] arm64: cpuinfo: Split AArch32 registers out into a separate struct Date: Tue, 18 May 2021 10:47:05 +0100 Message-Id: <20210518094725.7701-2-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024741_693388_12286DC9 X-CRM114-Status: GOOD ( 19.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for late initialisation of the "sanitised" AArch32 register state, move the AArch32 registers out of 'struct cpuinfo' and into their own struct definition. Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/cpu.h | 44 +++++++++++---------- arch/arm64/kernel/cpufeature.c | 71 ++++++++++++++++++---------------- arch/arm64/kernel/cpuinfo.c | 53 +++++++++++++------------ 3 files changed, 89 insertions(+), 79 deletions(-) diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h index 7faae6ff3ab4..f4e01aa0f442 100644 --- a/arch/arm64/include/asm/cpu.h +++ b/arch/arm64/include/asm/cpu.h @@ -12,26 +12,7 @@ /* * Records attributes of an individual CPU. */ -struct cpuinfo_arm64 { - struct cpu cpu; - struct kobject kobj; - u32 reg_ctr; - u32 reg_cntfrq; - u32 reg_dczid; - u32 reg_midr; - u32 reg_revidr; - - u64 reg_id_aa64dfr0; - u64 reg_id_aa64dfr1; - u64 reg_id_aa64isar0; - u64 reg_id_aa64isar1; - u64 reg_id_aa64mmfr0; - u64 reg_id_aa64mmfr1; - u64 reg_id_aa64mmfr2; - u64 reg_id_aa64pfr0; - u64 reg_id_aa64pfr1; - u64 reg_id_aa64zfr0; - +struct cpuinfo_32bit { u32 reg_id_dfr0; u32 reg_id_dfr1; u32 reg_id_isar0; @@ -54,6 +35,29 @@ struct cpuinfo_arm64 { u32 reg_mvfr0; u32 reg_mvfr1; u32 reg_mvfr2; +}; + +struct cpuinfo_arm64 { + struct cpu cpu; + struct kobject kobj; + u32 reg_ctr; + u32 reg_cntfrq; + u32 reg_dczid; + u32 reg_midr; + u32 reg_revidr; + + u64 reg_id_aa64dfr0; + u64 reg_id_aa64dfr1; + u64 reg_id_aa64isar0; + u64 reg_id_aa64isar1; + u64 reg_id_aa64mmfr0; + u64 reg_id_aa64mmfr1; + u64 reg_id_aa64mmfr2; + u64 reg_id_aa64pfr0; + u64 reg_id_aa64pfr1; + u64 reg_id_aa64zfr0; + + struct cpuinfo_32bit aarch32; /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */ u64 reg_zcr; diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index efed2830d141..a4db25cd7122 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -863,6 +863,31 @@ static void __init init_cpu_hwcaps_indirect_list(void) static void __init setup_boot_cpu_capabilities(void); +static void __init init_32bit_cpu_features(struct cpuinfo_32bit *info) +{ + init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0); + init_cpu_ftr_reg(SYS_ID_DFR1_EL1, info->reg_id_dfr1); + init_cpu_ftr_reg(SYS_ID_ISAR0_EL1, info->reg_id_isar0); + init_cpu_ftr_reg(SYS_ID_ISAR1_EL1, info->reg_id_isar1); + init_cpu_ftr_reg(SYS_ID_ISAR2_EL1, info->reg_id_isar2); + init_cpu_ftr_reg(SYS_ID_ISAR3_EL1, info->reg_id_isar3); + init_cpu_ftr_reg(SYS_ID_ISAR4_EL1, info->reg_id_isar4); + init_cpu_ftr_reg(SYS_ID_ISAR5_EL1, info->reg_id_isar5); + init_cpu_ftr_reg(SYS_ID_ISAR6_EL1, info->reg_id_isar6); + init_cpu_ftr_reg(SYS_ID_MMFR0_EL1, info->reg_id_mmfr0); + init_cpu_ftr_reg(SYS_ID_MMFR1_EL1, info->reg_id_mmfr1); + init_cpu_ftr_reg(SYS_ID_MMFR2_EL1, info->reg_id_mmfr2); + init_cpu_ftr_reg(SYS_ID_MMFR3_EL1, info->reg_id_mmfr3); + init_cpu_ftr_reg(SYS_ID_MMFR4_EL1, info->reg_id_mmfr4); + init_cpu_ftr_reg(SYS_ID_MMFR5_EL1, info->reg_id_mmfr5); + init_cpu_ftr_reg(SYS_ID_PFR0_EL1, info->reg_id_pfr0); + init_cpu_ftr_reg(SYS_ID_PFR1_EL1, info->reg_id_pfr1); + init_cpu_ftr_reg(SYS_ID_PFR2_EL1, info->reg_id_pfr2); + init_cpu_ftr_reg(SYS_MVFR0_EL1, info->reg_mvfr0); + init_cpu_ftr_reg(SYS_MVFR1_EL1, info->reg_mvfr1); + init_cpu_ftr_reg(SYS_MVFR2_EL1, info->reg_mvfr2); +} + void __init init_cpu_features(struct cpuinfo_arm64 *info) { /* Before we start using the tables, make sure it is sorted */ @@ -882,29 +907,8 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info) init_cpu_ftr_reg(SYS_ID_AA64PFR1_EL1, info->reg_id_aa64pfr1); init_cpu_ftr_reg(SYS_ID_AA64ZFR0_EL1, info->reg_id_aa64zfr0); - if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) { - init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0); - init_cpu_ftr_reg(SYS_ID_DFR1_EL1, info->reg_id_dfr1); - init_cpu_ftr_reg(SYS_ID_ISAR0_EL1, info->reg_id_isar0); - init_cpu_ftr_reg(SYS_ID_ISAR1_EL1, info->reg_id_isar1); - init_cpu_ftr_reg(SYS_ID_ISAR2_EL1, info->reg_id_isar2); - init_cpu_ftr_reg(SYS_ID_ISAR3_EL1, info->reg_id_isar3); - init_cpu_ftr_reg(SYS_ID_ISAR4_EL1, info->reg_id_isar4); - init_cpu_ftr_reg(SYS_ID_ISAR5_EL1, info->reg_id_isar5); - init_cpu_ftr_reg(SYS_ID_ISAR6_EL1, info->reg_id_isar6); - init_cpu_ftr_reg(SYS_ID_MMFR0_EL1, info->reg_id_mmfr0); - init_cpu_ftr_reg(SYS_ID_MMFR1_EL1, info->reg_id_mmfr1); - init_cpu_ftr_reg(SYS_ID_MMFR2_EL1, info->reg_id_mmfr2); - init_cpu_ftr_reg(SYS_ID_MMFR3_EL1, info->reg_id_mmfr3); - init_cpu_ftr_reg(SYS_ID_MMFR4_EL1, info->reg_id_mmfr4); - init_cpu_ftr_reg(SYS_ID_MMFR5_EL1, info->reg_id_mmfr5); - init_cpu_ftr_reg(SYS_ID_PFR0_EL1, info->reg_id_pfr0); - init_cpu_ftr_reg(SYS_ID_PFR1_EL1, info->reg_id_pfr1); - init_cpu_ftr_reg(SYS_ID_PFR2_EL1, info->reg_id_pfr2); - init_cpu_ftr_reg(SYS_MVFR0_EL1, info->reg_mvfr0); - init_cpu_ftr_reg(SYS_MVFR1_EL1, info->reg_mvfr1); - init_cpu_ftr_reg(SYS_MVFR2_EL1, info->reg_mvfr2); - } + if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) + init_32bit_cpu_features(&info->aarch32); if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) { init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr); @@ -975,20 +979,12 @@ static void relax_cpu_ftr_reg(u32 sys_id, int field) WARN_ON(!ftrp->width); } -static int update_32bit_cpu_features(int cpu, struct cpuinfo_arm64 *info, - struct cpuinfo_arm64 *boot) +static int update_32bit_cpu_features(int cpu, struct cpuinfo_32bit *info, + struct cpuinfo_32bit *boot) { int taint = 0; u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1); - /* - * If we don't have AArch32 at all then skip the checks entirely - * as the register values may be UNKNOWN and we're not going to be - * using them for anything. - */ - if (!id_aa64pfr0_32bit_el0(pfr0)) - return taint; - /* * If we don't have AArch32 at EL1, then relax the strictness of * EL1-dependent register fields to avoid spurious sanity check fails. @@ -1135,10 +1131,17 @@ void update_cpu_features(int cpu, } /* + * If we don't have AArch32 at all then skip the checks entirely + * as the register values may be UNKNOWN and we're not going to be + * using them for anything. + * * This relies on a sanitised view of the AArch64 ID registers * (e.g. SYS_ID_AA64PFR0_EL1), so we call it last. */ - taint |= update_32bit_cpu_features(cpu, info, boot); + if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) { + taint |= update_32bit_cpu_features(cpu, &info->aarch32, + &boot->aarch32); + } /* * Mismatched CPU features are a recipe for disaster. Don't even diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c index 51fcf99d5351..264c119a6cae 100644 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -344,6 +344,32 @@ static void cpuinfo_detect_icache_policy(struct cpuinfo_arm64 *info) pr_info("Detected %s I-cache on CPU%d\n", icache_policy_str[l1ip], cpu); } +static void __cpuinfo_store_cpu_32bit(struct cpuinfo_32bit *info) +{ + info->reg_id_dfr0 = read_cpuid(ID_DFR0_EL1); + info->reg_id_dfr1 = read_cpuid(ID_DFR1_EL1); + info->reg_id_isar0 = read_cpuid(ID_ISAR0_EL1); + info->reg_id_isar1 = read_cpuid(ID_ISAR1_EL1); + info->reg_id_isar2 = read_cpuid(ID_ISAR2_EL1); + info->reg_id_isar3 = read_cpuid(ID_ISAR3_EL1); + info->reg_id_isar4 = read_cpuid(ID_ISAR4_EL1); + info->reg_id_isar5 = read_cpuid(ID_ISAR5_EL1); + info->reg_id_isar6 = read_cpuid(ID_ISAR6_EL1); + info->reg_id_mmfr0 = read_cpuid(ID_MMFR0_EL1); + info->reg_id_mmfr1 = read_cpuid(ID_MMFR1_EL1); + info->reg_id_mmfr2 = read_cpuid(ID_MMFR2_EL1); + info->reg_id_mmfr3 = read_cpuid(ID_MMFR3_EL1); + info->reg_id_mmfr4 = read_cpuid(ID_MMFR4_EL1); + info->reg_id_mmfr5 = read_cpuid(ID_MMFR5_EL1); + info->reg_id_pfr0 = read_cpuid(ID_PFR0_EL1); + info->reg_id_pfr1 = read_cpuid(ID_PFR1_EL1); + info->reg_id_pfr2 = read_cpuid(ID_PFR2_EL1); + + info->reg_mvfr0 = read_cpuid(MVFR0_EL1); + info->reg_mvfr1 = read_cpuid(MVFR1_EL1); + info->reg_mvfr2 = read_cpuid(MVFR2_EL1); +} + static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info) { info->reg_cntfrq = arch_timer_get_cntfrq(); @@ -371,31 +397,8 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info) info->reg_id_aa64pfr1 = read_cpuid(ID_AA64PFR1_EL1); info->reg_id_aa64zfr0 = read_cpuid(ID_AA64ZFR0_EL1); - /* Update the 32bit ID registers only if AArch32 is implemented */ - if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) { - info->reg_id_dfr0 = read_cpuid(ID_DFR0_EL1); - info->reg_id_dfr1 = read_cpuid(ID_DFR1_EL1); - info->reg_id_isar0 = read_cpuid(ID_ISAR0_EL1); - info->reg_id_isar1 = read_cpuid(ID_ISAR1_EL1); - info->reg_id_isar2 = read_cpuid(ID_ISAR2_EL1); - info->reg_id_isar3 = read_cpuid(ID_ISAR3_EL1); - info->reg_id_isar4 = read_cpuid(ID_ISAR4_EL1); - info->reg_id_isar5 = read_cpuid(ID_ISAR5_EL1); - info->reg_id_isar6 = read_cpuid(ID_ISAR6_EL1); - info->reg_id_mmfr0 = read_cpuid(ID_MMFR0_EL1); - info->reg_id_mmfr1 = read_cpuid(ID_MMFR1_EL1); - info->reg_id_mmfr2 = read_cpuid(ID_MMFR2_EL1); - info->reg_id_mmfr3 = read_cpuid(ID_MMFR3_EL1); - info->reg_id_mmfr4 = read_cpuid(ID_MMFR4_EL1); - info->reg_id_mmfr5 = read_cpuid(ID_MMFR5_EL1); - info->reg_id_pfr0 = read_cpuid(ID_PFR0_EL1); - info->reg_id_pfr1 = read_cpuid(ID_PFR1_EL1); - info->reg_id_pfr2 = read_cpuid(ID_PFR2_EL1); - - info->reg_mvfr0 = read_cpuid(MVFR0_EL1); - info->reg_mvfr1 = read_cpuid(MVFR1_EL1); - info->reg_mvfr2 = read_cpuid(MVFR2_EL1); - } + if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) + __cpuinfo_store_cpu_32bit(&info->aarch32); if (IS_ENABLED(CONFIG_ARM64_SVE) && id_aa64pfr0_sve(info->reg_id_aa64pfr0)) From patchwork Tue May 18 09:47:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E717C433ED for ; Tue, 18 May 2021 09:52:52 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 87B8B610A1 for ; Tue, 18 May 2021 09:52:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 87B8B610A1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=qPhbCRELTwz63Dc7AbjtFC+sFCga5EJXvaPWnc/wfiM=; b=mHU71hyI1cdfHWxVavryidmwe X3tXEBy5f6UpYZ46lxFAvvrgA3R91V7tUfk8d43FugXe33/C9ig3eGdvYzmYASx5OmleJPe5grEZ8 7tjCbltpS2P4ErgTm3mJgAbmLuT90O1cwP7faRTNUOEZoT1RUSHr+fwweK9uFM7JCc1uRh1pfD4aY 86wtkhTefCe9cj+gD6L/lntYs/eJlN4J9a67mIutHsBRePesYPpTJJM0itbgn7Mb4J5tOgS6Hk8Yz o0YtKXfFImaJswcMdJgaiMdfK/FbbcHT/xf+BjJM9z/SWPedp2ACjLoo3VcZburLWAawYt0Z8Slwb hXOaJBnZw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwNE-000EWm-DY; Tue, 18 May 2021 09:50:49 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKK-000DnI-9q for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:47:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=QCA1vWhh8X1TKt6sG9kjzyc2zuCAjEREYNzPN/VM6Kg=; b=lp8RDkbIihE5CtcCCBhFjL2wPl Fgt1OGkYeGRJ6D8yQQipWrhyJCtytm7nD///mh6L1RvZWmaOV4zxx1DwAL/V6VT8XcXgbvAi6NY1D PBHOwdnvQoIdZxfVfU8uRFj9IrwxRk0iF1Svq22xW3R6khxoFvMoMUP/LoYqSZLU22m2+1CuModVQ fbtQH2AMJWVtjl1LdXSabsZBNz5oIgtQs4PTLhQCquEntfip2eHQlMthJvPT3pkB72grsp9X/mHkd Wdtzsuk/xv+iH+r8urh5fg+SnH5RHyifCJUN8hM7NmBoZ5IBBXQYy6P2FPeEXGcvKQbxv2MM8D0ge m/LT12Cg==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKH-00EWa9-6w for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:47:47 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id BD116613AE; Tue, 18 May 2021 09:47:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331264; bh=V6IT4Uwp4gdlW8G/j6emkgNX4YIaIlSDqdG7WubTP4Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DhCL5j847ycJqdF280cSaRPsUnPhwczG91CdOA2D/0j2VX+2GeK8K6laPwEue4EpX J2AN7RuPJceyLgIkivtTuw024KdTwQFOIJYHhj4VIKFC7abwrnjBsubXqwDYKif1RJ ttEsVah9Ve09lzXS34l+4BzYmEABsIxYZEAs5R2qnG2jE65pSWSs5StQnX7deg2EXj MPbvkLqEXckrqCs6uWM8P31kpm/k41C1yxJbcn7VGneyVf41xR5etdy8/wo2RBCtme wd5H0ajsOhntJRY50Sh3GKCs8P3DtnDRsXScoYrn57jbE+fmYb5Z2RVWJXNzXJi2vc c4PSdhXAwWufw== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 02/21] arm64: Allow mismatched 32-bit EL0 support Date: Tue, 18 May 2021 10:47:06 +0100 Message-Id: <20210518094725.7701-3-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024745_317441_276A3581 X-CRM114-Status: GOOD ( 22.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When confronted with a mixture of CPUs, some of which support 32-bit applications and others which don't, we quite sensibly treat the system as 64-bit only for userspace and prevent execve() of 32-bit binaries. Unfortunately, some crazy folks have decided to build systems like this with the intention of running 32-bit applications, so relax our sanitisation logic to continue to advertise 32-bit support to userspace on these systems and track the real 32-bit capable cores in a cpumask instead. For now, the default behaviour remains but will be tied to a command-line option in a later patch. Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/cpucaps.h | 3 +- arch/arm64/include/asm/cpufeature.h | 8 +- arch/arm64/kernel/cpufeature.c | 114 ++++++++++++++++++++++++---- 3 files changed, 110 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index b0c5eda0498f..b87461490977 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -18,7 +18,8 @@ #define ARM64_HAS_NO_HW_PREFETCH 8 #define ARM64_HAS_VIRT_HOST_EXTN 11 #define ARM64_WORKAROUND_CAVIUM_27456 12 -#define ARM64_HAS_32BIT_EL0 13 +/* Unreliable: use system_supports_32bit_el0() instead. */ +#define ARM64_HAS_32BIT_EL0_DO_NOT_USE 13 #define ARM64_SPECTRE_V3A 14 #define ARM64_HAS_CNP 15 #define ARM64_HAS_NO_FPSIMD 16 diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 338840c00e8e..603bf4160cd6 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -630,9 +630,15 @@ static inline bool cpu_supports_mixed_endian_el0(void) return id_aa64mmfr0_mixed_endian_el0(read_cpuid(ID_AA64MMFR0_EL1)); } +const struct cpumask *system_32bit_el0_cpumask(void); +DECLARE_STATIC_KEY_FALSE(arm64_mismatched_32bit_el0); + static inline bool system_supports_32bit_el0(void) { - return cpus_have_const_cap(ARM64_HAS_32BIT_EL0); + u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1); + + return static_branch_unlikely(&arm64_mismatched_32bit_el0) || + id_aa64pfr0_32bit_el0(pfr0); } static inline bool system_supports_4kb_granule(void) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index a4db25cd7122..4194a47de62d 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -107,6 +107,24 @@ DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE); bool arm64_use_ng_mappings = false; EXPORT_SYMBOL(arm64_use_ng_mappings); +/* + * Permit PER_LINUX32 and execve() of 32-bit binaries even if not all CPUs + * support it? + */ +static bool __read_mostly allow_mismatched_32bit_el0; + +/* + * Static branch enabled only if allow_mismatched_32bit_el0 is set and we have + * seen at least one CPU capable of 32-bit EL0. + */ +DEFINE_STATIC_KEY_FALSE(arm64_mismatched_32bit_el0); + +/* + * Mask of CPUs supporting 32-bit EL0. + * Only valid if arm64_mismatched_32bit_el0 is enabled. + */ +static cpumask_var_t cpu_32bit_el0_mask __cpumask_var_read_mostly; + /* * Flag to indicate if we have computed the system wide * capabilities based on the boot time active CPUs. This @@ -767,7 +785,7 @@ static void __init sort_ftr_regs(void) * Any bits that are not covered by an arm64_ftr_bits entry are considered * RES0 for the system-wide value, and must strictly match. */ -static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new) +static void init_cpu_ftr_reg(u32 sys_reg, u64 new) { u64 val = 0; u64 strict_mask = ~0x0ULL; @@ -863,7 +881,7 @@ static void __init init_cpu_hwcaps_indirect_list(void) static void __init setup_boot_cpu_capabilities(void); -static void __init init_32bit_cpu_features(struct cpuinfo_32bit *info) +static void init_32bit_cpu_features(struct cpuinfo_32bit *info) { init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0); init_cpu_ftr_reg(SYS_ID_DFR1_EL1, info->reg_id_dfr1); @@ -979,6 +997,22 @@ static void relax_cpu_ftr_reg(u32 sys_id, int field) WARN_ON(!ftrp->width); } +static void update_mismatched_32bit_el0_cpu_features(struct cpuinfo_arm64 *info, + struct cpuinfo_arm64 *boot) +{ + static bool boot_cpu_32bit_regs_overridden = false; + + if (!allow_mismatched_32bit_el0 || boot_cpu_32bit_regs_overridden) + return; + + if (id_aa64pfr0_32bit_el0(boot->reg_id_aa64pfr0)) + return; + + boot->aarch32 = info->aarch32; + init_32bit_cpu_features(&boot->aarch32); + boot_cpu_32bit_regs_overridden = true; +} + static int update_32bit_cpu_features(int cpu, struct cpuinfo_32bit *info, struct cpuinfo_32bit *boot) { @@ -1139,6 +1173,7 @@ void update_cpu_features(int cpu, * (e.g. SYS_ID_AA64PFR0_EL1), so we call it last. */ if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) { + update_mismatched_32bit_el0_cpu_features(info, boot); taint |= update_32bit_cpu_features(cpu, &info->aarch32, &boot->aarch32); } @@ -1251,6 +1286,28 @@ has_cpuid_feature(const struct arm64_cpu_capabilities *entry, int scope) return feature_matches(val, entry); } +const struct cpumask *system_32bit_el0_cpumask(void) +{ + if (!system_supports_32bit_el0()) + return cpu_none_mask; + + if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) + return cpu_32bit_el0_mask; + + return cpu_possible_mask; +} + +static bool has_32bit_el0(const struct arm64_cpu_capabilities *entry, int scope) +{ + if (!has_cpuid_feature(entry, scope)) + return allow_mismatched_32bit_el0; + + if (scope == SCOPE_SYSTEM) + pr_info("detected: 32-bit EL0 Support\n"); + + return true; +} + static bool has_useable_gicv3_cpuif(const struct arm64_cpu_capabilities *entry, int scope) { bool has_sre; @@ -1869,10 +1926,9 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpu_enable = cpu_copy_el2regs, }, { - .desc = "32-bit EL0 Support", - .capability = ARM64_HAS_32BIT_EL0, + .capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE, .type = ARM64_CPUCAP_SYSTEM_FEATURE, - .matches = has_cpuid_feature, + .matches = has_32bit_el0, .sys_reg = SYS_ID_AA64PFR0_EL1, .sign = FTR_UNSIGNED, .field_pos = ID_AA64PFR0_EL0_SHIFT, @@ -2381,7 +2437,7 @@ static const struct arm64_cpu_capabilities compat_elf_hwcaps[] = { {}, }; -static void __init cap_set_elf_hwcap(const struct arm64_cpu_capabilities *cap) +static void cap_set_elf_hwcap(const struct arm64_cpu_capabilities *cap) { switch (cap->hwcap_type) { case CAP_HWCAP: @@ -2426,7 +2482,7 @@ static bool cpus_have_elf_hwcap(const struct arm64_cpu_capabilities *cap) return rc; } -static void __init setup_elf_hwcaps(const struct arm64_cpu_capabilities *hwcaps) +static void setup_elf_hwcaps(const struct arm64_cpu_capabilities *hwcaps) { /* We support emulation of accesses to CPU ID feature registers */ cpu_set_named_feature(CPUID); @@ -2601,7 +2657,7 @@ static void check_early_cpu_features(void) } static void -verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps) +__verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps) { for (; caps->matches; caps++) @@ -2612,6 +2668,14 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps) } } +static void verify_local_elf_hwcaps(void) +{ + __verify_local_elf_hwcaps(arm64_elf_hwcaps); + + if (id_aa64pfr0_32bit_el0(read_cpuid(ID_AA64PFR0_EL1))) + __verify_local_elf_hwcaps(compat_elf_hwcaps); +} + static void verify_sve_features(void) { u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1); @@ -2676,11 +2740,7 @@ static void verify_local_cpu_capabilities(void) * on all secondary CPUs. */ verify_local_cpu_caps(SCOPE_ALL & ~SCOPE_BOOT_CPU); - - verify_local_elf_hwcaps(arm64_elf_hwcaps); - - if (system_supports_32bit_el0()) - verify_local_elf_hwcaps(compat_elf_hwcaps); + verify_local_elf_hwcaps(); if (system_supports_sve()) verify_sve_features(); @@ -2815,6 +2875,34 @@ void __init setup_cpu_features(void) ARCH_DMA_MINALIGN); } +static int enable_mismatched_32bit_el0(unsigned int cpu) +{ + struct cpuinfo_arm64 *info = &per_cpu(cpu_data, cpu); + bool cpu_32bit = id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0); + + if (cpu_32bit) { + cpumask_set_cpu(cpu, cpu_32bit_el0_mask); + static_branch_enable_cpuslocked(&arm64_mismatched_32bit_el0); + setup_elf_hwcaps(compat_elf_hwcaps); + } + + return 0; +} + +static int __init init_32bit_el0_mask(void) +{ + if (!allow_mismatched_32bit_el0) + return 0; + + if (!zalloc_cpumask_var(&cpu_32bit_el0_mask, GFP_KERNEL)) + return -ENOMEM; + + return cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, + "arm64/mismatched_32bit_el0:online", + enable_mismatched_32bit_el0, NULL); +} +subsys_initcall_sync(init_32bit_el0_mask); + static void __maybe_unused cpu_enable_cnp(struct arm64_cpu_capabilities const *cap) { cpu_replace_ttbr1(lm_alias(swapper_pg_dir)); From patchwork Tue May 18 09:47:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61526C433ED for ; Tue, 18 May 2021 09:53:17 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1EFD460BD3 for ; Tue, 18 May 2021 09:53:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1EFD460BD3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ccqB1JYRdvdsO1e+9XA27cX+ZVZIKcKyJ+nKdUtQfLA=; b=Xr5+1pjw7gFtf0cjZY491A7/P cz0iLz1H8fUSiKFZ2glv5OcFKc1D1hg3yiyan2Q+k65yh16SS+gRopIrmnKCXWsSteiPiyIoVMMnt eR0bdV7Ejko4VRAA3qtqyiyjxnJs4pPnnlceyK57KXgxpSb2FYVNI+jCHyu6/6VAfJQwef6pU3aBY ptUC/j70Cd2FAnlh5mJXNpc3OQML1lLn/6kmhT07cYpteGC6RTjfxFP2zyEBygs1UPoV9HPf1KePJ oyZ6R3icYB844qFxmBuA6q3NVU4TFeM/MbhXIpNvSF5mIHoawMU588/8q1B2ABj5/OB7LMMAbyuYZ nx77CZ3bQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwO5-000Ej3-2N; Tue, 18 May 2021 09:51:41 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKO-000DoH-Bn for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:47:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=is8gJR47maHbQv14/EL+XcM6Kns0E/nUDmBQUn7eGDs=; b=DTs2+NZecK2Za/c9PxhMLfnXQB DeUEfbSt7T9nwfgb4FOPHZwbpsCKRCslzWt3tHwwNioXdf4/PI0qVlO3c6jWQwbq4w2Mtz17RVbt5 9m1HEoOLU7IgsyUqJAbSt/ybJ6bRcRlK/KS8erqYwi9EyDArrMJbpdvtRi4OSe1DyouBFbRPNCjN+ XQ6VvSeoFxHFJJgSm/N/7l7qBWQTMwFlQMaMOL4i+TflqDycqyj2pP5T90RbWiyfZqjC5HuZ3+T2p mkDOGjFE95aQrmG3V8sqZ21bTfL3PePEPu5uoIImMUSd7cYNadXELDjeovFxYM6OUIzXQvlsmDpVo lDn8wZBg==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKK-00EWb3-Nm for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:47:51 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 53BD8613B9; Tue, 18 May 2021 09:47:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331268; bh=NS4QEgIKSv/TTYjJalSTuvXiLJYF+wZzDqr0D/nk7gg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A+Hero/Z/jjMxzs9M4fBC2NACqp7Ij5irKXXGRtGUOXUYFIRki+CVYeQbVZaLKVPc yOEKw+nrm5ZqFTHkA1qeR4XD2WX0C3ZAqtt9j5gBazPzls9Di6lKR6zY6FTZIQXdM2 +ubc1/vbB16Jvy956HahOqZzZ8+H/yOck3ft7bqfCrTA4Gs4KSnYYknMWFm8DdOuXN QvG0HZufgLu51uqSRFHk9bBKoybZWGM1+Er0yIXExGS15jD38M4MobmDWGjrhl95Bv qfjhKibJ1tQ8ALu0N07p0mt4sZdac9VptqK5O9zu5lfNdaGYB62jVJxc+NZbbdgLlD WYu+m9QNRJLMw== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 03/21] KVM: arm64: Kill 32-bit vCPUs on systems with mismatched EL0 support Date: Tue, 18 May 2021 10:47:07 +0100 Message-Id: <20210518094725.7701-4-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024748_830077_877E2A19 X-CRM114-Status: GOOD ( 12.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If a vCPU is caught running 32-bit code on a system with mismatched support at EL0, then we should kill it. Acked-by: Marc Zyngier Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- arch/arm64/kvm/arm.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 1cb39c0803a4..5bdba97a7654 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -692,6 +692,15 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) } } +static bool vcpu_mode_is_bad_32bit(struct kvm_vcpu *vcpu) +{ + if (likely(!vcpu_mode_is_32bit(vcpu))) + return false; + + return !system_supports_32bit_el0() || + static_branch_unlikely(&arm64_mismatched_32bit_el0); +} + /** * kvm_arch_vcpu_ioctl_run - the main VCPU run function to execute guest code * @vcpu: The VCPU pointer @@ -875,7 +884,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) * with the asymmetric AArch32 case), return to userspace with * a fatal error. */ - if (!system_supports_32bit_el0() && vcpu_mode_is_32bit(vcpu)) { + if (vcpu_mode_is_bad_32bit(vcpu)) { /* * As we have caught the guest red-handed, decide that * it isn't fit for purpose anymore by making the vcpu From patchwork Tue May 18 09:47:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264411 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AC9DC433ED for ; Tue, 18 May 2021 09:54:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 00DD0610A1 for ; Tue, 18 May 2021 09:54:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 00DD0610A1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VrpcspceiMMEPL6Cjtj8vPK7m7Yrm/PPDlaruPvlyJ4=; b=nHiazr2ucZEnGU4Mi89TbYSXo 75IakMyaTRWsOak++5wurRMudUQ75VW/tZqk4a//ypJQbz6Qcdrs4092fZ+qPNyVI7Y5VhuAwXUEp ZatwN+ctErgMAxvPV33plxAjuirw7fBQqrc+5bAg1TMmhbfYcEYRuw+GKchYCaaypGB+jxtC5qFm9 fVsBDJpwGBCG0RKtrJtvqzy8pu/1OSRefecBULEBl2TKi8iq5njcVbT73NWyIujd4Ldj5fmcNLix4 sr1lNYW7kE8kppHQgtGFDZtCwkM67kFOuqT0gNJYV9UMp0asXqc0PnsCM+RvLYPQO8+wvAHt7E1H9 u97tcRJiw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwP3-000EvR-Co; Tue, 18 May 2021 09:52:42 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKR-000Doy-M3 for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:47:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=H+gluzK/c87K8IKp3ObwtqZ2D5iBrtAAx4x/MZ/PQPw=; b=g3vcUR1gSxFuxhM/Fido6rL9B4 hgy7X1MA0ZRnPgGappQKWSehORa+pVPP2FqIu89WJxFXsOIXffBnJPsG2K8e76ajGbS2UqbxwpJVl 33DpL64qlFx4HKqYxH4bXzxj8oDs4TAMowW9m/DIKjEJR0iAW/3bWh5B7If0/39Y0Ubwjw17VIwgt 5sRgB5gAcX3hrFCLJEtElPuvAy09ES6hq+mNztfiJc+x4cgckOBMohZur711EIOnWX18GAPuhC+rH L+Rsmp2EuCt6ALxj4dTx7S/Epydm+NCYkYY0JdI3ujxfmNMZ2hFoZw5ODJWyaWNCkVUxHE49kuLm4 0BGtew9A==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKO-00EWcY-B6 for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:47:54 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id E073B61396; Tue, 18 May 2021 09:47:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331272; bh=RTAwS1Pb2T4YH9RTWzE3GXmCSLarrFRCCZxfeC0IjGo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GWOXp+zS1a58Dta3KChjDG5CJSAxD2/cHljq4pP1pAYtJd7zAhjfF9NQE9x23csXy N8MBfE/duY99QsBbvPB2PlmyTVMH0XJI57ErJ0MU+EzCWNTRbdbBbqGp8HTT64W1LL gYdiyfNbLozlC37iqdcQtttiQpriKoprIdRxSFlKaVoLc35LortpU/SKFa2vno1R4V +bI1Jv4qPJlwXn8SRZXTKI6Z8NkGgcT3UDV1BatVuWJOE0FrVwH0DaHmISPQVg3pHy xoApFmXHckl+UbvFmk5RMZhI9EIzE+KyTrPOY6Va37bK3ZKU42ebAtliYmWsgwtIRl pOzJM3y2TiX0w== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 04/21] arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs Date: Tue, 18 May 2021 10:47:08 +0100 Message-Id: <20210518094725.7701-5-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024752_444883_091B4ADD X-CRM114-Status: GOOD ( 17.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Scheduling a 32-bit application on a 64-bit-only CPU is a bad idea. Ensure that 32-bit applications always take the slow-path when returning to userspace on a system with mismatched support at EL0, so that we can avoid trying to run on a 64-bit-only CPU and force a SIGKILL instead. Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- arch/arm64/kernel/process.c | 19 ++++++++++++++++++- arch/arm64/kernel/signal.c | 26 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index b4bb67f17a2c..f4a91bf1ce0c 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -527,6 +527,15 @@ static void erratum_1418040_thread_switch(struct task_struct *prev, write_sysreg(val, cntkctl_el1); } +static void compat_thread_switch(struct task_struct *next) +{ + if (!is_compat_thread(task_thread_info(next))) + return; + + if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) + set_tsk_thread_flag(next, TIF_NOTIFY_RESUME); +} + static void update_sctlr_el1(u64 sctlr) { /* @@ -568,6 +577,7 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev, ssbs_thread_switch(next); erratum_1418040_thread_switch(prev, next); ptrauth_thread_switch_user(next); + compat_thread_switch(next); /* * Complete any pending TLB or cache maintenance on this CPU in case @@ -633,8 +643,15 @@ unsigned long arch_align_stack(unsigned long sp) */ void arch_setup_new_exec(void) { - current->mm->context.flags = is_compat_task() ? MMCF_AARCH32 : 0; + unsigned long mmflags = 0; + + if (is_compat_task()) { + mmflags = MMCF_AARCH32; + if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) + set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); + } + current->mm->context.flags = mmflags; ptrauth_thread_init_user(); mte_thread_init_user(); diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 6237486ff6bb..f8192f4ae0b8 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -911,6 +911,19 @@ static void do_signal(struct pt_regs *regs) restore_saved_sigmask(); } +static bool cpu_affinity_invalid(struct pt_regs *regs) +{ + if (!compat_user_mode(regs)) + return false; + + /* + * We're preemptible, but a reschedule will cause us to check the + * affinity again. + */ + return !cpumask_test_cpu(raw_smp_processor_id(), + system_32bit_el0_cpumask()); +} + asmlinkage void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags) { @@ -938,6 +951,19 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, if (thread_flags & _TIF_NOTIFY_RESUME) { tracehook_notify_resume(regs); rseq_handle_notify_resume(NULL, regs); + + /* + * If we reschedule after checking the affinity + * then we must ensure that TIF_NOTIFY_RESUME + * is set so that we check the affinity again. + * Since tracehook_notify_resume() clears the + * flag, ensure that the compiler doesn't move + * it after the affinity check. + */ + barrier(); + + if (cpu_affinity_invalid(regs)) + force_sig(SIGKILL); } if (thread_flags & _TIF_FOREIGN_FPSTATE) From patchwork Tue May 18 09:47:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264413 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84AD4C433B4 for ; Tue, 18 May 2021 09:55:35 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0D7D161185 for ; Tue, 18 May 2021 09:55:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D7D161185 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=D59imcGuxsENzuS3Mi+V1o7LMV2L8Pi+tcP5eQW1XjM=; b=XGM7w8Okia70mm026cmcvpJAh K6UbOsV65h6olcYf57Ra1Lva1hsgNxQwfKItvGbVXeqXb9ay2rLA5Y+bQV4A8m3fJTMjHAQeddnV0 Kht1/XFSpTU5xNtuTetYzM0M5ALwRIxTBWUpV/8ds3O+VBl01+F7/dzaIrElcVDX1fvJuuXMuItNl f172BvuAl0qVkILySbY5AwWAjpk8Bm4YMUExj4BnG06nhMlquN0/1RcAg/jGolYW+zbfrNevcH6hj jZgntr3tVTs/vEEipOXsv+wE+OFkJU7sJX+khaDVWGamkM68ukzgC+I5gevKkIS89uoMEX7q5MV5p gfGHDuGqA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwPv-000F60-GW; Tue, 18 May 2021 09:53:36 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKU-000Dri-Gf for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:47:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=92rxMjpe7Z887tAwC2LVTLvwNh4rev7lxvrCntKF1Vw=; b=0sWGYcS5YLeZCANI15lFd7mtlU tMBJF8afBjQVskjexuKL4MlMF8BVNFhAjSf6vRcZbUcpPMuQbXOOfSqDqed6rGfJNYt74nI+WfS5L taYgA0c6NQ2R2qRlyydJNokKVTkVdvlT/XQUmw8syCrTFCzDL01DQ7dj6FILkE8o3FhZV1q7ST3xF yGnrqTSr/S75PU8+Tch1gPuVao/Y2ksNI+F2w8N0aP1qIOgMd4tqC51a0Gn1D0MN2OlGfuxGNapi2 U+sm9Q8Oe0xxx7JohEMbHHFQmGeEa3qVykslhGC7kOOT6IZFykoM4q4nwKbGNZMrcO91EZ/RlRnRo YqC/tNOQ==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKR-00EWdT-Sh for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:47:57 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 7460E613C8; Tue, 18 May 2021 09:47:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331275; bh=k5KbCUN0F2YlNMUf2xfvaa60/l+3klPfLthUfAzmzGA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IUh4fN2yCR5HduaVygauxlNzp5jizxbUnEaUdzEo1k9g8pVcD0mg1gChO3grPPfS3 7xyJoUy1xX6PebVXgM4JgUFgWwbAR+anDJagqO2iXeGou2OKQ5kR87kAzk4O8PuuAt iDTmVXug7o5AbFjurqVgN3ohJ8Zr1Eyre91fQHrLA8dLl/l0kNlqHoZlIOQphMyojX 6YfAah9AqPPsc/KPZmu+UrPAfhXnHaWaYnTp7GApF6LJ3UxUTqiFzEstcqv5njR41E 91bMYM2M/OxQGfy3u6X12L7dw65enYb7fICh1NpBrx9wUnZQAk3rEenjtS1sS9vdOH 2vQ19b/57LRfQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 05/21] arm64: Advertise CPUs capable of running 32-bit applications in sysfs Date: Tue, 18 May 2021 10:47:09 +0100 Message-Id: <20210518094725.7701-6-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024755_986385_B68342E2 X-CRM114-Status: GOOD ( 12.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since 32-bit applications will be killed if they are caught trying to execute on a 64-bit-only CPU in a mismatched system, advertise the set of 32-bit capable CPUs to userspace in sysfs. Reviewed-by: Greg Kroah-Hartman Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- .../ABI/testing/sysfs-devices-system-cpu | 9 +++++++++ arch/arm64/kernel/cpufeature.c | 19 +++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu index fe13baa53c59..899377b2715a 100644 --- a/Documentation/ABI/testing/sysfs-devices-system-cpu +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu @@ -494,6 +494,15 @@ Description: AArch64 CPU registers 'identification' directory exposes the CPU ID registers for identifying model and revision of the CPU. +What: /sys/devices/system/cpu/aarch32_el0 +Date: May 2021 +Contact: Linux ARM Kernel Mailing list +Description: Identifies the subset of CPUs in the system that can execute + AArch32 (32-bit ARM) applications. If present, the same format as + /sys/devices/system/cpu/{offline,online,possible,present} is used. + If absent, then all or none of the CPUs can execute AArch32 + applications and execve() will behave accordingly. + What: /sys/devices/system/cpu/cpu#/cpu_capacity Date: December 2016 Contact: Linux kernel mailing list diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 4194a47de62d..959442f76ed7 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -67,6 +67,7 @@ #include #include #include +#include #include #include #include @@ -1297,6 +1298,24 @@ const struct cpumask *system_32bit_el0_cpumask(void) return cpu_possible_mask; } +static ssize_t aarch32_el0_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + const struct cpumask *mask = system_32bit_el0_cpumask(); + + return sysfs_emit(buf, "%*pbl\n", cpumask_pr_args(mask)); +} +static const DEVICE_ATTR_RO(aarch32_el0); + +static int __init aarch32_el0_sysfs_init(void) +{ + if (!allow_mismatched_32bit_el0) + return 0; + + return device_create_file(cpu_subsys.dev_root, &dev_attr_aarch32_el0); +} +device_initcall(aarch32_el0_sysfs_init); + static bool has_32bit_el0(const struct arm64_cpu_capabilities *entry, int scope) { if (!has_cpuid_feature(entry, scope)) From patchwork Tue May 18 09:47:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3786C433ED for ; Tue, 18 May 2021 09:56:38 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3EF4E6024A for ; Tue, 18 May 2021 09:56:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3EF4E6024A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=atrW2wmseOV4HvLT3JFnuGknE2oI062xjcc5s0+slJo=; b=Eb/ACntJDQDRGXMlCPqLs6aUz itRBPOy3e57wuxYt8PqFQuqMqiM+7TEtoVPz5QhZaz0jzKlrMJRs4EL3n7vZjurfeq5ghMBtu42kz YONH8P84g9o8tWoi7Nd7KwodO1Qmy0R52bR7vdjIQumX7rrXagi76RrBu/TkagLYz9J/B6NqZQF6F buIpqqT6Mt2eRh1Tph26mRbAKyJlfJ+AIxyPaeOzbEGAwvQqNmfp4GF75oZIMj43G3JEa0JMydU9t IqLskK0YHV1RleKoyEQSq8aoHOSuol/vv9NHbU4Dz1yAhUIRt2bWrv+RHvKiOOoyd6KEw4aznwazM WkCSNM6eg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwQq-000FFo-I6; Tue, 18 May 2021 09:54:32 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKY-000Dsf-6M for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=iVMYSKzq3hfG2og4eSIkeTuf05Jn8RhTmkKMBnY1U0c=; b=u6lyPoPFmjXEhguns3vO+VoGax zI1apoD4u8hiTMX6hh9bgwQ4pDTLCopMYMKXvYjg2D7sfd9aHiKsETAWDTmNjSGhglvBBuD3KzOvm er3X2U2H61ZLXOjfDPsye0iJA/23aWF4qaWNjbN3nnkorGXyBV8jyfdGI/Xua5VUXig5UTjoFXReM Y2fRmc2Xa/nvXs5Ap85qpcB2osbk5abUK19Dpppi17/k+n9bzRuhtrT8r+hMV4UwAD44ByuC5VAzE ah4DxGLbCIjYavUoUNA1ZgR7UgPQRHonF6XphUGMJ5jiU+uKUKnKKkpAgkrAgpkcj5acKQsDzG7oC +rXWS8aA==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKV-00EWeb-FU for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:00 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0C391613CE; Tue, 18 May 2021 09:47:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331279; bh=EV9JgCxKjLj4KVtD7cKw871l26IXkBBXbX6fJNN8AkA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QqN0MSxlEWysNMl3QzC9VV1pquDbqXqjrrvGQioYU6YK/JDHtOJmFhEn4e6Yz9pPn krlxtnuEtmgB7W/xdaMVwHhPp2EFjsqVSWh9KzZJ7ixKZXtz7sprabF5LJuvT9aLqm zXlIbJNaSAqN05og4Uev+SmBGglmP6/ifaR/Z6/0mv41joOk+ZyJiyT7KJa5hUsNCh FHDpi7QSMs4GSKVLwtNkXznYWi/PAcTZ2KtJ3uIhrR+j/7yMp/3SZ/9Z7zAoJu+MBE fQ4YekLvZPq0FsLtAyGntl6Ci5qj3Dyw6dCOD1RltEn7LhmiK97uvujTMP6B98bhvf IXxQ9YVOktUPg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 06/21] sched: Introduce task_cpu_possible_mask() to limit fallback rq selection Date: Tue, 18 May 2021 10:47:10 +0100 Message-Id: <20210518094725.7701-7-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024759_577770_BCA9BAA6 X-CRM114-Status: GOOD ( 16.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. On such a system, we must take care not to migrate a task to an unsupported CPU when forcefully moving tasks in select_fallback_rq() in response to a CPU hot-unplug operation. Introduce a task_cpu_possible_mask() hook which, given a task argument, allows an architecture to return a cpumask of CPUs that are capable of executing that task. The default implementation returns the cpu_possible_mask, since sane machines do not suffer from per-cpu ISA limitations that affect scheduling. The new mask is used when selecting the fallback runqueue as a last resort before forcing a migration to the first active CPU. Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- include/linux/mmu_context.h | 8 ++++++++ kernel/sched/core.c | 10 ++++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/include/linux/mmu_context.h b/include/linux/mmu_context.h index 03dee12d2b61..bc4ac3c525e6 100644 --- a/include/linux/mmu_context.h +++ b/include/linux/mmu_context.h @@ -14,4 +14,12 @@ static inline void leave_mm(int cpu) { } #endif +/* + * CPUs that are capable of running task @p. By default, we assume a sane, + * homogeneous system. Must contain at least one active CPU. + */ +#ifndef task_cpu_possible_mask +# define task_cpu_possible_mask(p) cpu_possible_mask +#endif + #endif diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 5226cc26a095..482f7fdca0e8 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1813,8 +1813,11 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu) return cpu_online(cpu); /* Non kernel threads are not allowed during either online or offline. */ - if (!(p->flags & PF_KTHREAD)) - return cpu_active(cpu); + if (!(p->flags & PF_KTHREAD)) { + if (cpu_active(cpu)) + return cpumask_test_cpu(cpu, task_cpu_possible_mask(p)); + return false; + } /* KTHREAD_IS_PER_CPU is always allowed. */ if (kthread_is_per_cpu(p)) @@ -2792,10 +2795,9 @@ static int select_fallback_rq(int cpu, struct task_struct *p) * * More yuck to audit. */ - do_set_cpus_allowed(p, cpu_possible_mask); + do_set_cpus_allowed(p, task_cpu_possible_mask(p)); state = fail; break; - case fail: BUG(); break; From patchwork Tue May 18 09:47:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD6DDC433B4 for ; Tue, 18 May 2021 09:57:40 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 406D66024A for ; Tue, 18 May 2021 09:57:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 406D66024A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RxLgwrfDN0aBzWnWcuZ7b4f7XW30PYrf/VHckiELBfI=; b=bFhLlODlip6oIyRsgC6gDlv2Q n9P2kQ74piQvE/873P+WMBo7anCnnB6SFDZtZf7J3+2+SO5anVN/yZ25vdQRph77+ei2w+4MAwVB2 YTLu8EUbQ5diLypqh/sY1HwxHcLBdE5dk69yVTAyHZxqsvEHpDRZH3511uQGQLze4z3jKrQnEDTGA Ih+qSm6LFySz4b8JP9BCpboi0mah4jqrqkZhPCQ8LrXjKGoQsOoGxjHjlHnhmRd8f6AOcyjD9Vpxn H4C02ETcDXFkr9XmAp2cjyz8VTkkme9KmcDNSSkDQfECjfBcjsaEo4Y5Mqdm+v2ipDEEkONk2sof2 eH58JSOHQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwRs-000FRt-6d; Tue, 18 May 2021 09:55:36 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKb-000DtG-Hx for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=8pfE3GGF8GP/H9dVMEMQaTGttX4ip0gEojsKCJpvGoU=; b=J+6qOgR1xA8tHqTTJHA4qZpL4L Tzcv+c9mVlTe/lRNQofloKiPj5LbFAeJhKTncYoigLJXDkPdP+C/F3LlyIDI9Bl8rIKlm30ZD2sDR IC+J3qqNZR87Jqe9VYfqE4udaAn6Upf3Vfj8AmtFUVPtWRdyXxsTU5Nj7gJPj8nRlNBhWwhd+zSPS CKIzjhcvIbtmW51aUIuJAhtLB07veFw0xuvWH4LKiFz2HNDuHLQKBJxXzL0Pw6fTogLzBjvLs/1zC YmeG3KLtMJFRYFW0Ipf0U03Mj1BgNNhh7ryr+eMEg43LV+KDCpEye+cC8xAnggP4doXeNLF3ZJrsx yCM5cPAQ==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKZ-00EWfH-1f for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:04 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 987FB613CA; Tue, 18 May 2021 09:47:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331282; bh=NvwSaFd4j1U9x7MuiRuhuzS0QYyzwo36gVpUtBca09k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PrnyXh5dDh1IBvExWmit09/FbBGiBbuPKeY+VMUJw8xXrf5z1gD6Mq5/GTrEf2e53 Ss1XprfORyl2NnRvt2oGg1RpmHMV4Sjwk3oZA/1j83FYtPvuXxNZlmJhlMNbUSwpwP kE14LKvtyOjuJLTLLTA6KK+VT/JmdQbX/Ul4WJO7mqzTR43GDkfe+CAuBWI/8jbR6U zQsSsD5Ykng8/nOF2AZ9uIo1azDZZUcBsMqZ9GCQ6MQGB9j6WjwXcFyHtp4pO8b6Z6 LmF2ljTd4DQIiYNwlltUyhN9v/4/J+y+n+oKeZzyXYC9X8mdtWS3U1Wavdlm3prYkT 7ArvJUG2dnRDg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 07/21] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Date: Tue, 18 May 2021 10:47:11 +0100 Message-Id: <20210518094725.7701-8-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024803_126590_21376FE8 X-CRM114-Status: GOOD ( 13.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If the scheduler cannot find an allowed CPU for a task, cpuset_cpus_allowed_fallback() will widen the affinity to cpu_possible_mask if cgroup v1 is in use. In preparation for allowing architectures to provide their own fallback mask, just return early if we're either using cgroup v1 or we're using cgroup v2 with a mask that contains invalid CPUs. This will allow select_fallback_rq() to figure out the mask by itself. Cc: Li Zefan Cc: Tejun Heo Cc: Johannes Weiner Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- include/linux/cpuset.h | 1 + kernel/cgroup/cpuset.c | 12 ++++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index 04c20de66afc..ed6ec677dd6b 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #ifdef CONFIG_CPUSETS diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index a945504c0ae7..8c799260a4a2 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -3322,9 +3322,17 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) void cpuset_cpus_allowed_fallback(struct task_struct *tsk) { + const struct cpumask *cs_mask; + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); + rcu_read_lock(); - do_set_cpus_allowed(tsk, is_in_v2_mode() ? - task_cs(tsk)->cpus_allowed : cpu_possible_mask); + cs_mask = task_cs(tsk)->cpus_allowed; + + if (!is_in_v2_mode() || !cpumask_subset(cs_mask, possible_mask)) + goto unlock; /* select_fallback_rq will try harder */ + + do_set_cpus_allowed(tsk, cs_mask); +unlock: rcu_read_unlock(); /* From patchwork Tue May 18 09:47:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3EAAC433ED for ; Tue, 18 May 2021 09:58:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7AD1E6024A for ; Tue, 18 May 2021 09:58:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AD1E6024A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=AAHCPKBnaokUK/LEkiWVXyFWeeWrAChJGhpjTCk7isQ=; b=ihcYD+XoIPzmc0qDeXPP4uubr tuWMfJyeIvJpPbQufMEG3DPqVaLS6iCB4zKGTa1qb0eLTTrKsFq5VauS92en6e6ayQVfVPzCwMZeq Gupv6qQPuvV87VX4hmXaJTaGUMl4pZIl28Y/zSI3q+fGnlsuzjWg8WHA/IQFMLfiaJ1jJZq4BvXmv +kc5yOlwzWkocV+7HT4AeZnhNMq5iGCGDJOwRSh5t1Qxr1KsR+rBDmxPlTWcGEbPY5HM5Hudb5hxz 6uO8JlXMWhpyF7cVnNnCdMXnzKFaRvcuuG23JaQvWvPR6RJuMIZgwQRA1yQ4DSlcJq5hywJRRNmXo N30NoBcWA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwT0-000FeT-Vd; Tue, 18 May 2021 09:56:47 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKf-000DuA-El for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=JmoWdrGPJy6iEhkr+OpJss1ufIUv1Uc6WHY2TyWPXXE=; b=pIAUk/HLe++5BhgJkT7C0RVclH mnQrPfnf9XB7OYliWxPxgDVQEG/RGw8Pn3s1oZhVLMescF5nOUhrAgon39y9PVCggNVaoyK9y9xLp lV5rdQG7QZAQHv3yfabQT/J0mnFOU/etQNvGHS0xUrnby0gvvl23Q0gTLGx6p209xITGSepX1MLAs 5H4+A8ey8dqERlFaLWFCecpvwMsrvjPw1hTuSKd8WVS9lJkKrkqeCB6/366JGMCNGHjJvNdR9GfZX hQRXvyQJNkFv3PWHwliT9wsx98ZpYsTZHGbzkRu7/A2G1CE12Vbvj2exgCTXhK9T10cl1QMDrAYii BJRCZ94A==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKc-00EWg6-Km for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:08 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 358BD613DC; Tue, 18 May 2021 09:48:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331286; bh=wMv2PdJYwMEJ9tSikM8uIF6fm2qkuS94TU9x613bwLc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KMkIKGq+1e4eB6KgumfgKe9peqbooTnY+8iKzWKWLYC3kuNzEA6kyIsR2uySffZz3 lu2u0iNi5PAh0yfH3RH8UWDz8u9BI4QzNpoXaCuu5SV7328L6pN0X8wolSR4xR6oT4 6cf6BNJxS8gFEut0DYy/7+hxCVkzQRDgGw3Lx8nkPWfeuhaLycJ3B8kcZmpwnLew6N MbuwLKjmFaL07RKLkigg5My4F5jxaYpXzVj3AmBFrMrVJdv92Fra2sJtZJh7CbNFHW ag+e9wq9p1cwZZ1MXr9e9/Qh4c299ix1NUcLZuvmtFXE0dU/hhHxj2otiOXER6EhNw imuCsKOwBpYJQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 08/21] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Date: Tue, 18 May 2021 10:47:12 +0100 Message-Id: <20210518094725.7701-9-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024806_745497_710347AE X-CRM114-Status: GOOD ( 19.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. Modify guarantee_online_cpus() to take task_cpu_possible_mask() into account when trying to find a suitable set of online CPUs for a given task. This will avoid passing an invalid mask to set_cpus_allowed_ptr() during ->attach() and will subsequently allow the cpuset hierarchy to be taken into account when forcefully overriding the affinity mask for a task which requires migration to a compatible CPU. Cc: Li Zefan Cc: Tejun Heo Cc: Johannes Weiner Signed-off-by: Will Deacon --- include/linux/cpuset.h | 2 +- kernel/cgroup/cpuset.c | 33 +++++++++++++++++++-------------- 2 files changed, 20 insertions(+), 15 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index ed6ec677dd6b..414a8e694413 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -185,7 +185,7 @@ static inline void cpuset_read_unlock(void) { } static inline void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask) { - cpumask_copy(mask, cpu_possible_mask); + cpumask_copy(mask, task_cpu_possible_mask(p)); } static inline void cpuset_cpus_allowed_fallback(struct task_struct *p) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 8c799260a4a2..b532a5333ff9 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -372,18 +372,26 @@ static inline bool is_in_v2_mode(void) } /* - * Return in pmask the portion of a cpusets's cpus_allowed that - * are online. If none are online, walk up the cpuset hierarchy - * until we find one that does have some online cpus. + * Return in pmask the portion of a task's cpusets's cpus_allowed that + * are online and are capable of running the task. If none are found, + * walk up the cpuset hierarchy until we find one that does have some + * appropriate cpus. * * One way or another, we guarantee to return some non-empty subset * of cpu_online_mask. * * Call with callback_lock or cpuset_mutex held. */ -static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) +static void guarantee_online_cpus(struct task_struct *tsk, + struct cpumask *pmask) { - while (!cpumask_intersects(cs->effective_cpus, cpu_online_mask)) { + struct cpuset *cs = task_cs(tsk); + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); + + if (WARN_ON(!cpumask_and(pmask, possible_mask, cpu_online_mask))) + cpumask_copy(pmask, cpu_online_mask); + + while (!cpumask_intersects(cs->effective_cpus, pmask)) { cs = parent_cs(cs); if (unlikely(!cs)) { /* @@ -393,11 +401,10 @@ static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) * cpuset's effective_cpus is on its way to be * identical to cpu_online_mask. */ - cpumask_copy(pmask, cpu_online_mask); return; } } - cpumask_and(pmask, cs->effective_cpus, cpu_online_mask); + cpumask_and(pmask, pmask, cs->effective_cpus); } /* @@ -2199,15 +2206,13 @@ static void cpuset_attach(struct cgroup_taskset *tset) percpu_down_write(&cpuset_rwsem); - /* prepare for attach */ - if (cs == &top_cpuset) - cpumask_copy(cpus_attach, cpu_possible_mask); - else - guarantee_online_cpus(cs, cpus_attach); - guarantee_online_mems(cs, &cpuset_attach_nodemask_to); cgroup_taskset_for_each(task, css, tset) { + if (cs != &top_cpuset) + guarantee_online_cpus(task, cpus_attach); + else + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); /* * can_attach beforehand should guarantee that this doesn't * fail. TODO: have a better way to handle failure here @@ -3303,7 +3308,7 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) spin_lock_irqsave(&callback_lock, flags); rcu_read_lock(); - guarantee_online_cpus(task_cs(tsk), pmask); + guarantee_online_cpus(tsk, pmask); rcu_read_unlock(); spin_unlock_irqrestore(&callback_lock, flags); } From patchwork Tue May 18 09:47:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D831C433B4 for ; Tue, 18 May 2021 10:00:01 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DACB16024A for ; Tue, 18 May 2021 10:00:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DACB16024A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6l7eUTDlnE4K0ER5KNZETO+MxrVfBpiZV0Y3EVSwNWw=; b=M4UX9LDKNYoDoete8ZNiBE4la 8qo6VW+g2rgWdYON5Rh6rhu2SMHLQhSoIN0pAv21b3oBaDJ4HB5fgXDMyPfTkAtB8x1wSyU51keBM fw3lJ4uAzxDWbLLk0JEjyHz4s1bKF7Q/e3CGWHCbV4QLs32Iud0OBZNF2SjkiKJwlrtjDykEVbjRW +rFJW3Lf4Wy0jdASmNRP09aosQj06P2AIvapzLak+1/dPuM5kruLqBMFeYwVjs40lEmfZQ70jaXzd X3/zQEmEmGE1L0q9N1W6RDWTzuGDEn+DD+X0+tKNaNoio65Ulw/I8wIh4vuDwHpM0GkzYsPZvl6Dl OW+qa6Lrw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwUH-000Fs2-1p; Tue, 18 May 2021 09:58:05 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKi-000DuW-MM for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=S0EkUCKK02CAMFl032LIMUDv1WWyfhhYTZA5ouSwM3o=; b=JGlbhsri/L7lcGKyQQwGiMMEE6 HozWhYEI1Vu7Lr5mEr0Sk8vomZvQlw0CWUe+vumDG+2WjPtDFMtwdp2YJbK4MgIkIOLmx93NbwxYN hC9P1K29JWBX8sHa9o+FV7elV6kkhlJ2E8wEFShzGj1dLBuGyMAhMjKwvPSmM4XRoyOfTOVcT6wjq ND0G9EHtiGykcM0cjWw1TIvOkZhcBXo/Lue7+g9MinDzQnqxZY3t5fWgiWHTiU0HHKD0hxCPtOUSw fNg8iz2AUAn4XxlbVQJqXQMgTyOPiQ0VK2XWDr1G0YHeaOZULvGgd9SBtnDAirZ6VjGVlFxEFG8xu 9shlCJwg==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKg-00EWgu-6y for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:11 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id BD153613CB; Tue, 18 May 2021 09:48:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331289; bh=nWVmEPeR0hM9A1fN5XIKmBdskSrJ1G92PsqLDEhYaSM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bJZqDMYB1gHPG9YMTA0G3FEGVzf5TtzSg0BgVnZHPLaioxVDGWkxXh8gwz/OvrV6a d9QkWGIU208DCKa/xvr3mO7xf8vPcLQ2XYb3DyrfBR/iom3FBG3Bh4TFhJb7gMXYbQ nny2uINhbOB96DwiMBfoVcXVt+l1BQ9+RR2EQxK65Ltxuh4hAQU5XREskaMI56htpu 7oRAlRX4ZpVgEAUlSkoz7G/o6qi6RIp7p37jAJ9tBXOK7HbX2TYOPr3BPeo8Ck8A2P 7NQ/uHCz1LgDyo8aUGO95xjD2rMEqBqzCOTyLYbV3v25w5ag6+TYDrqIc3VPeuOO9y 1NELW1ysbLwmQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 09/21] sched: Reject CPU affinity changes based on task_cpu_possible_mask() Date: Tue, 18 May 2021 10:47:13 +0100 Message-Id: <20210518094725.7701-10-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024810_290706_DC0A95A3 X-CRM114-Status: GOOD ( 12.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Reject explicit requests to change the affinity mask of a task via set_cpus_allowed_ptr() if the requested mask is not a subset of the mask returned by task_cpu_possible_mask(). This ensures that the 'cpus_mask' for a given task cannot contain CPUs which are incapable of executing it, except in cases where the affinity is forced. Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- kernel/sched/core.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 482f7fdca0e8..21da3d7f9e47 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2350,6 +2350,7 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, u32 flags) { const struct cpumask *cpu_valid_mask = cpu_active_mask; + const struct cpumask *cpu_allowed_mask = task_cpu_possible_mask(p); unsigned int dest_cpu; struct rq_flags rf; struct rq *rq; @@ -2370,6 +2371,9 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, * set_cpus_allowed_common() and actually reset p->cpus_ptr. */ cpu_valid_mask = cpu_online_mask; + } else if (!cpumask_subset(new_mask, cpu_allowed_mask)) { + ret = -EINVAL; + goto out; } /* From patchwork Tue May 18 09:47:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7649DC433B4 for ; Tue, 18 May 2021 10:01:14 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F1C986024A for ; Tue, 18 May 2021 10:01:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F1C986024A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=j/F+iTWRmjBjLiKKF8ITNEyGCvvNdITkDTb0hMkAk0o=; b=ku/sle7XeCQVqcbYc3i+mx05x gDOEV6C8QdCf2pcc6fIoVE3LpkWSiBTLJt70c4M8JNsOM7OXxCnUfUCCLAsBUFidTwW1tp/elP/Ou LB/JzCo2cbQGcK4BsOAofLrYYZ6M1ftAsO6O1zbyEP95PPl1322fUrApjuj+3MzZaTN5IvP/Qj474 Tk49ur1xa8MsEtBmUFq4FbytRS58MpwAVlJqzRKcgmib7Ku8PXnLS7qj+zz9+fw2M9KzPsK26P/s9 lMcjV+YEAGiDM5rQNRyF6wv9umk1EUOUaksWqIHN++ReULe60qAmw9fzfYKcrNoEUKfDhhLjPnQUY eEvl+ia2Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwVX-000G6s-AR; Tue, 18 May 2021 09:59:24 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKm-000DvS-NR for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=PErYAO4RwcsAe3B1Llbjj15mMeoNwmDiNctF5O/2p1I=; b=gISYYgEB3QGEv61R0rkeycDRKV ZqIu4Xa8g7CX6ZNJyysh2diBfKMFIPnh/zguq7DAcdAx1n36G7vLvIZMbqL+ApTLXojZSp24kbmNA p/96klWz2adZFFlc8yGcDVIslvitkeu+k3SlZu5Dg61j+BZh4xdT0O0MXGuTLXUSzJpJvOmepfoAG Ti8i5Gl8oZ71+oOqsL4KV9W0MTIQYPWg2uoMdMTfK0SjTIM1eCt3jCncvIw34eGSbW1hjaA13/SpN JZvVa0RFwhgMAVcl67bepji64PPmvj43AS5m+GANksLTSgpzIzf9DRCGlkW5AP0UhwT28cjE5+SJM wXTby6qQ==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKj-00EWhi-On for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:15 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 57CF1613E6; Tue, 18 May 2021 09:48:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331293; bh=3IIlWuzmNSSaLHDQQBcc0aUminZ817yeuijTww75wDE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jhIWlDKt5jYI09Emq1vje2nk2fkytA9qWdsOJveSi29sEf7xh5MH/O61vAAaZgLLY xmGF2hJM9JuETDkUpDVM48Wr6uyNi9rn3KkRge+l+ujaYNLqFgmf5v/kdeOqsTWOSG Udz/fowf5cFwGnCz5UctZQlz6M7BP9HQXjtE0cYhHBlwod8x4N0FKdcOpoqyIQuV1n tEaum75VtMAiznWVNebBUwCpQ2L+XyHbaDI3PtoXPdvb6KXQqgR12EToCZPUfP0sBp oluPnLaOrjqgKubHYAZaHuSXgvB39WbZ5xmGOzQDhEcahqfibLF2K8yfppeAuFev3T VsYJN5MqEu/Yw== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 10/21] sched: Introduce task_struct::user_cpus_ptr to track requested affinity Date: Tue, 18 May 2021 10:47:14 +0100 Message-Id: <20210518094725.7701-11-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024813_872230_438CFEEF X-CRM114-Status: GOOD ( 15.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for saving and restoring the user-requested CPU affinity mask of a task, add a new cpumask_t pointer to 'struct task_struct'. If the pointer is non-NULL, then the mask is copied across fork() and freed on task exit. Signed-off-by: Will Deacon --- include/linux/sched.h | 13 +++++++++++++ init/init_task.c | 1 + kernel/fork.c | 2 ++ kernel/sched/core.c | 20 ++++++++++++++++++++ 4 files changed, 36 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index d2c881384517..db32d4f7e5b3 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -730,6 +730,7 @@ struct task_struct { unsigned int policy; int nr_cpus_allowed; const cpumask_t *cpus_ptr; + cpumask_t *user_cpus_ptr; cpumask_t cpus_mask; void *migration_pending; #ifdef CONFIG_SMP @@ -1688,6 +1689,8 @@ extern int task_can_attach(struct task_struct *p, const struct cpumask *cs_cpus_ #ifdef CONFIG_SMP extern void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask); extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask); +extern int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node); +extern void release_user_cpus_ptr(struct task_struct *p); #else static inline void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) { @@ -1698,6 +1701,16 @@ static inline int set_cpus_allowed_ptr(struct task_struct *p, const struct cpuma return -EINVAL; return 0; } +static inline int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node) +{ + if (src->user_cpus_ptr) + return -EINVAL; + return 0; +} +static inline void release_user_cpus_ptr(struct task_struct *p) +{ + WARN_ON(p->user_cpus_ptr); +} #endif extern int yield_to(struct task_struct *p, bool preempt); diff --git a/init/init_task.c b/init/init_task.c index 8b08c2e19cbb..158c2b1689e1 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -80,6 +80,7 @@ struct task_struct init_task .normal_prio = MAX_PRIO - 20, .policy = SCHED_NORMAL, .cpus_ptr = &init_task.cpus_mask, + .user_cpus_ptr = NULL, .cpus_mask = CPU_MASK_ALL, .nr_cpus_allowed= NR_CPUS, .mm = NULL, diff --git a/kernel/fork.c b/kernel/fork.c index dc06afd725cb..d3710e7f1686 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -446,6 +446,7 @@ void put_task_stack(struct task_struct *tsk) void free_task(struct task_struct *tsk) { + release_user_cpus_ptr(tsk); scs_release(tsk); #ifndef CONFIG_THREAD_INFO_IN_TASK @@ -918,6 +919,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) #endif if (orig->cpus_ptr == &orig->cpus_mask) tsk->cpus_ptr = &tsk->cpus_mask; + dup_user_cpus_ptr(tsk, orig, node); /* * One for the user space visible state that goes away when reaped. diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 21da3d7f9e47..9512623d5a60 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2129,6 +2129,26 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) __do_set_cpus_allowed(p, new_mask, 0); } +int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, + int node) +{ + if (!src->user_cpus_ptr) + return 0; + + dst->user_cpus_ptr = kmalloc_node(cpumask_size(), GFP_KERNEL, node); + if (!dst->user_cpus_ptr) + return -ENOMEM; + + cpumask_copy(dst->user_cpus_ptr, src->user_cpus_ptr); + return 0; +} + +void release_user_cpus_ptr(struct task_struct *p) +{ + kfree(p->user_cpus_ptr); + p->user_cpus_ptr = NULL; +} + /* * This function is wildly self concurrent; here be dragons. * From patchwork Tue May 18 09:47:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AA0DC433B4 for ; Tue, 18 May 2021 10:02:33 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DFD87610CB for ; Tue, 18 May 2021 10:02:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DFD87610CB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zhSbqE7K9x5VIhb0rX4f4aDUOCo8fgENAUg6ckOHekI=; b=OpySOt2uoN3DgSsE2SUQrq/ZN MIyee6mO/Z5BbLl7pr4nZiRxIAmDREhJKEbBpf6V+RA2ouLBt2bGPoHtYoRqbbOdJgPeGa6zAYhG9 I9t/6ABJUZep37+T4xU0LpKm0MEuON3be0hX3CwG1d3D28Sqjo8PhDiNfiGjHLnVJGYooYiF/D6wT vDFHVJXr8YBoJdEfxbi25wnGg+cEUzNkp7MrN2YVxOVNkDPrmOWfVkI0Fw72aMnIAF10iUE6RYKKr gayVjv5ankqMHp3hSiEFj5QcQCeSHPlnC4qjn3EATCHUgo78gz/CeZ4iLopcBOFcKpx8/qX+tlWGz cNIwwoTkQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwWd-000GI5-JU; Tue, 18 May 2021 10:00:32 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKq-000Dwe-4Z for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=dzRLRVMKxIMoLf6Q+b/8jkbjgkjftPiV2WnZBxrNtrM=; b=RHK83tlPcFPxSUNVuLzcP6LbT8 Sx+Fgz2XJdPKIVuJ8FHa2NUIurc1xqJ7EN7hQivVm0KpRf1s6xR7Mf5HIHaBtNmSu+OTWqtHi7QAM AdHURDnUGJ9sMixaSuDTTqnutU5ROkyX3FnXlgltnsPI6KsUi7UgZG8N79VXtDqv8hx1/b5JOuPXC xX93VRM5kjuKTKNsDnVIEGY1nzlS33RVvbyaVBNpIsMQkYtAuzEQ9PcWDEJyAH4lucE07YnDg6jc7 rs9f7Acgr13iGYaBwnUHXOl8lu/g4ez1hId/uJO/MfARTvOvMdjyDPdgFTmw1wOlmJphnB+iAnRUf Ok2K9+Xw==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKn-00EWiY-Aw for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:18 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id DE8E2613DB; Tue, 18 May 2021 09:48:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331297; bh=P87T2EfUL7peRiOL9j9h5VxX48Zfl+w2CWwNb/BHI/k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fEetJlCzqOez2e7J4B7XUnVnt3Uw1kt/v1RkvMkHpV/oc7I5qwF1IVj6EnkT8iqqu ezAtVrzs18o6PT/4U5I84PqtvXE6xaKZ7aEuHQMXidCvT6jjTsKVAvWxqZ1PdcAC/A UpZOBjToLwld7ULDIUXBJJyIP7S+IsVEB5NjUB3vpaiLa+LvdrYTftk5SnnYYmOSWb aKCvIGouAjn/2Y9kl02SmS/558f/bw3p1qgkUGNUS7+S9M+KFJTLwYwFXQyQj4a/cw DFUQABCuIHoe8TI51N+iNHceWzLnEpr4w/Q5eUA9sMI+tri175AxAZAMCkrIUjVc0i SR0zvVxQpNJ0A== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 11/21] sched: Split the guts of sched_setaffinity() into a helper function Date: Tue, 18 May 2021 10:47:15 +0100 Message-Id: <20210518094725.7701-12-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024817_438301_DB59578B X-CRM114-Status: GOOD ( 17.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for replaying user affinity requests using a saved mask, split sched_setaffinity() up so that the initial task lookup and security checks are only performed when the request is coming directly from userspace. Signed-off-by: Will Deacon --- kernel/sched/core.c | 110 +++++++++++++++++++++++--------------------- 1 file changed, 58 insertions(+), 52 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 9512623d5a60..808bbe669a6d 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6788,9 +6788,61 @@ SYSCALL_DEFINE4(sched_getattr, pid_t, pid, struct sched_attr __user *, uattr, return retval; } -long sched_setaffinity(pid_t pid, const struct cpumask *in_mask) +static int +__sched_setaffinity(struct task_struct *p, const struct cpumask *mask) { + int retval; cpumask_var_t cpus_allowed, new_mask; + + if (!alloc_cpumask_var(&cpus_allowed, GFP_KERNEL)) + return -ENOMEM; + + if (!alloc_cpumask_var(&new_mask, GFP_KERNEL)) + return -ENOMEM; + + cpuset_cpus_allowed(p, cpus_allowed); + cpumask_and(new_mask, mask, cpus_allowed); + + /* + * Since bandwidth control happens on root_domain basis, + * if admission test is enabled, we only admit -deadline + * tasks allowed to run on all the CPUs in the task's + * root_domain. + */ +#ifdef CONFIG_SMP + if (task_has_dl_policy(p) && dl_bandwidth_enabled()) { + rcu_read_lock(); + if (!cpumask_subset(task_rq(p)->rd->span, new_mask)) { + retval = -EBUSY; + rcu_read_unlock(); + goto out_free_masks; + } + rcu_read_unlock(); + } +#endif +again: + retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK); + if (retval) + goto out_free_masks; + + cpuset_cpus_allowed(p, cpus_allowed); + if (!cpumask_subset(new_mask, cpus_allowed)) { + /* + * We must have raced with a concurrent cpuset update. + * Just reset the cpumask to the cpuset's cpus_allowed. + */ + cpumask_copy(new_mask, cpus_allowed); + goto again; + } + +out_free_masks: + free_cpumask_var(new_mask); + free_cpumask_var(cpus_allowed); + return retval; +} + +long sched_setaffinity(pid_t pid, const struct cpumask *in_mask) +{ struct task_struct *p; int retval; @@ -6810,68 +6862,22 @@ long sched_setaffinity(pid_t pid, const struct cpumask *in_mask) retval = -EINVAL; goto out_put_task; } - if (!alloc_cpumask_var(&cpus_allowed, GFP_KERNEL)) { - retval = -ENOMEM; - goto out_put_task; - } - if (!alloc_cpumask_var(&new_mask, GFP_KERNEL)) { - retval = -ENOMEM; - goto out_free_cpus_allowed; - } - retval = -EPERM; + if (!check_same_owner(p)) { rcu_read_lock(); if (!ns_capable(__task_cred(p)->user_ns, CAP_SYS_NICE)) { rcu_read_unlock(); - goto out_free_new_mask; + retval = -EPERM; + goto out_put_task; } rcu_read_unlock(); } retval = security_task_setscheduler(p); if (retval) - goto out_free_new_mask; - - - cpuset_cpus_allowed(p, cpus_allowed); - cpumask_and(new_mask, in_mask, cpus_allowed); - - /* - * Since bandwidth control happens on root_domain basis, - * if admission test is enabled, we only admit -deadline - * tasks allowed to run on all the CPUs in the task's - * root_domain. - */ -#ifdef CONFIG_SMP - if (task_has_dl_policy(p) && dl_bandwidth_enabled()) { - rcu_read_lock(); - if (!cpumask_subset(task_rq(p)->rd->span, new_mask)) { - retval = -EBUSY; - rcu_read_unlock(); - goto out_free_new_mask; - } - rcu_read_unlock(); - } -#endif -again: - retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK); + goto out_put_task; - if (!retval) { - cpuset_cpus_allowed(p, cpus_allowed); - if (!cpumask_subset(new_mask, cpus_allowed)) { - /* - * We must have raced with a concurrent cpuset - * update. Just reset the cpus_allowed to the - * cpuset's cpus_allowed - */ - cpumask_copy(new_mask, cpus_allowed); - goto again; - } - } -out_free_new_mask: - free_cpumask_var(new_mask); -out_free_cpus_allowed: - free_cpumask_var(cpus_allowed); + retval = __sched_setaffinity(p, in_mask); out_put_task: put_task_struct(p); return retval; From patchwork Tue May 18 09:47:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B2F0C433ED for ; Tue, 18 May 2021 10:03:47 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 88192610CB for ; Tue, 18 May 2021 10:03:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 88192610CB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=E1wZOmyLWuxr3TXIp/TybbYyLm+v5LMZZXekoMA1z2Y=; b=DmcwOtCuda0w5KHNVSwaP4OCX 12E5AXgXhu9UO97rDzkIOLhIBWuU1FZCKWNuqCbGXsJhsoN2fKroIjOPRuXcyTAxD/yHzQuC02R6B CT44DzCvitTLkHmLhO0dO37RB570eA0em076beLTMlOtIFS0ycFw/ZhpmYGjKlOpngbP38I+IYJ1u PsNJcZwUiZPT3Sc0NS2Owon0GRXLo2HYB4QhcmIpNd3TEDwH89eghcj5DemZRWw9b4gcA29OKwN7S 2TpXj2EaXFxPG5NZvSaQjKfP+My09zre7RtPDr8X2Q6JlFh+Yn/3ymqhp70tchKAiPSJbtkCBjTCc rgHLg9J8g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwXc-000GTT-5l; Tue, 18 May 2021 10:01:32 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKv-000Dxs-05 for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=l0EMf5aisBlMmSr8Ot3jlaofafUoIDkZHz4GgoOmR9Y=; b=qwb95kit6ImatZRP/5+GZ5XlhZ 8UqVeJuO5CHNRf/T2qDVUQvmS/qC9C287wnUzp+PWwHRS8V5X1rr1sblZ1nTnOr/x9ky6cBy9kZRW v35Fe63XkmzIVW0Cpt9DYZ7/EJ5ljW/12DYXyKGSzx3Pan3IMTbEgYa5he3jJfRcpzPVXqfqyO0jf /lEG4LaWXT82sCzS21tI++Skv5EqjEtpuaa5IYkr4m6VuwYuryZLbFAWmOPD3gNL9fihAmQh/0yME 7xNPIWOS10MCEZ44HAI2UdJVMCIOXRGrDImr1bIrbzvn8rFiVMVjD8wzSicAPYWuGpON5P181fDgT XmCfVCsQ==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKr-00EWjO-PE for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:23 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 7474D613E8; Tue, 18 May 2021 09:48:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331300; bh=1kXNF+3IJHvkImbSqosGhbu75mb7vOSzQ/dwxKcJifY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hySB9KhTYXh5y8iJbIjb3GaWRVuzYUA9gukd+nhcjMstAwygw4oyNANjEscqXHMrM 0uH500Yk4oJKs23bkKkl1IfHrVtQz4gn++pWk5fwwQRbxbhyc60DlRDaYIjnvx5opo Lj36lWMA3rOgcqQOpSh+X3xHehl03/nTRU4UJwclORjPDGtKtslWSV5zTrI0+vr1M5 o8X0vu5OQD0E1zcdsvVVe0vYx4oUslbYTmYPx6eNdQRh5JsjaZGwoq6rYwV9M/wzxO fFalIXtk22/44K6ZASWVHbZpaZqyYvSQyYQ8dr3isDQTVTtyYdDlhYLyEagUNrEa+C zrERwpynn99lQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 12/21] sched: Allow task CPU affinity to be restricted on asymmetric systems Date: Tue, 18 May 2021 10:47:16 +0100 Message-Id: <20210518094725.7701-13-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024821_886160_549600D2 X-CRM114-Status: GOOD ( 31.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. Although userspace can carefully manage the affinity masks for such tasks, one place where it is particularly problematic is execve() because the CPU on which the execve() is occurring may be incompatible with the new application image. In such a situation, it is desirable to restrict the affinity mask of the task and ensure that the new image is entered on a compatible CPU. From userspace's point of view, this looks the same as if the incompatible CPUs have been hotplugged off in the task's affinity mask. Similarly, if a subsequent execve() reverts to a compatible image, then the old affinity is restored if it is still valid. In preparation for restricting the affinity mask for compat tasks on arm64 systems without uniform support for 32-bit applications, introduce {force,relax}_compatible_cpus_allowed_ptr(), which respectively restrict and restore the affinity mask for a task based on the compatible CPUs. Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- include/linux/sched.h | 2 + kernel/sched/core.c | 165 ++++++++++++++++++++++++++++++++++++++---- kernel/sched/sched.h | 1 + 3 files changed, 152 insertions(+), 16 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index db32d4f7e5b3..91a6cfeae242 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1691,6 +1691,8 @@ extern void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask); extern int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node); extern void release_user_cpus_ptr(struct task_struct *p); +extern void force_compatible_cpus_allowed_ptr(struct task_struct *p); +extern void relax_compatible_cpus_allowed_ptr(struct task_struct *p); #else static inline void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) { diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 808bbe669a6d..ba66bcf8e812 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2357,26 +2357,21 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag } /* - * Change a given task's CPU affinity. Migrate the thread to a - * proper CPU and schedule it away if the CPU it's executing on - * is removed from the allowed bitmask. - * - * NOTE: the caller must have a valid reference to the task, the - * task must not exit() & deallocate itself prematurely. The - * call is not atomic; no spinlocks may be held. + * Called with both p->pi_lock and rq->lock held; drops both before returning. */ -static int __set_cpus_allowed_ptr(struct task_struct *p, - const struct cpumask *new_mask, - u32 flags) +static int __set_cpus_allowed_ptr_locked(struct task_struct *p, + const struct cpumask *new_mask, + u32 flags, + struct rq *rq, + struct rq_flags *rf) + __releases(rq->lock) + __releases(p->pi_lock) { const struct cpumask *cpu_valid_mask = cpu_active_mask; const struct cpumask *cpu_allowed_mask = task_cpu_possible_mask(p); unsigned int dest_cpu; - struct rq_flags rf; - struct rq *rq; int ret = 0; - rq = task_rq_lock(p, &rf); update_rq_clock(rq); if (p->flags & PF_KTHREAD || is_migration_disabled(p)) { @@ -2430,20 +2425,158 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, __do_set_cpus_allowed(p, new_mask, flags); - return affine_move_task(rq, p, &rf, dest_cpu, flags); + if (flags & SCA_USER) + release_user_cpus_ptr(p); + + return affine_move_task(rq, p, rf, dest_cpu, flags); out: - task_rq_unlock(rq, p, &rf); + task_rq_unlock(rq, p, rf); return ret; } +/* + * Change a given task's CPU affinity. Migrate the thread to a + * proper CPU and schedule it away if the CPU it's executing on + * is removed from the allowed bitmask. + * + * NOTE: the caller must have a valid reference to the task, the + * task must not exit() & deallocate itself prematurely. The + * call is not atomic; no spinlocks may be held. + */ +static int __set_cpus_allowed_ptr(struct task_struct *p, + const struct cpumask *new_mask, u32 flags) +{ + struct rq_flags rf; + struct rq *rq; + + rq = task_rq_lock(p, &rf); + return __set_cpus_allowed_ptr_locked(p, new_mask, flags, rq, &rf); +} + int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask) { return __set_cpus_allowed_ptr(p, new_mask, 0); } EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr); +/* + * Change a given task's CPU affinity to the intersection of its current + * affinity mask and @subset_mask, writing the resulting mask to @new_mask + * and pointing @p->user_cpus_ptr to a copy of the old mask. + * If the resulting mask is empty, leave the affinity unchanged and return + * -EINVAL. + */ +static int restrict_cpus_allowed_ptr(struct task_struct *p, + struct cpumask *new_mask, + const struct cpumask *subset_mask) +{ + struct rq_flags rf; + struct rq *rq; + int err; + struct cpumask *user_mask = NULL; + + if (!p->user_cpus_ptr) + user_mask = kmalloc(cpumask_size(), GFP_KERNEL); + + rq = task_rq_lock(p, &rf); + + /* + * We're about to butcher the task affinity, so keep track of what + * the user asked for in case we're able to restore it later on. + */ + if (user_mask) { + cpumask_copy(user_mask, p->cpus_ptr); + p->user_cpus_ptr = user_mask; + } + + /* + * Forcefully restricting the affinity of a deadline task is + * likely to cause problems, so fail and noisily override the + * mask entirely. + */ + if (task_has_dl_policy(p) && dl_bandwidth_enabled()) { + err = -EPERM; + goto err_unlock; + } + + if (!cpumask_and(new_mask, &p->cpus_mask, subset_mask)) { + err = -EINVAL; + goto err_unlock; + } + + return __set_cpus_allowed_ptr_locked(p, new_mask, false, rq, &rf); + +err_unlock: + task_rq_unlock(rq, p, &rf); + return err; +} + +/* + * Restrict the CPU affinity of task @p so that it is a subset of + * task_cpu_possible_mask() and point @p->user_cpu_ptr to a copy of the + * old affinity mask. If the resulting mask is empty, we warn and walk + * up the cpuset hierarchy until we find a suitable mask. + */ +void force_compatible_cpus_allowed_ptr(struct task_struct *p) +{ + cpumask_var_t new_mask; + const struct cpumask *override_mask = task_cpu_possible_mask(p); + + if (!alloc_cpumask_var(&new_mask, GFP_KERNEL)) + goto out_set_mask; + + if (!restrict_cpus_allowed_ptr(p, new_mask, override_mask)) + goto out_free_mask; + + /* + * We failed to find a valid subset of the affinity mask for the + * task, so override it based on its cpuset hierarchy. + */ + cpuset_cpus_allowed(p, new_mask); + override_mask = new_mask; + +out_set_mask: + if (printk_ratelimit()) { + printk_deferred("Overriding affinity for process %d (%s) to CPUs %*pbl\n", + task_pid_nr(p), p->comm, + cpumask_pr_args(override_mask)); + } + + set_cpus_allowed_ptr(p, override_mask); +out_free_mask: + free_cpumask_var(new_mask); +} + +static int +__sched_setaffinity(struct task_struct *p, const struct cpumask *mask); + +/* + * Restore the affinity of a task @p which was previously restricted by a + * call to force_compatible_cpus_allowed_ptr(). This will clear (and free) + * @p->user_cpus_ptr. + */ +void relax_compatible_cpus_allowed_ptr(struct task_struct *p) +{ + unsigned long flags; + struct cpumask *mask = p->user_cpus_ptr; + + if (!mask) + return; + + /* + * Try to restore the old affinity mask. If this fails, then + * we free the mask explicitly to avoid it being inherited across + * a subsequent fork(). + */ + if (__sched_setaffinity(p, mask)) { + raw_spin_lock_irqsave(&p->pi_lock, flags); + release_user_cpus_ptr(p); + raw_spin_unlock_irqrestore(&p->pi_lock, flags); + } +} + void set_task_cpu(struct task_struct *p, unsigned int new_cpu) { #ifdef CONFIG_SCHED_DEBUG @@ -6821,7 +6954,7 @@ __sched_setaffinity(struct task_struct *p, const struct cpumask *mask) } #endif again: - retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK); + retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK | SCA_USER); if (retval) goto out_free_masks; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index a189bec13729..29c35b51411b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1956,6 +1956,7 @@ extern struct task_struct *pick_next_task_idle(struct rq *rq); #define SCA_CHECK 0x01 #define SCA_MIGRATE_DISABLE 0x02 #define SCA_MIGRATE_ENABLE 0x04 +#define SCA_USER 0x08 #ifdef CONFIG_SMP From patchwork Tue May 18 09:47:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82567C43461 for ; Tue, 18 May 2021 10:04:41 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 141A2610CB for ; Tue, 18 May 2021 10:04:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 141A2610CB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dh153rgEbdL34wZklmCH0TmlwQFYbS/rpl/ZSgqBD08=; b=ePj8PevuhIVhN/Wjaqkf3vNga uXH+KyYnPRQr1sNG5/DEOF7jiDDIZi6la14JR+34oSFP25p3T0V7uKCTDbS4Efvl4x0zVLtnAA1ML RQ4ECvxHehEb9GQMvNlLf4T/ioNGXLSU7YRTVz3YnD5IPOSB1XNOdsJB6WiopBqwz250EAfvoK1m8 cTCnzGQoUAJlw+sZYDl8QWL8PwioUxvn+mWJc8+zwNlmMrcmUSw29XmVhLtDFTkoYR8f9jiRn5Cy3 Q+waBwjEtHOlvy5sLaEbqDNteu22+e4AiiJ+fUdJIxtNv6K4lia4JCCje62o1dDxIjdneuvRymDPX +B49Cc+Qg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwYp-000Ggj-Ke; Tue, 18 May 2021 10:02:48 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKx-000DyJ-1Z for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=ykl5GInEicBrXAQCD7ZOD64wvKf+8lVn/3ZZzg6geLs=; b=Cx1PMkdcq6Y4xrLqTZoQJMY5Cg j0nSqiqdMBSXp2OxOs4F0d3jdTujvJNBCMiW2pOuqjsvjjBi8MEy2DcFjRoLCwE4FCfHE/OEy6bHc iNoqpLQY7jwukGz0IZk5MRA9Vcqkr9095BSgBmZm9wxPt8/wbjbcprahUucGou2/LSjzF9C85cfU2 sXt0AHgWZQSNqWDripRqSiDL9qsS+xmQrRp6wAhJKqKLGhHBWsF6JAtpPkpgVFJqbbbrCHtojBKoN pKLvN00GYNA5XB24P/sUWbO3A2BqI1Ryq08jo2yQagiWzeLV09iBJD8IJ/untOMHcvIuB4aIt0eCW EZ2RVFxA==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKu-00EWk5-E9 for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:25 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0A95B613F4; Tue, 18 May 2021 09:48:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331304; bh=CXXSJFTLguIC0wnRzVNZjaiEiV5Q9Ps2gUQifPQxHXY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nVxji+ySFLgIHKy055Xm1GNP1NonfdXGEDwTzeg4wNyAOJ5ruRznguUI8CcX2GpKY FMM0QlUPUDq6Ugowra6N7/hicfp/zkzLQKQxQSkeMGXd0jP5LJRlVIZ5HMJ5zrIRMz 4yBQB33CX/K6WLegiITaoiNArqpRa9ybEKWitb0A210zhqZNxJz+FOsPehG2vVwcv+ bQgyVRTAB7FERd6FEDduJGLLQPXUhGZry4E8mlU3AKIdFoB0GCFrz+TT40UVn89+fK 39vHAVB+VZYbgai9BGmpnAjaoagZ47Fd3lLdqxHqCU4Q9wc4eA3lN+O0FeC5K44w0M M7OVoBumfEQTQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 13/21] sched: Admit forcefully-affined tasks into SCHED_DEADLINE Date: Tue, 18 May 2021 10:47:17 +0100 Message-Id: <20210518094725.7701-14-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024824_516159_61EEFFBC X-CRM114-Status: GOOD ( 14.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On asymmetric systems where the affinity of a task is restricted to contain only the CPUs capable of running it, admission to the deadline scheduler is likely to fail because the span of the sched domain contains incompatible CPUs. Although this is arguably the right thing to do, it is inconsistent with the case where the affinity of a task is restricted after already having been admitted to the deadline scheduler. For example, on an arm64 system where not all CPUs support 32-bit applications, a 64-bit deadline task can exec() a 32-bit image and have its affinity forcefully restricted. Rather than reject these tasks altogether, favour the requested user affinity saved in 'task_struct::user_cpus_ptr' over the actual affinity of the task which has been restricted by the kernel. Signed-off-by: Will Deacon --- kernel/sched/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index ba66bcf8e812..d7d058fc012e 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6403,13 +6403,14 @@ static int __sched_setscheduler(struct task_struct *p, if (dl_bandwidth_enabled() && dl_policy(policy) && !(attr->sched_flags & SCHED_FLAG_SUGOV)) { cpumask_t *span = rq->rd->span; + const cpumask_t *aff = p->user_cpus_ptr ?: p->cpus_ptr; /* * Don't allow tasks with an affinity mask smaller than * the entire root_domain to become SCHED_DEADLINE. We * will also fail if there's no bandwidth available. */ - if (!cpumask_subset(span, p->cpus_ptr) || + if (!cpumask_subset(span, aff) || rq->rd->dl_bw.bw == 0) { retval = -EPERM; goto unlock; From patchwork Tue May 18 09:47:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A57CC433ED for ; Tue, 18 May 2021 10:05:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CF81060725 for ; Tue, 18 May 2021 10:05:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF81060725 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MOjiB/eVo0pIroS30PVmJQL9SnOALC+iVDCB1IDHKK8=; b=U8WfClW8fBablr0QemOd0J8yu EQVRXnz5cbsxJd6XyvbMPJAiNvWGPzpkzeWf19z6dJvAu7wbvXn4qbFQSETHZC5hGOgmG8nGnfDmq PbmHWVkNkyYPVFQ95MKInzp2nloAJpc3tWzuhvDzVWm4RJPoGk6ZxMSjdyCM5AC324bJ2Vxr/tgdz KUvkUskwbYYlnjikHHvrxooT6Db1WOuUj4zjOIzdp6NBkWUDInYwq0rx1RErmb6qUl4zwVyT5M+bq Y1ksPdX2ZWLvKC0pEAKYJ0Jk2fa6YyBJMMAeJK3qqUBcT8xsjY72sepNcskdQxl/ylAzF40LEUJwZ Gx4ibQ4Lg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwZu-000Gti-Td; Tue, 18 May 2021 10:03:55 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwL0-000Dz1-L1 for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=bjr5vEOw53CCMoZJdtKmOzHU4bbp/tC78/DJIl5Tzo0=; b=lIFkTa7U0wqKKLEMixQ1NeBVUI MVBcZ2N1wKVyqWxleNYIaBr2ZNdVkSvByTWhfQGV2tvz3g5BJYkh+odboTa4+xBwd1fVS2/StAmgo T61rBgG/noX/5xQcgNinLXdLOS/uj3RJ2JScRwQV9JOwCuKou27oa0PosfbQNgT1LtFozcK4gommr iTSvRYQXiIgbxICkLuHlibFRTCZEFSql9TB2xKoh/2ow//j+x2DICnBLAcCd8cRlT4M17KSKh6qwn Uqvohz7qBDNwjDxD/CfflqziQF5n3au8bVe6Frp5qgrj6oRYHkaen4seOcutTu+QDZC8o1lxQgDZz 8GX+Ra8A==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwKy-00EWkc-0c for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:29 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9224A613BD; Tue, 18 May 2021 09:48:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331307; bh=Q01LQeqZQsXv68AWl8KkMRey0NZmXny9qiWfV/iWJII=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JPz5w/gR25XyMMQKrC4eho1EemJnl53pOfF1NzmpglNZe2RSOE96Kds2ka+HXolIl ptG9/FImIbUYt4QTHoTw4vEoQczjItxnD9p0/oCyeyvLWXoDBMKJr6fM9PoEvqivdt Qeuo8RRkkYfZK/9d3w9v09581ROVM6otSwdPPkj3tXEhj365tYyngsuSoVk/+OVtSO 8ciHGLSgJFmejrB5u2ec+P0dWCRnf4M8egmOVAOC5trF57+RELuZXxxmX6uvul1dS4 0HtzSjMscx1fmZ5IerrD+r7Rbp4Ji0yqMpCqS2/VBuSDB7Zq46J+KSAenmLALaEuYy eEiR99C+3kROg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 14/21] freezer: Add frozen_or_skipped() helper function Date: Tue, 18 May 2021 10:47:18 +0100 Message-Id: <20210518094725.7701-15-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024828_092611_FE12EB51 X-CRM114-Status: GOOD ( 13.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Occasionally it is necessary to see if a task is either frozen or sleeping in the PF_FREEZER_SKIP state. In preparation for adding additional users of this check, introduce a frozen_or_skipped() helper function and convert the hung task detector over to using it. Signed-off-by: Will Deacon --- include/linux/freezer.h | 6 ++++++ kernel/hung_task.c | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/linux/freezer.h b/include/linux/freezer.h index 0621c5f86c39..b9e1e4200101 100644 --- a/include/linux/freezer.h +++ b/include/linux/freezer.h @@ -27,6 +27,11 @@ static inline bool frozen(struct task_struct *p) return p->flags & PF_FROZEN; } +static inline bool frozen_or_skipped(struct task_struct *p) +{ + return p->flags & (PF_FROZEN | PF_FREEZER_SKIP); +} + extern bool freezing_slow_path(struct task_struct *p); /* @@ -270,6 +275,7 @@ static inline int freezable_schedule_hrtimeout_range(ktime_t *expires, #else /* !CONFIG_FREEZER */ static inline bool frozen(struct task_struct *p) { return false; } +static inline bool frozen_or_skipped(struct task_struct *p) { return false; } static inline bool freezing(struct task_struct *p) { return false; } static inline void __thaw_task(struct task_struct *t) {} diff --git a/kernel/hung_task.c b/kernel/hung_task.c index 396ebaebea3f..d2d4c4159b23 100644 --- a/kernel/hung_task.c +++ b/kernel/hung_task.c @@ -92,8 +92,8 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout) * Ensure the task is not frozen. * Also, skip vfork and any other user process that freezer should skip. */ - if (unlikely(t->flags & (PF_FROZEN | PF_FREEZER_SKIP))) - return; + if (unlikely(frozen_or_skipped(t))) + return; /* * When a freshly created task is scheduled once, changes its state to From patchwork Tue May 18 09:47:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 362A9C433B4 for ; Tue, 18 May 2021 10:07:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BD38E61185 for ; Tue, 18 May 2021 10:07:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD38E61185 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=br9qTpv+UvD00yFIFP2iik7pT33uOoAd89HloO6q9HU=; b=OyhfGY0WL6MeFixrqjRBk+qJe v6TQ+aL/UstdPbXsRwlFz+CjGyYKwFI/KJwzxjgCxfBvwvcW1RrG7Q1a1yy07itCfwVcHWMTLNDHB FZ5raX7C80uQUJFjVIKcITxwFH4lpqMORmjVjKdDWbxEjtQBrS+mNYWUfqsE5cP9EyU5xQXAbvqs3 nE9ks2vmN4NA0zqYW5vyBib/Z5W+vuUfambhP5bnvacwqXYbp5bCo2dJbK3p3kOpGxskqxK3/AKHU 1sm3x1aaow2amnpNUxfwrmmIuHz8xFWr6sVx9gAU357t0HZI7DOjEyM+NTLuOOowUYj5O762goKP/ pDd9+bvDw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwav-000H4v-An; Tue, 18 May 2021 10:04:57 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwL5-000E0R-Ii for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:35 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=O4M1tuPpeMcv+HwLI+koJa4o1Si/SzgLQivtStAuziI=; b=o6flcBM2I8zjvaJoS4Up7B10cu hNb+5jiGv9ByMp043EBPbd/vRqd6At+e5CDD2KjmeVTBTxOLVeX/tA3Br2lK7+eXmuq5up6Dg+K6T f0jPThCcrjtCO7JRCZdd8lbsSozPDpRFAPlAs+OyVn/oSllnKKM4VXBkQCu1+kUlhsFX9mhaVFPTF 8UPdpkhy+xJ1DOqP/mnZ4jQR2FXge3c0b++zXK23EjSkyRaYw3QRJ4OEcL9JmoID8r/OoqwD4WoKZ gPlC+eXVt7DAkGEhG6yhTsMg93AcoooNiSnIgMp8ahTaJBxWebauYeSluwKB0O/ZQC+8caFELWsBI lfdefrHA==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwL1-00EWlU-IK for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:34 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 23B95613F1; Tue, 18 May 2021 09:48:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331311; bh=44h3JsYZ/KIsAP+Upj/qBCyu3PQIrAfBaeDZkj8kEHs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MMayGdvqj7sLozMbYFnwY2WFufoESelFZSH8VuLRTDX3yLwAATjm9J81bgPDOuugT hCTg2WgfvbgkYufvVeb6qBR1RxTWc5mKSSyDGC2a+ksgEFM/o4wU3UPORukgbOdirx XS1frfOXkyKZnlH/Ii+ARG8h7cFNv7BP21kof+VBHR5KYXRVQ6IRHC3tLCeC1xp1Wg 0s3eWa2ZL08a+qhYd73PdJwGSwbR9dYH3QW3noMKqynRPt+Jqgh18gDZMgfE+s9fIc ydGY2gtkHFAEJ3Rl/mLyW7P4N5vPGXKWmjuSWk3RRTWvkiytoyFyj12egmW3tIMlSu Gd2JZAmEk6fsg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 15/21] sched: Defer wakeup in ttwu() for unschedulable frozen tasks Date: Tue, 18 May 2021 10:47:19 +0100 Message-Id: <20210518094725.7701-16-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024831_664087_04C88BF7 X-CRM114-Status: GOOD ( 21.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. Although we take care to prevent explicit hot-unplug of all 32-bit capable CPUs on such a system, this is required when suspending on some SoCs where the firmware mandates that the suspend/resume operation is handled by CPU 0, which may not be capable of running 32-bit tasks. Consequently, there is a window on the resume path where no 32-bit capable CPUs are available for scheduling and waking up a 32-bit task will result in a scheduler BUG() due to failure of select_fallback_rq(): | kernel BUG at kernel/sched/core.c:2858! | Internal error: Oops - BUG: 0 [#1] PREEMPT SMP | ... | Call trace: | select_fallback_rq+0x4b0/0x4e4 | try_to_wake_up.llvm.4388853297126348405+0x460/0x5b0 | default_wake_function+0x1c/0x30 | autoremove_wake_function+0x1c/0x60 | __wake_up_common.llvm.11763074518265335900+0x100/0x1b8 | __wake_up+0x78/0xc4 | ep_poll_callback+0x20c/0x3fc Prevent wakeups of unschedulable frozen tasks in ttwu() and instead defer the wakeup to __thaw_tasks(), which runs only once all the secondary CPUs are back online. Signed-off-by: Will Deacon --- kernel/freezer.c | 10 +++++++++- kernel/sched/core.c | 13 +++++++++++++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/kernel/freezer.c b/kernel/freezer.c index dc520f01f99d..8f3d950c2a87 100644 --- a/kernel/freezer.c +++ b/kernel/freezer.c @@ -11,6 +11,7 @@ #include #include #include +#include /* total number of freezing conditions in effect */ atomic_t system_freezing_cnt = ATOMIC_INIT(0); @@ -146,9 +147,16 @@ bool freeze_task(struct task_struct *p) void __thaw_task(struct task_struct *p) { unsigned long flags; + const struct cpumask *mask = task_cpu_possible_mask(p); spin_lock_irqsave(&freezer_lock, flags); - if (frozen(p)) + /* + * Wake up frozen tasks. On asymmetric systems where tasks cannot + * run on all CPUs, ttwu() may have deferred a wakeup generated + * before thaw_secondary_cpus() had completed so we generate + * additional wakeups here for tasks in the PF_FREEZER_SKIP state. + */ + if (frozen(p) || (frozen_or_skipped(p) && mask != cpu_possible_mask)) wake_up_process(p); spin_unlock_irqrestore(&freezer_lock, flags); } diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d7d058fc012e..f5ff55786344 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3525,6 +3525,19 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags) if (!(p->state & state)) goto unlock; +#ifdef CONFIG_FREEZER + /* + * If we're going to wake up a thread which may be frozen, then + * we can only do so if we have an active CPU which is capable of + * running it. This may not be the case when resuming from suspend, + * as the secondary CPUs may not yet be back online. See __thaw_task() + * for the actual wakeup. + */ + if (unlikely(frozen_or_skipped(p)) && + !cpumask_intersects(cpu_active_mask, task_cpu_possible_mask(p))) + goto unlock; +#endif + trace_sched_waking(p); /* We're going to change ->state: */ From patchwork Tue May 18 09:47:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2E29C433ED for ; Tue, 18 May 2021 10:07:56 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 94686611AD for ; Tue, 18 May 2021 10:07:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 94686611AD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MaJca/DWGSGxRX9TpJhnTfGze2tYrpOvtItBJd63nwQ=; b=eTCXwrzUwvQHflz2N8cvp/ycd iBEFbJ7l3m/KfM1V14U0lYv4iqKRprW4vFNHkuNVf5vkeDZ07omScI5e8bv5AmXHLUFE+SaGqNyG8 MRtbCPtMcfZ+CTVGMltqM/F19zW5AX4SuYHxhzsQeLpVZmeOu/2yLyIAkUScHQIyW3qdqvsHCiNgc lS3G3FKOwDjxdQ6wd7l1TYV3qJJAkgR+ddBawjy8ldkDUnwddxbfK6L3Xd/L6CFw69T9GLdBzihgR XloFmEMm1T2K0IDg7U/YuIkOaX+00eLucTAbQkFStF+nhgU+cft4PQ+5pxEBDajJTDx/ArM8Dhe9u Ywv8Ij9Ug==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwbt-000HFb-OT; Tue, 18 May 2021 10:05:58 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwL7-000E17-P9 for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=QWdJLdtbkVKcmFgSpDyUxiktHy5X5kT7KmOcrItqULQ=; b=WZTLvubXUkQSvY//zL/OwarEm5 ll7o9tGrTcSqgNsHfnN7jPCCJl9MXE672n/YPBsOe4w66ICIMichcy4cvTLJSrMZUIDsbpHWZb3NN UE2riELV61bNV5+N9T+W6ZSi0FJWFBF6qNgw5+P+v90e0SI8goWqupqEFVl5rZh5ITOkIR3YI5xbX HORw/zIKuPUA/9uL1DL8PeH4pbuC5m+VeQDxa0Fqr9bLmJ56POirf52knRvCPA6wSocXT9ZMdx/RB VE0GlEYFV/Roah3InBikCAOvw8zrz0CsW8XvFd+8Ao52FTlT9c8Cs74T5JXymVoeI3yroqleF4qOh z6ObNBxg==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwL5-00EWm1-5K for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:36 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id ADD1F613F6; Tue, 18 May 2021 09:48:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331314; bh=Dh/Va0X09dy++LO+W/9hE+3xOUtyUgHSE0+ZKx7G1GY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AD3ZY9USRC7qBSNI+UlwJXnlO/hP51HMdwLExYlkSvChWUqzEezJD+4JY1iVaG3fn 0ck3hG27/ib9mDAvXmOuRnlC/QinvFDCdlolRh5L/grcfifI5uIm3n3pZXFrQL5hgG pLxQYytpQIS2wXxEka1hruQPbIRjtzlbtcdrXk4C4Z9PdTv985Utk/PHwrv1s5r3za XoZ0+WvsoDD2k8Ak1Ug1m1HwsCrS5WnqogmKBgQK+w4lThzaQWmfxSz000knyS7msW 0OmfLkLhtvSdih0E6Q1zfjT4wTHBgtW7ZXDzYG+MG1M4Dw0LnSa548pKHVlGD5VJ+f +5LlMTJ7h9ZzQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 16/21] arm64: Implement task_cpu_possible_mask() Date: Tue, 18 May 2021 10:47:20 +0100 Message-Id: <20210518094725.7701-17-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024835_243618_3AA33DF7 X-CRM114-Status: UNSURE ( 9.51 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Provide an implementation of task_cpu_possible_mask() so that we can prevent 64-bit-only cores being added to the 'cpus_mask' for compat tasks on systems with mismatched 32-bit support at EL0, Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/mmu_context.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index d3cef9133539..bb9b7510f334 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -231,6 +231,19 @@ switch_mm(struct mm_struct *prev, struct mm_struct *next, update_saved_ttbr0(tsk, next); } +static inline const struct cpumask * +task_cpu_possible_mask(struct task_struct *p) +{ + if (!static_branch_unlikely(&arm64_mismatched_32bit_el0)) + return cpu_possible_mask; + + if (!is_compat_thread(task_thread_info(p))) + return cpu_possible_mask; + + return system_32bit_el0_cpumask(); +} +#define task_cpu_possible_mask task_cpu_possible_mask + void verify_cpu_asid_bits(void); void post_ttbr_update_workaround(void); From patchwork Tue May 18 09:47:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1E24C433ED for ; Tue, 18 May 2021 10:08:45 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 836DA61185 for ; Tue, 18 May 2021 10:08:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 836DA61185 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=rxFmdFHVatO5c1VVmoORm4Jy4UmWStTjuk+K+0NHlR8=; b=pL1N8rJfzHTRzvBRX5mlt2qcp 3YaX5tk955lCbK6wgU+yqoNG5RMeaH7ru/boI4pyHEQ83rv6CVeB+3CbwIhO/wN64uv6WCVIBxlPU Vbt6Db43Eb7EurBVQpRkwNL1fLWCX4MgJe+GzK6SGAW4Lg9AKEzHMjaD76hq/1JHQx1T179aP8gTD eGsiAA+KsdmBUgfiEk8tknsq/kOt8bejykAJ/K2AQdi5YoAWKn0LJdmdWGH45BnBlT7UAxoQs2JU+ /gJ9aqC8h5SSbiaXEo/OwuE1NsT9mYCKbwNMQLV+VnjlKUAunpAlqE4mKCqWOdTSkWE9h7PZYeMRh ZumxEZGCA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwcv-000HQd-23; Tue, 18 May 2021 10:07:01 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLB-000E2E-77 for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=21zMtIdWAx7slZJEppOPAVloi7y0XLQyP0p6eiABDQg=; b=CXRNCwRHY0cBDp4o/QD8cIuLuf lO3n+WkJZ/dZyU0jNc3c0Z+HzSHc71OMwo1YPDgqlvPHa4E+x4Qjf6Ywf6XWTVHOf3GLY31GZ/rQT 2xjqOT+nbGrVXPlpNL59Z9a+i6OTaz3649bFQs3vfpkcHYyKuGreqnOmH0DXshuuz7jOZ3b7QSkpf G0ODR0mPqqnTsFQikZajlyiCcby1Z1B4SC8eut97wM874DRDBQxGxcUsLSKT3tR7vXmz9636Rye5j LISzifru8W+mrKCUfSfCiS6gWLvmd/1GNGRpyYUqXE4XEhwGCi1ocLJ4APQAdyTz8jNtzTvsB2agS zCvQE8tA==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwL8-00EWmU-Kd for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:39 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3E50961400; Tue, 18 May 2021 09:48:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331318; bh=R9BW6P5F0BMcxqhawbpZj55WsxoJwynPUjScPCDvTIs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DHYBKJslYl+OETsyFh81LkpiXQ1cFVBDLzO/naT40uQr2B1dNxMIsDbm5QRs9oU2Z S704c6CqM37DEqMAX0U1NLMEWmSGkBhw5PpD5NpFNbIGNQ3riHGvsT8egS7XIvaRYc ddfJGNtVY1A3jNpKyKyqog2WD5Q8DJX9RdW2aVHD2GoCAvv+3AA1rEkASemwoWStaC STSqemDuneUVF9rEaqO9Azitdtec7reffxVPEc8srAEWY16QDZHqSLl5ly02jgPuPx I/jsjzw8Vp6IGqYqyJGRJLNXDL35IPBuaz+TLXET7FpPkHE7OAVtibCRod2500RmUR LwPBykiMdhBBQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 17/21] arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit EL0 Date: Tue, 18 May 2021 10:47:21 +0100 Message-Id: <20210518094725.7701-18-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024838_714955_C669A5C2 X-CRM114-Status: GOOD ( 14.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When exec'ing a 32-bit task on a system with mismatched support for 32-bit EL0, try to ensure that it starts life on a CPU that can actually run it. Similarly, when exec'ing a 64-bit task on such a system, try to restore the old affinity mask if it was previously restricted. Reviewed-by: Quentin Perret Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- arch/arm64/kernel/process.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index f4a91bf1ce0c..8e0da06c4e77 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -647,8 +647,22 @@ void arch_setup_new_exec(void) if (is_compat_task()) { mmflags = MMCF_AARCH32; - if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) + + /* + * Restrict the CPU affinity mask for a 32-bit task so that + * it contains only 32-bit-capable CPUs. + * + * From the perspective of the task, this looks similar to + * what would happen if the 64-bit-only CPUs were hot-unplugged + * at the point of execve(), although we try a bit harder to + * honour the cpuset hierarchy. + */ + if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { + force_compatible_cpus_allowed_ptr(current); set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); + } + } else if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { + relax_compatible_cpus_allowed_ptr(current); } current->mm->context.flags = mmflags; From patchwork Tue May 18 09:47:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28E3CC433ED for ; Tue, 18 May 2021 10:10:04 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A63D0611BD for ; Tue, 18 May 2021 10:10:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A63D0611BD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Sr5N+HvZKKFgaeewiecIB9ZWnDtXQjGBP/wjYwC5qmA=; b=iN/QvszvldDOgND5+xjeVLl3D +IeLlxkuFbH/UQpQqobRfoVc4QvMTsZ3QpwKaoHFH56MKZqXmPTqzxIZY4W9LwaAaEZ29I4fX0eJ4 VCYqdkNinKFiD91C0DWakmS0cIYFOmWYKk2Eo4fHYxCRV9wHHGm+9rB8xOOiYnlj4KaXjI4e328uf vyuxLardekG1OLi4zistJ01cRxDWilcpLfzpEsmL3zNLiyRydNFXPebeHlrZ1TIl9/2k9vSzTHZE8 Kp3NZfAjn0pZwCBp1+fTpSoXKc25DnxOhYiWtGoy+X7nKXDTQcwGeRfPyIfZGrSLrJIu7Rlmx0oOY mOHOAsFyQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwdq-000HcR-6k; Tue, 18 May 2021 10:07:58 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLF-000E33-Nj for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=mJrRcI1ieCOpDitI8/IIj8QtBN6jl43+OSHaWCO6GBc=; b=OpW6rf/+RRxc3bLoP/jqNzJhyI Ze6RDPqJcy3a6gk25dcchnhZNHGzuZ+mE9PAYgbV2DRRRBIc5NVBNEuT2CtiHwOrTs7UcFrwm2sOF t4spezRbfyBchuYhm/J3NbEnTrOqZDTtyzLs8Y41ZXpHVog4cHM+4g+67vuW8XrIraBzkwRZSKH0n m+n7APxtn4tvpt4a9+QiLuhEbZvvPJ2hCeLojgAgD7hrWHRd7mOXuGYOgdkItE7SDiBxYvzrHCfaX 9fRxwmKWxtZcdivTgDUFsjqPEHZ5ppXn3kMLVvVQLh5+pht3lwefOiNLnYScEH7cJ4DxLC2JTyUf4 X9MuCoig==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLC-00EWnG-73 for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:44 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id C1FDA61401; Tue, 18 May 2021 09:48:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331321; bh=6g9NVbkObRxGunwkaQ5JxyE6uDt34ueHBKqyl0oVxYE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kbkOc+WKgFFl0HV+uacnOtz0MtG+VG7IhssMjNOO3i3md9AUXwFLFbGvOlpJpP2UK f2exwaDNgrJDxMFDR5OlCqmVex9lfkqHocqc6c3LInUzFdAFzLPcI9gCtH4YNVksco f+lMksvAgtytTY+DC1MbXoWtJDcR2JormPvecHdqDcBPcLYqp0pzb2jYqu/Wn1RQNT zJ+Y73ZOzrAKdEYP/S3kDgZqofiXT0z6KJlaprpYRMtofQNdN0g7rm9ysamPRUDA7D bDugtVJzZ5+Nx0fCM7DDgQvkYtTx8Zn3xOlpnl7S9DmXA/SsDgYKH3N9wC7BOyL6Lt SMBvCK4BI4tJQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 18/21] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Date: Tue, 18 May 2021 10:47:22 +0100 Message-Id: <20210518094725.7701-19-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024842_297220_B3227E54 X-CRM114-Status: GOOD ( 15.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we want to support 32-bit applications, then when we identify a CPU with mismatched 32-bit EL0 support we must ensure that we will always have an active 32-bit CPU available to us from then on. This is important for the scheduler, because is_cpu_allowed() will be constrained to 32-bit CPUs for compat tasks and forced migration due to a hotplug event will hang if no 32-bit CPUs are available. On detecting a mismatch, prevent offlining of either the mismatching CPU if it is 32-bit capable, or find the first active 32-bit capable CPU otherwise. Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- arch/arm64/kernel/cpufeature.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 959442f76ed7..72efdc611b14 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2896,15 +2896,33 @@ void __init setup_cpu_features(void) static int enable_mismatched_32bit_el0(unsigned int cpu) { + static int lucky_winner = -1; + struct cpuinfo_arm64 *info = &per_cpu(cpu_data, cpu); bool cpu_32bit = id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0); if (cpu_32bit) { cpumask_set_cpu(cpu, cpu_32bit_el0_mask); static_branch_enable_cpuslocked(&arm64_mismatched_32bit_el0); - setup_elf_hwcaps(compat_elf_hwcaps); } + if (cpumask_test_cpu(0, cpu_32bit_el0_mask) == cpu_32bit) + return 0; + + if (lucky_winner >= 0) + return 0; + + /* + * We've detected a mismatch. We need to keep one of our CPUs with + * 32-bit EL0 online so that is_cpu_allowed() doesn't end up rejecting + * every CPU in the system for a 32-bit task. + */ + lucky_winner = cpu_32bit ? cpu : cpumask_any_and(cpu_32bit_el0_mask, + cpu_active_mask); + get_cpu_device(lucky_winner)->offline_disabled = true; + setup_elf_hwcaps(compat_elf_hwcaps); + pr_info("Asymmetric 32-bit EL0 support detected on CPU %u; CPU hot-unplug disabled on CPU %u\n", + cpu, lucky_winner); return 0; } From patchwork Tue May 18 09:47:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF4B6C433B4 for ; Tue, 18 May 2021 10:10:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 435E061185 for ; Tue, 18 May 2021 10:10:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 435E061185 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=glLvVpxgsHnVmz71hQWRRDq8yYQw3SFd1p5kc+ka9U4=; b=qbej1JPm1FGRh0Ipv2DCKGDgT 3inM21KxEj+YnsEumkLVv10xJRifQG2U7ifhOvmvLcdZ94l/beGtoxMylpYRahm08nbfTOKyoZa9O 4ag8Qlzq93+iUJEmtDjaXSrHC19TcUbwHgbklieJs8QNATyK7W13KXKQjG7ltOdn/QU+F0KTDSobA Qx0endfAUU/flJ7lh7bXBzgCy7Vhq7/i12U7bUC1e4RcMZrsHv3IEvkQMW8vhsgbh9qCvYeJc05L0 dBeNhIAiV4ZzXSB75npAPt5QA8F1jFLx/q5DlSZVexOJpoX7J/iYqc8bec9DIfkUTj9H0oZMUdlTS /9hubpI5g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liweb-000Hmp-KX; Tue, 18 May 2021 10:08:46 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLI-000E3k-9u for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=H1u3jivJEPCiVXSzPUEDO4O32WzfJt0Yji9f9bgjb4o=; b=dkQEGMjUGdNwoW53qxV4mZY+Nw NXe+WulyO3PKFUvVZlQF2m/x6a7s5Nwl3R5qsIDR5Th5kgZGkpuYnWGsiWkOpnZVC3GGTJr68aTiG prKg5/hKg8/+ra9cyuILJDomTyWhIKdV/0HTsLzd3PeH9HOqWEJI3vufgOoWSoLU4sphExYp9hRja 9pS8PTGkBX1DveT1+4b9IL6/viQcY7cJtaIgMdplQ8QQiwkA4XqMy+jvZgdbCsFw/fD3viveFNzul sLwcSNJmTgLwY6xBbXW44SkXJKonakuZDieR2aQKcpG8OzI5Vlavs4Bm1a3wjqpUZvV4BoFR4gBdE 2jpPDgDg==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLF-00EWny-NF for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:46 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5329561403; Tue, 18 May 2021 09:48:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331325; bh=s9Bz6q2x/QfoapW/bvF+kNoVHzS747ZjQm5kJMTQ8ug=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=h7zS00zStct6l9agEnN+I9aZki2iULUfO6eXw/xDHXgUQvmUu3xE/mUjpzT3DviMR 3amO2G0va/GqityTOw3Etvhd/5exc6AAUv0WSMO3L6kaRkWICFIdgeMhZjWGK816lk lkCsAcHxy5SKh+ZfEbO0jca7RfulSvydQQfkm2u0stlKeh2bGqmYsnbogjv1+RcZnQ RYsaP+5wET5HnX/2kYsZAaix6vD1ujhvVw7FfH7F94gkZ2T2Ny9c9sjuVZUQc1BLY+ ltA+vSja2r8ubSMaD2LcQ7Af0AvqYzrpweCKQ+qccs9EZ9XBDWjPs4k7uwcrTU0CT7 /ia6iySGKWlpA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 19/21] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Date: Tue, 18 May 2021 10:47:23 +0100 Message-Id: <20210518094725.7701-20-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024845_796020_EB5B550F X-CRM114-Status: GOOD ( 12.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow systems with mismatched 32-bit support at EL0 to run 32-bit applications based on a new kernel parameter. Signed-off-by: Will Deacon Acked-by: Catalin Marinas --- Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++ arch/arm64/kernel/cpufeature.c | 7 +++++++ 2 files changed, 15 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index cb89dbdedc46..a2e453919bb6 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -287,6 +287,14 @@ do not want to use tracing_snapshot_alloc() as it needs to be done where GFP_KERNEL allocations are allowed. + allow_mismatched_32bit_el0 [ARM64] + Allow execve() of 32-bit applications and setting of the + PER_LINUX32 personality on systems where only a strict + subset of the CPUs support 32-bit EL0. When this + parameter is present, the set of CPUs supporting 32-bit + EL0 is indicated by /sys/devices/system/cpu/aarch32_el0 + and hot-unplug operations may be restricted. + amd_iommu= [HW,X86-64] Pass parameters to the AMD IOMMU driver in the system. Possible values are: diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 72efdc611b14..f2c97baa050f 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1298,6 +1298,13 @@ const struct cpumask *system_32bit_el0_cpumask(void) return cpu_possible_mask; } +static int __init parse_32bit_el0_param(char *str) +{ + allow_mismatched_32bit_el0 = true; + return 0; +} +early_param("allow_mismatched_32bit_el0", parse_32bit_el0_param); + static ssize_t aarch32_el0_show(struct device *dev, struct device_attribute *attr, char *buf) { From patchwork Tue May 18 09:47:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86D90C433B4 for ; Tue, 18 May 2021 10:11:42 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1923A60BD3 for ; Tue, 18 May 2021 10:11:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1923A60BD3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gy56GjVXMKHSj/aTt6kVn9gQM25jTQDStUeUQcTbOXU=; b=ZOo3jvIuvOFG48jzs+BNUzwG+ cNFypkooIE3+OJRS4poV7GNZnpKvE59/JHr70GhlPOTqSFfDNITQ8E3UObnwzhmnh2WI+eEA2bUy/ xJ+1ciQ8XxAhwH8RCZq2SboWWdpuX1/w6qmirfOoL+ZYQAo67pllgh09iCiqwkQ8HnBDksqydJDuE RHvTI5WkR64e8QCe9lljZ41OMLCymGWwAN3I6yUHoXOVv60tEoTi+muQ6QU5RfWLBvovxBohmVX+W h5+aNY6XvlmjoaEUa5k4o37qN89mxtB565w5zZ0fSCDmfc7ifHRTmTmrWbaLGstTps4BT/bhC9++g 3Q+tO9MTQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwfQ-000HxN-Gb; Tue, 18 May 2021 10:09:36 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLM-000E4W-26 for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=IPBAwtBdNfOl2ou4HntULZzpESobs8OaXnbxndEUnHM=; b=kh98XLPBiNmQxxNuRAi4t6dw8d YyCOxCDb7l6kj64skFWDB2mhbmVvgCNBt1V4FMCDZOPjT4m1bPDHgX/LZyfiYWs4FoODkJFHTl7cg F16RM2A0mIvYWev9tHQ0HE/Qu7hH5AIJmHrX6jIrl5tVdyVtiNNPnKnJyHUFjekxaMHH5okx6riyL Bz3M4630/dkkIK/+SGS035jNvQWIaHS3KFS0hQYJwCmIX381vrWYgMOtZmBm5dI+fpWM6py1Ecu0z p9efThvKn6kBC2Ma2Gb3P3/y2WXk2CRPWWAYVBx5DtiW+kC0sPtZmDomG/ei1TZuuV56Ycbl2thR6 tQemnI/w==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLJ-00EWoW-CP for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:50 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id D7446613EB; Tue, 18 May 2021 09:48:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331329; bh=cxh0O3aWsznuhosSjWq+pFkb/mSjtZzHnDThwdnjZQg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GoWEGGqCZBOkoC2+SS/nheiXu/2Hmo+WC9UVd5Z+j0CNnv8h4OqnLXGjaGl7iNbry ShR1l32aTvoPswFIknrj2CogVOF2bOIy8UbnShkpro4zTF4Mdp8mHB6mLZ7YqKw2Sq w9LRuhE9hHajS9IrAYzQaanN21jDJo3euMA8ZbUGi9rgadI90kZ6lo4/KMWo8q/dQb Ebq0jOV4AnXerbs0ME7zLpmBDWxOotDxU+HPHaiQOld41NDrnWWqNmic55rsFPGAm+ J7pigE6MSeZTmxNgfxbMfZiMdMfYw1D7Py2GuJm8bLRwvsECD0VSpBEn9LwhvuqBfk 0FAbg8A9ApvZA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 20/21] arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores Date: Tue, 18 May 2021 10:47:24 +0100 Message-Id: <20210518094725.7701-21-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024849_482757_0E3DAECC X-CRM114-Status: GOOD ( 14.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The scheduler now knows enough about these braindead systems to place 32-bit tasks accordingly, so throw out the safety checks and allow the ret-to-user path to avoid do_notify_resume() if there is nothing to do. Signed-off-by: Will Deacon Acked-by: Catalin Marinas --- arch/arm64/kernel/process.c | 14 +------------- arch/arm64/kernel/signal.c | 26 -------------------------- 2 files changed, 1 insertion(+), 39 deletions(-) diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 8e0da06c4e77..de9833f08742 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -527,15 +527,6 @@ static void erratum_1418040_thread_switch(struct task_struct *prev, write_sysreg(val, cntkctl_el1); } -static void compat_thread_switch(struct task_struct *next) -{ - if (!is_compat_thread(task_thread_info(next))) - return; - - if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) - set_tsk_thread_flag(next, TIF_NOTIFY_RESUME); -} - static void update_sctlr_el1(u64 sctlr) { /* @@ -577,7 +568,6 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev, ssbs_thread_switch(next); erratum_1418040_thread_switch(prev, next); ptrauth_thread_switch_user(next); - compat_thread_switch(next); /* * Complete any pending TLB or cache maintenance on this CPU in case @@ -657,10 +647,8 @@ void arch_setup_new_exec(void) * at the point of execve(), although we try a bit harder to * honour the cpuset hierarchy. */ - if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { + if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) force_compatible_cpus_allowed_ptr(current); - set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); - } } else if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { relax_compatible_cpus_allowed_ptr(current); } diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index f8192f4ae0b8..6237486ff6bb 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -911,19 +911,6 @@ static void do_signal(struct pt_regs *regs) restore_saved_sigmask(); } -static bool cpu_affinity_invalid(struct pt_regs *regs) -{ - if (!compat_user_mode(regs)) - return false; - - /* - * We're preemptible, but a reschedule will cause us to check the - * affinity again. - */ - return !cpumask_test_cpu(raw_smp_processor_id(), - system_32bit_el0_cpumask()); -} - asmlinkage void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags) { @@ -951,19 +938,6 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, if (thread_flags & _TIF_NOTIFY_RESUME) { tracehook_notify_resume(regs); rseq_handle_notify_resume(NULL, regs); - - /* - * If we reschedule after checking the affinity - * then we must ensure that TIF_NOTIFY_RESUME - * is set so that we check the affinity again. - * Since tracehook_notify_resume() clears the - * flag, ensure that the compiler doesn't move - * it after the affinity check. - */ - barrier(); - - if (cpu_affinity_invalid(regs)) - force_sig(SIGKILL); } if (thread_flags & _TIF_FOREIGN_FPSTATE) From patchwork Tue May 18 09:47:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12264485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96417C43460 for ; Tue, 18 May 2021 10:12:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1B06F61185 for ; Tue, 18 May 2021 10:12:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1B06F61185 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Gme75XANzf3Ifobwt+8XeMi3Va2ZfIWEFHCFiFwk5v8=; b=VghvJilcQ45J62tVfzAgtXUy+ QPvSb0LQwYeuaIQZrYgwdpxiH0MPqYPZakbWbR58cyzwzO7Jpbd1Cv+dCgBQIUlrg0GIVFZyDcYXu kWLcc7cmD760cvKBuzmq7s8npHnf8odCbyiYzqlKpmQ0INgZHqj+inj0p52SRr95oKY4LO4wFNYoU v3+emeDtwCQxcOIrmI3Hk9htREaQu3HXVToX2D+vSntf42tK1nQymbk/iBUT8a54L60vmf95l53Li h9cs5VMLqLyhLCX9qQ/uQnhodUjds+Ok+HbwOcCt5UYE2/wpFVaPsDHJJBiSx49MMOnE3WUQS8aI1 djUaPfMKw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1liwgK-000ID1-QM; Tue, 18 May 2021 10:10:33 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLP-000E5G-Vi for linux-arm-kernel@desiato.infradead.org; Tue, 18 May 2021 09:48:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=yANpBrPC8xtK64IcMuS6drLEs04eN8sfpSmdS9Ywib0=; b=Aj52/Q6jCWIFDoSCHm5SczokcQ C71c9y2WUzglV7n8ZTvG5NBoJ3gCjbc9AcByzzWzcWvXnYjhCqBZlQV4LSB+xFprcVzj2tIxsmt4I Kq9kKupm1n7UfBkm3WyQhfd8P55W0moxK6uUCZVYNdW9bxQEaaFLFOfVRryJpTOjYNlTzbhQ7NyYg de9ng4W87gBmXOMjNBquudOE+GaxJG2REXl9Lzuy/lXcQ6E61eSGM2//BCkBlDp9XvZl+HquUGPKH lkfwxygDb3EeDrrz/oEmdJj8LUKv8OhObF/M132YxSiKBjLebOMxKdBemmRh8lpo03Bw3uqPKtIam 9M1L5rWQ==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1liwLM-00EWpC-Tw for linux-arm-kernel@lists.infradead.org; Tue, 18 May 2021 09:48:54 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8011561404; Tue, 18 May 2021 09:48:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621331332; bh=cLV1ruPvNZQWbLhrPTW5PTi59kS93zO0HxNZ8xc2n5I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PpUmRDuQPotlvWn6Zdm8S/5OhittrHyEVMvHp0uVcz4ZQfZit07yzmo/gvmt5fJsE cANOh+AlooAZYy7DCWLxMZulfgk9zCvL47X8Tn92OpSXCSPJeD3/tzsPsoTZXrUQkC vqHcsROicuUlVI0xVzlbqTLW7e+1kAKB8hVDeVnGS1ZX22G1ESF0XRx9ExjyuyR1OM gvoxgDN/VeMMK2hlyigf3JRWaHnKV60msIvhPv6RakToZXn5bVmQZpnB46ht3jbu8Z 5C6SSpXjuLXDH9n7Hwurn7z7rPVcR4gNK9M0rCfL/1BZha1gy5IxmVW1IvUiZCR5YY CpM3nGLV19GbQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: [PATCH v6 21/21] Documentation: arm64: describe asymmetric 32-bit support Date: Tue, 18 May 2021 10:47:25 +0100 Message-Id: <20210518094725.7701-22-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210518094725.7701-1-will@kernel.org> References: <20210518094725.7701-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210518_024853_031118_694B2100 X-CRM114-Status: GOOD ( 32.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Document support for running 32-bit tasks on asymmetric 32-bit systems and its impact on the user ABI when enabled. Signed-off-by: Will Deacon Reviewed-by: Catalin Marinas --- .../admin-guide/kernel-parameters.txt | 3 + Documentation/arm64/asymmetric-32bit.rst | 149 ++++++++++++++++++ Documentation/arm64/index.rst | 1 + 3 files changed, 153 insertions(+) create mode 100644 Documentation/arm64/asymmetric-32bit.rst diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index a2e453919bb6..5a1dc7e628a5 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -295,6 +295,9 @@ EL0 is indicated by /sys/devices/system/cpu/aarch32_el0 and hot-unplug operations may be restricted. + See Documentation/arm64/asymmetric-32bit.rst for more + information. + amd_iommu= [HW,X86-64] Pass parameters to the AMD IOMMU driver in the system. Possible values are: diff --git a/Documentation/arm64/asymmetric-32bit.rst b/Documentation/arm64/asymmetric-32bit.rst new file mode 100644 index 000000000000..baf02c143363 --- /dev/null +++ b/Documentation/arm64/asymmetric-32bit.rst @@ -0,0 +1,149 @@ +====================== +Asymmetric 32-bit SoCs +====================== + +Author: Will Deacon + +This document describes the impact of asymmetric 32-bit SoCs on the +execution of 32-bit (``AArch32``) applications. + +Date: 2021-05-17 + +Introduction +============ + +Some Armv9 SoCs suffer from a big.LITTLE misfeature where only a subset +of the CPUs are capable of executing 32-bit user applications. On such +a system, Linux by default treats the asymmetry as a "mismatch" and +disables support for both the ``PER_LINUX32`` personality and +``execve(2)`` of 32-bit ELF binaries, with the latter returning +``-ENOEXEC``. If the mismatch is detected during late onlining of a +64-bit-only CPU, then the onlining operation fails and the new CPU is +unavailable for scheduling. + +Surprisingly, these SoCs have been produced with the intention of +running legacy 32-bit binaries. Unsurprisingly, that doesn't work very +well with the default behaviour of Linux. + +It seems inevitable that future SoCs will drop 32-bit support +altogether, so if you're stuck in the unenviable position of needing to +run 32-bit code on one of these transitionary platforms then you would +be wise to consider alternatives such as recompilation, emulation or +retirement. If neither of those options are practical, then read on. + +Enabling kernel support +======================= + +Since the kernel support is not completely transparent to userspace, +allowing 32-bit tasks to run on an asymmetric 32-bit system requires an +explicit "opt-in" and can be enabled by passing the +``allow_mismatched_32bit_el0`` parameter on the kernel command-line. + +For the remainder of this document we will refer to an *asymmetric +system* to mean an SoC running Linux with this kernel command-line +option enabled. + +Userspace impact +================ + +32-bit tasks running on an asymmetric system behave in mostly the same +way as on a homogeneous system, with a few key differences relating to +CPU affinity. + +sysfs +----- + +The subset of CPUs capable of running 32-bit tasks is described in +``/sys/devices/system/cpu/aarch32_el0`` and is documented further in +``Documentation/ABI/testing/sysfs-devices-system-cpu``. + +**Note:** CPUs are advertised by this file as they are detected and so +late-onlining of 32-bit-capable CPUs can result in the file contents +being modified by the kernel at runtime. Once advertised, CPUs are never +removed from the file. + +``execve(2)`` +------------- + +On a homogeneous system, the CPU affinity of a task is preserved across +``execve(2)``. This is not always possible on an asymmetric system, +specifically when the new program being executed is 32-bit yet the +affinity mask contains 64-bit-only CPUs. In this situation, the kernel +determines the new affinity mask as follows: + + 1. If the 32-bit-capable subset of the affinity mask is not empty, + then the affinity is restricted to that subset and the old affinity + mask is saved. This saved mask is inherited over ``fork(2)`` and + preserved across ``execve(2)`` of 32-bit programs. + + **Note:** This step does not apply to ``SCHED_DEADLINE`` tasks. + See `SCHED_DEADLINE`_. + + 2. Otherwise, the cpuset hierarchy of the task is walked until an + ancestor is found containing at least one 32-bit-capable CPU. The + affinity of the task is then changed to match the 32-bit-capable + subset of the cpuset determined by the walk. + + 3. On failure (i.e. out of memory), the affinity is changed to the set + of all 32-bit-capable CPUs of which the kernel is aware. + +A subsequent ``execve(2)`` of a 64-bit program by the 32-bit task will +invalidate the affinity mask saved in (1) and attempt to restore the CPU +affinity of the task using the saved mask if it was previously valid. +This restoration may fail due to intervening changes to the deadline +policy or cpuset hierarchy, in which case the ``execve(2)`` continues +with the affinity unchanged. + +Calls to ``sched_setaffinity(2)`` for a 32-bit task will consider only +the 32-bit-capable CPUs of the requested affinity mask. On success, the +affinity for the task is updated and any saved mask from a prior +``execve(2)`` is invalidated. + +``SCHED_DEADLINE`` +------------------ + +Admitting a 32-bit task to the deadline scheduler (e.g. by calling +``sched_setattr(2)``) will, if valid, consider the affinity mask saved +by a previous call to ``execve(2)`` for the purposes of input validation +in preference to the running affinity of the task. 64-bit deadline tasks +will skip step (1) of the process described in `execve(2)`_ when +executed a 32-bit program. + +**Note:** It is recommended that the 32-bit-capable CPUs are placed into +a separate root domain if ``SCHED_DEADLINE`` is to be used with 32-bit +tasks on an asymmetric system. Failure to do so is likely to result in +missed deadlines. + +Cpusets +------- + +The affinity of a 32-bit task may include CPUs that are not explicitly +allowed by the cpuset to which it is attached. This can occur as a +result of the following two situations: + + - A 64-bit task attached to a cpuset which allows only 64-bit CPUs + executes a 32-bit program. + + - All of the 32-bit-capable CPUs allowed by a cpuset containing a + 32-bit task are offlined. + +In both of these cases, the new affinity is calculated according to step +(2) of the process described in `execve(2)`_ and the cpuset hierarchy is +unchanged irrespective of the cgroup version. + +CPU hotplug +----------- + +When the kernel detects asymmetric 32-bit hardware, the first detected +32-bit-capable CPU is prevented from being offlined by userspace and any +such attempt will return ``-EPERM``. Note that suspend is still +permitted even if the primary CPU (i.e. CPU 0) is 64-bit-only. + +KVM +--- + +Although KVM will not advertise 32-bit EL0 support to any vCPUs on an +asymmetric system, a broken guest at EL1 could still attempt to execute +32-bit code at EL0. In this case, an exit from a vCPU thread in 32-bit +mode will return to host userspace with an ``exit_reason`` of +``KVM_EXIT_FAIL_ENTRY``. diff --git a/Documentation/arm64/index.rst b/Documentation/arm64/index.rst index 97d65ba12a35..4f840bac083e 100644 --- a/Documentation/arm64/index.rst +++ b/Documentation/arm64/index.rst @@ -10,6 +10,7 @@ ARM64 Architecture acpi_object_usage amu arm-acpi + asymmetric-32bit booting cpu-feature-registers elf_hwcaps