From patchwork Fri Jul 30 11:24:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA5BBC432BE for ; Fri, 30 Jul 2021 11:27:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B57516101C for ; Fri, 30 Jul 2021 11:27:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B57516101C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Za+XYNtqrx2OaDQP3YSJbBDKeWjGPP5Pxf/6PNves/Y=; b=T002nA/ELB5j0W swdUPUzmJH8kVNIH4GQ6QyBcwEa8RwKZaOoFt5kAQ8PUgLiEPvQs9NjCZW8BJwmTqEg7dLr5XTsSz ZGrUMBx4GU6WJ9leGZBefI/lN/iSz8QlzPBAT1zBCJa+YG5Iu7j8WaHDhY+mzEgyeGEkG/slxXsxU IHQdWTvMp/1SyrEJkSJH9kM2jQII82gF5G0i5AnO8nYo/QyRkZO1SDg1B/nPJMx5CjlnFKwXz4Qc+ zSOeHEX278CzdW1IKCsC+oKDLjEfaJKhy4CvVHyUmfCMAZZx8HXJgKNboS2zeoxgnkk7TeNLPf/Io pMg71x4+BJOTv1QZAKJA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QdV-008HOi-Nh; Fri, 30 Jul 2021 11:25:05 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QdL-008HLx-Fl for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:24:56 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8B8EB61054; Fri, 30 Jul 2021 11:24:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644295; bh=cMR+kmfN+LR4V8K3b19C27mYClwkFpH+EgWdYLvABQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HsfUg5e2IISXfWTjXPkysFNubh3ludK+aWAEQc8/gOUQbIfqlfyfNYSiaXBkAC0qP tfcZKhQa7coD2ecATLTGiLAYyOsLGzqwoqyGcCPvAAkbMcW7tKgLsOCtsaQWE2aSfn J2Q+5FBbzH3erHhvz6EMIalxK0NR+6ibob5jvwukU+gOSpfSiN3C6lclmDm1Xc+bOP 3JyHpCBxbYeOEWakm170h7e1cLsBffQDMWZQdDti1gOGUwGi6ySe3oauId4BWwnaO/ 9bZqf/Niqxsi9F6KU18HSjkXNaCAUVlUuvo/2Jj0B/W/H6YA6o5ZXTKLU8EUrxGA+K 9XMDKjVjTB0iQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com, Valentin Schneider Subject: [PATCH v11 01/16] sched: Introduce task_cpu_possible_mask() to limit fallback rq selection Date: Fri, 30 Jul 2021 12:24:28 +0100 Message-Id: <20210730112443.23245-2-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042455_602351_80B2076B X-CRM114-Status: GOOD ( 18.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. On such a system, we must take care not to migrate a task to an unsupported CPU when forcefully moving tasks in select_fallback_rq() in response to a CPU hot-unplug operation. Introduce a task_cpu_possible_mask() hook which, given a task argument, allows an architecture to return a cpumask of CPUs that are capable of executing that task. The default implementation returns the cpu_possible_mask, since sane machines do not suffer from per-cpu ISA limitations that affect scheduling. The new mask is used when selecting the fallback runqueue as a last resort before forcing a migration to the first active CPU. Reviewed-by: Valentin Schneider Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- include/linux/mmu_context.h | 14 ++++++++++++++ kernel/sched/core.c | 9 +++------ 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/include/linux/mmu_context.h b/include/linux/mmu_context.h index 03dee12d2b61..b9b970f7ab45 100644 --- a/include/linux/mmu_context.h +++ b/include/linux/mmu_context.h @@ -14,4 +14,18 @@ static inline void leave_mm(int cpu) { } #endif +/* + * CPUs that are capable of running user task @p. Must contain at least one + * active CPU. It is assumed that the kernel can run on all CPUs, so calling + * this for a kernel thread is pointless. + * + * By default, we assume a sane, homogeneous system. + */ +#ifndef task_cpu_possible_mask +# define task_cpu_possible_mask(p) cpu_possible_mask +# define task_cpu_possible(cpu, p) true +#else +# define task_cpu_possible(cpu, p) cpumask_test_cpu((cpu), task_cpu_possible_mask(p)) +#endif + #endif diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2d9ff40f4661..84b20feb3214 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2163,7 +2163,7 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu) /* Non kernel threads are not allowed during either online or offline. */ if (!(p->flags & PF_KTHREAD)) - return cpu_active(cpu); + return cpu_active(cpu) && task_cpu_possible(cpu, p); /* KTHREAD_IS_PER_CPU is always allowed. */ if (kthread_is_per_cpu(p)) @@ -3114,9 +3114,7 @@ static int select_fallback_rq(int cpu, struct task_struct *p) /* Look for allowed, online CPU in same node. */ for_each_cpu(dest_cpu, nodemask) { - if (!cpu_active(dest_cpu)) - continue; - if (cpumask_test_cpu(dest_cpu, p->cpus_ptr)) + if (is_cpu_allowed(p, dest_cpu)) return dest_cpu; } } @@ -3146,10 +3144,9 @@ static int select_fallback_rq(int cpu, struct task_struct *p) * * More yuck to audit. */ - do_set_cpus_allowed(p, cpu_possible_mask); + do_set_cpus_allowed(p, task_cpu_possible_mask(p)); state = fail; break; - case fail: BUG(); break; From patchwork Fri Jul 30 11:24:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A444C4338F for ; Fri, 30 Jul 2021 11:28:07 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DEFA06101C for ; Fri, 30 Jul 2021 11:28:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DEFA06101C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3cXJr1oYglbG/Q+7oD/YYL8+WN2ZTN6Gd7wrtT0YJ+k=; b=WHsVuUjRYiRwkA br2tAZ3ZH1/pRcMm31DVKDr4MnNuRs2H78uQoqlUAJmVmQmhwNp/VOHeb8wu08UwezUfv5sfh1NT5 JTz+z56O9r9vg7kUhjUhEVRpCjD3UVoAsfXqEcYRMdGrxCNIsTwkH5YMF6e7lF+FZw2VoBeDmjlv2 9HoXU24xKtpMzekVUgoP/gnj3bU3CRKM6gx2lr/g5orP+Ad6UnLypAcsbpKbBVkFiERj8TjIu//iR fq1KWsQ0Lto3AHfCDb7xjhECXJd3IAQvR2bRqqVo6GWKUHepEsCtJFuB318JPtd9L7pf0BdJom+YK UpbW/LRV051IHmDtAw+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qdg-008HRK-Jd; Fri, 30 Jul 2021 11:25:16 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QdP-008HMr-IE for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:00 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id AD1D561042; Fri, 30 Jul 2021 11:24:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644299; bh=K5kU/PzOO4ShY8PqsOhOT7CTKWSZRzwEMBYkXqNWnNg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TfkWA4ve4UekRNgF4b6qwssXf41d1X4H3+/o2bt4GnIYOK5np42/4VBwLEdXNDLYx 2V5mGkr1uACLZFXGuDPyqHAeUZtXxPmiMXnhCdRyQWZo9u1Ab+IXEtH/u3RvUk5ipx nWKChKqFetN8G4Dr0RKlnEBGCVm3t0GWTCacxjlcuwNQwfCL8i3jIjJW8raEBrGcnH vKeYcIenVzJ31vUMnhEjgf65Us4KhPjUx4lo9fiJXBqYU208iGnApCaG4RCkcwUsz1 tGHwDLhhpj+A47b5qu5eHkRb2UYK4aSu23TsZp9zSJOxWlsyZx317yVaef6qBlikdC 06Ha+10Md0ZnA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 02/16] cpuset: Don't use the cpu_possible_mask as a last resort for cgroup v1 Date: Fri, 30 Jul 2021 12:24:29 +0100 Message-Id: <20210730112443.23245-3-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042459_661701_53D5E4FB X-CRM114-Status: GOOD ( 13.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If the scheduler cannot find an allowed CPU for a task, cpuset_cpus_allowed_fallback() will widen the affinity to cpu_possible_mask if cgroup v1 is in use. In preparation for allowing architectures to provide their own fallback mask, just return early if we're either using cgroup v1 or we're using cgroup v2 with a mask that contains invalid CPUs. This will allow select_fallback_rq() to figure out the mask by itself. Cc: Tejun Heo Cc: Johannes Weiner Reviewed-by: Valentin Schneider Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- include/linux/cpuset.h | 1 + kernel/cgroup/cpuset.c | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index 04c20de66afc..ed6ec677dd6b 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #ifdef CONFIG_CPUSETS diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index adb5190c4429..6000d7fbf5da 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -3322,9 +3322,13 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) void cpuset_cpus_allowed_fallback(struct task_struct *tsk) { + const struct cpumask *cs_mask; + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); + rcu_read_lock(); - do_set_cpus_allowed(tsk, is_in_v2_mode() ? - task_cs(tsk)->cpus_allowed : cpu_possible_mask); + cs_mask = task_cs(tsk)->cpus_allowed; + if (is_in_v2_mode() && cpumask_subset(cs_mask, possible_mask)) + do_set_cpus_allowed(tsk, cs_mask); rcu_read_unlock(); /* From patchwork Fri Jul 30 11:24:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15E52C4338F for ; Fri, 30 Jul 2021 11:28:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D85C26103B for ; Fri, 30 Jul 2021 11:28:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D85C26103B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IZVX/Ksej7OaYTAeabj+iRcGCB8nbckrcZ7vnakWUCI=; b=bQHMv0Ft3Mx0lv UrDDvmJFnnUynxfzux6hUSfHkZH0zChHLN9QSaSoC8fuxKo0YnxLznXAJI3pq0QbLJua1LAzU1jw7 kA0dic9bI2CWh6jppK3H0XulA5NmSEp70XQ9xgmc1eNmunSv5dfqC6IV2lFCrkhhxxdZD9dzw7IkQ seSDY8HUeY3xk6niy8YqiasONq4T7y3RKXM2ZKuVBmHy/w9Fx+/1uv9hCMEnCkTkm8tsnbGC8HFl4 x1ZHRjcHPEwCpXd4pdj85M9TEtrVwErF1XS9eA+tyxZI1cErTCwV3C74i++sZWTLBbq+WJ20X+PbK QA1w32faNO//LfgMJJCw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qds-008HUf-LV; Fri, 30 Jul 2021 11:25:28 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QdT-008HOG-MF for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:05 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id B2E9A60E76; Fri, 30 Jul 2021 11:24:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644303; bh=C6JSXnDEz3VZcU1vdbB6SFjc0icBulmiQcPDWKZ8SX8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j00pYhSr+EPU1app3tsMBIN1Ykaq1NImjdZcW2OLc+1aY4UGjG5ejq/bS4Db8o9Tq uYiHTzjxTPlaIt3m/N5KqsRrVWYfE/Kg6NLFiP0SXBWCBUdEWsHE1ULkOndQ1uSg49 UmOMUL51YbYB1E1aFL8wt+w4oYtPShxuHp9WVv7leb+NsI97J7iMhkyWkWTXQQ61d+ 8qIMKtdSCAXD2qBsNz/mYsU6hT1Qoj0mDscBzUL6icHAaMyxCz7t1lwhCV16bfQgBM vWOLG4/3vI+2Bey/4uN9NRkzCVk80b9FLe02li7IVBIRgiGhxhOjvScRJ+esRwJuqF 4mzEETJcM/lVQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com, Valentin Schneider Subject: [PATCH v11 03/16] cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus() Date: Fri, 30 Jul 2021 12:24:30 +0100 Message-Id: <20210730112443.23245-4-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042503_844072_D3056A5F X-CRM114-Status: GOOD ( 20.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. Modify guarantee_online_cpus() to take task_cpu_possible_mask() into account when trying to find a suitable set of online CPUs for a given task. This will avoid passing an invalid mask to set_cpus_allowed_ptr() during ->attach() and will subsequently allow the cpuset hierarchy to be taken into account when forcefully overriding the affinity mask for a task which requires migration to a compatible CPU. Cc: Tejun Heo Cc: Johannes Weiner Reviewed-by: Valentin Schneider Signed-off-by: Will Deacon --- include/linux/cpuset.h | 2 +- kernel/cgroup/cpuset.c | 43 +++++++++++++++++++++++++----------------- 2 files changed, 27 insertions(+), 18 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index ed6ec677dd6b..414a8e694413 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -185,7 +185,7 @@ static inline void cpuset_read_unlock(void) { } static inline void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask) { - cpumask_copy(mask, cpu_possible_mask); + cpumask_copy(mask, task_cpu_possible_mask(p)); } static inline void cpuset_cpus_allowed_fallback(struct task_struct *p) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 6000d7fbf5da..3984284c76bd 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -372,18 +372,29 @@ static inline bool is_in_v2_mode(void) } /* - * Return in pmask the portion of a cpusets's cpus_allowed that - * are online. If none are online, walk up the cpuset hierarchy - * until we find one that does have some online cpus. + * Return in pmask the portion of a task's cpusets's cpus_allowed that + * are online and are capable of running the task. If none are found, + * walk up the cpuset hierarchy until we find one that does have some + * appropriate cpus. * * One way or another, we guarantee to return some non-empty subset * of cpu_online_mask. * * Call with callback_lock or cpuset_mutex held. */ -static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) +static void guarantee_online_cpus(struct task_struct *tsk, + struct cpumask *pmask) { - while (!cpumask_intersects(cs->effective_cpus, cpu_online_mask)) { + const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); + struct cpuset *cs; + + if (WARN_ON(!cpumask_and(pmask, possible_mask, cpu_online_mask))) + cpumask_copy(pmask, cpu_online_mask); + + rcu_read_lock(); + cs = task_cs(tsk); + + while (!cpumask_intersects(cs->effective_cpus, pmask)) { cs = parent_cs(cs); if (unlikely(!cs)) { /* @@ -393,11 +404,13 @@ static void guarantee_online_cpus(struct cpuset *cs, struct cpumask *pmask) * cpuset's effective_cpus is on its way to be * identical to cpu_online_mask. */ - cpumask_copy(pmask, cpu_online_mask); - return; + goto out_unlock; } } - cpumask_and(pmask, cs->effective_cpus, cpu_online_mask); + cpumask_and(pmask, pmask, cs->effective_cpus); + +out_unlock: + rcu_read_unlock(); } /* @@ -2199,15 +2212,13 @@ static void cpuset_attach(struct cgroup_taskset *tset) percpu_down_write(&cpuset_rwsem); - /* prepare for attach */ - if (cs == &top_cpuset) - cpumask_copy(cpus_attach, cpu_possible_mask); - else - guarantee_online_cpus(cs, cpus_attach); - guarantee_online_mems(cs, &cpuset_attach_nodemask_to); cgroup_taskset_for_each(task, css, tset) { + if (cs != &top_cpuset) + guarantee_online_cpus(task, cpus_attach); + else + cpumask_copy(cpus_attach, task_cpu_possible_mask(task)); /* * can_attach beforehand should guarantee that this doesn't * fail. TODO: have a better way to handle failure here @@ -3302,9 +3313,7 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) unsigned long flags; spin_lock_irqsave(&callback_lock, flags); - rcu_read_lock(); - guarantee_online_cpus(task_cs(tsk), pmask); - rcu_read_unlock(); + guarantee_online_cpus(tsk, pmask); spin_unlock_irqrestore(&callback_lock, flags); } From patchwork Fri Jul 30 11:24:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC7DEC4338F for ; Fri, 30 Jul 2021 11:28:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 901FD6101C for ; Fri, 30 Jul 2021 11:28:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 901FD6101C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zRzqskKriaDn3dZHq3r3Y5lqVtXHSoFDJnwzbdmafHU=; b=yFB94W9upXBhEF x4uyT4lSHgpofq61eTTrcCbllV/H2QLVk9R/KAYjSn0YQpeMfQ8fuqgFY3h6MKL0uE4s5NwPHv9oB YjHWDN845/K2I2lYwTY+mJgsz7JwfomsKGvBDlzEEJy8J52ay5RpOtXwUWDtXClxS+TlUfe8Asxn4 ajx+tgt0hRwokoxh9FQDhGU0r1GsqIz8WDpupTUG9LoYS7tIlLIck7yRK8uEKKSAmH9K9Sdw3qbX1 RNci+oXeYbAdUlRxf5LHbc8Zwbl686Z6ZsB9qwqckmKIOZ1MLZY577TaoqoA3Kkmnli6ROlT6Y84k OJAXxmPQJkm9/2Okh+oA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qe6-008HYb-47; Fri, 30 Jul 2021 11:25:42 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QdX-008HPK-MJ for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:09 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id D456E6101C; Fri, 30 Jul 2021 11:25:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644307; bh=0kH0eyvwCTWwHw6ma6y+fNZDarOFlKoPkj8nC+/vVWM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LDwxyVeegmrWV6Q4Zud4AtwqOcNq+BmxdITDZOnrJxXI+rYxfpo6AmWDmU0dt+fP8 K4uxYj99ahhzVcqEZ8skmUNQeC+MNQibaTi8K3w9SZwxrkwDGi0KlazVhBvELHAMUP Q/TJiKsTQQq5TlOhd/6yuMKeyg88QRUSxoCDkRHYKLLYePkkC7fLtCGUvPt4Bhsh/p zX9xt7PbgoL0AnEMDOyj5uhJLYFcnEERw583Wg79Z/zWK+AXVVlU4roddAqw3HL7E0 aG22z8Iw9od0YTsIT/YQ1JDceHydEA/63AQAUIiTc/PWN43QCcMTOLt76zi0mfa7AY w+Mw7Qkv7BzqA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 04/16] cpuset: Cleanup cpuset_cpus_allowed_fallback() use in select_fallback_rq() Date: Fri, 30 Jul 2021 12:24:31 +0100 Message-Id: <20210730112443.23245-5-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042507_815987_F4174014 X-CRM114-Status: GOOD ( 17.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org select_fallback_rq() only needs to recheck for an allowed CPU if the affinity mask of the task has changed since the last check. Return a 'bool' from cpuset_cpus_allowed_fallback() to indicate whether the affinity mask was updated, and use this to elide the allowed check when the mask has been left alone. No functional change. Suggested-by: Valentin Schneider Reviewed-by: Valentin Schneider Signed-off-by: Will Deacon --- include/linux/cpuset.h | 5 +++-- kernel/cgroup/cpuset.c | 10 ++++++++-- kernel/sched/core.c | 3 +-- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index 414a8e694413..d2b9c41c8edf 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -59,7 +59,7 @@ extern void cpuset_wait_for_hotplug(void); extern void cpuset_read_lock(void); extern void cpuset_read_unlock(void); extern void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask); -extern void cpuset_cpus_allowed_fallback(struct task_struct *p); +extern bool cpuset_cpus_allowed_fallback(struct task_struct *p); extern nodemask_t cpuset_mems_allowed(struct task_struct *p); #define cpuset_current_mems_allowed (current->mems_allowed) void cpuset_init_current_mems_allowed(void); @@ -188,8 +188,9 @@ static inline void cpuset_cpus_allowed(struct task_struct *p, cpumask_copy(mask, task_cpu_possible_mask(p)); } -static inline void cpuset_cpus_allowed_fallback(struct task_struct *p) +static inline bool cpuset_cpus_allowed_fallback(struct task_struct *p) { + return false; } static inline nodemask_t cpuset_mems_allowed(struct task_struct *p) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 3984284c76bd..e78271a1b2fa 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -3327,17 +3327,22 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask) * which will not contain a sane cpumask during cases such as cpu hotplugging. * This is the absolute last resort for the scheduler and it is only used if * _every_ other avenue has been traveled. + * + * Returns true if the affinity of @tsk was changed, false otherwise. **/ -void cpuset_cpus_allowed_fallback(struct task_struct *tsk) +bool cpuset_cpus_allowed_fallback(struct task_struct *tsk) { const struct cpumask *cs_mask; + bool changed = false; const struct cpumask *possible_mask = task_cpu_possible_mask(tsk); rcu_read_lock(); cs_mask = task_cs(tsk)->cpus_allowed; - if (is_in_v2_mode() && cpumask_subset(cs_mask, possible_mask)) + if (is_in_v2_mode() && cpumask_subset(cs_mask, possible_mask)) { do_set_cpus_allowed(tsk, cs_mask); + changed = true; + } rcu_read_unlock(); /* @@ -3357,6 +3362,7 @@ void cpuset_cpus_allowed_fallback(struct task_struct *tsk) * select_fallback_rq() will fix things ups and set cpu_possible_mask * if required. */ + return changed; } void __init cpuset_init_current_mems_allowed(void) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 84b20feb3214..9fd598b8dac5 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3131,8 +3131,7 @@ static int select_fallback_rq(int cpu, struct task_struct *p) /* No more Mr. Nice Guy. */ switch (state) { case cpuset: - if (IS_ENABLED(CONFIG_CPUSETS)) { - cpuset_cpus_allowed_fallback(p); + if (cpuset_cpus_allowed_fallback(p)) { state = possible; break; } From patchwork Fri Jul 30 11:24:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFEEDC4320E for ; Fri, 30 Jul 2021 11:27:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7DBDE6103B for ; Fri, 30 Jul 2021 11:27:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7DBDE6103B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RS0yFlbyyqqggOyEfd1ufNQJMvnifeoy9IBp+Xgc2RQ=; b=5A7H2sD4O2Bpnu tDM3OivqnhCsApT07SomxUvSjC29TUcR6EnbUz4IJXzp9B7D/QtBCQ6bN11FLFZNtywGQkmETktBG n5e2Fr9jJJIf5FEevCCdcrYIVJ79xQxlWNs4sOzbJUYINPqENl0I844DWXWEqOw+zaQTx/7j6ZN3S bDOoZEpSpYJaNnrIJppvG+hv5j+fGVE/ZPi5HKxfEJUnXzveXrePoLDX0BIOgFyOWCngez75fclJy /bNxuP+Yw0DbZDXrZWasQjw16NunaTwYkA1xiXSW4QwsPPyAPOndpED+2/j/c9OE9koCmOqmutMQS 5haJOaccT0UTQcBxjVqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QeP-008HiH-GO; Fri, 30 Jul 2021 11:26:01 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qdb-008HQK-QS for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:13 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id D444961050; Fri, 30 Jul 2021 11:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644311; bh=7LEh2aSH3zOeZLBW6RvesFNL9tENH6Ea6cYaqsgwy6w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ozkspCQJ9DpTuoIeGCM/4uUwKbVKkHbVMiC0KEvv6XbS1PVA/kZ6mdQ53FueMXklP WheblWgaAXsP74bkbNQTnN5+Vp57tDXVrVi89tYSEE2PDLjd8S7XI0M+OPRdlnJ8VE boZBzgHXOOlC2KyaByjBmOHZm4PDgoHR3HGXSowAcmxrwQDXSxtQxtptPPJoXMugd5 FgwFshx/uhv8iXf3lLXQCPdJ4H8BXRFJtbyDrwWZPR3Z1tYyHwGjEnk+ff+aCwPiad XRzYq4NTTAlBT0tDVDMuBlGB6UCq+DnsqcKFt6Eryf+kSSsZxzLSM1yr+Q57MrXzKE M/e//lQTmGnyA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com, Valentin Schneider Subject: [PATCH v11 05/16] sched: Reject CPU affinity changes based on task_cpu_possible_mask() Date: Fri, 30 Jul 2021 12:24:32 +0100 Message-Id: <20210730112443.23245-6-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042511_926849_A24E14D3 X-CRM114-Status: GOOD ( 15.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Reject explicit requests to change the affinity mask of a task via set_cpus_allowed_ptr() if the requested mask is not a subset of the mask returned by task_cpu_possible_mask(). This ensures that the 'cpus_mask' for a given task cannot contain CPUs which are incapable of executing it, except in cases where the affinity is forced. Reviewed-by: Valentin Schneider Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- kernel/sched/core.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 9fd598b8dac5..dbce9cd83a53 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2700,15 +2700,17 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, u32 flags) { const struct cpumask *cpu_valid_mask = cpu_active_mask; + const struct cpumask *cpu_allowed_mask = task_cpu_possible_mask(p); unsigned int dest_cpu; struct rq_flags rf; struct rq *rq; int ret = 0; + bool kthread = p->flags & PF_KTHREAD; rq = task_rq_lock(p, &rf); update_rq_clock(rq); - if (p->flags & PF_KTHREAD || is_migration_disabled(p)) { + if (kthread || is_migration_disabled(p)) { /* * Kernel threads are allowed on online && !active CPUs, * however, during cpu-hot-unplug, even these might get pushed @@ -2722,6 +2724,11 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, cpu_valid_mask = cpu_online_mask; } + if (!kthread && !cpumask_subset(new_mask, cpu_allowed_mask)) { + ret = -EINVAL; + goto out; + } + /* * Must re-check here, to close a race against __kthread_bind(), * sched_setaffinity() is not guaranteed to observe the flag. From patchwork Fri Jul 30 11:24:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9168C4338F for ; Fri, 30 Jul 2021 11:28:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7DD4460E93 for ; Fri, 30 Jul 2021 11:28:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7DD4460E93 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wtBv6jMiuZn6kyfI19RhI8fBPibpnrkDz5pf44cx3hs=; b=iGaDbVTqauj4Af TaJ/U9gJ8f+w4TDP2aX82r2jZTUnuAmqt2i48ruUhSGtbo6d6jRKpiZuCzckMotdUeri4+kNP4wyX 1t+N/cF3kNUEicpxvoPgN5/du795hofa8Q04XqctA0/eNmRqHTG6NKmNmxPJMWWUCmvgbEE3CQDlZ XE8zgwmtCgRew2ZFpzC8CAq6v+tXXlfSqvTe5NK9NZeSXQT0cvx+ZRWkzqzrWGgnkVerfor/lkfax dZmLRsOe/Ltc3Iu8JMRA/2+LdfNaDY8w/RZ1mgaz/pXqqn231tA5YY2m09CZ0sk2ZYs9MygEt4C5l PwYhIk1jQudb/bHfXilw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qej-008HpE-LA; Fri, 30 Jul 2021 11:26:22 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qdf-008HRO-VE for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:17 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 02DB561052; Fri, 30 Jul 2021 11:25:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644315; bh=c3XeuGdd25e7O6CSURs0DdZO6Yl72U7eD1ENjOcRYTY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ugGOPJSLGXplYmM4I/M7RglzRGfe0aNzpUoRwY8UwfAiVamtUKwfq+gNcSjlJFVk8 hX35+hVXy5hq5q8g0/i+XR+LWd4prJX3ehI7+MDjz7o1HJhSeYupSkPx8b62s1lAoM FsL+IK+RZZwHfcHlBCOBIWzal4jKbo6XkoZ8yJzbsADYBpjgEjtzVLlAtblHRK51bK fWyHlxxhiNrKo/bsTI2VBzVm5fGkcEz/qRS9CaGM+3QVwrxs9qdTWX5PUjrru1ewHR Uih868csnAwWnkOORyQp1TS3jEqB7rDFhp1T65haU7qZd8XUPYlFIcO3G8GrdMDW6E /uRxEIVb/+Vmw== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com, Valentin Schneider Subject: [PATCH v11 06/16] sched: Introduce task_struct::user_cpus_ptr to track requested affinity Date: Fri, 30 Jul 2021 12:24:33 +0100 Message-Id: <20210730112443.23245-7-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042516_088772_078919D2 X-CRM114-Status: GOOD ( 16.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for saving and restoring the user-requested CPU affinity mask of a task, add a new cpumask_t pointer to 'struct task_struct'. If the pointer is non-NULL, then the mask is copied across fork() and freed on task exit. Reviewed-by: Valentin Schneider Signed-off-by: Will Deacon --- include/linux/sched.h | 13 +++++++++++++ init/init_task.c | 1 + kernel/fork.c | 2 ++ kernel/sched/core.c | 20 ++++++++++++++++++++ 4 files changed, 36 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index ec8d07d88641..91dab7a62aa1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -748,6 +748,7 @@ struct task_struct { unsigned int policy; int nr_cpus_allowed; const cpumask_t *cpus_ptr; + cpumask_t *user_cpus_ptr; cpumask_t cpus_mask; void *migration_pending; #ifdef CONFIG_SMP @@ -1705,6 +1706,8 @@ extern int task_can_attach(struct task_struct *p, const struct cpumask *cs_cpus_ #ifdef CONFIG_SMP extern void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask); extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask); +extern int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node); +extern void release_user_cpus_ptr(struct task_struct *p); #else static inline void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) { @@ -1715,6 +1718,16 @@ static inline int set_cpus_allowed_ptr(struct task_struct *p, const struct cpuma return -EINVAL; return 0; } +static inline int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node) +{ + if (src->user_cpus_ptr) + return -EINVAL; + return 0; +} +static inline void release_user_cpus_ptr(struct task_struct *p) +{ + WARN_ON(p->user_cpus_ptr); +} #endif extern int yield_to(struct task_struct *p, bool preempt); diff --git a/init/init_task.c b/init/init_task.c index 562f2ef8d157..2d024066e27b 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -80,6 +80,7 @@ struct task_struct init_task .normal_prio = MAX_PRIO - 20, .policy = SCHED_NORMAL, .cpus_ptr = &init_task.cpus_mask, + .user_cpus_ptr = NULL, .cpus_mask = CPU_MASK_ALL, .nr_cpus_allowed= NR_CPUS, .mm = NULL, diff --git a/kernel/fork.c b/kernel/fork.c index bc94b2cc5995..bd0e165b8397 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -446,6 +446,7 @@ void put_task_stack(struct task_struct *tsk) void free_task(struct task_struct *tsk) { + release_user_cpus_ptr(tsk); scs_release(tsk); #ifndef CONFIG_THREAD_INFO_IN_TASK @@ -924,6 +925,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) #endif if (orig->cpus_ptr == &orig->cpus_mask) tsk->cpus_ptr = &tsk->cpus_mask; + dup_user_cpus_ptr(tsk, orig, node); /* * One for the user space visible state that goes away when reaped. diff --git a/kernel/sched/core.c b/kernel/sched/core.c index dbce9cd83a53..a139ed8be7e3 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2470,6 +2470,26 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) __do_set_cpus_allowed(p, new_mask, 0); } +int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, + int node) +{ + if (!src->user_cpus_ptr) + return 0; + + dst->user_cpus_ptr = kmalloc_node(cpumask_size(), GFP_KERNEL, node); + if (!dst->user_cpus_ptr) + return -ENOMEM; + + cpumask_copy(dst->user_cpus_ptr, src->user_cpus_ptr); + return 0; +} + +void release_user_cpus_ptr(struct task_struct *p) +{ + kfree(p->user_cpus_ptr); + p->user_cpus_ptr = NULL; +} + /* * This function is wildly self concurrent; here be dragons. * From patchwork Fri Jul 30 11:24:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F93DC4338F for ; Fri, 30 Jul 2021 11:28:36 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4429A6101C for ; Fri, 30 Jul 2021 11:28:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4429A6101C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AitBC29os0ptxPiVlaqS6Fk0mBnB9tToq2shbQEgv1A=; b=d7mMdsANQ2w6+X vYhEJhbfgASam6epr+ynOeEZmsdiTdalILWAfS3jRpjv4BG/4HaS/sPf45z+MJ7D/MLAY/0HYj/vR qDwl15F+6tw7fZ4kSDBhFj5IMpocNO4T6UYM3vgsuWkmulwiubastcRrSvjH6VjaSDtg/wVYGlwP8 LxEWltHTzTYP6SV+K770wrhL3RXXXEmZqX7DrgloREeQXg4Ph3XAJO0zhy4AUXqPCo+8FahcE9d+0 D9Jp4OgpxLIK36bwJyr4+fwvfOsogaQR8UWqjGRytr3gL2VIBt47aLYf5Xhf6HMqkyY4zg35v1drg 2xAglKcG/MHz67JLs9rQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QfN-008I46-PM; Fri, 30 Jul 2021 11:27:03 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qdk-008HSt-3k for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:21 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2379E6101C; Fri, 30 Jul 2021 11:25:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644319; bh=WAepL21rbrjFTc8SrzUcBXFOjwYdNAPaTXAlLZdQFdI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fRDl5aUnYj4076IBQA1FmhVXpz1lWR0ifeC9iVcFXEqGhYTIOrfVEb8WKaLM0hZrs 8nasfVicG9X/R8r0UTrqjxQGVexRJzkxLLj9HakUQYWNMyBZGBIZopV2dveAGn98hp DEqDIGV9CKbSxcMWj2KbICdy0mskfKOhsWvzc8FGqXkCQNXgtg2C53jkpePrO8rENz ezUOijMz83fncmcdr4xt7oIPzG5xgWrFUH4oAuSxp0K6igcvFemC+5kFcZBgemLJyC o5nTgsah4cITIyQNoHxVGwwYLD9HgYbe5EP0zQ9vLjMaXeiLxeLSQ1zPLlpi2vSlHD lghg3IqatMzIQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com, Valentin Schneider Subject: [PATCH v11 07/16] sched: Split the guts of sched_setaffinity() into a helper function Date: Fri, 30 Jul 2021 12:24:34 +0100 Message-Id: <20210730112443.23245-8-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042520_228282_F41AD3D7 X-CRM114-Status: GOOD ( 17.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for replaying user affinity requests using a saved mask, split sched_setaffinity() up so that the initial task lookup and security checks are only performed when the request is coming directly from userspace. Reviewed-by: Valentin Schneider Signed-off-by: Will Deacon --- kernel/sched/core.c | 105 ++++++++++++++++++++++++-------------------- 1 file changed, 57 insertions(+), 48 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a139ed8be7e3..d4219d366103 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7578,53 +7578,22 @@ SYSCALL_DEFINE4(sched_getattr, pid_t, pid, struct sched_attr __user *, uattr, return retval; } -long sched_setaffinity(pid_t pid, const struct cpumask *in_mask) +static int +__sched_setaffinity(struct task_struct *p, const struct cpumask *mask) { - cpumask_var_t cpus_allowed, new_mask; - struct task_struct *p; int retval; + cpumask_var_t cpus_allowed, new_mask; - rcu_read_lock(); - - p = find_process_by_pid(pid); - if (!p) { - rcu_read_unlock(); - return -ESRCH; - } - - /* Prevent p going away */ - get_task_struct(p); - rcu_read_unlock(); + if (!alloc_cpumask_var(&cpus_allowed, GFP_KERNEL)) + return -ENOMEM; - if (p->flags & PF_NO_SETAFFINITY) { - retval = -EINVAL; - goto out_put_task; - } - if (!alloc_cpumask_var(&cpus_allowed, GFP_KERNEL)) { - retval = -ENOMEM; - goto out_put_task; - } if (!alloc_cpumask_var(&new_mask, GFP_KERNEL)) { retval = -ENOMEM; goto out_free_cpus_allowed; } - retval = -EPERM; - if (!check_same_owner(p)) { - rcu_read_lock(); - if (!ns_capable(__task_cred(p)->user_ns, CAP_SYS_NICE)) { - rcu_read_unlock(); - goto out_free_new_mask; - } - rcu_read_unlock(); - } - - retval = security_task_setscheduler(p); - if (retval) - goto out_free_new_mask; - cpuset_cpus_allowed(p, cpus_allowed); - cpumask_and(new_mask, in_mask, cpus_allowed); + cpumask_and(new_mask, mask, cpus_allowed); /* * Since bandwidth control happens on root_domain basis, @@ -7645,23 +7614,63 @@ long sched_setaffinity(pid_t pid, const struct cpumask *in_mask) #endif again: retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK); + if (retval) + goto out_free_new_mask; - if (!retval) { - cpuset_cpus_allowed(p, cpus_allowed); - if (!cpumask_subset(new_mask, cpus_allowed)) { - /* - * We must have raced with a concurrent cpuset - * update. Just reset the cpus_allowed to the - * cpuset's cpus_allowed - */ - cpumask_copy(new_mask, cpus_allowed); - goto again; - } + cpuset_cpus_allowed(p, cpus_allowed); + if (!cpumask_subset(new_mask, cpus_allowed)) { + /* + * We must have raced with a concurrent cpuset update. + * Just reset the cpumask to the cpuset's cpus_allowed. + */ + cpumask_copy(new_mask, cpus_allowed); + goto again; } + out_free_new_mask: free_cpumask_var(new_mask); out_free_cpus_allowed: free_cpumask_var(cpus_allowed); + return retval; +} + +long sched_setaffinity(pid_t pid, const struct cpumask *in_mask) +{ + struct task_struct *p; + int retval; + + rcu_read_lock(); + + p = find_process_by_pid(pid); + if (!p) { + rcu_read_unlock(); + return -ESRCH; + } + + /* Prevent p going away */ + get_task_struct(p); + rcu_read_unlock(); + + if (p->flags & PF_NO_SETAFFINITY) { + retval = -EINVAL; + goto out_put_task; + } + + if (!check_same_owner(p)) { + rcu_read_lock(); + if (!ns_capable(__task_cred(p)->user_ns, CAP_SYS_NICE)) { + rcu_read_unlock(); + retval = -EPERM; + goto out_put_task; + } + rcu_read_unlock(); + } + + retval = security_task_setscheduler(p); + if (retval) + goto out_put_task; + + retval = __sched_setaffinity(p, in_mask); out_put_task: put_task_struct(p); return retval; From patchwork Fri Jul 30 11:24:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21B30C4338F for ; Fri, 30 Jul 2021 11:29:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E3E4F6103B for ; Fri, 30 Jul 2021 11:29:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E3E4F6103B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bIdSxyUWSlcmlAf584mzgOnNqkFzBLj3ZbZpxqemXxE=; b=rRqQyZNLScBOMU I4wYg8Di0kNkdWMGBvgmXLiU0M6P61MbCPl17YsaoC/q50TrwANfXgPhyn0zl+J+DYvUo/6YjFVvg xWqPc5F4zNPkYryeXvMaKHB+D8oeul9+XXC6d5vFbjqg8kO6eY2AOZXUJgAgEs6p91Y/vlvjtxH7n WT9S28Ev7nhwDHIMoV37EHd3WLbyTY1gEFTMHegFI8/LbQneZjFbFXCcwyvZdljYTey+5/okYnrQ0 1KvL7Pi6JNFQs51WIw/QqXk89rGQ5+pSTsFFx/cQWs9iYEjTSX+yDzToOCDXjP2Z9oaZu5pOosVWe MLvPOslyKJLyEIclObaA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qg7-008IUa-W4; Fri, 30 Jul 2021 11:27:48 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qdo-008HTs-42 for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:26 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4618861052; Fri, 30 Jul 2021 11:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644323; bh=/xV+B6YkGUKsuRqCsRod36aQoKNwqGQIrjvP2SH2eIM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kSKAsUARHBkBdoEoome15laphgjsBNjIOy44tgDvvLkw92V9SQCFCOEd6CzDEGDwF 65T42EsQmO4YGI5M6yMmKo5zDXPcPXM0tlXwxME892lebrvnKjnHHlAHEFFNOKZURR AbDJ3U3R7KezD3a5KjYDmMTLKfWNVRKQYKUZz57q71HqK2Ofpkjtg8Mm8q9g895FJB +vZe9JyULJ62f8p4r00EtLwQJUbm4l68IAfuPLLorxH3ywAuktSKoqyNEgMU37jsWU f8nAXE61OVPFfkmXs35Rx17o8LdWSj2qFOjvRxIbw6hDvuj1lREw6W9QJSa4y7Wjkr 5o+aTpPO4icZQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 08/16] sched: Allow task CPU affinity to be restricted on asymmetric systems Date: Fri, 30 Jul 2021 12:24:35 +0100 Message-Id: <20210730112443.23245-9-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042524_263150_D8E908B2 X-CRM114-Status: GOOD ( 33.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Asymmetric systems may not offer the same level of userspace ISA support across all CPUs, meaning that some applications cannot be executed by some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do not feature support for 32-bit applications on both clusters. Although userspace can carefully manage the affinity masks for such tasks, one place where it is particularly problematic is execve() because the CPU on which the execve() is occurring may be incompatible with the new application image. In such a situation, it is desirable to restrict the affinity mask of the task and ensure that the new image is entered on a compatible CPU. From userspace's point of view, this looks the same as if the incompatible CPUs have been hotplugged off in the task's affinity mask. Similarly, if a subsequent execve() reverts to a compatible image, then the old affinity is restored if it is still valid. In preparation for restricting the affinity mask for compat tasks on arm64 systems without uniform support for 32-bit applications, introduce {force,relax}_compatible_cpus_allowed_ptr(), which respectively restrict and restore the affinity mask for a task based on the compatible CPUs. Reviewed-by: Valentin Schneider Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- include/linux/sched.h | 2 + kernel/sched/core.c | 180 ++++++++++++++++++++++++++++++++++++++---- kernel/sched/sched.h | 1 + 3 files changed, 167 insertions(+), 16 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 91dab7a62aa1..2ebe3d6f8f0c 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1708,6 +1708,8 @@ extern void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask); extern int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node); extern void release_user_cpus_ptr(struct task_struct *p); +extern void force_compatible_cpus_allowed_ptr(struct task_struct *p); +extern void relax_compatible_cpus_allowed_ptr(struct task_struct *p); #else static inline void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask) { diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d4219d366103..aec75ec1d257 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2707,27 +2707,22 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag } /* - * Change a given task's CPU affinity. Migrate the thread to a - * proper CPU and schedule it away if the CPU it's executing on - * is removed from the allowed bitmask. - * - * NOTE: the caller must have a valid reference to the task, the - * task must not exit() & deallocate itself prematurely. The - * call is not atomic; no spinlocks may be held. + * Called with both p->pi_lock and rq->lock held; drops both before returning. */ -static int __set_cpus_allowed_ptr(struct task_struct *p, - const struct cpumask *new_mask, - u32 flags) +static int __set_cpus_allowed_ptr_locked(struct task_struct *p, + const struct cpumask *new_mask, + u32 flags, + struct rq *rq, + struct rq_flags *rf) + __releases(rq->lock) + __releases(p->pi_lock) { const struct cpumask *cpu_valid_mask = cpu_active_mask; const struct cpumask *cpu_allowed_mask = task_cpu_possible_mask(p); unsigned int dest_cpu; - struct rq_flags rf; - struct rq *rq; int ret = 0; bool kthread = p->flags & PF_KTHREAD; - rq = task_rq_lock(p, &rf); update_rq_clock(rq); if (kthread || is_migration_disabled(p)) { @@ -2783,20 +2778,173 @@ static int __set_cpus_allowed_ptr(struct task_struct *p, __do_set_cpus_allowed(p, new_mask, flags); - return affine_move_task(rq, p, &rf, dest_cpu, flags); + if (flags & SCA_USER) + release_user_cpus_ptr(p); + + return affine_move_task(rq, p, rf, dest_cpu, flags); out: - task_rq_unlock(rq, p, &rf); + task_rq_unlock(rq, p, rf); return ret; } +/* + * Change a given task's CPU affinity. Migrate the thread to a + * proper CPU and schedule it away if the CPU it's executing on + * is removed from the allowed bitmask. + * + * NOTE: the caller must have a valid reference to the task, the + * task must not exit() & deallocate itself prematurely. The + * call is not atomic; no spinlocks may be held. + */ +static int __set_cpus_allowed_ptr(struct task_struct *p, + const struct cpumask *new_mask, u32 flags) +{ + struct rq_flags rf; + struct rq *rq; + + rq = task_rq_lock(p, &rf); + return __set_cpus_allowed_ptr_locked(p, new_mask, flags, rq, &rf); +} + int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask) { return __set_cpus_allowed_ptr(p, new_mask, 0); } EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr); +/* + * Change a given task's CPU affinity to the intersection of its current + * affinity mask and @subset_mask, writing the resulting mask to @new_mask + * and pointing @p->user_cpus_ptr to a copy of the old mask. + * If the resulting mask is empty, leave the affinity unchanged and return + * -EINVAL. + */ +static int restrict_cpus_allowed_ptr(struct task_struct *p, + struct cpumask *new_mask, + const struct cpumask *subset_mask) +{ + struct rq_flags rf; + struct rq *rq; + int err; + struct cpumask *user_mask = NULL; + + if (!p->user_cpus_ptr) { + user_mask = kmalloc(cpumask_size(), GFP_KERNEL); + + if (!user_mask) + return -ENOMEM; + } + + rq = task_rq_lock(p, &rf); + + /* + * Forcefully restricting the affinity of a deadline task is + * likely to cause problems, so fail and noisily override the + * mask entirely. + */ + if (task_has_dl_policy(p) && dl_bandwidth_enabled()) { + err = -EPERM; + goto err_unlock; + } + + if (!cpumask_and(new_mask, &p->cpus_mask, subset_mask)) { + err = -EINVAL; + goto err_unlock; + } + + /* + * We're about to butcher the task affinity, so keep track of what + * the user asked for in case we're able to restore it later on. + */ + if (user_mask) { + cpumask_copy(user_mask, p->cpus_ptr); + p->user_cpus_ptr = user_mask; + } + + return __set_cpus_allowed_ptr_locked(p, new_mask, 0, rq, &rf); + +err_unlock: + task_rq_unlock(rq, p, &rf); + kfree(user_mask); + return err; +} + +/* + * Restrict the CPU affinity of task @p so that it is a subset of + * task_cpu_possible_mask() and point @p->user_cpu_ptr to a copy of the + * old affinity mask. If the resulting mask is empty, we warn and walk + * up the cpuset hierarchy until we find a suitable mask. + */ +void force_compatible_cpus_allowed_ptr(struct task_struct *p) +{ + cpumask_var_t new_mask; + const struct cpumask *override_mask = task_cpu_possible_mask(p); + + alloc_cpumask_var(&new_mask, GFP_KERNEL); + + /* + * __migrate_task() can fail silently in the face of concurrent + * offlining of the chosen destination CPU, so take the hotplug + * lock to ensure that the migration succeeds. + */ + cpus_read_lock(); + if (!cpumask_available(new_mask)) + goto out_set_mask; + + if (!restrict_cpus_allowed_ptr(p, new_mask, override_mask)) + goto out_free_mask; + + /* + * We failed to find a valid subset of the affinity mask for the + * task, so override it based on its cpuset hierarchy. + */ + cpuset_cpus_allowed(p, new_mask); + override_mask = new_mask; + +out_set_mask: + if (printk_ratelimit()) { + printk_deferred("Overriding affinity for process %d (%s) to CPUs %*pbl\n", + task_pid_nr(p), p->comm, + cpumask_pr_args(override_mask)); + } + + WARN_ON(set_cpus_allowed_ptr(p, override_mask)); +out_free_mask: + cpus_read_unlock(); + free_cpumask_var(new_mask); +} + +static int +__sched_setaffinity(struct task_struct *p, const struct cpumask *mask); + +/* + * Restore the affinity of a task @p which was previously restricted by a + * call to force_compatible_cpus_allowed_ptr(). This will clear (and free) + * @p->user_cpus_ptr. + * + * It is the caller's responsibility to serialise this with any calls to + * force_compatible_cpus_allowed_ptr(@p). + */ +void relax_compatible_cpus_allowed_ptr(struct task_struct *p) +{ + unsigned long flags; + struct cpumask *mask = p->user_cpus_ptr; + + /* + * Try to restore the old affinity mask. If this fails, then + * we free the mask explicitly to avoid it being inherited across + * a subsequent fork(). + */ + if (!mask || !__sched_setaffinity(p, mask)) + return; + + raw_spin_lock_irqsave(&p->pi_lock, flags); + release_user_cpus_ptr(p); + raw_spin_unlock_irqrestore(&p->pi_lock, flags); +} + void set_task_cpu(struct task_struct *p, unsigned int new_cpu) { #ifdef CONFIG_SCHED_DEBUG @@ -7613,7 +7761,7 @@ __sched_setaffinity(struct task_struct *p, const struct cpumask *mask) } #endif again: - retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK); + retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK | SCA_USER); if (retval) goto out_free_new_mask; diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 14a41a243f7b..e88c2d399f0d 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2234,6 +2234,7 @@ extern struct task_struct *pick_next_task_idle(struct rq *rq); #define SCA_CHECK 0x01 #define SCA_MIGRATE_DISABLE 0x02 #define SCA_MIGRATE_ENABLE 0x04 +#define SCA_USER 0x08 #ifdef CONFIG_SMP From patchwork Fri Jul 30 11:24:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02BF1C4338F for ; Fri, 30 Jul 2021 11:30:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C2D3060E93 for ; Fri, 30 Jul 2021 11:30:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C2D3060E93 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eqLaMGsQ77rCjiAcTF7/qzwK2hG3VGGlwctyYSjEGaE=; b=YI1lvCK6Cg45GA fNddeL44damAD6+wV7VZV6Wzg0UugL/s6fwdHLId3iWToEFiIzj6fV8WRD8OMANhpqXcI4Zo8Tvq2 IQUFdionOlE/CBM4aCs+a2pg9AF3qBlLojLZyPjDgKRqjZlwX//Y3QLHL/Dp4Kv0HbK3J/nACJReh 9t+5EHyr4vb3cessOqzrbi9bJGIcsZuXaZ3MvhSQ9AF0ZGJeAayEyRhzXm3VUCCqtpYMsD4QOwuXg i5LjZyOqEJXK0ritrtq5ZA7J0a4lxamqv3yTEalfM4mcE3WHEG/C/vCpaJNr4u8MbahFpBQs9I2ZT U9m6qvf9l3Q/rBwMjk+A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qgz-008J0M-UW; Fri, 30 Jul 2021 11:28:42 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qds-008HUj-3U for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:29 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 460B26101C; Fri, 30 Jul 2021 11:25:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644327; bh=kCuwgKXzoXBcro+Ij+LOlC2qKGyVspA9C9To3M75IWw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U+7htRhfZ0BZMQ4K1VLVCMPwr2rQKWP4+Rh6vjhdb8xQe4gQUUIIm9piXoHW410nu azUcBu3SLHBvQf7oHdgWcjH9J/mEcShpiARucX6q8iwFRZLpBnO+smgZO0EffeN74O zFP++MdFVoUVrEqGeTBT3JO6UINqMTvBCFjUTjTDTVB7PnVeydcrSx5fiv2ZCIrvEI hVJ60dzL29lnmfhbVH4gUfGIlQaw25KC/Hh4gHs8+nd+fUWrpyp0FfPWXf0QgnOEnv GbcPte3tDE6d6FIXoB0g3VW92fjCKEKIvo+BhNkSc/wVMgAS/9GxNosw9yj4O44BYn G93Qx3uQ6nvQg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 09/16] sched: Introduce dl_task_check_affinity() to check proposed affinity Date: Fri, 30 Jul 2021 12:24:36 +0100 Message-Id: <20210730112443.23245-10-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042528_270966_C35E4FED X-CRM114-Status: GOOD ( 16.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for restricting the affinity of a task during execve() on arm64, introduce a new dl_task_check_affinity() helper function to give an indication as to whether the restricted mask is admissible for a deadline task. Reviewed-by: Daniel Bristot de Oliveira Signed-off-by: Will Deacon --- include/linux/sched.h | 6 ++++++ kernel/sched/core.c | 46 +++++++++++++++++++++++++++---------------- 2 files changed, 35 insertions(+), 17 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 2ebe3d6f8f0c..6ecd02e2ca1e 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1708,6 +1708,7 @@ extern void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new extern int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask); extern int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, int node); extern void release_user_cpus_ptr(struct task_struct *p); +extern int dl_task_check_affinity(struct task_struct *p, const struct cpumask *mask); extern void force_compatible_cpus_allowed_ptr(struct task_struct *p); extern void relax_compatible_cpus_allowed_ptr(struct task_struct *p); #else @@ -1730,6 +1731,11 @@ static inline void release_user_cpus_ptr(struct task_struct *p) { WARN_ON(p->user_cpus_ptr); } + +static inline int dl_task_check_affinity(struct task_struct *p, const struct cpumask *mask) +{ + return 0; +} #endif extern int yield_to(struct task_struct *p, bool preempt); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index aec75ec1d257..9f576a67bc31 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7726,6 +7726,32 @@ SYSCALL_DEFINE4(sched_getattr, pid_t, pid, struct sched_attr __user *, uattr, return retval; } +#ifdef CONFIG_SMP +int dl_task_check_affinity(struct task_struct *p, const struct cpumask *mask) +{ + int ret = 0; + + /* + * If the task isn't a deadline task or admission control is + * disabled then we don't care about affinity changes. + */ + if (!task_has_dl_policy(p) || !dl_bandwidth_enabled()) + return 0; + + /* + * Since bandwidth control happens on root_domain basis, + * if admission test is enabled, we only admit -deadline + * tasks allowed to run on all the CPUs in the task's + * root_domain. + */ + rcu_read_lock(); + if (!cpumask_subset(task_rq(p)->rd->span, mask)) + ret = -EBUSY; + rcu_read_unlock(); + return ret; +} +#endif + static int __sched_setaffinity(struct task_struct *p, const struct cpumask *mask) { @@ -7743,23 +7769,9 @@ __sched_setaffinity(struct task_struct *p, const struct cpumask *mask) cpuset_cpus_allowed(p, cpus_allowed); cpumask_and(new_mask, mask, cpus_allowed); - /* - * Since bandwidth control happens on root_domain basis, - * if admission test is enabled, we only admit -deadline - * tasks allowed to run on all the CPUs in the task's - * root_domain. - */ -#ifdef CONFIG_SMP - if (task_has_dl_policy(p) && dl_bandwidth_enabled()) { - rcu_read_lock(); - if (!cpumask_subset(task_rq(p)->rd->span, new_mask)) { - retval = -EBUSY; - rcu_read_unlock(); - goto out_free_new_mask; - } - rcu_read_unlock(); - } -#endif + retval = dl_task_check_affinity(p, new_mask); + if (retval) + goto out_free_new_mask; again: retval = __set_cpus_allowed_ptr(p, new_mask, SCA_CHECK | SCA_USER); if (retval) From patchwork Fri Jul 30 11:24:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92791C4338F for ; Fri, 30 Jul 2021 11:31:28 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 63A9461040 for ; Fri, 30 Jul 2021 11:31:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 63A9461040 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DZ+z4wRozmogtaW2UCLFPAq6RK9Ur406uja8T0y2kI8=; b=WHMh/37Nh3M5Wb lVGw48lz2dmg3UldYPGGOqNhZ9G1Q06ahvStR1DW052jcbz14KhavU8zdb3J/U8htBN7cLgDXb5bd PzYXtdZsDZbtvIqsAe2menLZXKjYeaZmFU4Hc+DBH6jh2UEc12hG60SlF2A1d0niwMDZvcr0Pt3rP n+QFNizH3IGZWQAB9Rvigiz/u+o3lxQlxOFR4KMDPLHUoJfXFfxZU+5iH4gOF578CA20bYqnJEQOf WNnKXFsXH5K7s+usu4Fh2aT7jJXoPJ83uRCankWkNAPQ4TVv4a9PL8CzLDRKJ7tV6hXzLwLelMBhl uy0YdOlIp1A8H9ELGniQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QhZ-008JFr-45; Fri, 30 Jul 2021 11:29:18 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qdw-008HWG-2s for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:33 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 42F9C61042; Fri, 30 Jul 2021 11:25:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644331; bh=41tFYtagGI1Lhiz3zWu2NlhWV1lTY0kbWv/RrqQ/NNQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=txOI8EoYqTcyfYriEygEnpS9AUmzRS/hTVAMDDtPpJAOyk+pBnKsszcKdnWVETGKH RGxgHIhsmbHsRm/HBH3DtIZSKP90plIeYjtDiDFHw2FC6H0XQGNZUNC7F3RKsNBokF znlajlq4J94PvqVbJ1hJStn8e+7G5o5DpvBZAzJlsrWyw37a/ufBc7fGtTAogXuYTR AP4UAjopdDNR3KfKC7y1013aTpz2/QbALkTsI334N5DY3r4179mjP33AKFZEkKHEV4 w0TWsT82jr3z6oLc/lZaJPjWiSPuyntp28F2pzPlXvm8C9qq2Z3aNoZU3D22JVIG65 wP/YciuWaYtWw== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 10/16] arm64: Implement task_cpu_possible_mask() Date: Fri, 30 Jul 2021 12:24:37 +0100 Message-Id: <20210730112443.23245-11-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042532_214736_AADF7302 X-CRM114-Status: UNSURE ( 9.96 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Provide an implementation of task_cpu_possible_mask() so that we can prevent 64-bit-only cores being added to the 'cpus_mask' for compat tasks on systems with mismatched 32-bit support at EL0, Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon --- arch/arm64/include/asm/mmu_context.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index eeb210997149..f4ba93d4ffeb 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -231,6 +231,19 @@ switch_mm(struct mm_struct *prev, struct mm_struct *next, update_saved_ttbr0(tsk, next); } +static inline const struct cpumask * +task_cpu_possible_mask(struct task_struct *p) +{ + if (!static_branch_unlikely(&arm64_mismatched_32bit_el0)) + return cpu_possible_mask; + + if (!is_compat_thread(task_thread_info(p))) + return cpu_possible_mask; + + return system_32bit_el0_cpumask(); +} +#define task_cpu_possible_mask task_cpu_possible_mask + void verify_cpu_asid_bits(void); void post_ttbr_update_workaround(void); From patchwork Fri Jul 30 11:24:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21832C4338F for ; Fri, 30 Jul 2021 11:32:06 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D4B236103B for ; Fri, 30 Jul 2021 11:32:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D4B236103B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=j9i1hGXbSlKgUwU9bK9eR+FebipL86U8kamH99BzfvQ=; b=Fep4AwYtP9otuf Ge/e5jvf+4FEUhSXwn8q7qnYyr0Sqbsdm2peb43fDr1M9dZ+M7W1tPn2RtkYW6V/+R6DEWMF6C1XC R8m1tf57+qxZVRGs42J9h80v2CvF0UhhcxK3zoXYCQsqIVS8RXzT6Aiia596kSJrhR4TTbRfZkld7 DjlmuIPHhcuvQ1INjj5Ku+yBstKhukyQDMe+CmG4gIM2w6GsyLOIWB83i7aVeTeL/S9b9ZivwVXMV CJF8/mRgcLtKBIvZV/BWlQbNf8NzQ84MbvSC6j1ZixPLwNwzo8op++4ZsoOI9kORTAN9jGt1gvuzJ s+WDrR6QixzZ3EtF9ccA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qi2-008JSB-Qh; Fri, 30 Jul 2021 11:29:46 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qe0-008HXE-1n for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:37 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4105261052; Fri, 30 Jul 2021 11:25:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644335; bh=XYxWvbxyG0p9zfJU0ZUjM8nEm2BIbKxsvYb0x1zSw14=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oXn/lVUuDMaltds6be6oJVxjRZ/VBeOCSaFGWu6JFjW0/SyeQnRXW9UyXBsGaArEt azn5Fq0iRMy/W+Q+LhfiMIbvqB7IhREw/AzOJqzZaNwtZxjQw34UKEkGg2Gk159NXp OiQnmbbdtuVo9ZG6ldfBylNLA3OSOe8uQNqoHb02XF8CTDhFvv+OybPvkWw2CnAXJo iDd3J7eWI4VeVNu9+efYuSbX1Ak8M/t84b+orAXLy1nK6yNCv90kyZwv8TldoUDf5c bgg1yuwNQCEv5mhjIRJp+1n0n8RXRnvqMTRdmA0MZIgBhgEgXini2tXU591nOm+IKm ITZ5e1N3x5RYQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 11/16] arm64: exec: Adjust affinity for compat tasks with mismatched 32-bit EL0 Date: Fri, 30 Jul 2021 12:24:38 +0100 Message-Id: <20210730112443.23245-12-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042536_178677_F7BF6F12 X-CRM114-Status: GOOD ( 18.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When exec'ing a 32-bit task on a system with mismatched support for 32-bit EL0, try to ensure that it starts life on a CPU that can actually run it. Similarly, when exec'ing a 64-bit task on such a system, try to restore the old affinity mask if it was previously restricted. Reviewed-by: Daniel Bristot de Oliveira Reviewed-by: Quentin Perret Signed-off-by: Will Deacon --- arch/arm64/include/asm/elf.h | 6 ++---- arch/arm64/kernel/process.c | 39 +++++++++++++++++++++++++++++++++++- 2 files changed, 40 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h index 8d1c8dcb87fd..97932fbf973d 100644 --- a/arch/arm64/include/asm/elf.h +++ b/arch/arm64/include/asm/elf.h @@ -213,10 +213,8 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG]; /* AArch32 EABI. */ #define EF_ARM_EABI_MASK 0xff000000 -#define compat_elf_check_arch(x) (system_supports_32bit_el0() && \ - ((x)->e_machine == EM_ARM) && \ - ((x)->e_flags & EF_ARM_EABI_MASK)) - +int compat_elf_check_arch(const struct elf32_hdr *); +#define compat_elf_check_arch compat_elf_check_arch #define compat_start_thread compat_start_thread /* * Unlike the native SET_PERSONALITY macro, the compat version maintains diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index c8989b999250..583ee58f8c9c 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -579,6 +580,28 @@ unsigned long arch_align_stack(unsigned long sp) return sp & ~0xf; } +#ifdef CONFIG_COMPAT +int compat_elf_check_arch(const struct elf32_hdr *hdr) +{ + if (!system_supports_32bit_el0()) + return false; + + if ((hdr)->e_machine != EM_ARM) + return false; + + if (!((hdr)->e_flags & EF_ARM_EABI_MASK)) + return false; + + /* + * Prevent execve() of a 32-bit program from a deadline task + * if the restricted affinity mask would be inadmissible on an + * asymmetric system. + */ + return !static_branch_unlikely(&arm64_mismatched_32bit_el0) || + !dl_task_check_affinity(current, system_32bit_el0_cpumask()); +} +#endif + /* * Called from setup_new_exec() after (COMPAT_)SET_PERSONALITY. */ @@ -588,8 +611,22 @@ void arch_setup_new_exec(void) if (is_compat_task()) { mmflags = MMCF_AARCH32; - if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) + + /* + * Restrict the CPU affinity mask for a 32-bit task so that + * it contains only 32-bit-capable CPUs. + * + * From the perspective of the task, this looks similar to + * what would happen if the 64-bit-only CPUs were hot-unplugged + * at the point of execve(), although we try a bit harder to + * honour the cpuset hierarchy. + */ + if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { + force_compatible_cpus_allowed_ptr(current); set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); + } + } else if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { + relax_compatible_cpus_allowed_ptr(current); } current->mm->context.flags = mmflags; From patchwork Fri Jul 30 11:24:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B48F1C4338F for ; Fri, 30 Jul 2021 11:33:04 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7C8DC60EFE for ; Fri, 30 Jul 2021 11:33:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7C8DC60EFE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RJ0ckGBV0Yr/08hYP/0Yj8eSrZp9V+i2FOA3R4ZlRIY=; b=LqMAj/N6Rl4P+S cJ55VbeXrlLpHE4zwwLkKu3EhSdx3oqZM7AEtHJKwQVhRan1fpK2NkiP1tC+wKbL/e5539P9rg4JX NQW++uhw1dZF27FQ4lCVoDcwaJtPi9vPpWul4hXY2fodAOXxpekB69jgHl+z40PeBWbmtsS2hE211 GOgmM6Prk/83Z33KuH5o0wKYTkgBticDn2uej3k3lbxPJUhN/KYSv6gJNmUJblxNZiN3/cMgRB2qd oGmNtPOhAyLSJptYxCJ7hYtHF2QHxchgpk2BXkwIKq5Q84ureK/mGE4gInTFbsbomyk2n//cvSIBx J/lFJSE/nIOgzHy/hv7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qid-008JgA-Gi; Fri, 30 Jul 2021 11:30:24 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qe4-008HYX-2v for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:41 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3D9B76101C; Fri, 30 Jul 2021 11:25:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644339; bh=YiNgY8+kTBdYeqURjU5xY5wvK1Ur/VHfTq38waThEcQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B67CT5aFUcNhTgH3L8ctFQCwWykZZrvqOvBKcnUHgW/h50xBHxQhK4vIJaB3af6be C0fNTSFujlB5bT/U1j0qfXspeVQXejynDJKo84pFuJT7PZUYfpPIEtlPSzfV/k/6Jn J6H3MqxVthdE+HbgR+YLJKivoV1u9e6SRlfxRdqZfd30QD56lh3xP/is2iHEQHkYFe CDYjmoceAL5DkNqft0GS0VIw0LwL+Acf6BCz/GPGohEh3HJtxKcUqaAcAovRIZeJtt si06Z8RzXWazWG6MVHidhsOrEyuyQsUtcVDIb75Tn+KjRMtmjrrv7nUgML+DTJBBNm pWl+cdQvLiOMA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 12/16] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Date: Fri, 30 Jul 2021 12:24:39 +0100 Message-Id: <20210730112443.23245-13-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042540_223498_0DD18A39 X-CRM114-Status: GOOD ( 16.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we want to support 32-bit applications, then when we identify a CPU with mismatched 32-bit EL0 support we must ensure that we will always have an active 32-bit CPU available to us from then on. This is important for the scheduler, because is_cpu_allowed() will be constrained to 32-bit CPUs for compat tasks and forced migration due to a hotplug event will hang if no 32-bit CPUs are available. On detecting a mismatch, prevent offlining of either the mismatching CPU if it is 32-bit capable, or find the first active 32-bit capable CPU otherwise. Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon --- arch/arm64/kernel/cpufeature.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 125d5c9471ac..d99a29f52aa1 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -2900,15 +2900,38 @@ void __init setup_cpu_features(void) static int enable_mismatched_32bit_el0(unsigned int cpu) { + /* + * The first 32-bit-capable CPU we detected and so can no longer + * be offlined by userspace. -1 indicates we haven't yet onlined + * a 32-bit-capable CPU. + */ + static int lucky_winner = -1; + struct cpuinfo_arm64 *info = &per_cpu(cpu_data, cpu); bool cpu_32bit = id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0); if (cpu_32bit) { cpumask_set_cpu(cpu, cpu_32bit_el0_mask); static_branch_enable_cpuslocked(&arm64_mismatched_32bit_el0); - setup_elf_hwcaps(compat_elf_hwcaps); } + if (cpumask_test_cpu(0, cpu_32bit_el0_mask) == cpu_32bit) + return 0; + + if (lucky_winner >= 0) + return 0; + + /* + * We've detected a mismatch. We need to keep one of our CPUs with + * 32-bit EL0 online so that is_cpu_allowed() doesn't end up rejecting + * every CPU in the system for a 32-bit task. + */ + lucky_winner = cpu_32bit ? cpu : cpumask_any_and(cpu_32bit_el0_mask, + cpu_active_mask); + get_cpu_device(lucky_winner)->offline_disabled = true; + setup_elf_hwcaps(compat_elf_hwcaps); + pr_info("Asymmetric 32-bit EL0 support detected on CPU %u; CPU hot-unplug disabled on CPU %u\n", + cpu, lucky_winner); return 0; } From patchwork Fri Jul 30 11:24:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48540C4338F for ; Fri, 30 Jul 2021 11:33:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1005861040 for ; Fri, 30 Jul 2021 11:33:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1005861040 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LYA7q0YY6mg4LmX5ZxKDFjVNp0n3bWEQ1YPOkbPM3SQ=; b=3E/ikU+vbvxkq3 IgUl/MOiyuaFRqNXd8yXqyl06YYkcjSQYve/1tZNnrDSFOHybBK5DUIKNWmZdEc7Q0o6Unee5bbr0 Vl4OJv0y/0KLR8OD7APGxNiGtaNC3OsYtkH1oEjb+XCJAkvI1QcVJk55cq8tSSYQbta/p58eEZzS8 2a7dp2GiQSIbAklDFH1DA6jmiAsXHiDjr7V6XBsUKC5On5DI8XNJfYHj+rn9F8ePBmvmEdruW3Dfs vz0eRuHUJAxqXJTHRiw1meFvIahyB7fSg80uTW/yY1q7g7qIi38Pn/FWNEezqEjicoZaLRlqERnPY w2am1HUERmwD1aoONTpg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QjD-008Jzv-G8; Fri, 30 Jul 2021 11:30:59 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qe8-008HaB-21 for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:45 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3D69761055; Fri, 30 Jul 2021 11:25:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644343; bh=VcjaQPzybbGlvtzMZMZPB8b5mPi8TxhDPTMLtmuqXPU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uPJLNBAq9EWh5Ch+oqsURAMrDIMMJZJ0bHbeueS0fIIKjv+VuBdXp9WWHI7r05VSp vNf+qMi0pfqDEm2XlU4JVuXCOMlFs+dj4a3tWWpHqVAnRxGMikDDAU9JLTMYRbtoZZ UEmFwv9nWdLdiDGQvIu1/zG+n3808lISU6eodl+5HgJTwWbOHHRomS1i7f5h+7dLNm asuerZzynWzfMk4HTtUlZ/eQyllQDEyoq1q75WdFinOfmIl9kGTp1XOFm9pp8rp4RV qXdCa3Ni0Yh/iVvc3FIpVjPNcacdGn7EvNV9uVHGuMSYMq9ds/DyR32apaIhgamfYX iojvukfFf37yQ== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 13/16] arm64: Advertise CPUs capable of running 32-bit applications in sysfs Date: Fri, 30 Jul 2021 12:24:40 +0100 Message-Id: <20210730112443.23245-14-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042544_212582_CAD0F4D1 X-CRM114-Status: GOOD ( 13.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since 32-bit applications will be killed if they are caught trying to execute on a 64-bit-only CPU in a mismatched system, advertise the set of 32-bit capable CPUs to userspace in sysfs. Reviewed-by: Greg Kroah-Hartman Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon --- .../ABI/testing/sysfs-devices-system-cpu | 9 +++++++++ arch/arm64/kernel/cpufeature.c | 19 +++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu index 160b10c029c0..69edbd99e0b7 100644 --- a/Documentation/ABI/testing/sysfs-devices-system-cpu +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu @@ -494,6 +494,15 @@ Description: AArch64 CPU registers 'identification' directory exposes the CPU ID registers for identifying model and revision of the CPU. +What: /sys/devices/system/cpu/aarch32_el0 +Date: May 2021 +Contact: Linux ARM Kernel Mailing list +Description: Identifies the subset of CPUs in the system that can execute + AArch32 (32-bit ARM) applications. If present, the same format as + /sys/devices/system/cpu/{offline,online,possible,present} is used. + If absent, then all or none of the CPUs can execute AArch32 + applications and execve() will behave accordingly. + What: /sys/devices/system/cpu/cpu#/cpu_capacity Date: December 2016 Contact: Linux kernel mailing list diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index d99a29f52aa1..7ee1095a4585 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -67,6 +67,7 @@ #include #include #include +#include #include #include #include @@ -1320,6 +1321,24 @@ const struct cpumask *system_32bit_el0_cpumask(void) return cpu_possible_mask; } +static ssize_t aarch32_el0_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + const struct cpumask *mask = system_32bit_el0_cpumask(); + + return sysfs_emit(buf, "%*pbl\n", cpumask_pr_args(mask)); +} +static const DEVICE_ATTR_RO(aarch32_el0); + +static int __init aarch32_el0_sysfs_init(void) +{ + if (!allow_mismatched_32bit_el0) + return 0; + + return device_create_file(cpu_subsys.dev_root, &dev_attr_aarch32_el0); +} +device_initcall(aarch32_el0_sysfs_init); + static bool has_32bit_el0(const struct arm64_cpu_capabilities *entry, int scope) { if (!has_cpuid_feature(entry, scope)) From patchwork Fri Jul 30 11:24:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 761FBC4338F for ; Fri, 30 Jul 2021 11:35:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4405861042 for ; Fri, 30 Jul 2021 11:35:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4405861042 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BSIu4lbvo/9Hs9ljdPEO+H/xPwPC3Wlmat92lq5Qt08=; b=eRATVvkgshcCf1 xzAaR8EgW4b7d/CmUKev36wBYoBuRX4v+pTM0o5DfH7bsW8doFqn7UbHVts+18l3cYl78MrQ+zVdP JzBiJsc3t8SqA2lYjhqe/jgjbxA7tyDvEyN3mDe4TunDIHjlIV5LRYZ0M6UubXbxLmn9swBJxHcgz aUaIn7ysBV3m+bAH3oXMY+rLtOiSWW2VrFfWcGjhd7xiqipoGoTt8Xxd0F/nrYRc4zQO8RUSsZv2W x+JTQxHh8oBV0CcWadbGz7ammxA0SqmoeBnWiEGlqYp3AaD3b23FZSpS5IR7O/N2++/3/70z0Gdr1 ZgYz1N9GbMCrY+kxT0QQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Ql3-008Kkh-HU; Fri, 30 Jul 2021 11:32:54 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QeC-008HcD-22 for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:49 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3C38061052; Fri, 30 Jul 2021 11:25:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644347; bh=TCVWFCgzUBmJBt37TzROmSpk2UW8evs3tu2IpPF1E/s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pRhw08r1vUHHmuR84BHVI846ZNn3oCtHdEaaEwLKcTognoC+dHe9WgIDmGOs6Gzwo 089IvMeq6Mkl/DO/7ZYR0x8ZsLr0CZkT8ArsGeFRMwWYBW9vUhfr/Mj7yg4wMBXtfL 2kd+8WYGTvybi43xB0rdO2lk7N0AoaHV1PrUNsZsu3NxOhWe3QLm8X0OH93D1vrRHu sfUugggl+7x/i2Qbu/ApCk3n2brOfPTOwsZQUa8ZKtow0mP7pgYXzsWqmmJlfyELBY uWCPmiVO1wvEhxbXUWA6AnnNb/wgAkL9LT4zd9EwU1bukeuptZYuK5B9e2E51Ld4fk 6tOZ6F/luICVg== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 14/16] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Date: Fri, 30 Jul 2021 12:24:41 +0100 Message-Id: <20210730112443.23245-15-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042548_161444_0E8E1893 X-CRM114-Status: GOOD ( 12.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow systems with mismatched 32-bit support at EL0 to run 32-bit applications based on a new kernel parameter. Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon --- Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++ arch/arm64/kernel/cpufeature.c | 7 +++++++ 2 files changed, 15 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index bdb22006f713..6ab625dea8c0 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -287,6 +287,14 @@ do not want to use tracing_snapshot_alloc() as it needs to be done where GFP_KERNEL allocations are allowed. + allow_mismatched_32bit_el0 [ARM64] + Allow execve() of 32-bit applications and setting of the + PER_LINUX32 personality on systems where only a strict + subset of the CPUs support 32-bit EL0. When this + parameter is present, the set of CPUs supporting 32-bit + EL0 is indicated by /sys/devices/system/cpu/aarch32_el0 + and hot-unplug operations may be restricted. + amd_iommu= [HW,X86-64] Pass parameters to the AMD IOMMU driver in the system. Possible values are: diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 7ee1095a4585..a306ed5a6549 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1321,6 +1321,13 @@ const struct cpumask *system_32bit_el0_cpumask(void) return cpu_possible_mask; } +static int __init parse_32bit_el0_param(char *str) +{ + allow_mismatched_32bit_el0 = true; + return 0; +} +early_param("allow_mismatched_32bit_el0", parse_32bit_el0_param); + static ssize_t aarch32_el0_show(struct device *dev, struct device_attribute *attr, char *buf) { From patchwork Fri Jul 30 11:24:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEA66C4338F for ; Fri, 30 Jul 2021 11:36:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 809E360EBB for ; Fri, 30 Jul 2021 11:36:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 809E360EBB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xP2+X7N1JM2hGkVbkWqCLCJTGrGz81Qb076r0PucBzk=; b=FD+8/owcgGPD4B +WhgCtRIiHO1DOeFoAPRVuMpQErtYQeCGX7vQhUuWkPxq8SBgFDlMElE5XiAEe0/JdWXJsejdNYBO 0w/ajJLfxOAlRnCzEFWT/pt5d8AxUXGi8cb1L7ngZiretgWiltI5A3eZVtvF7rsm6+7BTf0v66/UV hZx1DUO8AQHcsUiegZ2/Wtz4NImkFPWpTaQxCLKj/0Fjcg3JxlzJE38R1dde0vsuIc44RVoIJM9Jr CN30UxBzYW5CR/S4S1HjkDR5s+KobRzr1qjCxuShR12UsrtJ81BdVcJlvVxN1EnV+FgCOE5Oe6d6n aTCIrUH+u1+94R8enhIw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qme-008LPV-6H; Fri, 30 Jul 2021 11:34:33 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QeG-008He7-2Z for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:53 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 390C360E76; Fri, 30 Jul 2021 11:25:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644351; bh=sg1rRxyn26PY6OXld7jhPzFtDo8GvzQTxzM27NbOHS4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ppJuqC0cvdqm8c5b40UZIlIdZLNugRZoA5OS57qvkm0+nQNTthMR8YuXGbFwj0SVF ucudqTOpPR+WsSoK/uSM1dk5tq+7uaTAe+XnTiaAp6A+9wN3vl0mQWLs0w/KYVfTAJ 6bRKj8hK29ZEHe1ae4C7ubr9p4y+DTXUc1nnliRY0Z1waqeA76FCeXvDGpXcYBbvVI ZdVg6LnXWz31/weKxFZptk3K+fHiEHUs6Y/unCRCc6/IaFWilXjO6EXG1DvQ5oxwHG I52wcZI9hzIXbaE2WlimjxyokQ5ohMqQp1A2qMsNuhhSh34+STq4W+mQwRTn2P/T9H 7+vyEt5faip7g== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 15/16] arm64: Remove logic to kill 32-bit tasks on 64-bit-only cores Date: Fri, 30 Jul 2021 12:24:42 +0100 Message-Id: <20210730112443.23245-16-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042552_244165_10E24CCD X-CRM114-Status: GOOD ( 15.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The scheduler now knows enough about these braindead systems to place 32-bit tasks accordingly, so throw out the safety checks and allow the ret-to-user path to avoid do_notify_resume() if there is nothing to do. Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon --- arch/arm64/kernel/process.c | 14 +------------- arch/arm64/kernel/signal.c | 26 -------------------------- 2 files changed, 1 insertion(+), 39 deletions(-) diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 583ee58f8c9c..e0e7f4e9b607 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -469,15 +469,6 @@ static void erratum_1418040_thread_switch(struct task_struct *prev, write_sysreg(val, cntkctl_el1); } -static void compat_thread_switch(struct task_struct *next) -{ - if (!is_compat_thread(task_thread_info(next))) - return; - - if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) - set_tsk_thread_flag(next, TIF_NOTIFY_RESUME); -} - static void update_sctlr_el1(u64 sctlr) { /* @@ -519,7 +510,6 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev, ssbs_thread_switch(next); erratum_1418040_thread_switch(prev, next); ptrauth_thread_switch_user(next); - compat_thread_switch(next); /* * Complete any pending TLB or cache maintenance on this CPU in case @@ -621,10 +611,8 @@ void arch_setup_new_exec(void) * at the point of execve(), although we try a bit harder to * honour the cpuset hierarchy. */ - if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { + if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) force_compatible_cpus_allowed_ptr(current); - set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); - } } else if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) { relax_compatible_cpus_allowed_ptr(current); } diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index f8192f4ae0b8..6237486ff6bb 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -911,19 +911,6 @@ static void do_signal(struct pt_regs *regs) restore_saved_sigmask(); } -static bool cpu_affinity_invalid(struct pt_regs *regs) -{ - if (!compat_user_mode(regs)) - return false; - - /* - * We're preemptible, but a reschedule will cause us to check the - * affinity again. - */ - return !cpumask_test_cpu(raw_smp_processor_id(), - system_32bit_el0_cpumask()); -} - asmlinkage void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags) { @@ -951,19 +938,6 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, if (thread_flags & _TIF_NOTIFY_RESUME) { tracehook_notify_resume(regs); rseq_handle_notify_resume(NULL, regs); - - /* - * If we reschedule after checking the affinity - * then we must ensure that TIF_NOTIFY_RESUME - * is set so that we check the affinity again. - * Since tracehook_notify_resume() clears the - * flag, ensure that the compiler doesn't move - * it after the affinity check. - */ - barrier(); - - if (cpu_affinity_invalid(regs)) - force_sig(SIGKILL); } if (thread_flags & _TIF_FOREIGN_FPSTATE) From patchwork Fri Jul 30 11:24:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12411089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAA29C4338F for ; Fri, 30 Jul 2021 11:38:16 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 71D1861042 for ; Fri, 30 Jul 2021 11:38:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 71D1861042 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TXM2owVd9tBqad0gHkLYWAYKw7kZSVOUc8LuH/cvVuU=; b=ZDmBavjsEShj4+ Dx/Rar2qo5dkJ0R4J2T9UEgYvv7DU9QnyiCagYlfizN6e6s73+YpAPN5urAcW8Hg4NaORH5dG31yN tg2Gx4S+imR8/nVvtdGGm33PxXPgnlHPgElynYPVwgwlmqgVHITgCKPF7X2jbIVN/TuT2TdpF+YNx fMFqzJ+nMZ/2MeTRvyu6eHpBfTGYWEgXzHJ5BNrtenI/SjTmh8bouMy1PQ1dGIzIMukZ8xAxgHbb9 yywmJ8G+C2asxkk/AcGd+3bHd5sMvkHliQeaEk9e2HS8XtdvhD1TdckC8A09dqXpCpx3Wn8CJo3oL uxvlMRBCnjaxJ0ys3OCQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9Qny-008Lwl-7L; Fri, 30 Jul 2021 11:35:55 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9QeK-008Hgb-38 for linux-arm-kernel@lists.infradead.org; Fri, 30 Jul 2021 11:25:58 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 37BBE6101C; Fri, 30 Jul 2021 11:25:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627644355; bh=58RaxNDrM4Q3j9mW7XXfbQ/URRqdiy8OCrN2808XFlI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gimJjdxCpcP5+6drREnkcJJDzRCfAXoQR2MMMtMbaOSNCr3wJKzskSjnV8QKAa4lQ 4gKcKXvG4TXMAHtSiNvNgg5K8LpVwNdO/tOvm9Yu18gJolVy5iDcjf5e0uL6zsJsXE da/rsLR5xEm+1Jx5gtn4tk4NIrCXQvqgNlHQTwiHwMrQTTHSrARtyEuR012iLx/AhY sbp30F8XYhs6DYIHMaGi5//T/VpgpuQQZrfDD2XT67IfXgxc3tQ4dhNJKCmvztzsDw HaRtpchcBI9Yp7Kq5J0Nvm8O9lCDjPI3cDcCA2iHE6qW90LwGd/CjkYto8vbBgUMfW 592bCND5pSZIA== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Will Deacon , Catalin Marinas , Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , Dietmar Eggemann , Daniel Bristot de Oliveira , Valentin Schneider , Mark Rutland , kernel-team@android.com Subject: [PATCH v11 16/16] Documentation: arm64: describe asymmetric 32-bit support Date: Fri, 30 Jul 2021 12:24:43 +0100 Message-Id: <20210730112443.23245-17-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210730112443.23245-1-will@kernel.org> References: <20210730112443.23245-1-will@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210730_042556_224555_70D07244 X-CRM114-Status: GOOD ( 32.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Document support for running 32-bit tasks on asymmetric 32-bit systems and its impact on the user ABI when enabled. Signed-off-by: Will Deacon --- .../admin-guide/kernel-parameters.txt | 3 + Documentation/arm64/asymmetric-32bit.rst | 155 ++++++++++++++++++ Documentation/arm64/index.rst | 1 + 3 files changed, 159 insertions(+) create mode 100644 Documentation/arm64/asymmetric-32bit.rst diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 6ab625dea8c0..b2f5dd4ea805 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -295,6 +295,9 @@ EL0 is indicated by /sys/devices/system/cpu/aarch32_el0 and hot-unplug operations may be restricted. + See Documentation/arm64/asymmetric-32bit.rst for more + information. + amd_iommu= [HW,X86-64] Pass parameters to the AMD IOMMU driver in the system. Possible values are: diff --git a/Documentation/arm64/asymmetric-32bit.rst b/Documentation/arm64/asymmetric-32bit.rst new file mode 100644 index 000000000000..64a0b505da7d --- /dev/null +++ b/Documentation/arm64/asymmetric-32bit.rst @@ -0,0 +1,155 @@ +====================== +Asymmetric 32-bit SoCs +====================== + +Author: Will Deacon + +This document describes the impact of asymmetric 32-bit SoCs on the +execution of 32-bit (``AArch32``) applications. + +Date: 2021-05-17 + +Introduction +============ + +Some Armv9 SoCs suffer from a big.LITTLE misfeature where only a subset +of the CPUs are capable of executing 32-bit user applications. On such +a system, Linux by default treats the asymmetry as a "mismatch" and +disables support for both the ``PER_LINUX32`` personality and +``execve(2)`` of 32-bit ELF binaries, with the latter returning +``-ENOEXEC``. If the mismatch is detected during late onlining of a +64-bit-only CPU, then the onlining operation fails and the new CPU is +unavailable for scheduling. + +Surprisingly, these SoCs have been produced with the intention of +running legacy 32-bit binaries. Unsurprisingly, that doesn't work very +well with the default behaviour of Linux. + +It seems inevitable that future SoCs will drop 32-bit support +altogether, so if you're stuck in the unenviable position of needing to +run 32-bit code on one of these transitionary platforms then you would +be wise to consider alternatives such as recompilation, emulation or +retirement. If neither of those options are practical, then read on. + +Enabling kernel support +======================= + +Since the kernel support is not completely transparent to userspace, +allowing 32-bit tasks to run on an asymmetric 32-bit system requires an +explicit "opt-in" and can be enabled by passing the +``allow_mismatched_32bit_el0`` parameter on the kernel command-line. + +For the remainder of this document we will refer to an *asymmetric +system* to mean an asymmetric 32-bit SoC running Linux with this kernel +command-line option enabled. + +Userspace impact +================ + +32-bit tasks running on an asymmetric system behave in mostly the same +way as on a homogeneous system, with a few key differences relating to +CPU affinity. + +sysfs +----- + +The subset of CPUs capable of running 32-bit tasks is described in +``/sys/devices/system/cpu/aarch32_el0`` and is documented further in +``Documentation/ABI/testing/sysfs-devices-system-cpu``. + +**Note:** CPUs are advertised by this file as they are detected and so +late-onlining of 32-bit-capable CPUs can result in the file contents +being modified by the kernel at runtime. Once advertised, CPUs are never +removed from the file. + +``execve(2)`` +------------- + +On a homogeneous system, the CPU affinity of a task is preserved across +``execve(2)``. This is not always possible on an asymmetric system, +specifically when the new program being executed is 32-bit yet the +affinity mask contains 64-bit-only CPUs. In this situation, the kernel +determines the new affinity mask as follows: + + 1. If the 32-bit-capable subset of the affinity mask is not empty, + then the affinity is restricted to that subset and the old affinity + mask is saved. This saved mask is inherited over ``fork(2)`` and + preserved across ``execve(2)`` of 32-bit programs. + + **Note:** This step does not apply to ``SCHED_DEADLINE`` tasks. + See `SCHED_DEADLINE`_. + + 2. Otherwise, the cpuset hierarchy of the task is walked until an + ancestor is found containing at least one 32-bit-capable CPU. The + affinity of the task is then changed to match the 32-bit-capable + subset of the cpuset determined by the walk. + + 3. On failure (i.e. out of memory), the affinity is changed to the set + of all 32-bit-capable CPUs of which the kernel is aware. + +A subsequent ``execve(2)`` of a 64-bit program by the 32-bit task will +invalidate the affinity mask saved in (1) and attempt to restore the CPU +affinity of the task using the saved mask if it was previously valid. +This restoration may fail due to intervening changes to the deadline +policy or cpuset hierarchy, in which case the ``execve(2)`` continues +with the affinity unchanged. + +Calls to ``sched_setaffinity(2)`` for a 32-bit task will consider only +the 32-bit-capable CPUs of the requested affinity mask. On success, the +affinity for the task is updated and any saved mask from a prior +``execve(2)`` is invalidated. + +``SCHED_DEADLINE`` +------------------ + +Explicit admission of a 32-bit deadline task to the default root domain +(e.g. by calling ``sched_setattr(2)``) is rejected on an asymmetric +32-bit system unless admission control is disabled by writing -1 to +``/proc/sys/kernel/sched_rt_runtime_us``. + +``execve(2)`` of a 32-bit program from a 64-bit deadline task will +return ``-ENOEXEC`` if the root domain for the task contains any +64-bit-only CPUs and admission control is enabled. Concurrent offlining +of 32-bit-capable CPUs may still necessitate the procedure described in +`execve(2)`_, in which case step (1) is skipped and a warning is +emitted on the console. + +**Note:** It is recommended that a set of 32-bit-capable CPUs are placed +into a separate root domain if ``SCHED_DEADLINE`` is to be used with +32-bit tasks on an asymmetric system. Failure to do so is likely to +result in missed deadlines. + +Cpusets +------- + +The affinity of a 32-bit task on an asymmetric system may include CPUs +that are not explicitly allowed by the cpuset to which it is attached. +This can occur as a result of the following two situations: + + - A 64-bit task attached to a cpuset which allows only 64-bit CPUs + executes a 32-bit program. + + - All of the 32-bit-capable CPUs allowed by a cpuset containing a + 32-bit task are offlined. + +In both of these cases, the new affinity is calculated according to step +(2) of the process described in `execve(2)`_ and the cpuset hierarchy is +unchanged irrespective of the cgroup version. + +CPU hotplug +----------- + +On an asymmetric system, the first detected 32-bit-capable CPU is +prevented from being offlined by userspace and any such attempt will +return ``-EPERM``. Note that suspend is still permitted even if the +primary CPU (i.e. CPU 0) is 64-bit-only. + +KVM +--- + +Although KVM will not advertise 32-bit EL0 support to any vCPUs on an +asymmetric system, a broken guest at EL1 could still attempt to execute +32-bit code at EL0. In this case, an exit from a vCPU thread in 32-bit +mode will return to host userspace with an ``exit_reason`` of +``KVM_EXIT_FAIL_ENTRY`` and will remain non-runnable until successfully +re-initialised by a subsequent ``KVM_ARM_VCPU_INIT`` operation. diff --git a/Documentation/arm64/index.rst b/Documentation/arm64/index.rst index 97d65ba12a35..4f840bac083e 100644 --- a/Documentation/arm64/index.rst +++ b/Documentation/arm64/index.rst @@ -10,6 +10,7 @@ ARM64 Architecture acpi_object_usage amu arm-acpi + asymmetric-32bit booting cpu-feature-registers elf_hwcaps