| Message ID | 20201124155039.13804-9-will@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | An alternative series for asymmetric AArch32 systems | expand |
On Tuesday 24 Nov 2020 at 15:50:33 (+0000), Will Deacon wrote: > When exec'ing a 32-bit task on a system with mismatched support for > 32-bit EL0, try to ensure that it starts life on a CPU that can actually > run it. > > Signed-off-by: Will Deacon <will@kernel.org> > --- > arch/arm64/kernel/process.c | 42 ++++++++++++++++++++++++++++++++++++- > 1 file changed, 41 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c > index 1540ab0fbf23..72116b0c7c73 100644 > --- a/arch/arm64/kernel/process.c > +++ b/arch/arm64/kernel/process.c > @@ -31,6 +31,7 @@ > #include <linux/interrupt.h> > #include <linux/init.h> > #include <linux/cpu.h> > +#include <linux/cpuset.h> > #include <linux/elfcore.h> > #include <linux/pm.h> > #include <linux/tick.h> > @@ -625,6 +626,45 @@ unsigned long arch_align_stack(unsigned long sp) > return sp & ~0xf; > } > > +static void adjust_compat_task_affinity(struct task_struct *p) > +{ > + cpumask_var_t cpuset_mask; > + const struct cpumask *possible_mask = system_32bit_el0_cpumask(); > + const struct cpumask *newmask = possible_mask; > + > + /* > + * Restrict the CPU affinity mask for a 32-bit task so that it contains > + * only the 32-bit-capable subset of its original CPU mask. If this is > + * empty, then try again with the cpuset allowed mask. If that fails, > + * forcefully override it with the set of all 32-bit-capable CPUs that > + * we know about. > + * > + * From the perspective of the task, this looks similar to what would > + * happen if the 64-bit-only CPUs were hot-unplugged at the point of > + * execve(). > + */ > + if (!restrict_cpus_allowed_ptr(p, possible_mask)) > + goto out; > + > + if (alloc_cpumask_var(&cpuset_mask, GFP_KERNEL)) { > + cpuset_cpus_allowed(p, cpuset_mask); > + if (cpumask_and(cpuset_mask, cpuset_mask, possible_mask)) { > + newmask = cpuset_mask; > + goto out_set_mask; > + } > + } > + > + if (printk_ratelimit()) { > + printk_deferred("Overriding affinity for 32-bit process %d (%s) to CPUs %*pbl\n", > + task_pid_nr(p), p->comm, cpumask_pr_args(newmask)); > + } > +out_set_mask: > + set_cpus_allowed_ptr(p, newmask); > + free_cpumask_var(cpuset_mask); > +out: > + set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); > +} This starts to look an awful lot like select_fallback_rq(), but I suppose we shouldn't bother factoring out that code yet as we probably don't want this pattern to be re-used all over, so: Reviewed-by: Quentin Perret <qperret@google.com> Thanks, Quentin
On 11/24/20 15:50, Will Deacon wrote: > When exec'ing a 32-bit task on a system with mismatched support for > 32-bit EL0, try to ensure that it starts life on a CPU that can actually > run it. > > Signed-off-by: Will Deacon <will@kernel.org> > --- > arch/arm64/kernel/process.c | 42 ++++++++++++++++++++++++++++++++++++- > 1 file changed, 41 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c > index 1540ab0fbf23..72116b0c7c73 100644 > --- a/arch/arm64/kernel/process.c > +++ b/arch/arm64/kernel/process.c > @@ -31,6 +31,7 @@ > #include <linux/interrupt.h> > #include <linux/init.h> > #include <linux/cpu.h> > +#include <linux/cpuset.h> > #include <linux/elfcore.h> > #include <linux/pm.h> > #include <linux/tick.h> > @@ -625,6 +626,45 @@ unsigned long arch_align_stack(unsigned long sp) > return sp & ~0xf; > } > > +static void adjust_compat_task_affinity(struct task_struct *p) > +{ > + cpumask_var_t cpuset_mask; > + const struct cpumask *possible_mask = system_32bit_el0_cpumask(); > + const struct cpumask *newmask = possible_mask; > + > + /* > + * Restrict the CPU affinity mask for a 32-bit task so that it contains > + * only the 32-bit-capable subset of its original CPU mask. If this is > + * empty, then try again with the cpuset allowed mask. If that fails, > + * forcefully override it with the set of all 32-bit-capable CPUs that > + * we know about. > + * > + * From the perspective of the task, this looks similar to what would > + * happen if the 64-bit-only CPUs were hot-unplugged at the point of > + * execve(). > + */ > + if (!restrict_cpus_allowed_ptr(p, possible_mask)) > + goto out; > + > + if (alloc_cpumask_var(&cpuset_mask, GFP_KERNEL)) { > + cpuset_cpus_allowed(p, cpuset_mask); > + if (cpumask_and(cpuset_mask, cpuset_mask, possible_mask)) { > + newmask = cpuset_mask; > + goto out_set_mask; > + } > + } Wouldn't it be better to move this logic to restrict_cpus_allowed_ptr()? I think it should always take cpusets into account and it's not special to this particular handling here, no? > + > + if (printk_ratelimit()) { > + printk_deferred("Overriding affinity for 32-bit process %d (%s) to CPUs %*pbl\n", > + task_pid_nr(p), p->comm, cpumask_pr_args(newmask)); > + } We have 2 cases where the affinity could have been overridden but we won't print anything: 1. restrict_cpus_allowed_ptr() 2. intersection of cpuset_mask and possible mask drops some cpus. Shouldn't we print something in these cases too? IMO it would be better to move this print to restrict_cpus_allowed_ptr() too. Thanks -- Qais Yousef > +out_set_mask: > + set_cpus_allowed_ptr(p, newmask); > + free_cpumask_var(cpuset_mask); > +out: > + set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); > +} > + > /* > * Called from setup_new_exec() after (COMPAT_)SET_PERSONALITY. > */ > @@ -635,7 +675,7 @@ void arch_setup_new_exec(void) > if (is_compat_task()) { > mmflags = MMCF_AARCH32; > if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) > - set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); > + adjust_compat_task_affinity(current); > } > > current->mm->context.flags = mmflags; > -- > 2.29.2.454.gaff20da3a2-goog >
On Fri, Nov 27, 2020 at 01:23:06PM +0000, Qais Yousef wrote: > On 11/24/20 15:50, Will Deacon wrote: > > When exec'ing a 32-bit task on a system with mismatched support for > > 32-bit EL0, try to ensure that it starts life on a CPU that can actually > > run it. > > > > Signed-off-by: Will Deacon <will@kernel.org> > > --- > > arch/arm64/kernel/process.c | 42 ++++++++++++++++++++++++++++++++++++- > > 1 file changed, 41 insertions(+), 1 deletion(-) > > > > diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c > > index 1540ab0fbf23..72116b0c7c73 100644 > > --- a/arch/arm64/kernel/process.c > > +++ b/arch/arm64/kernel/process.c > > @@ -31,6 +31,7 @@ > > #include <linux/interrupt.h> > > #include <linux/init.h> > > #include <linux/cpu.h> > > +#include <linux/cpuset.h> > > #include <linux/elfcore.h> > > #include <linux/pm.h> > > #include <linux/tick.h> > > @@ -625,6 +626,45 @@ unsigned long arch_align_stack(unsigned long sp) > > return sp & ~0xf; > > } > > > > +static void adjust_compat_task_affinity(struct task_struct *p) > > +{ > > + cpumask_var_t cpuset_mask; > > + const struct cpumask *possible_mask = system_32bit_el0_cpumask(); > > + const struct cpumask *newmask = possible_mask; > > + > > + /* > > + * Restrict the CPU affinity mask for a 32-bit task so that it contains > > + * only the 32-bit-capable subset of its original CPU mask. If this is > > + * empty, then try again with the cpuset allowed mask. If that fails, > > + * forcefully override it with the set of all 32-bit-capable CPUs that > > + * we know about. > > + * > > + * From the perspective of the task, this looks similar to what would > > + * happen if the 64-bit-only CPUs were hot-unplugged at the point of > > + * execve(). > > + */ > > + if (!restrict_cpus_allowed_ptr(p, possible_mask)) > > + goto out; > > + > > + if (alloc_cpumask_var(&cpuset_mask, GFP_KERNEL)) { > > + cpuset_cpus_allowed(p, cpuset_mask); > > + if (cpumask_and(cpuset_mask, cpuset_mask, possible_mask)) { > > + newmask = cpuset_mask; > > + goto out_set_mask; > > + } > > + } > > Wouldn't it be better to move this logic to restrict_cpus_allowed_ptr()? > I think it should always take cpusets into account and it's not special to > this particular handling here, no? I did actually try this but didn't pursue it further because I was worried that I was putting too much of the "can't run a 32-bit task on a 64-bit-only CPU" logic into what would otherwise be a potentially useful library function if/when other architectures want something similar. But I'll have another look because there were a couple of ideas I didn't try out. > > + if (printk_ratelimit()) { > > + printk_deferred("Overriding affinity for 32-bit process %d (%s) to CPUs %*pbl\n", > > + task_pid_nr(p), p->comm, cpumask_pr_args(newmask)); > > + } > > We have 2 cases where the affinity could have been overridden but we won't > print anything: > > 1. restrict_cpus_allowed_ptr() > 2. intersection of cpuset_mask and possible mask drops some cpus. > > Shouldn't we print something in these cases too? I don't think so: in these cases we've found a subset of CPUs that we can run on, and so there's no need to warn. Nothing says we _have_ to use all the CPUs available to us. The case where we override the affinity mask altogether, however, does warrant a warning. This is very similar to the hotplug behaviour in select_fallback_rq(). Will
On 12/01/20 16:55, Will Deacon wrote: > > > +static void adjust_compat_task_affinity(struct task_struct *p) > > > +{ > > > + cpumask_var_t cpuset_mask; > > > + const struct cpumask *possible_mask = system_32bit_el0_cpumask(); > > > + const struct cpumask *newmask = possible_mask; > > > + > > > + /* > > > + * Restrict the CPU affinity mask for a 32-bit task so that it contains > > > + * only the 32-bit-capable subset of its original CPU mask. If this is > > > + * empty, then try again with the cpuset allowed mask. If that fails, > > > + * forcefully override it with the set of all 32-bit-capable CPUs that > > > + * we know about. > > > + * > > > + * From the perspective of the task, this looks similar to what would > > > + * happen if the 64-bit-only CPUs were hot-unplugged at the point of > > > + * execve(). > > > + */ > > > + if (!restrict_cpus_allowed_ptr(p, possible_mask)) > > > + goto out; > > > + > > > + if (alloc_cpumask_var(&cpuset_mask, GFP_KERNEL)) { > > > + cpuset_cpus_allowed(p, cpuset_mask); > > > + if (cpumask_and(cpuset_mask, cpuset_mask, possible_mask)) { > > > + newmask = cpuset_mask; > > > + goto out_set_mask; > > > + } > > > + } > > > > Wouldn't it be better to move this logic to restrict_cpus_allowed_ptr()? > > I think it should always take cpusets into account and it's not special to > > this particular handling here, no? > > I did actually try this but didn't pursue it further because I was worried > that I was putting too much of the "can't run a 32-bit task on a 64-bit-only > CPU" logic into what would otherwise be a potentially useful library function > if/when other architectures want something similar. But I'll have another > look because there were a couple of ideas I didn't try out. If we improve the cpuset handling issues to take into account arch_task_cpu_possible_mask() as discussed in the other thread, I think we can drop the cpuset handling here. > > > > + if (printk_ratelimit()) { > > > + printk_deferred("Overriding affinity for 32-bit process %d (%s) to CPUs %*pbl\n", > > > + task_pid_nr(p), p->comm, cpumask_pr_args(newmask)); > > > + } > > > > We have 2 cases where the affinity could have been overridden but we won't > > print anything: > > > > 1. restrict_cpus_allowed_ptr() > > 2. intersection of cpuset_mask and possible mask drops some cpus. > > > > Shouldn't we print something in these cases too? > > I don't think so: in these cases we've found a subset of CPUs that we can > run on, and so there's no need to warn. Nothing says we _have_ to use all > the CPUs available to us. The case where we override the affinity mask > altogether, however, does warrant a warning. This is very similar to the > hotplug behaviour in select_fallback_rq(). Okay. It is just to warn when we actually break the affinity because we ended up with empty mask; not just because we changed the affinity to an intersecting one. I think this makes sense, yes. We might be able to drop this too if we improve cpuset handling. The devil is in the details I guess. Thanks -- Qais Yousef
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 1540ab0fbf23..72116b0c7c73 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -31,6 +31,7 @@ #include <linux/interrupt.h> #include <linux/init.h> #include <linux/cpu.h> +#include <linux/cpuset.h> #include <linux/elfcore.h> #include <linux/pm.h> #include <linux/tick.h> @@ -625,6 +626,45 @@ unsigned long arch_align_stack(unsigned long sp) return sp & ~0xf; } +static void adjust_compat_task_affinity(struct task_struct *p) +{ + cpumask_var_t cpuset_mask; + const struct cpumask *possible_mask = system_32bit_el0_cpumask(); + const struct cpumask *newmask = possible_mask; + + /* + * Restrict the CPU affinity mask for a 32-bit task so that it contains + * only the 32-bit-capable subset of its original CPU mask. If this is + * empty, then try again with the cpuset allowed mask. If that fails, + * forcefully override it with the set of all 32-bit-capable CPUs that + * we know about. + * + * From the perspective of the task, this looks similar to what would + * happen if the 64-bit-only CPUs were hot-unplugged at the point of + * execve(). + */ + if (!restrict_cpus_allowed_ptr(p, possible_mask)) + goto out; + + if (alloc_cpumask_var(&cpuset_mask, GFP_KERNEL)) { + cpuset_cpus_allowed(p, cpuset_mask); + if (cpumask_and(cpuset_mask, cpuset_mask, possible_mask)) { + newmask = cpuset_mask; + goto out_set_mask; + } + } + + if (printk_ratelimit()) { + printk_deferred("Overriding affinity for 32-bit process %d (%s) to CPUs %*pbl\n", + task_pid_nr(p), p->comm, cpumask_pr_args(newmask)); + } +out_set_mask: + set_cpus_allowed_ptr(p, newmask); + free_cpumask_var(cpuset_mask); +out: + set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); +} + /* * Called from setup_new_exec() after (COMPAT_)SET_PERSONALITY. */ @@ -635,7 +675,7 @@ void arch_setup_new_exec(void) if (is_compat_task()) { mmflags = MMCF_AARCH32; if (static_branch_unlikely(&arm64_mismatched_32bit_el0)) - set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); + adjust_compat_task_affinity(current); } current->mm->context.flags = mmflags;
When exec'ing a 32-bit task on a system with mismatched support for 32-bit EL0, try to ensure that it starts life on a CPU that can actually run it. Signed-off-by: Will Deacon <will@kernel.org> --- arch/arm64/kernel/process.c | 42 ++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-)