| Message ID | 20220125091453.1475246-9-ardb@kernel.org (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | ARM vmap'ed and IRQ stacks roundup | expand |
On Tue, Jan 25, 2022 at 1:15 AM Ard Biesheuvel <ardb@kernel.org> wrote: > > The get_current() and __my_cpu_offset() accessors evaluate to only a > single instruction emitted inline, but due to the size of the asm string > that is created for SMP+v6 configurations, the compiler assumes > otherwise, and may emit the functions out of line instead. > > So use __always_inline to avoid this. > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Alternatively, you could use the inline qualifier on the asm stmt. i.e. `asm inline ("my asm string")`. Only supported since gcc-8.3+ and all kernel-supported versions of clang though. See `asm_inline` in include/linux/compiler_types.h. Either way, Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> > --- > arch/arm/include/asm/current.h | 2 +- > arch/arm/include/asm/percpu.h | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/arch/arm/include/asm/current.h b/arch/arm/include/asm/current.h > index 131a89bbec6b..1e1178bf176d 100644 > --- a/arch/arm/include/asm/current.h > +++ b/arch/arm/include/asm/current.h > @@ -14,7 +14,7 @@ struct task_struct; > > extern struct task_struct *__current; > > -static inline __attribute_const__ struct task_struct *get_current(void) > +static __always_inline __attribute_const__ struct task_struct *get_current(void) > { > struct task_struct *cur; > > diff --git a/arch/arm/include/asm/percpu.h b/arch/arm/include/asm/percpu.h > index a09034ae45a1..7545c87c251f 100644 > --- a/arch/arm/include/asm/percpu.h > +++ b/arch/arm/include/asm/percpu.h > @@ -25,7 +25,7 @@ static inline void set_my_cpu_offset(unsigned long off) > asm volatile("mcr p15, 0, %0, c13, c0, 4" : : "r" (off) : "memory"); > } > > -static inline unsigned long __my_cpu_offset(void) > +static __always_inline unsigned long __my_cpu_offset(void) > { > unsigned long off; > > -- > 2.30.2 >
diff --git a/arch/arm/include/asm/current.h b/arch/arm/include/asm/current.h index 131a89bbec6b..1e1178bf176d 100644 --- a/arch/arm/include/asm/current.h +++ b/arch/arm/include/asm/current.h @@ -14,7 +14,7 @@ struct task_struct; extern struct task_struct *__current; -static inline __attribute_const__ struct task_struct *get_current(void) +static __always_inline __attribute_const__ struct task_struct *get_current(void) { struct task_struct *cur; diff --git a/arch/arm/include/asm/percpu.h b/arch/arm/include/asm/percpu.h index a09034ae45a1..7545c87c251f 100644 --- a/arch/arm/include/asm/percpu.h +++ b/arch/arm/include/asm/percpu.h @@ -25,7 +25,7 @@ static inline void set_my_cpu_offset(unsigned long off) asm volatile("mcr p15, 0, %0, c13, c0, 4" : : "r" (off) : "memory"); } -static inline unsigned long __my_cpu_offset(void) +static __always_inline unsigned long __my_cpu_offset(void) { unsigned long off;
The get_current() and __my_cpu_offset() accessors evaluate to only a single instruction emitted inline, but due to the size of the asm string that is created for SMP+v6 configurations, the compiler assumes otherwise, and may emit the functions out of line instead. So use __always_inline to avoid this. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> --- arch/arm/include/asm/current.h | 2 +- arch/arm/include/asm/percpu.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)