| Message ID | 20250114175143.81438-28-vschneid@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | context_tracking,x86: Defer some IPIs until a user->kernel transition | expand |
On Tue, Jan 14, 2025, Valentin Schneider wrote: > Later patches will require issuing a __flush_tlb_all() from noinstr code. > This requires making both __flush_tlb_local() and __flush_tlb_global() > noinstr-compliant. > > For __flush_tlb_local(), xen_flush_tlb() has already been made noinstr, so > it's just native_flush_tlb_global(), and simply __always_inline'ing > invalidate_user_asid() gets us there > > Signed-off-by: Valentin Schneider <vschneid@redhat.com> > --- ... > @@ -1206,7 +1206,7 @@ STATIC_NOPV noinstr void native_flush_tlb_global(void) > /* > * Flush the entire current user mapping > */ > -STATIC_NOPV void native_flush_tlb_local(void) > +STATIC_NOPV noinstr void native_flush_tlb_local(void) native_write_cr3() and __native_read_cr3() need to be __always_inline. vmlinux.o: warning: objtool: native_flush_tlb_local+0x8: call to __native_read_cr3() leaves .noinstr.text section
diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index b3daee3d46677..0c0dd186c03e6 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -70,7 +70,7 @@ void native_flush_tlb_one_user(unsigned long addr); void native_flush_tlb_multi(const struct cpumask *cpumask, const struct flush_tlb_info *info); -static inline void __flush_tlb_local(void) +static __always_inline void __flush_tlb_local(void) { PVOP_VCALL0(mmu.flush_tlb_user); } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 2d2ab3e221f0c..18b40bbc2fa15 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -257,7 +257,7 @@ static void choose_new_asid(struct mm_struct *next, u64 next_tlb_gen, * * See SWITCH_TO_USER_CR3. */ -static inline void invalidate_user_asid(u16 asid) +static __always_inline void invalidate_user_asid(u16 asid) { /* There is no user ASID if address space separation is off */ if (!IS_ENABLED(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION)) @@ -1206,7 +1206,7 @@ STATIC_NOPV noinstr void native_flush_tlb_global(void) /* * Flush the entire current user mapping */ -STATIC_NOPV void native_flush_tlb_local(void) +STATIC_NOPV noinstr void native_flush_tlb_local(void) { /* * Preemption or interrupts must be disabled to protect the access
Later patches will require issuing a __flush_tlb_all() from noinstr code. This requires making both __flush_tlb_local() and __flush_tlb_global() noinstr-compliant. For __flush_tlb_local(), xen_flush_tlb() has already been made noinstr, so it's just native_flush_tlb_global(), and simply __always_inline'ing invalidate_user_asid() gets us there Signed-off-by: Valentin Schneider <vschneid@redhat.com> --- arch/x86/include/asm/paravirt.h | 2 +- arch/x86/mm/tlb.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)