From patchwork Mon Jul 24 14:28:53 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?SsO8cmdlbiBHcm/Dnw==?= X-Patchwork-Id: 9859595 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 26F4F600F5 for ; Mon, 24 Jul 2017 14:31:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 180CB28573 for ; Mon, 24 Jul 2017 14:31:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0CD692857E; Mon, 24 Jul 2017 14:31:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C68912859E for ; Mon, 24 Jul 2017 14:31:45 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dZeM7-0006eH-Rq; Mon, 24 Jul 2017 14:29:07 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dZeM7-0006e8-2N for xen-devel@lists.xenproject.org; Mon, 24 Jul 2017 14:29:07 +0000 Received: from [85.158.143.35] by server-8.bemta-6.messagelabs.com id E4/AB-09901-23406795; Mon, 24 Jul 2017 14:29:06 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPLMWRWlGSWpSXmKPExsVyuP0Ov64hS1m kQW+rlsX3LZOZHBg9Dn+4whLAGMWamZeUX5HAmnHw8Svmglv+FasvX2BvYLzt3MXIySEhYCTx duI/pi5GLg4hgYWMEucOn2IFSbAJqEpsuA5hiwgESWzoaGUCsZkF+hglbr4SBrGFBdwk/t/5A 1bDAlR/e99rli5GDg5eAROJXbPSIebLS3QcmMwygZFzASPDKkaN4tSistQiXUMDvaSizPSMkt zEzBwgz0wvN7W4ODE9NScxqVgvOT93EyPQWwxAsIPx3rKAQ4ySHExKorxCTGWRQnxJ+SmVGYn FGfFFpTmpxYcYZTg4lCR4O5iBcoJFqempFWmZOcCwgUlLcPAoifC+B2nlLS5IzC3OTIdInWI0 5tiwev0XJo5XE/5/YxJiycvPS5US5xUAmSQAUppRmgc3CBbOlxhlpYR5GYFOE+IpSC3KzSxBl X/FKM7BqCTMex5kIU9mXgncvldApzABnTJnRinIKSWJCCmpBsZADZZH9xVKAviYbp6Ic957WI z70mW9EOWvAvL8Ucfn216bL8zyhGvi+c9Ho37zbk7d0up41v1YT+C7X7ee6Z9+m64cVS1/XCP qw19Gm6Rdun5ZvWtXtVamWZze1sIUL/BJcc/qsGf+ret8tlwQP2TD0L9ngSn30Ts8vXOmHvoW 4PRD/es/owVKLMUZiYZazEXFiQCFmCZiYgIAAA== X-Env-Sender: jgross@suse.com X-Msg-Ref: server-10.tower-21.messagelabs.com!1500906542!68052437!1 X-Originating-IP: [195.135.220.15] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 63905 invoked from network); 24 Jul 2017 14:29:05 -0000 Received: from mx2.suse.de (HELO mx1.suse.de) (195.135.220.15) by server-10.tower-21.messagelabs.com with DHE-RSA-CAMELLIA256-SHA encrypted SMTP; 24 Jul 2017 14:29:05 -0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D6773ABB3; Mon, 24 Jul 2017 14:29:01 +0000 (UTC) From: Juergen Gross To: linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, x86@kernel.org Date: Mon, 24 Jul 2017 16:28:53 +0200 Message-Id: <20170724142853.26448-1-jgross@suse.com> X-Mailer: git-send-email 2.12.3 Cc: Juergen Gross , luto@amacapital.net, mingo@redhat.com, hpa@zytor.com, boris.ostrovsky@oracle.com, tglx@linutronix.de Subject: [Xen-devel] [PATCH v1] xen: get rid of paravirt op adjust_exception_frame X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP When running as Xen pv-guest the exception frame on the stack contains %r11 and %rcx additional to the other data pushed by the processor. Instead of having a paravirt op being called for each exception type prepend the Xen specific code to each exception entry. When running as Xen pv-guest just use the exception entry with prepended instructions, otherwise use the entry without the Xen specific code. Signed-off-by: Juergen Gross Reviewed-by: Boris Ostrovsky --- arch/x86/entry/calling.h | 6 ++++++ arch/x86/entry/entry_64.S | 19 ++----------------- arch/x86/entry/entry_64_compat.S | 3 +-- arch/x86/include/asm/desc.h | 7 +++++++ arch/x86/include/asm/paravirt.h | 5 ----- arch/x86/include/asm/paravirt_types.h | 4 ---- arch/x86/kernel/asm-offsets_64.c | 1 - arch/x86/kernel/paravirt.c | 3 --- arch/x86/xen/enlighten_pv.c | 8 ++++++-- arch/x86/xen/irq.c | 3 --- arch/x86/xen/setup.c | 3 ++- arch/x86/xen/smp_pv.c | 2 +- arch/x86/xen/xen-asm_64.S | 6 ------ arch/x86/xen/xen-ops.h | 2 +- 14 files changed, 26 insertions(+), 46 deletions(-) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 05ed3d393da7..8b315ee49c93 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -227,3 +227,9 @@ For 32-bit we have the following conventions - kernel is built with .Lafter_call_\@: #endif .endm + +#ifdef CONFIG_XEN_PV +#define PV_ENTRY(sym) ENTRY(_xen_##sym); pop %rcx; pop %r11; .globl sym; sym: +#else +#define PV_ENTRY(sym) ENTRY(sym) +#endif diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index a9a8027a6c0e..94b6b56fa005 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -738,14 +738,13 @@ apicinterrupt IRQ_WORK_VECTOR irq_work_interrupt smp_irq_work_interrupt #define CPU_TSS_IST(x) PER_CPU_VAR(cpu_tss) + (TSS_ist + ((x) - 1) * 8) .macro idtentry sym do_sym has_error_code:req paranoid=0 shift_ist=-1 -ENTRY(\sym) +PV_ENTRY(\sym) /* Sanity check */ .if \shift_ist != -1 && \paranoid == 0 .error "using shift_ist requires paranoid=1" .endif ASM_CLAC - PARAVIRT_ADJUST_EXCEPTION_FRAME .ifeq \has_error_code pushq $-1 /* ORIG_RAX: no syscall to restart */ @@ -967,8 +966,6 @@ ENTRY(xen_failsafe_callback) movq 8(%rsp), %r11 addq $0x30, %rsp pushq $0 /* RIP */ - pushq %r11 - pushq %rcx jmp general_protection 1: /* Segment mismatch => Category 1 (Bad segment). Retry the IRET. */ movq (%rsp), %rcx @@ -1159,19 +1156,7 @@ ENTRY(error_exit) END(error_exit) /* Runs on exception stack */ -ENTRY(nmi) - /* - * Fix up the exception frame if we're on Xen. - * PARAVIRT_ADJUST_EXCEPTION_FRAME is guaranteed to push at most - * one value to the stack on native, so it may clobber the rdx - * scratch slot, but it won't clobber any of the important - * slots past it. - * - * Xen is a different story, because the Xen frame itself overlaps - * the "NMI executing" variable. - */ - PARAVIRT_ADJUST_EXCEPTION_FRAME - +PV_ENTRY(nmi) /* * We allow breakpoints in NMIs. If a breakpoint occurs, then * the iretq it performs will take us out of NMI context. diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S index e1721dafbcb1..9fd8c8f6004e 100644 --- a/arch/x86/entry/entry_64_compat.S +++ b/arch/x86/entry/entry_64_compat.S @@ -290,11 +290,10 @@ END(entry_SYSCALL_compat) * edi arg5 * ebp arg6 */ -ENTRY(entry_INT80_compat) +PV_ENTRY(entry_INT80_compat) /* * Interrupts are off on entry. */ - PARAVIRT_ADJUST_EXCEPTION_FRAME ASM_CLAC /* Do this early to minimize exposure */ SWAPGS diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h index d0a21b12dd58..ff19be44877a 100644 --- a/arch/x86/include/asm/desc.h +++ b/arch/x86/include/asm/desc.h @@ -85,9 +85,16 @@ static inline phys_addr_t get_cpu_gdt_paddr(unsigned int cpu) #ifdef CONFIG_X86_64 +#ifdef CONFIG_XEN_PV +extern unsigned int pv_idt_prologue; +#else +#define pv_idt_prologue 0 +#endif + static inline void pack_gate(gate_desc *gate, unsigned type, unsigned long func, unsigned dpl, unsigned ist, unsigned seg) { + func -= pv_idt_prologue; gate->offset_low = PTR_LOW(func); gate->segment = __KERNEL_CS; gate->ist = ist; diff --git a/arch/x86/include/asm/paravirt.h b/arch/x86/include/asm/paravirt.h index 9ccac1926587..c25dd22f7c70 100644 --- a/arch/x86/include/asm/paravirt.h +++ b/arch/x86/include/asm/paravirt.h @@ -960,11 +960,6 @@ extern void default_banner(void); #define GET_CR2_INTO_RAX \ call PARA_INDIRECT(pv_mmu_ops+PV_MMU_read_cr2) -#define PARAVIRT_ADJUST_EXCEPTION_FRAME \ - PARA_SITE(PARA_PATCH(pv_irq_ops, PV_IRQ_adjust_exception_frame), \ - CLBR_NONE, \ - call PARA_INDIRECT(pv_irq_ops+PV_IRQ_adjust_exception_frame)) - #define USERGS_SYSRET64 \ PARA_SITE(PARA_PATCH(pv_cpu_ops, PV_CPU_usergs_sysret64), \ CLBR_NONE, \ diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h index 9ffc36bfe4cd..c55106726938 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -195,10 +195,6 @@ struct pv_irq_ops { void (*safe_halt)(void); void (*halt)(void); - -#ifdef CONFIG_X86_64 - void (*adjust_exception_frame)(void); -#endif } __no_randomize_layout; struct pv_mmu_ops { diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets_64.c index 99332f550c48..cf42206926af 100644 --- a/arch/x86/kernel/asm-offsets_64.c +++ b/arch/x86/kernel/asm-offsets_64.c @@ -20,7 +20,6 @@ static char syscalls_ia32[] = { int main(void) { #ifdef CONFIG_PARAVIRT - OFFSET(PV_IRQ_adjust_exception_frame, pv_irq_ops, adjust_exception_frame); OFFSET(PV_CPU_usergs_sysret64, pv_cpu_ops, usergs_sysret64); OFFSET(PV_CPU_swapgs, pv_cpu_ops, swapgs); BLANK(); diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index bc0a849589bb..a14df9eecfed 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -319,9 +319,6 @@ __visible struct pv_irq_ops pv_irq_ops = { .irq_enable = __PV_IS_CALLEE_SAVE(native_irq_enable), .safe_halt = native_safe_halt, .halt = native_halt, -#ifdef CONFIG_X86_64 - .adjust_exception_frame = paravirt_nop, -#endif }; __visible struct pv_cpu_ops pv_cpu_ops = { diff --git a/arch/x86/xen/enlighten_pv.c b/arch/x86/xen/enlighten_pv.c index 811e4ddb3f37..27e0d964be7b 100644 --- a/arch/x86/xen/enlighten_pv.c +++ b/arch/x86/xen/enlighten_pv.c @@ -88,6 +88,7 @@ #include "pmu.h" void *xen_initial_gdt; +unsigned int pv_idt_prologue; static int xen_cpu_up_prepare_pv(unsigned int cpu); static int xen_cpu_dead_pv(unsigned int cpu); @@ -589,7 +590,7 @@ static int cvt_gate_to_trap(int vector, const gate_desc *val, info->vector = vector; - addr = gate_offset(*val); + addr = gate_offset(*val) + pv_idt_prologue; #ifdef CONFIG_X86_64 /* * Look for known traps using IST, and substitute them @@ -626,7 +627,7 @@ static int cvt_gate_to_trap(int vector, const gate_desc *val, return 0; } #endif /* CONFIG_X86_64 */ - info->address = addr; + info->address = addr - pv_idt_prologue; info->cs = gate_segment(*val); info->flags = val->dpl; @@ -1246,6 +1247,9 @@ asmlinkage __visible void __init xen_start_kernel(void) pv_info = xen_info; pv_init_ops = xen_init_ops; pv_cpu_ops = xen_cpu_ops; +#ifdef CONFIG_X86_64 + pv_idt_prologue = 3; /* size of pop %rcx; pop %r11; */ +#endif x86_platform.get_nmi_reason = xen_get_nmi_reason; diff --git a/arch/x86/xen/irq.c b/arch/x86/xen/irq.c index 33e92955e09d..d4eff5676cfa 100644 --- a/arch/x86/xen/irq.c +++ b/arch/x86/xen/irq.c @@ -123,9 +123,6 @@ static const struct pv_irq_ops xen_irq_ops __initconst = { .safe_halt = xen_safe_halt, .halt = xen_halt, -#ifdef CONFIG_X86_64 - .adjust_exception_frame = xen_adjust_exception_frame, -#endif }; void __init xen_init_irq_ops(void) diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c index c81046323ebc..2f56dd849ea3 100644 --- a/arch/x86/xen/setup.c +++ b/arch/x86/xen/setup.c @@ -1012,7 +1012,8 @@ void __init xen_pvmmu_arch_setup(void) HYPERVISOR_vm_assist(VMASST_CMD_enable, VMASST_TYPE_pae_extended_cr3); - if (register_callback(CALLBACKTYPE_event, xen_hypervisor_callback) || + if (register_callback(CALLBACKTYPE_event, + xen_hypervisor_callback - pv_idt_prologue) || register_callback(CALLBACKTYPE_failsafe, xen_failsafe_callback)) BUG(); diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c index 51471408fdd1..4be2c5e08dac 100644 --- a/arch/x86/xen/smp_pv.c +++ b/arch/x86/xen/smp_pv.c @@ -323,7 +323,7 @@ cpu_initialize_context(unsigned int cpu, struct task_struct *idle) ctxt->gs_base_kernel = per_cpu_offset(cpu); #endif ctxt->event_callback_eip = - (unsigned long)xen_hypervisor_callback; + (unsigned long)xen_hypervisor_callback - pv_idt_prologue; ctxt->failsafe_callback_eip = (unsigned long)xen_failsafe_callback; ctxt->user_regs.cs = __KERNEL_CS; diff --git a/arch/x86/xen/xen-asm_64.S b/arch/x86/xen/xen-asm_64.S index c3df43141e70..8db45fdba96d 100644 --- a/arch/x86/xen/xen-asm_64.S +++ b/arch/x86/xen/xen-asm_64.S @@ -22,12 +22,6 @@ #include "xen-asm.h" -ENTRY(xen_adjust_exception_frame) - mov 8+0(%rsp), %rcx - mov 8+8(%rsp), %r11 - ret $16 -ENDPROC(xen_adjust_exception_frame) - hypercall_iret = hypercall_page + __HYPERVISOR_iret * 32 /* * Xen64 iret frame: diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index 0d5004477db6..f08f7cd722ab 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -17,6 +17,7 @@ void xen_syscall32_target(void); #endif extern void *xen_initial_gdt; +extern unsigned int pv_idt_prologue; struct trap_info; void xen_copy_trap_info(struct trap_info *traps); @@ -145,7 +146,6 @@ DECL_ASM(void, xen_restore_fl_direct, unsigned long); __visible void xen_iret(void); __visible void xen_sysret32(void); __visible void xen_sysret64(void); -__visible void xen_adjust_exception_frame(void); extern int xen_panic_handler_init(void);