From patchwork Fri Nov 7 23:58:19 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Lutomirski X-Patchwork-Id: 5255951 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 95037C11AC for ; Fri, 7 Nov 2014 23:58:48 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C7DB620145 for ; Fri, 7 Nov 2014 23:58:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id D9B6F20149 for ; Fri, 7 Nov 2014 23:58:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753439AbaKGX6g (ORCPT ); Fri, 7 Nov 2014 18:58:36 -0500 Received: from mail-pa0-f42.google.com ([209.85.220.42]:60873 "EHLO mail-pa0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753416AbaKGX6d (ORCPT ); Fri, 7 Nov 2014 18:58:33 -0500 Received: by mail-pa0-f42.google.com with SMTP id bj1so4514206pad.1 for ; Fri, 07 Nov 2014 15:58:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=tjb+NjfGCwa4ssseLPZMJ+P9+K+unYsdXau3OwelmSQ=; b=LVyqkCaKtdJdbIW6CcFE97NU2QnFkzZKKqQF4YcuHOFtEQdtwC4QeFQf2UhJpQqReb 6/gk9sc5tzC0K02T8uWoJF9L2pTRh9f/REeaCQ5o4eVGBGjegIC0l5QQN3JhKpbP/clz W7rwIwc10p2sg7VS35RP9+cuIhzjEdxL3fR7adtDLb00//C/atpHoCGST2tpuYX4ylcS PaPC/hxABIJK77q2b5dwm0k69t81YMvhh0b8/OSDJmZvYyUbnOGX+V2r0jO+6EDemeMh BU64Lyew5s/e58KM3gEmD34X1Hp8OvhkYN4EOykFESnww11rW7Y5zN9rIbCoGo9ma4vS 0YTw== X-Gm-Message-State: ALoCoQlgvUVaDS76JTdDBjFshMRqnDkyY6kX00/0TH+w6E3DP/GiViZnSBwjNAWN6cKhdJqAvejF X-Received: by 10.68.65.10 with SMTP id t10mr14912322pbs.156.1415404712779; Fri, 07 Nov 2014 15:58:32 -0800 (PST) Received: from localhost (50-76-60-73-ip-static.hfc.comcastbusiness.net. [50.76.60.73]) by mx.google.com with ESMTPSA id mt6sm9777735pbb.45.2014.11.07.15.58.30 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Nov 2014 15:58:31 -0800 (PST) From: Andy Lutomirski To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20Weisbecker?= , Oleg Nesterov , kvm list , Andy Lutomirski Subject: [PATCH 3/3] x86_64, entry: Remove the syscall exit audit and schedule optimizations Date: Fri, 7 Nov 2014 15:58:19 -0800 Message-Id: <1072a16a8d4ad1b11b8062f76e3236b9771b0fb6.1415403984.git.luto@amacapital.net> X-Mailer: git-send-email 1.9.3 In-Reply-To: References: In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We used to optimize rescheduling and audit on syscall exit. Now that the full slow path is reasonably fast, remove these optimizations. This adds something like 10ns to the previously optimized paths on my computer, presumably due mostly to SAVE_REST / RESTORE_REST. I think that we should eventually replace both the syscall and non-paranoid interrupt exit slow paths with a pair of C functions along the lines of the syscall entry hooks. Signed-off-by: Andy Lutomirski Acked-by: Borislav Petkov --- arch/x86/kernel/entry_64.S | 52 +++++----------------------------------------- 1 file changed, 5 insertions(+), 47 deletions(-) diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index a5afdf0f7fa4..222dc5c45ac3 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S @@ -427,15 +427,12 @@ system_call_fastpath: * Has incomplete stack frame and undefined top of stack. */ ret_from_sys_call: - movl $_TIF_ALLWORK_MASK,%edi - /* edi: flagmask */ -sysret_check: + testl $_TIF_ALLWORK_MASK,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET) + jnz int_ret_from_sys_call_fixup /* Go the the slow path */ + LOCKDEP_SYS_EXIT DISABLE_INTERRUPTS(CLBR_NONE) TRACE_IRQS_OFF - movl TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET),%edx - andl %edi,%edx - jnz sysret_careful CFI_REMEMBER_STATE /* * sysretq will re-enable interrupts: @@ -449,49 +446,10 @@ sysret_check: USERGS_SYSRET64 CFI_RESTORE_STATE - /* Handle reschedules */ - /* edx: work, edi: workmask */ -sysret_careful: - bt $TIF_NEED_RESCHED,%edx - jnc sysret_signal - TRACE_IRQS_ON - ENABLE_INTERRUPTS(CLBR_NONE) - pushq_cfi %rdi - SCHEDULE_USER - popq_cfi %rdi - jmp sysret_check - /* Handle a signal */ -sysret_signal: - TRACE_IRQS_ON - ENABLE_INTERRUPTS(CLBR_NONE) -#ifdef CONFIG_AUDITSYSCALL - bt $TIF_SYSCALL_AUDIT,%edx - jc sysret_audit -#endif - /* - * We have a signal, or exit tracing or single-step. - * These all wind up with the iret return path anyway, - * so just join that path right now. - */ +int_ret_from_sys_call_fixup: FIXUP_TOP_OF_STACK %r11, -ARGOFFSET - jmp int_check_syscall_exit_work - -#ifdef CONFIG_AUDITSYSCALL - /* - * Return fast path for syscall audit. Call __audit_syscall_exit() - * directly and then jump back to the fast path with TIF_SYSCALL_AUDIT - * masked off. - */ -sysret_audit: - movq RAX-ARGOFFSET(%rsp),%rsi /* second arg, syscall return value */ - cmpq $-MAX_ERRNO,%rsi /* is it < -MAX_ERRNO? */ - setbe %al /* 1 if so, 0 if not */ - movzbl %al,%edi /* zero-extend that into %edi */ - call __audit_syscall_exit - movl $(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT),%edi - jmp sysret_check -#endif /* CONFIG_AUDITSYSCALL */ + jmp int_ret_from_sys_call /* Do syscall tracing */ tracesys: