From patchwork Wed Oct 25 20:52:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pawan Gupta X-Patchwork-Id: 13436779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C63DCC25B6B for ; Wed, 25 Oct 2023 20:52:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230054AbjJYUw5 (ORCPT ); Wed, 25 Oct 2023 16:52:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230233AbjJYUwv (ORCPT ); Wed, 25 Oct 2023 16:52:51 -0400 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA653136; Wed, 25 Oct 2023 13:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1698267169; x=1729803169; h=date:from:to:cc:subject:message-id:mime-version; bh=nFU0gWVfaBmzYZu/sQ4XkeGrRDvj/BfRgOvVsVQkje4=; b=Y5ZOcJEvkN6QrjZ9b3ItbSYy5d9G9MwqOVOaptKEePeD4uAHKzrrMQ+4 6oz/0MXKteweeLUlEM0YMGOzf7ImQwSJwVSvayzm2q9EXUhFZyQbtZphX UlxNLDLacqwqYfuagf7LODrXjMeaDSGlk2vbpw6rMJFfCziLLvziQWCuw vX9zytI3Nq4pUzMQcu7UElIY7JstWRf+EKup2j0l4tCOcodsjM9TqibOl OhChClHt+HWNe2xVwyEut59Gp+491d+DMz6gbK0EdwZk6lMMkqefDYV25 dAQUbozr6Uij74KgLCjVyPJabaR/MvsUWJOLwSFcG0sZ9E1LOgWR2Pr6r g==; X-IronPort-AV: E=McAfee;i="6600,9927,10874"; a="390255461" X-IronPort-AV: E=Sophos;i="6.03,250,1694761200"; d="scan'208";a="390255461" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Oct 2023 13:52:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.03,250,1694761200"; d="scan'208";a="259486" Received: from kkomeyli-mobl.amr.corp.intel.com (HELO desk) ([10.251.29.139]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Oct 2023 13:52:36 -0700 Date: Wed, 25 Oct 2023 13:52:44 -0700 From: Pawan Gupta To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Josh Poimboeuf , Andy Lutomirski , Jonathan Corbet , Sean Christopherson , Paolo Bonzini , tony.luck@intel.com, ak@linux.intel.com, tim.c.chen@linux.intel.com Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, Alyssa Milburn , Daniel Sneddon , antonio.gomez.iglesias@linux.intel.com, Pawan Gupta , Alyssa Milburn , Andrew Cooper , Dave Hansen , Nikolay Borisov Subject: [PATCH v3 0/6] Delay VERW Message-ID: <20231025-delay-verw-v3-0-52663677ee35@linux.intel.com> X-B4-Tracking: v=1; b=H4sIAH19OWUC/2XMSw6CMBSF4a2Qji3pSyiO3IdxgO2t3ASLabFCC Hu3wQnG4fmT8y0kQkCI5FQsJEDCiIPPQx4KYrrW34GizZsIJiRnnFMLfTvTBOFNLVO1anR1E0a SfHgGcDht2IWQay4dxnEI86YnvvUvJNgeSpwyapw7KtZUUlT23KN/TSX6EfrSDI8NS2IPqB9AZ MBxrblUzGhW/wPrun4A7q9Tae0AAAA= X-Mailer: b4 0.12.3 MIME-Version: 1.0 Content-Disposition: inline Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org v3: - Use .entry.text section for VERW memory operand. (Andrew/PeterZ) - Fix the duplicate header inclusion. (Chao) v2: https://lore.kernel.org/r/20231024-delay-verw-v2-0-f1881340c807@linux.intel.com - Removed the extra EXEC_VERW macro layers. (Sean) - Move NOPL before VERW. (Sean) - s/USER_CLEAR_CPU_BUFFERS/CLEAR_CPU_BUFFERS/. (Josh/Dave) - Removed the comments before CLEAR_CPU_BUFFERS. (Josh) - Remove CLEAR_CPU_BUFFERS from NMI returning to kernel and document the reason. (Josh/Dave) - Reformat comment in md_clear_update_mitigation(). (Josh) - Squash "x86/bugs: Cleanup mds_user_clear" patch. (Nikolay) - s/GUEST_CLEAR_CPU_BUFFERS/CLEAR_CPU_BUFFERS/. (Josh) - Added a patch from Sean to use CFLAGS.CF for VMLAUNCH/VMRESUME selection. This facilitates a single CLEAR_CPU_BUFFERS location for both VMLAUNCH and VMRESUME. (Sean) v1: https://lore.kernel.org/r/20231020-delay-verw-v1-0-cff54096326d@linux.intel.com Hi, Legacy instruction VERW was overloaded by some processors to clear micro-architectural CPU buffers as a mitigation of CPU bugs. This series moves VERW execution to a later point in exit-to-user path. This is needed because in some cases it may be possible for kernel data to be accessed after VERW in arch_exit_to_user_mode(). Such accesses may put data into MDS affected CPU buffers, for example: 1. Kernel data accessed by an NMI between VERW and return-to-user can remain in CPU buffers (since NMI returning to kernel does not execute VERW to clear CPU buffers). 2. Alyssa reported that after VERW is executed, CONFIG_GCC_PLUGIN_STACKLEAK=y scrubs the stack used by a system call. Memory accesses during stack scrubbing can move kernel stack contents into CPU buffers. 3. When caller saved registers are restored after a return from function executing VERW, the kernel stack accesses can remain in CPU buffers(since they occur after VERW). Although these cases are less practical to exploit, moving VERW closer to ring transition reduces the attack surface. Overview of the series: Patch 1: Prepares VERW macros for use in asm. Patch 2: Adds macros to 64-bit entry/exit points. Patch 3: Adds macros to 32-bit entry/exit points. Patch 4: Enables the new macros. Patch 5: Uses CFLAGS.CF for VMLAUNCH/VMRESUME selection. Patch 6: Adds macro to VMenter. Below is some performance data collected with v1 on a Skylake client compared with previous implementation: Baseline: v6.6-rc5 | Test | Configuration | Relative | | ------------------ | ---------------------- | -------- | | build-linux-kernel | defconfig | 1.00 | | hackbench | 32 - Process | 1.02 | | nginx | Short Connection - 500 | 1.01 | Signed-off-by: Pawan Gupta --- Pawan Gupta (5): x86/bugs: Add asm helpers for executing VERW x86/entry_64: Add VERW just before userspace transition x86/entry_32: Add VERW just before userspace transition x86/bugs: Use ALTERNATIVE() instead of mds_user_clear static key KVM: VMX: Move VERW closer to VMentry for MDS mitigation Sean Christopherson (1): KVM: VMX: Use BT+JNC, i.e. EFLAGS.CF to select VMRESUME vs. VMLAUNCH Documentation/arch/x86/mds.rst | 39 ++++++++++++++++++++++++++---------- arch/x86/entry/entry.S | 16 +++++++++++++++ arch/x86/entry/entry_32.S | 3 +++ arch/x86/entry/entry_64.S | 11 ++++++++++ arch/x86/entry/entry_64_compat.S | 1 + arch/x86/include/asm/cpufeatures.h | 2 +- arch/x86/include/asm/entry-common.h | 1 - arch/x86/include/asm/nospec-branch.h | 27 ++++++++++++++----------- arch/x86/kernel/cpu/bugs.c | 15 ++++++-------- arch/x86/kernel/nmi.c | 2 -- arch/x86/kvm/vmx/run_flags.h | 7 +++++-- arch/x86/kvm/vmx/vmenter.S | 9 ++++++--- arch/x86/kvm/vmx/vmx.c | 10 ++++++--- 13 files changed, 99 insertions(+), 44 deletions(-) --- base-commit: 05d3ef8bba77c1b5f98d941d8b2d4aeab8118ef1 change-id: 20231011-delay-verw-d0474986b2c3 Best regards,