From patchwork Fri Jan 5 18:30:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Kelley X-Patchwork-Id: 13512440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76FA8C47077 for ; Fri, 5 Jan 2024 18:30:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9E826B0258; Fri, 5 Jan 2024 13:30:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D76F26B0284; Fri, 5 Jan 2024 13:30:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA00C6B0285; Fri, 5 Jan 2024 13:30:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A73EF6B0258 for ; Fri, 5 Jan 2024 13:30:46 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 70B2116076F for ; Fri, 5 Jan 2024 18:30:46 +0000 (UTC) X-FDA: 81646098492.02.97AA222 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf30.hostedemail.com (Postfix) with ESMTP id 8A5688001E for ; Fri, 5 Jan 2024 18:30:44 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DKmoTgPv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of mhkelley58@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=mhkelley58@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704479444; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/DDBxixv8I/6JBD0uZ/FnCePGFgbAZpyF75SjVpOH3s=; b=maj11rRn3AMcsc0Ccd8ybmfV9L4wvyr2SMDwFCPy7WcI/Qy442GXiseGrn7Yta1COueue2 RBlAMTOOAbSjj7Ocvt0VM332NbErtQulfZlkjQIk8OwRXAcMUvfGoh2DU8PlFMgPP5Edkw 7Th++bFTLCaB9imvHOr5EVS5j5w65Sk= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=DKmoTgPv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of mhkelley58@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=mhkelley58@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704479444; a=rsa-sha256; cv=none; b=bJLttz7rD6maYA/Iqmq5FE28VZ4byaEc4X7Sb2CqSMezSQMJ/HsVQT4uswu5l0xpAHRxX0 M8bYTprYiK2ReHY1m6FT2tZSZBMSOXkkIoZ8CuXKev89wM9/vp2zcHbpvLL0tt7ou7/AqF q8lrDYVdISyr6K5npuD4QiooD3NL5Ak= Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-28cb3bc3fe7so589492a91.1 for ; Fri, 05 Jan 2024 10:30:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704479443; x=1705084243; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/DDBxixv8I/6JBD0uZ/FnCePGFgbAZpyF75SjVpOH3s=; b=DKmoTgPvjXF/8oA/4R0l9jiZpPrhRARoJFDVf8EyaWHFLGkqnJAYSl2nAls3OR1np4 eSNskbToO9ugmJXuaR7T/Lxg7MAKsifylA2zG+oIGt1/bveT/stTk0MfyloMH1ze84Vk 6AheZsDWRYODt6NY2ol9tiV+6uMds+8S6/nvxjvM4EbmFQvgwv0li6HpZlTmPc3KkVuR 7pp7pXxycsyaQc9BuuFm5iXy4pFx2OUDhaWEqmA67BaoieZ52ACALyOy61f2jVUsGFuQ eKGZC0OtE4st4UXwJZ9j4Se2bM2NYIVMaumpA136dBMq77tk/hBapObF4XTrT+0gR+VC 5prg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704479443; x=1705084243; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=/DDBxixv8I/6JBD0uZ/FnCePGFgbAZpyF75SjVpOH3s=; b=WVj1XmEtzwFxBXrisas14N8DYNU7JQQgVQYKBPImE7gDl8zzyX1fM9L7/h1a2EGoeg MQt9a5yPi8B85FQTf3B70YlCvVCeahe+c1OBv71fs3KEs4ycbePJtFsKOeqTimwctlDq e7BCk/riQQZBM+VGExvhg+o0mmSB+zYLQSUug1ptIVNChFdk6s9+gAsgHL6dyC2AjSGO IG33ZO+bKBKN5TQFeqUNVYcIH2VB/5VIKyZjfMPemDbQS+SvnWdiG+MGVaTa0r/pAd6M oMrJbNrsP90SgrM51UsdNachJ6p3MWMkGyO7Cquk5o3gxV9+dwFTb1fdVl+25+HJDlji sfPw== X-Gm-Message-State: AOJu0YxHd27xbPRy8Qbe1QwOUdDaxK9cVz8c9RayDtXxXskioOModHoe Uhe+Bl9ZlSfz+hJovineEVo= X-Google-Smtp-Source: AGHT+IHZB8hxoZEO9XFseOS+q+ucI/+i6AVwNZFHhru24uO3VXpK5t8sRYDrNHcHS8sHO6AGVonlVw== X-Received: by 2002:a17:90a:de11:b0:28c:3620:b5ee with SMTP id m17-20020a17090ade1100b0028c3620b5eemr3251196pjv.28.1704479443490; Fri, 05 Jan 2024 10:30:43 -0800 (PST) Received: from localhost.localdomain (c-73-254-87-52.hsd1.wa.comcast.net. [73.254.87.52]) by smtp.gmail.com with ESMTPSA id 23-20020a17090a195700b002868abc0e6dsm1687293pjh.11.2024.01.05.10.30.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 10:30:43 -0800 (PST) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, kirill.shutemov@linux.intel.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, luto@kernel.org, peterz@infradead.org, akpm@linux-foundation.org, urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, thomas.lendacky@amd.com, ardb@kernel.org, jroedel@suse.de, seanjc@google.com, rick.p.edgecombe@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, linux-kernel@vger.kernel.org, linux-coco@lists.linux.dev, linux-hyperv@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 3/3] x86/hyperv: Make encrypted/decrypted changes safe for load_unaligned_zeropad() Date: Fri, 5 Jan 2024 10:30:25 -0800 Message-Id: <20240105183025.225972-4-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240105183025.225972-1-mhklinux@outlook.com> References: <20240105183025.225972-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 9b5adkobq5gcicbz5f31xd1rhd3gs4ib X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8A5688001E X-HE-Tag: 1704479444-90321 X-HE-Meta: U2FsdGVkX1/CYJOIx1ycf/i8Svjk34qQF/agdCre82zgJgv4dZkjLM3abs/ZRuA4I+Y2ugx3Df2Jh9ybXu3wdmrEsinjGBG0kbgmxczDJXAzGHjXsTp+Pqsr57hZ8oTa78Jweh6XYeL3ZkuOMaSlhP8XPGH9QVjaDNTUWsXtngZNGv/yUl4oEOYuIV4GLLlu879PQi5H0GiyvSOS44YX/Ixc7eXc+3+hGfRnr21u5ONj2VW2puUGUbvWpaWEXKcRV3nY/vEzLMMW4m/xbNoFTEXZAOAqR0sdvbiD8wpvge3tOE5qJrLy2wIzYY1lf7kmsnZ8+n4cUvxDkQcttWxAvg41gF+chFptoe5NvTmuaXL15Uq0GfBPicg5RRgc0z7jmkekfFOebh9GFO02rTosIoEjag4QR2kVrGEWPMnNQJSB5Ojzl5MfR5xgNfWSyn0mMg6DNA4dlcUxZ9yuKWDnEhOji/ph4wCqlao5cLHZQzqUeJX2VMlwJB6KCN0tz1i7Tbwknoub8yqXIO2JIZnD7LeZzyCBIJI24RSAAFla/sRlO8glaogMwSYD4GyUPq91Or5M/0eP+Asu5XLp8MATT7WiyLqUArisCvszmGpexDLizclx0ica8Pzhf2lDbmyCjdhMr+fu74QdnnVpqdd4ELHaeWdamXp9E94uxy8lt4/HJqDiJsZoFdMD9rKKoHnS5D5lQx00WA7jd8PXJ4fv0lwn5FFKAFePBfzCjz24vZxKlRg+XWGY+YgSjhE841JWZZ5C9Pml47uDfVucL5gmXdjv3GKW1NYHDVfSbxMbk1RlTWsnqDoVta+w+Zj3cDuoMEZnXt4oODiDLkhKaoX578NrwXOdA/PNaFeqnq9OvXMCNtBen2Q5it7BoDbD0Y39a2JqkUMu1Pje15ZmS0s6X+Tkts/a3BExdWgNGvU+b5dJ0OPx3x5eh/TZ08HsxxjCXBNMYYs6gSpgCZ/SKuS 87UzROjs vN8oN8G+bXb/EzsnS9IQBFPYvDO3H3z+Ysf642h0yUwHUNrSo3kRztBtAfE0bUYxwygN72SF7fqVpzsOvecFmyh0Dp4NKHeoAbFq9naLAq3Ho1DbrqYuH3fSPXenPU9+CGlgxxYIruSV8FmhHWSGz5xoooqa0FMQTxLoAx3rC6oIBN4UGmkBAb8Y71xrptnRmejqMW+o2eRY/fHqzi2phVqWZZQhNM0RrKFj5qEY9FYQ4BdhDaFBvyMPlDF7i17rTNj6BLE1WmtsOfuTL/8gGSLa9N4WhMmuCYTN9f5Pfc+BDwabXYgnyC/6htEmXG0r/QQJayWTUR1OD74W+zN4z+97x/u1SDnYBJyZVDttEOFFWv8L3ZI9KvTOfq8vEW3+l3bfiKMFb+OzuLmCy/pVmT9rjw2zoWFzgjG+F9GXFrcaE8x6RA3if5D42LG+ljsInwnjluxx3a1GPVOkxHUBcQRMSGvNIKhujVj7GNp9iTwQzZ5nYOZjNYLFKC6I/UQfdDVREegv4fBVEkWDwIMhs5VXj+0lv1Eldoujbr5vUUZI7Sex9FFXFRu6caiDUPR96u5s687D9ejXb9Z//oVEPtgLfz65Sb4bSAdEY84cy8eJKOMD1zwlsIprYLQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Michael Kelley In a CoCo VM, when transitioning memory from encrypted to decrypted, or vice versa, the caller of set_memory_encrypted() or set_memory_decrypted() is responsible for ensuring the memory isn't in use and isn't referenced while the transition is in progress. The transition has multiple steps, and the memory is in an inconsistent state until all steps are complete. A reference while the state is inconsistent could result in an exception that can't be cleanly fixed up. However, the kernel load_unaligned_zeropad() mechanism could cause a stray reference that can't be prevented by the caller of set_memory_encrypted() or set_memory_decrypted(), so there's specific code to handle this case. But a CoCo VM running on Hyper-V may be configured to run with a paravisor, with the #VC or #VE exception routed to the paravisor. There's no architectural way to forward the exceptions back to the guest kernel, and in such a case, the load_unaligned_zeropad() specific code doesn't work. To avoid this problem, mark pages as "not present" while a transition is in progress. If load_unaligned_zeropad() causes a stray reference, a normal page fault is generated instead of #VC or #VE, and the page-fault-based fixup handlers for load_unaligned_zeropad() resolve the reference. When the encrypted/decrypted transition is complete, mark the pages as "present" again. Signed-off-by: Michael Kelley Reviewed-by: Kuppuswamy Sathyanarayanan --- arch/x86/hyperv/ivm.c | 49 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 46 insertions(+), 3 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 8ba18635e338..5ad39256a5d2 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -502,6 +503,31 @@ static int hv_mark_gpa_visibility(u16 count, const u64 pfn[], return -EFAULT; } +/* + * When transitioning memory between encrypted and decrypted, the caller + * of set_memory_encrypted() or set_memory_decrypted() is responsible for + * ensuring that the memory isn't in use and isn't referenced while the + * transition is in progress. The transition has multiple steps, and the + * memory is in an inconsistent state until all steps are complete. A + * reference while the state is inconsistent could result in an exception + * that can't be cleanly fixed up. + * + * But the Linux kernel load_unaligned_zeropad() mechanism could cause a + * stray reference that can't be prevented by the caller, so Linux has + * specific code to handle this case. But when the #VC and #VE exceptions + * routed to a paravisor, the specific code doesn't work. To avoid this + * problem, mark the pages as "not present" while the transition is in + * progress. If load_unaligned_zeropad() causes a stray reference, a normal + * page fault is generated instead of #VC or #VE, and the page-fault-based + * handlers for load_unaligned_zeropad() resolve the reference. When the + * transition is complete, hv_vtom_set_host_visibility() marks the pages + * as "present" again. + */ +static bool hv_vtom_clear_present(unsigned long kbuffer, int pagecount, bool enc) +{ + return !set_memory_np(kbuffer, pagecount); +} + /* * hv_vtom_set_host_visibility - Set specified memory visible to host. * @@ -521,7 +547,7 @@ static bool hv_vtom_set_host_visibility(unsigned long kbuffer, int pagecount, bo pfn_array = kmalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL); if (!pfn_array) - return false; + goto err_set_memory_p; for (i = 0, pfn = 0; i < pagecount; i++) { /* @@ -545,14 +571,30 @@ static bool hv_vtom_set_host_visibility(unsigned long kbuffer, int pagecount, bo } } - err_free_pfn_array: +err_free_pfn_array: kfree(pfn_array); + +err_set_memory_p: + /* + * Set the PTE PRESENT bits again to revert what hv_vtom_clear_present() + * did. Do this even if there is an error earlier in this function in + * order to avoid leaving the memory range in a "broken" state. Setting + * the PRESENT bits shouldn't fail, but return an error if it does. + */ + if (set_memory_p(kbuffer, pagecount)) + result = false; + return result; } static bool hv_vtom_tlb_flush_required(bool private) { - return true; + /* + * Since hv_vtom_clear_present() marks the PTEs as "not present" + * and flushes the TLB, they can't be in the TLB. That makes the + * flush controlled by this function redundant, so return "false". + */ + return false; } static bool hv_vtom_cache_flush_required(void) @@ -615,6 +657,7 @@ void __init hv_vtom_init(void) x86_platform.hyper.is_private_mmio = hv_is_private_mmio; x86_platform.guest.enc_cache_flush_required = hv_vtom_cache_flush_required; x86_platform.guest.enc_tlb_flush_required = hv_vtom_tlb_flush_required; + x86_platform.guest.enc_status_change_prepare = hv_vtom_clear_present; x86_platform.guest.enc_status_change_finish = hv_vtom_set_host_visibility; /* Set WB as the default cache mode. */