From patchwork Mon May 13 14:38:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AD221390 for ; Mon, 13 May 2019 14:39:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B3CD27FAE for ; Mon, 13 May 2019 14:39:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1EF482817F; Mon, 13 May 2019 14:39:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 952F527FAE for ; Mon, 13 May 2019 14:39:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00DD46B000A; Mon, 13 May 2019 10:39:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E8B786B000C; Mon, 13 May 2019 10:39:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB6F06B000D; Mon, 13 May 2019 10:39:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by kanga.kvack.org (Postfix) with ESMTP id 9E52C6B000A for ; Mon, 13 May 2019 10:39:10 -0400 (EDT) Received: by mail-io1-f70.google.com with SMTP id t7so9968082iof.21 for ; Mon, 13 May 2019 07:39:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=1/A8dPg6N3BCp1dVVHZldE7IkiS+8kPsjdwBuin2g4s=; b=ERTGHeUuiioq1OWvXy6sEDTDYd7llPdRfyr9iKsmfOKLYWUUAeircmg8EHe+QaSFJM 1kDlA7L7tbhamUCF36fiQOh3jDH6Jff8U9qsYhA9K0B8zCeAdeyDcOXWrXQkL7ah98Cm 3Ru+qn3IsMt19KemE46PB5ryXppc3TEjdYfTV720QsT4tjWQiyiflSl99wq7MTQ9BSCX HmPqC06ivWBXkly3Zrptzy2f5xhqcz2hL0Y2gs8lVNFJsfzBl/HrInyn/QP4T00sWX+h gMb37R9//Cy6+KEFPDEhwtOGNWxZ3kNd6eoKnbntVkZDzVxM488p3E5afMJxlCf7de7N /rPQ== X-Gm-Message-State: APjAAAXd8pWjJyx2iYh0t+w2j5pwBP+4Fv6ooZEnjYU+sJV/Ej9oVfJ9 jfqn3eC2Ksr0gqWs+TMWCpVtmvJcbO4A0nZ8neL2+bmSZJkCLe4CkeTW2yytApcE035KNI7ECl+ z+hLYy/WtTCYTOMbaIlkD2cWAShUhlQNGrSIkeqvdtLRsXfJIRIvQF3xR1q5SL3pL7Q== X-Received: by 2002:a5e:cb47:: with SMTP id h7mr13273714iok.69.1557758350389; Mon, 13 May 2019 07:39:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqwQRIyc6T+QCpSHaHEnXG902/BpiESTYryP4+wPP4TN7G+goh2cLkfWU/YrGAHTV+W+gnzq X-Received: by 2002:a5e:cb47:: with SMTP id h7mr13273671iok.69.1557758349703; Mon, 13 May 2019 07:39:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758349; cv=none; d=google.com; s=arc-20160816; b=XbeooR9vGBUgnKe+RMnUiRuxBB0pQJVJe/iW5VKViCGZ2s/2II5sqnmaDyV8UQm6tc ju90yF5zO8fUhDVEPbVZZ/GsuXDurv6F+nU8BLmiVe2geImFkBEZA6hNASKscOvqUh1+ eZ5/qgpkCgjq62o+9DkhggNn0C+AKeEI3Ny1xtYmlsLbZKphQ9z5jzoIjLbQPv/h+vK1 fQN6FAZBzeGheTHyI/tjoowENreeRYDBNoWT9YkJuOxH5BU8EF5n560KJT9Xgc81EFyk tjIBCNwqiPJP2nPUfA5RCbSOyZrsYq9xPkO751UjSlDiijA0tfP7Lurd5V0D1T9JxKYZ ORbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1/A8dPg6N3BCp1dVVHZldE7IkiS+8kPsjdwBuin2g4s=; b=qoEYpSZd0TTjq1dNK3yNuzXxIZkC7S5UpKQIzFYrydEirbvDBGVvIb8iBxkh3MVy02 pMeB/yFnsjhXIuvrw/XNhOTavS8qF0vp7qXYI1tgaLGTDNzhCqTRizNyGvE6DXaQd3ys ucQF3ufGIjnM1ZJx3ZZRhtbv637eEGsWPK1BqgCAqXcnmMmLEwnZW4i186gN8RmmWbVK 6FKgQZs2SDJalK3T6tZO7e12JMycCwViyibKz8pe0yhZ3M05wbTAjSRLdgVwmKToftK8 rYbGZmf5/hvqDwj5WQ8B63iVdyHWdsrqOduoHgk0yTe+Ztc0dF4iWgTIW6gfr8ob3TPx BpQw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=IIHbvoJP; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id h1si8195330itl.18.2019.05.13.07.39.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:09 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=IIHbvoJP; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DESwxh183032; Mon, 13 May 2019 14:38:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=1/A8dPg6N3BCp1dVVHZldE7IkiS+8kPsjdwBuin2g4s=; b=IIHbvoJP5+K/6hJmjo6mwDCr+XYe5WkcmTj/jZgsHQd++1A3nrRugabuYhfepHXTNKMa V+W5LQ4oABEgMC2/bRAoDFfs1URTJrv1FkjV3nZfJhEh3aTydJM7O9zVMLE+XCLnCNo0 mgaCYQLJsGwxxqQvTsOBwgrZT+TlGys5Y+xXj8Q2b2tjQKrJs5qkBiGxQsdVfxbT4cvm AvsvkAn4MmTs1z4WmXxi/1KuzDaA9HLi5wH7yN5hYhVDuggPf0x5mUO+bX8AKL3WJ1Pd Vh+iMejI0gUwJY+npXZCvzmO6WQnPg//1CrX7b7e8J9CRrNauyV7gxxNmcg6wsYTegjc qw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfksf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:38:50 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQ4022780; Mon, 13 May 2019 14:38:42 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 01/27] kernel: Export memory-management symbols required for KVM address space isolation Date: Mon, 13 May 2019 16:38:09 +0200 Message-Id: <1557758315-12667-2-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130102 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon Export symbols needed to create, manage, populate and switch a mm from a kernel module (kvm in this case). This is a hacky way for now to start. This should be changed to some suitable memory-management API. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/kernel/ldt.c | 1 + arch/x86/mm/tlb.c | 3 ++- mm/memory.c | 5 +++++ 3 files changed, 8 insertions(+), 1 deletions(-) diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c index b2463fc..19a86e0 100644 --- a/arch/x86/kernel/ldt.c +++ b/arch/x86/kernel/ldt.c @@ -401,6 +401,7 @@ void destroy_context_ldt(struct mm_struct *mm) free_ldt_struct(mm->context.ldt); mm->context.ldt = NULL; } +EXPORT_SYMBOL_GPL(destroy_context_ldt); void ldt_arch_exit_mmap(struct mm_struct *mm) { diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 7f61431..a4db7f5 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -70,7 +70,7 @@ static void clear_asid_other(void) } atomic64_t last_mm_ctx_id = ATOMIC64_INIT(1); - +EXPORT_SYMBOL_GPL(last_mm_ctx_id); static void choose_new_asid(struct mm_struct *next, u64 next_tlb_gen, u16 *new_asid, bool *need_flush) @@ -159,6 +159,7 @@ void switch_mm(struct mm_struct *prev, struct mm_struct *next, switch_mm_irqs_off(prev, next, tsk); local_irq_restore(flags); } +EXPORT_SYMBOL_GPL(switch_mm); static void sync_current_stack_to_mm(struct mm_struct *mm) { diff --git a/mm/memory.c b/mm/memory.c index 36aac68..ede9335 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -434,6 +434,7 @@ int __pte_alloc(struct mm_struct *mm, pmd_t *pmd) pte_free(mm, new); return 0; } +EXPORT_SYMBOL_GPL(__pte_alloc); int __pte_alloc_kernel(pmd_t *pmd) { @@ -453,6 +454,7 @@ int __pte_alloc_kernel(pmd_t *pmd) pte_free_kernel(&init_mm, new); return 0; } +EXPORT_SYMBOL_GPL(__pte_alloc_kernel); static inline void init_rss_vec(int *rss) { @@ -4007,6 +4009,7 @@ int __p4d_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address) spin_unlock(&mm->page_table_lock); return 0; } +EXPORT_SYMBOL_GPL(__p4d_alloc); #endif /* __PAGETABLE_P4D_FOLDED */ #ifndef __PAGETABLE_PUD_FOLDED @@ -4039,6 +4042,7 @@ int __pud_alloc(struct mm_struct *mm, p4d_t *p4d, unsigned long address) spin_unlock(&mm->page_table_lock); return 0; } +EXPORT_SYMBOL_GPL(__pud_alloc); #endif /* __PAGETABLE_PUD_FOLDED */ #ifndef __PAGETABLE_PMD_FOLDED @@ -4072,6 +4076,7 @@ int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address) spin_unlock(ptl); return 0; } +EXPORT_SYMBOL_GPL(__pmd_alloc); #endif /* __PAGETABLE_PMD_FOLDED */ static int __follow_pte_pmd(struct mm_struct *mm, unsigned long address, From patchwork Mon May 13 14:38:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941035 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ABB796C5 for ; Mon, 13 May 2019 14:39:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9B5C627FAE for ; Mon, 13 May 2019 14:39:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8DFEA2817F; Mon, 13 May 2019 14:39:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 09CFF27FAE for ; Mon, 13 May 2019 14:39:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEF2A6B0007; Mon, 13 May 2019 10:39:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D78EF6B0008; Mon, 13 May 2019 10:39:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C19716B000A; Mon, 13 May 2019 10:39:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) by kanga.kvack.org (Postfix) with ESMTP id A18576B0007 for ; Mon, 13 May 2019 10:39:09 -0400 (EDT) Received: by mail-io1-f69.google.com with SMTP id y15so9985515iod.10 for ; Mon, 13 May 2019 07:39:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=03NJAuy8gQRMw5NMdO5oIpncwVrdcNhEn/qnXsnIh+Q=; b=hzaFK8hVnJVyjg4MTGgXJakwvRGRjh6j8DG3hmU75JphWnfrfvWKEdTjqZyswnYItj qEXjuUR/rGXXjk4E3X8oJC+BuvfIJpxsnKimjizuG6LSxTWxNmPb0jEaL8zwOH9VjcJL qZ2L89CHSrIYLCB4IV3FEnksJ1ngL3OQq1Qpz091sBdnegicTSbVmYqGHYV5faQucfAi tpuZKa8hoc+MP42bT9RxOL7Aht5+mp27lORdnekt2Ve7yJ6QmJoStsBVnQIhG0iYzzW1 CjZXAgfdUgbikwRgK9LRH5HeJzz4F5/M1767qcx6wTGfy/jAcdEeyGBN0xYjY+ehE1Rs Qjxw== X-Gm-Message-State: APjAAAUEDG3sY17PX/oBhmAce+s/VWyYyuKZCL5MfSzeX1VEg+mpyTV0 XEUruJdcbvPS2AzCXsJ7LOXfrvL+ThEs5aAn0brBLs/J4eHthY8dXiMZUlbdw1bPEz3d+RJ9vyO 30Gq+NL7/ldC1+KO3ur+KOR9B0ZJNuqyok2tbkkA87LpdW5f3UgjTiU4lpkbrVAOmqA== X-Received: by 2002:a02:1a45:: with SMTP id 66mr17674807jai.124.1557758349296; Mon, 13 May 2019 07:39:09 -0700 (PDT) X-Google-Smtp-Source: APXvYqxmAEsDXCkJdMabxYV5G37EOXIhrFAI5/rbbw5RsQdWFiza607CIJKqVO6rmCcxJZjNzOoW X-Received: by 2002:a02:1a45:: with SMTP id 66mr17674744jai.124.1557758348550; Mon, 13 May 2019 07:39:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758348; cv=none; d=google.com; s=arc-20160816; b=pNYF/ZqkUvIUU24sB7Ko/wuMehgceK+XAXO5PxEleFRhaOQPPUDLzccgOo9x6QqoMk Ci7HGiyzcYnpJg25/LwcgFRCbd1tVtc7TYWameGJtK/KLpYbXSHcUZgCcx6bA4Noz1dC pdF1/08GWGpXIAMgq/5xtZMwwJICaZpQWO0Cki+5aJCyYX4G0N/9DLypr66gHVulIjev vBO5Nbx2pGbxLLKGEL+rAcrjASSdQu6W5/IBDWE5/kJ+woYOCiPAfuiG0PLeBRvxzixl oQvYCRmXg+tAgYueh0APyoVTjfbHMHqd0jJyFw5gDizjH52ouOooioNRXSqRzszqKyHm vvBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=03NJAuy8gQRMw5NMdO5oIpncwVrdcNhEn/qnXsnIh+Q=; b=iL0u9YCcBcxk14eMcOK0ZXCJPND+iK/UFoop3jbnvQf/Gl2lpD4bmT+tvPoHNKj0uz r8naNgcmMyuWYlcetIB0MMOarPbgoIt6FWgY5qo3nhpehbeKQe+6utoDH2KG4wAjuMeT aVh0E3Yjl2U49NWFVa+/o13/ZKQVbu/0ckRzjqx74XGtQAlM8uaih6VGtX3DSVM6NV4v nvLAEjAFhCH/MYeb6fb7PXtlxeooC3pEyyzLoaDrh0BdOxIYRR20IyaZ7Nn7IjV4Z4a+ fkiOzAappmmoUHYa3v8q79+iHUr2Sb3Vu0ts6Lani9CXw1xHL5G66+Ykw+Ap6fwUXVU9 cUMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=g5WaOGjM; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id o192si8317943itb.38.2019.05.13.07.39.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:08 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=g5WaOGjM; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DESr12184826; Mon, 13 May 2019 14:38:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=03NJAuy8gQRMw5NMdO5oIpncwVrdcNhEn/qnXsnIh+Q=; b=g5WaOGjMCQGcw3yFQID9Xz5SR+orOYV1jEvqta9pP1dW+3vk6S7rk3Xf0Xqs9c72Ro1w /TCodkQB+5Z4YQjeExvs0QvsKAbjAzfxUXW21GiZ7ZoPR7nSWQ/5N76ENeSlLIaBPOuq oraTRR4s8rulSGRtJmfzQ6I6tr45ficozvt9Zf9IR0tS1V0R/j/qCwAykrzu4TBWku12 vyAVK6nNFxTgtyozBfwL3irrf/S3fPxxNjChwJoT7jqQ+3BTuw9SWcfOTckw2b1l05Nd oG/FOBhG00v9BsVeDYzcuV3eTXQRhjHKRHxD7Luw1u+N5LvXr+eEUi8yT9Gj8XmE4JX0 Bg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7aqq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:38:48 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQ5022780; Mon, 13 May 2019 14:38:45 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 02/27] KVM: x86: Introduce address_space_isolation module parameter Date: Mon, 13 May 2019 16:38:10 +0200 Message-Id: <1557758315-12667-3-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130102 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon Add the address_space_isolation parameter to the kvm module. When set to true, KVM #VMExit handlers run in isolated address space which maps only KVM required code and per-VM information instead of entire kernel address space. This mechanism is meant to mitigate memory-leak side-channels CPU vulnerabilities (e.g. Spectre, L1TF and etc.) but can also be viewed as security in-depth as it also helps generically against info-leaks vulnerabilities in KVM #VMExit handlers and reduce the available gadgets for ROP attacks. This is set to false by default because it incurs a performance hit which some users will not want to take for security gain. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/isolation.c | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletions(-) create mode 100644 arch/x86/kvm/isolation.c diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 31ecf7a..9f404e9 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -10,7 +10,7 @@ kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \ - hyperv.o page_track.o debugfs.o + hyperv.o page_track.o debugfs.o isolation.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o vmx/evmcs.o vmx/nested.o kvm-amd-y += svm.o pmu_amd.o diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c new file mode 100644 index 0000000..e25f663 --- /dev/null +++ b/arch/x86/kvm/isolation.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved. + * + * KVM Address Space Isolation + */ + +#include +#include + +/* + * When set to true, KVM #VMExit handlers run in isolated address space + * which maps only KVM required code and per-VM information instead of + * entire kernel address space. + * + * This mechanism is meant to mitigate memory-leak side-channels CPU + * vulnerabilities (e.g. Spectre, L1TF and etc.) but can also be viewed + * as security in-depth as it also helps generically against info-leaks + * vulnerabilities in KVM #VMExit handlers and reduce the available + * gadgets for ROP attacks. + * + * This is set to false by default because it incurs a performance hit + * which some users will not want to take for security gain. + */ +static bool __read_mostly address_space_isolation; +module_param(address_space_isolation, bool, 0444); From patchwork Mon May 13 14:38:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED95F1390 for ; Mon, 13 May 2019 14:39:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE79D27FAE for ; Mon, 13 May 2019 14:39:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D1E4D2817F; Mon, 13 May 2019 14:39:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4009127FAE for ; Mon, 13 May 2019 14:39:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69CE86B0008; Mon, 13 May 2019 10:39:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 626216B000A; Mon, 13 May 2019 10:39:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C5CD6B000C; Mon, 13 May 2019 10:39:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f197.google.com (mail-it1-f197.google.com [209.85.166.197]) by kanga.kvack.org (Postfix) with ESMTP id 2FB7B6B0008 for ; Mon, 13 May 2019 10:39:10 -0400 (EDT) Received: by mail-it1-f197.google.com with SMTP id z125so12371547itf.4 for ; Mon, 13 May 2019 07:39:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=N+bhv6OONCtHadzhgbU1e3MAiku0pVi0en6iW2c0TrE=; b=S8ABYTe2IRA8df7yr9dp+8UgCcn64Rmvw7VL0XpYSc2gF7J4GX5tPKZ4B6SCsimI75 5xG+t3O8XtLM26k0c2I8AMRqFoGLNAONseEZLgP/E/VPWh9k7TyMKpXqZnzu6FG7qBSC aPMbR5oVjwv9wnAIBSV3KbWK8cwzzGwvrskdF5ptnNoiXla9vOM2rFnJh1Mw60ON/Hho /3/CH6zfZ6NYc6ivKENt0kkC2IBkrsTWsujN28gxd5GK79odCWjd7WI7M4ZnxtgG6ZpC Znyt9QwBVzgp+fcZ9CxMVlFuU4da89nvetbenJjYHzWjSPuV8etqutXSn4nH2BUpjM8t rmPw== X-Gm-Message-State: APjAAAXfSWtYe5t23Ai+oDS7ladS8zNCvNjMYHn8md2Q4RUZwrcww2kN tzQAJrkp6XrqyR8L0JN6+TJatcd2kLqDVYMZPqYWx1gFnxiDZviK4JMe9QcYk7XWrvjccs1Xzhx j9r5r33fKKlIDuzVZ5jNQ8Il5NLdvIL9OOWGOqruMfGfp9baidGsgPc6onnCz56gImg== X-Received: by 2002:a24:4f4a:: with SMTP id c71mr19979814itb.65.1557758349853; Mon, 13 May 2019 07:39:09 -0700 (PDT) X-Google-Smtp-Source: APXvYqyhB1+omrxlUzOuk8BY18l7BBCzoolPoDQLGO86ucfvN/xgkCwv20SwjVRy6NHVuAN9B4+n X-Received: by 2002:a24:4f4a:: with SMTP id c71mr19979710itb.65.1557758348553; Mon, 13 May 2019 07:39:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758348; cv=none; d=google.com; s=arc-20160816; b=f1B6xduM9cUUq+NoaRctMMBqmRSfexjnSH33ravRlE8cwvu+hCf3JAWLXrOYE5OMDG 6+wnoh8VBXZhIKijyDWKXiGu1JCtjBfuaYwCp+SiCuORJUP5/+mUCVaACbi8ODjX+rWZ dQhsytTrqEsuEcRY5DImAsnaarYnKvGBQgmamS9H56OGSe0UZaFRKci8OFfmoz+cv3pK dWTjgRkkvamMAn/R+hDcW+Sy19o6cPzpTI/oSKd2ZEo/+sCCbqW2zXKyVbAkEf8fvjIy gmTD2ZUEVnPsc/z+d7/zMSGAq0ZcPp4nm01NF5c7FfjjCb4jXQDjvqs7MPNsKsBKbits xIUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=N+bhv6OONCtHadzhgbU1e3MAiku0pVi0en6iW2c0TrE=; b=KLD0uit/9xK042XGSkN6DEynNR8748BewZfwCnqBZrmO9d5KX8NutrRlqFFOljb4pE /39cxlu0MOnrZrxXb0d+Ysh6GQG/+l8vblVZIbFmbgst90haUT4supycGZ0CUxckPZRc lUzB1wai+Bh8ZbGsCmWFyhDOUjhXfBLMJNRyEVqqhxXx4YxyBH3NRv5GPASQoAgnFcqL VYtqbN/Pm1NpnUr2K4qtdNTTwmTsJOj5M33ichNjHmpuSUvdxnia+ZLrStfUcMY/x8Ip jobTWxh/irY/SAUEB5skA52PfLsfr9RBXl8ZlO4OLGsVzDUWTmqhAgBPghGt2PRWPuFx QsgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=P2E6fPrV; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id w1si7553576iop.69.2019.05.13.07.39.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:08 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=P2E6fPrV; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DESuQq184844; Mon, 13 May 2019 14:38:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=N+bhv6OONCtHadzhgbU1e3MAiku0pVi0en6iW2c0TrE=; b=P2E6fPrVnhroaq8muel0nn2IGVthj4KsZusNGeNWJzt9GowNKpeWiTFc/b69anfJ0Jmu oveuhNt89Ikw71ZvGW5vaL++kydq+j4PLd4quL0ofYa2FFOqRahZ4/VLfHaGhpQ4vHLA m5ceIwPWsPji5If/gseVw1NXQQCm7CUpEO9gRgNKuigMbubyHEiBRLZ2oXHfdXvE5+yg 4Vxc4Q0CWq9Ih+MZt7QCjMIpS8BEJfgPaEsyU1ULUJzIGUT76jHo7eJhX4dTAjgX0kjG sJYESl+2wGBWqLkwymFi6MuDOCTCpC/bVqNxgtfnEiuBaKSkXU7jAwqSWP/dnuQqoRnn 9A== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7as5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:38:56 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQ6022780; Mon, 13 May 2019 14:38:47 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 03/27] KVM: x86: Introduce KVM separate virtual address space Date: Mon, 13 May 2019 16:38:11 +0200 Message-Id: <1557758315-12667-4-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130102 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon Create a separate mm for KVM that will be active when KVM #VMExit handlers run. Up until the point which we architectully need to access host (or other VM) sensitive data. This patch just create kvm_mm but never makes it active yet. This will be done by next commits. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 8 ++++ arch/x86/kvm/x86.c | 10 ++++- 3 files changed, 112 insertions(+), 1 deletions(-) create mode 100644 arch/x86/kvm/isolation.h diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index e25f663..74bc0cd 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -7,6 +7,21 @@ #include #include +#include + +#include +#include + +#include "isolation.h" + +struct mm_struct kvm_mm = { + .mm_rb = RB_ROOT, + .mm_users = ATOMIC_INIT(2), + .mm_count = ATOMIC_INIT(1), + .mmap_sem = __RWSEM_INITIALIZER(kvm_mm.mmap_sem), + .page_table_lock = __SPIN_LOCK_UNLOCKED(kvm_mm.page_table_lock), + .mmlist = LIST_HEAD_INIT(kvm_mm.mmlist), +}; /* * When set to true, KVM #VMExit handlers run in isolated address space @@ -24,3 +39,83 @@ */ static bool __read_mostly address_space_isolation; module_param(address_space_isolation, bool, 0444); + +static int kvm_isolation_init_mm(void) +{ + pgd_t *kvm_pgd; + gfp_t gfp_mask; + + gfp_mask = GFP_KERNEL | __GFP_ZERO; + kvm_pgd = (pgd_t *)__get_free_pages(gfp_mask, PGD_ALLOCATION_ORDER); + if (!kvm_pgd) + return -ENOMEM; + +#ifdef CONFIG_PAGE_TABLE_ISOLATION + /* + * With PTI, we have two PGDs: one the kernel page table, and one + * for the user page table. The PGD with the kernel page table has + * to be the entire kernel address space because paranoid faults + * will unconditionally use it. So we define the KVM address space + * in the user table space, although it will be used in the kernel. + */ + + /* initialize the kernel page table */ + memcpy(kvm_pgd, current->active_mm->pgd, sizeof(pgd_t) * PTRS_PER_PGD); + + /* define kvm_mm with the user page table */ + kvm_mm.pgd = kernel_to_user_pgdp(kvm_pgd); +#else /* CONFIG_PAGE_TABLE_ISOLATION */ + kvm_mm.pgd = kvm_pgd; +#endif /* CONFIG_PAGE_TABLE_ISOLATION */ + mm_init_cpumask(&kvm_mm); + init_new_context(NULL, &kvm_mm); + + return 0; +} + +static void kvm_isolation_uninit_mm(void) +{ + pgd_t *kvm_pgd; + + BUG_ON(current->active_mm == &kvm_mm); + + destroy_context(&kvm_mm); + +#ifdef CONFIG_PAGE_TABLE_ISOLATION + /* + * With PTI, the KVM address space is defined in the user + * page table space, but the full PGD starts with the kernel + * page table space. + */ + kvm_pgd = user_to_kernel_pgdp(kvm_pgd); +#else /* CONFIG_PAGE_TABLE_ISOLATION */ + kvm_pgd = kvm_mm.pgd; +#endif /* CONFIG_PAGE_TABLE_ISOLATION */ + kvm_mm.pgd = NULL; + free_pages((unsigned long)kvm_pgd, PGD_ALLOCATION_ORDER); +} + +int kvm_isolation_init(void) +{ + int r; + + if (!address_space_isolation) + return 0; + + r = kvm_isolation_init_mm(); + if (r) + return r; + + pr_info("KVM: x86: Running with isolated address space\n"); + + return 0; +} + +void kvm_isolation_uninit(void) +{ + if (!address_space_isolation) + return; + + kvm_isolation_uninit_mm(); + pr_info("KVM: x86: End of isolated address space\n"); +} diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h new file mode 100644 index 0000000..cf8c7d4 --- /dev/null +++ b/arch/x86/kvm/isolation.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ARCH_X86_KVM_ISOLATION_H +#define ARCH_X86_KVM_ISOLATION_H + +extern int kvm_isolation_init(void); +extern void kvm_isolation_uninit(void); + +#endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b5edc8e..4b7cec2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -29,6 +29,7 @@ #include "cpuid.h" #include "pmu.h" #include "hyperv.h" +#include "isolation.h" #include #include @@ -6972,10 +6973,14 @@ int kvm_arch_init(void *opaque) goto out_free_x86_fpu_cache; } - r = kvm_mmu_module_init(); + r = kvm_isolation_init(); if (r) goto out_free_percpu; + r = kvm_mmu_module_init(); + if (r) + goto out_uninit_isolation; + kvm_set_mmio_spte_mask(); kvm_x86_ops = ops; @@ -7000,6 +7005,8 @@ int kvm_arch_init(void *opaque) return 0; +out_uninit_isolation: + kvm_isolation_uninit(); out_free_percpu: free_percpu(shared_msrs); out_free_x86_fpu_cache: @@ -7024,6 +7031,7 @@ void kvm_arch_exit(void) #ifdef CONFIG_X86_64 pvclock_gtod_unregister_notifier(&pvclock_gtod_notifier); #endif + kvm_isolation_uninit(); kvm_x86_ops = NULL; kvm_mmu_module_exit(); free_percpu(shared_msrs); From patchwork Mon May 13 14:38:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941059 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 782C315AB for ; Mon, 13 May 2019 14:39:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68FE027FAE for ; Mon, 13 May 2019 14:39:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5D2042817F; Mon, 13 May 2019 14:39:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA6B527FAE for ; Mon, 13 May 2019 14:39:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0C056B026C; Mon, 13 May 2019 10:39:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A94296B026D; Mon, 13 May 2019 10:39:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 872A86B026E; Mon, 13 May 2019 10:39:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by kanga.kvack.org (Postfix) with ESMTP id 647EF6B026D for ; Mon, 13 May 2019 10:39:24 -0400 (EDT) Received: by mail-io1-f70.google.com with SMTP id t7so9968534iof.21 for ; Mon, 13 May 2019 07:39:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=GS3GihsPyz7Y99NHOiQDTtmA0apAWQh5Zi8mHVLNyoo=; b=FLYyjDQcBuD4avZ+Qy/pj+cV38PJNjW9Mix+dRF5KIrLvTyQJejuOlFWpzcuMxZfu0 USa2MiJl5dLCPwMdO0X0HWPdNcrea90UjWSWBx2ywn3Cbi7+e0WOStJK1Jd80JkTop+R 4WbR1+17PZAAIv9MWjfC1JNGz8WyeD09fzMpu6+4Nczo8vJ1BOGQxXM67rEZPTW/xXzJ UhrXgxZzxoQn7dm6SXMOI2UziP11mu1fbsqRCODtmw+3PGZhyfrT5TG3spkb9YxuAjeq KXLcop1juMuR1vVoVRrDfNvP/bQ8lj0jBQmvHsSMUk82iOOVLB47M4DSqbJS1zyz703E RdNg== X-Gm-Message-State: APjAAAV5WCkgg9Do5LpDUiste5MCb1Tf4cxKfcQo0NB3E3oRckX18Ea1 o8HtbyBkOB111c0guELr38nNpNxWTj9A5UV0XtetRRlUT57UovBpM0pz9qomBzQypF8C76Gkfg1 Cv56H6MOlPRJHT+sOyVsfx2zwuAE5TSGZwuXXeXongrd/haxdORybqhR5c1glijrP7Q== X-Received: by 2002:a6b:c046:: with SMTP id q67mr17080730iof.157.1557758364147; Mon, 13 May 2019 07:39:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqwdbo7JLruPmcq4GNDhZvuhPWyq3SGvkcS+u81HPBOdD15TxIRqdvugTpsGX5FZzY3sDpwK X-Received: by 2002:a6b:c046:: with SMTP id q67mr17080681iof.157.1557758363420; Mon, 13 May 2019 07:39:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758363; cv=none; d=google.com; s=arc-20160816; b=iA82Hb/iXOH2W8fS6BqNqH39ley0pD27DxfTxGbTCpnXIY1WkFDJYMM2e9JuRmDh6V 4DgZv6KtEz1rDpJUwmPCKAta5moio224omhl0KdA1wVBEYMW0E684cBYfspO6AXAkvUZ dn+FWYp9faYJaEj/clhA3BHXn4vct7+XWW/LdDCxw08WvBalfHmUv3/8ivUSiMvxRDU0 ++bEkVtS0tAJO6Zm2mFzEVXH8vU3GSIzV3+l0qHWw7PivHe/54aLvlt8K43kyaQgwaRP 0YtxhmMeq6p2i/4Ofj9kp5w1hhB7UItG0ryzN0zoG85AZg7TlXxAHhPRGBauz9ZiAfzP p7Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=GS3GihsPyz7Y99NHOiQDTtmA0apAWQh5Zi8mHVLNyoo=; b=iZJIS1uSSblQRFIhJgTDTqsd/08RUlExVcA78YB4NtFhQKFSOmzxvA5Pk0QLgupf8T UQ/K/i3pmUYft2PH45utW4FcIr+2JaOwUixyZ/PGaWOsGqo9wSR/5o0KDIsZlp+Fi2Vj TGcJzI7lGvo3CkXPU/I2atlB2ajbmEMhmg6kfq41oqagfdsGwpPtD8sj3uFI8Eh1wmk4 uK/dh1fktwwhJTUB6uOHuQTlrCGgNF5HvtxJDVFRq5P/ZVfTjqkjhDTUOljXeDmJRM54 qdu1qSoUn/ZNIdaBEktJl16b9u1rIB4IMonv5qr5/F1upMii5h8o46YpmiwlKMUmXIJX n1GA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=mJcTtw57; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2130.oracle.com (userp2130.oracle.com. [156.151.31.86]) by mx.google.com with ESMTPS id b123si8132262ith.61.2019.05.13.07.39.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:23 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) client-ip=156.151.31.86; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=mJcTtw57; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DESlCI171427; Mon, 13 May 2019 14:38:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=GS3GihsPyz7Y99NHOiQDTtmA0apAWQh5Zi8mHVLNyoo=; b=mJcTtw571vBY/DqchePtoK+a2fMaLqNsWff99ts41cz3uW81AVh8k2kIjQIXfkadX5Q3 6F/E6e4/O6myTDqiwNYcUSpAOVMeeIBt5z6sBwlWmvIiYKdfzx0Qg4nsfGs0iR0Ee/ws 8fyDT/ryZpc/Obi/2zRq7DJqs0810bOvF26HLCD7ir3wtStKFUFAa7Ws6PKz+Pu1UJQL dU9UJa5EK8+LYETRyQ5DuH2DbGMtHYVtA2jgoPUK3ziD1XjXnCkAD9TDmm7V4635R9i5 XflzqnTQJehHeZrV5CJoVBXa8ETCmtisHC5rPPYYhukBl0cMbZnPTDNSWhzzDSLkexDm Fw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2sdnttfecf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:38:54 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQ7022780; Mon, 13 May 2019 14:38:50 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 04/27] KVM: x86: Switch to KVM address space on entry to guest Date: Mon, 13 May 2019 16:38:12 +0200 Message-Id: <1557758315-12667-5-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130102 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon Switch to KVM address space on entry to guest and switch out on immediately at exit (before enabling host interrupts). For now, this is not effectively switching, we just remain on the kernel address space. In addition, we switch back as soon as we exit guest, which makes KVM #VMExit handlers still run with full host address space. However, this introduces the entry points and places for switching. Next commits will change switch to happen only when necessary. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 20 ++++++++++++++++++++ arch/x86/kvm/isolation.h | 2 ++ arch/x86/kvm/x86.c | 8 ++++++++ 3 files changed, 30 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 74bc0cd..35aa659 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -119,3 +119,23 @@ void kvm_isolation_uninit(void) kvm_isolation_uninit_mm(); pr_info("KVM: x86: End of isolated address space\n"); } + +void kvm_isolation_enter(void) +{ + if (address_space_isolation) { + /* + * Switches to kvm_mm should happen from vCPU thread, + * which should not be a kernel thread with no mm + */ + BUG_ON(current->active_mm == NULL); + /* TODO: switch to kvm_mm */ + } +} + +void kvm_isolation_exit(void) +{ + if (address_space_isolation) { + /* TODO: Kick sibling hyperthread before switch to host mm */ + /* TODO: switch back to original mm */ + } +} diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index cf8c7d4..595f62c 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -4,5 +4,7 @@ extern int kvm_isolation_init(void); extern void kvm_isolation_uninit(void); +extern void kvm_isolation_enter(void); +extern void kvm_isolation_exit(void); #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4b7cec2..85700e0 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7896,6 +7896,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) goto cancel_injection; } + kvm_isolation_enter(); + if (req_immediate_exit) { kvm_make_request(KVM_REQ_EVENT, vcpu); kvm_x86_ops->request_immediate_exit(vcpu); @@ -7946,6 +7948,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); + /* + * TODO: Move this to where we architectually need to access + * host (or other VM) sensitive data + */ + kvm_isolation_exit(); + vcpu->mode = OUTSIDE_GUEST_MODE; smp_wmb(); From patchwork Mon May 13 14:38:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941043 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9978E1390 for ; Mon, 13 May 2019 14:39:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89EFD27FAE for ; Mon, 13 May 2019 14:39:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7DBFB2817F; Mon, 13 May 2019 14:39:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 038EA27FAE for ; Mon, 13 May 2019 14:39:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 95FBD6B000D; Mon, 13 May 2019 10:39:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8E9156B000E; Mon, 13 May 2019 10:39:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 678176B0010; Mon, 13 May 2019 10:39:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f198.google.com (mail-it1-f198.google.com [209.85.166.198]) by kanga.kvack.org (Postfix) with ESMTP id 3C5AC6B000E for ; Mon, 13 May 2019 10:39:12 -0400 (EDT) Received: by mail-it1-f198.google.com with SMTP id m20so6802820itn.3 for ; Mon, 13 May 2019 07:39:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=ivlXRYid08bCLk7EeeAFLog7R/4eiJfJ3h7LO8cUdeI=; b=fbS1/mQSbh6PMhkEnIiUhin95oaoTBBGP7DNXRQduzQKI/JkyjXHhA6IgJjmJi5tm5 cJhzNhiE0N/5K+jz6eNKHMXNl+K6FlennT6HfGQKjNbulyA8J5RZ4BoY10DcmmEvJF57 Xf8k+oyifoBVv6YSnsPT1IDXJsXMkBSazqEs1lDkbMGtyI4BT2BITmPijVKfdWuI3Gmf l+XZC5VxDoYby+gm+hS849cd2ACvZ6gb6Vc3ezIskZxgYUGA0IRWO2/E5xjOm0uE+h6f lspNrXBzeb2En46wioRtN8itsLHzGV1RmTUwR3TddXnoM9oHEMa/297zXdGJT9BC7nHk ml6A== X-Gm-Message-State: APjAAAX/AXrfxTDzUn+iG66IugVXfxXH85fzsnNoapNC0o8tFiTaOU6i 4iHfKZEnRRnWrh89MvgdLj1QiO50K98E2SLl0lIzAQLyJnGeBkzDqh4db88El+Fzori9ZE5UWl0 v7GxVQkRR5HHB4bCS0OMzL/mDG9tQivGSPtK4ZwJcROb42awfcZaxFBP+rDbG0q2BQA== X-Received: by 2002:a05:6602:21d7:: with SMTP id c23mr17786544ioc.66.1557758351971; Mon, 13 May 2019 07:39:11 -0700 (PDT) X-Google-Smtp-Source: APXvYqwdhifnTXo9MXDplt9ld2K9itr4UowiHd6RQXWJ33HTToHgvefN9jETHF7ouGLqTP9DyIFm X-Received: by 2002:a05:6602:21d7:: with SMTP id c23mr17786503ioc.66.1557758351358; Mon, 13 May 2019 07:39:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758351; cv=none; d=google.com; s=arc-20160816; b=kgOsGn8ikRf9pxLCp2S5bQZXkl6dVunizLbCGyoxlqKm3LoC5V+C1KNEl3J+D9OXfN l8kOQVmnWoJlcbv7YHr4wkcFIyJVUS59Y0f565KqtfRAyq7BBYon75mwI6ZQNzTL20Ul jOSxkR9x7YHFcc7IHFBxQsOZFUsnbQMLmKoLEOq31I8bSnl2djoJAJEjK1gNGs4f0H3T 2TrWRY+czoloSASKq6JMFNkGbKBbYs4AMLmWIwpQSXI4B8ratYyH6j4hjWRhO6KHLWHa Y5efPeYEi4XW5hdQfMIAR/tH2V6CmJOfBTejCSJANHmXJO4R6OxkE2jx3WM9W3V14aaG 9IEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ivlXRYid08bCLk7EeeAFLog7R/4eiJfJ3h7LO8cUdeI=; b=rB6syKYztLnCHs7ilES2xClj/sX5+ou1TIFh1an9Jr/tX3kVjSeYVKbQZ30ty6mVWd l9WmWccPXvl8b1zY8QzPasBop7843RwgPG4baeiVHkO/a+UqCqiEae6gUNQ63mzCR3Tn PMPQ30zAwNkGuEBUd4ay92j67ehHOg95G6x6kPf1R5yEijD1IujzajGciwShuU/Mk2GA KEgVjsyTlEJB1U4m/0xDdzcbQA46q9Tfz5qOSc6lyz0HIUQVarPrCX83cPXMkLqn8BJI 6DG/X1JtU3MaLyPn/54fN3lN/h6PLmNGZ7RGmmDW8Bguk+pqAmCwM8nmmtenaP2lhGcK ckrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="z94gwv/h"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id i141si403951iti.137.2019.05.13.07.39.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:11 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="z94gwv/h"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd3QK194955; Mon, 13 May 2019 14:39:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=ivlXRYid08bCLk7EeeAFLog7R/4eiJfJ3h7LO8cUdeI=; b=z94gwv/h6a2eKxGWAbRUGkTU6KQJtXJ5d9W6ZAsILmq9BIFlV0LuM3OrSOh/UH7Pz5NG 96qNkm9FPETMU1CK7gybTPwq+ay4+eI/5ZBk7+Q+y/kQOV/x7sjuvp8KNOiVPI3sWHJp aavXu55k8608Ik9cqQUYuPI190YDw9stqWSKV2ovjULIm12NLn2fEyPVAmeiTcuHgcon 8059HBgFrgSpkKo1gDcIsSnBF1vNWz4rpcg52v7RzoR3yGYo4g+LLYhNJsd8LIgK7av1 KVYEFQoIDdxdVwMIjw1yVhHummBBRuJpdrOk80g5n+iCI4jKYptxDL+3/5eTCqCqtgeN 4g== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7atx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:03 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQ8022780; Mon, 13 May 2019 14:38:53 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 05/27] KVM: x86: Add handler to exit kvm isolation Date: Mon, 13 May 2019 16:38:13 +0200 Message-Id: <1557758315-12667-6-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon Interrupt handlers will need this handler to switch from the KVM address space back to the kernel address space on their prelog. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/include/asm/irq.h | 1 + arch/x86/kernel/irq.c | 11 +++++++++++ arch/x86/kvm/isolation.c | 13 +++++++++++++ 3 files changed, 25 insertions(+), 0 deletions(-) diff --git a/arch/x86/include/asm/irq.h b/arch/x86/include/asm/irq.h index 8f95686..eb32abc 100644 --- a/arch/x86/include/asm/irq.h +++ b/arch/x86/include/asm/irq.h @@ -29,6 +29,7 @@ static inline int irq_canonicalize(int irq) extern __visible void smp_kvm_posted_intr_ipi(struct pt_regs *regs); extern __visible void smp_kvm_posted_intr_wakeup_ipi(struct pt_regs *regs); extern __visible void smp_kvm_posted_intr_nested_ipi(struct pt_regs *regs); +extern void kvm_set_isolation_exit_handler(void (*handler)(void)); #endif extern void (*x86_platform_ipi_callback)(void); diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index 59b5f2e..e68483b 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -295,6 +295,17 @@ void kvm_set_posted_intr_wakeup_handler(void (*handler)(void)) } EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wakeup_handler); +void (*kvm_isolation_exit_handler)(void) = dummy_handler; + +void kvm_set_isolation_exit_handler(void (*handler)(void)) +{ + if (handler) + kvm_isolation_exit_handler = handler; + else + kvm_isolation_exit_handler = dummy_handler; +} +EXPORT_SYMBOL_GPL(kvm_set_isolation_exit_handler); + /* * Handler for POSTED_INTERRUPT_VECTOR. */ diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 35aa659..22ff9c2 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -5,6 +5,7 @@ * KVM Address Space Isolation */ +#include #include #include #include @@ -95,6 +96,16 @@ static void kvm_isolation_uninit_mm(void) free_pages((unsigned long)kvm_pgd, PGD_ALLOCATION_ORDER); } +static void kvm_isolation_set_handlers(void) +{ + kvm_set_isolation_exit_handler(kvm_isolation_exit); +} + +static void kvm_isolation_clear_handlers(void) +{ + kvm_set_isolation_exit_handler(NULL); +} + int kvm_isolation_init(void) { int r; @@ -106,6 +117,7 @@ int kvm_isolation_init(void) if (r) return r; + kvm_isolation_set_handlers(); pr_info("KVM: x86: Running with isolated address space\n"); return 0; @@ -116,6 +128,7 @@ void kvm_isolation_uninit(void) if (!address_space_isolation) return; + kvm_isolation_clear_handlers(); kvm_isolation_uninit_mm(); pr_info("KVM: x86: End of isolated address space\n"); } From patchwork Mon May 13 14:38:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F8C91390 for ; Mon, 13 May 2019 14:39:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2FE2227FAE for ; Mon, 13 May 2019 14:39:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 237142817F; Mon, 13 May 2019 14:39:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D0BA27FAE for ; Mon, 13 May 2019 14:39:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D1E56B000E; Mon, 13 May 2019 10:39:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 937596B0010; Mon, 13 May 2019 10:39:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7393A6B0266; Mon, 13 May 2019 10:39:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f199.google.com (mail-it1-f199.google.com [209.85.166.199]) by kanga.kvack.org (Postfix) with ESMTP id 50DB96B000E for ; Mon, 13 May 2019 10:39:13 -0400 (EDT) Received: by mail-it1-f199.google.com with SMTP id l193so2754622ita.8 for ; Mon, 13 May 2019 07:39:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=k+F2ikPI91eVYLszQvHNmFnwWVR/o1Jhouj4HTjs9uc=; b=b6infYx7IQ1ziFj9MMmNigNJphzbSFVgfXvt35l7gYUzGgWDYSH6RroqtiqbUgy4C7 m93sB4F9bJ0D0r2R++ADmgjLoBNNVLUdrAshnxqmCOs3erjwmnsWtbpxJgZj5R1s/QuF 933qyjk9dzh/e9Y6p2J2SZt5kT2PDlj5CwV1JtNWQMAMQ0NT0UIXBsy5dTuoMcBfc1gQ Ta0XQ4/Moe2tC0d+20xf5IRjdDu8oJXG45Ncc8JZyq/ASmQEBnIWdVm1f0V4y/HPucEM 75lq+UwDzqghKj5n/BpYr2ZQPiRFpe/bAovno0HvukdmyxwTGri1biB4UZLapLZ1HcQE muHg== X-Gm-Message-State: APjAAAUrT7PsWohhPKqUi/Sut6a1gC2Bc6iIr8caThDRmLvO6nCvkpZr a92xeIVXC7LQHfC/rXrVuDhvr8ymPXQYtuLFQ7TKb4hVZ4SQWistsB2j2Ihr309j16lGAkIkb8G 6Mxe6vyDwMRU3LRsyMR1zr6IULKY62cPQS2ydovKLSL4aiyY6HFgR7E0seKf687OjZA== X-Received: by 2002:a02:b895:: with SMTP id p21mr20003737jam.80.1557758353043; Mon, 13 May 2019 07:39:13 -0700 (PDT) X-Google-Smtp-Source: APXvYqyNasLAbNJSVN/xkQ3g1I3hYCD0qA/QK1ArVNurZOHqESslaPxyWzRzBx9T+q3AkoP2ZDac X-Received: by 2002:a02:b895:: with SMTP id p21mr20003700jam.80.1557758352431; Mon, 13 May 2019 07:39:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758352; cv=none; d=google.com; s=arc-20160816; b=K7MRQwqyr7+fSo45+0iy2kc61KcYdwV7HhNakTrd31507Q7BePUDsuUWtFlHYXD/dd a9QoM2PW5jIQe2e29I5FwPqq+pdK0a2F7DRyCkkZ1qlpkF8hzpW/qufRz7quKHvTMJ0j Z42WkhkwVXSzAsrnd9DicBrc3VKtV6ABrh05AxQ3G/S1Pgir2BPlr/edgRyoLxp0NzGw ZBl2o4mRH9dQcOcoRnD5UB6dKFyihRJfioX7Zg9B6E87yyV16xFDtnSle3zL73VLg8QH U2qv0gtdg5w0ExKNIe7D/kWapONEUSecGDI24nmVFySu5bU2IgCA33xXAvi/8zJ2++0J Y+6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=k+F2ikPI91eVYLszQvHNmFnwWVR/o1Jhouj4HTjs9uc=; b=uAw46+HRV8IisIiUtfCBgayuDYvCH+M++C4OgyjV0T400U/JB44pvq0VOYIn2+z2NS 76yJQXosjWslnXqmbtZbo8LVSFoghdg702Q23Uj0+0SnO7wLmJBGhwCNMusThbTHcp6U ne6OhpUocFEAtcwnXO0t2a0Td6Ki6C76KMkObWgIUFK+ucfwVZjfCQdTdaGDmOyXARlN 98yRhLzF2bvu44uOO5UugwenHRYx0ymybDAu/fAMgumNDhq0YShZ/5zog4HmeSNzhtmx +77gDICZDcndsI0t3I8FtzYpxNFZPCfE+tqKzJIT/tGwGE5ATUVk2qaspuq3QyrFoTvK qT1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=sPuRCB7A; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id b8si8497089itl.66.2019.05.13.07.39.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:12 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=sPuRCB7A; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd4le195008; Mon, 13 May 2019 14:39:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=k+F2ikPI91eVYLszQvHNmFnwWVR/o1Jhouj4HTjs9uc=; b=sPuRCB7AsbZZWT9m3DPio3N73cCl1qZSGvYcOtO5wcwsHy2zHWKroJG67uw/Upj6MB8h as5U+fXDw92G4CC+u+VRhy4iiGPx4okU0TKnt3mZX/RUGaqFhGWJu4ijIoE5Zs9d9SNL u//amNhRfak4bvPH2HOHsboCM+M+dxOBIVZGLn3TN6a2pmSvuViBD6jXP4r86MOGViz9 jN0fBYhsPujDrb6O6Mzol1ztWZq7wn6RT/LFtqlQE5JCw2X/1/QkZ/0lzXFxFT78JDD8 PNKqvAwiyWmQ5SmRKDUE6R4j7mSNKvwv3s4EEuyCXYkMX1kBdTCqgLL8P3W8I/5vm5L5 rw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7ata-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:04 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQ9022780; Mon, 13 May 2019 14:38:56 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 06/27] KVM: x86: Exit KVM isolation on IRQ entry Date: Mon, 13 May 2019 16:38:14 +0200 Message-Id: <1557758315-12667-7-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon Next commits will change most of KVM #VMExit handlers to run in KVM isolated address space. Any interrupt handler raised during execution in KVM address space needs to switch back to host address space. This patch makes sure that IRQ handlers will run in full host address space instead of KVM isolated address space. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/include/asm/apic.h | 4 ++-- arch/x86/include/asm/hardirq.h | 10 ++++++++++ arch/x86/kernel/smp.c | 2 +- arch/x86/platform/uv/tlb_uv.c | 2 +- 4 files changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 130e81e..606da8f 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -515,7 +515,7 @@ static inline unsigned int read_apic_id(void) static inline void entering_irq(void) { irq_enter(); - kvm_set_cpu_l1tf_flush_l1d(); + kvm_cpu_may_access_sensitive_data(); } static inline void entering_ack_irq(void) @@ -528,7 +528,7 @@ static inline void ipi_entering_ack_irq(void) { irq_enter(); ack_APIC_irq(); - kvm_set_cpu_l1tf_flush_l1d(); + kvm_cpu_may_access_sensitive_data(); } static inline void exiting_irq(void) diff --git a/arch/x86/include/asm/hardirq.h b/arch/x86/include/asm/hardirq.h index d9069bb..e082ecb 100644 --- a/arch/x86/include/asm/hardirq.h +++ b/arch/x86/include/asm/hardirq.h @@ -80,4 +80,14 @@ static inline bool kvm_get_cpu_l1tf_flush_l1d(void) static inline void kvm_set_cpu_l1tf_flush_l1d(void) { } #endif /* IS_ENABLED(CONFIG_KVM_INTEL) */ +#ifdef CONFIG_HAVE_KVM +extern void (*kvm_isolation_exit_handler)(void); + +static inline void kvm_cpu_may_access_sensitive_data(void) +{ + kvm_set_cpu_l1tf_flush_l1d(); + kvm_isolation_exit_handler(); +} +#endif + #endif /* _ASM_X86_HARDIRQ_H */ diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 04adc8d..b99fda0 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -261,7 +261,7 @@ __visible void __irq_entry smp_reschedule_interrupt(struct pt_regs *regs) { ack_APIC_irq(); inc_irq_stat(irq_resched_count); - kvm_set_cpu_l1tf_flush_l1d(); + kvm_cpu_may_access_sensitive_data(); if (trace_resched_ipi_enabled()) { /* diff --git a/arch/x86/platform/uv/tlb_uv.c b/arch/x86/platform/uv/tlb_uv.c index 1297e18..83a17ca 100644 --- a/arch/x86/platform/uv/tlb_uv.c +++ b/arch/x86/platform/uv/tlb_uv.c @@ -1285,7 +1285,7 @@ void uv_bau_message_interrupt(struct pt_regs *regs) struct msg_desc msgdesc; ack_APIC_irq(); - kvm_set_cpu_l1tf_flush_l1d(); + kvm_cpu_may_access_sensitive_data(); time_start = get_cycles(); bcp = &per_cpu(bau_control, smp_processor_id()); From patchwork Mon May 13 14:38:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941049 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A42566C5 for ; Mon, 13 May 2019 14:39:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9180627FAE for ; Mon, 13 May 2019 14:39:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8549528305; Mon, 13 May 2019 14:39:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE43D27FAE for ; Mon, 13 May 2019 14:39:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E18B86B0266; Mon, 13 May 2019 10:39:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D9F306B0269; Mon, 13 May 2019 10:39:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C24206B026A; Mon, 13 May 2019 10:39:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by kanga.kvack.org (Postfix) with ESMTP id 9D2016B0266 for ; Mon, 13 May 2019 10:39:20 -0400 (EDT) Received: by mail-io1-f72.google.com with SMTP id i16so2516877ioj.4 for ; Mon, 13 May 2019 07:39:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=wqdeci0HyBGB7YKvZMkvhcTkiB8MUENeuIRahUviMQQ=; b=p/glfFEFzh6IVRB9pnsmcdQZl0HnbkeKHZ2D1M5/q+JO6Jd8lPK/mCMV12rgo8gDQC +Ry1TSFDRmvqHMHqBLIBrtHWHvJo3NOhqB+MPsDpE+o/IFkR2UEwJGzwGyS2nikO+LTi hFX6dSJ/pnSG5bdyUBi1aG9vCD0ayxXPKJvO1rbsTLkY2b0c108J9TDJOLKhzwm5Aq0A Si4AUNQu6iKwQOg2ukcpd89dFf0VKrKL47BaxdRCPYSr+8L8fjpx3l11y9Gqf50tRKTX 9AWmeIUqS9vlGdmm0KefMRsNXmufS8p0ZdNsvpkzxMadE++yMQeaRHpH6xl/JwhSy0BG ByCw== X-Gm-Message-State: APjAAAVUNQRV6K9aC8PzsRl7hX5p9mhPkTDCYDBasWzSaBDZ043PY9wM /6EpG/utl8zAHvc8ktTQ0EITsGsibabLei5eb3mPwDRFxnz+edV9yO3LhvBcFR2fhMwHYZHD/ho NXdvqFOmnW+GkO1StjhOHzjwj7Fqr5FmJKv9wQt9OTDr7CmrIMrV0temQsTU6qHtoDw== X-Received: by 2002:a5d:9dd2:: with SMTP id 18mr16342290ioo.7.1557758360369; Mon, 13 May 2019 07:39:20 -0700 (PDT) X-Google-Smtp-Source: APXvYqwbHyYgAUX4Aq3vb3o3mQEzZ8auOj/lQzJqO3TmDMqsVX5kZMF+1FmSAoWTemeOXBOfw+BV X-Received: by 2002:a5d:9dd2:: with SMTP id 18mr16342255ioo.7.1557758359629; Mon, 13 May 2019 07:39:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758359; cv=none; d=google.com; s=arc-20160816; b=dpYEev6WzlRVo7uK7UAiQzEILQLqI5l8Qb2dBD8xthE8s5VX/RCRxEPrhi8pRWRA8S u19Ry6t9wzwxrhY0ShB3RUqoS+whO+eEo8fKnkU2D+gQSqo3tHKGpqslmOsjJ3KsD8uE e6xV6J6DxSIMJQSJpV9WGEPuSasekTruol3014cxeW64J/K/LJm7HC9+atQFlmhS+s5n yeq0oJyHWAx1CT7K6PxZTGA6CFJPLi8YHNxyAdKU+FQU976tTtvqNOBVLFdK806HkVdh kproeiOOFJusS9gNfH8dXZqocTtI58EQKpDkbu3zhOk/rVK43KVjlXZKu7AaZnAK0TPO KTxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=wqdeci0HyBGB7YKvZMkvhcTkiB8MUENeuIRahUviMQQ=; b=gGU6yI2JgyerelolOLgg1YAEEWuGBdhlPKvLAsnJJLORkz6zmeDGUHyVnkp3TCFAFG hdZjbyB6XUNLOaYDSuuuwGfN+s+fbG+rUe9B2dVdwdSxSzddHMWPiy5UKZkRBuWnud+x akDxNQoWDQaw8z683RRdNMmsNhdbQFofTPCh2QYH0ImI8da16T7FRboJBMDQabH28SeR 5SoCuGK52VI4R9VsuVJKF4nWimnI/0wN2oQRD+qmL8sSin8/zPu6diLqw0wh0MO3dCff 0FEq/ckBBn3EJoM/2mVQZ08EOmUduy71jNmq8EJyL6hY6o/Xc0e7bOxIQfI9Cht2NOGa G1Tw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=qAq8bRmq; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2130.oracle.com (userp2130.oracle.com. [156.151.31.86]) by mx.google.com with ESMTPS id k195si8417716itb.11.2019.05.13.07.39.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:19 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) client-ip=156.151.31.86; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=qAq8bRmq; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd6qH181584; Mon, 13 May 2019 14:39:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=wqdeci0HyBGB7YKvZMkvhcTkiB8MUENeuIRahUviMQQ=; b=qAq8bRmqjLI4ZmGhxqxLQzFJ+Y7ZGe9B4tzqFa4rSPttIDYPwO8CyJSVFyohA4WGqJmF PiEooyxx92pLb5i0rofSV4dZw0lAf84hAXIH/yxG0GncuXG6InmO4dJUyzjA+ryXZa3+ kpAk+nls/3Op+xDAehiDEtWmVkdLl1RWyWJ2r/dqXCkn+On4KPyzKvKpd3t0q+Al/+8t OnREbE+Cqw+DpqL4613DUOyPi8UvLz/4anaHm7nytPek57wpfSd6QSOe3Wz14bi6e5IO vMHSdlEGcaFkmj7ADxMP0X19oYbCmNWG6TEextjbaF9y0FnEIo95HqGb3w9D+rgQ3QCW PQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2sdnttfeff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:07 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQA022780; Mon, 13 May 2019 14:38:59 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 07/27] KVM: x86: Switch to host address space when may access sensitive data Date: Mon, 13 May 2019 16:38:15 +0200 Message-Id: <1557758315-12667-8-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=851 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon Before this patch, we exited from KVM isolated address space to host address space as soon as we exit guest. Change code such that most of KVM #VMExit handlers will run in KVM isolated address space and switch back to host address space only before accessing sensitive data. Sensitive data is defined as either host data or other VM data. Currently, we switch from kvm_mm to host_mm on the following scenarios: 1) When handling guest page-faults: As this will access SPTs which contains host PFNs. 2) On schedule-out of vCPU thread 3) On write to guest virtual memory (kvm_write_guest_virt_system() can pull in tons of pages) 4) On return to userspace (e.g. QEMU) 5) On prelog of IRQ handlers Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 7 ++++++- arch/x86/kvm/isolation.h | 3 +++ arch/x86/kvm/mmu.c | 3 ++- arch/x86/kvm/x86.c | 12 +++++------- 4 files changed, 16 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 22ff9c2..eeb60c4 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -5,7 +5,6 @@ * KVM Address Space Isolation */ -#include #include #include #include @@ -133,6 +132,12 @@ void kvm_isolation_uninit(void) pr_info("KVM: x86: End of isolated address space\n"); } +void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu) +{ + vcpu->arch.l1tf_flush_l1d = true; + kvm_isolation_exit(); +} + void kvm_isolation_enter(void) { if (address_space_isolation) { diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 595f62c..1290d32 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -2,9 +2,12 @@ #ifndef ARCH_X86_KVM_ISOLATION_H #define ARCH_X86_KVM_ISOLATION_H +#include + extern int kvm_isolation_init(void); extern void kvm_isolation_uninit(void); extern void kvm_isolation_enter(void); extern void kvm_isolation_exit(void); +extern void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu); #endif diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index d9c7b45..a2b38de 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -23,6 +23,7 @@ #include "x86.h" #include "kvm_cache_regs.h" #include "cpuid.h" +#include "isolation.h" #include #include @@ -4059,7 +4060,7 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code, { int r = 1; - vcpu->arch.l1tf_flush_l1d = true; + kvm_may_access_sensitive_data(vcpu); switch (vcpu->arch.apf.host_apf_reason) { default: trace_kvm_page_fault(fault_address, error_code); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 85700e0..1db72c3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3307,6 +3307,8 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) * guest. do_debug expects dr6 to be cleared after it runs, do the same. */ set_debugreg(0, 6); + + kvm_may_access_sensitive_data(vcpu); } static int kvm_vcpu_ioctl_get_lapic(struct kvm_vcpu *vcpu, @@ -5220,7 +5222,7 @@ int kvm_write_guest_virt_system(struct kvm_vcpu *vcpu, gva_t addr, void *val, unsigned int bytes, struct x86_exception *exception) { /* kvm_write_guest_virt_system can pull in tons of pages. */ - vcpu->arch.l1tf_flush_l1d = true; + kvm_may_access_sensitive_data(vcpu); return kvm_write_guest_virt_helper(addr, val, bytes, vcpu, PFERR_WRITE_MASK, exception); @@ -7948,12 +7950,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); - /* - * TODO: Move this to where we architectually need to access - * host (or other VM) sensitive data - */ - kvm_isolation_exit(); - vcpu->mode = OUTSIDE_GUEST_MODE; smp_wmb(); @@ -8086,6 +8082,8 @@ static int vcpu_run(struct kvm_vcpu *vcpu) srcu_read_unlock(&kvm->srcu, vcpu->srcu_idx); + kvm_may_access_sensitive_data(vcpu); + return r; } From patchwork Mon May 13 14:38:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941047 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 050EF6C5 for ; Mon, 13 May 2019 14:39:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E8EDC27FAE for ; Mon, 13 May 2019 14:39:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DCBE92817F; Mon, 13 May 2019 14:39:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A9AE27FAE for ; Mon, 13 May 2019 14:39:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D7CEF6B0010; Mon, 13 May 2019 10:39:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D06A86B0266; Mon, 13 May 2019 10:39:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B81FF6B0269; Mon, 13 May 2019 10:39:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f198.google.com (mail-it1-f198.google.com [209.85.166.198]) by kanga.kvack.org (Postfix) with ESMTP id 7BEC16B0010 for ; Mon, 13 May 2019 10:39:16 -0400 (EDT) Received: by mail-it1-f198.google.com with SMTP id g63so12279408ita.6 for ; Mon, 13 May 2019 07:39:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=BckiUE2qwd6ce7s4e94ZCEH08ViWVb2Uhow4MmaOmmg=; b=cCW7EJyyyiEMkzyWSb5Y9qMt4w+4+WGLbAoqk2HAf5Ny5u3KR3sLUEy+hm2G65mprQ 2aKBozA85jNAcexthHymh7Sm4jsm6YcDh2lZiyBtsMvkT1maGmZ/svtepFt5DsRh0qBA Un5l2jME6f/4a1pNHZMH43nL20ZO4zn/mabBKOzTgT5VTbj18GBoqzHLQkXItsLYag70 ON0DSpC+lfUy4LMrr99IlRQSAWzjZNC8alVcJmzSi7K+qSAD/YFxdBzLObJPiaR340ad PniEpPAPVM1O9dlwr1PrN9/HR39Z1X7KahyH0es7mmnGN4+Gro9k8hBc1zGDsDUMprG0 zgMQ== X-Gm-Message-State: APjAAAW10c8o4bYlmpSFngQQTTkri50/teT1UXZYn2vfrZxQQqEiqiKe JW9SoyFSsREaiWQyxoDCfLeJO0wmrYBGGX2Ih52n4XBudUn0h2A7yY76eSBjeLVwhwyaMsIJhME owYBIeaEOidpXUfPRQiNfgt/kwO6zqz19UTtCYh50JnEecproC/uDQNQGlyNPaHO8AQ== X-Received: by 2002:a24:1f50:: with SMTP id d77mr17645435itd.25.1557758356238; Mon, 13 May 2019 07:39:16 -0700 (PDT) X-Google-Smtp-Source: APXvYqxqfgMm/81O7+/n8LVlwszWYEPY/hPItHtexhaRLbnP4Da9OGzrHkgkp2B2zoqudlVVXDVf X-Received: by 2002:a24:1f50:: with SMTP id d77mr17645396itd.25.1557758355606; Mon, 13 May 2019 07:39:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758355; cv=none; d=google.com; s=arc-20160816; b=NRrpcRnuidmR/nzXWJ0ow3NhscjF84FrGBa58Pri/POz0cSnEkV34dSfKBFDyGHJ6w fd6g9cMSIKPkDkbeiNvFv65vskcZcurfp2DL9SwEAp2/ZF+eA7HvrwhYcrXvuseEnY4S MXaLiWL7G3hdXAgXj/32Mxyx3R0/D/vOMjOoECPguOgP1Jus3iaal6H9Jcbu0vKm4wAD JPjyFtj2+h7yW669S2PMcY68BrarvcOh0TbOq9n3S5KxW1onLfCYYBnKAgv8j20wzD6F uNgKEZV3L5u9hYQOyJLaxcNpl4WxI70ip1KNtuIrWqdGr61ey2SD2UMNHjhFxBQ72bBF OjEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BckiUE2qwd6ce7s4e94ZCEH08ViWVb2Uhow4MmaOmmg=; b=y8OGPgDjh+kmlJ7bTzQGxxO7ygD+R/SvEtUATuB1Jo+5qedHUhQeY7pxIFu0mshAeN d9C5BZPxzFGenkRTwPKirEGlxHwnRj2q39w+MMk7+O9TfBCfknNm/pWVND1/6FhkTssG hf4PbeUwr11aEdr1nILxNvDjQ5/VNoikxFLkjTnBg+Ss8Ckog7iyNTuX6evafOxLNP78 2TYDSBx6zoEYFh18s4xvcAO5rjrvMhjuaraNlUdtcteqFZI7rqgvGUuHxgdcBqCmOlvK FQcT/K4jRTjIuT0mhCTUEqjAhu7RlRBnUj1SAjoFeSyEGKryNqKsi+q32dPWcogc9V2j e2Yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=MqxRlocD; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id a7si8602214jap.6.2019.05.13.07.39.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:15 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=MqxRlocD; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2YY194903; Mon, 13 May 2019 14:39:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=BckiUE2qwd6ce7s4e94ZCEH08ViWVb2Uhow4MmaOmmg=; b=MqxRlocDO4PBVSKI4njGqiBJNQ3Dwy7bXoxOg336yVqnxDPoOo4C8bDcoA+hRinUelY/ IfBGNpsi5hC0VByASVpiWMEgObQsMVJz2ebCqcSbAU7+bP+dXw4f4OP72FR/CyrWZMzm C5l9JlMKOqhD3L7yBz6qxyBISnDBFSOI418qbfcikgzUXjTPisA4nha1WqGfH00pB+fR MAPL3suB/BEgJ7y+NOdtYN4R08kIlmFj7UCcos/Hh23ufmYdF9aqYoILoddmclvElloo MtVyLdQhfuOp3oe6RJY+QcC2ZMynhm3bA4JOkdVlLXIFA7MOvRKhnhe+hFi0uBip4m9o lw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7aum-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:05 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQB022780; Mon, 13 May 2019 14:39:02 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 08/27] KVM: x86: Optimize branches which checks if address space isolation enabled Date: Mon, 13 May 2019 16:38:16 +0200 Message-Id: <1557758315-12667-9-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon As every entry to guest checks if should switch from host_mm to kvm_mm, these branches is at very hot path. Optimize them by using static_branch. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 11 ++++++++--- arch/x86/kvm/isolation.h | 7 +++++++ 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index eeb60c4..43fd924 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -23,6 +23,9 @@ struct mm_struct kvm_mm = { .mmlist = LIST_HEAD_INIT(kvm_mm.mmlist), }; +DEFINE_STATIC_KEY_FALSE(kvm_isolation_enabled); +EXPORT_SYMBOL(kvm_isolation_enabled); + /* * When set to true, KVM #VMExit handlers run in isolated address space * which maps only KVM required code and per-VM information instead of @@ -118,15 +121,17 @@ int kvm_isolation_init(void) kvm_isolation_set_handlers(); pr_info("KVM: x86: Running with isolated address space\n"); + static_branch_enable(&kvm_isolation_enabled); return 0; } void kvm_isolation_uninit(void) { - if (!address_space_isolation) + if (!kvm_isolation()) return; + static_branch_disable(&kvm_isolation_enabled); kvm_isolation_clear_handlers(); kvm_isolation_uninit_mm(); pr_info("KVM: x86: End of isolated address space\n"); @@ -140,7 +145,7 @@ void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu) void kvm_isolation_enter(void) { - if (address_space_isolation) { + if (kvm_isolation()) { /* * Switches to kvm_mm should happen from vCPU thread, * which should not be a kernel thread with no mm @@ -152,7 +157,7 @@ void kvm_isolation_enter(void) void kvm_isolation_exit(void) { - if (address_space_isolation) { + if (kvm_isolation()) { /* TODO: Kick sibling hyperthread before switch to host mm */ /* TODO: switch back to original mm */ } diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 1290d32..aa5e979 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -4,6 +4,13 @@ #include +DECLARE_STATIC_KEY_FALSE(kvm_isolation_enabled); + +static inline bool kvm_isolation(void) +{ + return static_branch_likely(&kvm_isolation_enabled); +} + extern int kvm_isolation_init(void); extern void kvm_isolation_uninit(void); extern void kvm_isolation_enter(void); From patchwork Mon May 13 14:38:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941051 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF9C51390 for ; Mon, 13 May 2019 14:39:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BC80227FAE for ; Mon, 13 May 2019 14:39:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AA3A72817F; Mon, 13 May 2019 14:39:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E148028173 for ; Mon, 13 May 2019 14:39:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 47BB56B0269; Mon, 13 May 2019 10:39:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3DC2D6B026A; Mon, 13 May 2019 10:39:21 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20A1D6B026B; Mon, 13 May 2019 10:39:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f199.google.com (mail-it1-f199.google.com [209.85.166.199]) by kanga.kvack.org (Postfix) with ESMTP id E8BFB6B026A for ; Mon, 13 May 2019 10:39:20 -0400 (EDT) Received: by mail-it1-f199.google.com with SMTP id m20so6803168itn.3 for ; Mon, 13 May 2019 07:39:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=lqyn+UMxH2H0mFstbeYlyK/O28QxJu29o92QL7NqRo0=; b=oMBvnk3WW+IFxibxIHG2zxvtzWyK1FeS8ZELKRWLpSUpywjtDY9w5tWhS12KxIJLZC MgSapfhrqjJ71t3X3W7+NYj+i18JGGG/fabtXkcGJUFE590EnaLPQXg4KF1RHtRRjYJ/ AIVaWsqkEm/Qy/hmEVPNRtLc3cvWOYxsgWtESc28g26Tz1e0ZOtrDNQy+jVqqUfKfSHc 0XvYChYXY4xHYyCEsnLns7bMFpz4i63xmB8nGojiv9Tm1B9TMBSg2QPbsvb4pjUs7so8 39eeU2fvkNYFHn4RDXGg+b4IXgBhjJoVL4cQGs/qrpibSLtwd5ajFO0pIhjSeUi9goIN Gvig== X-Gm-Message-State: APjAAAVjQpbMYWyD6RHyg2RewLIy1SF1z04pr1wrJqP6NrmHbNjV+5IW 5vO1IryuhOHhYDbbLpatxYRf1iwsJFjaOuXEj204FMEle/sIjXtOE/P/bL3t5tgCi5Pc1BVPp0M Tw1ZGfcgVrbiKjZcoihrKAuhd7x0jO53eQtXg97bUc9BQpk95jMd1KtGZGC9OTz2gvA== X-Received: by 2002:a24:56c7:: with SMTP id o190mr18572988itb.120.1557758360664; Mon, 13 May 2019 07:39:20 -0700 (PDT) X-Google-Smtp-Source: APXvYqwCyBpbZ+ZUQa8HX1Woieu8tdY2X5ppMZzvd3aAtzO2LdJtrYB+9iWF3+6w/qTfEvq6XMyX X-Received: by 2002:a24:56c7:: with SMTP id o190mr18572906itb.120.1557758359640; Mon, 13 May 2019 07:39:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758359; cv=none; d=google.com; s=arc-20160816; b=BxxeQma1NwlN36teOC8SFulbshrmo/3MpLWtP7zA1PDRW5hEwkCjnBzYZL1UmXErAx CY2dxHSoZrJjyvQID0DRTiVtP6pQWkqBguIHRpVyJq+YxrdceG6sNIvZZdwm+NVyKd03 Vj2OBxyz9QwwsiH3VulfcA2z+MD3yIqhjVaKqXihwbfsO4HGs9Aa0Qtmtm8O7dlh6T1S fPVy09DXoyf2bhVBjcIyFo8Fi4v3vxuS5zHlabpQ2JfnoqipB3KGyu8OqlsXVXtWSBE2 Jxd17+c9zweB6GodnnLru694jeP/Kx2nT5/xd1OYJIvwhzObF2Vrz9EW2Z4mO9HD3f3N f82w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=lqyn+UMxH2H0mFstbeYlyK/O28QxJu29o92QL7NqRo0=; b=QwNdhxvUwMENZT/jP3G7nfnJh/ror0dQJ5/JGiMedFR65AR8qsKbIUkaSG6/pjjQ5d zZ+ZB5rmsXiTDfHUqjQNxHhyaVaYOURlULwsoRtopgNB30B0g2lVNKDaOAW29qROsfPJ CVA6iYVqu7exBMwbao1W0RfJNxnQQqX1l0Ekp3xZTa38dvS7zZnduVaTbOxUJGZPQByx Y2rTbKr1GN+DdDGCb856zv5x8xr6A2ve4xW3Qy9MUHfiNwlvd6TkKcuIBGTtOY2Iy6rp wLHEu0CRuBBxuApp00ed0FbnMBnmajC3G5c63NEX+t763MSltU3Io/o/NXJh468NucXd CKhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="Jr5TCZD/"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id m69si8151116itb.96.2019.05.13.07.39.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:19 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="Jr5TCZD/"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2hT193056; Mon, 13 May 2019 14:39:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=lqyn+UMxH2H0mFstbeYlyK/O28QxJu29o92QL7NqRo0=; b=Jr5TCZD/u/xNJPZvrSFuZJzX/+fkjXUeKcGKj+LAFsPLKhL5qgqU35GRnUUc8xQu8pLv kJ2cgm2Uaj8fWZEV+R26AC0h34/IHr0xch9O9VC8DBfJZs45MDHArPHOrKH2BZgkZn0g Vo1Qsw+WtWbru1nSWeUWRDzk0S8BMwbmJOSxY9hDJpCKuZ4Rjcl9KR+8ZAUPwwUnSucl F6NkZr7jZnrzkGGfd72bek1kv1u5Rb/uWnD+/v6jia+yUhHtbKzvABFRbX3KACRbuEhG GYu3aCArrPJv0oQqS9LptBfUuYpEaXqIKNe+OKRLzFVYsoDNcshbQCQH2UMO7meQY53G qQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfkvq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:08 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQC022780; Mon, 13 May 2019 14:39:05 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 09/27] kvm/isolation: function to track buffers allocated for the KVM page table Date: Mon, 13 May 2019 16:38:17 +0200 Message-Id: <1557758315-12667-10-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The KVM page table will have direct references to the kernel page table, at different levels (PGD, P4D, PUD, PMD). When freeing the KVM page table, we should make sure that we free parts actually allocated for the KVM page table, and not parts of the kernel page table referenced from the KVM page table. To do so, we will keep track of buffers when building the KVM page table. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 119 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 119 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 43fd924..1efdab1 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -8,12 +8,60 @@ #include #include #include +#include #include #include #include "isolation.h" + +enum page_table_level { + PGT_LEVEL_PTE, + PGT_LEVEL_PMD, + PGT_LEVEL_PUD, + PGT_LEVEL_P4D, + PGT_LEVEL_PGD +}; + +/* + * The KVM page table can have direct references to the kernel page table, + * at different levels (PGD, P4D, PUD, PMD). When freeing the KVM page + * table, we should make sure that we free parts actually allocated for + * the KVM page table, and not parts of the kernel page table referenced + * from the KVM page table. + * + * To do so, page table directories (struct pgt_directory) are used to keep + * track of buffers allocated when building the KVM page table. Also, as + * a page table can have many buffers, page table directory groups (struct + * (pgt_directory_group) are used to group page table directories and save + * some space (instead of allocating each directory individually). + */ + +#define PGT_DIRECTORY_GROUP_SIZE 64 + +struct pgt_directory { + enum page_table_level level; + void *ptr; +}; + +struct pgt_directory_group { + struct list_head list; + int count; + struct pgt_directory directory[PGT_DIRECTORY_GROUP_SIZE]; +}; + +static LIST_HEAD(kvm_pgt_dgroup_list); +static DEFINE_MUTEX(kvm_pgt_dgroup_lock); + +/* + * Get the pointer to the beginning of a page table directory from a page + * table directory entry. + */ +#define PGTD_ALIGN(entry) \ + ((typeof(entry))(((unsigned long)(entry)) & PAGE_MASK)) + + struct mm_struct kvm_mm = { .mm_rb = RB_ROOT, .mm_users = ATOMIC_INIT(2), @@ -43,6 +91,77 @@ struct mm_struct kvm_mm = { static bool __read_mostly address_space_isolation; module_param(address_space_isolation, bool, 0444); + +static struct pgt_directory_group *pgt_directory_group_create(void) +{ + struct pgt_directory_group *dgroup; + + dgroup = kzalloc(sizeof(struct pgt_directory_group), GFP_KERNEL); + if (!dgroup) + return NULL; + + INIT_LIST_HEAD(&dgroup->list); + dgroup->count = 0; + + return dgroup; +} + +static bool kvm_add_pgt_directory(void *ptr, enum page_table_level level) +{ + struct pgt_directory_group *dgroup; + int index; + + mutex_lock(&kvm_pgt_dgroup_lock); + + if (list_empty(&kvm_pgt_dgroup_list)) + dgroup = NULL; + else + dgroup = list_entry(kvm_pgt_dgroup_list.next, + struct pgt_directory_group, list); + + if (!dgroup || dgroup->count >= PGT_DIRECTORY_GROUP_SIZE) { + dgroup = pgt_directory_group_create(); + if (!dgroup) { + mutex_unlock(&kvm_pgt_dgroup_lock); + return false; + } + list_add_tail(&dgroup->list, &kvm_pgt_dgroup_list); + } + + index = dgroup->count; + dgroup->directory[index].level = level; + dgroup->directory[index].ptr = PGTD_ALIGN(ptr); + dgroup->count = index + 1; + + mutex_unlock(&kvm_pgt_dgroup_lock); + + return true; +} + +static bool kvm_valid_pgt_entry(void *ptr) +{ + struct pgt_directory_group *dgroup; + int i; + + mutex_lock(&kvm_pgt_dgroup_lock); + + ptr = PGTD_ALIGN(ptr); + list_for_each_entry(dgroup, &kvm_pgt_dgroup_list, list) { + for (i = 0; i < dgroup->count; i++) { + if (dgroup->directory[i].ptr == ptr) { + mutex_unlock(&kvm_pgt_dgroup_lock); + return true; + } + } + } + + mutex_unlock(&kvm_pgt_dgroup_lock); + + return false; + +} + + static int kvm_isolation_init_mm(void) { pgd_t *kvm_pgd; From patchwork Mon May 13 14:38:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941055 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6855F1390 for ; Mon, 13 May 2019 14:39:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 591EB27FAE for ; Mon, 13 May 2019 14:39:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4D06128305; Mon, 13 May 2019 14:39:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D3E5328173 for ; Mon, 13 May 2019 14:39:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A7366B026A; Mon, 13 May 2019 10:39:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 20B8B6B026B; Mon, 13 May 2019 10:39:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05B5E6B026C; Mon, 13 May 2019 10:39:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by kanga.kvack.org (Postfix) with ESMTP id D75186B026A for ; Mon, 13 May 2019 10:39:21 -0400 (EDT) Received: by mail-io1-f71.google.com with SMTP id l6so10001974ioc.15 for ; Mon, 13 May 2019 07:39:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=4cIDR6PZl2rutLAEYSA22jeItW1PJPdhgsRn6XGFWEA=; b=am1x7bJ5EthjY4914FelVsh5/dAlYPP3xsDd6R07wANsy+vgqwUe6+pXsmwTGRxZPU aD+hHP4vvsKaWHIpO9xCqyj++hidJNheZIuyzGagRbIIR3CRY9sPMlMJLucFTerjyB8s cVyiH5AXU9Hv/df12q10V1dia/3o/P3EA5m9LJ7ocPEME4YUwac2h4pOLY5qw/w4PXuB 4fQeH5mdOCfsW6fLckigbUweGplEhEqRwY3nT53xJusqDwpUDUVx8ywATS7ajq2r+Bgm nc7XHj6TWPY0LnGMT3EeB9cdsrFGN956RREjgmedoREzQI3jnniYDYSOgpK7MzLbJPSF HGaw== X-Gm-Message-State: APjAAAXALsRXv2OHiSS7HjDk8LCQG5GZp9+0YkyIjBRPIFox15U+CjCf v0OrkrvgSRdQ89aeNU9BmfH4gtgwAmjcgicEw+MMKDVRZJ2Ltch3rO4Ffdu/BHS+areCEtjbLGA 6miNw3ZJIeb/eYjZWJBsOCVGnrZNkvu2UDWO/QT6Xqv30CnkoydpGcdB3XFBfMUHJYg== X-Received: by 2002:a24:8207:: with SMTP id t7mr20260483itd.78.1557758361612; Mon, 13 May 2019 07:39:21 -0700 (PDT) X-Google-Smtp-Source: APXvYqzch60aIEizryxqhQSyLd0TrJaC0tQIf7wapwcNBvAEkrflEJQ6IcYVky+TUXzjiG431YwY X-Received: by 2002:a24:8207:: with SMTP id t7mr20260434itd.78.1557758360947; Mon, 13 May 2019 07:39:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758360; cv=none; d=google.com; s=arc-20160816; b=fZ1gjdrClA80H4OJjzTqm6HNoSYkHdTbuRgxLEtLhOrfw8lz8AgQNN2ZHax6xbGYBo gx3J+rsRLOtW9uYpP6F+Cgfc6uwHv12Gvr/HMkIsMB3Ft2ZJYCbOaNWreLoZGRrbs1TS xUKuZ8iKjSa4h/uk6Zucrla8QV8d9gHGPsUXmQlbCmKYRMZb/cQY5gtYseMPl2iLIHcf 8Hiy7F2XCtA0cNwkM6Y7UoB+n/4a0INa67BffH/215Cruuxn+u33MmkmNUV7xIdvFtu6 xe0/WrxIGgVU2xWAwStishCpuWT7PvJoxaKvhjBhHis0Yout/zpilmBSco9WiQ9hqPup 5JIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=4cIDR6PZl2rutLAEYSA22jeItW1PJPdhgsRn6XGFWEA=; b=arWkfpLl0zl7Rkvkj0TcHKJNJGtVaVGy9uO2RN9WyUc1DpraK1z2mh/mB1MFSs5sl2 /fVo1CFlUuxUQ0XnptTQ+iKrn6vyDO4upx9tdPHtKvtW8Oos2yGWlSp/i9XaHyhtxJpP 1G2DHSR04Qh8W8k6cb59tvqEQ3jGdz8U6TEVG5/bLQ8cBhdfFwyIHE9YU9YgYITCXjec ANxLmLSUGK7CS99Vixsp/pSiFn537buFOYx6/SBg4b/OqXyUnM0e76C9HKqk637QVmDC 7g4ScSY5jTMnCmEUW8KY74gs5a6ytxhIKBfqDtoOxiomZJJi+IeuvYGsWbatm5lDg+QK huYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=VC6jjd4Q; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id x197si3223284itb.72.2019.05.13.07.39.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:20 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=VC6jjd4Q; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd84M195057; Mon, 13 May 2019 14:39:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=4cIDR6PZl2rutLAEYSA22jeItW1PJPdhgsRn6XGFWEA=; b=VC6jjd4Q/95/JOy5PgCZ8AdKsMFlE4fQCQOC2dyJBtX/9iwCN4mgh4f7m5XZlJT41CUT CWiyU1PZYsAhyoP4PcbGhS++vcss56Xjgewi7dXzHWILBy4ssid7xPlSLghegxyz3MzM lRsyU6oTrYGTNnLa5CF5DZEESUPecwfArax8wnk/8fA+g1/lxpeUfKmiNGTDKimHFL0k BUWpMG9d2DNNstgdU8bqlCEnHiwjnEh1BuYVAwNxdf4+s9KmtLq6O/FzAGHQaEbNfMUf 6YUvBPOPUK+XTc5r2FauxEIQJUNK5TtoYUDMw6GnP5h/kShdnJWw0oXvReVp36peQCHR rw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7avk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:11 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQD022780; Mon, 13 May 2019 14:39:08 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 10/27] kvm/isolation: add KVM page table entry free functions Date: Mon, 13 May 2019 16:38:18 +0200 Message-Id: <1557758315-12667-11-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP These functions are wrappers around the p4d/pud/pmd/pte free function which can be used with any pointer in the directory. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 26 ++++++++++++++++++++++++++ 1 files changed, 26 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 1efdab1..61df750 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -161,6 +161,32 @@ static bool kvm_valid_pgt_entry(void *ptr) } +/* + * kvm_pXX_free() functions are equivalent to kernel pXX_free() + * functions but they can be used with any PXX pointer in the + * directory. + */ + +static inline void kvm_pte_free(struct mm_struct *mm, pte_t *pte) +{ + pte_free_kernel(mm, PGTD_ALIGN(pte)); +} + +static inline void kvm_pmd_free(struct mm_struct *mm, pmd_t *pmd) +{ + pmd_free(mm, PGTD_ALIGN(pmd)); +} + +static inline void kvm_pud_free(struct mm_struct *mm, pud_t *pud) +{ + pud_free(mm, PGTD_ALIGN(pud)); +} + +static inline void kvm_p4d_free(struct mm_struct *mm, p4d_t *p4d) +{ + p4d_free(mm, PGTD_ALIGN(p4d)); +} + static int kvm_isolation_init_mm(void) { From patchwork Mon May 13 14:38:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941057 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1927415AB for ; Mon, 13 May 2019 14:39:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0B40B27FAE for ; Mon, 13 May 2019 14:39:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F39F528334; Mon, 13 May 2019 14:39:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCE1C27FAE for ; Mon, 13 May 2019 14:39:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F0476B026B; Mon, 13 May 2019 10:39:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 47C186B026C; Mon, 13 May 2019 10:39:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2303C6B026D; Mon, 13 May 2019 10:39:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f200.google.com (mail-it1-f200.google.com [209.85.166.200]) by kanga.kvack.org (Postfix) with ESMTP id EFDE36B026B for ; Mon, 13 May 2019 10:39:23 -0400 (EDT) Received: by mail-it1-f200.google.com with SMTP id x143so12246879itb.9 for ; Mon, 13 May 2019 07:39:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=gIeFN0ziPc52a8KLvMF4BYUuKgdMORhtc90dfJFFcK8=; b=YUaDAaKEVGhQWcOTfDNZw4XPtQF980jgJbtBuaMX/QLCmEfl7jI6MvFHtRUzQjEV7+ ZnX1Qw3PVceCWpa9tb0AkS0kld7yz0UrX+WNNdgW9IJQ75Zl7r+jcb4N0IbVNoHQfJri HTiDr6b1ZSGQYprb6ZGjK79aRFD5M7elfP9hawBXlsaYsnDNWz8Ti6lwKhAya03h0bGR e87QTdRJ8ZR3bbOmZZct5J5feoWSky3lmiPfIs3M3AnvBpkByNpDXNucv1dDmqcjHdBI v3ZVCJWtW9FHb36FoDKdFGkmSUBmInZpj3W8HcRG4VCBdeYwBS6o1c1WyhVRTQAlG50w wk7Q== X-Gm-Message-State: APjAAAWr1PdD2K5paU3t6bFcQOp04/GyUHqmzXxTYDYGs1B3kWVcL8+p hO/toC1QkXwSzdQywViYyeSQt7rPO3h9e9WmrDXzyx2mKeZID90zjQd5Ps5msmY5dSJytjCD5t7 eo7lTMFuZWv+2XynkHZB7jMxeH2ZPnJjAKIgHWT1jyqwgYKz3jEygzNL66yMwdMctVw== X-Received: by 2002:a6b:e917:: with SMTP id u23mr7822275iof.136.1557758363746; Mon, 13 May 2019 07:39:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqxB3yzRIYngYNpxOJrkCbxEDtVS7mx19z/PkLRqDIWlNd95HOmobZp4LqiAacAJhp+N6XYN X-Received: by 2002:a6b:e917:: with SMTP id u23mr7822238iof.136.1557758363077; Mon, 13 May 2019 07:39:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758363; cv=none; d=google.com; s=arc-20160816; b=J6rozUWPKS5PVmeCQxjJ9YaCrVU5hLkkVP6Sbr5O1ygFJ5jmHy/2zo27vKYOmg90QY rdyGNm00Gu+4OwawKscO67DScAJlGaYfdbrDAT9sBjQjIFTl9rpkps+qAiRXmQZ0XKUJ 0dcet0PdirNEnynSRxANsxx+Oia+Tbxud0FXENMnyi4svn463YquZ8m6KTg6f+a2bNyt hYjIwLu1G3R/Wg8/Nm+0ThkOqkvzDeKbqH4+GTjJen+0QRxNHBIssavUFZZ+wkp1TLF8 BYk8eOWH2HCX5C6GtU8J9apWKvzNZ+XzlHzba8IdGIq8xg/2wmK60H2WFWauMnOS1CJX V0dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=gIeFN0ziPc52a8KLvMF4BYUuKgdMORhtc90dfJFFcK8=; b=PFQQHb/cDlK0+31KIAxLNdC0vlcQ4QoHeXwWnGM0UlCeCZj25WsV5aIfIveY6hpl6e P0oXzK/EjP9LmJbqaK6mmseI6DmJStLCxROiDDsl0m3dR6ELNkhXCZxJt05pWvZt+Oxz WvVhtQFmJLQ/zNKWh9HqiNwl2RoROED+t5jdnb8ONkn2Rt6vUuyr+q34oKejJEuUEAHw FqIosVGOU9OW7BRNwZfn8Z0a+S9sy2WWRSV90EFnqcOcP41ouU+7dVXmjvb8X6fA3Job oRn9GWw8nIEUQnvDGlRj/ulccUnJZWW2UT4hyH0Rm6IVwL2X4O44t/OwB2STUQrHELIG Belw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=hG+Eotkw; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id z7si7144987ioc.47.2019.05.13.07.39.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:23 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=hG+Eotkw; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd4lg195008; Mon, 13 May 2019 14:39:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=gIeFN0ziPc52a8KLvMF4BYUuKgdMORhtc90dfJFFcK8=; b=hG+EotkwfW0afgyQ6GQFcjNoUZvzVSUEOYaoXlfVoV2cUsog6OXPvWLhJn7Oel/wqLNU q93k3vhzcUlchxQbCpQ1datYttq5bU2CVnCcfgAzotf47Y8hQH+8rBgoN2OmMBnGwcKO LEQEsJMpMtvD5hREw/USDtI1pRBmvG/plElU+Xrot/aCMJUcLyH6vUNMyX9iYSD4MPrI Y0R0Lk4RBx9aTbJnKTE11caIJKBYXFjCKw9NZy0zBsZERP6ZtLnDVZy0YizHnM93iBtI eV66j3vQdt+zdewHBfSeob4gdmSsVVx7BhZaz3q+MBojcv1hUk06rRc6rPcuwhVdDGFV rg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7avt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:14 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQE022780; Mon, 13 May 2019 14:39:11 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 11/27] kvm/isolation: add KVM page table entry offset functions Date: Mon, 13 May 2019 16:38:19 +0200 Message-Id: <1557758315-12667-12-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP These functions are wrappers are the p4d/pud/pmd/pte offset functions which ensure that page table pointers are in the KVM page table. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 61 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 61df750..b29a09b 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -162,6 +162,67 @@ static bool kvm_valid_pgt_entry(void *ptr) } /* + * kvm_pXX_offset() functions are equivalent to kernel pXX_offset() + * functions but, in addition, they ensure that page table pointers + * are in the KVM page table. Otherwise an error is returned. + */ + +static pte_t *kvm_pte_offset(pmd_t *pmd, unsigned long addr) +{ + pte_t *pte; + + pte = pte_offset_map(pmd, addr); + if (!kvm_valid_pgt_entry(pte)) { + pr_err("PTE %px is not in KVM page table\n", pte); + return ERR_PTR(-EINVAL); + } + + return pte; +} + +static pmd_t *kvm_pmd_offset(pud_t *pud, unsigned long addr) +{ + pmd_t *pmd; + + pmd = pmd_offset(pud, addr); + if (!kvm_valid_pgt_entry(pmd)) { + pr_err("PMD %px is not in KVM page table\n", pmd); + return ERR_PTR(-EINVAL); + } + + return pmd; +} + +static pud_t *kvm_pud_offset(p4d_t *p4d, unsigned long addr) +{ + pud_t *pud; + + pud = pud_offset(p4d, addr); + if (!kvm_valid_pgt_entry(pud)) { + pr_err("PUD %px is not in KVM page table\n", pud); + return ERR_PTR(-EINVAL); + } + + return pud; +} + +static p4d_t *kvm_p4d_offset(pgd_t *pgd, unsigned long addr) +{ + p4d_t *p4d; + + p4d = p4d_offset(pgd, addr); + /* + * p4d is the same has pgd if we don't have a 5-level page table. + */ + if ((p4d != (p4d_t *)pgd) && !kvm_valid_pgt_entry(p4d)) { + pr_err("P4D %px is not in KVM page table\n", p4d); + return ERR_PTR(-EINVAL); + } + + return p4d; +} + +/* * kvm_pXX_free() functions are equivalent to kernel pXX_free() * functions but they can be used with any PXX pointer in the * directory. From patchwork Mon May 13 14:38:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941061 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 954BA1390 for ; Mon, 13 May 2019 14:39:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 86FD727FAE for ; Mon, 13 May 2019 14:39:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7B50128305; Mon, 13 May 2019 14:39:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5F1527FAE for ; Mon, 13 May 2019 14:39:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 988016B026D; Mon, 13 May 2019 10:39:27 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8EB566B026E; Mon, 13 May 2019 10:39:27 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73CBB6B026F; Mon, 13 May 2019 10:39:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f198.google.com (mail-it1-f198.google.com [209.85.166.198]) by kanga.kvack.org (Postfix) with ESMTP id 484FE6B026D for ; Mon, 13 May 2019 10:39:27 -0400 (EDT) Received: by mail-it1-f198.google.com with SMTP id l193so2755147ita.8 for ; Mon, 13 May 2019 07:39:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=3oQZsnmQ7NshrUK5gBFOiQdwJCmvyMk7rTDgLZ4eiR8=; b=piDYExRTFfeUF6XkxVMbxVOtozhjY1EJGNFLHaPUBNjCrUaEZ40gDY/w/dKL2M76r4 sGcm9llkyIHRWPJzwnJVOIY3LKsB5zpxTuGCkIXJstW0GOq80MRLap+LJZNOt9QGOKuz dncf7OYexlPypqL5q307QEq7FiASrTfNGDJNDb7D8x03VrkPlyEz/TcxlpmY4x1/woCN 7c7kCnFyb3WaNJNcOjLQYYegjxAoxhCbdgOF5FXYvpvthnQTUclkbDrYVTQORYGZrZ8Z +GRmFGKRrdmegLGiLaUBnZ2AF0ka2L7WK1rLBT7OAvpXzecV2FAnGSll20M9Wq5jYxPe ksgQ== X-Gm-Message-State: APjAAAVMRCq/4QmIcNd7g/aa3fRHgnbrERxFwZFOvPaeBjmYJJL18o9o ujMnHpJx0pwOhN/FuLQTwXhhU2ctq7UXD1oaqZSX/5w1u5XQGn6i+hKnTHWE0+xRy06Upuz8fwt xuYM6t61eG2gCXHvF9Td8ESwFdqVO7grKIhq9iqEwm3fCbVyD9LGnFccbHjnq5zPJUg== X-Received: by 2002:a6b:14ce:: with SMTP id 197mr15661240iou.29.1557758367073; Mon, 13 May 2019 07:39:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqwQExO2FUE+CYKtkjFEET3dW29Ayk44bbP+TIruMPLyd5GcaoyGddH65NbT4h1mRqLQVZ6N X-Received: by 2002:a6b:14ce:: with SMTP id 197mr15661192iou.29.1557758366316; Mon, 13 May 2019 07:39:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758366; cv=none; d=google.com; s=arc-20160816; b=I+2T+mtdXaBoCI0oknwsio7WhmYH6N2haYMa2omVmQCQcg6DSORUncOpNkAhZTdeoj q1QnpDC3azh9bXnhIS5PQfohfLgWzmnZ9lgMeHPtXU1hq3GoM5HvBpQwyQHNofDq7ucy JsmwRdQSZ8iP0lZg/9jSeOIqgKk8C6h8aDK8BjovT71Mskn5QpaO7ulYEz33nrcnjHft lYS1D5XsxgQTSiwl8o406qAEL0wXciTvP8OYWOH0v+zA0WqHHZBBE4NhcbPeZl8DJi8W VLkSeidhDDVeYH3DrcaNaeQst4izZlP/lLQHNyjQyEZSBm90NwkKhOlrGzN4JhOF7J8+ mUPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=3oQZsnmQ7NshrUK5gBFOiQdwJCmvyMk7rTDgLZ4eiR8=; b=qk1WRF8rtcOYXHlINqEwmHYiv9u3zKNy6cBYRZ5gQhG7q+uT1SZVMYuC2UtQAs8mx4 lwuBK1+0YKSUoR0nyteMY5ajImKbxRZbmg6mdwYJuGzIU/RD4iEgrVAaTkz+vOEWGA2y 9+VhwcozHY0ftQRD1/dXKjW79VEQjnZ3yqfvr6DJjVE/6jbrf6ozJVgCkT3t6066Q1uG FQ1+5MsaaMJPgUz8WgczjnNqBBTz0C+jQczhisVHmWHsg2Gb05XjHoprdJ6kQs/ujTm0 KPvPLc5JbgCh4R/AWFiINk0rbfeR/WjcKCisU87WlyktCv4MRKTDKv/rTlZPUunvMc4p RZUw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="cackC5e/"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id c17si82803itc.40.2019.05.13.07.39.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:26 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="cackC5e/"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd28F193025; Mon, 13 May 2019 14:39:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=3oQZsnmQ7NshrUK5gBFOiQdwJCmvyMk7rTDgLZ4eiR8=; b=cackC5e/rN14Atf5QlnsuYve24xrmHKk5OAv4LozaAp9vhiUL7E7xPH3kbnOM8VYLH+M qQZDMDajYAWzXDX4pYfxSi/cFR9+miTzGtpQvwsGANTuO936Hac7155wikox7cO9zHM6 D6VSW0wFlH4kS9hhjEHDuhBzZL61Y1YPaaQ9QapuqevDj5phnii9KR4rMc6oFyvj+0cK chnwMcSACXVNjOAa1xqmZO14HcVVu9E4v0xi+ZQhdjbaq/Q1oelYLJg/hz12bN7CBmtg 1H1C0ZDiqvGmCSUtMdjVlrxOudApkaffggciBhogmTfWK5aks2i24a2IdgYkJ3LSxlPk OA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfkwq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:16 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQF022780; Mon, 13 May 2019 14:39:13 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 12/27] kvm/isolation: add KVM page table entry allocation functions Date: Mon, 13 May 2019 16:38:20 +0200 Message-Id: <1557758315-12667-13-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP These functions allocate p4d/pud/pmd/pte pages and ensure that pages are in the KVM page table. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 94 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index b29a09b..6ec86df 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -248,6 +248,100 @@ static inline void kvm_p4d_free(struct mm_struct *mm, p4d_t *p4d) p4d_free(mm, PGTD_ALIGN(p4d)); } +/* + * kvm_pXX_alloc() functions are equivalent to kernel pXX_alloc() + * functions but, in addition, they ensure that page table pointers + * are in the KVM page table. Otherwise an error is returned. + */ + +static pte_t *kvm_pte_alloc(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr) +{ + pte_t *pte; + + if (pmd_none(*pmd)) { + pte = pte_alloc_kernel(pmd, addr); + if (!pte) { + pr_debug("PTE: ERR ALLOC\n"); + return ERR_PTR(-ENOMEM); + } + if (!kvm_add_pgt_directory(pte, PGT_LEVEL_PTE)) { + kvm_pte_free(mm, pte); + return ERR_PTR(-EINVAL); + } + } else { + pte = kvm_pte_offset(pmd, addr); + } + + return pte; +} + +static pmd_t *kvm_pmd_alloc(struct mm_struct *mm, pud_t *pud, + unsigned long addr) +{ + pmd_t *pmd; + + if (pud_none(*pud)) { + pmd = pmd_alloc(mm, pud, addr); + if (!pmd) { + pr_debug("PMD: ERR ALLOC\n"); + return ERR_PTR(-ENOMEM); + } + if (!kvm_add_pgt_directory(pmd, PGT_LEVEL_PMD)) { + kvm_pmd_free(mm, pmd); + return ERR_PTR(-EINVAL); + } + } else { + pmd = kvm_pmd_offset(pud, addr); + } + + return pmd; +} + +static pud_t *kvm_pud_alloc(struct mm_struct *mm, p4d_t *p4d, + unsigned long addr) +{ + pud_t *pud; + + if (p4d_none(*p4d)) { + pud = pud_alloc(mm, p4d, addr); + if (!pud) { + pr_debug("PUD: ERR ALLOC\n"); + return ERR_PTR(-ENOMEM); + } + if (!kvm_add_pgt_directory(pud, PGT_LEVEL_PUD)) { + kvm_pud_free(mm, pud); + return ERR_PTR(-EINVAL); + } + } else { + pud = kvm_pud_offset(p4d, addr); + } + + return pud; +} + +static p4d_t *kvm_p4d_alloc(struct mm_struct *mm, pgd_t *pgd, + unsigned long addr) +{ + p4d_t *p4d; + + if (pgd_none(*pgd)) { + p4d = p4d_alloc(mm, pgd, addr); + if (!p4d) { + pr_debug("P4D: ERR ALLOC\n"); + return ERR_PTR(-ENOMEM); + } + if (!kvm_add_pgt_directory(p4d, PGT_LEVEL_P4D)) { + kvm_p4d_free(mm, p4d); + return ERR_PTR(-EINVAL); + } + } else { + p4d = kvm_p4d_offset(pgd, addr); + } + + return p4d; +} + static int kvm_isolation_init_mm(void) { From patchwork Mon May 13 14:38:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941065 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 496496C5 for ; Mon, 13 May 2019 14:39:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 381A027FAE for ; Mon, 13 May 2019 14:39:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 28A3A2817F; Mon, 13 May 2019 14:39:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 976E428173 for ; Mon, 13 May 2019 14:39:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA2E06B026F; Mon, 13 May 2019 10:39:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D2CE06B0270; Mon, 13 May 2019 10:39:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B805C6B0271; Mon, 13 May 2019 10:39:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by kanga.kvack.org (Postfix) with ESMTP id 919576B026F for ; Mon, 13 May 2019 10:39:34 -0400 (EDT) Received: by mail-io1-f71.google.com with SMTP id s16so4234526ioe.22 for ; Mon, 13 May 2019 07:39:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=5Qtz3s/XQzDItLwgII6a7yqyolD8+y7jKItUNNZ5jHw=; b=aBB/HOTqohJeyniyrqo5cwGyxpRUgXgXhN400akjn6xsXH27xumJ35SsARxt3tjsP+ 2a4vFpnv2vpzCamv28k+gpCkvwzBRHRJ0bzEZGmStMB4FFLdRAOpHUl2J7Hvid0gRqye 8eryjp/rguI9+ZWuqr9fO36EudTagk5H+SInqGSz8z0QplqAyJGjCwbjS4d3QCI5laJF CmMND1VvYUz2SQ35FsG0mpECopNoAXDF2VAKiX6q++MeQaqEEs+tO5sDFKc4ynf8LtDU 3qxp906pqweMOL+KPvdANqO/vPo+jZn5SqVpzvF04EFVw84SO6EVQ8bbf6otsvCiyIU/ 5qEw== X-Gm-Message-State: APjAAAVmYo2QsieFKxkbtHNwqwQsKtCGZ4fGD8+6voiDBWhTYCnXYtaN 6iVaIZPASuMVBj7BmIT5Zm6Nk5cdgw5h4w7E78yNXlWk4ss2athsOPc3IXCsoHUObJ3gfzGJYMs WGrXdnxH6t1bUlsMOXEz+KyJ7kExvfwj7Ri3ZBpmyyns1r6FWNdGdH+JowWD95MeVUQ== X-Received: by 2002:a6b:b485:: with SMTP id d127mr15669209iof.273.1557758374327; Mon, 13 May 2019 07:39:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqyvDU3Nwf8mFFBikeEMO8251HzfLWluPQgB5K1DuBv6JyaWP/FInrEOS7bTZ+Wf5b8/jj8v X-Received: by 2002:a6b:b485:: with SMTP id d127mr15669150iof.273.1557758373499; Mon, 13 May 2019 07:39:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758373; cv=none; d=google.com; s=arc-20160816; b=XyJpFLr7AzBQxSCT2TyqOplsySinFTHlAwQqPpVb+bZ/m5cx48wpEENWVoGCAiTads z9pUoHG7LQOYDSSgpE9WU991A51PUFJ8eiuM53pIynp3hYK/kSheycOnr92GBgRCmb3+ uaudqaS9bPL4i4oFON5yRJRyd07VGdlhA1zWArT57KHcItL72x6TgfszvTy1Tf/ggDFj 9SsVXxglvLcImzWTaQ+rfjeZvp0F4y+hV3kPqI6TIkKPtv7KiKFBGWuV1pKEVpKOaPDc D2ULNEADTQRLERmzYr1vwiclQqh5SGHHXc+LVTmtM10sia3t1Oacx9oVNPzeofHQqrXI 8d1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5Qtz3s/XQzDItLwgII6a7yqyolD8+y7jKItUNNZ5jHw=; b=FPhPMztzTTWJtnEqNp+ZYdfynEeDY81Pzd7F5624tGG18AH2FceR27gT/80Hmat3Of P4Z/1TNiSQUsDNDD3Dhh8Bsbw7A+aDhlbLUes37Nv5649G6x9J5m+RdXJOlgDGkAA1KQ S63jaoZh5myPO/WvLn5wyuCLdbZgFDqd4r5l2haD8yOIFN6/sDIgTUEhnzObFnhmFHt9 LLwuWN6pEubaDzc2JUELXjiiPaUkcvkk2m98sUPEkOOPhLiEAAfnmrlz1FXMcR0X2OJv 5Rs+1tuoITMB74rpiVwNmnZlf7+fWdqiclzwo9gHx15cuhr2fxJM9dNNbYBD0YpJcvt7 QhKw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=bc0UAXJu; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id b13si8210483itb.143.2019.05.13.07.39.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:33 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=bc0UAXJu; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd3jQ193102; Mon, 13 May 2019 14:39:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=5Qtz3s/XQzDItLwgII6a7yqyolD8+y7jKItUNNZ5jHw=; b=bc0UAXJuEUDdxvyu9LwCdXHqD4qh6pNojWMqcM9jSLBAxGM6igASeeKPL3IWePb5ch0s fbCFf/kSYwqFzCOuWD0pbNuB78gAyLFK5Ed4veTVvUvCkVI2DR43VeRRk0g9ufQAr0sw e6QhuGNeEnywPiXsoWdU2luDBNsmNNUqdcfqgUJyVy51fsmTOMsXHwAuC4cslhey/L9A UYFWoKFzlhctdRm9y9d9iC7Djqvsf9+FCNhU/NbXFxMpjmDvfMOaaoR1Y5lpKRNKz4fW VMMMFGBaxysR4n+y/P893dNcj5PpeJwOEyHs1Tnsryx4DvPOgcmS7i7F6ug4vAd8pZZL 3A== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfkxg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:24 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQG022780; Mon, 13 May 2019 14:39:16 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 13/27] kvm/isolation: add KVM page table entry set functions Date: Mon, 13 May 2019 16:38:21 +0200 Message-Id: <1557758315-12667-14-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Add wrappers around the page table entry (pgd/p4d/pud/pmd) set function to check that an existing entry is not being overwritten. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 107 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 107 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 6ec86df..b681e4f 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -342,6 +342,113 @@ static inline void kvm_p4d_free(struct mm_struct *mm, p4d_t *p4d) return p4d; } +/* + * kvm_set_pXX() functions are equivalent to kernel set_pXX() functions + * but, in addition, they ensure that they are not overwriting an already + * existing reference in the page table. Otherwise an error is returned. + * + * Note that this is not used for PTE because a PTE entry points to page + * frames containing the actual user data, and not to another entry in the + * page table. However this is used for PGD. + */ + +static int kvm_set_pmd(pmd_t *pmd, pmd_t pmd_value) +{ +#ifdef DEBUG + /* + * The pmd pointer should come from kvm_pmd_alloc() or kvm_pmd_offset() + * both of which check if the pointer is in the KVM page table. So this + * is a paranoid check to ensure the pointer is really in the KVM page + * table. + */ + if (!kvm_valid_pgt_entry(pmd)) { + pr_err("PMD %px is not in KVM page table\n", pmd); + return -EINVAL; + } +#endif + if (pmd_val(*pmd) == pmd_val(pmd_value)) + return 0; + + if (!pmd_none(*pmd)) { + pr_err("PMD %px: overwriting %lx with %lx\n", + pmd, pmd_val(*pmd), pmd_val(pmd_value)); + return -EBUSY; + } + + set_pmd(pmd, pmd_value); + + return 0; +} + +static int kvm_set_pud(pud_t *pud, pud_t pud_value) +{ +#ifdef DEBUG + /* + * The pud pointer should come from kvm_pud_alloc() or kvm_pud_offset() + * both of which check if the pointer is in the KVM page table. So this + * is a paranoid check to ensure the pointer is really in the KVM page + * table. + */ + if (!kvm_valid_pgt_entry(pud)) { + pr_err("PUD %px is not in KVM page table\n", pud); + return -EINVAL; + } +#endif + if (pud_val(*pud) == pud_val(pud_value)) + return 0; + + if (!pud_none(*pud)) { + pr_err("PUD %px: overwriting %lx\n", pud, pud_val(*pud)); + return -EBUSY; + } + + set_pud(pud, pud_value); + + return 0; +} + +static int kvm_set_p4d(p4d_t *p4d, p4d_t p4d_value) +{ +#ifdef DEBUG + /* + * The p4d pointer should come from kvm_p4d_alloc() or kvm_p4d_offset() + * both of which check if the pointer is in the KVM page table. So this + * is a paranoid check to ensure the pointer is really in the KVM page + * table. + */ + if (!kvm_valid_pgt_entry(p4d)) { + pr_err("P4D %px is not in KVM page table\n", p4d); + return -EINVAL; + } +#endif + if (p4d_val(*p4d) == p4d_val(p4d_value)) + return 0; + + if (!p4d_none(*p4d)) { + pr_err("P4D %px: overwriting %lx\n", p4d, p4d_val(*p4d)); + return -EBUSY; + } + + set_p4d(p4d, p4d_value); + + return 0; +} + +static int kvm_set_pgd(pgd_t *pgd, pgd_t pgd_value) +{ + if (pgd_val(*pgd) == pgd_val(pgd_value)) + return 0; + + if (!pgd_none(*pgd)) { + pr_err("PGD %px: overwriting %lx\n", pgd, pgd_val(*pgd)); + return -EBUSY; + } + + set_pgd(pgd, pgd_value); + + return 0; +} + static int kvm_isolation_init_mm(void) { From patchwork Mon May 13 14:38:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941063 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92DE86C5 for ; Mon, 13 May 2019 14:39:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82ACC2817F for ; Mon, 13 May 2019 14:39:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 75FFC28173; Mon, 13 May 2019 14:39:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A844628173 for ; Mon, 13 May 2019 14:39:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F27B6B026E; Mon, 13 May 2019 10:39:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 455D46B026F; Mon, 13 May 2019 10:39:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25BCD6B0270; Mon, 13 May 2019 10:39:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by kanga.kvack.org (Postfix) with ESMTP id EE4C96B026E for ; Mon, 13 May 2019 10:39:32 -0400 (EDT) Received: by mail-io1-f72.google.com with SMTP id y15so9986212iod.10 for ; Mon, 13 May 2019 07:39:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=HIfaAaJqDpQgZqOlqv93OYEN222yx/NWgo2RgdnE7Ys=; b=Hb9+4ID65uTiE9KUVx+wbYarbn0Hh5dS5B6dm51whdZiCMV1IFNQ+shr1Pq8kRL3Mo wRXknBlJ0wgqYU/s5IBEOollbpHnq2QCoU0ike1ES0zaNc/vnnfHjfzv1NUrbekpnKZn Dmjz7X6UUMRDmg36LWeICov++xKtbU4se8ssrQLW10a7C/92cw5eXhaIFmjc7a0pI/Sg sIBG0K74K6HVC/5uyG0VIFQ3HYDaq/wnznV5Og78ihkCX385yYiiZa+lmC57R5o2FsST jez9sk2H8e4HA/JYC4ZqMa1zfDcDXAqJqHdXaRLwq6T9I1kw2/zYsqgVRg/5CUoZiShn 16lw== X-Gm-Message-State: APjAAAUoQz3KFTnCxOQ3BsuI5HL0sTN2/p8GjacY2vhm+ieL5jpGQPUm ITNoXDpN7MeY10L6bOdwqYgDnzhkBY4IUs7SDwE/LbHdTCobR1c3YSFhTo/j9cjJKzp0kbFdd5W TjcDihSOY6CWudph4S6QYVbPDmUXYtdimU2h1/it9DZViY5MTk9IVuAtpE0d1NynHRQ== X-Received: by 2002:a6b:14ce:: with SMTP id 197mr15661550iou.29.1557758372679; Mon, 13 May 2019 07:39:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqyrk6602vsJype93i5KY1HXl3KP7JuUx9bhHHCbgiye3/oiI4BhL8ekBX+N/x8xHo5Xjs/6 X-Received: by 2002:a6b:14ce:: with SMTP id 197mr15661496iou.29.1557758371709; Mon, 13 May 2019 07:39:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758371; cv=none; d=google.com; s=arc-20160816; b=uRtRJ0CQfXDZad5M7fIKeeDL7Rh4LVuIawUA3g6yDhRiHLK7BeCbauMxTJbIsk/IXK dcEJcBQjGdYI2cKgMfn0WiTEPIkNSbAzYMvL0Akyk9FNOOvcj34pMfj55lCtwcCvjmDF eYHTVwYXMIqyP2H2LcZwaCV/eS+j8atPHg79cDpHTBM0mKBbRSRUhN73vBJum6+X9D3b Yqx87tjG0Av619OIJSLqUNh57PU5EmX+buVvjghS8PHzuS+ng1bdtdGO5h6A0QMb+SE5 6ZKRlagqBzgru0YMtwQNjpgB7GcTHJi3YCelnM3cc+KGnsd+lgWmNlWvm4qrE4Az8vDC V8NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HIfaAaJqDpQgZqOlqv93OYEN222yx/NWgo2RgdnE7Ys=; b=OICrLbwuoyogil5BoM5r39sDP4kpzDn+daqS8kvXCBwofG+tuiWnRnUojfRcrfTJuk YgfMhUaZCszbxcwsCapRSZ18NcOrBYeFPdor2qggl4fqta4tP92MrQX1omNkLa1c3ker Jc0fF+PTt2JyRaNJamiPFGHbkTirrGmqAxZeasxxLn8aBHk2kaV7hNprxHCMr1WZpSZ6 fxGh/OwJbAqYZYibaKoTJDWFtRpokmBj1UNueCD+5XHaEcJL4OSA+Cc2P19tQ86PLGNl ZGNqSmR9Ckf9Awuawyn1HE7ifJP/d/L8y2jDF/C6gIzMiO7MqdYksQ6UIDZDwYIoIgp1 z4EA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="rJqFVfy/"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id g127si7640610jag.119.2019.05.13.07.39.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:31 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="rJqFVfy/"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2pd194892; Mon, 13 May 2019 14:39:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=HIfaAaJqDpQgZqOlqv93OYEN222yx/NWgo2RgdnE7Ys=; b=rJqFVfy/BraQo2VV+vTf4vEU8uKTxNMJYqNUMPJntKe3SPbYIqJswWfe8FSX9DOUDUvq NMeBPV/EFppUnPmGQYbDoMnmg/dC+wvoFJX/MAxWhRp9eR7c70Ir05eJ9ObgYhuTZIN0 D6O2EwvBsR4WuKaiukLz+63YyUEe8qIjPm7QE5ehY3t9v0MmShxoS6X2fFxg5nPXKcjD 5e42nLFl9jyGEOT3WQUgTW0ZEa0BSwnYUeiOjo0jQFfif/6jx2982K9zu7DiWOea0CSj W4Xl7iGJz5g3uHt6WP7akNTkktkKiod+MAJFVIj50sV+mpTRrNCEYkdwKX02CtXlEBbR ag== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7aws-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:23 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQH022780; Mon, 13 May 2019 14:39:19 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 14/27] kvm/isolation: functions to copy page table entries for a VA range Date: Mon, 13 May 2019 16:38:22 +0200 Message-Id: <1557758315-12667-15-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP These functions are based on the copy_pxx_range() functions defined in mm/memory.c. The main difference is that a level parameter is specified to indicate the page table level (PGD, P4D, PUD PMD, PTE) at which the copy should be done. Also functions don't use a vma parameter, and don't alter the source page table even if an entry is bad. Also kvm_copy_pte_range() can be called with a non page-aligned buffer, so the buffer should be aligned with the page start so that the entire buffer is mapped if the end of buffer crosses a page. These functions will be used to populate the KVM page table. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 1 + 2 files changed, 230 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index b681e4f..4f1b511 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -450,6 +450,235 @@ static int kvm_set_pgd(pgd_t *pgd, pgd_t pgd_value) } +static int kvm_copy_pte_range(struct mm_struct *dst_mm, + struct mm_struct *src_mm, pmd_t *dst_pmd, + pmd_t *src_pmd, unsigned long addr, + unsigned long end) +{ + pte_t *src_pte, *dst_pte; + + dst_pte = kvm_pte_alloc(dst_mm, dst_pmd, addr); + if (IS_ERR(dst_pte)) + return PTR_ERR(dst_pte); + + addr &= PAGE_MASK; + src_pte = pte_offset_map(src_pmd, addr); + + do { + pr_debug("PTE: %lx/%lx set[%lx] = %lx\n", + addr, addr + PAGE_SIZE, (long)dst_pte, pte_val(*src_pte)); + set_pte_at(dst_mm, addr, dst_pte, *src_pte); + + } while (dst_pte++, src_pte++, addr += PAGE_SIZE, addr < end); + + return 0; +} + +static int kvm_copy_pmd_range(struct mm_struct *dst_mm, + struct mm_struct *src_mm, + pud_t *dst_pud, pud_t *src_pud, + unsigned long addr, unsigned long end, + enum page_table_level level) +{ + pmd_t *src_pmd, *dst_pmd; + unsigned long next; + int err; + + dst_pmd = kvm_pmd_alloc(dst_mm, dst_pud, addr); + if (IS_ERR(dst_pmd)) + return PTR_ERR(dst_pmd); + + src_pmd = pmd_offset(src_pud, addr); + + do { + next = pmd_addr_end(addr, end); + if (level == PGT_LEVEL_PMD || pmd_none(*src_pmd)) { + pr_debug("PMD: %lx/%lx set[%lx] = %lx\n", + addr, next, (long)dst_pmd, pmd_val(*src_pmd)); + err = kvm_set_pmd(dst_pmd, *src_pmd); + if (err) + return err; + continue; + } + + if (!pmd_present(*src_pmd)) { + pr_warn("PMD: not present for [%lx,%lx]\n", + addr, next - 1); + pmd_clear(dst_pmd); + continue; + } + + if (pmd_trans_huge(*src_pmd) || pmd_devmap(*src_pmd)) { + pr_debug("PMD: %lx/%lx set[%lx] = %lx (huge/devmap)\n", + addr, next, (long)dst_pmd, pmd_val(*src_pmd)); + err = kvm_set_pmd(dst_pmd, *src_pmd); + if (err) + return err; + continue; + } + + err = kvm_copy_pte_range(dst_mm, src_mm, dst_pmd, src_pmd, + addr, next); + if (err) { + pr_err("PMD: ERR PTE addr=%lx next=%lx\n", addr, next); + return err; + } + + } while (dst_pmd++, src_pmd++, addr = next, addr < end); + + return 0; +} + +static int kvm_copy_pud_range(struct mm_struct *dst_mm, + struct mm_struct *src_mm, + p4d_t *dst_p4d, p4d_t *src_p4d, + unsigned long addr, unsigned long end, + enum page_table_level level) +{ + pud_t *src_pud, *dst_pud; + unsigned long next; + int err; + + dst_pud = kvm_pud_alloc(dst_mm, dst_p4d, addr); + if (IS_ERR(dst_pud)) + return PTR_ERR(dst_pud); + + src_pud = pud_offset(src_p4d, addr); + + do { + next = pud_addr_end(addr, end); + if (level == PGT_LEVEL_PUD || pud_none(*src_pud)) { + pr_debug("PUD: %lx/%lx set[%lx] = %lx\n", + addr, next, (long)dst_pud, pud_val(*src_pud)); + err = kvm_set_pud(dst_pud, *src_pud); + if (err) + return err; + continue; + } + + if (pud_trans_huge(*src_pud) || pud_devmap(*src_pud)) { + pr_debug("PUD: %lx/%lx set[%lx] = %lx (huge/devmap)\n", + addr, next, (long)dst_pud, pud_val(*src_pud)); + err = kvm_set_pud(dst_pud, *src_pud); + if (err) + return err; + continue; + } + + err = kvm_copy_pmd_range(dst_mm, src_mm, dst_pud, src_pud, + addr, next, level); + if (err) { + pr_err("PUD: ERR PMD addr=%lx next=%lx\n", addr, next); + return err; + } + + } while (dst_pud++, src_pud++, addr = next, addr < end); + + return 0; +} + +static int kvm_copy_p4d_range(struct mm_struct *dst_mm, + struct mm_struct *src_mm, + pgd_t *dst_pgd, pgd_t *src_pgd, + unsigned long addr, unsigned long end, + enum page_table_level level) +{ + p4d_t *src_p4d, *dst_p4d; + unsigned long next; + int err; + + dst_p4d = kvm_p4d_alloc(dst_mm, dst_pgd, addr); + if (IS_ERR(dst_p4d)) + return PTR_ERR(dst_p4d); + + src_p4d = p4d_offset(src_pgd, addr); + + do { + next = p4d_addr_end(addr, end); + if (level == PGT_LEVEL_P4D || p4d_none(*src_p4d)) { + pr_debug("P4D: %lx/%lx set[%lx] = %lx\n", + addr, next, (long)dst_p4d, p4d_val(*src_p4d)); + + err = kvm_set_p4d(dst_p4d, *src_p4d); + if (err) + return err; + continue; + } + + err = kvm_copy_pud_range(dst_mm, src_mm, dst_p4d, src_p4d, + addr, next, level); + if (err) { + pr_err("P4D: ERR PUD addr=%lx next=%lx\n", addr, next); + return err; + } + + } while (dst_p4d++, src_p4d++, addr = next, addr < end); + + return 0; +} + +static int kvm_copy_pgd_range(struct mm_struct *dst_mm, + struct mm_struct *src_mm, unsigned long addr, + unsigned long end, enum page_table_level level) +{ + pgd_t *src_pgd, *dst_pgd; + unsigned long next; + int err; + + dst_pgd = pgd_offset(dst_mm, addr); + src_pgd = pgd_offset(src_mm, addr); + + do { + next = pgd_addr_end(addr, end); + if (level == PGT_LEVEL_PGD || pgd_none(*src_pgd)) { + pr_debug("PGD: %lx/%lx set[%lx] = %lx\n", + addr, next, (long)dst_pgd, pgd_val(*src_pgd)); + err = kvm_set_pgd(dst_pgd, *src_pgd); + if (err) + return err; + continue; + } + + err = kvm_copy_p4d_range(dst_mm, src_mm, dst_pgd, src_pgd, + addr, next, level); + if (err) { + pr_err("PGD: ERR P4D addr=%lx next=%lx\n", addr, next); + return err; + } + + } while (dst_pgd++, src_pgd++, addr = next, addr < end); + + return 0; +} + +/* + * Copy page table entries from the current page table (i.e. from the + * kernel page table) to the KVM page table. The level parameter specifies + * the page table level (PGD, P4D, PUD PMD, PTE) at which the copy should + * be done. + */ +static int kvm_copy_mapping(void *ptr, size_t size, enum page_table_level level) +{ + unsigned long addr = (unsigned long)ptr; + unsigned long end = addr + ((unsigned long)size); + + BUG_ON(current->mm == &kvm_mm); + pr_debug("KERNMAP COPY addr=%px size=%lx\n", ptr, size); + return kvm_copy_pgd_range(&kvm_mm, current->mm, addr, end, level); +} + + +/* + * Copy page table PTE entries from the current page table to the KVM + * page table. + */ +int kvm_copy_ptes(void *ptr, unsigned long size) +{ + return kvm_copy_mapping(ptr, size, PGT_LEVEL_PTE); +} +EXPORT_SYMBOL(kvm_copy_ptes); + + static int kvm_isolation_init_mm(void) { pgd_t *kvm_pgd; diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index aa5e979..e8c018a 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -16,5 +16,6 @@ static inline bool kvm_isolation(void) extern void kvm_isolation_enter(void); extern void kvm_isolation_exit(void); extern void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu); +extern int kvm_copy_ptes(void *ptr, unsigned long size); #endif From patchwork Mon May 13 14:38:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941067 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3401B1390 for ; Mon, 13 May 2019 14:39:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2498D27FAE for ; Mon, 13 May 2019 14:39:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 178A72817F; Mon, 13 May 2019 14:39:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BFA128173 for ; Mon, 13 May 2019 14:39:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CDFF86B0270; Mon, 13 May 2019 10:39:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C69356B0271; Mon, 13 May 2019 10:39:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0CCA6B0272; Mon, 13 May 2019 10:39:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by kanga.kvack.org (Postfix) with ESMTP id 8AE776B0270 for ; Mon, 13 May 2019 10:39:35 -0400 (EDT) Received: by mail-io1-f70.google.com with SMTP id s16so4234557ioe.22 for ; Mon, 13 May 2019 07:39:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=bWRO35KV4FFQ3ievQKGZUgx84GSk+HyQdow97r7zwRo=; b=Js76tlH/DOubF1QxoOSPElcuDoIto+/8mfO9+o6RoMhGX4RLEHytnB+z14R4d5ToSv e7hrYsTd2O7sHgV3HWrkDUUt4Gz6pOf7Lrw00y73T0jI+6IW224TogLB9RM6G8XxOlM9 NwHbbPNmXlHm0mqnzMp1e+vAzI9tm6trnykrAIcpJV+18+qMwoNRRRE/qIFbHhJc9Kjl qnuT6It+Y+c9mi8fWkQHTfMq7Gmq9nJOK/X0AhLxxmmJpopd4YMM0VabyqBPbStdOrTZ nlOtQ9R0BVfqHRhAWGhbYzozILZ8CGnKWaplVTJG3mr4VhnB8Sr/wc1pwhApSxq1Nsgl xZtg== X-Gm-Message-State: APjAAAUsWvGY8GOBNRRA9dePaEJQL3isPZW9KEB5F1UXyt/IsZsFFXqh Gh9ex6AFApNoPBSE+cz9ob4gnCE173yq/f54qtQef2wWavXoFCTHRPEMgXGPwWDPx8vXarJHctc jKFuKFAZrXGgahkO/GC+rsCC33w0phmlPmVAqlBoauwe6omWwlkbQohdajHH7o4ZQSw== X-Received: by 2002:a6b:6619:: with SMTP id a25mr15889849ioc.131.1557758375294; Mon, 13 May 2019 07:39:35 -0700 (PDT) X-Google-Smtp-Source: APXvYqy+E1gvQJxoObq4NNNB9Vdx/wTnS4lNxJkOG5Z5SU7wyoo7yPtnUuGBA6ceN7dJojbAkySg X-Received: by 2002:a6b:6619:: with SMTP id a25mr15889800ioc.131.1557758374483; Mon, 13 May 2019 07:39:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758374; cv=none; d=google.com; s=arc-20160816; b=Q5rY7r9pfL/zdRi27+7G3TLHLcPcIt+YLNXKDwdVidmrj5Avcj+MyWiwuzRm3bn4WT LzivXgNLYQ5I4CabRybVwD+qrc3mwLKFEkVbwQN0jjesRPAT4qG+DZgPVfaWds39LrSd gHvzOMW7UdlYj/XwTOxbMOX3isNrb/SHUrlLL33/7ZlbmHIrHXdGcRYrjn2cFzmfkpox 8VTieLEa0zVdgGKOSeWv38TM6jCAlqMusQRirWD+qvsCOVxKe5DX16lqJxoaSD4zE85b 7iB9TEAMddoGBr+FMMlK8LqJpfFV/7KEdoUIyuihRdM1YRi8POdQJoueV8NLyVan0aVy XYyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bWRO35KV4FFQ3ievQKGZUgx84GSk+HyQdow97r7zwRo=; b=a8PY4rHSQZFxbCssC2ux8ZYOa4fbk25Adg2aPhk2M/Qrb4cpxkwHedlrsZoEUcCxO3 nzu1DA5UDGfLzpeCXI71k/X2ApM2pqacdbtt/EITcfet4BmBuLC3QJQC69m8uqfQw2am BvUbUsMM+kwxjLS5D4CKQRF82UuWE+Krr8iklkH3funcF3aUbfBFXnilUZOOTIaeAAYC ZpEeZfPrqYtH3AKO60OeneP3q1y86RJYYEAeWF6H9yzc+GDjUQcFvKUEjpbqagGYUw/0 B/D2zy6Bm6UPSZEQA9lnpwvGLc1YRmz7XFAiwAJinaumdlAL+Jh4WvwR8C+t55MqqGlm 6VUw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=m4RXoI1f; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2130.oracle.com (userp2130.oracle.com. [156.151.31.86]) by mx.google.com with ESMTPS id f17si8580946itk.6.2019.05.13.07.39.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:34 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) client-ip=156.151.31.86; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=m4RXoI1f; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd3Xk181510; Mon, 13 May 2019 14:39:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=bWRO35KV4FFQ3ievQKGZUgx84GSk+HyQdow97r7zwRo=; b=m4RXoI1fsAKJ65HwcxYTTiRielpSLksHyApCP0jeXXPvtewp5rwRVYDUYtxPM3t1rURV OUQPSWm2qaD3jmmQMsTFovUD4DnaSZ9GrLWp+D/fBi4LLvQVebxgVSc9OJHFf2CRAqzr jMs2DWV9OiHzkIdKAyWvrDZ3MS42p42Wdt2fSuBYwE1/w/asqQj5umS4w9GtEYvIA23u 3rWtsxf8YqaVnsN0/d1amhOkTseV2bABVDPs67GwZ9yZi2jVBvqt2G58KXhAHzSc5cet 2eTD9xR3JIRNVohIEeFBP6T1HxJGuEpp2qFDS0MQNFcDFncFDyVl1u6MqmF255tAofwm Xw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2sdnttfeh6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:25 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQI022780; Mon, 13 May 2019 14:39:22 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 15/27] kvm/isolation: keep track of VA range mapped in KVM address space Date: Mon, 13 May 2019 16:38:23 +0200 Message-Id: <1557758315-12667-16-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This will be used when we have to clear mappings to ensure the same range is cleared at the same page table level it was copied. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 86 ++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 84 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 4f1b511..c8358a9 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -61,6 +61,20 @@ struct pgt_directory_group { #define PGTD_ALIGN(entry) \ ((typeof(entry))(((unsigned long)(entry)) & PAGE_MASK)) +/* + * Variables to keep track of address ranges mapped into the KVM + * address space. + */ +struct kvm_range_mapping { + struct list_head list; + void *ptr; + size_t size; + enum page_table_level level; +}; + +static LIST_HEAD(kvm_range_mapping_list); +static DEFINE_MUTEX(kvm_range_mapping_lock); + struct mm_struct kvm_mm = { .mm_rb = RB_ROOT, @@ -91,6 +105,52 @@ struct mm_struct kvm_mm = { static bool __read_mostly address_space_isolation; module_param(address_space_isolation, bool, 0444); +static struct kvm_range_mapping *kvm_get_range_mapping_locked(void *ptr, + bool *subset) +{ + struct kvm_range_mapping *range; + + list_for_each_entry(range, &kvm_range_mapping_list, list) { + if (range->ptr == ptr) { + if (subset) + *subset = false; + return range; + } + if (ptr > range->ptr && ptr < range->ptr + range->size) { + if (subset) + *subset = true; + return range; + } + } + + return NULL; +} + +static struct kvm_range_mapping *kvm_get_range_mapping(void *ptr, bool *subset) +{ + struct kvm_range_mapping *range; + + mutex_lock(&kvm_range_mapping_lock); + range = kvm_get_range_mapping_locked(ptr, subset); + mutex_unlock(&kvm_range_mapping_lock); + + return range; +} + +static void kvm_free_all_range_mapping(void) +{ + struct kvm_range_mapping *range, *range_next; + + mutex_lock(&kvm_range_mapping_lock); + + list_for_each_entry_safe(range, range_next, + &kvm_range_mapping_list, list) { + list_del(&range->list); + kfree(range); + } + + mutex_unlock(&kvm_range_mapping_lock); +} static struct pgt_directory_group *pgt_directory_group_create(void) { @@ -661,10 +721,30 @@ static int kvm_copy_mapping(void *ptr, size_t size, enum page_table_level level) { unsigned long addr = (unsigned long)ptr; unsigned long end = addr + ((unsigned long)size); + struct kvm_range_mapping *range_mapping; + bool subset; + int err; BUG_ON(current->mm == &kvm_mm); - pr_debug("KERNMAP COPY addr=%px size=%lx\n", ptr, size); - return kvm_copy_pgd_range(&kvm_mm, current->mm, addr, end, level); + pr_debug("KERNMAP COPY addr=%px size=%lx level=%d\n", ptr, size, level); + + range_mapping = kmalloc(sizeof(struct kvm_range_mapping), GFP_KERNEL); + if (!range_mapping) + return -ENOMEM; + + err = kvm_copy_pgd_range(&kvm_mm, current->mm, addr, end, level); + if (err) { + kfree(range_mapping); + return err; + } + + INIT_LIST_HEAD(&range_mapping->list); + range_mapping->ptr = ptr; + range_mapping->size = size; + range_mapping->level = level; + list_add(&range_mapping->list, &kvm_range_mapping_list); + + return 0; } @@ -720,6 +800,8 @@ static void kvm_isolation_uninit_mm(void) destroy_context(&kvm_mm); + kvm_free_all_range_mapping(); + #ifdef CONFIG_PAGE_TABLE_ISOLATION /* * With PTI, the KVM address space is defined in the user From patchwork Mon May 13 14:38:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941073 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 449561390 for ; Mon, 13 May 2019 14:40:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 320A127861 for ; Mon, 13 May 2019 14:40:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2067227E5A; Mon, 13 May 2019 14:40:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6681127861 for ; Mon, 13 May 2019 14:40:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D0C36B0271; Mon, 13 May 2019 10:39:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 95C586B0272; Mon, 13 May 2019 10:39:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ED446B0273; Mon, 13 May 2019 10:39:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f197.google.com (mail-it1-f197.google.com [209.85.166.197]) by kanga.kvack.org (Postfix) with ESMTP id 4CC256B0271 for ; Mon, 13 May 2019 10:39:38 -0400 (EDT) Received: by mail-it1-f197.google.com with SMTP id q1so12293558itc.3 for ; Mon, 13 May 2019 07:39:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=kkqkMbnz+iek9dkeudKGJuXmnkn48lWfRSXOX7tv6jE=; b=NiMKXPVbTLtNuPInDRiayZHjkO97F0TQteqQjDxiXn0yR8b6gqk+sZRwtTm5mNcAx/ R9xMkakA0q12Jc9+yVtYJi4sIRT6a6axeRXdwJk9kxvu9R6LWlQJeKsA6Yeo6ujocVBN mqjkD09sVMnG32JsdDOo13GrxQ2E5AEiMWgdJyQJGc04DkoCYI25GfwjSLN0bIrX4Z4p /sCgYCichmaSE3E7P4uJsdIX/gGx8gRp2DGQeGg3PLv2bEuNn9JWQAuUWjukKp4BN2Ul smgXUbPRXTRs7+0JXgw27v6juAt2uTpr+W6v21zWs6IHBIu82bMDcLdqGV4YKU+gn7gm UYEQ== X-Gm-Message-State: APjAAAUHV2CpbtTj/t7qZ9O1sRIK3XrZuOc8YQt8ectvbuKGt2ERviie bvz+1qfynzSPIv6vWFvkdusNN5bOBvSJ+7wb58XcZCqBbcvkywYRYKUlErbLFSUv+4kyHZRXWpE 859Jq7QdkY47fX6aZMGyK18ueME80lL0CCsyUWMrsXOxLvOA4o2QTXgMoACp6bLMtNA== X-Received: by 2002:a24:5302:: with SMTP id n2mr1594214itb.27.1557758378018; Mon, 13 May 2019 07:39:38 -0700 (PDT) X-Google-Smtp-Source: APXvYqzmlGIflOsNIb1t35hwc1/JYO1dK1Z0yk7pUSYV3x9O1HXdG6vmERvtN8HzShv5RT7U4/GF X-Received: by 2002:a24:5302:: with SMTP id n2mr1594141itb.27.1557758377062; Mon, 13 May 2019 07:39:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758377; cv=none; d=google.com; s=arc-20160816; b=Eu0MRpn5ejIXt1k00OgZix2wojIQxfx8O1qHdS7wWvjdUo2AU4+DdeznctTuwZJwAG cNfbbqAKwDZarPuJz5Vlno13sIvjsyVLNmMUVExUfNs1flA9ijJkvHBdq6Foej9zKNpU m+IumeCU8GPmggsBoOXP8RxY67DIFlciN4KvN6ieOp8FYAGACwK0a/AP/nZTiX8iXapt A3U0qHCMfArSHKXClfE4qBa2Bv3niwNzhXCHPcf/LkJGAdV/6TD4qrz2VBm8hql4blRt pMDvHQERagv+M2N0sVzmH0LjXTRpxjsCr5V4l50HWmCjzl7rnQZFna9q1OaMnjJQtGIs kx8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kkqkMbnz+iek9dkeudKGJuXmnkn48lWfRSXOX7tv6jE=; b=uAXk1kwJG4YxTWDihseU5FstclDdTCN8i93BYN39+xlomAVtsZe14YrTurSRLrsoWX xwK3OCOqKwGNRPEZD2EZ7DRkAxf0Tr4vSXEV9GeJoACneyGJVN/OjrS8/iAD0kt6Cqwp DAO3a0njldV3/X4xtrhVoHnH5mA2l1wqt0woUI/FScn8fUonFHwgiIV4nZh8ssmXp9yp JzdCALgyeIr37SwbwpLMsvLmFlyFqZ0A3Z8y+0ZMJWCR2S+pB8rMBpY/NiFNQYclWPvx 7+/milrRnCB2+otvsTeAQ9Z34IhW6fC8P8wXuHSwt89WKQhuOKk1uWL/mIQQX2FmJAtn TP7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=2ar9gUe8; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id f63si8070581itg.124.2019.05.13.07.39.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:37 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=2ar9gUe8; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2tv193095; Mon, 13 May 2019 14:39:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=kkqkMbnz+iek9dkeudKGJuXmnkn48lWfRSXOX7tv6jE=; b=2ar9gUe8PEkk2N7E5R2hXBN+S8EK1bK+MAG3miC+Uo2QLiQpRtXqL81wyT3WFUlvaGIV yFj+PnJKxx3M+EJiG2N5Wy2Ca0GxXFOvdr9H+PUOpN9ibTPdSwSw+PUxnSclUc/j8iqx wwwphVvx1m+Qjgrj3A5zXZqSIA9u83IyoHXzPJ4+e91MJFKjD0UNIlFLxXBq84f1OvIs RoXbf7KaYRZRzbEmfMM66PYK9PkQLGNLAw9R0GErJlNjc4FdSPRQilul0tfbzsHbejxk biccPs2shvHGp4MSsXlJ7ox721gE8q5sCTOHoWye2NivkamyCR56V4G1+yL8MEIsUd/r kQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfkxs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:28 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQJ022780; Mon, 13 May 2019 14:39:25 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 16/27] kvm/isolation: functions to clear page table entries for a VA range Date: Mon, 13 May 2019 16:38:24 +0200 Message-Id: <1557758315-12667-17-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP These functions will be used to unmapped memory from the KVM address space. When clearing mapping in the KVM page table, check that the clearing effectively happens in the KVM page table and there is no crossing of the KVM page table boundary (with references to the kernel page table), so that the kernel page table isn't mistakenly modified. Information (address, size, page table level) about address ranges mapped to the KVM page table is tracked, so mapping clearing is done with just specified the start address of the range. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 172 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 1 + 2 files changed, 173 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index c8358a9..e494a15 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -758,6 +758,178 @@ int kvm_copy_ptes(void *ptr, unsigned long size) } EXPORT_SYMBOL(kvm_copy_ptes); +static void kvm_clear_pte_range(pmd_t *pmd, unsigned long addr, + unsigned long end) +{ + pte_t *pte; + + pte = kvm_pte_offset(pmd, addr); + if (IS_ERR(pte)) { + pr_debug("PTE not found, skip clearing\n"); + return; + } + do { + pr_debug("PTE: %lx/%lx clear[%lx]\n", addr, end, (long)pte); + pte_clear(NULL, addr, pte); + } while (pte++, addr += PAGE_SIZE, addr < end); +} + +static void kvm_clear_pmd_range(pud_t *pud, unsigned long addr, + unsigned long end, enum page_table_level level) +{ + pmd_t *pmd; + unsigned long next; + + pmd = kvm_pmd_offset(pud, addr); + if (IS_ERR(pmd)) { + pr_debug("PMD not found, skip clearing\n"); + return; + } + do { + next = pmd_addr_end(addr, end); + if (pmd_none(*pmd)) + continue; + BUG_ON(!pmd_present(*pmd)); + if (level == PGT_LEVEL_PMD || pmd_trans_huge(*pmd) || + pmd_devmap(*pmd)) { + pr_debug("PMD: %lx/%lx clear[%lx]\n", + addr, end, (long)pmd); + pmd_clear(pmd); + continue; + } + kvm_clear_pte_range(pmd, addr, next); + } while (pmd++, addr = next, addr < end); +} + +static void kvm_clear_pud_range(p4d_t *p4d, unsigned long addr, + unsigned long end, enum page_table_level level) +{ + pud_t *pud; + unsigned long next; + + pud = kvm_pud_offset(p4d, addr); + if (IS_ERR(pud)) { + pr_debug("PUD not found, skip clearing\n"); + return; + } + do { + next = pud_addr_end(addr, end); + if (pud_none(*pud)) + continue; + if (level == PGT_LEVEL_PUD || pud_trans_huge(*pud) || + pud_devmap(*pud)) { + pr_debug("PUD: %lx/%lx clear[%lx]\n", + addr, end, (long)pud); + pud_clear(pud); + continue; + } + kvm_clear_pmd_range(pud, addr, next, level); + } while (pud++, addr = next, addr < end); +} + +static void kvm_clear_p4d_range(pgd_t *pgd, unsigned long addr, + unsigned long end, enum page_table_level level) +{ + p4d_t *p4d; + unsigned long next; + + p4d = kvm_p4d_offset(pgd, addr); + if (IS_ERR(p4d)) { + pr_debug("P4D not found, skip clearing\n"); + return; + } + + do { + next = p4d_addr_end(addr, end); + if (p4d_none(*p4d)) + continue; + if (level == PGT_LEVEL_P4D) { + pr_debug("P4D: %lx/%lx clear[%lx]\n", + addr, end, (long)p4d); + p4d_clear(p4d); + continue; + } + kvm_clear_pud_range(p4d, addr, next, level); + } while (p4d++, addr = next, addr < end); +} + +static void kvm_clear_pgd_range(struct mm_struct *mm, unsigned long addr, + unsigned long end, enum page_table_level level) +{ + pgd_t *pgd; + unsigned long next; + + pgd = pgd_offset(mm, addr); + do { + next = pgd_addr_end(addr, end); + if (pgd_none(*pgd)) + continue; + if (level == PGT_LEVEL_PGD) { + pr_debug("PGD: %lx/%lx clear[%lx]\n", + addr, end, (long)pgd); + pgd_clear(pgd); + continue; + } + kvm_clear_p4d_range(pgd, addr, next, level); + } while (pgd++, addr = next, addr < end); +} + +/* + * Clear page table entries in the KVM page table. The level parameter + * specifies the page table level (PGD, P4D, PUD PMD, PTE) at which the + * clear should be done. + * + * WARNING: The KVM page table can have direct references to the kernel + * page table, at different levels (PGD, P4D, PUD, PMD). When clearing + * such references, if the level is incorrect (for example, clear at the + * PTE level while the mapping was done at PMD level), then the clearing + * will occur in the kernel page table and the system will likely crash + * on an unhandled page fault. + */ +static void kvm_clear_mapping(void *ptr, size_t size, + enum page_table_level level) +{ + unsigned long start = (unsigned long)ptr; + unsigned long end = start + ((unsigned long)size); + + pr_debug("CLEAR %px, %lx [%lx,%lx], level=%d\n", + ptr, size, start, end, level); + kvm_clear_pgd_range(&kvm_mm, start, end, level); +} + +/* + * Clear a range mapping in the KVM page table. + */ +void kvm_clear_range_mapping(void *ptr) +{ + struct kvm_range_mapping *range_mapping; + bool subset; + + mutex_lock(&kvm_range_mapping_lock); + + range_mapping = kvm_get_range_mapping_locked(ptr, &subset); + if (!range_mapping) { + mutex_unlock(&kvm_range_mapping_lock); + pr_debug("CLEAR %px - range not found\n", ptr); + return; + } + if (subset) { + mutex_unlock(&kvm_range_mapping_lock); + pr_debug("CLEAR %px - ignored, subset of %px/%lx/%d\n", + ptr, range_mapping->ptr, range_mapping->size, + range_mapping->level); + return; + } + + kvm_clear_mapping(range_mapping->ptr, range_mapping->size, + range_mapping->level); + list_del(&range_mapping->list); + mutex_unlock(&kvm_range_mapping_lock); + + kfree(range_mapping); +} +EXPORT_SYMBOL(kvm_clear_range_mapping); + static int kvm_isolation_init_mm(void) { diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index e8c018a..7d3c985 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -17,5 +17,6 @@ static inline bool kvm_isolation(void) extern void kvm_isolation_exit(void); extern void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu); extern int kvm_copy_ptes(void *ptr, unsigned long size); +extern void kvm_clear_range_mapping(void *ptr); #endif From patchwork Mon May 13 14:38:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941077 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4FE5B1390 for ; Mon, 13 May 2019 14:40:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4192D27861 for ; Mon, 13 May 2019 14:40:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3485427E5A; Mon, 13 May 2019 14:40:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A294127BA5 for ; Mon, 13 May 2019 14:40:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89A576B0272; Mon, 13 May 2019 10:39:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8255F6B0273; Mon, 13 May 2019 10:39:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64F906B0274; Mon, 13 May 2019 10:39:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by kanga.kvack.org (Postfix) with ESMTP id 3EA9A6B0272 for ; Mon, 13 May 2019 10:39:41 -0400 (EDT) Received: by mail-io1-f70.google.com with SMTP id e126so10001391ioa.8 for ; Mon, 13 May 2019 07:39:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=+MaSC7D5NYyFHcoLlt21rqwn+ky+e8tS8duegev3cnQ=; b=KkgIVfnbAFe051tkHrnXaKXf457PsKzxnWgSeVM+31XRSDXtkugXHiaOPKbH3XMRuf 7AUNaa/uEMTgEnWoMxHe6KTJa5d4Jkp/X4peYYBRAfa/Um8jFgSDBtxW+J9k9388xxPZ tFZX3w/QUT7srqqGBqJOsN8zBBjY9ZXCkFYcMYXJ8OcOKH8NBdXjY7RL305VVSKtUnhu 2jznaZybx0yLilhlKr5YMy+qupFtKTCGqRnNRc0ZLFSGRhdvHUrod5XWvUDpi3Lq6xkY VErK71riFO5fJ+jN18BczGd7JD/Y2o/Y31+zNJBX5m2QSz4C0rqdwALhae6tiOdJSml7 eI1g== X-Gm-Message-State: APjAAAUKrM64rfyH5q2T1winQPRShYRUaqHnUlIRI9YvYl5C9xMEIf1L U7kSiwS7rizmWsm+ovdkdFa6J7f7oz4oJV8B2SK4yblWHeLUMlLfNe8rdp/w+LjY4eenv+0rYte kGqaRluxyxt4x+7LkCLd2mdflr+mtOuMJa5auVJNrn7+gvS6TyRqSFAhX3On5dAz4ZQ== X-Received: by 2002:a24:4d1:: with SMTP id 200mr12654563itb.92.1557758380961; Mon, 13 May 2019 07:39:40 -0700 (PDT) X-Google-Smtp-Source: APXvYqyaZLboFPbkYyt7Ssz3BnwPe4ie2rJj0JtHP/Kxi/vfR20A5on/PfVY91i7nTMDFE1ZOUSw X-Received: by 2002:a24:4d1:: with SMTP id 200mr12654510itb.92.1557758380133; Mon, 13 May 2019 07:39:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758380; cv=none; d=google.com; s=arc-20160816; b=UgB/Ey/4XBY3hBBnkt9bR/cmXzydQslMOSZdZO/uR0bO3lRSpiOwVSChIG1V92wZ04 nKod0VK3SpFsunbuOqbVwU42ZB7iWH+2SxqxSrdL7rCAT04636MrhKmYmTPGt6R5xjFm 6o8K44wGmA9ApGf8uR1ZUTL6LX5rt3gjsfBdJIetCAoZd8e46dtnilDeQ1zCu1+vm+06 Hj4huYpGRU5FU/AlnRf+VLB9SJWcj3kDu68ej90jWgC9rKrEHy4POXNbQdCYSQjAakje q+RSg1uQRJqBxDJ/E9xh8qRletnbGNQO3Fno0Gy9CWr8svMywXnwvGcLzJgz/pLYTlqb bcXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+MaSC7D5NYyFHcoLlt21rqwn+ky+e8tS8duegev3cnQ=; b=LNARPmmFElAlU0rxdecCFGlNTqtzjrHJ+6am7f6V67EBaUhVM5Pdu0jwK2Ia9SwR2h FbbMX3sY8a1k5w2UTe6Gs6V4pTbHlMGhNQ0Ina597C27flccWGpBBHHZ99uLMTXDmtGt 1A4Ogyf6VwuWyF7S0ijWFDqXws3qtGSBDSgO6MUylF+R1qpQb4H66CP5y+X+LHqLmau9 CGWNS6Sgp/79mXFLSF172SyU99uGMgzhWsRFHPyQvFPysPVFQTws9QsdFGu7fWC4cOVQ io5scaxE+FcFPiym39DFgnrhhRrOClyYwa/MFZhOd8ce30v/ZLoa9umdn276tz8qz37g M/cg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=z6slNgJ9; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id b64si8275836iti.113.2019.05.13.07.39.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:40 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=z6slNgJ9; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2Gn194925; Mon, 13 May 2019 14:39:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=+MaSC7D5NYyFHcoLlt21rqwn+ky+e8tS8duegev3cnQ=; b=z6slNgJ9MrFp8uPsBPJmJU6GEaQ8wt3ZidHa0OAjWLM8dA3+hvZO3Q2zqNmvKu+Qfvha f39YKjJiNdGzsIVZtiZmHgCAk3/RyTNp7XRWQJE/vV0AXPAhhrT+G+j8gHuyTYUQNX0f AIhcCp0PH4W5K5mBi7O15eBaUEQLLjjyWKwdghJjGta7Pwvvxw11nROudMT9WBKTJRLS bc5hFg0tiztOk+LrZZH2/SQPAOQVHBG6LvLcBcZDJcfhj/iVjW1L4APtEZtPI5sxvMGw RUUksDo71NnDOv56zatbXlgzV01aZsNu7msINVw4f0f0NjXKNnCv8fOY+KxWkWNogtPN 5Q== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7axg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:31 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQK022780; Mon, 13 May 2019 14:39:28 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 17/27] kvm/isolation: improve mapping copy when mapping is already present Date: Mon, 13 May 2019 16:38:25 +0200 Message-Id: <1557758315-12667-18-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP A mapping can already exist if a buffer was mapped in the KVM address space, and then the buffer was freed but there was no request to unmap from the KVM address space. In that case, clear the existing mapping before mapping the new buffer. Also if the new mapping is a subset of an already larger mapped range, then remap the entire larger map. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 67 +++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 63 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index e494a15..539e287 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -88,6 +88,9 @@ struct mm_struct kvm_mm = { DEFINE_STATIC_KEY_FALSE(kvm_isolation_enabled); EXPORT_SYMBOL(kvm_isolation_enabled); +static void kvm_clear_mapping(void *ptr, size_t size, + enum page_table_level level); + /* * When set to true, KVM #VMExit handlers run in isolated address space * which maps only KVM required code and per-VM information instead of @@ -721,6 +724,7 @@ static int kvm_copy_mapping(void *ptr, size_t size, enum page_table_level level) { unsigned long addr = (unsigned long)ptr; unsigned long end = addr + ((unsigned long)size); + unsigned long range_addr, range_end; struct kvm_range_mapping *range_mapping; bool subset; int err; @@ -728,22 +732,77 @@ static int kvm_copy_mapping(void *ptr, size_t size, enum page_table_level level) BUG_ON(current->mm == &kvm_mm); pr_debug("KERNMAP COPY addr=%px size=%lx level=%d\n", ptr, size, level); - range_mapping = kmalloc(sizeof(struct kvm_range_mapping), GFP_KERNEL); - if (!range_mapping) - return -ENOMEM; + mutex_lock(&kvm_range_mapping_lock); + + /* + * A mapping can already exist if the buffer was mapped and then + * freed but there was no request to unmap it. We might also be + * trying to map a subset of an already mapped buffer. + */ + range_mapping = kvm_get_range_mapping_locked(ptr, &subset); + if (range_mapping) { + if (subset) { + pr_debug("range %px/%lx/%d is a subset of %px/%lx/%d already mapped, remapping\n", + ptr, size, level, range_mapping->ptr, + range_mapping->size, range_mapping->level); + range_addr = (unsigned long)range_mapping->ptr; + range_end = range_addr + + ((unsigned long)range_mapping->size); + err = kvm_copy_pgd_range(&kvm_mm, current->mm, + range_addr, range_end, + range_mapping->level); + if (end <= range_end) { + /* + * We effectively have a subset, fully contained + * in the superset. So we are done. + */ + mutex_unlock(&kvm_range_mapping_lock); + return err; + } + /* + * The new range is larger than the existing mapped + * range. So we need an extra mapping to map the end + * of the range. + */ + addr = range_end; + range_mapping = NULL; + pr_debug("adding extra range %lx-%lx (%d)\n", addr, + end, level); + } else { + pr_debug("range %px size=%lx level=%d already mapped, clearing\n", + range_mapping->ptr, range_mapping->size, + range_mapping->level); + kvm_clear_mapping(range_mapping->ptr, + range_mapping->size, + range_mapping->level); + list_del(&range_mapping->list); + } + } + + if (!range_mapping) { + range_mapping = kmalloc(sizeof(struct kvm_range_mapping), + GFP_KERNEL); + if (!range_mapping) { + mutex_unlock(&kvm_range_mapping_lock); + return -ENOMEM; + } + INIT_LIST_HEAD(&range_mapping->list); + } err = kvm_copy_pgd_range(&kvm_mm, current->mm, addr, end, level); if (err) { + mutex_unlock(&kvm_range_mapping_lock); kfree(range_mapping); return err; } - INIT_LIST_HEAD(&range_mapping->list); range_mapping->ptr = ptr; range_mapping->size = size; range_mapping->level = level; list_add(&range_mapping->list, &kvm_range_mapping_list); + mutex_unlock(&kvm_range_mapping_lock); + return 0; } From patchwork Mon May 13 14:38:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941081 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0E706C5 for ; Mon, 13 May 2019 14:40:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A2A1A27861 for ; Mon, 13 May 2019 14:40:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9446627E5A; Mon, 13 May 2019 14:40:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2184D27861 for ; Mon, 13 May 2019 14:40:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 95B1C6B0275; Mon, 13 May 2019 10:39:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8BDBA6B0277; Mon, 13 May 2019 10:39:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C42F6B0276; Mon, 13 May 2019 10:39:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by kanga.kvack.org (Postfix) with ESMTP id 4C2E26B0274 for ; Mon, 13 May 2019 10:39:48 -0400 (EDT) Received: by mail-io1-f71.google.com with SMTP id t7so9991700iod.17 for ; Mon, 13 May 2019 07:39:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=bOd6NTuOO3w4Tse88wsGsS3lTxF7t3wBy812EA6WdBU=; b=HAqzUXcoP5AygszEcSDrHn/s34Ifxl0M53HmH66dG0JAROs/c2nCvLZUGvpSyfWpUG 7uV8UVPW6EHTRa+yo4Cc5JGOc97XFPHAgUQHJZKvUHKdW4QrhGtVrsrZMhCbBj7/euxU 6aPxXXBjpYecHyxEJSzwTqg7P2lA+XaFyAoDL1/OEO35NZ2xCkYRq3jcBM8If1S9kBKH bXWxKPyClU+7d9VN0qz7vF2B7pUB3QCoNLaRVVEhj+QxozC2zFAfjpafVLV+7S70DgU0 CykNZr6CHR66kQDXHc2aDVYJYxDkXdG2zJ6RRQ9L4QQgoK8g/UseYxYyuO7xrdQ+yPKh oyig== X-Gm-Message-State: APjAAAVYxT4T/BwM3syelq3V4OwiYHCBzz2QMfRTl2h21JzKvpMPhNfF zNNVGV7CVGWcGFmo1JXZU8fhnO52iVXaoXg3P3aYupwhDsmiFUZhs3EdlrJjjiWydK+6cnaWaIT T2NFD7wGUKQj8CIdqBPxeYFNfjS7nny7GbQrSjhy/jnerMw+GZ15l5WPZEacZrmB8zw== X-Received: by 2002:a6b:6f08:: with SMTP id k8mr11352559ioc.104.1557758388065; Mon, 13 May 2019 07:39:48 -0700 (PDT) X-Google-Smtp-Source: APXvYqwacQS30NO8TdbhchRhWCO7+AAeD7BJizMLV0/DWnu9RBhwPaxBIluZo1GkJtF4dlfGzSt5 X-Received: by 2002:a6b:6f08:: with SMTP id k8mr11352525ioc.104.1557758387409; Mon, 13 May 2019 07:39:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758387; cv=none; d=google.com; s=arc-20160816; b=b7STVkggKOGNCA+g4wyDWRdooseMglbPM6an6J6QCqn+0Sz0mzkfaRzKYHfSg+PQ9x ny0RKTAX1sOGeY6/t1VmxcPoTcLTbN5kG18g7M7TtPDmQ4ChLG8BSByqeUjVgarcEUo8 ZCJkFCnhgrrb/ZeOMWC9EzbUh8gwlwNgJKHfoNTCtNnVCJDb9e9C9+STqoMNKrJgPcO/ 0JLSHMLCHL2IdXJy33LdW5sqIb/w9ci5JSl1msFXAqytrlMRn0Xz1zqDCw8p0jdsKMYn de/7DTY97KOMdTQ4V7epIyqoNEKwrOWfT1osc5BoJIMIy42mteMVGfDdOQW2TrLAio+T 6dJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bOd6NTuOO3w4Tse88wsGsS3lTxF7t3wBy812EA6WdBU=; b=Hhxq/4S5Ev/uLjCSDALTQeP64BHWsDpAs1TXDfnQ1oF8pxqyjdTGn+MwVA2v5f6Y4d sfEgawjP6C4FCLtifDp+nTXUqS0LWVR+tOdcww2ZCelmOKfaFXMrVd/G5xn5dIwBeOjk Doo+VAUgRpfITTUmj8glSeOnXIz/vT17VRMa3b3zUZargyVVl+s5B4SfjY+P0oHg3COv pdP8iTNsHFCtu/IUJAR8zrYsBgG+7K201rsEEs3pw5zo4cufQg846yk23z/5zEQr2tpV wKRhglH1eKxPCg2ihuttS50O3Rei5dFFqqZK6f8BD+TEHFNewpKdvZnHq0jxnJv6mmfF M9lw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=Y9nXfUJN; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2130.oracle.com (userp2130.oracle.com. [156.151.31.86]) by mx.google.com with ESMTPS id x197si3224008itb.72.2019.05.13.07.39.47 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:47 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) client-ip=156.151.31.86; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=Y9nXfUJN; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd3dB181544; Mon, 13 May 2019 14:39:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=bOd6NTuOO3w4Tse88wsGsS3lTxF7t3wBy812EA6WdBU=; b=Y9nXfUJNbdNXiy0WXZfnXcvJMfJnbW6Wd/jhOsfaH+Nku6LO73tegfk4kUXz6qGARmee jxG1wNCbZiHDX8SNNrTewc3cLIlDZsttlVSwGklecgW/HROVU+I3mtazS3Yvqh11anbo Zv4jHKjLPjfduxRLu4X8LAzZ3P36HyfHmP1W/DyLiXegpYJeMKnqKZFwITYW5qzlSHm5 RtesqMSGG/6fDOLSNgWvXF9bfkYj0YrLp3UqSNuvKowRK44FvGuaRvjBi3RpZ0NPhj/l QmdYQ9IaU3TVKzTuW8I6wo8hxvktW9E/r6qI9QWOuBYsblTK3/WgBv+O9a9uDwsYhX/x cg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2sdnttfeja-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:39 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQL022780; Mon, 13 May 2019 14:39:31 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 18/27] kvm/isolation: function to copy page table entries for percpu buffer Date: Mon, 13 May 2019 16:38:26 +0200 Message-Id: <1557758315-12667-19-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP pcpu_base_addr is already mapped to the KVM address space, but this represents the first percpu chunk. To access a per-cpu buffer not allocated in the first chunk, add a function which maps all cpu buffers corresponding to that per-cpu buffer. Also add function to clear page table entries for a percpu buffer. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 34 ++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 2 ++ 2 files changed, 36 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 539e287..2052abf 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -990,6 +990,40 @@ void kvm_clear_range_mapping(void *ptr) EXPORT_SYMBOL(kvm_clear_range_mapping); +void kvm_clear_percpu_mapping(void *percpu_ptr) +{ + void *ptr; + int cpu; + + pr_debug("PERCPU CLEAR percpu=%px\n", percpu_ptr); + for_each_possible_cpu(cpu) { + ptr = per_cpu_ptr(percpu_ptr, cpu); + kvm_clear_range_mapping(ptr); + } +} +EXPORT_SYMBOL(kvm_clear_percpu_mapping); + +int kvm_copy_percpu_mapping(void *percpu_ptr, size_t size) +{ + void *ptr; + int cpu, err; + + pr_debug("PERCPU COPY percpu=%px size=%lx\n", percpu_ptr, size); + for_each_possible_cpu(cpu) { + ptr = per_cpu_ptr(percpu_ptr, cpu); + pr_debug("PERCPU COPY cpu%d addr=%px\n", cpu, ptr); + err = kvm_copy_ptes(ptr, size); + if (err) { + kvm_clear_range_mapping(percpu_ptr); + return err; + } + } + + return 0; +} +EXPORT_SYMBOL(kvm_copy_percpu_mapping); + + static int kvm_isolation_init_mm(void) { pgd_t *kvm_pgd; diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 7d3c985..3ef2060 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -18,5 +18,7 @@ static inline bool kvm_isolation(void) extern void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu); extern int kvm_copy_ptes(void *ptr, unsigned long size); extern void kvm_clear_range_mapping(void *ptr); +extern int kvm_copy_percpu_mapping(void *percpu_ptr, size_t size); +extern void kvm_clear_percpu_mapping(void *percpu_ptr); #endif From patchwork Mon May 13 14:38:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941079 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A58316C5 for ; Mon, 13 May 2019 14:40:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96A8927BA5 for ; Mon, 13 May 2019 14:40:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8A3AB27FA1; Mon, 13 May 2019 14:40:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C705A27BA5 for ; Mon, 13 May 2019 14:40:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 822FC6B0273; Mon, 13 May 2019 10:39:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7D50C6B0274; Mon, 13 May 2019 10:39:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 626906B0275; Mon, 13 May 2019 10:39:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) by kanga.kvack.org (Postfix) with ESMTP id 3ABD96B0273 for ; Mon, 13 May 2019 10:39:47 -0400 (EDT) Received: by mail-io1-f69.google.com with SMTP id h189so10041930ioa.13 for ; Mon, 13 May 2019 07:39:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=l5aKMAk3Fh+gtpNvYGvrecFqK8ir23pbpb3HmvcDshc=; b=iWw4x9LAEYYPqYI8++QiuoGNSi8rtFuqXyctEJ3urSw8Wf66aNV8fsRSFrSZBnweTM sYKo4ZmxRxoloUBihkPsbZ218pZWTY0T86e/FJbDyfbfRcPzBudGjjwIP+aiU2ZaV73C 6tACvDSvcj5GKiYCgFvxQ+468h/Rp2n2157Htzsmkv226bgGPRQFUC2GKInR6Ak24aFi BfYdaQVfACtMS4nrheE2k9A1NBUyU5WTMhx/KnbkMlYkAxTFJES7mvtJJqbhQUzP/r5q icj0692Ss8fuqS3LMJHT63rtk+ek+3pukESXX0Gj9BfkN+PumWsUTfrLkulH0EmzWMda IFfw== X-Gm-Message-State: APjAAAUX4YkKbYZhxVZClBvFsTkiMwYA1ANAphUuLm6e0HB6TpnnUwwx rRwn2kysLpD58k2tGkcn/uN7i0vsNz5OxfCDogZ554ORnZPVP7E9cxt1+wWCDa4XWAN4IRKW0ho TwW0lYSoZ1JaN416iRDqyQjGgvhy8iIIAJlpzLkmxKiS88Ht+mCLo43/SPBgUCQhOXQ== X-Received: by 2002:a24:edcb:: with SMTP id r194mr18444148ith.164.1557758386965; Mon, 13 May 2019 07:39:46 -0700 (PDT) X-Google-Smtp-Source: APXvYqygEQrhf27FYMiIZFm4DqTBSODe3d70r6ad5hPNBj55bZKU/VyilK8D+eiWp7lHYrN0+eYM X-Received: by 2002:a24:edcb:: with SMTP id r194mr18444086ith.164.1557758385964; Mon, 13 May 2019 07:39:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758385; cv=none; d=google.com; s=arc-20160816; b=HewYbC9+iD4iKtC6Rqv5ppvgbrPFOkOzEVzNZix3HOr3PbDZFbUX1HHq5+QZBhuTRk i1vRDYDvLzP9CjYD3s7fqtf6DJ3xT+pM1YHee0lxEKyRIK5Zw25Onna4ylxQiOViUjLc Oof68FyhHW54FaS5iBh7Oxe3YKwKhC6RQlNr7pWA55r1qm5gSjnXDrZZFGl9yE4NKnRw 2WiHkLpr74v1esrDu7wc0XPfoWbgP1NS3R75seUhfeWHdVVcnvcUXwAF2ETSEXdmRbU5 Y1Q6P3wniphUji838j4iEgrohEBm2NhMqeqO1L3so2mER7tOsGWS7MJr0pRkNVMFwn3s 9UMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=l5aKMAk3Fh+gtpNvYGvrecFqK8ir23pbpb3HmvcDshc=; b=tLpRFpr7b86rPV03yZQ5puQjOeIuyRoVhPOMCD8wR+VmuD793Z8dZi8DvaYAZi+eqK Vf3OB0hTLcZ6CU+KWE6OXm89jhCiAsiVmTbfDr6IjRnWk7gh7ZPI+B+Tr9U3fc+TcVGS j8NEICas0uCPfChezGIg8nWGSaVVE1GDT3Y016UMH/h5RCRDiZYUDaSkEX9fXIjna7xV KY/YiID0DD9XD7nzLjIcGxLHKCirBnuZi8tbwiq4wksh0tXCKdZmYYdu+NmkxKL0kZWR iWWvkYjQ6w72OV6qK0QHb0xwtEfTTMmIQhcek21DlFwEoNCJ86wqOKAAG35p6OtGH3W0 NoDQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=NPL0tURB; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id q129si7546890jaq.37.2019.05.13.07.39.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:45 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=NPL0tURB; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd28I193025; Mon, 13 May 2019 14:39:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=l5aKMAk3Fh+gtpNvYGvrecFqK8ir23pbpb3HmvcDshc=; b=NPL0tURBc+BNNRYWXut9QJPxP0shoD0HzAqoBR5E6EV0J0revf1EigBcQsuJWtb30CCC upBrgbB61tjujiQwsK2QAD4JDlF2ymOmSTZCeesv44XBs0QlN2E2lJ976YC4UOUGYYBG yj4HV+wTXtHjArhgs933THSajLYTjqu3Pm7eFTZXAp/pY1YUghyA/Rlag0ritPGg8VZh 0vV3GFARwn18IkWwCCTC4TZudBcx6LFynGTBEWs9PS6Pa3VOJP3RXhwmMCToyY+MwubI lgNu9vsCkFz4pSpo1VcWzLZNKsLSUinM/ps8GUa/cBmdF5gC/ozgE5MprNsOpOKW+J8h TA== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfkye-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:36 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQM022780; Mon, 13 May 2019 14:39:33 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 19/27] kvm/isolation: initialize the KVM page table with core mappings Date: Mon, 13 May 2019 16:38:27 +0200 Message-Id: <1557758315-12667-20-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The KVM page table is initialized with adding core memory mappings: the kernel text, the per-cpu memory, the kvm module, the cpu_entry_area, %esp fixup stacks, IRQ stacks. Signed-off-by: Alexandre Chartre --- arch/x86/kernel/cpu/common.c | 2 + arch/x86/kvm/isolation.c | 131 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 10 +++ include/linux/percpu.h | 2 + mm/percpu.c | 6 +- 5 files changed, 149 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 3764054..0fa44b1 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -1511,6 +1511,8 @@ static __init int setup_clearcpuid(char *arg) EXPORT_PER_CPU_SYMBOL(current_task); DEFINE_PER_CPU(struct irq_stack *, hardirq_stack_ptr); +EXPORT_PER_CPU_SYMBOL_GPL(hardirq_stack_ptr); + DEFINE_PER_CPU(unsigned int, irq_count) __visible = -1; DEFINE_PER_CPU(int, __preempt_count) = INIT_PREEMPT_COUNT; diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 2052abf..cf5ee0d 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -10,6 +10,8 @@ #include #include +#include +#include #include #include @@ -88,6 +90,8 @@ struct mm_struct kvm_mm = { DEFINE_STATIC_KEY_FALSE(kvm_isolation_enabled); EXPORT_SYMBOL(kvm_isolation_enabled); +static void kvm_isolation_uninit_page_table(void); +static void kvm_isolation_uninit_mm(void); static void kvm_clear_mapping(void *ptr, size_t size, enum page_table_level level); @@ -1024,10 +1028,130 @@ int kvm_copy_percpu_mapping(void *percpu_ptr, size_t size) EXPORT_SYMBOL(kvm_copy_percpu_mapping); +static int kvm_isolation_init_page_table(void) +{ + void *stack; + int cpu, rv; + + /* + * Copy the mapping for all the kernel text. We copy at the PMD + * level since the PUD is shared with the module mapping space. + */ + rv = kvm_copy_mapping((void *)__START_KERNEL_map, KERNEL_IMAGE_SIZE, + PGT_LEVEL_PMD); + if (rv) + goto out_uninit_page_table; + + /* copy the mapping of per cpu memory */ + rv = kvm_copy_mapping(pcpu_base_addr, pcpu_unit_size * pcpu_nr_units, + PGT_LEVEL_PMD); + if (rv) + goto out_uninit_page_table; + + /* + * Copy the mapping for cpu_entry_area and %esp fixup stacks + * (this is based on the PTI userland address space, but probably + * not needed because the KVM address space is not directly + * enterered from userspace). They can both be copied at the P4D + * level since they each have a dedicated P4D entry. + */ + rv = kvm_copy_mapping((void *)CPU_ENTRY_AREA_PER_CPU, P4D_SIZE, + PGT_LEVEL_P4D); + if (rv) + goto out_uninit_page_table; + +#ifdef CONFIG_X86_ESPFIX64 + rv = kvm_copy_mapping((void *)ESPFIX_BASE_ADDR, P4D_SIZE, + PGT_LEVEL_P4D); + if (rv) + goto out_uninit_page_table; +#endif + +#ifdef CONFIG_VMAP_STACK + /* + * Interrupt stacks are vmap'ed with guard pages, so we need to + * copy mappings. + */ + for_each_possible_cpu(cpu) { + stack = per_cpu(hardirq_stack_ptr, cpu); + pr_debug("IRQ Stack %px\n", stack); + if (!stack) + continue; + rv = kvm_copy_ptes(stack - IRQ_STACK_SIZE, IRQ_STACK_SIZE); + if (rv) + goto out_uninit_page_table; + } + +#endif + + /* copy mapping of the current module (kvm) */ + rv = kvm_copy_module_mapping(); + if (rv) + goto out_uninit_page_table; + + return 0; + +out_uninit_page_table: + kvm_isolation_uninit_page_table(); + return rv; +} + +/* + * Free all buffers used by the kvm page table. These buffers are stored + * in the kvm_pgt_dgroup_list. + */ +static void kvm_isolation_uninit_page_table(void) +{ + struct pgt_directory_group *dgroup, *dgroup_next; + enum page_table_level level; + void *ptr; + int i; + + mutex_lock(&kvm_pgt_dgroup_lock); + + list_for_each_entry_safe(dgroup, dgroup_next, + &kvm_pgt_dgroup_list, list) { + + for (i = 0; i < dgroup->count; i++) { + ptr = dgroup->directory[i].ptr; + level = dgroup->directory[i].level; + + switch (dgroup->directory[i].level) { + + case PGT_LEVEL_PTE: + kvm_pte_free(NULL, ptr); + break; + + case PGT_LEVEL_PMD: + kvm_pmd_free(NULL, ptr); + break; + + case PGT_LEVEL_PUD: + kvm_pud_free(NULL, ptr); + break; + + case PGT_LEVEL_P4D: + kvm_p4d_free(NULL, ptr); + break; + + default: + pr_err("unexpected page directory %d for %px\n", + level, ptr); + } + } + + list_del(&dgroup->list); + kfree(dgroup); + } + + mutex_unlock(&kvm_pgt_dgroup_lock); +} + static int kvm_isolation_init_mm(void) { pgd_t *kvm_pgd; gfp_t gfp_mask; + int rv; gfp_mask = GFP_KERNEL | __GFP_ZERO; kvm_pgd = (pgd_t *)__get_free_pages(gfp_mask, PGD_ALLOCATION_ORDER); @@ -1054,6 +1178,12 @@ static int kvm_isolation_init_mm(void) mm_init_cpumask(&kvm_mm); init_new_context(NULL, &kvm_mm); + rv = kvm_isolation_init_page_table(); + if (rv) { + kvm_isolation_uninit_mm(); + return rv; + } + return 0; } @@ -1065,6 +1195,7 @@ static void kvm_isolation_uninit_mm(void) destroy_context(&kvm_mm); + kvm_isolation_uninit_page_table(); kvm_free_all_range_mapping(); #ifdef CONFIG_PAGE_TABLE_ISOLATION diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 3ef2060..1f79e28 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -3,6 +3,16 @@ #define ARCH_X86_KVM_ISOLATION_H #include +#include + +/* + * Copy the memory mapping for the current module. This is defined as a + * macro to ensure it is expanded in the module making the call so that + * THIS_MODULE has the correct value. + */ +#define kvm_copy_module_mapping() \ + (kvm_copy_ptes(THIS_MODULE->core_layout.base, \ + THIS_MODULE->core_layout.size)) DECLARE_STATIC_KEY_FALSE(kvm_isolation_enabled); diff --git a/include/linux/percpu.h b/include/linux/percpu.h index 70b7123..fb0ab9a 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -70,6 +70,8 @@ extern void *pcpu_base_addr; extern const unsigned long *pcpu_unit_offsets; +extern int pcpu_unit_size; +extern int pcpu_nr_units; struct pcpu_group_info { int nr_units; /* aligned # of units */ diff --git a/mm/percpu.c b/mm/percpu.c index 68dd2e7..b68b3d8 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -119,8 +119,10 @@ #endif /* CONFIG_SMP */ static int pcpu_unit_pages __ro_after_init; -static int pcpu_unit_size __ro_after_init; -static int pcpu_nr_units __ro_after_init; +int pcpu_unit_size __ro_after_init; +EXPORT_SYMBOL(pcpu_unit_size); +int pcpu_nr_units __ro_after_init; +EXPORT_SYMBOL(pcpu_nr_units); static int pcpu_atom_size __ro_after_init; int pcpu_nr_slots __ro_after_init; static size_t pcpu_chunk_struct_size __ro_after_init; From patchwork Mon May 13 14:38:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941087 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF83D1390 for ; Mon, 13 May 2019 14:40:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A156B27FAE for ; Mon, 13 May 2019 14:40:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93C6427BA5; Mon, 13 May 2019 14:40:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2596727BA5 for ; Mon, 13 May 2019 14:40:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1469C6B0278; Mon, 13 May 2019 10:39:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0AA816B027A; Mon, 13 May 2019 10:39:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB3166B027B; Mon, 13 May 2019 10:39:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f200.google.com (mail-it1-f200.google.com [209.85.166.200]) by kanga.kvack.org (Postfix) with ESMTP id C9FD36B0278 for ; Mon, 13 May 2019 10:39:53 -0400 (EDT) Received: by mail-it1-f200.google.com with SMTP id d12so12370735itl.5 for ; Mon, 13 May 2019 07:39:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=gJ5tt/4IYtOQ64SjXc4XK/hg4MlEde9v0jqeQXYD/eQ=; b=XXG99HjVjAHO3dlGwAOaiMdnrIzcPi4ZGAH94ZWWE86bNav75GOqln8etclmAsaGqO YLvP5A5afZOIs/R2pEma0Pov/Wo0EJrEd/Tdm4LyP76qrCtP/ltQsgmo9Hfm7f0OIb2U OiFsQ9J/Z5ggVRwnm+CDfFzp4FXJxMFE2XvrrxLrn8oQGn1vuTSrPNJfqnnMQ+6YmVJD UbE7/pJ2UT/EE7Pfvl3mjNcqp/702mL2B6MXcE2fBAoOZepmrf/2Cc2GMKC1kWKyY+67 udq3D/gc0P/e76v11nWTdDCf/rm8qec8zSl9BRyPmSOuK0NEzf7zcNNoWnku15zJ1oii /Bgw== X-Gm-Message-State: APjAAAUrAyS7yzyw/voGund3KfW2+JyRicWs/yR3vXAhF8KgHdDlqklT kCxG0R1bTl2sr73i/eKWkINPYp5oKDwHSLuWwapgiM3L3Qtz6RN5sX7pb/vgaW1GPjlJNgsx7Gs XF3iwjQep26KA1/iZnBUVAtXwy69iCKXpEZ9sbPZAFcAUyhP5kulF25ZRJSuHeFO2Og== X-Received: by 2002:a24:a088:: with SMTP id o130mr10509551ite.86.1557758393561; Mon, 13 May 2019 07:39:53 -0700 (PDT) X-Google-Smtp-Source: APXvYqyjRB4QKtcyrveHdDov/7YcDVsgCnvCWUtbj692gEbcgpAngW4hr3V6mvRKZH8wpDNf3iUY X-Received: by 2002:a24:a088:: with SMTP id o130mr10509498ite.86.1557758392854; Mon, 13 May 2019 07:39:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758392; cv=none; d=google.com; s=arc-20160816; b=vWqz/xI4NE2+Wt2deyI3aYeh+BW8zJMFnCrhh5FX1285VcinvaBymS/Zw3/fUyaiRm QGTra8WP/Lye9NZncs6CbkPlqhIDB4t8UZ3pk5hFvP47nQTt2ulMTcRlZwMU1y2QrbpN gb5w3o5YHUJT3NBBNUD7a8aHKF3GVYgksaBPH6zMgnAxC9FjRnCIgcKCEHWi+c6peFgg KjhIspGWUFYo99qc+OG8qf1T9pFo7PEdlgwv+3SOaLdH+r72P9OhDDuM8xQUC89CGm8g aM6Sqo6EDMGCVgBSwsynFTqEDMtZoc3Zb3MkCJuOIno0cZOJ7nmunOBtojsf+UiUzLAw jajg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=gJ5tt/4IYtOQ64SjXc4XK/hg4MlEde9v0jqeQXYD/eQ=; b=pjcJ+Q6RGZ6VSQR5xoMl966sTiUjnohJsmZmNDbVpQJ6OArL4+0n3ajvH6zyTXyFvj INYizz3GlPDE6CVjBY+e9/GT9E3/2sZJWWrZuMHFkuTzl1zlQBnnHzv5XiYlv3faHQ/c 5vdP9DqD+DAO+SwGUvjOpO0BfNwcsT3MN1U+qtTU4dJc9DJOxvHgqyEB6VKYlAQ8M1gx ZzIYQvXebsKIgkMYtJUQ+MQLwvbr8lU7UzPhLYB3JksoHvJQco+jmvb82QWv/IhSx8LM OS5UErGmXnub7A/q1IzhjaUvUc0768RtIUsWcX99jfKW3jKjbbJK72hRbBBPA2yvJmNQ TgVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=ZVeO12rO; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id 80si8588911itl.77.2019.05.13.07.39.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:52 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=ZVeO12rO; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEdHlu193231; Mon, 13 May 2019 14:39:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=gJ5tt/4IYtOQ64SjXc4XK/hg4MlEde9v0jqeQXYD/eQ=; b=ZVeO12rORSnjNn30oKP4IfrRQOnpuWsZZpTFew29AW/A1AopB4vNTvJPzoBUNnLwecR5 RxfhLqEFwtp98NkffYHLWW/xGFZAMdCGFC1Cm5FhGyQSMCaCSB6w20I2qahvkzKCsFa5 m/f/wbDM5y5zJuEnz5wG8g9nfWjmPhU8RB2bXp/ASv1kj1gRX0Hrz8Moy8R160Qf+K6H gpnLK5eK57XeoRIkkZcloR/MsoTBMRliLnztAGJlyy4fIbv6fkl19QVpaSKUSvXbYn60 PuCO41ERbdVUgxGByWPWdcyJSJL7JyT3ve6bcXI+B2FmQnOupQrAXYuphyQi5YY9ctn+ Fg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfm05-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:44 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQN022780; Mon, 13 May 2019 14:39:36 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 20/27] kvm/isolation: initialize the KVM page table with vmx specific data Date: Mon, 13 May 2019 16:38:28 +0200 Message-Id: <1557758315-12667-21-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In addition of core memory mappings, the KVM page table has to be initialized with vmx specific data. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/vmx/vmx.c | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 0c955bb..f181b3c 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -63,6 +63,7 @@ #include "vmcs12.h" #include "vmx.h" #include "x86.h" +#include "isolation.h" MODULE_AUTHOR("Qumranet"); MODULE_LICENSE("GPL"); @@ -7830,6 +7831,24 @@ static int __init vmx_init(void) } } + if (kvm_isolation()) { + pr_debug("mapping vmx init"); + /* copy mapping of the current module (kvm_intel) */ + r = kvm_copy_module_mapping(); + if (r) { + vmx_exit(); + return r; + } + if (vmx_l1d_flush_pages) { + r = kvm_copy_ptes(vmx_l1d_flush_pages, + PAGE_SIZE << L1D_CACHE_ORDER); + if (r) { + vmx_exit(); + return r; + } + } + } + #ifdef CONFIG_KEXEC_CORE rcu_assign_pointer(crash_vmclear_loaded_vmcss, crash_vmclear_local_loaded_vmcss); From patchwork Mon May 13 14:38:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941085 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 739F51390 for ; Mon, 13 May 2019 14:40:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 641A827861 for ; Mon, 13 May 2019 14:40:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5835727E5A; Mon, 13 May 2019 14:40:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9FFE27861 for ; Mon, 13 May 2019 14:40:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A47B86B0277; Mon, 13 May 2019 10:39:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 983286B0278; Mon, 13 May 2019 10:39:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 785BE6B0279; Mon, 13 May 2019 10:39:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by kanga.kvack.org (Postfix) with ESMTP id 572B26B0277 for ; Mon, 13 May 2019 10:39:53 -0400 (EDT) Received: by mail-io1-f72.google.com with SMTP id n24so497361ioo.23 for ; Mon, 13 May 2019 07:39:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=OxanbwcMAhQGR2MgwGG1WhL6eejs6scvJBA++WEfeAY=; b=LkJiSepkvLmNXqrDMMLHUfXKxUYKvVV9LOg9VBsisJDC0I1dS37ow4DYa9XdMlnLwg AjXr1l/VAEKh8HUTKnd5ywUgjDW6PIRnEGCZSAsNP+98tRTTCKzBpEMw3TP/ulwqVtq0 Bkzl46+VyCyTEohGXCPhfNrkSV63ZuRbIwLDT1AvumuqiqTC6EhJMhEA2FXHqyhpus52 IauWS8mX8cgwCSdmbiqm55xprqI9XzsGkE9yzmdXVJcEt6f18uzNkbLX3jHNvtFS9QZB yHFcV66rpdhdm/VmAzlsrc5J3hNjoNrz+r3l3sXKTPzzc85I73LJ/BM/pgjeCD3ZSYli skOg== X-Gm-Message-State: APjAAAULVQzDTWyaeaFdNed1cCNtcyoyGBF6UflPgS+qzibup2aKz3Cz TFYFPGH5yQO3gt3KjN/VdUBzucUZcFzvT+ax69EcJCtPQ6vhpVbll9auNWV2iRZI6oA8QJ1PdxW dCvjCkJRzf738lQsFxfbQ1VuePldIRHeEO7HrVqpoxQGoPwSwGz2UJPJ2mvdnmbqlPA== X-Received: by 2002:a24:274e:: with SMTP id g75mr17314962ita.34.1557758393095; Mon, 13 May 2019 07:39:53 -0700 (PDT) X-Google-Smtp-Source: APXvYqxUYv0YQ5TP5w40sgJLkPrNWnEwDO7MEqo1zEgN9Bguiy6+bZtW99b0h3Qb/D+SQIa84KwA X-Received: by 2002:a24:274e:: with SMTP id g75mr17314906ita.34.1557758392267; Mon, 13 May 2019 07:39:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758392; cv=none; d=google.com; s=arc-20160816; b=MUIONStcK1rPTUWTkxnmq/rsCumOZWK5hLNBL6Wq+ylAu8m1iVhTVFnz0GeoI4vHgH 4zSQveymIYwgear2mYQuTCxvglJMnNfi33VvunUVTYRZtenmUatMdGC90lzWINitlz7B oC5et0e4LPFP9l83jkKlBXmqTIFgPOXNvijeYQjhovQxJIETtkq80sBqibvgecoTyBNl qseHwzXrY2R1UFG4XVmYBrCds/7dssRlNODDUAjxfE3vIaNjDdN4AHMFegtBN7spTpgW QU2VJIaASLYcgYuUtJUViq1kzAbVnfj0WiVYY1Yj8GgReBUzUI+xEu7MR6OogMxQWxuG 9+fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=OxanbwcMAhQGR2MgwGG1WhL6eejs6scvJBA++WEfeAY=; b=TjbKGdIc8bbmVDCQ1dOFBeGKiwkA0+ADcTUYUGMteAewPo6k9mIf1rfifN0yJuf+R8 dZzE9RdzFFiPRkiddleC9avTvruCw6bAbtU7M9l9y8/Ym7ZsIua6YayPxUbAz5E64IcP yzroy3WI4+CeI67FSZwpeLkAVq0bmkbN1pwvbUw+skj9TqiPx0ssGE8nZ6aa+9YuV1en Ixp1TXMo6ARxjGqpx9lqLf5UkIjYPGcjTa+2jPD7GuJ49Eq44MWGGF0gpvnkwfsYfr2O WHzcN1fsszfTAGDtcFmkBunAfL8q57oqKmjR5aSVOF/0kaW7KH6IZ6pRt2Z3Y/RvDrD1 kfVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=sU2LVxvs; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2130.oracle.com (userp2130.oracle.com. [156.151.31.86]) by mx.google.com with ESMTPS id 7si8146824itv.107.2019.05.13.07.39.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:52 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) client-ip=156.151.31.86; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=sU2LVxvs; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd3dD181544; Mon, 13 May 2019 14:39:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=OxanbwcMAhQGR2MgwGG1WhL6eejs6scvJBA++WEfeAY=; b=sU2LVxvsJrs8djlyFSIdZHIRCpnGERg2UYPNUhkjBZS5y33p5PwL4YfHc1GeuRzDdQDc tI0T4yhU0dK1zriLrw7oc9ar2kWJb8ndOWDLgtO2S3p47ONlJjMT8DUNuiH6bnk/HBS8 4pRTCY+QUM4kQQE3oKH19lJQLedFyMmlQevJV2qxd27nI0pcuQGDzWA0LWRRFA/SWQEU nPdKdbh2dWUUqgGbATEXLToM31SW1rDh84aMrX31dZ5ifOINbXk1GaZ1ybT260MDzhmO Rv/V6YNnfjMVWcuyQ7jnbONxErhqEaY2u+iG8myukAxzaJ4GJAWzjMX/sNhelawAFCXL Aw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2sdnttfejj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:42 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQO022780; Mon, 13 May 2019 14:39:39 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 21/27] kvm/isolation: initialize the KVM page table with vmx VM data Date: Mon, 13 May 2019 16:38:29 +0200 Message-Id: <1557758315-12667-22-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=2 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Map VM data, in particular the kvm structure data. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 17 +++++++++++++++++ arch/x86/kvm/isolation.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 31 ++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 12 ++++++++++++ include/linux/kvm_host.h | 1 + virt/kvm/arm/arm.c | 4 ++++ virt/kvm/kvm_main.c | 2 +- 7 files changed, 67 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index cf5ee0d..d3ac014 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -1222,6 +1222,23 @@ static void kvm_isolation_clear_handlers(void) kvm_set_isolation_exit_handler(NULL); } +int kvm_isolation_init_vm(struct kvm *kvm) +{ + if (!kvm_isolation()) + return 0; + + return (kvm_copy_percpu_mapping(kvm->srcu.sda, + sizeof(struct srcu_data))); +} + +void kvm_isolation_destroy_vm(struct kvm *kvm) +{ + if (!kvm_isolation()) + return; + + kvm_clear_percpu_mapping(kvm->srcu.sda); +} + int kvm_isolation_init(void) { int r; diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 1f79e28..33e9a87 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -23,6 +23,8 @@ static inline bool kvm_isolation(void) extern int kvm_isolation_init(void); extern void kvm_isolation_uninit(void); +extern int kvm_isolation_init_vm(struct kvm *kvm); +extern void kvm_isolation_destroy_vm(struct kvm *kvm); extern void kvm_isolation_enter(void); extern void kvm_isolation_exit(void); extern void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f181b3c..5b52e8c 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6523,6 +6523,33 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx_complete_interrupts(vmx); } +static void vmx_unmap_vm(struct kvm *kvm) +{ + struct kvm_vmx *kvm_vmx = to_kvm_vmx(kvm); + + if (!kvm_isolation()) + return; + + pr_debug("unmapping kvm %p", kvm_vmx); + kvm_clear_range_mapping(kvm_vmx); +} + +static int vmx_map_vm(struct kvm *kvm) +{ + struct kvm_vmx *kvm_vmx = to_kvm_vmx(kvm); + + if (!kvm_isolation()) + return 0; + + pr_debug("mapping kvm %p", kvm_vmx); + /* + * Only copy kvm_vmx struct mapping because other + * attributes (like kvm->srcu) are not initialized + * yet. + */ + return kvm_copy_ptes(kvm_vmx, sizeof(struct kvm_vmx)); +} + static struct kvm *vmx_vm_alloc(void) { struct kvm_vmx *kvm_vmx = __vmalloc(sizeof(struct kvm_vmx), @@ -6533,6 +6560,7 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu) static void vmx_vm_free(struct kvm *kvm) { + vmx_unmap_vm(kvm); vfree(to_kvm_vmx(kvm)); } @@ -6702,7 +6730,8 @@ static int vmx_vm_init(struct kvm *kvm) break; } } - return 0; + + return (vmx_map_vm(kvm)); } static void __init vmx_check_processor_compat(void *rtn) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1db72c3..e1cc3a6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9207,6 +9207,17 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) return 0; } +void kvm_arch_vm_postcreate(struct kvm *kvm) +{ + /* + * The kvm structure is mapped in vmx.c so that the full kvm_vmx + * structure can be mapped. Attributes allocated in the kvm + * structure (like kvm->srcu) are mapped by kvm_isolation_init_vm() + * because they are not initialized when vmx.c maps the kvm structure. + */ + kvm_isolation_init_vm(kvm); +} + static void kvm_unload_vcpu_mmu(struct kvm_vcpu *vcpu) { vcpu_load(vcpu); @@ -9320,6 +9331,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm) x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, 0, 0); x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); } + kvm_isolation_destroy_vm(kvm); if (kvm_x86_ops->vm_destroy) kvm_x86_ops->vm_destroy(kvm); kvm_pic_destroy(kvm); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 640a036..ad24d9e 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -932,6 +932,7 @@ static inline bool kvm_arch_intc_initialized(struct kvm *kvm) int kvm_arch_init_vm(struct kvm *kvm, unsigned long type); void kvm_arch_destroy_vm(struct kvm *kvm); +void kvm_arch_vm_postcreate(struct kvm *kvm); void kvm_arch_sync_events(struct kvm *kvm); int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu); diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c index f412ebc..0921cb3 100644 --- a/virt/kvm/arm/arm.c +++ b/virt/kvm/arm/arm.c @@ -156,6 +156,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) return ret; } +void kvm_arch_vm_postcreate(struct kvm *kvm) +{ +} + bool kvm_arch_has_vcpu_debugfs(void) { return false; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index a704d1f..3c0c3db 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3366,7 +3366,7 @@ static int kvm_dev_ioctl_create_vm(unsigned long type) return -ENOMEM; } kvm_uevent_notify_change(KVM_EVENT_CREATE_VM, kvm); - + kvm_arch_vm_postcreate(kvm); fd_install(r, file); return r; From patchwork Mon May 13 14:38:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941089 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 89C866C5 for ; Mon, 13 May 2019 14:40:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C44327BA5 for ; Mon, 13 May 2019 14:40:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 706BC27FA1; Mon, 13 May 2019 14:40:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F18B227BA5 for ; Mon, 13 May 2019 14:40:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF48A6B027A; Mon, 13 May 2019 10:39:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B56CC6B027B; Mon, 13 May 2019 10:39:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F8426B027C; Mon, 13 May 2019 10:39:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by kanga.kvack.org (Postfix) with ESMTP id 7FB256B027A for ; Mon, 13 May 2019 10:39:55 -0400 (EDT) Received: by mail-io1-f71.google.com with SMTP id z2so9986496iog.12 for ; Mon, 13 May 2019 07:39:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=K8i6wzsowphEeVDFXNpunuRUppmUJV415MqqD0nUaBI=; b=qKVLh4C09kUMZkSesT7hs/4uvX+0wtWPs/1/03fAA8iEePcFMmSfNBII4/2TZBcJIQ QgLcv049X2CVoFMCN+x/BFVQ4zfCFdOmn0pyFBjnsHXa6LZAfo3XauT5IHK/P/Fp7nN6 vDAteIQDzYCerc1dFTYvhJ6WIGp0HvsiGu2w9LMmCW6sDhYtiXYwWrhtwWnS3y+faYmy iNMIubILbeu8LQogHW9FksH7F8OsF1o4kg1pXVRBSWfr33/CgNvDtEdUua2VlhI/m3PF mCSYdugAimLzhudltBLOf02IljAg7WvQBMxmoVqw4KdSpBHgrMupPN3sZvxvRaxm2Ypk zZHA== X-Gm-Message-State: APjAAAXhWuG8n75c/rIDhk7uFf/WWtAHFqy68Co5XT5IhuhxbSor/sJI 8Ccy7/HbWtk1Crd0QVguyBdAz940NN9JBBzL7L7tnly2LTsbM18x5f/6MJRN21o5q7PJquM2E07 hlz7v4hkj056nZhQvEP5qTLhv8yrkuh/gn+CNYz6D1gNDJo9xaoS2ab+nKo4uu00pVQ== X-Received: by 2002:a24:5ec2:: with SMTP id h185mr21078501itb.19.1557758395268; Mon, 13 May 2019 07:39:55 -0700 (PDT) X-Google-Smtp-Source: APXvYqwSjrcMyhd3iLiDNEvT5iYJdeTV4ms7y4kxXD/3s4evRRW7UbeoS7SvDSZDValgIBnCMG8e X-Received: by 2002:a24:5ec2:: with SMTP id h185mr21078461itb.19.1557758394648; Mon, 13 May 2019 07:39:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758394; cv=none; d=google.com; s=arc-20160816; b=K5rZlNRJzzbPZcvA6ZPuZIdSdt53SQNaevlQ5rnbeYZWXpxw3bjXciaKF73F7t3lWC UKgWyncgLHwyiwgjpbBfeYpskWYbn8KeopXrLmKkg+Zvt5r+qCEY+LCIngY45JnGV/Z0 sBW6BiOA05GmvYMTHCB3W1s9B66qB4gI8UnDfMwbMQkDK3OJtbPKRrdUhu0uP769VT8i US7/iwg+4/YtcU5QBNqCal893zZBDnyvzut1QtizpY+L4V4HQeY/nZDS7mKThK9WBORZ dLxDMIipt6XGex/ZIqhxtfvbH2fHOsAIlBg+6OXw7JSShuVowTbva9q5/k7OAY7y7JBw tpTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=K8i6wzsowphEeVDFXNpunuRUppmUJV415MqqD0nUaBI=; b=0PsTS69htCxmF5C+LhuVeSZEUE96ZtwSRw+MKMLswnRvfUGVsKD25qCWhI//8kd9uz 8n2cZawEmmUtg9YTOamsGRHlESOHu0u4qW2C2EsqswbFdojUkj9qo5qXrFPW14UN4Hez 9tSYisNNMawiIpeBIYtN1T06d0EuxbfJBaSa+a8xejGJnBas1u7SXP859tvU9AL8DijS ZBl7NsEKZLSRcrW5Xk1TZ/vFJEvkHcfYOUQM4OP9lgcQCI9xArWrF0J6M/oANGmcTCrU OdIE3hRwNez0MFMQSeQc5bqTeLxjet9/vyitdhFy4+lFEQ+mY1u9L4hKD8piZErXPis8 lB5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=W22kvYLf; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2120.oracle.com (userp2120.oracle.com. [156.151.31.85]) by mx.google.com with ESMTPS id l186si8093217itb.59.2019.05.13.07.39.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:54 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) client-ip=156.151.31.85; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=W22kvYLf; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.85 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2Tx194906; Mon, 13 May 2019 14:39:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=K8i6wzsowphEeVDFXNpunuRUppmUJV415MqqD0nUaBI=; b=W22kvYLfLPm+lj8NF2wS7sDx9KGLeptevfeDTUzv+GZagg1XtUp9vZNWcihPEkWLSaPG Lx/9JGXoffT/Eb0lGLC1sDrq7K90utKJBv6oHFBVipO5+pMWF/TFSJYeuAR0q5E3kmrb Z8y5MymYgg/n3HDgLeCgwZcunQle8i2tJ4Yh1yNjfZX7YXAq/W6CISOEnlueSKA6ARpD 7dxEJ5PcvUpwerEg1uRhKhIlwA7KzrsVOb2FbKrBGoKH6oKfCK78Z0vuFk+sXe8ZpF4h vFYm9C8pwOMgg5LaQcKAYKtRiG+FEuIFLEcRxD9EaawYkjfOPClZpb0QoK/QdH7HCyaA Yg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2sdq1q7ayc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:45 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQP022780; Mon, 13 May 2019 14:39:42 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 22/27] kvm/isolation: initialize the KVM page table with vmx cpu data Date: Mon, 13 May 2019 16:38:30 +0200 Message-Id: <1557758315-12667-23-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Map vmx cpu to the KVM address space when a vmx cpu is created, and unmap when it is freed. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/vmx/vmx.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 65 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 5b52e8c..cbbaf58 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6564,10 +6564,69 @@ static void vmx_vm_free(struct kvm *kvm) vfree(to_kvm_vmx(kvm)); } +static void vmx_unmap_vcpu(struct vcpu_vmx *vmx) +{ + pr_debug("unmapping vmx %p", vmx); + + kvm_clear_range_mapping(vmx); + if (enable_pml) + kvm_clear_range_mapping(vmx->pml_pg); + kvm_clear_range_mapping(vmx->guest_msrs); + kvm_clear_range_mapping(vmx->vmcs01.vmcs); + kvm_clear_range_mapping(vmx->vmcs01.msr_bitmap); + kvm_clear_range_mapping(vmx->vcpu.arch.pio_data); + kvm_clear_range_mapping(vmx->vcpu.arch.apic); +} + +static int vmx_map_vcpu(struct vcpu_vmx *vmx) +{ + int rv; + + pr_debug("mapping vmx %p", vmx); + + rv = kvm_copy_ptes(vmx, sizeof(struct vcpu_vmx)); + if (rv) + goto out_unmap_vcpu; + + if (enable_pml) { + rv = kvm_copy_ptes(vmx->pml_pg, PAGE_SIZE); + if (rv) + goto out_unmap_vcpu; + } + + rv = kvm_copy_ptes(vmx->guest_msrs, PAGE_SIZE); + if (rv) + goto out_unmap_vcpu; + + rv = kvm_copy_ptes(vmx->vmcs01.vmcs, PAGE_SIZE << vmcs_config.order); + if (rv) + goto out_unmap_vcpu; + + rv = kvm_copy_ptes(vmx->vmcs01.msr_bitmap, PAGE_SIZE); + if (rv) + goto out_unmap_vcpu; + + rv = kvm_copy_ptes(vmx->vcpu.arch.pio_data, PAGE_SIZE); + if (rv) + goto out_unmap_vcpu; + + rv = kvm_copy_ptes(vmx->vcpu.arch.apic, sizeof(struct kvm_lapic)); + if (rv) + goto out_unmap_vcpu; + + return 0; + +out_unmap_vcpu: + vmx_unmap_vcpu(vmx); + return rv; +} + static void vmx_free_vcpu(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); + if (kvm_isolation()) + vmx_unmap_vcpu(vmx); if (enable_pml) vmx_destroy_pml_buffer(vmx); free_vpid(vmx->vpid); @@ -6679,6 +6738,12 @@ static void vmx_free_vcpu(struct kvm_vcpu *vcpu) vmx->ept_pointer = INVALID_PAGE; + if (kvm_isolation()) { + err = vmx_map_vcpu(vmx); + if (err) + goto free_vmcs; + } + return &vmx->vcpu; free_vmcs: From patchwork Mon May 13 14:38:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941093 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E0E41390 for ; Mon, 13 May 2019 14:40:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B4E62833E for ; Mon, 13 May 2019 14:40:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0D97528346; Mon, 13 May 2019 14:40:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4075A28334 for ; Mon, 13 May 2019 14:40:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0BA96B027C; Mon, 13 May 2019 10:39:58 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A47476B027D; Mon, 13 May 2019 10:39:58 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BE1C6B027E; Mon, 13 May 2019 10:39:58 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by kanga.kvack.org (Postfix) with ESMTP id 642B36B027C for ; Mon, 13 May 2019 10:39:58 -0400 (EDT) Received: by mail-io1-f72.google.com with SMTP id c16so453493ioo.20 for ; Mon, 13 May 2019 07:39:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=Dw7jnDOehrItQ8IBhhkiUXlVuHtLBnaP+UdwDVV0Wkc=; b=eZVveuJ5FKDcVPdNWk74LVN43neofEpnaLykim0xyye6iZb5LSBeX45wernV1Xn6zR MUeBqpTqsRl6T4oNUWkWzEnR/hbxbWVlZ9il7ZdMj09xE/ptjuACCdB8KRF2wqingDYa ogV/3AJ+PVCaoITAtAg1dpJybZErccwBKrRs8wYfko9Gkuw4x0WBzSAoaaBzcEoB7D9O rYMRqNlcue6n8xrpOrTWnrSJ3hC9H2V2w51IdA9H51xNnLG/NW9kdCIdqg0rEGasrU9l xgIw++gPfP3Tz2XeBo1TLqEHR7jEjuD+T/BkOKkNQRZ8opRkh+7x1Kw8BBAWDkVCpkgX Ssqg== X-Gm-Message-State: APjAAAWNLtaWrl9p6xCDWQh5aGfg3WgvTk440lOm9mdkBznfHy3trf9m XDXIy6im4A7bzrK69/bOfqohYmLW2QgMVhqgM7uXzLYcUCCsyYNUw0/QXOJOQ1O23A0jtQJDfrV uFBLpOTzdnHInG8Xd/HfaTrN77ou7UIlhIcOedNWQkiXagT4FUfd7UyvMUpZtEhtD8g== X-Received: by 2002:a24:5e10:: with SMTP id h16mr18314652itb.170.1557758398128; Mon, 13 May 2019 07:39:58 -0700 (PDT) X-Google-Smtp-Source: APXvYqwmZzp24QMdm2u5WRfPN52Ruc76iXNWbEd6BxfBqqxFHYwddvcmEbiRPr2dfZbiNlar8n8g X-Received: by 2002:a24:5e10:: with SMTP id h16mr18314592itb.170.1557758397138; Mon, 13 May 2019 07:39:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758397; cv=none; d=google.com; s=arc-20160816; b=AITsqq21tOiw5cxr5wwVsoKJY1aChy/XQSh0UWEvOLsH5nLuvsCw+8HgJS3QLmDsXU Poel2iN1t9+FolJbX9UXIsDx+urQ9o1F/cZ6TowpCbYUSPVWSokwHek/BEuEe5t9txBL hPb5dhk9mRjih7JiZUB9cSND/9cGgpdvqYnniRFNLBcLehPxrX3PqYcJuoBXj/FmA86i CCxoM2trxJJkjCYnIDO5z8t9+sA6KTNI8Wro+AufCcenr0rWeMbvMZ//LNYswnnRva3l E8vZP7bLU2ehOtMFyO/jMBU+NAjyho2p/F0jgJdsqR5mIXLUSZHW7uGpIgBOTfR8GQk2 TxLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Dw7jnDOehrItQ8IBhhkiUXlVuHtLBnaP+UdwDVV0Wkc=; b=MOc2KyM5IPsh0FqX7FXPckc5eEqnFNcUjnnQIo3ID3G3IxkWbjvs28jJb0Uo5sZ5Ky B5AjFkdDrFvd6noiMnPX/BBypLDQT/RrYOWjk0tV4lQ51MM70PpZEXFa0ajA8grW3W9P Y1GUVFh8Mh2KHTtVC09EKVr11igltt1cfGGP6r/VXYHzXcNX+6rNlepKC9GEagRKZmlz wFLGo8dqO4bflkpVzZWqvmV8lb69J2pE3sJSzQfdbhiCrCBAMEHwUyLEl2kVuVFslzp2 GD0a8TWVCoNX3V6vlMhITHiK1W3GvgZlP+22wrfPwF3u3fRadU3YHfmRAXiWxWgiyHCT Vjfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=zSrYKyEo; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id 74si4802001itk.24.2019.05.13.07.39.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:57 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=zSrYKyEo; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2cM193026; Mon, 13 May 2019 14:39:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=Dw7jnDOehrItQ8IBhhkiUXlVuHtLBnaP+UdwDVV0Wkc=; b=zSrYKyEoOAdq0QWpvZ0qd6kNFKnCiopVoHRqhqc1ZV4bi48RNUFvLgth3il0pLxTxJ2n CePloH/migBtqYtd6xbrITTyQDKGmELS+eGvp8q4r6EsdjFT9Aha0voKsbZbmYzkK/aH 4zsWyBkbuScAy0o39X/qZ0peTs4L6A5vj+f8j0R0RpMOLecXpCbYiaN7tvdV6Zj7SkBb LqMBBHLoFSVmQtxPLbBms9KAeMYrLYP9+8QQeRm3CM/TahwAKsdeOMTJKAgWYkztBied FDur+uo2FG0gAOb5iBm4+gebU8igLetDFFzIk9XWaWLWwgf4BEpCgqcLaJ6fAkuU2QB3 PQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfm0d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:47 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQQ022780; Mon, 13 May 2019 14:39:45 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 23/27] kvm/isolation: initialize the KVM page table with the vcpu tasks Date: Mon, 13 May 2019 16:38:31 +0200 Message-Id: <1557758315-12667-24-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Tasks which are going to be running with the KVM address space have to be mapped with their core data (stack, mm, pgd..) so that they can (at least) switch back to the kernel address space. For now, assume that these tasks are the ones running vcpu, and that there's a 1:1 mapping between a task and vcpu. This should eventually be improved to be independent of any task/vcpu mapping. Also check that the task effectively entering the KVM address space is mapped. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 182 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 2 + arch/x86/kvm/vmx/vmx.c | 8 ++ include/linux/sched.h | 5 + 4 files changed, 197 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index d3ac014..e7979b3 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -64,6 +64,20 @@ struct pgt_directory_group { ((typeof(entry))(((unsigned long)(entry)) & PAGE_MASK)) /* + * Variables to keep track of tasks mapped into the KVM address space. + */ +struct kvm_task_mapping { + struct list_head list; + struct task_struct *task; + void *stack; + struct mm_struct *mm; + pgd_t *pgd; +}; + +static LIST_HEAD(kvm_task_mapping_list); +static DEFINE_MUTEX(kvm_task_mapping_lock); + +/* * Variables to keep track of address ranges mapped into the KVM * address space. */ @@ -1027,6 +1041,160 @@ int kvm_copy_percpu_mapping(void *percpu_ptr, size_t size) } EXPORT_SYMBOL(kvm_copy_percpu_mapping); +static void kvm_clear_task_mapping(struct kvm_task_mapping *task_mapping) +{ + if (task_mapping->task) { + kvm_clear_range_mapping(task_mapping->task); + task_mapping->task = NULL; + } + if (task_mapping->stack) { + kvm_clear_range_mapping(task_mapping->stack); + task_mapping->stack = NULL; + } + if (task_mapping->mm) { + kvm_clear_range_mapping(task_mapping->mm); + task_mapping->mm = NULL; + } + if (task_mapping->pgd) { + kvm_clear_range_mapping(task_mapping->pgd); + task_mapping->pgd = NULL; + } +} + +static int kvm_copy_task_mapping(struct task_struct *tsk, + struct kvm_task_mapping *task_mapping) +{ + int err; + + err = kvm_copy_ptes(tsk, sizeof(struct task_struct)); + if (err) + goto out_clear_task_mapping; + task_mapping->task = tsk; + + err = kvm_copy_ptes(tsk->stack, THREAD_SIZE); + if (err) + goto out_clear_task_mapping; + task_mapping->stack = tsk->stack; + + err = kvm_copy_ptes(tsk->active_mm, sizeof(struct mm_struct)); + if (err) + goto out_clear_task_mapping; + task_mapping->mm = tsk->active_mm; + + err = kvm_copy_ptes(tsk->active_mm->pgd, + PAGE_SIZE << PGD_ALLOCATION_ORDER); + if (err) + goto out_clear_task_mapping; + task_mapping->pgd = tsk->active_mm->pgd; + + return 0; + +out_clear_task_mapping: + kvm_clear_task_mapping(task_mapping); + return err; +} + +int kvm_add_task_mapping(struct task_struct *tsk) +{ + struct kvm_task_mapping *task_mapping; + int err; + + mutex_lock(&kvm_task_mapping_lock); + + if (tsk->kvm_mapped) { + mutex_unlock(&kvm_task_mapping_lock); + return 0; + } + + task_mapping = kzalloc(sizeof(struct kvm_task_mapping), GFP_KERNEL); + if (!task_mapping) { + mutex_unlock(&kvm_task_mapping_lock); + return -ENOMEM; + } + INIT_LIST_HEAD(&task_mapping->list); + + /* + * Ensure that the task and its stack are mapped into the KVM + * address space. Also map the task mm to be able to switch back + * to the original mm, and its PGD directory. + */ + pr_debug("mapping task %px\n", tsk); + err = kvm_copy_task_mapping(tsk, task_mapping); + if (err) { + kfree(task_mapping); + mutex_unlock(&kvm_task_mapping_lock); + return err; + } + + get_task_struct(tsk); + list_add(&task_mapping->list, &kvm_task_mapping_list); + tsk->kvm_mapped = true; + + mutex_unlock(&kvm_task_mapping_lock); + + return 0; +} +EXPORT_SYMBOL(kvm_add_task_mapping); + +static struct kvm_task_mapping *kvm_find_task_mapping(struct task_struct *tsk) +{ + struct kvm_task_mapping *task_mapping; + + list_for_each_entry(task_mapping, &kvm_task_mapping_list, list) { + if (task_mapping->task == tsk) + return task_mapping; + } + return NULL; +} + +void kvm_cleanup_task_mapping(struct task_struct *tsk) +{ + struct kvm_task_mapping *task_mapping; + + if (!tsk->kvm_mapped) + return; + + task_mapping = kvm_find_task_mapping(tsk); + if (!task_mapping) { + pr_debug("KVM isolation: mapping not found for mapped task %px\n", + tsk); + tsk->kvm_mapped = false; + mutex_unlock(&kvm_task_mapping_lock); + return; + } + + pr_debug("unmapping task %px\n", tsk); + + list_del(&task_mapping->list); + kvm_clear_task_mapping(task_mapping); + kfree(task_mapping); + tsk->kvm_mapped = false; + put_task_struct(tsk); + mutex_unlock(&kvm_task_mapping_lock); +} +EXPORT_SYMBOL(kvm_cleanup_task_mapping); + +/* + * Mark all tasks which have being mapped into the KVM address space + * as not mapped. This only clears the mapping attribute in the task + * structure, but page table mappings remain in the KVM page table. + * They will be effectively removed when deleting the KVM page table. + */ +static void kvm_reset_all_task_mapping(void) +{ + struct kvm_task_mapping *task_mapping; + struct task_struct *tsk; + + mutex_lock(&kvm_task_mapping_lock); + list_for_each_entry(task_mapping, &kvm_task_mapping_list, list) { + tsk = task_mapping->task; + pr_debug("clear mapping for task %px\n", tsk); + tsk->kvm_mapped = false; + put_task_struct(tsk); + } + mutex_unlock(&kvm_task_mapping_lock); +} + static int kvm_isolation_init_page_table(void) { @@ -1195,6 +1363,7 @@ static void kvm_isolation_uninit_mm(void) destroy_context(&kvm_mm); + kvm_reset_all_task_mapping(); kvm_isolation_uninit_page_table(); kvm_free_all_range_mapping(); @@ -1227,6 +1396,8 @@ int kvm_isolation_init_vm(struct kvm *kvm) if (!kvm_isolation()) return 0; + pr_debug("mapping kvm srcu sda\n"); + return (kvm_copy_percpu_mapping(kvm->srcu.sda, sizeof(struct srcu_data))); } @@ -1236,6 +1407,8 @@ void kvm_isolation_destroy_vm(struct kvm *kvm) if (!kvm_isolation()) return; + pr_debug("unmapping kvm srcu sda\n"); + kvm_clear_percpu_mapping(kvm->srcu.sda); } @@ -1276,12 +1449,21 @@ void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu) void kvm_isolation_enter(void) { + int err; + if (kvm_isolation()) { /* * Switches to kvm_mm should happen from vCPU thread, * which should not be a kernel thread with no mm */ BUG_ON(current->active_mm == NULL); + + err = kvm_add_task_mapping(current); + if (err) { + pr_err("KVM isolation cancelled (failed to map task %px)", + current); + return; + } /* TODO: switch to kvm_mm */ } } diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 33e9a87..2d7d016 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -32,5 +32,7 @@ static inline bool kvm_isolation(void) extern void kvm_clear_range_mapping(void *ptr); extern int kvm_copy_percpu_mapping(void *percpu_ptr, size_t size); extern void kvm_clear_percpu_mapping(void *percpu_ptr); +extern int kvm_add_task_mapping(struct task_struct *tsk); +extern void kvm_cleanup_task_mapping(struct task_struct *tsk); #endif diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index cbbaf58..9ed31c2 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6576,6 +6576,9 @@ static void vmx_unmap_vcpu(struct vcpu_vmx *vmx) kvm_clear_range_mapping(vmx->vmcs01.msr_bitmap); kvm_clear_range_mapping(vmx->vcpu.arch.pio_data); kvm_clear_range_mapping(vmx->vcpu.arch.apic); + + /* XXX assume there's a 1:1 mapping between a task and a vcpu */ + kvm_cleanup_task_mapping(current); } static int vmx_map_vcpu(struct vcpu_vmx *vmx) @@ -6614,6 +6617,11 @@ static int vmx_map_vcpu(struct vcpu_vmx *vmx) if (rv) goto out_unmap_vcpu; + /* XXX assume there's a 1:1 mapping between a task and a vcpu */ + rv = kvm_add_task_mapping(current); + if (rv) + goto out_unmap_vcpu; + return 0; out_unmap_vcpu: diff --git a/include/linux/sched.h b/include/linux/sched.h index 50606a6..80e1d75 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1199,6 +1199,11 @@ struct task_struct { unsigned long prev_lowest_stack; #endif +#ifdef CONFIG_HAVE_KVM + /* Is the task mapped into the KVM address space? */ + bool kvm_mapped; +#endif + /* * New fields for task_struct should be added above here, so that * they are included in the randomized portion of task_struct. From patchwork Mon May 13 14:38:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941095 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C03A6C5 for ; Mon, 13 May 2019 14:40:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A48B28306 for ; Mon, 13 May 2019 14:40:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D79C28334; Mon, 13 May 2019 14:40:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3547928306 for ; Mon, 13 May 2019 14:40:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B7606B027E; Mon, 13 May 2019 10:40:01 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 366C56B027F; Mon, 13 May 2019 10:40:01 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 193256B0280; Mon, 13 May 2019 10:40:01 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f200.google.com (mail-it1-f200.google.com [209.85.166.200]) by kanga.kvack.org (Postfix) with ESMTP id EB81F6B027E for ; Mon, 13 May 2019 10:40:00 -0400 (EDT) Received: by mail-it1-f200.google.com with SMTP id o126so4939625itc.5 for ; Mon, 13 May 2019 07:40:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=I1kUHuKiSH9QzRVSTdzQMnQ4Y46T3IjTtH8oraLYUKM=; b=pqJ5ZdPNTVddHcmLeF+Me/SSXUwbjIfOTcybL7FbavinksGxr2uRKMDI+zW8f+xnlD GmrXlYLPmwTqhWfioroIJV/vyskjlrt6v3whkhvYZkoV7VIEpc3D6cXAnh2mVlPJWQCD 17uwvAvie0qngcV5tcmXb0Q8vLqfcNxVjGGtuyyikExYz8AkONvrbqi9HLeqx0GZBvoH +XD+GxEsPCSrHT6g8WMBS4+jlc8hGmsylBI0XRmG3L+6H5wGcrOKJCFNtKFiN4ac9TBf 3MgAVQTnar96UGihe1NDC54rr470Tjpx2M5pN2/9XyilpAJ13zWLluInX9M413SD846Y C7Vg== X-Gm-Message-State: APjAAAV65CpckmadAfcQnNyBDVQf9jUXv5JQoQM/McrD7exd/T4EhRuD PYNpF/ksZDnR7sgjz+92f5LXTJo6s9cBWA5liS5dsMjKThzEewvMT9SFJG2dGTEaoK8yOMlRUn/ KCHz3zB4GjGKlLpwQs6RLSVwQ10z+cAbREBfhivaSx18dGCoNHv/Y4Ne1+W0J4iChWw== X-Received: by 2002:a05:6602:4f:: with SMTP id z15mr17131337ioz.108.1557758400665; Mon, 13 May 2019 07:40:00 -0700 (PDT) X-Google-Smtp-Source: APXvYqzgtn3D25Hys2NNIC/hBt8Wtzkl2WWO/E8RxnCtdru1Og0ahqlOst4ENzZDuAkbeYTgtP2z X-Received: by 2002:a05:6602:4f:: with SMTP id z15mr17131283ioz.108.1557758399633; Mon, 13 May 2019 07:39:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758399; cv=none; d=google.com; s=arc-20160816; b=eADWenpWpkC31qvR/LqYvYeBL7b5/oGMt6IK18ccC5ibkqO2WrTIWOrC5+Ud0G6w1H 0GdMY8Nqx2TvEh1VF2/5vinQnaMUc7WkCDu2uzFSF0lXTaxbrXIiWJPYF5vQZBr1Kbp7 vJ7xnNovoarlxpJbVJ4ENxdnUz/ccnsXioxcKsQg0S9NTxIDemcJAY3nGw5QvF1INEpd 97tNaXKNaKjClJ73bErCAVMIRRNsGnObFMnTX5OGUSrdMHUglJDimky5qXhHUE7Y2LMV j6U2SLN31EwWSZB0XxyCFJMaasU1ioa1mEAQPqlwyoBYILQq4bZzEKVJxsKtjecv3Mla tzaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=I1kUHuKiSH9QzRVSTdzQMnQ4Y46T3IjTtH8oraLYUKM=; b=R87JqUTh+yPOQvNihXeuTOCZtouIPSoah5dnNw8Bqs8dV/z/+2nBBMD9db6MTZJB+t EgDJxAHLgOwKS0vD1FUjy+D3F4LE0WON5d2cZ5iK6wsNalAewPvbFfjATmNamtezcUAb tlPJqcaEA5X4qq7nOjp1ohj0uKH4WLFGKicdzBK2nmLsbFq3idyjzvl0nO7b4n24zKIX lZa2ELbzmypdH+9OEhVCrTGM9kFWgRMo+ClPqkhtC/FtrNCZwtDxGUSYjWon0Mxj3vko DfZ4J1J39Srg0NcL5Sx9LOiAMkb587VVp0w4EQBHFXowJxqwJihNLmT4Y9YarnTcRIjm /Gow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=ajrelu+2; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id b2si8578116iti.141.2019.05.13.07.39.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:39:59 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=ajrelu+2; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEdhZb193417; Mon, 13 May 2019 14:39:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=I1kUHuKiSH9QzRVSTdzQMnQ4Y46T3IjTtH8oraLYUKM=; b=ajrelu+23EpPoZzOm46NJouVlFfpY/7VuA8j/AA2maWWZv6PEWO74sb0JEOk8WTm7MlX zkCzIK+uulNLm3Uc5+fKPE0rSLiDjjJLejX1G+rGS4A2aaX6wwImF2XGsend6ZiCm1fm P8RiX9tW+RIc4zR0FuNImR/j0P1fr1IsIkAxcMSKq1XNd8xla45EpEAzSsqq0KaMJbcU mGTuD8FX0uYid5B79ED/WxOVxHhFvK/EkRkDNZuXjosZo4JvYLUzGo5D2aW2dE1YBaP0 wQ7ypO/K2a5rX9NiNfuwOPDx8wO0Hl7bt9/GOcZqO+YBs4IdO/Hb+PDVBfNbd7Ra1TXZ Dw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfm0s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:50 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQR022780; Mon, 13 May 2019 14:39:47 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 24/27] kvm/isolation: KVM page fault handler Date: Mon, 13 May 2019 16:38:32 +0200 Message-Id: <1557758315-12667-25-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The KVM page fault handler handles page fault occurring while using the KVM address space by switching to the kernel address space and retrying the access (except if the fault occurs while switching to the kernel address space). Processing of page faults occurring while using the kernel address space is unchanged. Page fault log is cleared when creating a vm so that page fault information doesn't persist when qemu is stopped and restarted. The KVM module parameter page_fault_stack can be used to disable dumping stack trace when a page fault occurs while using the KVM address space. The fault will still be reported but without the stack trace. Signed-off-by: Alexandre Chartre --- arch/x86/kernel/dumpstack.c | 1 + arch/x86/kvm/isolation.c | 202 +++++++++++++++++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 12 +++ 3 files changed, 215 insertions(+), 0 deletions(-) diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c index 2b58864..aa28763 100644 --- a/arch/x86/kernel/dumpstack.c +++ b/arch/x86/kernel/dumpstack.c @@ -292,6 +292,7 @@ void show_stack(struct task_struct *task, unsigned long *sp) show_trace_log_lvl(task, NULL, sp, KERN_DEFAULT); } +EXPORT_SYMBOL(show_stack); void show_stack_regs(struct pt_regs *regs) { diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index e7979b3..db0a7ce 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -17,6 +18,9 @@ #include "isolation.h" +extern bool (*kvm_page_fault_handler)(struct pt_regs *regs, + unsigned long error_code, + unsigned long address); enum page_table_level { PGT_LEVEL_PTE, @@ -91,6 +95,25 @@ struct kvm_range_mapping { static LIST_HEAD(kvm_range_mapping_list); static DEFINE_MUTEX(kvm_range_mapping_lock); +/* + * When a page fault occurs, while running with the KVM address space, + * the KVM page fault handler prints information about the fault (in + * particular the stack trace), and it switches back to the kernel + * address space. + * + * Information printed by the KVM page fault handler can be used to find + * out data not mapped in the KVM address space. Then the KVM address + * space can be augmented to include the missing mapping so that we don't + * fault at that same place anymore. + * + * The following variables keep track of page faults occurring while running + * with the KVM address space to prevent displaying the same information. + */ + +#define KVM_LAST_FAULT_COUNT 128 + +static unsigned long kvm_last_fault[KVM_LAST_FAULT_COUNT]; + struct mm_struct kvm_mm = { .mm_rb = RB_ROOT, @@ -126,6 +149,14 @@ static void kvm_clear_mapping(void *ptr, size_t size, static bool __read_mostly address_space_isolation; module_param(address_space_isolation, bool, 0444); +/* + * When set to true, KVM dumps the stack when a page fault occurs while + * running with the KVM address space. Otherwise the page fault is still + * reported but without the stack trace. + */ +static bool __read_mostly page_fault_stack = true; +module_param(page_fault_stack, bool, 0444); + static struct kvm_range_mapping *kvm_get_range_mapping_locked(void *ptr, bool *subset) { @@ -1195,6 +1226,173 @@ static void kvm_reset_all_task_mapping(void) mutex_unlock(&kvm_task_mapping_lock); } +static int bad_address(void *p) +{ + unsigned long dummy; + + return probe_kernel_address((unsigned long *)p, dummy); +} + +static void kvm_dump_pagetable(pgd_t *base, unsigned long address) +{ + pgd_t *pgd = base + pgd_index(address); + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + pr_info("BASE %px ", base); + + if (bad_address(pgd)) + goto bad; + + pr_cont("PGD %lx ", pgd_val(*pgd)); + + if (!pgd_present(*pgd)) + goto out; + + p4d = p4d_offset(pgd, address); + if (bad_address(p4d)) + goto bad; + + pr_cont("P4D %lx ", p4d_val(*p4d)); + if (!p4d_present(*p4d) || p4d_large(*p4d)) + goto out; + + pud = pud_offset(p4d, address); + if (bad_address(pud)) + goto bad; + + pr_cont("PUD %lx ", pud_val(*pud)); + if (!pud_present(*pud) || pud_large(*pud)) + goto out; + + pmd = pmd_offset(pud, address); + if (bad_address(pmd)) + goto bad; + + pr_cont("PMD %lx ", pmd_val(*pmd)); + if (!pmd_present(*pmd) || pmd_large(*pmd)) + goto out; + + pte = pte_offset_kernel(pmd, address); + if (bad_address(pte)) + goto bad; + + pr_cont("PTE %lx", pte_val(*pte)); +out: + pr_cont("\n"); + return; +bad: + pr_info("BAD\n"); +} + +static void kvm_clear_page_fault(void) +{ + int i; + + for (i = 0; i < KVM_LAST_FAULT_COUNT; i++) + kvm_last_fault[i] = 0; +} + +static void kvm_log_page_fault(struct pt_regs *regs, unsigned long error_code, + unsigned long address) +{ + int i; + + /* + * Log information about the fault only if this is a fault + * we don't know about yet (or if the fault tracking buffer + * is full). + */ + for (i = 0; i < KVM_LAST_FAULT_COUNT; i++) { + if (!kvm_last_fault[i]) { + kvm_last_fault[i] = regs->ip; + break; + } + if (kvm_last_fault[i] == regs->ip) + return; + } + + if (i >= KVM_LAST_FAULT_COUNT) + pr_warn("KVM isolation: fault tracking buffer is full [%d]\n", + i); + + pr_info("KVM isolation: page fault #%d (%ld) at %pS on %px (%pS)\n", + i, error_code, (void *)regs->ip, + (void *)address, (void *)address); + if (page_fault_stack) + show_stack(NULL, (unsigned long *)regs->sp); +} + +/* + * KVM Page Fault Handler. The handler handles two simple cases: + * + * - If the fault occurs while using the kernel address space, then let + * the kernel handles the fault normally. + * + * - If the fault occurs while using the KVM address space, then switch + * to the kernel address space, and retry. + * + * It also handles a tricky case: if the fault occurs when using the KVM + * address space but while switching to the kernel address space then the + * switch is failing and we can't recover. In that case, we force switching + * to the kernel address space, print information and let the kernel + * handles the fault. + */ +static bool kvm_page_fault(struct pt_regs *regs, unsigned long error_code, + unsigned long address) +{ + struct mm_struct *active_mm = current->active_mm; + unsigned long cr3; + + /* + * First, do a quick and simple test to see if we are using + * the KVM address space. If we do then exit KVM isolation, + * log the fault and report that we have handled the fault. + */ + if (likely(active_mm == &kvm_mm)) { + kvm_isolation_exit(); + kvm_log_page_fault(regs, error_code, address); + return true; + } + + /* + * Verify that we are effectively using the kernel address space. + * When switching address space, active_mm is not necessarily up + * to date as it can already be set with the next mm while %cr3 + * has not been updated yet. So check loaded_mm which is updated + * after %cr3. + * + * If we are effectively using the kernel address space then report + * that we haven't handled the fault. + */ + if (this_cpu_read(cpu_tlbstate.loaded_mm) != &kvm_mm) + return false; + + /* + * We are actually using the KVM address space and faulting while + * switching address space. Force swiching to the kernel address + * space, log information and reported that we haven't handled + * the fault. + */ + cr3 = __read_cr3(); + write_cr3(build_cr3(active_mm->pgd, 0)); + kvm_dump_pagetable(kvm_mm.pgd, address); + kvm_dump_pagetable(active_mm->pgd, address); + printk(KERN_DEFAULT "KVM isolation: page fault %ld at %pS on %lx (%pS) while switching mm\n" + " cr3=%lx\n" + " kvm_mm=%px pgd=%px\n" + " active_mm=%px pgd=%px\n", + error_code, (void *)regs->ip, address, (void *)address, + cr3, + &kvm_mm, kvm_mm.pgd, + active_mm, active_mm->pgd); + dump_stack(); + + return false; +} + static int kvm_isolation_init_page_table(void) { @@ -1384,11 +1582,13 @@ static void kvm_isolation_uninit_mm(void) static void kvm_isolation_set_handlers(void) { kvm_set_isolation_exit_handler(kvm_isolation_exit); + kvm_page_fault_handler = kvm_page_fault; } static void kvm_isolation_clear_handlers(void) { kvm_set_isolation_exit_handler(NULL); + kvm_page_fault_handler = NULL; } int kvm_isolation_init_vm(struct kvm *kvm) @@ -1396,6 +1596,8 @@ int kvm_isolation_init_vm(struct kvm *kvm) if (!kvm_isolation()) return 0; + kvm_clear_page_fault(); + pr_debug("mapping kvm srcu sda\n"); return (kvm_copy_percpu_mapping(kvm->srcu.sda, diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 46df4c6..317e105 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -33,6 +33,10 @@ #define CREATE_TRACE_POINTS #include +bool (*kvm_page_fault_handler)(struct pt_regs *regs, unsigned long error_code, + unsigned long address); +EXPORT_SYMBOL(kvm_page_fault_handler); + /* * Returns 0 if mmiotrace is disabled, or if the fault is not * handled by mmiotrace: @@ -1253,6 +1257,14 @@ static int fault_in_kernel_space(unsigned long address) WARN_ON_ONCE(hw_error_code & X86_PF_PK); /* + * KVM might be able to handle the fault when running with the + * KVM address space. + */ + if (kvm_page_fault_handler && + kvm_page_fault_handler(regs, hw_error_code, address)) + return; + + /* * We can fault-in kernel-space virtual memory on-demand. The * 'reference' page table is init_mm.pgd. * From patchwork Mon May 13 14:38:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941101 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 797496C5 for ; Mon, 13 May 2019 14:40:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C69928334 for ; Mon, 13 May 2019 14:40:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 60C7528346; Mon, 13 May 2019 14:40:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E349328334 for ; Mon, 13 May 2019 14:40:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D21F6B027F; Mon, 13 May 2019 10:40:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 082176B0281; Mon, 13 May 2019 10:40:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF1086B0282; Mon, 13 May 2019 10:40:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by kanga.kvack.org (Postfix) with ESMTP id B9F896B027F for ; Mon, 13 May 2019 10:40:03 -0400 (EDT) Received: by mail-io1-f72.google.com with SMTP id y15so9987135iod.10 for ; Mon, 13 May 2019 07:40:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=LfHNGKdGVZ8SXafCfcR3Zof+WadTo/bKWh5zIaesw/M=; b=EWcqFMyoNVU3seUxHOCJtm95pI5i2CKxh700IW/ksVSZp4pWpMUepPfNzJ3kR1TvwG K1IDUUdXfJco47K8DxIoMhLvT4/EJRuLbHJXfhZX28sZqJfMJYBueGPJpMrw2oZ535g1 V5cScC1dhKAxUX8TRYZTyhZmJdzXv/lsYXpYSZc1mDJHt2riAe5dGUCZ8ycitKYmv7XM 0GxhSbeiAxN/OSSqPSMGXzz+pjo2QXuMC3mEAkzq5Th40iGMI/tR2Q2T+ieaJQY0Gt6y fyI3dckDnebNKTGRCJ1pnL8rW+m/8Eb72Q5vqKF2f1AKafmW+jtRfCfkH7+ueXsY5pZJ 2BBw== X-Gm-Message-State: APjAAAXQIhaFvr2KPJusonnWQ6GF9hnX9lULjhHseUVDTsT56IzEWObM tVf3KP76WDfk8pw8A8BtuWNMW7m+zUb8AOfI7saRc1NtzP0N/gxrs1gO3xtcR763xA7WooOHOE2 kCdcsHJsDXtESxtrnIlbyni43RaVB6iBqulQ0Dob+XikTasOFCziZxelNxxHPqdLwow== X-Received: by 2002:a24:fe0b:: with SMTP id w11mr13264438ith.6.1557758403510; Mon, 13 May 2019 07:40:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqwj2j+fJMr/ZBCFPplBnrlnPf/ooYu4wBVgxBuLWTsHUYTazyz3RmVW9SBPfVbkgQtI619I X-Received: by 2002:a24:fe0b:: with SMTP id w11mr13264372ith.6.1557758402788; Mon, 13 May 2019 07:40:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758402; cv=none; d=google.com; s=arc-20160816; b=cR0GL47AVI1uO/rxwitokhBjUHNpPb7C246Mhelr17Jpj0Xd5uLxf1uEJaclGNbU5Y MBHafXTTSEcJ/bUObDXNhheTdEXWHeR88tdGNObCcVGzwr4uYCxpeEQRs7LG/9zEPNTW 8ejV2KYWU0wLrB/8219Gi9qWpjHGhZBgIlLFGgHhELRkvNG3qbPP32upx9tfTExYOkYJ fie8HEgaIeRVZMQFxVNAV5Wf5Tp16fpVZw42CyOMsPO9Zo5Vb383FD7auOO5OeI5CX+O FzhQiWfbG/gwWgYLHgvqdi0T4s0vKOjf/NYdFjOz5ki9J04tS+rJF5Qa4RqyPUh6MoTK NY7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=LfHNGKdGVZ8SXafCfcR3Zof+WadTo/bKWh5zIaesw/M=; b=cwX3B/VC4JlX6z4qZL4OVNbzRt5Fii/z5fOF9rAhacWza4YJKyhBQDqm0mZbkf3HZF g05pfZI44kNacfuFCyfvMaG6GtchPFC0X+p0JTnS4Pi1kA7C0IU3lKteVFBJGIhCBSZ1 CVlK/qLDdc+2O/M+0WNZWEXzGfmFuWFD+LaS6PVaZS5yZF3GdCwDrubTcZ2sh8CFcD6S 1YKwY5Dl/zVTdJCNAw/RZeTR8+SAutbSa7RTkfTOErPZa/41Gw9dJHfAsVpuynkjE6R2 Jwb16YjMYxZ2IQWqMXqtXzXj4ZB69C7NadqSw2UtUeof4Xv7vOODdvC7mye5rgDbAfrn sKaw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="m2arcl/O"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id c17si83681itc.40.2019.05.13.07.40.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:40:02 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="m2arcl/O"; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEdHlv193231; Mon, 13 May 2019 14:39:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=LfHNGKdGVZ8SXafCfcR3Zof+WadTo/bKWh5zIaesw/M=; b=m2arcl/OoSz25ufQlPkSDyXa5f1frMRq2QeB0JzA4PV/uhaSpAPsJfmP6YeH0R+Y7+PC 50HJ+Yx7Z8gUn3m69WqI9tJvyt5ZPxzpbW+60gErtelRwYfFJw+X3V78E3pi6uU708Zl uEVUq4MhUxXUf1DFx84dYgGV/93RXZwfztVCXTVvknZhM6YZw2KSvN8qBdHsbd2ELVmx /7qs4qjDnvxh+fcskoo9PGQvTX2vY0RinWTrcvKjzeXlWoP9/wWiom7xFgGIssxbulhP IvPfd4bMncK92V8msleNSWffh5EcxE1AX3Pw6AsvfuevV3vA7E7XlcOF1Gt+oJHTRqAq Ww== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfm13-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:39:53 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQS022780; Mon, 13 May 2019 14:39:50 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 25/27] kvm/isolation: implement actual KVM isolation enter/exit Date: Mon, 13 May 2019 16:38:33 +0200 Message-Id: <1557758315-12667-26-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Liran Alon KVM isolation enter/exit is done by switching between the KVM address space and the kernel address space. Signed-off-by: Liran Alon Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 30 ++++++++++++++++++++++++------ arch/x86/mm/tlb.c | 1 + include/linux/sched.h | 1 + 3 files changed, 26 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index db0a7ce..b0c789f 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -1383,11 +1383,13 @@ static bool kvm_page_fault(struct pt_regs *regs, unsigned long error_code, printk(KERN_DEFAULT "KVM isolation: page fault %ld at %pS on %lx (%pS) while switching mm\n" " cr3=%lx\n" " kvm_mm=%px pgd=%px\n" - " active_mm=%px pgd=%px\n", + " active_mm=%px pgd=%px\n" + " kvm_prev_mm=%px pgd=%px\n", error_code, (void *)regs->ip, address, (void *)address, cr3, &kvm_mm, kvm_mm.pgd, - active_mm, active_mm->pgd); + active_mm, active_mm->pgd, + current->kvm_prev_mm, current->kvm_prev_mm->pgd); dump_stack(); return false; @@ -1649,11 +1651,27 @@ void kvm_may_access_sensitive_data(struct kvm_vcpu *vcpu) kvm_isolation_exit(); } +static void kvm_switch_mm(struct mm_struct *mm) +{ + unsigned long flags; + + /* + * Disable interrupt before updating active_mm, otherwise if an + * interrupt occurs during the switch then the interrupt handler + * can be mislead about the mm effectively in use. + */ + local_irq_save(flags); + current->kvm_prev_mm = current->active_mm; + current->active_mm = mm; + switch_mm_irqs_off(current->kvm_prev_mm, mm, NULL); + local_irq_restore(flags); +} + void kvm_isolation_enter(void) { int err; - if (kvm_isolation()) { + if (kvm_isolation() && current->active_mm != &kvm_mm) { /* * Switches to kvm_mm should happen from vCPU thread, * which should not be a kernel thread with no mm @@ -1666,14 +1684,14 @@ void kvm_isolation_enter(void) current); return; } - /* TODO: switch to kvm_mm */ + kvm_switch_mm(&kvm_mm); } } void kvm_isolation_exit(void) { - if (kvm_isolation()) { + if (kvm_isolation() && current->active_mm == &kvm_mm) { /* TODO: Kick sibling hyperthread before switch to host mm */ - /* TODO: switch back to original mm */ + kvm_switch_mm(current->kvm_prev_mm); } } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index a4db7f5..7ad5ad1 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -444,6 +444,7 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, switch_ldt(real_prev, next); } } +EXPORT_SYMBOL_GPL(switch_mm_irqs_off); /* * Please ignore the name of this function. It should be called diff --git a/include/linux/sched.h b/include/linux/sched.h index 80e1d75..b03680d 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1202,6 +1202,7 @@ struct task_struct { #ifdef CONFIG_HAVE_KVM /* Is the task mapped into the KVM address space? */ bool kvm_mapped; + struct mm_struct *kvm_prev_mm; #endif /* From patchwork Mon May 13 14:38:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941103 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53D996C5 for ; Mon, 13 May 2019 14:40:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4597027861 for ; Mon, 13 May 2019 14:40:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 39C862834A; Mon, 13 May 2019 14:40:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 400AF27861 for ; Mon, 13 May 2019 14:40:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4EF986B0283; Mon, 13 May 2019 10:40:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4A2086B0284; Mon, 13 May 2019 10:40:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F4CE6B0285; Mon, 13 May 2019 10:40:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f199.google.com (mail-it1-f199.google.com [209.85.166.199]) by kanga.kvack.org (Postfix) with ESMTP id 098EB6B0283 for ; Mon, 13 May 2019 10:40:11 -0400 (EDT) Received: by mail-it1-f199.google.com with SMTP id z125so12373934itf.4 for ; Mon, 13 May 2019 07:40:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=WgSqF1N9AL8OkCOWJMjKijUQDrf/NZQ7sjOAy3KvX18=; b=m3QJnF/Rnvx0k+Zj7UTgzqxu0A1xm8FqueJXM0c67+GjhRIApbCJtJSdq0FfkNEu54 1AAuKStShGfvOp2XCdo7CzEPXOTp7eGtxVN7q1YpyyyxtDUVFyIIgicPVPAKzzxh9Ltt 9ICWkzkeJU73En1FUZ0pHu7Ll0yUZpo54qp4BG+v7fwg1P1GoppD8X9TbTW7WU7klzhk +zip7QVEKqMHWoLjJLME94x3BnL928GwrQ2AjL8JJZsLxVWC1Fjx2EsdMfmE4r7zkvdn GrgJS9QxiMWxxMWbexaMGgQ41xraSjUvXTpY1GBSF+33lXPHJnIZZ9kJz4oj9WmwwxHb qxzQ== X-Gm-Message-State: APjAAAUaK5M5ZHIfoN4vrMlNsHVlgWWRrzz/38zyT2xsj7ah42Sr2d0W 1ladJw194faQlce8xBloBET1Q/K3nJEv3IhTXkBnSpGGNx3EPSMUK9bmLebWgiHKj6vyq+5Qu1Y qTC3HbtzWqOtzI5HBvlW0LWvAFE5z9UJAGVgq0+QgXp3loxr3WHRRlvoy2VKuIlT/jg== X-Received: by 2002:a24:5491:: with SMTP id t139mr16658022ita.173.1557758410782; Mon, 13 May 2019 07:40:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqzVeLKx6PSPjXJYh1mLwqv2jmFke6Ov7fkO1P29FGmUxTc+YY3OfkU7+29bwYawV6lNgGhs X-Received: by 2002:a24:5491:: with SMTP id t139mr16657965ita.173.1557758409915; Mon, 13 May 2019 07:40:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758409; cv=none; d=google.com; s=arc-20160816; b=J1a/UvLzTlbzkX/1+MMO3XBUBjAT+iFgnHkRXjifSqc5JY/YHYDO5Prdt+1FlzZsq9 XXp+1fFVitopNOOVhgXdxPwrc/gpcP/vx4bMohyhgFRRkPKT9fwxl9r0RUEDL0TIbZnD DjGA0Rm7kWiJGAZHNigetGDBe1U75ZKpuBZwZ7jfpRJ8Ovmt+oAchCoe1fcANSdgEogM uDYgwMSSczEvnn27TYH+bGuXJ6f2c+GXYzYM/9BoKQ4UvlSY15fZUqHThBOuWdAbcz0+ hbqjGsEKxikMv4Mw7/23JZQRE42Nl31a+ZEDqqtVoh8ZH4QVZY2IYO0yruMOV3lPu6VT NZOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=WgSqF1N9AL8OkCOWJMjKijUQDrf/NZQ7sjOAy3KvX18=; b=rw0omjBWxhCwHjV9ahae/FcRtvBVQVlo7VgULxMpwRHLfpnbQLE5pnr3x3N6g7ZQkf HV0/mW3WvBRn1i4eY+PP4LQvVj4mAGBQxt5aAhRbn1R1X2NoFmsGICxk2t3YIBe2J3k7 yYvMHhv+xLJw0Pd9bgxv7UnL8k4FF56uryeY0bvvGrwesM9W17lz0TvNdpl9Aq09Rjei xRXgI4Io5WkR9a7o8KV9zWCKe9gRRZ09LnUwBYKpjegqe/xyaG1K6cGhcxMpNHHZ8cjk ldl/zkjKylr4slBPFhivITqMINdk/gyDwpDieWEfe8jZwgZJxP+m7E2NIXWkz9HL7XFP loag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=F3ixlg5D; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2130.oracle.com (aserp2130.oracle.com. [141.146.126.79]) by mx.google.com with ESMTPS id x15si8372480jaf.51.2019.05.13.07.40.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:40:09 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) client-ip=141.146.126.79; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=F3ixlg5D; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 141.146.126.79 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEdhZc193417; Mon, 13 May 2019 14:40:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=WgSqF1N9AL8OkCOWJMjKijUQDrf/NZQ7sjOAy3KvX18=; b=F3ixlg5Del/kZFpkm5V339YqcYZIvytCmmZa2JMsPdT9dTNJdzNgJr2su3ZaFPX7+xSo OgE5doXVsWLJE6lg24N70KQRsqrw7KDlX0Lmv11xaiukFvSmC1PL9Z+pb6KOsPd3f9/Q TysNHNbYW17DUsb88Xzm+0xLKL38OFY7GPCscqM5Sr2RGXWabgevtm0BY8LIkE1hebHi ZDG3ZUmmf/4/2FsM+aInuyKqWofpo40CdOK/HySYEBTuzbf6QiQlJM7oxHIAKfyG/xd8 7z7TkAUzLmadWyiur41dp0GSyhTRruaBBuRMWSLXH6YSLiSP0FUqhXsgwlGrUBetwxxp pQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp2130.oracle.com with ESMTP id 2sdkwdfm1k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:40:01 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQT022780; Mon, 13 May 2019 14:39:53 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 26/27] kvm/isolation: initialize the KVM page table with KVM memslots Date: Mon, 13 May 2019 16:38:34 +0200 Message-Id: <1557758315-12667-27-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=970 adultscore=15 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KVM memslots can change after they have been created so new memslots have to be mapped when they are created. TODO: we currently don't unmapped old memslots, they should be unmapped when they are freed. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 39 +++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 1 + arch/x86/kvm/x86.c | 3 +++ 3 files changed, 43 insertions(+), 0 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index b0c789f..255b2da 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -1593,13 +1593,45 @@ static void kvm_isolation_clear_handlers(void) kvm_page_fault_handler = NULL; } +void kvm_isolation_check_memslots(struct kvm *kvm) +{ + struct kvm_range_mapping *rmapping; + int i, err; + + if (!kvm_isolation()) + return; + + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { + rmapping = kvm_get_range_mapping(kvm->memslots[i], NULL); + if (rmapping) + continue; + pr_debug("remapping kvm memslots[%d]\n", i); + err = kvm_copy_ptes(kvm->memslots[i], + sizeof(struct kvm_memslots)); + if (err) + pr_debug("failed to map kvm memslots[%d]\n", i); + } + +} + int kvm_isolation_init_vm(struct kvm *kvm) { + int err, i; + if (!kvm_isolation()) return 0; kvm_clear_page_fault(); + pr_debug("mapping kvm memslots\n"); + + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { + err = kvm_copy_ptes(kvm->memslots[i], + sizeof(struct kvm_memslots)); + if (err) + return err; + } + pr_debug("mapping kvm srcu sda\n"); return (kvm_copy_percpu_mapping(kvm->srcu.sda, @@ -1608,9 +1640,16 @@ int kvm_isolation_init_vm(struct kvm *kvm) void kvm_isolation_destroy_vm(struct kvm *kvm) { + int i; + if (!kvm_isolation()) return; + pr_debug("unmapping kvm memslots\n"); + + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) + kvm_clear_range_mapping(kvm->memslots[i]); + pr_debug("unmapping kvm srcu sda\n"); kvm_clear_percpu_mapping(kvm->srcu.sda); diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 2d7d016..1e55799 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -32,6 +32,7 @@ static inline bool kvm_isolation(void) extern void kvm_clear_range_mapping(void *ptr); extern int kvm_copy_percpu_mapping(void *percpu_ptr, size_t size); extern void kvm_clear_percpu_mapping(void *percpu_ptr); +extern void kvm_isolation_check_memslots(struct kvm *kvm); extern int kvm_add_task_mapping(struct task_struct *tsk); extern void kvm_cleanup_task_mapping(struct task_struct *tsk); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e1cc3a6..7d98e9f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9438,6 +9438,7 @@ void kvm_arch_memslots_updated(struct kvm *kvm, u64 gen) * mmio generation may have reached its maximum value. */ kvm_mmu_invalidate_mmio_sptes(kvm, gen); + kvm_isolation_check_memslots(kvm); } int kvm_arch_prepare_memory_region(struct kvm *kvm, @@ -9537,6 +9538,8 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, */ if (change != KVM_MR_DELETE) kvm_mmu_slot_apply_flags(kvm, (struct kvm_memory_slot *) new); + + kvm_isolation_check_memslots(kvm); } void kvm_arch_flush_shadow_all(struct kvm *kvm) From patchwork Mon May 13 14:38:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Chartre X-Patchwork-Id: 10941107 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8FB0A1390 for ; Mon, 13 May 2019 14:40:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 815FA27861 for ; Mon, 13 May 2019 14:40:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 748CF2836F; Mon, 13 May 2019 14:40:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C70DC27861 for ; Mon, 13 May 2019 14:40:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5B946B0293; Mon, 13 May 2019 10:40:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AD9196B0294; Mon, 13 May 2019 10:40:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95BB76B0295; Mon, 13 May 2019 10:40:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f200.google.com (mail-it1-f200.google.com [209.85.166.200]) by kanga.kvack.org (Postfix) with ESMTP id 754896B0293 for ; Mon, 13 May 2019 10:40:14 -0400 (EDT) Received: by mail-it1-f200.google.com with SMTP id p23so12279171itc.7 for ; Mon, 13 May 2019 07:40:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=bVamqx26WofIGHx47Rok5tY+OeXg0YuA7C3C4gCjugU=; b=PcM0BrtnaYJPauU4M+QPFzHRSVHOMi9HhalkOvPQ+dsvJPACHAmLGkvgYUibRS6pDu 4IvzvjRBgo97H8RolSeegzyTd7/ifFUIln9HODzgWodr3luN+dV9wiuXmLg7y4Qb5PGe CLlFv9Ucyu2Ri2kRhN4xKradq2FaplcJbiy5d0hxHpSaA8qHRg7Dle0qn5j1iLxPLqUr Om5Y5qBQpJzpjzG+45TV1kwV1l3JCY+XqvVMFQ54ig7CCJu9tSuFghutNqKoTB+joTR9 xMOvKHL27ZbHRxI1Vk1G3nage7BXg2gp8pCvcXSc0+CHNg9aTnumlWqH7Fg7aHwFUC/z a7zA== X-Gm-Message-State: APjAAAUMO4IB6+6tl9lruLttMi35xrkr7Da9JdR8E7HX15JLri6+HJIb 0ObZ3YXq/Y6X70K+i9S76vWh/bTntz9AzcJZHxQpmSILdvybRqK8XnhYCiR9826hRVzv4NaPr68 8eTfCLMuJbTexhHW3UOsWAiTf9KN2d1t26K8AMBtJCxjryD6SwOAPrPaE2Zw1102MPg== X-Received: by 2002:a24:a088:: with SMTP id o130mr10510858ite.86.1557758414216; Mon, 13 May 2019 07:40:14 -0700 (PDT) X-Google-Smtp-Source: APXvYqzj5Ku0wlF057h5m6/H1UZiEKJu5IcutsDURxLQ8fcqi2GI6hAyObuoRS9dstQ0NTHALuwx X-Received: by 2002:a24:a088:: with SMTP id o130mr10510790ite.86.1557758413386; Mon, 13 May 2019 07:40:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1557758413; cv=none; d=google.com; s=arc-20160816; b=0Iwo/+tsGDDwMNP/cmgFnGXqzXDpbq2ks/O5IeT7PTxof/WbCiLXxN1AqRsKRTFvnG cXeechGbeEIxiL/TUM9tAElQr19ASzG45+GO7Aduqqmwvk3SbZKBeYQ8RofvNVeCL+np iCX+EhNU4iApwWj/SAVHokc6A20jOSgMsE+y3hMbLlOKKuTHiDnVHQjg/YnNu1d1j5V5 8N+3qSHEIg93eZeqFSVsJI4jHMovDhHD3QBpPYc6BVQ2D8iv4HmT/KEQrUMmf+pn9I+U qjHDw8m1t13HEUHl/V1jsSe8DbuB8mkNQhQPpw56/uE14D8dA/M4SkDwlvjmd3mVb9ku 91bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=bVamqx26WofIGHx47Rok5tY+OeXg0YuA7C3C4gCjugU=; b=L+3cpjyNOx0/4mdQlOZXsjLTWweay5qF9VWhzNiD42jTrHNAIZZ+iy5i/OVwU/IX+S E9+Kk4y1XE1/T5mGIS369+JVDVdt0nW9TtJaayJtfOXDV1Teoj1Ri7vAAoyZ6SaAx1K3 OM374jMDi0JuvPkbM3eJUkEB/f7dw7Gqu9IK4iEwO35exnZP7ch3TyXwOBK1+rux62oG 1yOQZd0A4VQVr+dhWCt3lQxanMmZRacbvMQEz6hekzoVJ+vLZITC2Dyqt2vG6jMQ2bG4 J7esZ++vLa7OI/BLHE6EtPdh7KzJzDN5zm6wGsznRy4sO3usPvk4isPTHMe1yvCLmHWI qSgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=phOTgVyR; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from userp2130.oracle.com (userp2130.oracle.com. [156.151.31.86]) by mx.google.com with ESMTPS id j63si8776272itb.19.2019.05.13.07.40.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 May 2019 07:40:13 -0700 (PDT) Received-SPF: pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) client-ip=156.151.31.86; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=phOTgVyR; spf=pass (google.com: domain of alexandre.chartre@oracle.com designates 156.151.31.86 as permitted sender) smtp.mailfrom=alexandre.chartre@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x4DEd2K2181455; Mon, 13 May 2019 14:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=bVamqx26WofIGHx47Rok5tY+OeXg0YuA7C3C4gCjugU=; b=phOTgVyRSmfJlShasjXR81DKx3htuPesB+i0d59gwRf4Pvj7Dim335VGDly6TaryasbH p79YnJpLHxV6ux2E6qwXe0qzqxgl9bjTmvmDzhmAzNh5lJ8rrhJ6ObfkWiFmfAq3lRJz K05CgDIVBI1/f6xo2nFMOruVDoE6XxhauS8j3PiqHmyqCTNH4xnpgnaVEUHrFTBXW7f2 VLrhtdpEyD7KijA3Esx4FX4nbFn/WkBHai46PD3BIoP4/Hkwmpdc6QjQUAlSMhB2yCbA Ui+mFwRtvI4wzRar5LmOK42XboaGtSbIhchHkhsgVIP9GVewhcqNUP1Lh2Qu6jIn+TwR /w== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2sdnttfemt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 May 2019 14:40:04 +0000 Received: from achartre-desktop.fr.oracle.com (dhcp-10-166-106-34.fr.oracle.com [10.166.106.34]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id x4DEcZQU022780; Mon, 13 May 2019 14:39:56 GMT From: Alexandre Chartre To: pbonzini@redhat.com, rkrcmar@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, kvm@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: konrad.wilk@oracle.com, jan.setjeeilers@oracle.com, liran.alon@oracle.com, jwadams@google.com, alexandre.chartre@oracle.com Subject: [RFC KVM 27/27] kvm/isolation: initialize the KVM page table with KVM buses Date: Mon, 13 May 2019 16:38:35 +0200 Message-Id: <1557758315-12667-28-git-send-email-alexandre.chartre@oracle.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> References: <1557758315-12667-1-git-send-email-alexandre.chartre@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9255 signatures=668686 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1905130103 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP KVM buses can change after they have been created so new buses have to be mapped when they are created. Signed-off-by: Alexandre Chartre --- arch/x86/kvm/isolation.c | 37 +++++++++++++++++++++++++++++++++++++ arch/x86/kvm/isolation.h | 1 + arch/x86/kvm/x86.c | 13 ++++++++++++- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 2 ++ 5 files changed, 53 insertions(+), 1 deletions(-) diff --git a/arch/x86/kvm/isolation.c b/arch/x86/kvm/isolation.c index 255b2da..329e769 100644 --- a/arch/x86/kvm/isolation.c +++ b/arch/x86/kvm/isolation.c @@ -1614,6 +1614,29 @@ void kvm_isolation_check_memslots(struct kvm *kvm) } +void kvm_isolation_check_buses(struct kvm *kvm) +{ + struct kvm_range_mapping *rmapping; + struct kvm_io_bus *bus; + int i, err; + + if (!kvm_isolation()) + return; + + for (i = 0; i < KVM_NR_BUSES; i++) { + bus = kvm->buses[i]; + rmapping = kvm_get_range_mapping(bus, NULL); + if (rmapping) + continue; + pr_debug("remapping kvm buses[%d]\n", i); + err = kvm_copy_ptes(bus, sizeof(*bus) + bus->dev_count * + sizeof(struct kvm_io_range)); + if (err) + pr_debug("failed to map kvm buses[%d]\n", i); + } + +} + int kvm_isolation_init_vm(struct kvm *kvm) { int err, i; @@ -1632,6 +1655,15 @@ int kvm_isolation_init_vm(struct kvm *kvm) return err; } + pr_debug("mapping kvm buses\n"); + + for (i = 0; i < KVM_NR_BUSES; i++) { + err = kvm_copy_ptes(kvm->buses[i], + sizeof(struct kvm_io_bus)); + if (err) + return err; + } + pr_debug("mapping kvm srcu sda\n"); return (kvm_copy_percpu_mapping(kvm->srcu.sda, @@ -1650,6 +1682,11 @@ void kvm_isolation_destroy_vm(struct kvm *kvm) for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) kvm_clear_range_mapping(kvm->memslots[i]); + pr_debug("unmapping kvm buses\n"); + + for (i = 0; i < KVM_NR_BUSES; i++) + kvm_clear_range_mapping(kvm->buses[i]); + pr_debug("unmapping kvm srcu sda\n"); kvm_clear_percpu_mapping(kvm->srcu.sda); diff --git a/arch/x86/kvm/isolation.h b/arch/x86/kvm/isolation.h index 1e55799..b048946 100644 --- a/arch/x86/kvm/isolation.h +++ b/arch/x86/kvm/isolation.h @@ -33,6 +33,7 @@ static inline bool kvm_isolation(void) extern int kvm_copy_percpu_mapping(void *percpu_ptr, size_t size); extern void kvm_clear_percpu_mapping(void *percpu_ptr); extern void kvm_isolation_check_memslots(struct kvm *kvm); +extern void kvm_isolation_check_buses(struct kvm *kvm); extern int kvm_add_task_mapping(struct task_struct *tsk); extern void kvm_cleanup_task_mapping(struct task_struct *tsk); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 7d98e9f..3ba1996 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9253,6 +9253,13 @@ void kvm_arch_sync_events(struct kvm *kvm) cancel_delayed_work_sync(&kvm->arch.kvmclock_sync_work); cancel_delayed_work_sync(&kvm->arch.kvmclock_update_work); kvm_free_pit(kvm); + /* + * Note that kvm_isolation_destroy_vm() has to be called from + * here, and not from kvm_arch_destroy_vm() because it will unmap + * buses which are already destroyed when kvm_arch_destroy_vm() + * is invoked. + */ + kvm_isolation_destroy_vm(kvm); } int __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, u32 size) @@ -9331,7 +9338,6 @@ void kvm_arch_destroy_vm(struct kvm *kvm) x86_set_memory_region(kvm, IDENTITY_PAGETABLE_PRIVATE_MEMSLOT, 0, 0); x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0); } - kvm_isolation_destroy_vm(kvm); if (kvm_x86_ops->vm_destroy) kvm_x86_ops->vm_destroy(kvm); kvm_pic_destroy(kvm); @@ -9909,6 +9915,11 @@ bool kvm_vector_hashing_enabled(void) } EXPORT_SYMBOL_GPL(kvm_vector_hashing_enabled); +void kvm_arch_buses_updated(struct kvm *kvm, struct kvm_io_bus *bus) +{ + kvm_isolation_check_buses(kvm); +} + EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_inj_virq); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ad24d9e..1291d8d 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -199,6 +199,7 @@ void kvm_io_bus_unregister_dev(struct kvm *kvm, enum kvm_bus bus_idx, struct kvm_io_device *dev); struct kvm_io_device *kvm_io_bus_get_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr); +void kvm_arch_buses_updated(struct kvm *kvm, struct kvm_io_bus *bus); #ifdef CONFIG_KVM_ASYNC_PF struct kvm_async_pf { diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 3c0c3db..374e79f 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3749,6 +3749,8 @@ int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, synchronize_srcu_expedited(&kvm->srcu); kfree(bus); + kvm_arch_buses_updated(kvm, new_bus); + return 0; }