From patchwork Tue Jan 18 11:06:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 12716175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 029D0C433EF for ; Tue, 18 Jan 2022 11:06:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238546AbiARLG5 (ORCPT ); Tue, 18 Jan 2022 06:06:57 -0500 Received: from mail-dm6nam08on2058.outbound.protection.outlook.com ([40.107.102.58]:50080 "EHLO NAM04-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238467AbiARLGz (ORCPT ); Tue, 18 Jan 2022 06:06:55 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bgA2jsS5hXB0eea7a9hQGhkQsUOxtTcmPBvv9tp7ihO+VDl7CIJb2PE2MpB3tE1m/oOyErJdyA8Qrx5b89CGW8s+gPWzmr678RSlQvpWRHIXk0RqnqPlm6cKsrjjF1wVOjTHc++w1SY8CR5qX0in4sqD2tuFGna7syUCqrygXk1TKOX1wjEMR6N37eGGwuk6VOYTunZo/6N8aEjW7lW9c3GZLmqPAx1gPIpfc2BAWL1aRvCk1CS+jlpRo2t3PGQ/eVS7mjgNxn32TchDmOFWYywgNWqlTXS/LgEH3nRkCbCtVswmOEU/oHR4r7TcVobMdoCYcn9T4mlHVQnjGZEMBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RUmfn8KV5dcGImEu8xWKxUf1N8/3Nk126wgHmM0KoLA=; b=BRbOyhCKDjJha3oIJTjSlenP0iBSSCZjFlQUsPFymzf7Ha54KGXbGz70Pg8DSLlTSXQW/JPHHeHWzG1u0jKPyyzcm9yx5NykNG8KqF3pOSrn/+rsnDvhYzEOocuujGxseCaEJFqVYyQPa32zTgHtNE9f9l7HRUop5BVFCKGraDkV2QFCcqEiuKVqKhKXViHQFxzKTw5i8lXA8sE/nddMV2g4KLRkXc6g7M4R4yk1FQnR/wQt/6SAcZ4sU7PfymoqRVH9WEOmOVnWBavfSsf+JiPenpeeiokfyf941JnT0Y8TJKA8qyBh8chd60Jmr717F2rhhuHJH6TySc9GlFp4oA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RUmfn8KV5dcGImEu8xWKxUf1N8/3Nk126wgHmM0KoLA=; b=S71cJu6iagZwP2AlSuVpP5VGtYm5lx812qNX6C/GBJUzLh+kbVIgDvQxE+ETItnazvuK4ErB1i80g52qSak2L9ubuJbh8D844VPXl3fSUn2ZEV8qAwny/B4/PO3DQKQUpIqmqAuAFjkuwOyKzDBdhQ9V338HlNXWn8965bLZcIE= Received: from CO1PR15CA0084.namprd15.prod.outlook.com (2603:10b6:101:20::28) by BL1PR12MB5079.namprd12.prod.outlook.com (2603:10b6:208:31a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.11; Tue, 18 Jan 2022 11:06:53 +0000 Received: from CO1NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:101:20:cafe::7a) by CO1PR15CA0084.outlook.office365.com (2603:10b6:101:20::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.7 via Frontend Transport; Tue, 18 Jan 2022 11:06:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT019.mail.protection.outlook.com (10.13.175.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4888.9 via Frontend Transport; Tue, 18 Jan 2022 11:06:52 +0000 Received: from gomati.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Tue, 18 Jan 2022 05:06:45 -0600 From: Nikunj A Dadhania To: Paolo Bonzini CC: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Brijesh Singh , Tom Lendacky , Peter Gonda , , , Nikunj A Dadhania Subject: [RFC PATCH 1/6] KVM: x86/mmu: Add hook to pin PFNs on demand in MMU Date: Tue, 18 Jan 2022 16:36:16 +0530 Message-ID: <20220118110621.62462-2-nikunj@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220118110621.62462-1-nikunj@amd.com> References: <20220118110621.62462-1-nikunj@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 51968e1e-7526-46c2-a451-08d9da72a1f7 X-MS-TrafficTypeDiagnostic: BL1PR12MB5079:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:935; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JXzDPNsa+nTk7GEwl2NtpTsb7hmxATbByoHPKNx65x25d99KELYyI4kyXKRODt7lcea9QLFjcUbxKdraSL1TQ9ko4JeQDpANTWdBRC46p74fz8ITC01ffh98PEbf4jYKi9IjEnHCYAhcZLkT82gn50addN7UIii/ImnJ+fMgiuSby7loDfRiNg6e+RIn1mQlloiVu+LB7a7X6tNgCAgGGkmq2yqkUtt/xT3XSQMZcljAQN94OQM5TNfjxSg7df0AsplWDy7Z31gGRngPUO7oWO9Xoyg7b9tjXF/LKCmXtxebtrvFe2/Ft40IspfkaMKfzTWkXLHsT2lpnsz3OiNU2z3lPwkq+C0FLtOsXN/7Mjgd42OnMVZMbnznpcMT96JgwB/7fNSRXVNWCxi+7shTBxPS909hJga0epAnaSdiuvf81engc0aJn6SDH7S5624N1uIpwEXMNpc/LXe1oWxyKpnq9ZIysZgepDSGsEZ0/fZfKSXyz3SZIifui5VkqT5QTuYD0YBp36eMEeis3GfViKW9l6eHwKIXAzh2n0eGq2VKS1Nk/5Yj6+22Nkv/TPrqzMEpJZvU34ATLk6ap/TJcvUwKi1aWoT3VzporScJygajIb2VX7h5XDAStsE3T0znFROnBGL+8DMHUQNzuOjhZd/4WzbnguHn+QlS9Oz7+jvJQcZoBKJzt/asLTdMfFFsRCE3ELe7J21Qc9ronug/p75hjjOd9bojILicnWgJ7MPy/j3j8cMh06IHeqncdnRiLvGKp6AOy60XYm505cNBb2UPWeS7BgdeqHNzCyCgj6k= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(46966006)(40470700002)(36840700001)(26005)(2906002)(36756003)(4326008)(5660300002)(316002)(81166007)(70206006)(70586007)(426003)(16526019)(6666004)(82310400004)(2616005)(186003)(6916009)(47076005)(40460700001)(36860700001)(8936002)(356005)(336012)(7696005)(8676002)(1076003)(508600001)(54906003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2022 11:06:52.6405 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 51968e1e-7526-46c2-a451-08d9da72a1f7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5079 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use vendor code via kvm_x86_ops hooks for pinning. Signed-off-by: Nikunj A Dadhania --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu/mmu.c | 3 +++ arch/x86/kvm/mmu/tdp_mmu.c | 7 +++++++ 4 files changed, 13 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index f658bb4dbb74..a96c52a99a04 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -87,6 +87,7 @@ KVM_X86_OP(set_tss_addr) KVM_X86_OP(set_identity_map_addr) KVM_X86_OP(get_mt_mask) KVM_X86_OP(load_mmu_pgd) +KVM_X86_OP(pin_spte) KVM_X86_OP_NULL(has_wbinvd_exit) KVM_X86_OP(get_l2_tsc_offset) KVM_X86_OP(get_l2_tsc_multiplier) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 0677b9ea01c9..1263a16dd588 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1417,6 +1417,8 @@ struct kvm_x86_ops { void (*load_mmu_pgd)(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); + void (*pin_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level, + kvm_pfn_t pfn); bool (*has_wbinvd_exit)(void); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1d275e9d76b5..62dda588eb99 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2977,6 +2977,9 @@ static int __direct_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) return ret; direct_pte_prefetch(vcpu, it.sptep); + if (!is_error_noslot_pfn(fault->pfn) && !kvm_is_reserved_pfn(fault->pfn)) + static_call_cond(kvm_x86_pin_spte)(vcpu->kvm, base_gfn, + it.level, fault->pfn); ++vcpu->stat.pf_fixed; return ret; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 7b1bc816b7c3..b7578fa02e9f 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -467,6 +467,13 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, (!is_present || !is_dirty_spte(new_spte) || pfn_changed)) kvm_set_pfn_dirty(spte_to_pfn(old_spte)); + /* + * Call the vendor code to handle the pinning + */ + if (is_present && is_leaf) + static_call_cond(kvm_x86_pin_spte)(kvm, gfn, level, + spte_to_pfn(new_spte)); + /* * Recursively handle child PTs if the change removed a subtree from * the paging structure. From patchwork Tue Jan 18 11:06:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 12716176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF199C433EF for ; Tue, 18 Jan 2022 11:07:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238649AbiARLG7 (ORCPT ); Tue, 18 Jan 2022 06:06:59 -0500 Received: from mail-mw2nam10on2055.outbound.protection.outlook.com ([40.107.94.55]:15965 "EHLO NAM10-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238550AbiARLG5 (ORCPT ); Tue, 18 Jan 2022 06:06:57 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WtS1npYODLFOyJLMNNUk1hq4sMMZx+fl08JOXy4H+vjKHmRGI1YwbZ5ER7PfPG6iMbuPqJzSh8Uwx0y4SmAwyVliIO3E3Ua0FA8ouMAovPYe8h184XujAl3yDGhYJIaGrVtp8nQyGpdPIS7PQuq51lfGPqC/VKZql9XWGBKnnxOITlw7pvX5tu8RNPXbS3e2aGBP481pM/y6a7a7ZvHLrlNXUcCc1AS9jbXIiaZ7BKSX5bi4N+5YdrYwQCGIy4O8uBuoN2BBWj1XPmfDBwcFfz4KWT5ideOOF7lDjbrpbMCvEYm5cfRebL6Vxo/ne6m1QwjylE3+VRnKFqSrqz8FEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2UKpjyRB8zKmbZhd9QXnlWZp/aC7b7ZPgU/15zOu8ow=; b=I4g9osP+CvLjlsMGMcSi0sYnPxrXc94fV1Oyg1NDwrBzj4XO02A2CPCU0mO8F5J7UaxHwFaQP9iDrF1F3hAdtfFrVU8UJde9vsPV8nfwg0AP6Yia/UZgHprXMqCMga0HHB4L8x7+Mrig4ZeWaFKBXbwPnsGv2PWVKRlLxllAEMuzbEd7e9aJZiPKlTZcz7Pv6Y5eiLHNIhmfU70TElVXuMx0SmwztzMySn5Ldy/z+M2bZFms/wuOGRUDl3YcFeDSMYbbSnztYliiRRchnIGCUWS4pWAmOwh0ECck/+AHgnn27HaWRTRuWdDvjUjoGoiLAC5s3HwUyxtoay30H5EwFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2UKpjyRB8zKmbZhd9QXnlWZp/aC7b7ZPgU/15zOu8ow=; b=vYAdpf5CTuiMNfjgIw1uBeeX47IsALzQ0t46lmVSqTW4GYgY0iazn9CBX42D73JDDMf5toBMCJaVkE/PpHuELpeSNtwnZGJLcneuGH5vWWhd22melp6c8hhEuukVbQVYXKmizPODNsFOw8CZarMJ+jlTO71JCaDi6bD4miKXDTE= Received: from CO1PR15CA0068.namprd15.prod.outlook.com (2603:10b6:101:20::12) by DM4PR12MB5341.namprd12.prod.outlook.com (2603:10b6:5:39e::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.11; Tue, 18 Jan 2022 11:06:55 +0000 Received: from CO1NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:101:20:cafe::4c) by CO1PR15CA0068.outlook.office365.com (2603:10b6:101:20::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.9 via Frontend Transport; Tue, 18 Jan 2022 11:06:55 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT019.mail.protection.outlook.com (10.13.175.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4888.9 via Frontend Transport; Tue, 18 Jan 2022 11:06:54 +0000 Received: from gomati.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Tue, 18 Jan 2022 05:06:49 -0600 From: Nikunj A Dadhania To: Paolo Bonzini CC: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Brijesh Singh , Tom Lendacky , Peter Gonda , , , Nikunj A Dadhania Subject: [RFC PATCH 2/6] KVM: SVM: Add pinning metadata in the arch memslot Date: Tue, 18 Jan 2022 16:36:17 +0530 Message-ID: <20220118110621.62462-3-nikunj@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220118110621.62462-1-nikunj@amd.com> References: <20220118110621.62462-1-nikunj@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3fc57edb-5c50-46f4-3ff8-08d9da72a35d X-MS-TrafficTypeDiagnostic: DM4PR12MB5341:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:751; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sWx8k7HcSjAvXgLyTqoItFEGxRaGK3WBvVFjtNSUS6lGM/OlK4K1PBL3kk7VfOkSO8tPMm6rYOy4BfbI3NidDqYnKi4yKkyTxkm3gT2TswdtNQYOpZbUr0gua8PuJYc6HIKefIYHKhtszpyM+oNO0hhOwJFW9cfJTyTBj690fRl9CxEEYWDnFBcpgvAA3BvpPdCuAPvRhjRY1QwzAUp/NSwy5TUD1Ly2AOaidpzSiBhzdIP7+m5vSluKFJ76igLYJ4K1jSCbq6j1scIbeY+/B/wireUfu0S4SngrcWpPvuHRIbOV8ZJoB5D3d4U9Y7HtjogufBCyZRRts0yWQUVmCCm6ZdEo4zb17enCLWudqgU2LCLzaVa4BktynNP8Mx9Fx+zh+TVAhClL+kTkPSXY7HRKpiENbBVzGVKd+u8Gr6F+rC+nChE6AO4fHFpXSRdV66/GnUUS2YOrsX9Q+4vPRsf1QUia2pkUHlDw98UijuOB6qU+Yb7sjEu1/Os4e5WEhB8G37CrjTEqVhDFhg4MGYNCsLTu2J500KWn6UAE2/IfuV/sXccfqDiBtPDeDLx+16bkdK/AcDDu9Cp7lB2fGHcsn71kl0FQB4tLmFo6dlwbfwVF/k14zRb8FmFC75BZeKthGBSOpyQ5EIHnwDcz7KiOvtjBKi3LBVswlq/zWirquaovjagf+CFxK96rlHjB++2sk0PIIeMfaKy5+DrxYv3svy9WYrccylQ/EcKPfhiiiwgghJIQ5ECJFGxiHQJ7YwgRvRtDY+Le2NCxy/F0bSHjPyC2CQwHJQv+ZshJwmo= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(40470700002)(46966006)(40460700001)(426003)(508600001)(6666004)(5660300002)(356005)(1076003)(336012)(26005)(81166007)(4326008)(16526019)(186003)(36756003)(2616005)(36860700001)(82310400004)(6916009)(8936002)(70206006)(7696005)(83380400001)(70586007)(8676002)(54906003)(47076005)(2906002)(316002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2022 11:06:54.9841 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3fc57edb-5c50-46f4-3ff8-08d9da72a35d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5341 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org AMD SEV guest requires the guest's pages to be pinned in host physical memory. The memory encryption scheme uses the physical address of the memory being encrypted. If guest pages are moved, content decrypted would be incorrect, corrupting guest's memory. For SEV/SEV-ES guests, the hypervisor doesn't know which pages are encrypted and when the guest is done using those pages. Hypervisor should treat all the guest pages as encrypted until the guest is destroyed. The KVM MMU needs to track the pages that are pinned and the corresponding pfns for unpinning them during the guest destroy path. Signed-off-by: Nikunj A Dadhania --- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 7 +++++ arch/x86/kvm/svm/sev.c | 49 ++++++++++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 3 ++ arch/x86/kvm/svm/svm.h | 6 ++++ arch/x86/kvm/x86.c | 11 ++++++- 6 files changed, 77 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index a96c52a99a04..da03250f503c 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -87,6 +87,8 @@ KVM_X86_OP(set_tss_addr) KVM_X86_OP(set_identity_map_addr) KVM_X86_OP(get_mt_mask) KVM_X86_OP(load_mmu_pgd) +KVM_X86_OP(alloc_memslot_metadata) +KVM_X86_OP(free_memslot) KVM_X86_OP(pin_spte) KVM_X86_OP_NULL(has_wbinvd_exit) KVM_X86_OP(get_l2_tsc_offset) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1263a16dd588..c235597f8442 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -927,6 +927,8 @@ struct kvm_arch_memory_slot { struct kvm_rmap_head *rmap[KVM_NR_PAGE_SIZES]; struct kvm_lpage_info *lpage_info[KVM_NR_PAGE_SIZES - 1]; unsigned short *gfn_track[KVM_PAGE_TRACK_MAX]; + unsigned long *pinned_bitmap; + kvm_pfn_t *pfns; }; /* @@ -1417,6 +1419,11 @@ struct kvm_x86_ops { void (*load_mmu_pgd)(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); + int (*alloc_memslot_metadata)(struct kvm *kvm, + const struct kvm_memory_slot *old, + struct kvm_memory_slot *new); + void (*free_memslot)(struct kvm *kvm, + struct kvm_memory_slot *slot); void (*pin_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level, kvm_pfn_t pfn); diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 6a22798eaaee..d972ab4956d4 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -2945,3 +2945,52 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, 1); } + +void sev_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) +{ + struct kvm_arch_memory_slot *aslot = &slot->arch; + + if (!sev_guest(kvm)) + return; + + if (aslot->pinned_bitmap) { + kvfree(aslot->pinned_bitmap); + aslot->pinned_bitmap = NULL; + } + + if (aslot->pfns) { + kvfree(aslot->pfns); + aslot->pfns = NULL; + } +} + +int sev_alloc_memslot_metadata(struct kvm *kvm, + const struct kvm_memory_slot *old, + struct kvm_memory_slot *new) +{ + struct kvm_arch_memory_slot *aslot = &new->arch; + unsigned long pinned_bytes = new->npages * sizeof(kvm_pfn_t); + + if (!sev_guest(kvm)) + return 0; + + if (old && old->arch.pinned_bitmap && old->arch.pfns) { + WARN_ON(old->npages != new->npages); + aslot->pinned_bitmap = old->arch.pinned_bitmap; + aslot->pfns = old->arch.pfns; + return 0; + } + + aslot->pfns = kvcalloc(new->npages, sizeof(*aslot->pfns), + GFP_KERNEL_ACCOUNT); + if (!aslot->pfns) + return -ENOMEM; + + aslot->pinned_bitmap = kvzalloc(pinned_bytes, GFP_KERNEL_ACCOUNT); + if (!aslot->pinned_bitmap) { + kvfree(aslot->pfns); + aslot->pfns = NULL; + return -ENOMEM; + } + return 0; +} diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 46bcc706f257..3fb19974f719 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4740,6 +4740,9 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .complete_emulated_msr = svm_complete_emulated_msr, .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector, + + .alloc_memslot_metadata = sev_alloc_memslot_metadata, + .free_memslot = sev_free_memslot, }; static struct kvm_x86_init_ops svm_init_ops __initdata = { diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 9f153c59f2c8..b2f8b3b52680 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -643,4 +643,10 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm); void __svm_sev_es_vcpu_run(unsigned long vmcb_pa); void __svm_vcpu_run(unsigned long vmcb_pa, unsigned long *regs); +int sev_alloc_memslot_metadata(struct kvm *kvm, + const struct kvm_memory_slot *old, + struct kvm_memory_slot *new); +void sev_free_memslot(struct kvm *kvm, + struct kvm_memory_slot *slot); + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 76b4803dd3bd..9e07e2ef8885 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11694,6 +11694,7 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) } kvm_page_track_free_memslot(slot); + static_call_cond(kvm_x86_free_memslot)(kvm, slot); } int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages) @@ -11719,6 +11720,7 @@ int memslot_rmap_alloc(struct kvm_memory_slot *slot, unsigned long npages) } static int kvm_alloc_memslot_metadata(struct kvm *kvm, + const struct kvm_memory_slot *old, struct kvm_memory_slot *slot) { unsigned long npages = slot->npages; @@ -11771,8 +11773,15 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm, if (kvm_page_track_create_memslot(kvm, slot, npages)) goto out_free; + if (kvm_x86_ops.alloc_memslot_metadata && + static_call(kvm_x86_alloc_memslot_metadata)(kvm, old, slot)) + goto out_free_page_track; + return 0; +out_free_page_track: + kvm_page_track_free_memslot(slot); + out_free: memslot_rmap_free(slot); @@ -11805,7 +11814,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, enum kvm_mr_change change) { if (change == KVM_MR_CREATE || change == KVM_MR_MOVE) - return kvm_alloc_memslot_metadata(kvm, new); + return kvm_alloc_memslot_metadata(kvm, old, new); if (change == KVM_MR_FLAGS_ONLY) memcpy(&new->arch, &old->arch, sizeof(old->arch)); From patchwork Tue Jan 18 11:06:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 12716178 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94875C433F5 for ; Tue, 18 Jan 2022 11:07:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238467AbiARLHH (ORCPT ); Tue, 18 Jan 2022 06:07:07 -0500 Received: from mail-co1nam11on2045.outbound.protection.outlook.com ([40.107.220.45]:63264 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238725AbiARLHE (ORCPT ); Tue, 18 Jan 2022 06:07:04 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fbL0xYKaDNAnry1atOMQAxzH+I6X9EBtsicGb5mlmruQTFvgHcaeXD3XV/zoGM4D67cAzodvilfi7dOUGRYCen4I16YHHCtQcc1ubIbXtIiZCbfolHMhqJLhq66fEUCYtApWFG1DlfKUZy6ZMKqr7uzqyAiXk3hRbERTksDGv2hSOvQKj7g4rgqxiu504VBuogDA5GNfwF52CVgigG9Ohtx19Jx3uTfR6cjpWWJv/HcpiCgafw6sRbNiXaD2ohT/lPf6M5YW+os35+ww85hHUopiJcgLTSEdQ9BlWMy3YF+6O0mJSJcz1HaEJUd3E04y6uOkR1zlDRh6opL4NM3e0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2HzJFzzY7RK0oQddNpfeGm2SoJJTaCj8YLnyvXw1h84=; b=ms2HwZQEeu/yn14s3bbQiwjbqLMDJUAKidk1Gi1G8RA1crFg7Arg5QQ4ZzIBUzbfhwQZjhX5itAePQf8ExEC71HqPFVkmwy1ymJSrGRdKApMsv0kXjqd+py7c1Pi4RBAlW+yI9EnK0y8GxkbjfFM65do9TlUhpnR3tZc0sXi9WK6yF0naaZ5aj9pzg0xdzayKgy512G1hqIIbhJK1IbDwUlI69pkma1gR4nvZx6PkImzpMonMiRjrwWMwmDjszsOpAb5nwDkgGVVLxZk0KsK/lj8tDrEFt2u9oowpd6Tc8q3qWd39w+hF1OKYfkzHCUUdiOoXfEu9OPJGbkHLO4DsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2HzJFzzY7RK0oQddNpfeGm2SoJJTaCj8YLnyvXw1h84=; b=yyXx6aHjDGVkhWSY005omyApEMzX4DxUxbsPsfUiZYxC+315xHUh6Bxbue+D5f6LRVN8/JS2DmreSFVgkW2nxhOX90CUf8jFYGVxUbnXrkPdMtp3SFHdeHbLKikZlaIINYcF7n3SzGS17r6PhTpE6OYkv7qYigNtIv+/OZZ7eBI= Received: from CO1PR15CA0077.namprd15.prod.outlook.com (2603:10b6:101:20::21) by BN7PR12MB2788.namprd12.prod.outlook.com (2603:10b6:408:30::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.9; Tue, 18 Jan 2022 11:06:58 +0000 Received: from CO1NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:101:20:cafe::bf) by CO1PR15CA0077.outlook.office365.com (2603:10b6:101:20::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.10 via Frontend Transport; Tue, 18 Jan 2022 11:06:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT019.mail.protection.outlook.com (10.13.175.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4888.9 via Frontend Transport; Tue, 18 Jan 2022 11:06:57 +0000 Received: from gomati.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Tue, 18 Jan 2022 05:06:53 -0600 From: Nikunj A Dadhania To: Paolo Bonzini CC: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Brijesh Singh , Tom Lendacky , Peter Gonda , , , Nikunj A Dadhania Subject: [RFC PATCH 3/6] KVM: SVM: Implement demand page pinning Date: Tue, 18 Jan 2022 16:36:18 +0530 Message-ID: <20220118110621.62462-4-nikunj@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220118110621.62462-1-nikunj@amd.com> References: <20220118110621.62462-1-nikunj@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d089fb51-c9a4-4ab6-d5ea-08d9da72a51f X-MS-TrafficTypeDiagnostic: BN7PR12MB2788:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2150; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 75qHYO9c/19BQmWGOlpAs0RKb0EzxX6tOCXOeKoDze97mUB3Ns91AYqJNQkQopZjGPTVHTx46Hoo+woVDBLgDNgHLY7OSh6uXc7tqlot5eOeOWGGE2keVn3YgQBRmFw6tAbfI/C4n3xFbqW8ac6tfl8eRbw/Q7wnUj3N9ksbx9TjFg/WQhG2O8C3YhGxi+AtNLzxjzTQMRniMMSQKui55yQYgyEW/EmSk7KYjXlo6gQRjKz1l0bR7UzZ3sIxJqrWCGH61skr+UzSWZuuBo6YARItYDovkmrX8nXDm4u0fVj8LXMjY+uVpvh3hKMkIMCdtdLVXkC6311GRUgtiCsCBl7BPIx/Fq3nIxwg32x/rojHgDdN4ZNloTSO2Wnwk4WiWiksd0pLEcKcXwceIU1GCzvQw6/GilRnwHfSzNWhnRE2yx6bgtdKDRBSQOM14iwFzph6I0IqCMAfH/lG16DuBSZe+67Md34gwnR6LQ+wWG4iPIiWHSIMoisyRwFOwni6/QO4JMfP7pjWzCmlOAJ0eBnfG5diPcgmN/U3eYcEEySCxwuUgZXGwPP+pvJRHJUqUzL2hLOpNk3Riy0EKtQ39kA6/8XlpfnFoYiUQznzAzAz/XnesKg7GzPMKcds74Rty3BbQW3ibEBXIDYArrBW6q0lQEIXZ64bUND3+PZYdBySDYro30xweOJ+UASAA5txn9TTMFLhy01UsX8+43+d/m67dno52enRYiaFOADFQ+8VsdC2HZDJJ4LjT1PAlcA50iToztcWxVA98VFDqgVzbcBpGcfGwX3xk3WbGuZWHuA= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(40470700002)(47076005)(5660300002)(356005)(16526019)(82310400004)(508600001)(316002)(186003)(6916009)(6666004)(54906003)(83380400001)(1076003)(8936002)(4326008)(26005)(40460700001)(70586007)(36756003)(7696005)(70206006)(336012)(81166007)(2616005)(36860700001)(8676002)(2906002)(426003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2022 11:06:57.9371 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d089fb51-c9a4-4ab6-d5ea-08d9da72a51f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR12MB2788 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use the memslot metadata to store the pinned data along with the pfns. This improves the SEV guest startup time from O(n) to a constant by deferring guest page pinning until the pages are used to satisfy nested page faults. The page reference will be dropped in the memslot free path. Remove the enc_region structure definition and the code which did upfront pinning, as they are no longer needed in view of the demand pinning support. Leave svm_register_enc_region() and svm_unregister_enc_region() as stubs since qemu is dependent on this API. Signed-off-by: Nikunj A Dadhania --- arch/x86/kvm/svm/sev.c | 208 ++++++++++++++++------------------------- arch/x86/kvm/svm/svm.c | 1 + arch/x86/kvm/svm/svm.h | 3 +- 3 files changed, 81 insertions(+), 131 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index d972ab4956d4..a962bed97a0b 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -66,14 +66,6 @@ static unsigned int nr_asids; static unsigned long *sev_asid_bitmap; static unsigned long *sev_reclaim_asid_bitmap; -struct enc_region { - struct list_head list; - unsigned long npages; - struct page **pages; - unsigned long uaddr; - unsigned long size; -}; - /* Called with the sev_bitmap_lock held, or on shutdown */ static int sev_flush_asids(int min_asid, int max_asid) { @@ -257,8 +249,6 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp) if (ret) goto e_free; - INIT_LIST_HEAD(&sev->regions_list); - return 0; e_free: @@ -1637,8 +1627,6 @@ static void sev_migrate_from(struct kvm_sev_info *dst, src->handle = 0; src->pages_locked = 0; src->enc_context_owner = NULL; - - list_cut_before(&dst->regions_list, &src->regions_list, &src->regions_list); } static int sev_es_migrate_from(struct kvm *dst, struct kvm *src) @@ -1861,115 +1849,13 @@ int svm_mem_enc_op(struct kvm *kvm, void __user *argp) int svm_register_enc_region(struct kvm *kvm, struct kvm_enc_region *range) { - struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; - struct enc_region *region; - int ret = 0; - - if (!sev_guest(kvm)) - return -ENOTTY; - - /* If kvm is mirroring encryption context it isn't responsible for it */ - if (is_mirroring_enc_context(kvm)) - return -EINVAL; - - if (range->addr > ULONG_MAX || range->size > ULONG_MAX) - return -EINVAL; - - region = kzalloc(sizeof(*region), GFP_KERNEL_ACCOUNT); - if (!region) - return -ENOMEM; - - mutex_lock(&kvm->lock); - region->pages = sev_pin_memory(kvm, range->addr, range->size, ®ion->npages, 1); - if (IS_ERR(region->pages)) { - ret = PTR_ERR(region->pages); - mutex_unlock(&kvm->lock); - goto e_free; - } - - region->uaddr = range->addr; - region->size = range->size; - - list_add_tail(®ion->list, &sev->regions_list); - mutex_unlock(&kvm->lock); - - /* - * The guest may change the memory encryption attribute from C=0 -> C=1 - * or vice versa for this memory range. Lets make sure caches are - * flushed to ensure that guest data gets written into memory with - * correct C-bit. - */ - sev_clflush_pages(region->pages, region->npages); - - return ret; - -e_free: - kfree(region); - return ret; -} - -static struct enc_region * -find_enc_region(struct kvm *kvm, struct kvm_enc_region *range) -{ - struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; - struct list_head *head = &sev->regions_list; - struct enc_region *i; - - list_for_each_entry(i, head, list) { - if (i->uaddr == range->addr && - i->size == range->size) - return i; - } - - return NULL; -} - -static void __unregister_enc_region_locked(struct kvm *kvm, - struct enc_region *region) -{ - sev_unpin_memory(kvm, region->pages, region->npages); - list_del(®ion->list); - kfree(region); + return 0; } int svm_unregister_enc_region(struct kvm *kvm, struct kvm_enc_region *range) { - struct enc_region *region; - int ret; - - /* If kvm is mirroring encryption context it isn't responsible for it */ - if (is_mirroring_enc_context(kvm)) - return -EINVAL; - - mutex_lock(&kvm->lock); - - if (!sev_guest(kvm)) { - ret = -ENOTTY; - goto failed; - } - - region = find_enc_region(kvm, range); - if (!region) { - ret = -EINVAL; - goto failed; - } - - /* - * Ensure that all guest tagged cache entries are flushed before - * releasing the pages back to the system for use. CLFLUSH will - * not do this, so issue a WBINVD. - */ - wbinvd_on_all_cpus(); - - __unregister_enc_region_locked(kvm, region); - - mutex_unlock(&kvm->lock); return 0; - -failed: - mutex_unlock(&kvm->lock); - return ret; } int svm_vm_copy_asid_from(struct kvm *kvm, unsigned int source_fd) @@ -2018,7 +1904,6 @@ int svm_vm_copy_asid_from(struct kvm *kvm, unsigned int source_fd) mirror_sev->fd = source_sev->fd; mirror_sev->es_active = source_sev->es_active; mirror_sev->handle = source_sev->handle; - INIT_LIST_HEAD(&mirror_sev->regions_list); ret = 0; /* @@ -2038,8 +1923,6 @@ int svm_vm_copy_asid_from(struct kvm *kvm, unsigned int source_fd) void sev_vm_destroy(struct kvm *kvm) { struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; - struct list_head *head = &sev->regions_list; - struct list_head *pos, *q; WARN_ON(sev->num_mirrored_vms); @@ -2066,18 +1949,6 @@ void sev_vm_destroy(struct kvm *kvm) */ wbinvd_on_all_cpus(); - /* - * if userspace was terminated before unregistering the memory regions - * then lets unpin all the registered memory. - */ - if (!list_empty(head)) { - list_for_each_safe(pos, q, head) { - __unregister_enc_region_locked(kvm, - list_entry(pos, struct enc_region, list)); - cond_resched(); - } - } - sev_unbind_asid(kvm, sev->handle); sev_asid_free(sev); } @@ -2946,13 +2817,90 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, 1); } +void sev_pin_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level, + kvm_pfn_t pfn) +{ + struct kvm_arch_memory_slot *aslot; + struct kvm_memory_slot *slot; + gfn_t rel_gfn, pin_pfn; + unsigned long npages; + kvm_pfn_t old_pfn; + int i; + + if (!sev_guest(kvm)) + return; + + if (WARN_ON_ONCE(is_error_noslot_pfn(pfn) || kvm_is_reserved_pfn(pfn))) + return; + + /* Tested till 1GB pages */ + if (KVM_BUG_ON(level > PG_LEVEL_1G, kvm)) + return; + + slot = gfn_to_memslot(kvm, gfn); + if (!slot || !slot->arch.pfns) + return; + + /* + * Use relative gfn index within the memslot for the bitmap as well as + * the pfns array + */ + rel_gfn = gfn - slot->base_gfn; + aslot = &slot->arch; + pin_pfn = pfn; + npages = KVM_PAGES_PER_HPAGE(level); + + /* Pin the page, KVM doesn't yet support page migration. */ + for (i = 0; i < npages; i++, rel_gfn++, pin_pfn++) { + if (test_bit(rel_gfn, aslot->pinned_bitmap)) { + old_pfn = aslot->pfns[rel_gfn]; + if (old_pfn == pin_pfn) + continue; + + put_page(pfn_to_page(old_pfn)); + } + + set_bit(rel_gfn, aslot->pinned_bitmap); + aslot->pfns[rel_gfn] = pin_pfn; + get_page(pfn_to_page(pin_pfn)); + } + + /* + * Flush any cached lines of the page being added since "ownership" of + * it will be transferred from the host to an encrypted guest. + */ + clflush_cache_range(__va(pfn << PAGE_SHIFT), page_level_size(level)); +} + void sev_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) { struct kvm_arch_memory_slot *aslot = &slot->arch; + kvm_pfn_t *pfns; + gfn_t gfn; + int i; if (!sev_guest(kvm)) return; + if (!aslot->pinned_bitmap || !slot->arch.pfns) + goto out; + + pfns = aslot->pfns; + + /* + * Iterate the memslot to find the pinned pfn using the bitmap and drop + * the pfn stored. + */ + for (i = 0, gfn = slot->base_gfn; i < slot->npages; i++, gfn++) { + if (test_and_clear_bit(i, aslot->pinned_bitmap)) { + if (WARN_ON(!pfns[i])) + continue; + + put_page(pfn_to_page(pfns[i])); + } + } + +out: if (aslot->pinned_bitmap) { kvfree(aslot->pinned_bitmap); aslot->pinned_bitmap = NULL; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 3fb19974f719..22535c680b3f 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4743,6 +4743,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .alloc_memslot_metadata = sev_alloc_memslot_metadata, .free_memslot = sev_free_memslot, + .pin_spte = sev_pin_spte, }; static struct kvm_x86_init_ops svm_init_ops __initdata = { diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index b2f8b3b52680..c731bc91ea8f 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -77,7 +77,6 @@ struct kvm_sev_info { unsigned int handle; /* SEV firmware handle */ int fd; /* SEV device fd */ unsigned long pages_locked; /* Number of pages locked */ - struct list_head regions_list; /* List of registered regions */ u64 ap_jump_table; /* SEV-ES AP Jump Table address */ struct kvm *enc_context_owner; /* Owner of copied encryption context */ unsigned long num_mirrored_vms; /* Number of VMs sharing this ASID */ @@ -648,5 +647,7 @@ int sev_alloc_memslot_metadata(struct kvm *kvm, struct kvm_memory_slot *new); void sev_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot); +void sev_pin_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level, + kvm_pfn_t pfn); #endif From patchwork Tue Jan 18 11:06:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 12716177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB9C2C433FE for ; Tue, 18 Jan 2022 11:07:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238830AbiARLHI (ORCPT ); Tue, 18 Jan 2022 06:07:08 -0500 Received: from mail-bn7nam10on2053.outbound.protection.outlook.com ([40.107.92.53]:27488 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238757AbiARLHE (ORCPT ); Tue, 18 Jan 2022 06:07:04 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WniC4Kvp+xXlkKAPLA4qsmfbZRZjDEy0FDagskJYWJM3uMJ0klcjw6wxlbTrnN20JP9HbklnN2oJE/ii+EV+Q95bNcPf53RseSFFdt1D/1x5qIYXWm5XsqevGuFVU/qS+40YVwYqNPWHtQ9LMtRh3a5i2DoJokZ8Fg136gAcd0T+lpRKVzaJlDrdabi5vYz6oiU9ke3BRT9+4SArF1R4Uqct57ro7ub2qpY+aeXZCYBlN8yPwJ7k4dSucTPTG3eYw9VaA5YqI7KZS2xeRMVhqhkyYTVNXLRKvws0niAClh3K20QxsZqGe6cQ5gLq1kJQSzLgUwXWoi51ZaEY0cW3Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DIjQgW+CjX+2R5ZsqvkdIeFD4XfzQrKHWkgE3XXhzA4=; b=JQzF7q8YFulzw9ldnG1ZxcdXGO9HXNjaDW6XwMwbz6YgoS7c9Ls3F4eO9hYV4u87CFznsuJQYp6etTqK7lMAKvjmtxVgEuCpZ88n4zmi78oK4Sg7cdsWaGmhOV0SEyI71+aSl25iCIGJNownQLOb/uE7f3I9BqguHyQ1mqzgM8py/eBpbhrSY8NGUSRBG8G6PmQPLvU4mSA8r5zu9YzdKpmLW7ggpUTbmGNUH+JG9xEZkam3FwS3YZ6RKmy9NgLtQq7oBsxsx6pi5R83cCNVe48lf3sj7wjbulKbE2AqjnOcykV8XzUXIRwbDPDj6USERlbZCEU/HGiT9DCA7aOlrw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DIjQgW+CjX+2R5ZsqvkdIeFD4XfzQrKHWkgE3XXhzA4=; b=y8TnGxQSeqfu3R5iHbFlxcGuIvw0yadWNSC16PfsJXeWQoVSiXkoy678zbtwkXeLU4VNVZfK10uBYirSFNNVMay52z3zalJujc5a7c3iLva+kSqGaRNj1jTJk26DgJ+/acrsmAwr7MNYpshoPw0B0D/cn/2ScT8jI92cJ19N5HE= Received: from CO1PR15CA0077.namprd15.prod.outlook.com (2603:10b6:101:20::21) by BY5PR12MB4210.namprd12.prod.outlook.com (2603:10b6:a03:203::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.7; Tue, 18 Jan 2022 11:07:02 +0000 Received: from CO1NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:101:20:cafe::bf) by CO1PR15CA0077.outlook.office365.com (2603:10b6:101:20::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.10 via Frontend Transport; Tue, 18 Jan 2022 11:07:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT019.mail.protection.outlook.com (10.13.175.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4888.9 via Frontend Transport; Tue, 18 Jan 2022 11:07:02 +0000 Received: from gomati.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Tue, 18 Jan 2022 05:06:57 -0600 From: Nikunj A Dadhania To: Paolo Bonzini CC: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Brijesh Singh , Tom Lendacky , Peter Gonda , , , Nikunj A Dadhania Subject: [RFC PATCH 4/6] KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by SEV/TDX Date: Tue, 18 Jan 2022 16:36:19 +0530 Message-ID: <20220118110621.62462-5-nikunj@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220118110621.62462-1-nikunj@amd.com> References: <20220118110621.62462-1-nikunj@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8235127c-47c7-4af2-39ba-08d9da72a795 X-MS-TrafficTypeDiagnostic: BY5PR12MB4210:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qGA4X84u7BcxAz7pacm46ed0MlxmlTF+KXHJTJjjYQLgom9MpXgKkB+5We9HFMlGPrc2IigXoBoV2GbC8XBXzrMUf68q3Bhyw1dcaa/oHljR3isECSugVUEMWBdHvkyXAn8XjHUPhS8Jsp3yYrNsWySYjOibrD6XrpcrJeBN4HgpJE8+ghhz3X7um3cPzZ5BCnRC1vA8gpDNSt/fIDAcf/LbJ/NwHmiIZUGRKruzarbpEB9A//mV5VrGFFefY0v/3nvFD2//te3g282DU4umRW8Roq4tVdiKnE/iMbU5pSw5NluPv1aUps23tSSZ84YogqwqLLD8HWLNiiJtvXZ8kDkgwM2xn9RnEKCGqdV6tJTJBEGsMW4z5cRyqGRGfpTvBErxIGzbdhIaYb6n45dT8aLM21NGjUcITIq6NbZYHE6GOMZ31XYI6Lp89zrX7htFt9m89rz1YvZi487ruEDYfFuLdihuOtHxmfBLL9lU2bMtQrQiyr5946C+YyvW/IAiqt9TC8fDV1mJ9KOIayIhlenX3m+aXMI5cauWa/QceVBbpxNri1E8WqONZzFiVp7slY5/HD0v4mH8bggUEHkgeg4xXP3cCulW6CuF8EGmr2noBIv6F8BdzrW8B3LFTYZ10qNJp9HV5IX3WIvJKmemyow0f7KmIZWytES6GWu8ia2fGaHRHeVZwlenhSbVdLC2in+9iUzZDGO/1M6/NnI/0iEAp2X6i+zmJ5oFf/HZ+sNfe6xpWV+frSpHgZvEQP1FBBFzta+/llKnC+JhN11lkwZUqrl6Fw0+J1X2MbzYA4s= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(40470700002)(46966006)(36840700001)(6666004)(40460700001)(26005)(82310400004)(1076003)(16526019)(8936002)(426003)(6916009)(36756003)(8676002)(5660300002)(508600001)(4326008)(2616005)(70586007)(70206006)(336012)(81166007)(356005)(54906003)(47076005)(83380400001)(2906002)(316002)(7696005)(186003)(36860700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2022 11:07:02.0618 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8235127c-47c7-4af2-39ba-08d9da72a795 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4210 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson Introduce a helper to directly (pun intended) fault-in a TDP page without having to go through the full page fault path. This allows SEV/TDX to pin pages before booting the guest, provides the resulting pfn to vendor code if should be needed in the future, and allows the RET_PF_* enums to stay in mmu.c where they belong. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Signed-off-by: Nikunj A Dadhania --- arch/x86/kvm/mmu.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index e9fbb2c8bbe2..0595891dd834 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -202,6 +202,9 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, return vcpu->arch.mmu->page_fault(vcpu, &fault); } +kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa, + u32 error_code, int max_level); + /* * Currently, we have two sorts of write-protection, a) the first one * write-protects guest page to sync the guest modification, b) another one is diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 62dda588eb99..de5d390e0dcc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4111,6 +4111,44 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) return direct_page_fault(vcpu, fault); } +kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa, + u32 error_code, int max_level) +{ + int r; + struct kvm_page_fault fault = (struct kvm_page_fault) { + .addr = gpa, + .error_code = error_code, + .exec = error_code & PFERR_FETCH_MASK, + .write = error_code & PFERR_WRITE_MASK, + .present = error_code & PFERR_PRESENT_MASK, + .rsvd = error_code & PFERR_RSVD_MASK, + .user = error_code & PFERR_USER_MASK, + .prefetch = false, + .is_tdp = true, + .nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(), + }; + + if (mmu_topup_memory_caches(vcpu, false)) + return KVM_PFN_ERR_FAULT; + + /* + * Loop on the page fault path to handle the case where an mmu_notifier + * invalidation triggers RET_PF_RETRY. In the normal page fault path, + * KVM needs to resume the guest in case the invalidation changed any + * of the page fault properties, i.e. the gpa or error code. For this + * path, the gpa and error code are fixed by the caller, and the caller + * expects failure if and only if the page fault can't be fixed. + */ + do { + fault.max_level = max_level; + fault.req_level = PG_LEVEL_4K; + fault.goal_level = PG_LEVEL_4K; + r = direct_page_fault(vcpu, &fault); + } while (r == RET_PF_RETRY && !is_error_noslot_pfn(fault.pfn)); + return fault.pfn; +} +EXPORT_SYMBOL_GPL(kvm_mmu_map_tdp_page); + static void nonpaging_init_context(struct kvm_mmu *context) { context->page_fault = nonpaging_page_fault; From patchwork Tue Jan 18 11:06:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 12716179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D475C433F5 for ; Tue, 18 Jan 2022 11:07:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238908AbiARLHP (ORCPT ); Tue, 18 Jan 2022 06:07:15 -0500 Received: from mail-bn8nam11on2049.outbound.protection.outlook.com ([40.107.236.49]:65024 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238749AbiARLHL (ORCPT ); Tue, 18 Jan 2022 06:07:11 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F4L5N0Lu4lNiPn2gm7V9tsmh/LF/8FS8R7Frh7tOtAxxUc4O0IIN5Knb9eAGlWaLJbLeteUJSfGtKukzXVjAdmrU9JHja5sd8tHiud7M7lEFVLHrhb9wBWOi099dNxJMBZeOmzr/VVNplWo2n7Ugs2SGVZ+P1SBu5EzmdHTdMhzWit/V4ZSowx1f3Xg5xpWggmzuM/a8DpjTZ/fH9YmPF1nU2lMOEZKGf+3pnPdkACBLk3p+EYzWfrc6fmijEHtK4ijW4vz7LdrlV0AUikcAmvrr35lMVwgr38p6TqWZYw+SctReT8EycgO9EOWSJIdteOMWZKbrYoxGvkbfhyr6qQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y4O1BTxQ+0AQpz7nmibwZM6cMfnh2ovbuPEI8zFPE9M=; b=DFYOutrUDSzWkSnFM8CQWQwPFfIHJsqcZU3qKVBfmcXt3SEB3sZzxj2VApQscu4hDdr4/hWrgMFAt4LOWybyMsRLtT2wvEMPtpQouruVsyOIzvw/2pJnwhz4h5LDx29BXb+Dx3C19y6BbkA9Ik9E/HQAbFXR4w7TP+rEekAHP+xCSMrc34vhFEV/rpepNwE2iH54003vYkST4qXT4f0f5qEdSKGVfdT1Wm+mh1l7O2FF5hwOpQ3Yw7YqV8q01n+hsnqLYw71Bxuge8n3nFRQ2y9geMpDuHZG+Le6OTydVh0j4mnMtIM05Wr1HDfkK0GMRPhHDcfTx41M+xXK86fMXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y4O1BTxQ+0AQpz7nmibwZM6cMfnh2ovbuPEI8zFPE9M=; b=PecZ6LYBhDeGvSsfIkK849wCug4oLeHbWzVQAkEYPTxTV39PzIbu1dyH56St9/rO+hOsJlKv3WEBHzC1DmKJcbLH2Rl/FYAzggoAq8+kAzuprTRXFTWnZCMmWSg24iWl6ISrbziGnXIHhcfRN5L1A0wyJg5KDw5I7IORnLPc4ak= Received: from MW4PR03CA0152.namprd03.prod.outlook.com (2603:10b6:303:8d::7) by BN9PR12MB5354.namprd12.prod.outlook.com (2603:10b6:408:103::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.12; Tue, 18 Jan 2022 11:07:09 +0000 Received: from CO1NAM11FT027.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8d:cafe::3a) by MW4PR03CA0152.outlook.office365.com (2603:10b6:303:8d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.10 via Frontend Transport; Tue, 18 Jan 2022 11:07:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT027.mail.protection.outlook.com (10.13.174.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4888.9 via Frontend Transport; Tue, 18 Jan 2022 11:07:08 +0000 Received: from gomati.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Tue, 18 Jan 2022 05:07:01 -0600 From: Nikunj A Dadhania To: Paolo Bonzini CC: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Brijesh Singh , Tom Lendacky , Peter Gonda , , , Nikunj A Dadhania Subject: [RFC PATCH 5/6] KVM: SEV: Carve out routine for allocation of pages Date: Tue, 18 Jan 2022 16:36:20 +0530 Message-ID: <20220118110621.62462-6-nikunj@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220118110621.62462-1-nikunj@amd.com> References: <20220118110621.62462-1-nikunj@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d00adf51-1c0b-4767-7c75-08d9da72ab31 X-MS-TrafficTypeDiagnostic: BN9PR12MB5354:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RXDzO2PzS6cDcOOlApQeJPofK2rXCvNNbcnjUXnlOzXO18FESbiHuyOTUTyXGI6OFy4jFaCjd8htRRsnf42+7m4p7cVK2WAmFBTiJy1ZfO4XK8dM8W0OkoTWu5gyqPvqQgzBeE3FqTq7KpIAQRKK9Z3v+qdacIyVHbgZDZ2MyCr4l6fXUYMiLJGOSDi8gX4cVodyZG9wB8snefzutHHl1gGp2caOB4CGGoHg7hcz1Je/czkEGcu0rxhMvof3kjbhcNy33kIGe2r41kJm39e/+6oueqxO79LGZPbbuewoASvcRtVf7L/pmPRKoiKCNUWEnlTM2umt8aQmfv7HPqpDsGNmXiDrEXYSrrvnRm7hSCy8RWSTUyLsz96X0Bb13p7JE+CAyq/TWZu9qX4zDGOmEUtOyeb2joVNU9K9YLvrdijwznjsV3JZsjAI2Z3EbwQcuuzwSE/XOX8iEYPLH7W91sB6XcJMgwYebK42K8SkWatMnWuqTtpceMx4Cm9duch566yWN3NGxdkTqi+NOCk94QBxXXbRuQ23TaiubrHzoQRCAdTbVQSKgpMLMy8OfBaKg+oxiDDYYkvTpUo4ljjWWiu9hZ0uMdGSeXhF6ydj8p4zy9A4rCG7sDOBFZPL8F1C5KfHAca9VGiremS2l3Aa6qzjAQGEDtlp0l+O8N6m2DSz2X5qfC7NqOgFoqsqxBnc381XRFMC5GGku2foo11UdlXUdgQ5qkJpB0hS3XD9G3NhD7cDwqFDHMZyF7n28Xe6mKKrZJOIXLOLj09QQr8YzASRqFgZiA/Cl0JnCgqkxbI= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(40470700002)(36840700001)(46966006)(36756003)(1076003)(40460700001)(16526019)(70206006)(8936002)(70586007)(508600001)(6916009)(2906002)(4326008)(8676002)(82310400004)(5660300002)(36860700001)(186003)(356005)(426003)(54906003)(81166007)(6666004)(47076005)(336012)(2616005)(316002)(83380400001)(26005)(7696005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2022 11:07:08.1347 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d00adf51-1c0b-4767-7c75-08d9da72ab31 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT027.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5354 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Create a separate routine sev_alloc_pages() for allocating sev pages. This will be used in the following MMU based pinning. While at it, validate the number of pages before the RLIMIT check and use kzalloc instead of kmalloc. Signed-off-by: Nikunj A Dadhania --- arch/x86/kvm/svm/sev.c | 44 +++++++++++++++++++++++++++--------------- 1 file changed, 28 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index a962bed97a0b..14aeccfc500b 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -368,19 +368,13 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) return ret; } -static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, - unsigned long ulen, unsigned long *n, - int write) +static void *sev_alloc_pages(struct kvm_sev_info *sev, unsigned long uaddr, + unsigned long ulen, unsigned long *n) { - struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; - unsigned long npages, size; - int npinned; unsigned long locked, lock_limit; - struct page **pages; + unsigned long npages, size; unsigned long first, last; - int ret; - - lockdep_assert_held(&kvm->lock); + struct page **pages; if (ulen == 0 || uaddr + ulen < uaddr) return ERR_PTR(-EINVAL); @@ -390,6 +384,9 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, last = ((uaddr + ulen - 1) & PAGE_MASK) >> PAGE_SHIFT; npages = (last - first + 1); + if (WARN_ON_ONCE(npages > INT_MAX)) + return ERR_PTR(-EINVAL); + locked = sev->pages_locked + npages; lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { @@ -397,19 +394,34 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, return ERR_PTR(-ENOMEM); } - if (WARN_ON_ONCE(npages > INT_MAX)) - return ERR_PTR(-EINVAL); - /* Avoid using vmalloc for smaller buffers. */ size = npages * sizeof(struct page *); if (size > PAGE_SIZE) pages = __vmalloc(size, GFP_KERNEL_ACCOUNT | __GFP_ZERO); else - pages = kmalloc(size, GFP_KERNEL_ACCOUNT); + pages = kzalloc(size, GFP_KERNEL_ACCOUNT); - if (!pages) - return ERR_PTR(-ENOMEM); + *n = pages ? npages : 0; + return pages; +} +static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, + unsigned long ulen, unsigned long *n, + int write) +{ + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; + unsigned long npages, locked; + struct page **pages; + int npinned; + int ret; + + lockdep_assert_held(&kvm->lock); + + pages = sev_alloc_pages(sev, uaddr, ulen, &npages); + if (IS_ERR(pages)) + return pages; + + locked = sev->pages_locked + npages; /* Pin the user virtual address. */ npinned = pin_user_pages_fast(uaddr, npages, write ? FOLL_WRITE : 0, pages); if (npinned != npages) { From patchwork Tue Jan 18 11:06:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 12716180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77651C43217 for ; Tue, 18 Jan 2022 11:07:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238959AbiARLHQ (ORCPT ); Tue, 18 Jan 2022 06:07:16 -0500 Received: from mail-mw2nam08on2060.outbound.protection.outlook.com ([40.107.101.60]:15840 "EHLO NAM04-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S238886AbiARLHM (ORCPT ); Tue, 18 Jan 2022 06:07:12 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BJwBubjBSPFyxTwH6vppkGK8Q4zHVzyvd0+j7IF6MKQ4fhRQro93+8/pJuvZ3P5hqPIIRFGAfPIFn6xj3jdKcHWLW/0PfD/Um45m5ub7ByIU3mRoM/zL8ghb9syqSW/rV3/dnsW/YklZyDLnVX3YpGubP19JqHFT9YGM5ww5Vb1PaFherU9AgEctd5bjGaqnn5Q9ybWFllv5/phqEid5VTz3MzlDMEIegveKCAKSlhg7J33p/AwiebB/kx2L18mnjXI1Mg+Jb8t7n0TXLWag7mGB61/A2v4blwZtsqJ0+7eH1tJMcB8vBHJD+ATjZf/NWJBj8PPwe1pJvEbQTzdnRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yGs/NxoPmwMYt2qms90KlE15xs/G5gfvCH3JjBjSFzY=; b=K01YcWMTXvEOmp+Qmd/BEiof/oo1z7+6tmTR4BMbcTDza0Fyz/3OnO7KsbXy2Me0Y7KuNQJxezrioEuyrPhK/fybCMDaLLKux2vWkxeN+2gfJ7wQ3jNNmbQm1DROQcHxtYN9GeAmmp0+qF4l2CKcrvGMM3BRt6DeDi9gQzLNJUaSk9YvSWzZSGQAPUEIfLrPCqi0VX6VREEUOCd0ws8FXQBqEo5o/0MRWQgTJzoHiTnXKBhVsTq7ZLMXLOivpU35Hdh1UlgKpY6tShx1wS0ScOzPkJ9kdLj6NQmP2dNkYcvxJGFf9w266bYn2G/cx/5bb90yl8mfanZKXYC2C/zoKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yGs/NxoPmwMYt2qms90KlE15xs/G5gfvCH3JjBjSFzY=; b=BfvcvGl/bBFD58Y+SV9MI70LsbWakVxIB8CDqawX3E0bt2W/VYDPe3I6rb7gWoj7ycUjpBKxhIhEarObhqzDhhKkGUZhBHUqeW/rJVAmZwTizd3vWd4h9elh2vOx2co7ZvH5C1NwjSCMEUtNJudTMDfizxQlTunMgf6nrRBfLCs= Received: from MWHPR14CA0048.namprd14.prod.outlook.com (2603:10b6:300:12b::34) by SJ0PR12MB5472.namprd12.prod.outlook.com (2603:10b6:a03:3bb::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.10; Tue, 18 Jan 2022 11:07:10 +0000 Received: from CO1NAM11FT053.eop-nam11.prod.protection.outlook.com (2603:10b6:300:12b:cafe::d5) by MWHPR14CA0048.outlook.office365.com (2603:10b6:300:12b::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.7 via Frontend Transport; Tue, 18 Jan 2022 11:07:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1NAM11FT053.mail.protection.outlook.com (10.13.175.63) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.4888.9 via Frontend Transport; Tue, 18 Jan 2022 11:07:10 +0000 Received: from gomati.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Tue, 18 Jan 2022 05:07:05 -0600 From: Nikunj A Dadhania To: Paolo Bonzini CC: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Brijesh Singh , Tom Lendacky , Peter Gonda , , , Nikunj A Dadhania Subject: [RFC PATCH 6/6] KVM: SVM: Pin SEV pages in MMU during sev_launch_update_data() Date: Tue, 18 Jan 2022 16:36:21 +0530 Message-ID: <20220118110621.62462-7-nikunj@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220118110621.62462-1-nikunj@amd.com> References: <20220118110621.62462-1-nikunj@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d680bbe2-6fc5-4471-51fc-08d9da72ac65 X-MS-TrafficTypeDiagnostic: SJ0PR12MB5472:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WGaQOJmurj8xUqnxz/YcrdMhnYLvCvKPh1xA9tkiPGWhOE6M/j5lGMMQGs6e8rQDwf/xmEFflE8cPChSDFZalrBUUSXG6lE02eEJfpxYtAxjapYGTKpptUXicBtRK9RvEOr/jaF/ciHcZnhUiWqtiQOwxCBa1ulAFqdmMTKAxDEukohc+3o9ydBDOH75XBIEBBE6KKxn4xpd1VnwOeYMxh+xneaSHP0/h1MeRo/de4JIbzEITlVp1sz7ossPvb//kxVEhMzx6aOo6fNHRdA4w9KtLUkPZtE5/HDvdcFe1omeL4BYWGO9z3Ohz42XzXJVkSQGjnZHZqXbIKuZxMZu6ilT179MPBI0el/rt5GEj317U8W313dBqlP9vKb+Kw9DQVSkm1eJcFLh7++u37SrkW4PlLa2zPwybdnLJ28baY0OFtMdgIK3nhnquByVXtbgenonYeMJuzP0kiHsNZlzCMNmt3Jb30S3Davt0sfHMYj6NTRM09aynjflVdU3Cl/bjZKqV1pLh9sRJ3hEtfnrKIsIiq/Xbr6n9vSnn+3+hBR6ax++NwxGPyOYfGwNDAWFgnw0aRNWjYuJmDj5Cws4QwkNMECZ33Lwo2FMdcPxkyB/Vy/h7woM0+dueEKTN5gcZOKcc4xY/vUQ8mAjSy7vWMIJYNmb9yyc31CDyN3WnOm8ylddkhvRMT7TQzAh5LFlHwxzt8DegNUArTL6E3M3T99WsGNjp4WgZ2deuG8RK6vj7kRBJbGw1DO3/qApL0JEkqwbPyQdFgJaMhY01C1+7uSXNgzreokEPxO9S0N1YQ8= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(40470700002)(46966006)(36840700001)(336012)(426003)(5660300002)(26005)(36756003)(81166007)(2616005)(1076003)(16526019)(47076005)(40460700001)(82310400004)(6916009)(186003)(83380400001)(54906003)(8676002)(316002)(2906002)(6666004)(7696005)(36860700001)(8936002)(70206006)(4326008)(70586007)(356005)(508600001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2022 11:07:10.1209 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d680bbe2-6fc5-4471-51fc-08d9da72ac65 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT053.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5472 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Sean Christopherson Pin the memory for the data being passed to launch_update_data() because it gets encrypted before the guest is first run and must not be moved which would corrupt it. Signed-off-by: Sean Christopherson [ * Changed hva_to_gva() to take an extra argument and return gpa_t. * Updated sev_pin_memory_in_mmu() error handling. * As pinning/unpining pages is handled within MMU, removed {get,put}_user(). ] Signed-off-by: Nikunj A Dadhania --- arch/x86/kvm/svm/sev.c | 122 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 119 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 14aeccfc500b..1ae714e83a3c 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -22,6 +22,7 @@ #include #include +#include "mmu.h" #include "x86.h" #include "svm.h" #include "svm_ops.h" @@ -490,6 +491,110 @@ static unsigned long get_num_contig_pages(unsigned long idx, return pages; } +#define SEV_PFERR_RO (PFERR_USER_MASK) +#define SEV_PFERR_RW (PFERR_WRITE_MASK | PFERR_USER_MASK) + +static struct kvm_memory_slot *hva_to_memslot(struct kvm *kvm, + unsigned long hva) +{ + struct kvm_memslots *slots = kvm_memslots(kvm); + struct kvm_memory_slot *memslot; + int bkt; + + kvm_for_each_memslot(memslot, bkt, slots) { + if (hva >= memslot->userspace_addr && + hva < memslot->userspace_addr + + (memslot->npages << PAGE_SHIFT)) + return memslot; + } + + return NULL; +} + +static gpa_t hva_to_gpa(struct kvm *kvm, unsigned long hva, bool *ro) +{ + struct kvm_memory_slot *memslot; + gpa_t gpa_offset; + + memslot = hva_to_memslot(kvm, hva); + if (!memslot) + return UNMAPPED_GVA; + + *ro = !!(memslot->flags & KVM_MEM_READONLY); + gpa_offset = hva - memslot->userspace_addr; + return ((memslot->base_gfn << PAGE_SHIFT) + gpa_offset); +} + +static struct page **sev_pin_memory_in_mmu(struct kvm *kvm, unsigned long addr, + unsigned long size, + unsigned long *npages) +{ + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; + struct kvm_vcpu *vcpu; + struct page **pages; + unsigned long i; + u32 error_code; + kvm_pfn_t pfn; + int idx, ret = 0; + gpa_t gpa; + bool ro; + + pages = sev_alloc_pages(sev, addr, size, npages); + if (IS_ERR(pages)) + return pages; + + vcpu = kvm_get_vcpu(kvm, 0); + if (mutex_lock_killable(&vcpu->mutex)) { + kvfree(pages); + return ERR_PTR(-EINTR); + } + + vcpu_load(vcpu); + idx = srcu_read_lock(&kvm->srcu); + + kvm_mmu_load(vcpu); + + for (i = 0; i < *npages; i++, addr += PAGE_SIZE) { + if (signal_pending(current)) { + ret = -ERESTARTSYS; + break; + } + + if (need_resched()) + cond_resched(); + + gpa = hva_to_gpa(kvm, addr, &ro); + if (gpa == UNMAPPED_GVA) { + ret = -EFAULT; + break; + } + + error_code = ro ? SEV_PFERR_RO : SEV_PFERR_RW; + + /* + * Fault in the page and sev_pin_page() will handle the + * pinning + */ + pfn = kvm_mmu_map_tdp_page(vcpu, gpa, error_code, PG_LEVEL_4K); + if (is_error_noslot_pfn(pfn)) { + ret = -EFAULT; + break; + } + pages[i] = pfn_to_page(pfn); + } + + kvm_mmu_unload(vcpu); + srcu_read_unlock(&kvm->srcu, idx); + vcpu_put(vcpu); + mutex_unlock(&vcpu->mutex); + + if (!ret) + return pages; + + kvfree(pages); + return ERR_PTR(ret); +} + static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) { unsigned long vaddr, vaddr_end, next_vaddr, npages, pages, size, i; @@ -510,15 +615,21 @@ static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) vaddr_end = vaddr + size; /* Lock the user memory. */ - inpages = sev_pin_memory(kvm, vaddr, size, &npages, 1); + if (atomic_read(&kvm->online_vcpus)) + inpages = sev_pin_memory_in_mmu(kvm, vaddr, size, &npages); + else + inpages = sev_pin_memory(kvm, vaddr, size, &npages, 1); if (IS_ERR(inpages)) return PTR_ERR(inpages); /* * Flush (on non-coherent CPUs) before LAUNCH_UPDATE encrypts pages in * place; the cache may contain the data that was written unencrypted. + * Flushing is automatically handled if the pages can be pinned in the + * MMU. */ - sev_clflush_pages(inpages, npages); + if (!atomic_read(&kvm->online_vcpus)) + sev_clflush_pages(inpages, npages); data.reserved = 0; data.handle = sev->handle; @@ -553,8 +664,13 @@ static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) set_page_dirty_lock(inpages[i]); mark_page_accessed(inpages[i]); } + /* unlock the user pages */ - sev_unpin_memory(kvm, inpages, npages); + if (atomic_read(&kvm->online_vcpus)) + kvfree(inpages); + else + sev_unpin_memory(kvm, inpages, npages); + return ret; }