From patchwork Fri Oct 20 14:30:53 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10020381 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 03EBF60211 for ; Fri, 20 Oct 2017 14:35:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E187128E13 for ; Fri, 20 Oct 2017 14:35:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D5F5628E19; Fri, 20 Oct 2017 14:35:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3007D28E13 for ; Fri, 20 Oct 2017 14:35:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753002AbdJTOe4 (ORCPT ); Fri, 20 Oct 2017 10:34:56 -0400 Received: from mail-dm3nam03on0088.outbound.protection.outlook.com ([104.47.41.88]:47104 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752855AbdJTObi (ORCPT ); Fri, 20 Oct 2017 10:31:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=X4xMhO3Gmk7+d9+fVB/pMHluEnHW9sPce0dnrsx0PbA=; b=s4wRdq5T3qhGZ5P7gKnfE1GGAmWB/ZA5x+1pbasxRhBYJ1n9PqSNXUL5GB2VIaYkJ8jGIom9HrItYWGoTMavUSnrHvLoQuWHs53z8eCky14KZbvtCNp3Do4gGFHiqB348o7gPfPHcHyYsVSny7GwfBKCG07b/WZsyo9l1zN3df4= Received: from ubuntu-010236106000.amd.com (165.204.78.1) by SN1PR12MB0160.namprd12.prod.outlook.com (10.162.3.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.156.4; Fri, 20 Oct 2017 14:31:27 +0000 From: Brijesh Singh To: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Borislav Petkov , Tom Lendacky , Brijesh Singh , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Borislav Petkov , Konrad Rzeszutek Wilk Subject: [Part1 PATCH v7 11/17] x86/mm: Add DMA support for SEV memory encryption Date: Fri, 20 Oct 2017 09:30:53 -0500 Message-Id: <20171020143059.3291-12-brijesh.singh@amd.com> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171020143059.3291-1-brijesh.singh@amd.com> References: <20171020143059.3291-1-brijesh.singh@amd.com> MIME-Version: 1.0 X-Originating-IP: [165.204.78.1] X-ClientProxiedBy: BN6PR14CA0029.namprd14.prod.outlook.com (10.171.172.143) To SN1PR12MB0160.namprd12.prod.outlook.com (10.162.3.147) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0036d384-e3e8-4424-4799-08d517c73ffe X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(48565401081)(4534020)(4602075)(4627075)(201703031133081)(201702281549075)(2017052603229); SRVR:SN1PR12MB0160; X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 3:v3XK174uW2DkXJ9bGYcXyzFQezamRrsY0maC3jGWUh6tFaaaZWi2wsq7Q5MH9+TIhUHIvWiXNUQSxHUbZ+CipcPJlbpLOU0ofLVxIA2jm4mOAWbAG8l6MRWXkt1wLyfOCL7pnCrQhAEDSMoRP4DHtH95h8tIAFWHRVnyQAGQgykx4f9nfJZ588XnpvPpsFaU2gO8xhpD6pA41Dung95TSr5twbL9HP9nnzzOYZA7gs+c8bVCSEoGWzLZ1Gkzti1X; 25:lelg1yty3M9E42YI3btOHTvQl6Lt8rCTMNVKJGESKS73y5POTUC2Ebz9rnAF0N75X07xSqbl212iuTrrmTloB/A1tPIs4ZmHgM4lP8gZZiYoL3pUJqGQnKgBH8TwA5ESYdlIsUuQTrwi2lb0BgxYEsni2HOOZ+ayxuj2QaT2l3id8YmAo+P2C8IqqsF+FllYRRd9nTYyQUO+w8856X7N82bsJ9zrf7uaRA55yr6o+xkZHAf/56N7HscJXXrhsiC9UrGQwjOS/8AK0U6OMzoBmGAFTloY5MF4pFiBNakVA4O555f6Fpe9VfFsI0F0PVohR5qAH7V4/R5D8Q0Yes0pqQ==; 31:j+hzj3uOxaNMmHM7UOlzPgiQblrgf3F1qzK5UZujlF9q3Ogk2Zne0V4Mvpzg4Pq2gPZA41YVTXcFu3ypH7TXE0Hj8Cz7aYhsABJa1J6BBOKnd+WjfUsJ3i3UxuQ5fxKyeE4vGZAsdmq6H6s0v6ZRpEsPfJ8WUqioVa/bXElLRHfXMEoRBQyOApooTWWf8kRm8MIUbmVbaWeiuP656aS50+3Lq19zgBLDRo21ooTF6Hc= X-MS-TrafficTypeDiagnostic: SN1PR12MB0160: X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 20:d+ItHVmQ/REpqg6xo6LNw2ynQsmIkiqe0VIKu71i30rNjIlIaNNW5DDLFXi9ltoQn8kDS7L5DrPYFaytdunOOfKJYaO/Rmmeiabhu77WnptD2cvQCAW5F3voCKVxUM4oz77M/oD2Z4U2ZBlXDZtWSCtODCZPhS+IjylepiwO3uNICGKdjkFax6ULZ+oWQidccnPbcNt+qHHXRGvSgY3ySzWOKkf995FRpY3lxYrKM2aTF9O4rcGFnukAfkRjr7o1H2qgDn2sC0lui7dB6jt4WCs2/vS6myVi/qNQyYzl+hiY5UbQoHLE/w0To8hbCbyyVTZrUdrbtxlf0l6eImmr+NlmO/HFJf2twRfmr81SDu5CuIhKQ4T30oQ4C4UL/xdfc4bBpLhWaZeMnhZlAiYnckkCasADoOS3QFlHbjpFfToCSfgnwWnNc/LbkA1X+LlUO8A3ZhTWXHdOUfowYQVpMPAWQg63g3ffRfhZFk9bGko2r9N12vsWv2F3lhwVmRH6; 4:xPLe4EExzZSZxuj+WlDjbmsCJM+Uwk8Y2s1Ak4835ZLsLTLisyzBrGyqlc50vRFYa+RjcaWbqjtiLQdIXpAdF15+31KZBtSsrgUoDw5i+/M70gP8gqHoaiD1lE9iIW6G/xGrQKTwOoMNMrG68RDssGw+51IcXr+73c18jmxhzRao3eJJs3trVy1GhUP5dOTKhHCTJwWUm/ghEcDBQRt1k5345mwLjQW9OnAON1T0Jm7AcBup6N7wqvHx5z9yJgbM7xizHf5nzklOh4J/0cMPRMDb9yEJ5OOxVnPRbwCMkg4rmA7PszzzuxHcZx37GXPGVRYVcvqGkYOWHF20Njm/tcXmym7Kfuh6zmiuPMtfl7g= X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(767451399110)(146099531331640); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(3231020)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(6055026)(6041248)(20161123562025)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:SN1PR12MB0160; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:SN1PR12MB0160; X-Forefront-PRVS: 0466CA5A45 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(39860400002)(376002)(346002)(199003)(189002)(54906003)(1076002)(6666003)(189998001)(68736007)(8676002)(305945005)(50226002)(7736002)(101416001)(76176999)(8936002)(97736004)(25786009)(81166006)(81156014)(16526018)(6486002)(478600001)(48376002)(2906002)(50466002)(4326008)(47776003)(66066001)(5660300001)(2950100002)(53936002)(106356001)(3846002)(50986999)(16586007)(53416004)(6116002)(5003940100001)(36756003)(33646002)(86362001)(105586002)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR12MB0160; H:ubuntu-010236106000.amd.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN1PR12MB0160; 23:PFxELyLiK5IhTg0SjLpq6THIyLxDGsXZHgq0c9XzP?= =?us-ascii?Q?3GlfWJdlrJlAE4HwRhfo2K6xPsKiYXRH4IhODJGAXB7qa9maReAcDHb2Yb4O?= =?us-ascii?Q?aesduCfNY4yLamXnTgpZE3aMVUJKT5iatW/oRUvZ3RXoAq7NKcymEPr7F64V?= =?us-ascii?Q?CU6YPTOy+O5aHvtXiYrI6mMnchyRfXmrEqgVUD21UlapjdVC8yw6uZMBOhm5?= =?us-ascii?Q?V1O12l2jowvt6VQCedZXr9VF7dkmeOkAG0T+v/ahV0deK1vjncz0DMqIfvWy?= =?us-ascii?Q?KbticLYGRBoXbdOHUOyNSOTmhNC3z49DqUxkIYkjV6U2bjWQT6ihbwqGA08j?= =?us-ascii?Q?gTZM+FA2Ejo1r5IoVous8vFO5h+8iRJ6Z64BcHYwC3k5ii+7gilNa32/jm5O?= =?us-ascii?Q?iRMGdloHJd+hpCHV7vYmdwuAjQPk+u1qz1oq6S2rxstSo4Yf9uMVcZajMGWU?= =?us-ascii?Q?48f0CRlJis0JgAGpORVKHQIykaiGSDCoHpS4939FpDgKLJlNfSIFJSUtWQL/?= =?us-ascii?Q?OfThdYPE8v47mf4eQfS9xHSNj2/P4+R7FRreL7zCAWJi11atzjXlv1uNUIZ2?= =?us-ascii?Q?xgseCj90IcDvqbr5kiLVn7SCAwCb4Db82YGsBFeRGe+4PI7PX1KHVlHnGRjZ?= =?us-ascii?Q?2dcmT5Va5rHnVT/HlkUVk/yUQfZv016b3wJX2v5DOfsiEG+AUaDexse0qR69?= =?us-ascii?Q?f6ydmwHobv1S1cEQYnJbGl6Sv/3KB7s4q05UhBKfFCG6KxeuozTy1VGw+l+V?= =?us-ascii?Q?5AKWVVWPLGnakZaHK8LqW6Y6BBm+yWsjQK36Dq5t7n+Yye1EzUS//6Ay9INL?= =?us-ascii?Q?0cyT/xTpLMm4RrNYFEUzyeh3PuZ/m5pajd1S6jLMjF6jxKqLYvi4Kt1UoKP3?= =?us-ascii?Q?4vqOgnXSznfqzBH07SeD0y0XM/xrXdF5uRSOw1TmjsHjccSaaG9JS0TvU5Qk?= =?us-ascii?Q?7FUFFQTm0c79a90aAatVO6GICVw+RJuyVkc8AczM7xMGFimtRuxhLouiSnQk?= =?us-ascii?Q?QzMHEknvhIBSzS7UADhLY2JtDOaZYYLbRG4cxaRWWF4m2thHYfMd9vB/njkd?= =?us-ascii?Q?jw9/YCPmXAU5CyLfBFyEBrJ9pnUdOzUQwePgTIzqQZLdwLkSA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 6:sbgu1dZo+okXg1fZuFzK3RYor2reXT/YG7KaoIIemViINyekI/F4HLIQvMJP9WA7pTLK7PUtWNecEWa1q6ECGNvrpDbKZA74IZ2SdA7EnSRg4AWJYoqD8jZZ1s+5dIB6ExV/NHcLvQUJRpqOvWiJyk7UOQUO7AWYf2E/D4AEVqo1DBSLO2vDJ0Nbh6F1dF+thT5b1gw8lcIuPzYoIlARPKyCteh434sfGLHsMR3fZVfxuxYUnEDFC8RMu/oncWdd2nqpYj4lZaDbFC65JQPjV4DWZPqXvfi82n+JdF9BmDXWtnWU4TU58pbymLoumWDUvo3MCQl1AtvGCB0eN5x8TA==; 5:9Cc+Cj6fCBQFUxFnI0KXoKLUrcPnMTEtf/z7r1yeW6kriYdEFx3V2b/5GFtqZd3wUTMsZwJVIWF1/uTuEdNjhR10Acae5yTw5Pk+GfpCLFv/VNcpm7ppzdIUf9EQVGBrSYSqKJI1cjZMDL88egzpJQ==; 24:wzI/oPEggMyCSCTl7EW2wXjxGApnlKjJdLFM5fv0Wl/laCuPDo1hGqKuu6A2e0HULzJwDerwnDQ6wTChNxCyBSHoLrCR4hVXVUd6hIz64IU=; 7:Rde1qWkmHDRG9dF0e3zap7ISB+Fo2mC3u20aLP222+LUqBAnW66OptkdFkqZMDpgPIyOVcflhy+U0L9A9FrPKzEnWw8EKuVy1aAeLgAg5zE4MPahVAxMze1/yAVyXi+0ZuKZAaiqhVaTAivPdk+4OHWcSVR9iQTHKUr9ZDHSb2lBdQXKNHoA1Brgta8IEjRwcFRdNifeWm5hvLSC8gUMmxosjpfOoHIJvVP+7gbVzMY= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0160; 20:Colb6yLSMy3Ixc/PRG49Vl+ELERg177IPmG15MB3xGyn6PucaSfm+RBPmoAuSObmgf9BQxbhJx+xyv6p4YYvDpgee9BPlmx/FaL7B9A7QQXG9pD+MTMQurV483FZLPNr4+8J+P8uJDvKwdRNcR/psS4l1TVKLfYy86N4JxLAupygIh6XoxhUgNIv0JrGy+bX6cAXvf30KH6C9DWiVIQLFACdOYUN2SXaODmdBGIlhDC7bEMlLJhS15G3UP+HvXzS X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Oct 2017 14:31:27.3446 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0036d384-e3e8-4424-4799-08d517c73ffe X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0160 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tom Lendacky DMA access to encrypted memory cannot be performed when SEV is active. In order for DMA to properly work when SEV is active, the SWIOTLB bounce buffers must be used. Signed-off-by: Tom Lendacky Signed-off-by: Brijesh Singh Reviewed-by: Borislav Petkov Tested-by: Borislav Petkov Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Borislav Petkov Cc: Konrad Rzeszutek Wilk Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org --- arch/x86/mm/mem_encrypt.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++ lib/swiotlb.c | 5 +-- 2 files changed, 89 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index add836d3d174..e8bfad75ba29 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -192,6 +192,70 @@ void __init sme_early_init(void) /* Update the protection map with memory encryption mask */ for (i = 0; i < ARRAY_SIZE(protection_map); i++) protection_map[i] = pgprot_encrypted(protection_map[i]); + + if (sev_active()) + swiotlb_force = SWIOTLB_FORCE; +} + +static void *sev_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, + gfp_t gfp, unsigned long attrs) +{ + unsigned long dma_mask; + unsigned int order; + struct page *page; + void *vaddr = NULL; + + dma_mask = dma_alloc_coherent_mask(dev, gfp); + order = get_order(size); + + /* + * Memory will be memset to zero after marking decrypted, so don't + * bother clearing it before. + */ + gfp &= ~__GFP_ZERO; + + page = alloc_pages_node(dev_to_node(dev), gfp, order); + if (page) { + dma_addr_t addr; + + /* + * Since we will be clearing the encryption bit, check the + * mask with it already cleared. + */ + addr = __sme_clr(phys_to_dma(dev, page_to_phys(page))); + if ((addr + size) > dma_mask) { + __free_pages(page, get_order(size)); + } else { + vaddr = page_address(page); + *dma_handle = addr; + } + } + + if (!vaddr) + vaddr = swiotlb_alloc_coherent(dev, size, dma_handle, gfp); + + if (!vaddr) + return NULL; + + /* Clear the SME encryption bit for DMA use if not swiotlb area */ + if (!is_swiotlb_buffer(dma_to_phys(dev, *dma_handle))) { + set_memory_decrypted((unsigned long)vaddr, 1 << order); + memset(vaddr, 0, PAGE_SIZE << order); + *dma_handle = __sme_clr(*dma_handle); + } + + return vaddr; +} + +static void sev_free(struct device *dev, size_t size, void *vaddr, + dma_addr_t dma_handle, unsigned long attrs) +{ + /* Set the SME encryption bit for re-use if not swiotlb area */ + if (!is_swiotlb_buffer(dma_to_phys(dev, dma_handle))) + set_memory_encrypted((unsigned long)vaddr, + 1 << get_order(size)); + + swiotlb_free_coherent(dev, size, vaddr, dma_handle); } /* @@ -218,6 +282,20 @@ bool sev_active(void) } EXPORT_SYMBOL_GPL(sev_active); +static const struct dma_map_ops sev_dma_ops = { + .alloc = sev_alloc, + .free = sev_free, + .map_page = swiotlb_map_page, + .unmap_page = swiotlb_unmap_page, + .map_sg = swiotlb_map_sg_attrs, + .unmap_sg = swiotlb_unmap_sg_attrs, + .sync_single_for_cpu = swiotlb_sync_single_for_cpu, + .sync_single_for_device = swiotlb_sync_single_for_device, + .sync_sg_for_cpu = swiotlb_sync_sg_for_cpu, + .sync_sg_for_device = swiotlb_sync_sg_for_device, + .mapping_error = swiotlb_dma_mapping_error, +}; + /* Architecture __weak replacement functions */ void __init mem_encrypt_init(void) { @@ -227,6 +305,14 @@ void __init mem_encrypt_init(void) /* Call into SWIOTLB to update the SWIOTLB DMA buffers */ swiotlb_update_mem_attributes(); + /* + * With SEV, DMA operations cannot use encryption. New DMA ops + * are required in order to mark the DMA areas as decrypted or + * to use bounce buffers. + */ + if (sev_active()) + dma_ops = &sev_dma_ops; + pr_info("AMD Secure Memory Encryption (SME) active\n"); } diff --git a/lib/swiotlb.c b/lib/swiotlb.c index 8c6c83ef57a4..cea19aaf303c 100644 --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -507,8 +507,9 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev, if (no_iotlb_memory) panic("Can not allocate SWIOTLB buffer earlier and can't now provide you with the DMA bounce buffer"); - if (sme_active()) - pr_warn_once("SME is active and system is using DMA bounce buffers\n"); + if (mem_encrypt_active()) + pr_warn_once("%s is active and system is using DMA bounce buffers\n", + sme_active() ? "SME" : "SEV"); mask = dma_get_seg_boundary(hwdev);