From patchwork Tue Jul 17 11:20:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A . Shutemov" X-Patchwork-Id: 10529019 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 427E760545 for ; Tue, 17 Jul 2018 11:22:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F9D9205A8 for ; Tue, 17 Jul 2018 11:22:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 23B9F28968; Tue, 17 Jul 2018 11:22:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A2FC928A05 for ; Tue, 17 Jul 2018 11:22:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4884C6B0269; Tue, 17 Jul 2018 07:21:51 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 437A86B0271; Tue, 17 Jul 2018 07:21:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23DE36B0272; Tue, 17 Jul 2018 07:21:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id D680D6B0269 for ; Tue, 17 Jul 2018 07:21:50 -0400 (EDT) Received: by mail-pf0-f199.google.com with SMTP id h14-v6so383140pfi.19 for ; Tue, 17 Jul 2018 04:21:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=4hSUWDZCUBTeeDsArIAUszGAWEjInBF+D7KzAtvRX+4=; b=EJVJv537bzYKM+LsCQG9NQoMkO/kg5SyLz8y1mV7mayzIshqZfMXZgxzz6pgEwiFQh zAu8zT/bt/k/LZVC7mRMuZ6RHr7RFbIBFuqoUFbYx4QteAGEEnMrvQzZ/FlEFfuBHF9R a9ER7tdAaShR6AslcZU/UjSRP70ujnieafp2FEATmLyK6Q6EFOfqWBSFv0VsZQfeKTtD pK+WyciABVKhtgBsUIsILVz1I6FEXFVteUECHArklmwsApacQogcpRJ3am3iQ85eBkRs 7y95nGxsPPfKgsw7nTVPW8xXymQMxxHCmHS6UMhjCvNChYnJRCzsLdVqfwok0gp2scbr vezg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AOUpUlEEjcb7yhxZokpNYW7u5wjnKPML83nt2eE/oYdQSOtXWwslG716 HGk8p/OT7Ub4Tr+GVPavzspg7thTtyEmib8DGh6I1BK9+s5tRPbOOqIxsTxg5tGsgxCVSGo1R4g OdVvUS1XhCZ3fHqMvbnk1YvrKLvXpFgN+0myqyN6uDcmJWC2t7cw33xzHeZwLiC0ksQ== X-Received: by 2002:a62:1089:: with SMTP id 9-v6mr285064pfq.30.1531826510573; Tue, 17 Jul 2018 04:21:50 -0700 (PDT) X-Google-Smtp-Source: AAOMgpd5/+NWgcp8b60LX5q2/rJcTKAVJtvk2yTDtop/bxjnG6QonJfGDKXwwRHu5/ZoB5iv5e1d X-Received: by 2002:a62:1089:: with SMTP id 9-v6mr285021pfq.30.1531826509775; Tue, 17 Jul 2018 04:21:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531826509; cv=none; d=google.com; s=arc-20160816; b=ppB12uKgGUyh9Tc979yO/gbVimmFn9rOOcxqeQOzia0TKs8KRvqgy+hN4Ngx4b877w FdtZNbYY/RZfRfXc771sVQmGhF+ipRD5QUqcujNco0pzxL2XejuWgV/GrT3MPBpV6AQD BYqDIJzGJvEk03c2tXVcfiCA3h95QvRNnVw2mF7b1/0fq218P84wPuxT/8EQNMxKCg8P a1I8ty8gD5VkDW9iWtdKJHGSWdRfp2nIigSAwcZVqZtJ5w7f71T8uzQNOx8JMHty45FR g3+91R/NVFjO6JgB/0ULtbPFavxB8bZ+Chf8cx9xvuMVHWBfQY8VfNPm9P5wdj5W2pZl wNLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=4hSUWDZCUBTeeDsArIAUszGAWEjInBF+D7KzAtvRX+4=; b=XHfAxXf5bhyZOSHvGeSetnQw4b3wKM1ME/1oX8M75rBmvxeRUm49F0sQJJZUCJGPI2 7Ro2muchDk2pMyDm7c3u6oEHw+aHSn3/8lMYnCUvoLId482vfH9zHo7Xab24/wqVaGfa 5lP6fOFoZ08bl7W7l8YQt2D0V+3SoUnJeBYYvNQKgLkyjVNN3jScwaxRSlw7ZInGqS2q vsrYIDUyWufWkshbdka9buD9McJkCAFp0AMgL7c9Ppl/+bPuqo95N2uvU88BV4RmzqyK vKHAilaN+2eq8WlPsdJgypZPKjOoZ8rLIL2CCpaVK2r+9s2BMRgiKwXeuQxJssqLh3yk H8BQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga17.intel.com (mga17.intel.com. [192.55.52.151]) by mx.google.com with ESMTPS id f62-v6si697286pfg.165.2018.07.17.04.21.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Jul 2018 04:21:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.151 as permitted sender) client-ip=192.55.52.151; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jul 2018 04:21:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,365,1526367600"; d="scan'208";a="67585445" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga003.jf.intel.com with ESMTP; 17 Jul 2018 04:21:45 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 3154E74D; Tue, 17 Jul 2018 14:21:49 +0300 (EEST) From: "Kirill A. Shutemov" To: Ingo Molnar , x86@kernel.org, Thomas Gleixner , "H. Peter Anvin" , Tom Lendacky Cc: Dave Hansen , Kai Huang , Jacob Pan , linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv5 12/19] x86/mm: Implement prep_encrypted_page() and arch_free_page() Date: Tue, 17 Jul 2018 14:20:22 +0300 Message-Id: <20180717112029.42378-13-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180717112029.42378-1-kirill.shutemov@linux.intel.com> References: <20180717112029.42378-1-kirill.shutemov@linux.intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The hardware/CPU does not enforce coherency between mappings of the same physical page with different KeyIDs or encryption keys. We are responsible for cache management. Flush cache on allocating encrypted page and on returning the page to the free pool. prep_encrypted_page() also takes care about zeroing the page. We have to do this after KeyID is set for the page. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/mktme.h | 6 +++++ arch/x86/mm/mktme.c | 49 ++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/arch/x86/include/asm/mktme.h b/arch/x86/include/asm/mktme.h index f0b7844e36a4..44409b8bbaca 100644 --- a/arch/x86/include/asm/mktme.h +++ b/arch/x86/include/asm/mktme.h @@ -19,6 +19,12 @@ int page_keyid(const struct page *page); #define vma_keyid vma_keyid int vma_keyid(struct vm_area_struct *vma); +#define prep_encrypted_page prep_encrypted_page +void prep_encrypted_page(struct page *page, int order, int keyid, bool zero); + +#define HAVE_ARCH_FREE_PAGE +void arch_free_page(struct page *page, int order); + #else #define mktme_keyid_mask ((phys_addr_t)0) #define mktme_nr_keyids 0 diff --git a/arch/x86/mm/mktme.c b/arch/x86/mm/mktme.c index a1f40ee61b25..1194496633ce 100644 --- a/arch/x86/mm/mktme.c +++ b/arch/x86/mm/mktme.c @@ -1,4 +1,5 @@ #include +#include #include phys_addr_t mktme_keyid_mask; @@ -49,3 +50,51 @@ int vma_keyid(struct vm_area_struct *vma) prot = pgprot_val(vma->vm_page_prot); return (prot & mktme_keyid_mask) >> mktme_keyid_shift; } + +void prep_encrypted_page(struct page *page, int order, int keyid, bool zero) +{ + int i; + + /* It's not encrypted page: nothing to do */ + if (!keyid) + return; + + /* + * The hardware/CPU does not enforce coherency between mappings of the + * same physical page with different KeyIDs or encryption keys. + * We are responsible for cache management. + * + * We flush cache before allocating encrypted page + */ + clflush_cache_range(page_address(page), PAGE_SIZE << order); + + for (i = 0; i < (1 << order); i++) { + /* All pages coming out of the allocator should have KeyID 0 */ + WARN_ON_ONCE(lookup_page_ext(page)->keyid); + lookup_page_ext(page)->keyid = keyid; + + /* Clear the page after the KeyID is set. */ + if (zero) + clear_highpage(page); + + page++; + } +} + +void arch_free_page(struct page *page, int order) +{ + int i; + + /* It's not encrypted page: nothing to do */ + if (!page_keyid(page)) + return; + + clflush_cache_range(page_address(page), PAGE_SIZE << order); + + for (i = 0; i < (1 << order); i++) { + /* Check if the page has reasonable KeyID */ + WARN_ON_ONCE(lookup_page_ext(page)->keyid > mktme_nr_keyids); + lookup_page_ext(page)->keyid = 0; + page++; + } +}