From patchwork Tue Jun 26 14:22:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "kirill.shutemov@linux.intel.com" X-Patchwork-Id: 10489189 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6B097602D8 for ; Tue, 26 Jun 2018 14:23:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B6AA1FF13 for ; Tue, 26 Jun 2018 14:23:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4F97E1FFE6; Tue, 26 Jun 2018 14:23:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D424A1FF13 for ; Tue, 26 Jun 2018 14:23:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D637E6B027D; Tue, 26 Jun 2018 10:23:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9E5A46B0285; Tue, 26 Jun 2018 10:23:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FF996B027E; Tue, 26 Jun 2018 10:23:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f71.google.com (mail-pl0-f71.google.com [209.85.160.71]) by kanga.kvack.org (Postfix) with ESMTP id EBF806B0280 for ; Tue, 26 Jun 2018 10:23:04 -0400 (EDT) Received: by mail-pl0-f71.google.com with SMTP id s3-v6so10135749plp.21 for ; Tue, 26 Jun 2018 07:23:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=4hSUWDZCUBTeeDsArIAUszGAWEjInBF+D7KzAtvRX+4=; b=n/vQzD5LKdNp+4LsVKCs8CSlUOUpEIu/fp7ITczwvnoBM8etq5qM7qr9L3cIr4G5J4 6qijJsS3yn07mdhzdV5YtvTWJnMzarJQLb1f4mbslrd+X12OC957OdR2V4KwBSZekDGv ea0chHDzaTg70Ee7SYT5R8DO2JI7V+/rQRZBOkJLJ0whcq7sOU07BuETunAHoh8R/2TE +/MoD3PYr4zgieZSiS/F4zd5RvaKCbmEBI9O1NR3noO+LtvRsRBGlWXMf+kGPodX1fFg 1GDVV+tDrV8w4d9Wc/8uvDVULS23A2tyi0J0PLGJ5hG5okxKNk6mLZWSCofKxJdwDOB2 njUw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: APt69E2wfYVcFTxC9Mba5bxg2ymsbM42VQ13oLpb2OfXo/hNjsdzOn2m Gh34z/v9rWnZvIyPlbbxXFAz0XlJU6OMcJND01BrLQJ65mxc+2zXq1f8jTZ7qWBgcH6vLJN4nY6 NPT+gZ6v8oc6xYwTmz7zsTP27JrSun39tqnP4avYpHn/Lop625oi8K1RUCxgKUjJ2Pg== X-Received: by 2002:a62:db05:: with SMTP id f5-v6mr1752291pfg.123.1530022984427; Tue, 26 Jun 2018 07:23:04 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdq99yCRqf0TfGKZ4CBOVCtmmukODuOy2fIH+gnu2UXPz2ePVz3/wSlrs34hhQv5h234MJf X-Received: by 2002:a62:db05:: with SMTP id f5-v6mr1751840pfg.123.1530022976759; Tue, 26 Jun 2018 07:22:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530022976; cv=none; d=google.com; s=arc-20160816; b=cMJclGjrY4YfqvqZaMJamgmLKv38EdGmPeNuAEvWeGhnnok9oRTWqBH5NEEXzTS58W Rc4bPlRadevhPy3jYz/Gy/RKXvk8QW1wChkUmopLvA2KaKH56vwm0w13nbXcwpv/XPZu Nfj1LZahYY4obVqknkrzoZqCMl7lkZ+EH++MkOThXFgYYLVgnr7mQbVZ7i5uZD1nGbcJ RM3GYjjzHKKxVbzNfgguu2Bcyg9RvCvxhmgj9AETSzDgSxJiqOp3u/1KNvZSQyJDX5bY dgXFKI29ZfEyEXfOvdMcOdEm/Euvotr/xSKhvhf18wuANuMB8/COl2SwKwtX3dR4D00B KImw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=4hSUWDZCUBTeeDsArIAUszGAWEjInBF+D7KzAtvRX+4=; b=s359YKFWSAaB6Z2AstPxjHUVAmjMMurLRk7qNpJwTV+sx2TsfLQlSbgffA9GijXbqQ QSR2RTaQTtl5Srr1u7Ke7B45USKuu5WVpqlcLOsvayU5Ji4PZLAGmNG+ery1zKbegTUz KU9Lx0sdVF6lc85dbMU10UyQD7eDsEDKC1tNe/+sNJbY17S8KUPIWwX7c06BlntGWSn8 /EKrQL1mmMCKhjMpuRun1XQTqDHGWMX3SQI6NNRUccbkCLtLitrGo1LjDpbG9j7D+1F5 VGFyGvL/k57bAfbrR6tCaQFKq8qwoxfGeI5/huohhQJJFdQdNQlX2cF2wJNi7ST3zz0m 9EpQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga11.intel.com (mga11.intel.com. [192.55.52.93]) by mx.google.com with ESMTPS id y16-v6si1608398pfl.11.2018.06.26.07.22.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Jun 2018 07:22:56 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.93 as permitted sender) client-ip=192.55.52.93; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Jun 2018 07:22:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,274,1526367600"; d="scan'208";a="62318729" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga003.jf.intel.com with ESMTP; 26 Jun 2018 07:22:53 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 05D2C7AA; Tue, 26 Jun 2018 17:22:51 +0300 (EEST) From: "Kirill A. Shutemov" To: Ingo Molnar , x86@kernel.org, Thomas Gleixner , "H. Peter Anvin" , Tom Lendacky Cc: Dave Hansen , Kai Huang , Jacob Pan , linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv4 11/18] x86/mm: Implement prep_encrypted_page() and arch_free_page() Date: Tue, 26 Jun 2018 17:22:38 +0300 Message-Id: <20180626142245.82850-12-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180626142245.82850-1-kirill.shutemov@linux.intel.com> References: <20180626142245.82850-1-kirill.shutemov@linux.intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The hardware/CPU does not enforce coherency between mappings of the same physical page with different KeyIDs or encryption keys. We are responsible for cache management. Flush cache on allocating encrypted page and on returning the page to the free pool. prep_encrypted_page() also takes care about zeroing the page. We have to do this after KeyID is set for the page. Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/mktme.h | 6 +++++ arch/x86/mm/mktme.c | 49 ++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/arch/x86/include/asm/mktme.h b/arch/x86/include/asm/mktme.h index f0b7844e36a4..44409b8bbaca 100644 --- a/arch/x86/include/asm/mktme.h +++ b/arch/x86/include/asm/mktme.h @@ -19,6 +19,12 @@ int page_keyid(const struct page *page); #define vma_keyid vma_keyid int vma_keyid(struct vm_area_struct *vma); +#define prep_encrypted_page prep_encrypted_page +void prep_encrypted_page(struct page *page, int order, int keyid, bool zero); + +#define HAVE_ARCH_FREE_PAGE +void arch_free_page(struct page *page, int order); + #else #define mktme_keyid_mask ((phys_addr_t)0) #define mktme_nr_keyids 0 diff --git a/arch/x86/mm/mktme.c b/arch/x86/mm/mktme.c index a1f40ee61b25..1194496633ce 100644 --- a/arch/x86/mm/mktme.c +++ b/arch/x86/mm/mktme.c @@ -1,4 +1,5 @@ #include +#include #include phys_addr_t mktme_keyid_mask; @@ -49,3 +50,51 @@ int vma_keyid(struct vm_area_struct *vma) prot = pgprot_val(vma->vm_page_prot); return (prot & mktme_keyid_mask) >> mktme_keyid_shift; } + +void prep_encrypted_page(struct page *page, int order, int keyid, bool zero) +{ + int i; + + /* It's not encrypted page: nothing to do */ + if (!keyid) + return; + + /* + * The hardware/CPU does not enforce coherency between mappings of the + * same physical page with different KeyIDs or encryption keys. + * We are responsible for cache management. + * + * We flush cache before allocating encrypted page + */ + clflush_cache_range(page_address(page), PAGE_SIZE << order); + + for (i = 0; i < (1 << order); i++) { + /* All pages coming out of the allocator should have KeyID 0 */ + WARN_ON_ONCE(lookup_page_ext(page)->keyid); + lookup_page_ext(page)->keyid = keyid; + + /* Clear the page after the KeyID is set. */ + if (zero) + clear_highpage(page); + + page++; + } +} + +void arch_free_page(struct page *page, int order) +{ + int i; + + /* It's not encrypted page: nothing to do */ + if (!page_keyid(page)) + return; + + clflush_cache_range(page_address(page), PAGE_SIZE << order); + + for (i = 0; i < (1 << order); i++) { + /* Check if the page has reasonable KeyID */ + WARN_ON_ONCE(lookup_page_ext(page)->keyid > mktme_nr_keyids); + lookup_page_ext(page)->keyid = 0; + page++; + } +}