From patchwork Fri May 18 11:30:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 10410033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AB57A602C2 for ; Fri, 18 May 2018 11:30:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C10928948 for ; Fri, 18 May 2018 11:30:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9A16F2894C; Fri, 18 May 2018 11:30:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D950F28948 for ; Fri, 18 May 2018 11:30:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 318E56B05D2; Fri, 18 May 2018 07:30:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2C9B46B05D3; Fri, 18 May 2018 07:30:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E1226B05D4; Fri, 18 May 2018 07:30:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f70.google.com (mail-pl0-f70.google.com [209.85.160.70]) by kanga.kvack.org (Postfix) with ESMTP id CDD786B05D2 for ; Fri, 18 May 2018 07:30:38 -0400 (EDT) Received: by mail-pl0-f70.google.com with SMTP id e1-v6so3451656pld.23 for ; Fri, 18 May 2018 04:30:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id; bh=IpUUU+AUffup74h+XmxL2kHa6DlhqjoSr3MyrwE1Ll0=; b=LB01h26XmbZMch0V5MCfFFUcTrOL8eio8LI1p7ib3fOpccQmgFoWOn/nyXWqC4L3Jy MqJ/ZU1y83SLXL9gjRVhfssde8sihKwpvSL06IWJLu2hJyhaScP3U4v1nmC3MSVmtE46 RpODvOOdzW0RzsnKQMoXpbcLBN/URtNHsM4xxiSKiRHSXDyhoqMBfrG3t1fV44TRfbGa mRilZ64zq3XfIvMR4gBelLBWtU4QbV8RaiXANfctlb/OKgFB9LMM30M0Q6Y3YjNL0lF6 JhGnZPznQoW6d6w0o3cDowXphDjgRdhju2rSRvvGIPgnkB8p12Ip8YsVpCGqOJQF0h4H VKbw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ALKqPweJ/CbDv2VJWOkBayctFpuD7NppLz3S1NLCJmGBOJk1pQaHMgp9 G36ASxsoigimAI3PapVcvaq79vd88Wpf7epxj1GquJ285KoWGKJ3FUdnhWyUxDESCETEapQC/Zg pGjSLxR/EMGY55GoB+l7FMzG5R0PLfKfrZh78vx6dQMZwXCidW5NYwv4NfAUERVL4UA== X-Received: by 2002:a17:902:9883:: with SMTP id s3-v6mr9291983plp.179.1526643038482; Fri, 18 May 2018 04:30:38 -0700 (PDT) X-Google-Smtp-Source: AB8JxZoqAVEq3Mbxz/KKjZc/i//80RSFBw7/T0vHhfFgpzeDd0idyK2fYxpQWDtsnZMvju+a8hct X-Received: by 2002:a17:902:9883:: with SMTP id s3-v6mr9291902plp.179.1526643037366; Fri, 18 May 2018 04:30:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526643037; cv=none; d=google.com; s=arc-20160816; b=PAAReu8q22cXJdgb1GEhr8hHE145EGY10tKyC4OdiWAjGzuweO1CkeTbHhaL//yxaM lYRNGxPK3Jy3b5OmTs9wdPiAuHtH/pN1gqvpSWC1Tj2vM82IQ3IC7XZgUXh2xV1PbEpF ItvMdC4X0NtGcm+HtG0i0oAJyxy9sC5F9Hqiky2KGHQnwvbjmN47+AIaVxySbZ08ovzy Iu08G63X91rlNEcuE4H1yEJGVRYY7wVptho8NgMTq6Y3Gz5Gp5lk4sKinBNlenLArZgk 1PT5pV5g5g5Ni4iXuOFAiHjHbdwp5TGMjIj70ws8Bo/P5ciMbPXQLgEVeA9+ufXMlMkP K7bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:arc-authentication-results; bh=IpUUU+AUffup74h+XmxL2kHa6DlhqjoSr3MyrwE1Ll0=; b=t1dy87wWp3q3uyyCdA5EEg+HtdAuoFqYySNjHQzecmSawlAXd4CtSzrRxRsqofcEtG yxzWQQgiBnpLLbUbeKzR8N90ue0rNsYUoeA2zqd24DRHE9LjXR84FnKTkN9Orc8YOppw kVzgm4+3KPOLvcI6cU3ehba8i23sErXATNyzsot3tNwFfpx14prqCzCt6wWguoHHyD8o 7orJGAyPE3HmGOwWUwnO30alNTmMmcSY8pqS8FKmsmdE9B9AvcNbnKYkuTgTv7Ha1n1G DcX0JyNHOHl/qGXgvnwQY42H5DDZoTfQIt3zwncDHrMHjcz1Gfjoxc488pOmMCWwZZrd /Vaw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga14.intel.com (mga14.intel.com. [192.55.52.115]) by mx.google.com with ESMTPS id d9-v6si7313933pls.334.2018.05.18.04.30.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 May 2018 04:30:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.115 as permitted sender) client-ip=192.55.52.115; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of kirill.shutemov@linux.intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 May 2018 04:30:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,414,1520924400"; d="scan'208";a="59721963" Received: from black.fi.intel.com ([10.237.72.28]) by orsmga002.jf.intel.com with ESMTP; 18 May 2018 04:30:33 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id EE63915D; Fri, 18 May 2018 14:30:32 +0300 (EEST) From: "Kirill A. Shutemov" To: Ingo Molnar , x86@kernel.org, Thomas Gleixner , "H. Peter Anvin" Cc: Tom Lendacky , linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCH, RESEND] x86/mm: Decouple dynamic __PHYSICAL_MASK from AMD SME Date: Fri, 18 May 2018 14:30:28 +0300 Message-Id: <20180518113028.79825-1-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.17.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP AMD SME claims one bit from physical address to indicate whether the page is encrypted or not. To achieve that we clear out the bit from __PHYSICAL_MASK. The capability to adjust __PHYSICAL_MASK is required beyond AMD SME. For instance for upcoming Intel Multi-Key Total Memory Encryption. Factor it out into a separate feature with own Kconfig handle. It also helps with overhead of AMD SME. It saves more than 3k in .text on defconfig + AMD_MEM_ENCRYPT: add/remove: 3/2 grow/shrink: 5/110 up/down: 189/-3753 (-3564) We would need to return to this once we have infrastructure to patch constants in code. That's good candidate for it. Signed-off-by: Kirill A. Shutemov Reviewed-by: Tom Lendacky --- arch/x86/Kconfig | 4 ++++ arch/x86/boot/compressed/kaslr_64.c | 5 +++++ arch/x86/include/asm/page_types.h | 8 +++++++- arch/x86/mm/mem_encrypt_identity.c | 3 +++ arch/x86/mm/pgtable.c | 5 +++++ 5 files changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index bcdd3e0e2ef5..8e2d0ee0e366 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -333,6 +333,9 @@ config ARCH_SUPPORTS_UPROBES config FIX_EARLYCON_MEM def_bool y +config DYNAMIC_PHYSICAL_MASK + bool + config PGTABLE_LEVELS int default 5 if X86_5LEVEL @@ -1504,6 +1507,7 @@ config ARCH_HAS_MEM_ENCRYPT config AMD_MEM_ENCRYPT bool "AMD Secure Memory Encryption (SME) support" depends on X86_64 && CPU_SUP_AMD + select DYNAMIC_PHYSICAL_MASK ---help--- Say yes to enable support for the encryption of system memory. This requires an AMD processor that supports Secure Memory diff --git a/arch/x86/boot/compressed/kaslr_64.c b/arch/x86/boot/compressed/kaslr_64.c index 522d11431433..748456c365f4 100644 --- a/arch/x86/boot/compressed/kaslr_64.c +++ b/arch/x86/boot/compressed/kaslr_64.c @@ -69,6 +69,8 @@ static struct alloc_pgt_data pgt_data; /* The top level page table entry pointer. */ static unsigned long top_level_pgt; +phys_addr_t physical_mask = (1ULL << __PHYSICAL_MASK_SHIFT) - 1; + /* * Mapping information structure passed to kernel_ident_mapping_init(). * Due to relocation, pointers must be assigned at run time not build time. @@ -81,6 +83,9 @@ void initialize_identity_maps(void) /* If running as an SEV guest, the encryption mask is required. */ set_sev_encryption_mask(); + /* Exclude the encryption mask from __PHYSICAL_MASK */ + physical_mask &= ~sme_me_mask; + /* Init mapping_info with run-time function/buffer pointers. */ mapping_info.alloc_pgt_page = alloc_pgt_page; mapping_info.context = &pgt_data; diff --git a/arch/x86/include/asm/page_types.h b/arch/x86/include/asm/page_types.h index 1e53560a84bb..c85e15010f48 100644 --- a/arch/x86/include/asm/page_types.h +++ b/arch/x86/include/asm/page_types.h @@ -17,7 +17,6 @@ #define PUD_PAGE_SIZE (_AC(1, UL) << PUD_SHIFT) #define PUD_PAGE_MASK (~(PUD_PAGE_SIZE-1)) -#define __PHYSICAL_MASK ((phys_addr_t)(__sme_clr((1ULL << __PHYSICAL_MASK_SHIFT) - 1))) #define __VIRTUAL_MASK ((1UL << __VIRTUAL_MASK_SHIFT) - 1) /* Cast *PAGE_MASK to a signed type so that it is sign-extended if @@ -55,6 +54,13 @@ #ifndef __ASSEMBLY__ +#ifdef CONFIG_DYNAMIC_PHYSICAL_MASK +extern phys_addr_t physical_mask; +#define __PHYSICAL_MASK physical_mask +#else +#define __PHYSICAL_MASK ((phys_addr_t)((1ULL << __PHYSICAL_MASK_SHIFT) - 1)) +#endif + extern int devmem_is_allowed(unsigned long pagenr); extern unsigned long max_low_pfn_mapped; diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c index 1b2197d13832..7ae36868aed2 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -527,6 +527,7 @@ void __init sme_enable(struct boot_params *bp) /* SEV state cannot be controlled by a command line option */ sme_me_mask = me_mask; sev_enabled = true; + physical_mask &= ~sme_me_mask; return; } @@ -561,4 +562,6 @@ void __init sme_enable(struct boot_params *bp) sme_me_mask = 0; else sme_me_mask = active_by_default ? me_mask : 0; + + physical_mask &= ~sme_me_mask; } diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 34cda7e0551b..0199b94e6b40 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -7,6 +7,11 @@ #include #include +#ifdef CONFIG_DYNAMIC_PHYSICAL_MASK +phys_addr_t physical_mask __ro_after_init = (1ULL << __PHYSICAL_MASK_SHIFT) - 1; +EXPORT_SYMBOL(physical_mask); +#endif + #define PGALLOC_GFP (GFP_KERNEL_ACCOUNT | __GFP_ZERO) #ifdef CONFIG_HIGHPTE