| Message ID | 20170216154348.19244.11884.stgit@tlendack-t1.amdoffice.net (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Thu, Feb 16, 2017 at 09:43:48AM -0600, Tom Lendacky wrote: > Add to the early_memremap support to be able to specify encrypted and early_memremap() Please append "()" to function names in your commit messages text. > decrypted mappings with and without write-protection. The use of > write-protection is necessary when encrypting data "in place". The > write-protect attribute is considered cacheable for loads, but not > stores. This implies that the hardware will never give the core a > dirty line with this memtype. By "hardware will never give" you mean that WP writes won't land dirty in the cache but will go out to mem and when some other core needs them, they will have to come from memory? > Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> > --- > arch/x86/Kconfig | 4 +++ > arch/x86/include/asm/fixmap.h | 13 ++++++++++ > arch/x86/include/asm/pgtable_types.h | 8 ++++++ > arch/x86/mm/ioremap.c | 44 ++++++++++++++++++++++++++++++++++ > include/asm-generic/early_ioremap.h | 2 ++ > mm/early_ioremap.c | 10 ++++++++ > 6 files changed, 81 insertions(+) > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig > index a3b8c71..581eae4 100644 > --- a/arch/x86/Kconfig > +++ b/arch/x86/Kconfig > @@ -1417,6 +1417,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT > If set to N, then the encryption of system memory can be > activated with the mem_encrypt=on command line option. > > +config ARCH_USE_MEMREMAP_PROT > + def_bool y > + depends on AMD_MEM_ENCRYPT Why do we need this? IOW, all those helpers below will end up being defined unconditionally, in practice. Think distro kernels. Then saving the couple of bytes is not really worth the overhead.
On 2/20/2017 9:43 AM, Borislav Petkov wrote: > On Thu, Feb 16, 2017 at 09:43:48AM -0600, Tom Lendacky wrote: >> Add to the early_memremap support to be able to specify encrypted and > > early_memremap() > > Please append "()" to function names in your commit messages text. > >> decrypted mappings with and without write-protection. The use of >> write-protection is necessary when encrypting data "in place". The >> write-protect attribute is considered cacheable for loads, but not >> stores. This implies that the hardware will never give the core a >> dirty line with this memtype. > > By "hardware will never give" you mean that WP writes won't land dirty > in the cache but will go out to mem and when some other core needs them, > they will have to come from memory? I think this best explains it, from Table 7-8 of the APM Vol 2: "Reads allocate cache lines on a cache miss, but only to the shared state. All writes update main memory. Cache lines are not allocated on a write miss. Write hits invalidate the cache line and update main memory." We're early enough that only the BSP is running and we don't have to worry about accesses from other cores. If this was to be used outside of early boot processing, then some safeties might have to be added. > >> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> >> --- >> arch/x86/Kconfig | 4 +++ >> arch/x86/include/asm/fixmap.h | 13 ++++++++++ >> arch/x86/include/asm/pgtable_types.h | 8 ++++++ >> arch/x86/mm/ioremap.c | 44 ++++++++++++++++++++++++++++++++++ >> include/asm-generic/early_ioremap.h | 2 ++ >> mm/early_ioremap.c | 10 ++++++++ >> 6 files changed, 81 insertions(+) >> >> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig >> index a3b8c71..581eae4 100644 >> --- a/arch/x86/Kconfig >> +++ b/arch/x86/Kconfig >> @@ -1417,6 +1417,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT >> If set to N, then the encryption of system memory can be >> activated with the mem_encrypt=on command line option. >> >> +config ARCH_USE_MEMREMAP_PROT >> + def_bool y >> + depends on AMD_MEM_ENCRYPT > > Why do we need this? > > IOW, all those helpers below will end up being defined unconditionally, > in practice. Think distro kernels. Then saving the couple of bytes is > not really worth the overhead. I added this because some other architectures use a u64 for the protection value instead of an unsigned long (i386 for one) and it was causing build errors/warnings on those archs. And trying to bring in the header to use pgprot_t instead of an unsigned long caused a ton of build issues. This seemed to be the simplest and least intrusive way to approach the issue. Thanks, Tom >
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index a3b8c71..581eae4 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1417,6 +1417,10 @@ config AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT If set to N, then the encryption of system memory can be activated with the mem_encrypt=on command line option. +config ARCH_USE_MEMREMAP_PROT + def_bool y + depends on AMD_MEM_ENCRYPT + # Common NUMA Features config NUMA bool "Numa Memory Allocation and Scheduler Support" diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h index 83e91f0..8233373 100644 --- a/arch/x86/include/asm/fixmap.h +++ b/arch/x86/include/asm/fixmap.h @@ -160,6 +160,19 @@ static inline void __set_fixmap(enum fixed_addresses idx, */ #define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_IO_NOCACHE +/* + * Early memremap routines used for in-place encryption. The mappings created + * by these routines are intended to be used as temporary mappings. + */ +void __init *early_memremap_encrypted(resource_size_t phys_addr, + unsigned long size); +void __init *early_memremap_encrypted_wp(resource_size_t phys_addr, + unsigned long size); +void __init *early_memremap_decrypted(resource_size_t phys_addr, + unsigned long size); +void __init *early_memremap_decrypted_wp(resource_size_t phys_addr, + unsigned long size); + #include <asm-generic/fixmap.h> #define __late_set_fixmap(idx, phys, flags) __set_fixmap(idx, phys, flags) diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h index 500fc60..f00e70f 100644 --- a/arch/x86/include/asm/pgtable_types.h +++ b/arch/x86/include/asm/pgtable_types.h @@ -161,6 +161,7 @@ enum page_cache_mode { #define _PAGE_CACHE_MASK (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT) #define _PAGE_NOCACHE (cachemode2protval(_PAGE_CACHE_MODE_UC)) +#define _PAGE_CACHE_WP (cachemode2protval(_PAGE_CACHE_MODE_WP)) #define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED) #define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \ @@ -189,6 +190,7 @@ enum page_cache_mode { #define __PAGE_KERNEL_VVAR (__PAGE_KERNEL_RO | _PAGE_USER) #define __PAGE_KERNEL_LARGE (__PAGE_KERNEL | _PAGE_PSE) #define __PAGE_KERNEL_LARGE_EXEC (__PAGE_KERNEL_EXEC | _PAGE_PSE) +#define __PAGE_KERNEL_WP (__PAGE_KERNEL | _PAGE_CACHE_WP) #define __PAGE_KERNEL_IO (__PAGE_KERNEL) #define __PAGE_KERNEL_IO_NOCACHE (__PAGE_KERNEL_NOCACHE) @@ -202,6 +204,12 @@ enum page_cache_mode { #define _KERNPG_TABLE (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED | \ _PAGE_DIRTY | _PAGE_ENC) +#define __PAGE_KERNEL_ENC (__PAGE_KERNEL | _PAGE_ENC) +#define __PAGE_KERNEL_ENC_WP (__PAGE_KERNEL_WP | _PAGE_ENC) + +#define __PAGE_KERNEL_NOENC (__PAGE_KERNEL) +#define __PAGE_KERNEL_NOENC_WP (__PAGE_KERNEL_WP) + #define PAGE_KERNEL __pgprot(__PAGE_KERNEL | _PAGE_ENC) #define PAGE_KERNEL_RO __pgprot(__PAGE_KERNEL_RO | _PAGE_ENC) #define PAGE_KERNEL_EXEC __pgprot(__PAGE_KERNEL_EXEC | _PAGE_ENC) diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c index c43b6b3..2385e70 100644 --- a/arch/x86/mm/ioremap.c +++ b/arch/x86/mm/ioremap.c @@ -419,6 +419,50 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr) iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK)); } +#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT +/* Remap memory with encryption */ +void __init *early_memremap_encrypted(resource_size_t phys_addr, + unsigned long size) +{ + return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC); +} + +/* + * Remap memory with encryption and write-protected - cannot be called + * before pat_init() is called + */ +void __init *early_memremap_encrypted_wp(resource_size_t phys_addr, + unsigned long size) +{ + /* Be sure the write-protect PAT entry is set for write-protect */ + if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP) + return NULL; + + return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_ENC_WP); +} + +/* Remap memory without encryption */ +void __init *early_memremap_decrypted(resource_size_t phys_addr, + unsigned long size) +{ + return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC); +} + +/* + * Remap memory without encryption and write-protected - cannot be called + * before pat_init() is called + */ +void __init *early_memremap_decrypted_wp(resource_size_t phys_addr, + unsigned long size) +{ + /* Be sure the write-protect PAT entry is set for write-protect */ + if (__pte2cachemode_tbl[_PAGE_CACHE_MODE_WP] != _PAGE_CACHE_MODE_WP) + return NULL; + + return early_memremap_prot(phys_addr, size, __PAGE_KERNEL_NOENC_WP); +} +#endif /* CONFIG_ARCH_USE_MEMREMAP_PROT */ + static pte_t bm_pte[PAGE_SIZE/sizeof(pte_t)] __page_aligned_bss; static inline pmd_t * __init early_ioremap_pmd(unsigned long addr) diff --git a/include/asm-generic/early_ioremap.h b/include/asm-generic/early_ioremap.h index 734ad4d..2edef8d 100644 --- a/include/asm-generic/early_ioremap.h +++ b/include/asm-generic/early_ioremap.h @@ -13,6 +13,8 @@ extern void *early_memremap(resource_size_t phys_addr, unsigned long size); extern void *early_memremap_ro(resource_size_t phys_addr, unsigned long size); +extern void *early_memremap_prot(resource_size_t phys_addr, + unsigned long size, unsigned long prot_val); extern void early_iounmap(void __iomem *addr, unsigned long size); extern void early_memunmap(void *addr, unsigned long size); diff --git a/mm/early_ioremap.c b/mm/early_ioremap.c index 6d5717b..d7d30da 100644 --- a/mm/early_ioremap.c +++ b/mm/early_ioremap.c @@ -226,6 +226,16 @@ void __init early_iounmap(void __iomem *addr, unsigned long size) } #endif +#ifdef CONFIG_ARCH_USE_MEMREMAP_PROT +void __init * +early_memremap_prot(resource_size_t phys_addr, unsigned long size, + unsigned long prot_val) +{ + return (__force void *)__early_ioremap(phys_addr, size, + __pgprot(prot_val)); +} +#endif + #define MAX_MAP_CHUNK (NR_FIX_BTMAPS << PAGE_SHIFT) void __init copy_from_early_mem(void *dest, phys_addr_t src, unsigned long size)
Add to the early_memremap support to be able to specify encrypted and decrypted mappings with and without write-protection. The use of write-protection is necessary when encrypting data "in place". The write-protect attribute is considered cacheable for loads, but not stores. This implies that the hardware will never give the core a dirty line with this memtype. Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> --- arch/x86/Kconfig | 4 +++ arch/x86/include/asm/fixmap.h | 13 ++++++++++ arch/x86/include/asm/pgtable_types.h | 8 ++++++ arch/x86/mm/ioremap.c | 44 ++++++++++++++++++++++++++++++++++ include/asm-generic/early_ioremap.h | 2 ++ mm/early_ioremap.c | 10 ++++++++ 6 files changed, 81 insertions(+)