From patchwork Fri Dec 21 18:14:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Stoppa X-Patchwork-Id: 10740837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1B436C5 for ; Fri, 21 Dec 2018 18:15:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D3D1E28426 for ; Fri, 21 Dec 2018 18:15:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C72CD28618; Fri, 21 Dec 2018 18:15:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id AB5A328426 for ; Fri, 21 Dec 2018 18:15:36 +0000 (UTC) Received: (qmail 32625 invoked by uid 550); 21 Dec 2018 18:15:05 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 32547 invoked from network); 21 Dec 2018 18:15:05 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=SN1YxRssqD/qbsJs6cWB56zoRuTrhLbtpCLtQvrTRuc=; b=TKdyKx/jrCPmAFRDV+JRiJANdImKAj933DMPS306pGi86XGkwj+0veYOqtcDmcyrjO vCfsX0EjPqV6JxEspMuCD7syOFu+fW7Ls0bgV1b4ZlTcTyIi5Oql5zNbSuc+80jK+/VX DM/4H8lprBVKz14tLSiADp5ICDkz9mVWmG/oMRH2xA0HOSQ6dNu953TqUVZjoQMNoDru 8FUS4oIGexrnmUsUPtj5ID/wr0hQgkpGKK+v8UuNNqz65pIaMEGbkr9wNg/hB4jMcz59 55PPj9akWYr8I0WhAzAG2xKK3oAvQD03wecwMctx3PGb9u8gNaL91wY0bxv5f9mXqxAo e80w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=SN1YxRssqD/qbsJs6cWB56zoRuTrhLbtpCLtQvrTRuc=; b=BJDZeYRAKC/Cd3ck5hqaB6eIW9EAKp3jqPhuy8PhspEJ2I9PiDQIR9lkGXpdX774Yv ONSSzefqrLQfAES5XCEtyclRPX092s03LImYxX7gx7JOnc3MXRZ6QH3UgE5wp8Shkmt5 IHCSoqIAKxBP9hqp8CBEsuDhHGXwSuHxLR9Pm0q0hlhTpuEyMcYl2RFyYPbxsIH+4ujl oCjy13J5VY86AwdfeoB0lgtKzQsh80npSm+hP/axqEqHPLId1SLtvCjcwoY8gE0Gq5x6 NHVD5HgWngR4tSMHm1TMmI3qEYBYfMSjMes2JLTs9MCxTIg4ehFvuNnHxiyazIE0Dh2a /MWQ== X-Gm-Message-State: AJcUukdG2QwWsnNPXb0ZqkJPxuejyT5OdyzDiqftsu2nl1EyBBQggDb5 ey4ZSH0uyNcx4AeZch2pipA= X-Google-Smtp-Source: ALg8bN6yuIckmNbKpfr+n67Or7uaJ2oFluosJDTSIQkUgvEiIqLM5ThHxUlnxLGzEbhgxJM4/tnQfw== X-Received: by 2002:a2e:98c9:: with SMTP id s9-v6mr1937913ljj.166.1545416094071; Fri, 21 Dec 2018 10:14:54 -0800 (PST) From: Igor Stoppa X-Google-Original-From: Igor Stoppa To: Andy Lutomirski , Matthew Wilcox , Peter Zijlstra , Dave Hansen , Mimi Zohar , Thiago Jung Bauermann Cc: igor.stoppa@huawei.com, Nadav Amit , Kees Cook , Ahmed Soliman , linux-integrity@vger.kernel.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 05/12] __wr_after_init: x86_64: __wr_op Date: Fri, 21 Dec 2018 20:14:16 +0200 Message-Id: <20181221181423.20455-6-igor.stoppa@huawei.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181221181423.20455-1-igor.stoppa@huawei.com> References: <20181221181423.20455-1-igor.stoppa@huawei.com> MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP Architecture-specific implementation of the core write rare operation. The implementation is based on code from Andy Lutomirski and Nadav Amit for patching the text on x86 [here goes reference to commits, once merged] The modification of write protected data is done through an alternate mapping of the same pages, as writable. This mapping is persistent, but active only for a core that is performing a write rare operation. And only for the duration of said operation. Local interrupts are disabled, while the alternate mapping is active. In theory, it could introduce a non-predictable delay, in a preemptible system, however the amount of data to be altered is likely to be far smaller than a page. Signed-off-by: Igor Stoppa CC: Andy Lutomirski CC: Nadav Amit CC: Matthew Wilcox CC: Peter Zijlstra CC: Kees Cook CC: Dave Hansen CC: Mimi Zohar CC: Thiago Jung Bauermann CC: Ahmed Soliman CC: linux-integrity@vger.kernel.org CC: kernel-hardening@lists.openwall.com CC: linux-mm@kvack.org CC: linux-kernel@vger.kernel.org --- arch/x86/Kconfig | 1 + arch/x86/include/asm/prmem.h | 72 ++++++++++++++++++++++++++++++++++++ arch/x86/mm/Makefile | 2 + arch/x86/mm/prmem.c | 69 ++++++++++++++++++++++++++++++++++ 4 files changed, 144 insertions(+) create mode 100644 arch/x86/include/asm/prmem.h create mode 100644 arch/x86/mm/prmem.c diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 8689e794a43c..e5e4fc4fa5c2 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -32,6 +32,7 @@ config X86_64 select SWIOTLB select X86_DEV_DMA_OPS select ARCH_HAS_SYSCALL_WRAPPER + select ARCH_HAS_PRMEM # # Arch settings diff --git a/arch/x86/include/asm/prmem.h b/arch/x86/include/asm/prmem.h new file mode 100644 index 000000000000..e1f09f881351 --- /dev/null +++ b/arch/x86/include/asm/prmem.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * prmem.h: Header for memory protection library + * + * (C) Copyright 2018 Huawei Technologies Co. Ltd. + * Author: Igor Stoppa + * + * Support for: + * - statically allocated write rare data + */ + +#ifndef _ASM_X86_PRMEM_H +#define _ASM_X86_PRMEM_H + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +typedef temporary_mm_state_t wr_state_t; + +extern __ro_after_init struct mm_struct *wr_poking_mm; +extern __ro_after_init unsigned long wr_poking_base; + +static inline void *__wr_addr(void *addr) +{ + return (void *)(wr_poking_base + (unsigned long)addr); +} + +static inline void __wr_enable(wr_state_t *state) +{ + *state = use_temporary_mm(wr_poking_mm); +} + +static inline void __wr_disable(wr_state_t *state) +{ + unuse_temporary_mm(*state); +} + + +/** + * __wr_memset() - sets len bytes of the destination p to the c value + * @p: beginning of the memory to write to + * @c: byte to replicate + * @len: amount of bytes to copy + * + * Returns pointer to the destination + */ +static inline void *__wr_memset(void *p, int c, __kernel_size_t len) +{ + return (void *)memset_user((void __user *)p, (u8)c, len); +} + +/** + * __wr_memcpy() - copyes size bytes from q to p + * @p: beginning of the memory to write to + * @q: beginning of the memory to read from + * @size: amount of bytes to copy + * + * Returns pointer to the destination + */ +static inline void *__wr_memcpy(void *p, const void *q, __kernel_size_t size) +{ + return (void *)copy_to_user((void __user *)p, q, size); +} + +#endif diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index 4b101dd6e52f..66652de1e2c7 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -53,3 +53,5 @@ obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_boot.o + +obj-$(CONFIG_PRMEM) += prmem.o diff --git a/arch/x86/mm/prmem.c b/arch/x86/mm/prmem.c new file mode 100644 index 000000000000..f4b36baa2f19 --- /dev/null +++ b/arch/x86/mm/prmem.c @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * prmem.c: Memory Protection Library + * + * (C) Copyright 2017-2018 Huawei Technologies Co. Ltd. + * Author: Igor Stoppa + */ + +#include +#include +#include +#include +#include +#include + +extern __ro_after_init bool wr_ready; +__ro_after_init struct mm_struct *wr_poking_mm; +__ro_after_init unsigned long wr_poking_base; + +/* + * The following two variables are statically allocated by the linker + * script at the the boundaries of the memory region (rounded up to + * multiples of PAGE_SIZE) reserved for __wr_after_init. + */ +extern long __start_wr_after_init; +extern long __end_wr_after_init; + +struct mm_struct *copy_init_mm(void); +void __init wr_poking_init(void) +{ + unsigned long start = (unsigned long)&__start_wr_after_init; + unsigned long end = (unsigned long)&__end_wr_after_init; + unsigned long i; + + wr_poking_mm = copy_init_mm(); + if (WARN_ONCE(!wr_poking_mm, "No alternate mapping available.")) + return; + + /* + * Place 64TB of kernel address space within 128TB of user address + * space, at a random page aligned offset. + */ + wr_poking_base = (((unsigned long)kaslr_get_random_long("WR Poke")) & + PAGE_MASK) % (64 * _BITUL(40)); + + /* Create alternate mapping for the entire wr_after_init range. */ + for (i = start; i < end; i += PAGE_SIZE) { + struct page *page; + spinlock_t *ptl; + pte_t pte; + pte_t *ptep; + unsigned long wr_poking_addr; + + page = virt_to_page(i); + if (WARN_ONCE(!page, "WR memory without physical page")) + return; + wr_poking_addr = i + wr_poking_base; + + /* The lock is not needed, but avoids open-coding. */ + ptep = get_locked_pte(wr_poking_mm, wr_poking_addr, &ptl); + if (WARN_ONCE(!ptep, "No pte for writable mapping")) + return; + + pte = mk_pte(page, PAGE_KERNEL); + set_pte_at(wr_poking_mm, wr_poking_addr, ptep, pte); + spin_unlock(ptl); + } + wr_ready = true; +}