From patchwork Fri Dec 21 18:14:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Stoppa X-Patchwork-Id: 10740813 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2327D924 for ; Fri, 21 Dec 2018 18:14:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1060D284AA for ; Fri, 21 Dec 2018 18:14:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0462F28622; Fri, 21 Dec 2018 18:14:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6CC27284AA for ; Fri, 21 Dec 2018 18:14:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5C3A8E0007; Fri, 21 Dec 2018 13:14:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A35B38E0001; Fri, 21 Dec 2018 13:14:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B1578E0007; Fri, 21 Dec 2018 13:14:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lj1-f199.google.com (mail-lj1-f199.google.com [209.85.208.199]) by kanga.kvack.org (Postfix) with ESMTP id 1E9BC8E0001 for ; Fri, 21 Dec 2018 13:14:52 -0500 (EST) Received: by mail-lj1-f199.google.com with SMTP id l12-v6so1870840ljb.11 for ; Fri, 21 Dec 2018 10:14:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:reply-to:mime-version :content-transfer-encoding; bh=IC72E1SkxQby2tnvUgxalWUUCvjmMLY+yVrjL3hsqg8=; b=BF1L1DraIB64y4N/pao4A6qaQbQBt7cuAqr7uJLAGfwuhTfZFm75UTj7jbJHLga6KJ 6Yl6o/8+e33iWK5jDZGpju18i02Lgk5I+wtgIHAljFhgsh5fx7Chw/iXwDeIiYRA0LLR V1juS+NRMQs8RVPwXxZnUVqBT5/b2GV0o1/tbxMC4bLeXa6JIhme0q3W6zBAFNeAQzgg 4tQO5rZZNx/rGkDqWTdHljPXfgbto26tn9+ZQdnGzwq/afKcUgLJOiD8lsTK4vhGqcYG oFSElBwPz84AaF9u9Cyp9F+oHmX3EKgc8YtyK5ww5Y9kelTErf+j4w3URceyKGC21AmS uhWA== X-Gm-Message-State: AA+aEWZ+QWLvNNlsrkBEUDn7GHD63NNT5PjmL8MWeJ5LK61RlUnoultj ddi12b8udPoxQ7BkzvTtdAsiG9I7r6ellTvnvWLBNkgHYKMfczofTmzaps7ntQH+2kJ8jITlB6t lsLMxEl4LKg/yVD4z7+N7IOnkc5HLYLx8KWYtxlPkTw4iVV1bVNzvqwAfsvf5KHaMKPpksk+yW2 3CODwxdCrw9Hnoloa+beJlJ1xiBjAB7Y9f+OsFhkEswK0wlM8PI0wjjW+FwnWUbXBRmg7vpDcZy WmOUllLn8TJwR0S44p6YHvXoAyyNZUfbP/0Od0WTOtMOfLhankFkckotpsGNDiZUPe/ht56yp0W 2CordzGVBmTe3bvI+1gA0j3I3f9qf+rZdyA/jhogAIo8Y1yvYx/E25Rjs8Y1GwgbkbWZr6bhrJH r X-Received: by 2002:a19:5e5d:: with SMTP id z29mr1960434lfi.105.1545416091457; Fri, 21 Dec 2018 10:14:51 -0800 (PST) X-Received: by 2002:a19:5e5d:: with SMTP id z29mr1960406lfi.105.1545416090366; Fri, 21 Dec 2018 10:14:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1545416090; cv=none; d=google.com; s=arc-20160816; b=zKvvsFY/PGV+tEQPYLXg6finnY7VE5dc73RIp4pZ9dInzpexxUU9pHv/AAEMbnDbwP vuao/vWJEkD27GagU6b/lc/iPiMtejaXj1cSRJ13AGvF1KC2IdkTvq4F5UyL+z0Abv5s SiQk0JF2oJFsJUPaq6IIX4vs5OJEseBUw3lH3KWgBWgfzrEhcw4Dqz/0ilUzAMeEJfzp XSYe8q3Io/q3xMW36WWPN/30elHCOXIiXPTzgGpJFzoET2Yx7kPgS8nfTbm4K2Lg8SAI Lu6Fjhu5aOg2A7jmXO9vhLC0ZZFLP2OePsEkHhKKMknrCw/nGNwbjGOIK2YZpHh2+LsL S8EQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature; bh=IC72E1SkxQby2tnvUgxalWUUCvjmMLY+yVrjL3hsqg8=; b=Cd/XedVn9A/DUfRXqYF6Z9OibtUbwPs2+S2lWfQIrCUSXKtpv0cMMSTRxxnLyCGloB C3yadExQwlDgS2JNMa+QoAYTLPDwRmDNniNPNwd8skQp89b/ZOWpo/Wkjgd072FWyb6i iSn9I75dQ8nCWvCHWsHwp6LHigA8cjF60+2VdHpHw2gLa316iJtD9h2pxtmvESikbDn3 QfKBu/FdkSas9B9/3Jp2psYQPanG1R+0jvcMZijaln1qDTrt//wa2KYqGAhRgK9kg2N1 wYqFc657yFq3NZIq/Hk8hh07zHvKN39g2Xw1Q9OwQAsktbWj6M1YmJqmOj9UlvgOk8T6 ovcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=DqfsEH5L; spf=pass (google.com: domain of igor.stoppa@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=igor.stoppa@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id a12-v6sor15952723lji.34.2018.12.21.10.14.50 for (Google Transport Security); Fri, 21 Dec 2018 10:14:50 -0800 (PST) Received-SPF: pass (google.com: domain of igor.stoppa@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=DqfsEH5L; spf=pass (google.com: domain of igor.stoppa@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=igor.stoppa@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=IC72E1SkxQby2tnvUgxalWUUCvjmMLY+yVrjL3hsqg8=; b=DqfsEH5L9YLdOJEXEYum6nwamDbHwKJwJCpeilO+Exni09zMXlLazjQWwG6UxGeyFW cthZ8/pTG4OViX9K/JQWfmJtS9F33LP1hb71YBXNsYDRjNxCpqV8NAzPvx7VOuAH5Tsc LPXPMnueI2EidByLqVwQ22AzvLQ144Ldw2HPEGliQyeXceXtPW1uaP/Sg86LlkmwCPHw /WOXCOYsw5cASKVRpV1s+hAEGzNAktCNUUXk5jWzgAJ/4jfpfwYuqnCr8CnsDS11dTI1 Ntel2Ew+lNIzH4pDzhUWw44FbLcUVoKFuI4LeX7oBViWiQnjx4Wqptqvm437EweXZB72 NKwQ== X-Google-Smtp-Source: AFSGD/UxlqHZez0NWd/HSVJmkToecB7S72thDSg2Cw/dydhEx7i1lDETl2T5I7D9XKZozHyTtVweIQ== X-Received: by 2002:a2e:4746:: with SMTP id u67-v6mr2238745lja.142.1545416089950; Fri, 21 Dec 2018 10:14:49 -0800 (PST) Received: from localhost.localdomain (dmhwpt3bffxn8z3-j6k-4.rev.dnainternet.fi. [2001:14bb:51:a4c8:5c24:24d7:ca5f:e7d2]) by smtp.gmail.com with ESMTPSA id m63-v6sm5444564lje.81.2018.12.21.10.14.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Dec 2018 10:14:49 -0800 (PST) From: Igor Stoppa X-Google-Original-From: Igor Stoppa To: Andy Lutomirski , Matthew Wilcox , Peter Zijlstra , Dave Hansen , Mimi Zohar , Thiago Jung Bauermann Cc: igor.stoppa@huawei.com, Nadav Amit , Kees Cook , Ahmed Soliman , linux-integrity@vger.kernel.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 03/12] __wr_after_init: generic functionality Date: Fri, 21 Dec 2018 20:14:14 +0200 Message-Id: <20181221181423.20455-4-igor.stoppa@huawei.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181221181423.20455-1-igor.stoppa@huawei.com> References: <20181221181423.20455-1-igor.stoppa@huawei.com> Reply-To: Igor Stoppa MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The patch provides: - the generic part of the write rare functionality for static data, based on code from Matthew Wilcox - the dummy functionality, in case an arch doesn't support write rare or the functionality is disabled The basic functions are: - wr_memset(): write rare counterpart of memset() - wr_memcpy(): write rare counterpart of memcpy() - wr_assign(): write rare counterpart of the assignment ('=') operator - wr_rcu_assign_pointer(): write rare counterpart of rcu_assign_pointer() Signed-off-by: Igor Stoppa CC: Andy Lutomirski CC: Nadav Amit CC: Matthew Wilcox CC: Peter Zijlstra CC: Kees Cook CC: Dave Hansen CC: Mimi Zohar CC: Thiago Jung Bauermann CC: Ahmed Soliman CC: linux-integrity@vger.kernel.org CC: kernel-hardening@lists.openwall.com CC: linux-mm@kvack.org CC: linux-kernel@vger.kernel.org --- include/linux/prmem.h | 106 ++++++++++++++++++++++++++++++++++++++++++ mm/Makefile | 1 + mm/prmem.c | 97 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 204 insertions(+) create mode 100644 include/linux/prmem.h create mode 100644 mm/prmem.c diff --git a/include/linux/prmem.h b/include/linux/prmem.h new file mode 100644 index 000000000000..12c1d0d1cb78 --- /dev/null +++ b/include/linux/prmem.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * prmem.h: Header for memory protection library + * + * (C) Copyright 2018 Huawei Technologies Co. Ltd. + * Author: Igor Stoppa + * + * Support for: + * - statically allocated write rare data + */ + +#ifndef _LINUX_PRMEM_H +#define _LINUX_PRMEM_H + +#include +#include +#include + + +/** + * memtst() - test len bytes starting at p to match the c value + * @p: beginning of the memory to test + * @c: byte to compare against + * @len: amount of bytes to test + * + * Returns 0 on success, non-zero otherwise. + */ +static inline int memtst(void *p, int c, __kernel_size_t len) +{ + __kernel_size_t i; + + for (i = 0; i < len; i++) { + u8 d = *(i + (u8 *)p) - (u8)c; + + if (unlikely(d)) + return d; + } + return 0; +} + + +#ifndef CONFIG_PRMEM + +static inline void *wr_memset(void *p, int c, __kernel_size_t len) +{ + return memset(p, c, len); +} + +static inline void *wr_memcpy(void *p, const void *q, __kernel_size_t size) +{ + return memcpy(p, q, size); +} + +#define wr_assign(var, val) ((var) = (val)) +#define wr_rcu_assign_pointer(p, v) rcu_assign_pointer(p, v) + +#else + +#include +#include +#include +#include + +#include + +void *wr_memset(void *p, int c, __kernel_size_t len); +void *wr_memcpy(void *p, const void *q, __kernel_size_t size); + +/** + * wr_assign() - sets a write-rare variable to a specified value + * @var: the variable to set + * @val: the new value + * + * Returns: the variable + * + * Note: it might be possible to optimize this, to use wr_memset in some + * cases (maybe with NULL?). + */ + +#define wr_assign(var, val) ({ \ + typeof(var) tmp = (typeof(var))val; \ + \ + wr_memcpy(&var, &tmp, sizeof(var)); \ + var; \ +}) + +/** + * wr_rcu_assign_pointer() - initialize a pointer in rcu mode + * @p: the rcu pointer - it MUST be aligned to a machine word + * @v: the new value + * + * Returns the value assigned to the rcu pointer. + * + * It is provided as macro, to match rcu_assign_pointer() + * The rcu_assign_pointer() is implemented as equivalent of: + * + * smp_mb(); + * WRITE_ONCE(); + */ +#define wr_rcu_assign_pointer(p, v) ({ \ + smp_mb(); \ + wr_assign(p, v); \ + p; \ +}) +#endif +#endif diff --git a/mm/Makefile b/mm/Makefile index d210cc9d6f80..ef3867c16ce0 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -58,6 +58,7 @@ obj-$(CONFIG_SPARSEMEM) += sparse.o obj-$(CONFIG_SPARSEMEM_VMEMMAP) += sparse-vmemmap.o obj-$(CONFIG_SLOB) += slob.o obj-$(CONFIG_MMU_NOTIFIER) += mmu_notifier.o +obj-$(CONFIG_PRMEM) += prmem.o obj-$(CONFIG_KSM) += ksm.o obj-$(CONFIG_PAGE_POISONING) += page_poison.o obj-$(CONFIG_SLAB) += slab.o diff --git a/mm/prmem.c b/mm/prmem.c new file mode 100644 index 000000000000..e1c1be3a1171 --- /dev/null +++ b/mm/prmem.c @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * prmem.c: Memory Protection Library + * + * (C) Copyright 2017-2018 Huawei Technologies Co. Ltd. + * Author: Igor Stoppa + */ + +#include +#include +#include +#include +#include +#include +#include + +__ro_after_init bool wr_ready; + +/* + * The following two variables are statically allocated by the linker + * script at the the boundaries of the memory region (rounded up to + * multiples of PAGE_SIZE) reserved for __wr_after_init. + */ +extern long __start_wr_after_init; +extern long __end_wr_after_init; +static unsigned long start = (unsigned long)&__start_wr_after_init; +static unsigned long end = (unsigned long)&__end_wr_after_init; + +static inline bool is_wr_after_init(void *p, __kernel_size_t size) +{ + unsigned long low = (unsigned long)p; + unsigned long high = low + size; + + return likely(start <= low && high <= end); +} + +/** + * wr_memcpy() - copyes size bytes from q to p + * @p: beginning of the memory to write to + * @q: beginning of the memory to read from + * @size: amount of bytes to copy + * + * Returns pointer to the destination + * + * The architecture code must provide: + * void __wr_enable(wr_state_t *state) + * void *__wr_addr(void *addr) + * void *__wr_memcpy(void *p, const void *q, __kernel_size_t size) + * void __wr_disable(wr_state_t *state) + */ +void *wr_memcpy(void *p, const void *q, __kernel_size_t size) +{ + wr_state_t wr_state; + void *wr_poking_addr = __wr_addr(p); + + if (WARN_ONCE(!wr_ready, "No writable mapping available") || + WARN_ONCE(!is_wr_after_init(p, size), "Invalid WR range.")) + return p; + + local_irq_disable(); + __wr_enable(&wr_state); + __wr_memcpy(wr_poking_addr, q, size); + __wr_disable(&wr_state); + local_irq_enable(); + return p; +} + +/** + * wr_memset() - sets len bytes of the destination p to the c value + * @p: beginning of the memory to write to + * @c: byte to replicate + * @len: amount of bytes to copy + * + * Returns pointer to the destination + * + * The architecture code must provide: + * void __wr_enable(wr_state_t *state) + * void *__wr_addr(void *addr) + * void *__wr_memset(void *p, int c, __kernel_size_t len) + * void __wr_disable(wr_state_t *state) + */ +void *wr_memset(void *p, int c, __kernel_size_t len) +{ + wr_state_t wr_state; + void *wr_poking_addr = __wr_addr(p); + + if (WARN_ONCE(!wr_ready, "No writable mapping available") || + WARN_ONCE(!is_wr_after_init(p, len), "Invalid WR range.")) + return p; + + local_irq_disable(); + __wr_enable(&wr_state); + __wr_memset(wr_poking_addr, c, len); + __wr_disable(&wr_state); + local_irq_enable(); + return p; +}