From patchwork Tue Apr 7 14:46:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478241 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 245D6174A for ; Tue, 7 Apr 2020 14:47:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 024FC2072A for ; Tue, 7 Apr 2020 14:47:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="jbcg3qDQ"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="jNjrdO2j" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 024FC2072A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tOQsL1UwAyeSD7aVc86VJnzoqXGbBrzuC9r/s6opFAQ=; b=jbcg3qDQZnNLoU mPiTc7evJ+w43wvBUJqAzHJ7qS3b9ES9eS+v6BWwaw3f6nTuy/Xuvjuypql+uzmB9v+KkuDiMrBGn cfjhlyq5zIi5HnjzGxSdMByr9WdgVcj0aCvFTmk14iZwvNgJPJJxFti2SWy7xzj6QEej/d1vczwsA k080Hcr7tQuJ1kojGxV5f6opiY6kbkR/i+5LSyXrT3GejKIW9SUi6XxpXDhXk1WM4Fca4wSXP6eJ4 njKjO4Ih/pgBfN3t8xd7sGEdUJ/eoDl7HY06rgoK9a8jDUmzBrzfRtwazmMaUZSB24ice5U9hulQ9 G2AvzFxakRR/jCy3rvrw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVI-00069y-E8; Tue, 07 Apr 2020 14:47:04 +0000 Received: from mail-pf1-x441.google.com ([2607:f8b0:4864:20::441]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVF-000683-Bg for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:02 +0000 Received: by mail-pf1-x441.google.com with SMTP id u65so908058pfb.4 for ; Tue, 07 Apr 2020 07:47:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tOQsL1UwAyeSD7aVc86VJnzoqXGbBrzuC9r/s6opFAQ=; b=jNjrdO2jGlUgSUD9yX89DPR2/2/A55WbbxmEk3D0BPwG9P1qqJkWGiOZ6K9PUE6NI4 ggW+RiFHKkMdsNH4G7JFuLFLB5oFS2LD3ruTt4WslQzsgp4tDE7Cfb8tUftvsBRs3alZ cRRTKvMno8t3cQ4xm6nyN5D0ZzW605zJkEX/DPb47MrtJhxxIEYXO48Fi6A2BOsIJkuu W4CCEZS9aswddXKsB5i3wPQ+2evp1ZGk6FxrS1iQhRL//uz2e11ktzh4CsfIymvLlXCB 2IISADHqCpS4TEXjaRGAoEN64Ct+cjeGOUPvrfx5zDlrSRDrt7GYwCwHG4f3SsBRy6sD gefw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tOQsL1UwAyeSD7aVc86VJnzoqXGbBrzuC9r/s6opFAQ=; b=hMj59sDtqR5T+9dHA+ka1re2zLul8jRN7UNVFmL+hfvA31O5RzqqVb2LHUXWb6+MCO XteEdV3JXqD9e8wcbKpveIjlubvBH2o1bCcvKRM9UoCG7NUc+6vHPtPv8eEiVtXNQYAE TMqabDHEExBehJ/eyLpn6BSItMG4pObduo2JhrHDLyhGj2MRlDATytYd6Pe3kNUeaqlK xYAVXw3gns3AcqL+gCLELQouyNOFyqQ/SkJRbvy5SpAgHKPFO4FHnAa3ufzsDDogzr9X 5CI0Q25+EhelFg5mRX4RGHkwbEQP32lt9uanVpyyLAj/Y4SO6kD/FjAfEMdvCozHemzt 6MDA== X-Gm-Message-State: AGi0PuaUHZy6dMQNN/qKTF1W/oP4wX4h3Aa1oCGTyOKoLXTOQZHohxPf Ve832fxs23epIv021lrpOinme+S00Vk= X-Google-Smtp-Source: APiQypJSbH9Ho0ehuvytIuygjbLMQeR8OPPI6bu3qmrH4iy9fTjQrkoOscBf4NTGkjiU3CFXYbADLA== X-Received: by 2002:a63:a052:: with SMTP id u18mr2473808pgn.210.1586270820635; Tue, 07 Apr 2020 07:47:00 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.46.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:00 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 1/9] riscv: add macro to get instruction length Date: Tue, 7 Apr 2020 22:46:46 +0800 Message-Id: <621303b9cdea215af57329b401b15750c1f683ab.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074701_394927_961B8E5E X-CRM114-Status: UNSURE ( 9.15 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:441 listed in] [list.dnswl.org] X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Extract the calculation of instruction length for common use. Signed-off-by: Zong Li --- arch/riscv/include/asm/bug.h | 8 ++++++++ arch/riscv/kernel/traps.c | 3 ++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/bug.h b/arch/riscv/include/asm/bug.h index 75604fec1b1b..d6f1ec08d97b 100644 --- a/arch/riscv/include/asm/bug.h +++ b/arch/riscv/include/asm/bug.h @@ -19,6 +19,14 @@ #define __BUG_INSN_32 _UL(0x00100073) /* ebreak */ #define __BUG_INSN_16 _UL(0x9002) /* c.ebreak */ +#define GET_INSN_LENGTH(insn) \ +({ \ + unsigned long __len; \ + __len = ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? \ + 4UL : 2UL; \ + __len; \ +}) + typedef u32 bug_insn_t; #ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index ffb3d94bf0cc..a4d136355f78 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -118,7 +118,8 @@ static inline unsigned long get_break_insn_length(unsigned long pc) if (probe_kernel_address((bug_insn_t *)pc, insn)) return 0; - return (((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? 4UL : 2UL); + + return GET_INSN_LENGTH(insn); } asmlinkage __visible void do_trap_break(struct pt_regs *regs) From patchwork Tue Apr 7 14:46:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478243 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C4373174A for ; Tue, 7 Apr 2020 14:47:11 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F38E2076E for ; Tue, 7 Apr 2020 14:47:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="aqzdUqty"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="P1AeqVor" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F38E2076E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=c7uXdqcuiavHxznT4lBdSfVCr+fRg7NRHIyZNAY7wD8=; b=aqzdUqtyUKVHCe SbrEiXo/ii+CfG4+hzXR2qzfoh/Hio4HmQKbYo1Dsfi1aAN4mb608GW4T4CY9MeinP7RjNojyChB6 PMZ2YZrQM/r8Wq286odk6T7yTmJtwTT0xjyGCVRhZ/yz/yLxwxCge/MTQ1a7Ihv/vYdYWYs88EX/8 73vfybxDY6Pd5ayRb4Ukt+E8G6o2bT+RpbKEmt7F/gAH7gM3qBEru8dXJLBkcMGezfV4GB2SzPoIS uLt+1qICPk0Xntk/K08W2ICmpxzVVfWJPechjcatRMQFqPlQKZ8FX/604rI+/mGcxPG4ShNDpj4Vs W7GMD1FsVRqaU9SmiJuQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVK-0006Cx-J8; Tue, 07 Apr 2020 14:47:06 +0000 Received: from mail-pj1-x1041.google.com ([2607:f8b0:4864:20::1041]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVH-00068W-Da for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:05 +0000 Received: by mail-pj1-x1041.google.com with SMTP id nu11so858273pjb.1 for ; Tue, 07 Apr 2020 07:47:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c7uXdqcuiavHxznT4lBdSfVCr+fRg7NRHIyZNAY7wD8=; b=P1AeqVorlKShJdhSjCpTu5VNpL+Za5BmA1B3JGSOcSzJ7IsnKO3GBuODRscgWd8G4+ U72bLLhsvWgXXJ9Z1Ys3LQZ496OgU8thm3JeaaBzwwPjF/H3FQJkdS03gosEpC5ynCmL nY8/ZsqWzGMBTjzKMdq8js54lm2TGULQE7TTAQS8L5PfeMfte0gj0xFKVGk9ezdp9Fc7 w9U63uhPj5rz3outYretqeP1s6X1T5TiNUFE37tqUZ095aTt0Y09G07DpmSKngy9k33Z qOUfgd9dMFrG2P/+48+pVd6EsnmDd0hNYKATwaX5XJVyR/fax2lE16ib2MIitH8AiofC e8hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c7uXdqcuiavHxznT4lBdSfVCr+fRg7NRHIyZNAY7wD8=; b=PzKGBCNjdGYnMpMzXXrD4i7W2BqPkBwv8yGWzz2fVWmx4z40Hfs/dj1wn1pvQ1skQu Pto0AqUfLs7znmdUQfNA4XOMedVlq1S5Ul2d9JkAoXTwv+MGitgfTjhvz/PRwDiVWyJ/ /8wJPKSCnSGOlbPfm0+tqPAG0z482UNU59+PFXg7ioyyhuD5/ibeGZt+rDbAsbDDoXMS z/WOX55cHyqb64jT1cLOoyt0WUzMEZXD+vgiwa788QbC8rTS2A+Iz/bW/LZoSioKb5HF JvX+YRrdsjjyeaJTzMZjcNDXLukfNzQqW1ILEcraiRnON4wn6iVpMlWiTk2B2V78f/fD /FKA== X-Gm-Message-State: AGi0PuZssCk3agqXdDUGT9rM2fYabcZvya6Q1FTPLOehLckORiYA/g1W oPHhDySBjEnoHWB3405jXlKesA== X-Google-Smtp-Source: APiQypILA25M+qgwgYPvpmKFU28Lv6GclMDIO1ifCU+inCy8FTJZT6CORAEugRPYmfSQcdv9GSOt8w== X-Received: by 2002:a17:90b:3714:: with SMTP id mg20mr3197336pjb.127.1586270822588; Tue, 07 Apr 2020 07:47:02 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:02 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 2/9] riscv: introduce interfaces to patch kernel code Date: Tue, 7 Apr 2020 22:46:47 +0800 Message-Id: X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074703_454609_E279ED12 X-CRM114-Status: GOOD ( 15.40 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:1041 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li Suggested-by: Masami Hiramatsu --- arch/riscv/include/asm/fixmap.h | 2 + arch/riscv/include/asm/patch.h | 12 ++++ arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/patch.c | 120 ++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/patch.h create mode 100644 arch/riscv/kernel/patch.c diff --git a/arch/riscv/include/asm/fixmap.h b/arch/riscv/include/asm/fixmap.h index 42d2c42f3cc9..2368d49eb4ef 100644 --- a/arch/riscv/include/asm/fixmap.h +++ b/arch/riscv/include/asm/fixmap.h @@ -27,6 +27,8 @@ enum fixed_addresses { FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1, FIX_PTE, FIX_PMD, + FIX_TEXT_POKE1, + FIX_TEXT_POKE0, FIX_EARLYCON_MEM_BASE, __end_of_fixed_addresses }; diff --git a/arch/riscv/include/asm/patch.h b/arch/riscv/include/asm/patch.h new file mode 100644 index 000000000000..9a7d7346001e --- /dev/null +++ b/arch/riscv/include/asm/patch.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2020 SiFive + */ + +#ifndef _ASM_RISCV_PATCH_H +#define _ASM_RISCV_PATCH_H + +int patch_text_nosync(void *addr, const void *insns, size_t len); +int patch_text(void *addr, u32 insn); + +#endif /* _ASM_RISCV_PATCH_H */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index f40205cb9a22..d189bd3d8501 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -4,7 +4,8 @@ # ifdef CONFIG_FTRACE -CFLAGS_REMOVE_ftrace.o = -pg +CFLAGS_REMOVE_ftrace.o = -pg +CFLAGS_REMOVE_patch.o = -pg endif extra-y += head.o @@ -26,6 +27,7 @@ obj-y += traps.o obj-y += riscv_ksyms.o obj-y += stacktrace.o obj-y += cacheinfo.o +obj-y += patch.o obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_M_MODE) += clint.o diff --git a/arch/riscv/kernel/patch.c b/arch/riscv/kernel/patch.c new file mode 100644 index 000000000000..5b4f0d37097f --- /dev/null +++ b/arch/riscv/kernel/patch.c @@ -0,0 +1,120 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 SiFive + */ + +#include +#include +#include +#include +#include +#include +#include + +struct patch_insn_patch { + void *addr; + u32 insn; + atomic_t cpu_count; +}; + +#ifdef CONFIG_MMU +static void *patch_map(void *addr, int fixmap) +{ + uintptr_t uintaddr = (uintptr_t) addr; + struct page *page; + + if (core_kernel_text(uintaddr)) + page = phys_to_page(__pa_symbol(addr)); + else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX)) + page = vmalloc_to_page(addr); + else + return addr; + + BUG_ON(!page); + + return (void *)set_fixmap_offset(fixmap, page_to_phys(page) + + (uintaddr & ~PAGE_MASK)); +} +NOKPROBE_SYMBOL(patch_map); + +static void patch_unmap(int fixmap) +{ + clear_fixmap(fixmap); +} +NOKPROBE_SYMBOL(patch_unmap); + +static int patch_insn_write(void *addr, const void *insn, size_t len) +{ + void *waddr = addr; + bool across_pages = (((uintptr_t) addr & ~PAGE_MASK) + len) > PAGE_SIZE; + int ret; + + if (across_pages) + patch_map(addr + len, FIX_TEXT_POKE1); + + waddr = patch_map(addr, FIX_TEXT_POKE0); + + ret = probe_kernel_write(waddr, insn, len); + + patch_unmap(FIX_TEXT_POKE0); + + if (across_pages) + patch_unmap(FIX_TEXT_POKE1); + + return ret; +} +NOKPROBE_SYMBOL(patch_insn_write); +#else +static int patch_insn_write(void *addr, const void *insn, size_t len) +{ + return probe_kernel_write(addr, insn, len); +} +NOKPROBE_SYMBOL(patch_insn_write); +#endif /* CONFIG_MMU */ + +int patch_text_nosync(void *addr, const void *insns, size_t len) +{ + u32 *tp = addr; + int ret; + + ret = patch_insn_write(tp, insns, len); + + if (!ret) + flush_icache_range((uintptr_t) tp, (uintptr_t) tp + len); + + return ret; +} +NOKPROBE_SYMBOL(patch_text_nosync); + +static int patch_text_cb(void *data) +{ + struct patch_insn_patch *patch = data; + int ret = 0; + + if (atomic_inc_return(&patch->cpu_count) == 1) { + ret = + patch_text_nosync(patch->addr, &patch->insn, + GET_INSN_LENGTH(patch->insn)); + atomic_inc(&patch->cpu_count); + } else { + while (atomic_read(&patch->cpu_count) <= num_online_cpus()) + cpu_relax(); + smp_mb(); + } + + return ret; +} +NOKPROBE_SYMBOL(patch_text_cb); + +int patch_text(void *addr, u32 insn) +{ + struct patch_insn_patch patch = { + .addr = addr, + .insn = insn, + .cpu_count = ATOMIC_INIT(0), + }; + + return stop_machine_cpuslocked(patch_text_cb, + &patch, cpu_online_mask); +} +NOKPROBE_SYMBOL(patch_text); From patchwork Tue Apr 7 14:46:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478245 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D22D81 for ; Tue, 7 Apr 2020 14:47:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B2162076E for ; Tue, 7 Apr 2020 14:47:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="GD+c0Md0"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="Dp2cyuiV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B2162076E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kYuVFCggRo9I8GvVGPv9n9TMuc3y+S8enCWZ0r9b2Rg=; b=GD+c0Md0v7B5Hx cffoYkK41lUhY51FpCpi1K6iJWjnYKRE6KyXPjNzXNEkaFWGfivrHcCn/fBuuD+OBy9/EgsBH3uKE Q5h7SsnuHtFjTHjTcGwdvaqmjd6CyCHTu/pd1U0tbQDBq3p/oEnjz86sH3H+NtOxWZEGgwNG3aFsb om0BFaCfa2/ifbu2B/X3PSrlIVKvWN514En2I0P34RbAN54nMoSqwTjHIB17qxPNTKUUosET4ntfc qi+wjYbYchyTm0QolZwlwCTfeSpd6ISozcAsVIKW8QfLqVl65ACjHr+iE3dhjgzeX43f0DvHjyZNm 7AywkuaNaIYoeK3vTlTQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVM-0006GO-PL; Tue, 07 Apr 2020 14:47:08 +0000 Received: from mail-pj1-x1041.google.com ([2607:f8b0:4864:20::1041]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVI-0006AZ-S0 for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:06 +0000 Received: by mail-pj1-x1041.google.com with SMTP id fh8so850305pjb.5 for ; Tue, 07 Apr 2020 07:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kYuVFCggRo9I8GvVGPv9n9TMuc3y+S8enCWZ0r9b2Rg=; b=Dp2cyuiV3Zq2urQAMh7yiuZ3tl4kdNTj0CnDofGx5CYcevhuKy6HPSG4E7W9b0Q7Y0 2uRjZtKohnI7E8ZgkiEw8px7x+qMhXPYizZnM37q5PEOBdTSXVfkFm1rceB9zklVQOkX 1LPAhRh1htLQX11lcxKXodjYAR/tQZSpZ31ln21dus97h07gwSbhgcUgQebQncnDJVxG FTmWLZgq3KfB2/OjR+yXpBM8YXWRHKJp+WLQ2haAVfKXzKbFkVmcTQIrch4ZHhaQgeM+ pnouesMXOGAoFz4QI/DgUrFHdTMGktQ4jJzfrqzYGwFkYxCBwnkXraurAUu5y22zedFW bdyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kYuVFCggRo9I8GvVGPv9n9TMuc3y+S8enCWZ0r9b2Rg=; b=EKTmE1/OmWUkUSOpltAYXjNyUCgZR/IaBXzKfWsGAHKB1sh9D4alogNBI0Ah1SjuuQ ggEtg/A0Pi5xUSdWHJ6JMYLaAqrxOjQI7N0AfsMBbPmAFALQ94QjHzvriTZSe790KUCd hM4fERDxyfGweLQpfful3Vq24Oy4YttFwrPRh8KTLNLDUwYhU8hGFwMORXZ5cbWVH+WW z9CrLYW6eOP8EMkFigt3ILsKEXgebbA0QzEAISKpk6i/ZLHlhz7++T9b7WVW36VQDCCY f2Kheb3mgEuNDMxlnt6VXANr+Bi3RCevcTfN87BeJuMy1n1VdZxqPOCZxd593JzwJFmY 9vqQ== X-Gm-Message-State: AGi0PuZR3J2CuN9ohulJkPRuREM6VSeFfQygbDXAn5J1ucZSSSN5j4B2 uUXi7QohxLvO7U4qoMOkQh0VmQ== X-Google-Smtp-Source: APiQypJeg8iV+Is/jjghotuLsws3LW3MbtvTfDLRSrO+mQADIEt6UxaVlUqnEGdv59l9qdnUHdBNhw== X-Received: by 2002:a17:90b:4c84:: with SMTP id my4mr3182095pjb.3.1586270824425; Tue, 07 Apr 2020 07:47:04 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:04 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 3/9] riscv: patch code by fixmap mapping Date: Tue, 7 Apr 2020 22:46:48 +0800 Message-Id: <0b6ad2759b47731a83008b46dbbed7c92e68cae2.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074704_925365_18C5B00C X-CRM114-Status: GOOD ( 10.54 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:1041 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org On strict kernel memory permission, the ftrace have to change the permission of text for dynamic patching the intructions. Use riscv_patch_text_nosync() to patch code instead of probe_kernel_write. Signed-off-by: Zong Li Suggested-by: Masami Hiramatsu Reviewed-by: Masami Hiramatsu --- arch/riscv/kernel/ftrace.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index c40fdcdeb950..08396614d6f4 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -7,9 +7,23 @@ #include #include +#include #include +#include #ifdef CONFIG_DYNAMIC_FTRACE +int ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) +{ + mutex_lock(&text_mutex); + return 0; +} + +int ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) +{ + mutex_unlock(&text_mutex); + return 0; +} + static int ftrace_check_current_call(unsigned long hook_pos, unsigned int *expected) { @@ -46,20 +60,14 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, { unsigned int call[2]; unsigned int nops[2] = {NOP4, NOP4}; - int ret = 0; make_call(hook_pos, target, call); - /* replace the auipc-jalr pair at once */ - ret = probe_kernel_write((void *)hook_pos, enable ? call : nops, - MCOUNT_INSN_SIZE); - /* return must be -EPERM on write error */ - if (ret) + /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */ + if (patch_text_nosync + ((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE)) return -EPERM; - smp_mb(); - flush_icache_range((void *)hook_pos, (void *)hook_pos + MCOUNT_INSN_SIZE); - return 0; } From patchwork Tue Apr 7 14:46:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478247 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9A49174A for ; Tue, 7 Apr 2020 14:47:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8516D2075E for ; Tue, 7 Apr 2020 14:47:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="IlAS7G4v"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="knovKfyo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8516D2075E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bdvS3ImXktegIGIUSt7DM4X3H//vtPG1T5XbtOqEIxE=; b=IlAS7G4vTotxUi ALvpJ9RbpEhNuu4P9hXe6Bx/VP6OynZnFvs1DP6OFIgjuqVOaxA5LAgB2al9Zh44fjIvzYBaYwLLu c//OLt9fF1M7K72VwPMpkfctwNhs6jF3DhSCk22tuXtiSXYB9sFuETipsQPCFp9I04+w30RMTSW/S u0DwSW0yYnIIOeHRqSBsmsySTY7E6RQx3LPAkujkzs6K+8ckfJRxhEanNTmuHSGCdPjXxCJi3Bgeh 8eIiRxQs4IjOeHxBxp6/RwD6wy8AK+1NwRPeY+9GksAryD3uE643XCjWF3GxMmZaJCLyT4JJm485l B6Nt9pHBnMGI+2v/KlHQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVR-0006Mz-4N; Tue, 07 Apr 2020 14:47:13 +0000 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVL-0006Et-VJ for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:09 +0000 Received: by mail-pl1-x643.google.com with SMTP id x1so1335495plm.4 for ; Tue, 07 Apr 2020 07:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bdvS3ImXktegIGIUSt7DM4X3H//vtPG1T5XbtOqEIxE=; b=knovKfyo4UZfNQvLumQh4YiND2YiuKKF1VEMXbmrk4DNL7JyU0zDLLAK12VKAzO69V No6OSJ5iiap2sww6JWv44GLP4xJ+EX6DctkaBBRIt1gJ4AIIO+Zny1L5JP+6vrtoyaGy 5xVmUOuerxLVWASbltXa4kMFmXTwckiw+dW6hmMzOdB/P6s4VgwcW+hJObtRQdoNuLG4 OiIbO+Ae757Uw9/9jPS3G+ZJT1DeYCWo7vXXQl1UaGlMOq5jwdNZuKfgz8VG0LatfZsc BtmTwRWewPKaw5Gj3ak0a9SFf3WY6jm4uRydCBo7ONIV/TeVBlkXGpJrsvxksFx6Mh4y J6Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bdvS3ImXktegIGIUSt7DM4X3H//vtPG1T5XbtOqEIxE=; b=ugZrmzIhPDaiccD0CILAvIO9k7vx1rqqukTWyhKdpWmR7uuUJB0g/qoChorFXgsrVZ YlaXErADNNWDGH7Oh06Xe5WARi7l755ain+PD6vCh6YxIwe8xu8uiBE66Z+dUZ+j2spm RyNBc0z9A53OKsM6y/u8O+t8U8nNXZ0jXAr7aIj3KT2uqyxN0naGaNEQ1d4Qe6I/+9WI zrCBKpYGQ+2qfTARNwSGSo0hU+nMPU+7i5K/RuQjtv8gxaDUIoFcshS4ldSVX3O6Y5Q+ 3dpdB8WofLqVJKQlYMXSGbFhorRyOv6AKZT+r7He4Kpa6D3I5Yj07rjxzxFrIJgq5A4p +97Q== X-Gm-Message-State: AGi0PuZkkqA7jhIieA5aTXkwk+05xHXL8Pt63phQ/h6qChyMB909VKMt 8zanPJ/0BCef409yV3oeG12d7Vf3mK4= X-Google-Smtp-Source: APiQypIjFwaw5F3CH133qsTVtPyL2XcdUIhQdqvu9Da4WywKjh+qJ7nnuc1kM1SSwgc9X3T8sZQNXw== X-Received: by 2002:a17:90a:e387:: with SMTP id b7mr3168370pjz.93.1586270826426; Tue, 07 Apr 2020 07:47:06 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:06 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 4/9] riscv: add ARCH_HAS_SET_MEMORY support Date: Tue, 7 Apr 2020 22:46:49 +0800 Message-Id: <783cadc46560fe173ac4900f36375b511185887e.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074708_046333_1BD728CB X-CRM114-Status: GOOD ( 13.35 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:643 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Add set_memory_ro/rw/x/nx architecture hooks to change the page attribution. Use own set_memory.h rather than generic set_memory.h (i.e. include/asm-generic/set_memory.h), because we want to add other function prototypes here. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/set_memory.h | 24 +++++ arch/riscv/mm/Makefile | 2 +- arch/riscv/mm/pageattr.c | 150 ++++++++++++++++++++++++++++ 4 files changed, 176 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/set_memory.h create mode 100644 arch/riscv/mm/pageattr.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 39c93ddff0cb..9044e0dd95ea 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -59,6 +59,7 @@ config RISCV select HAVE_EBPF_JIT if 64BIT select EDAC_SUPPORT select ARCH_HAS_GIGANTIC_PAGE + select ARCH_HAS_SET_MEMORY select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select SPARSEMEM_STATIC if 32BIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h new file mode 100644 index 000000000000..79a810f0f38b --- /dev/null +++ b/arch/riscv/include/asm/set_memory.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2019 SiFive + */ + +#ifndef _ASM_RISCV_SET_MEMORY_H +#define _ASM_RISCV_SET_MEMORY_H + +/* + * Functions to change memory attributes. + */ +#ifdef CONFIG_MMU +int set_memory_ro(unsigned long addr, int numpages); +int set_memory_rw(unsigned long addr, int numpages); +int set_memory_x(unsigned long addr, int numpages); +int set_memory_nx(unsigned long addr, int numpages); +#else +static inline int set_memory_ro(unsigned long addr, int numpages) { return 0; } +static inline int set_memory_rw(unsigned long addr, int numpages) { return 0; } +static inline int set_memory_x(unsigned long addr, int numpages) { return 0; } +static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } +#endif + +#endif /* _ASM_RISCV_SET_MEMORY_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 814e16a8d68a..363ef01c30b1 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -7,7 +7,7 @@ endif obj-y += init.o obj-y += extable.o -obj-$(CONFIG_MMU) += fault.o +obj-$(CONFIG_MMU) += fault.o pageattr.o obj-y += cacheflush.o obj-y += context.o diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c new file mode 100644 index 000000000000..fcd59ef2835b --- /dev/null +++ b/arch/riscv/mm/pageattr.c @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2019 SiFive + */ + +#include +#include +#include +#include + +struct pageattr_masks { + pgprot_t set_mask; + pgprot_t clear_mask; +}; + +static unsigned long set_pageattr_masks(unsigned long val, struct mm_walk *walk) +{ + struct pageattr_masks *masks = walk->private; + unsigned long new_val = val; + + new_val &= ~(pgprot_val(masks->clear_mask)); + new_val |= (pgprot_val(masks->set_mask)); + + return new_val; +} + +static int pageattr_pgd_entry(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pgd_t val = READ_ONCE(*pgd); + + if (pgd_leaf(val)) { + val = __pgd(set_pageattr_masks(pgd_val(val), walk)); + set_pgd(pgd, val); + } + + return 0; +} + +static int pageattr_p4d_entry(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + p4d_t val = READ_ONCE(*p4d); + + if (p4d_leaf(val)) { + val = __p4d(set_pageattr_masks(p4d_val(val), walk)); + set_p4d(p4d, val); + } + + return 0; +} + +static int pageattr_pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pud_t val = READ_ONCE(*pud); + + if (pud_leaf(val)) { + val = __pud(set_pageattr_masks(pud_val(val), walk)); + set_pud(pud, val); + } + + return 0; +} + +static int pageattr_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pmd_t val = READ_ONCE(*pmd); + + if (pmd_leaf(val)) { + val = __pmd(set_pageattr_masks(pmd_val(val), walk)); + set_pmd(pmd, val); + } + + return 0; +} + +static int pageattr_pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pte_t val = READ_ONCE(*pte); + + val = __pte(set_pageattr_masks(pte_val(val), walk)); + set_pte(pte, val); + + return 0; +} + +static int pageattr_pte_hole(unsigned long addr, unsigned long next, + int depth, struct mm_walk *walk) +{ + /* Nothing to do here */ + return 0; +} + +const static struct mm_walk_ops pageattr_ops = { + .pgd_entry = pageattr_pgd_entry, + .p4d_entry = pageattr_p4d_entry, + .pud_entry = pageattr_pud_entry, + .pmd_entry = pageattr_pmd_entry, + .pte_entry = pageattr_pte_entry, + .pte_hole = pageattr_pte_hole, +}; + +static int __set_memory(unsigned long addr, int numpages, pgprot_t set_mask, + pgprot_t clear_mask) +{ + int ret; + unsigned long start = addr; + unsigned long end = start + PAGE_SIZE * numpages; + struct pageattr_masks masks = { + .set_mask = set_mask, + .clear_mask = clear_mask + }; + + if (!numpages) + return 0; + + down_read(&init_mm.mmap_sem); + ret = walk_page_range_novma(&init_mm, start, end, &pageattr_ops, NULL, + &masks); + up_read(&init_mm.mmap_sem); + + flush_tlb_kernel_range(start, end); + + return ret; +} + +int set_memory_ro(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(_PAGE_READ), + __pgprot(_PAGE_WRITE)); +} + +int set_memory_rw(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(_PAGE_READ | _PAGE_WRITE), + __pgprot(0)); +} + +int set_memory_x(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(_PAGE_EXEC), __pgprot(0)); +} + +int set_memory_nx(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(0), __pgprot(_PAGE_EXEC)); +} From patchwork Tue Apr 7 14:46:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478249 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDDA7174A for ; Tue, 7 Apr 2020 14:47:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB601208FE for ; Tue, 7 Apr 2020 14:47:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="UxXehSUm"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="Z95WsXja" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB601208FE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1qkEQ73ghM61/BXRvlDJEs7SEg6hkPYIpkfLFdxJqkM=; b=UxXehSUmVa/dXd AFrY8HfGZeL/1eQQu5e3kClvWVvL1hXzJyA26PJiad70ovU1LAeoETctgyII9Lyx6BWl3+Kw1IPhi +MEAjk3FDJ10LrsZmK49XB85ghpXkQMr8/go5ou0zaMrUZrvSMYeo7anolMmI4jfzggyhMPBLG0If U/XYrSnspJvuOlEsrEShY/8fxFpsHdiNRLb9ZVBU8L+MFXrofuAN9Eg2u7mBI0M+66shXy9SdIkX6 0LEOHlKqiYqeXDWhFm2oQn9oljdjPWCLOOAGSqZ2jiKAgcg4VVp9PFaA+j9NPoRlFCM1SQJibdrZy v7y9O4FDJHSCpBQLJgJg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVT-0006Qs-Uz; Tue, 07 Apr 2020 14:47:15 +0000 Received: from mail-pj1-x1043.google.com ([2607:f8b0:4864:20::1043]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVO-0006HT-1n for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:11 +0000 Received: by mail-pj1-x1043.google.com with SMTP id kx8so852648pjb.5 for ; Tue, 07 Apr 2020 07:47:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1qkEQ73ghM61/BXRvlDJEs7SEg6hkPYIpkfLFdxJqkM=; b=Z95WsXjaZ4R1OiZAjtFWozqj9735nUzepdNDu+Rm+ykKmZ1QhMW1WXzh4mh+TxtIUE 3up1bhUw0eoHYPS2z8u6Mz4s5j4A2KaKT+b3o5aWzvtjB2dNihRQ+qXe8mFuGaUjOb7e PHR/jKA1Ybc0M0HAbqxdcQ4Id/syAJDH5PBKE8wm9I8phudmq4vsXT6ifv3WXy0nXAIB AH2lCtDzZ8wqQyWQ5t3ZAaLK/NdSJoIrbi78EcorwBkK9t45hMs/R6/CQqwUJ3hI8RZL W9QEv1DPVnBiArLheGHAnGn1vDRriUf+GqOdmk4bKQW2fK7NCvaaEyPjormnBQMmeELy Pchg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1qkEQ73ghM61/BXRvlDJEs7SEg6hkPYIpkfLFdxJqkM=; b=CEyl4aT5uzxoffqIuuuDoOSJfvPW8e3voZH+l2X4dmLu6WFJewxnOvqxnNRHz4fCAF TOsD8bRhTC/Mo2UKcaxLOvQN9GazO9GpaWJ0zKNeGEIYAMBpWmj6KJM86oAK42WLxwvI WMwnphPG5IVbkIMXdo8JYeTDaDVq0zCMiWpjRs5g1xQDt/lh2Mf24tInX1iVb/IXePwH 9okk7cAZZkTuqEfocvgvIYek7VomBu9oEKBCXvDPRZzD0V/ClXtgQcahQ8EWOoSfj6aR T+HZZ6UfttqQvx66zZW2FHF/NRLckeH7GeD3BG0ShguWIgQNxN40KEBcR/gQwzQIBZUw BsWA== X-Gm-Message-State: AGi0Pua0LoOhK3kxbDUoC9bEiXS7N7Vyj7nH8B1vaxiIyK/5Fo2gaibH y0sGLZN7132Z12IZjgmFnHul3A== X-Google-Smtp-Source: APiQypL0QZL2CpYGkPcslhSMRE9aqJtSat2xKN9cJ0wG3hkbm7awrwBq0Y0CDoR4ztQTWLcG3pAU6A== X-Received: by 2002:a17:902:8609:: with SMTP id f9mr2675736plo.203.1586270828988; Tue, 07 Apr 2020 07:47:08 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:08 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 5/9] riscv: add ARCH_HAS_SET_DIRECT_MAP support Date: Tue, 7 Apr 2020 22:46:50 +0800 Message-Id: <1b9ab6009c1d70451e21a35ed7fea47c139bc65f.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074710_127082_C92E9EC5 X-CRM114-Status: UNSURE ( 9.55 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:1043 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Add set_direct_map_*() functions for setting the direct map alias for the page to its default permissions and to an invalid state that cannot be cached in a TLB. (See d253ca0c ("x86/mm/cpa: Add set_direct_map_*() functions")) Add a similar implementation for RISC-V. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/set_memory.h | 3 +++ arch/riscv/mm/pageattr.c | 24 ++++++++++++++++++++++++ 3 files changed, 28 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 9044e0dd95ea..a94d0f064d9c 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -59,6 +59,7 @@ config RISCV select HAVE_EBPF_JIT if 64BIT select EDAC_SUPPORT select ARCH_HAS_GIGANTIC_PAGE + select ARCH_HAS_SET_DIRECT_MAP select ARCH_HAS_SET_MEMORY select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select SPARSEMEM_STATIC if 32BIT diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h index 79a810f0f38b..620d81c372d9 100644 --- a/arch/riscv/include/asm/set_memory.h +++ b/arch/riscv/include/asm/set_memory.h @@ -21,4 +21,7 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; } static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } #endif +int set_direct_map_invalid_noflush(struct page *page); +int set_direct_map_default_noflush(struct page *page); + #endif /* _ASM_RISCV_SET_MEMORY_H */ diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index fcd59ef2835b..7be6cd67e2ef 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -148,3 +148,27 @@ int set_memory_nx(unsigned long addr, int numpages) { return __set_memory(addr, numpages, __pgprot(0), __pgprot(_PAGE_EXEC)); } + +int set_direct_map_invalid_noflush(struct page *page) +{ + unsigned long start = (unsigned long)page_address(page); + unsigned long end = start + PAGE_SIZE; + struct pageattr_masks masks = { + .set_mask = __pgprot(0), + .clear_mask = __pgprot(_PAGE_PRESENT) + }; + + return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks); +} + +int set_direct_map_default_noflush(struct page *page) +{ + unsigned long start = (unsigned long)page_address(page); + unsigned long end = start + PAGE_SIZE; + struct pageattr_masks masks = { + .set_mask = PAGE_KERNEL, + .clear_mask = __pgprot(0) + }; + + return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks); +} From patchwork Tue Apr 7 14:46:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478251 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E50EA81 for ; Tue, 7 Apr 2020 14:47:23 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BEF5520BED for ; Tue, 7 Apr 2020 14:47:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="rf8Q1zMM"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="Dr3sAdrv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BEF5520BED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=v9zl9wdVM8GEYtYil3Mhwjs87sA3d02fBbOSD/V/2Io=; b=rf8Q1zMMGwRAz1 owCKoE2Zo7fzphuRr4uQx46Xj80jQepavAZi23dFR6keQ/cUI8qw9UkXSPzf9K64pJF7T1z2QTYYQ n4fVU0yC4kfdco9dpyFkvCT/gAb6etjkNfao3cnOoRhqjP+1Q2vfFfHivvS9DI2GZnyxTUI2EF6Ar tZkFpYCItNkW3bqquK/FuGbmidtijLSi2bIKacPkxbQpYCRIa/ifx7JhU7nQC/jJfyW9/cfFdpBLp OLjnw+ebJSHsVcKsZt5Jbu7Ki8SvrZZbjn7aWpTIZuj4tKBQI/cvYv1Gz3gL7zslNwwV9yZkOhqch loCF9Jf3S4j+9+FSBstw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVX-0006W0-T2; Tue, 07 Apr 2020 14:47:19 +0000 Received: from mail-pj1-x1042.google.com ([2607:f8b0:4864:20::1042]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVQ-0006Kh-Cl for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:13 +0000 Received: by mail-pj1-x1042.google.com with SMTP id z3so852927pjr.4 for ; Tue, 07 Apr 2020 07:47:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v9zl9wdVM8GEYtYil3Mhwjs87sA3d02fBbOSD/V/2Io=; b=Dr3sAdrvWqwbVgYRG3xI41G3QomvAew/Asd8fOt+Hzih8GAcGLY990zl5lzEDEUFfS rFz2r8PiTez4vQkSHyG6i66D37p45zDTF7mKWyqHS2KRTSW4HsCHenibJ08i0FlY9LCA RJATz1HEkSwg5S/VSgSpr6RXXPaavBo70UMlSGPI82JH1ercjcbnGTpTe132ZKGMZHAw nZksYVqpBq42d5GPn0kG3A1mMxp6m9j2bb+5m8WvKb6oXTdpyGm/tEZMU0avt92C3PwW WXhNzRtLZimrbB/nPTEE/DL7MZON4cRniYmXRoW32mBWz7TA80gs7IJIuu/Jwx1bBX1v bT4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v9zl9wdVM8GEYtYil3Mhwjs87sA3d02fBbOSD/V/2Io=; b=R3xUwMgAKr9r2ErLphaVVQpucNZ2yIOYpHtxhGODcWbsBefg5fqgH4KGT3GW37k1zS LrqhLWm3KJvXZmupsZm9CrYlS5iXOCKYSpwBYuxjqm2FuJFRSvd+y3+JnBdLU1DPeN4o XM/8OpN6hiEO9W0kvfAoe8Ij32bq97Dp1CdtfhsYYPThOxx/qpzRnZP+aT75XdpQSppx 3CgIiQDFu+4CnC0RQRS18GQch+V8VDyJS/IwTCua3UJZQt1QtYMLSDwMG6iVcGCyBzd4 dhL+CiQkprtltux2cXYJCWE6XTjInvotY94KM1bFn8BIP6Eo6H9CQHuJJcsviMLBiy9w Tjbw== X-Gm-Message-State: AGi0PuYx0WqxDmHHiu7GsRpvSWc/NsxHZ1kj9IUHu4pNy96LtDaBjHol Jt76lbnVazLB5knfNFvHJKc8+GGfh/g= X-Google-Smtp-Source: APiQypIxfe67P6X0LmkT8qgkBuwk7TGkhTTBndI07oSrQhzlvJyC2RaWNfS88nZVZw4Jjgn0w6QRAw== X-Received: by 2002:a17:902:aa94:: with SMTP id d20mr2788910plr.313.1586270831007; Tue, 07 Apr 2020 07:47:11 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:10 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 6/9] riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support Date: Tue, 7 Apr 2020 22:46:51 +0800 Message-Id: <9a9f8ff6d9ea84628511def8bacce8f8247945d1.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074712_431487_1DBC3922 X-CRM114-Status: UNSURE ( 7.83 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:1042 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org ARCH_SUPPORTS_DEBUG_PAGEALLOC provides a hook to map and unmap pages for debugging purposes. Implement the __kernel_map_pages functions to fill the poison pattern. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/Kconfig | 3 +++ arch/riscv/mm/pageattr.c | 13 +++++++++++++ 2 files changed, 16 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index a94d0f064d9c..1e1efc998baf 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -132,6 +132,9 @@ config ARCH_SELECT_MEMORY_MODEL config ARCH_WANT_GENERAL_HUGETLB def_bool y +config ARCH_SUPPORTS_DEBUG_PAGEALLOC + def_bool y + config SYS_SUPPORTS_HUGETLBFS def_bool y diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index 7be6cd67e2ef..728759eb530a 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -172,3 +172,16 @@ int set_direct_map_default_noflush(struct page *page) return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks); } + +void __kernel_map_pages(struct page *page, int numpages, int enable) +{ + if (!debug_pagealloc_enabled()) + return; + + if (enable) + __set_memory((unsigned long)page_address(page), numpages, + __pgprot(_PAGE_PRESENT), __pgprot(0)); + else + __set_memory((unsigned long)page_address(page), numpages, + __pgprot(0), __pgprot(_PAGE_PRESENT)); +} From patchwork Tue Apr 7 14:46:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478255 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BA25B81 for ; Tue, 7 Apr 2020 14:47:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C56820CC7 for ; Tue, 7 Apr 2020 14:47:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="RvM9wB1G"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="AHDDzmDi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C56820CC7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Quhy5/QZPA8/P4q/9fnjufmjHKuWjkZaQYF9/wabbVw=; b=RvM9wB1Gk70tGd yvzbZ5OcPIcCauabrp+ey32f1Wl1alwE2glFM5R9SI2D1oRJp1S0z25t5+5uuMrl+xjjZaQwq0gGw k0mrfFk5ONUdb3r9sS1+OI9lL62FpqCVRBXCbuj+88bVK1WVK4LkTaqWOhyDo9ccz/kVlCTMwf/lR eW3iq1S03enDP0cSwuS54DGAcCrWdzmozxQNia+gL4tpk4ER0BWtFGP1WN868S+d/n+p+LP0Oyg0A TO9c6MuXyZvEUvJCgBYsVWQ1ygBTQg9EdSOZuFzZ/fHz0C6jDpmkICkkytyh9asqsVAu3YERIu42i PyHOtLEJC2TFOKxYG48Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVk-0006nk-29; Tue, 07 Apr 2020 14:47:32 +0000 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVR-0006NU-QJ for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:15 +0000 Received: by mail-pg1-x544.google.com with SMTP id k191so1811334pgc.13 for ; Tue, 07 Apr 2020 07:47:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Quhy5/QZPA8/P4q/9fnjufmjHKuWjkZaQYF9/wabbVw=; b=AHDDzmDiFHiECrdMx/1zHadC9VdetNvA12RpoCsdBdCEK14bBlGUUaj0dd2H3A+BCH L6TKBWsntI3ec6isKAfpwrdF8gp0hEaVoFmoYPb+coyfeDRwebMhUEHgh1ipHAiVF+Gt 10VKtKonkggb5aWtTNpPY+j9eZcgqAFpLDjyDKnYkMUot/ajync8iCZ+8UgpSFPY7xSL gjON/Bei/AQgPSnbtG4wiHaMXKv+oyO1uxc/6HJCHDrs8EkA0debASne5ARpkVJ0R+FG OlfKbwiqhzHZt9pBeOyExXRS/dhnZB4yK/WSaSA6cpLorZk7MXsQvxmaE7X3t1hFGzjX 5ORw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Quhy5/QZPA8/P4q/9fnjufmjHKuWjkZaQYF9/wabbVw=; b=QMTgdnCr2E0JpB0etdNVeLrKkoaAv+z1v+PVFEniTOUM77z/ZrdfKvAMnXsofGKEqM Ps8Yw9gvb45Ih8IqPpSXtH46c8AAxW/XnoAdJ9PWDxbOWxg8kk7f3Rmkxy05rA5bHhnm EYkuOl1oMn6TIvDWKvqCCaAKnaVdCTbLDj1Fg1nFhld7iok/tdo7Qvct3H8PnscjDp9F rKzpW9G+zfgXkFK6z8nLpMtensotm8UcgDduhdg+eoOqqwu8sirfEbaX9DxB/4JAhd2Y FKvYBIaFWVWLvF4KKyeqZtkULmtmsaNcEWxJ/iwLmKUKvpNgKvMJrVBFoUoYTz6juhNq 6pKg== X-Gm-Message-State: AGi0PuZvENgVOnkzHMa8qb8fC1i1BusKPFdVUwkgNgnfY6CdVF/UYMwV LvWp/1PLc4ibSYYr5PPJrHG3fw== X-Google-Smtp-Source: APiQypIu7xZA7V1YhEkwC7XML1SD8uFsocdPfcN93OJKoMqGsb76XglFSxzjrLHKYnXDd+XsfzKmUg== X-Received: by 2002:a62:154:: with SMTP id 81mr2864787pfb.228.1586270833015; Tue, 07 Apr 2020 07:47:13 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:12 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 7/9] riscv: move exception table immediately after RO_DATA Date: Tue, 7 Apr 2020 22:46:52 +0800 Message-Id: <1d1d278ca3957db9103f40ddda008128365d6d72.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074713_862921_3CC360B7 X-CRM114-Status: UNSURE ( 9.52 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:544 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Move EXCEPTION_TABLE immediately after RO_DATA. Make it easy to set the attribution of the sections which should be read-only at a time. Add _data to specify the start of data section with write permission. This patch is prepared for STRICT_KERNEL_RWX support. Signed-off-by: Zong Li Suggested-by: Palmer Dabbelt --- arch/riscv/kernel/vmlinux.lds.S | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S index 1e0193ded420..02e948b620af 100644 --- a/arch/riscv/kernel/vmlinux.lds.S +++ b/arch/riscv/kernel/vmlinux.lds.S @@ -58,6 +58,10 @@ SECTIONS *(.srodata*) } + EXCEPTION_TABLE(0x10) + + _data = .; + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE) .sdata : { __global_pointer$ = . + 0x800; @@ -69,8 +73,6 @@ SECTIONS BSS_SECTION(PAGE_SIZE, PAGE_SIZE, 0) - EXCEPTION_TABLE(0x10) - .rel.dyn : { *(.rel.dyn*) } From patchwork Tue Apr 7 14:46:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478257 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F5CD81 for ; Tue, 7 Apr 2020 14:47:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E6F862082F for ; Tue, 7 Apr 2020 14:47:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="W9T/Egq4"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="JBxizfUV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6F862082F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=geDMvpdyp4eREUjGenxPiMAgszPzsf8hdgTrRUcyDVI=; b=W9T/Egq4kYJZh9 diSsjHdwzOQzSYrnWYgGGfBEkIjBiQVzkkQxEYymrggo4YDKUdmqVrmfmanxOybp2i7h09auTkQ1y ya+Yv6rIfQaY0tmLv8PgQ3Wh8+ccIGjJrVsUPqx8QGsd2mKO+Y7J/vchYh2EftForqDndeOC/M0+8 jdq6NJoHD6DbsbQRf8O8Sx96mBZQktzC3+dEpcvF4dTmeZzYsWRpdxjUzTso440rbbS7NMW/ONoVt UD9e+IirME/XeL9vxK//uQGKrRhl+KJcv90Dx7rmOyjfnAmXv9yYtaEroOwOWhID76jwH09aNcBFL rmYeftUJsl+oB6UZIq4g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVp-0006uO-6y; Tue, 07 Apr 2020 14:47:37 +0000 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVT-0006Qh-S1 for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:17 +0000 Received: by mail-pg1-x544.google.com with SMTP id r4so1835678pgg.4 for ; Tue, 07 Apr 2020 07:47:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=geDMvpdyp4eREUjGenxPiMAgszPzsf8hdgTrRUcyDVI=; b=JBxizfUVQ4VwCje90RdukKZBh/ywXEy+VAtWwgau09WHsbrygimeUgLaDGNihbFdPZ eDybrac2/Ma3vtwVtZ+dV10ejldg12V79mPPn+6PEcb+pYiamkJfHvdVCc3pQywEgffS 61nByA/bmGC+KvIQ961VXAVS+jeOKfdvsn2YvlzRhS4+b9g+s7hBJOhV9Zurmg6tfc88 hRFeb/pdtnq4jQdvIW1maqRVT/GoZMh6q3OCXYlUAxtVz2Sa7KW7AhM451d55reG1kbv bdvnAWUvHjr7PiJ0CRL0hEJnBVdf4VVNAcT2NTV9xOdaANofRCoMNuG4j0Ia+W6wAhRU xUww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=geDMvpdyp4eREUjGenxPiMAgszPzsf8hdgTrRUcyDVI=; b=P6+aA9ikySiuI0+rjfQmKUvmCC0dhMoyTrChmDY6b/BGLPcHNcLqKTVLN9TqR1xB/+ QTMiU44ZU2Oa/ivYJyBKHEH6Pef0sy2idCVGIQDoxkMoSCuhDIdTAslkWGT7Eqwu7t8f 9ljWubJaOzidHD1goJs2D0rt22A8gNZ+OXJOfpJMI0d6L1/FkdywO3lPDU1wtXTEz7cd 84oum21Of3jiAiu8EtkT6i4pQ37TncELLu9DJoNmobH7R3CBhXqx3xdrZr+F14fPzmio z1FSw7nHD6DKBp/9ib5d0bcwc9ZrNrdDhy8/oG3N7DLzSgY+Qh4sEIF7wIjpHfTKGD6a P1ig== X-Gm-Message-State: AGi0PuYjy7iaGLYcvPuIq3WqCtmt7RcmScmED24D2LpW6XkOhkENW3cE 6TnyIE7SJxjQbB0k9m9z3vPbVA== X-Google-Smtp-Source: APiQypJSC+TFUHwlK7PTtzsWm3RbNjRWth5EDoVm2mJoigQnMWAkG6PwXQSKKLPQx1AAHMmzFA/bEg== X-Received: by 2002:aa7:9a08:: with SMTP id w8mr2724871pfj.317.1586270835251; Tue, 07 Apr 2020 07:47:15 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:14 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 8/9] riscv: add alignment for text, rodata and data sections Date: Tue, 7 Apr 2020 22:46:53 +0800 Message-Id: <24d705447a3033ee94c3dc9687b97870e2521090.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074715_916028_8AA88D56 X-CRM114-Status: GOOD ( 11.91 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:544 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org The kernel mapping will tried to optimize its mapping by using bigger size. In rv64, it tries to use PMD_SIZE, and tryies to use PGDIR_SIZE in rv32. To ensure that the start address of these sections could fit the mapping entry size, make them align to the biggest alignment. Define a macro SECTION_ALIGN because the HPAGE_SIZE or PMD_SIZE, etc., are invisible in linker script. This patch is prepared for STRICT_KERNEL_RWX support. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/set_memory.h | 13 +++++++++++++ arch/riscv/kernel/vmlinux.lds.S | 5 ++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h index 620d81c372d9..4c5bae7ca01c 100644 --- a/arch/riscv/include/asm/set_memory.h +++ b/arch/riscv/include/asm/set_memory.h @@ -6,6 +6,7 @@ #ifndef _ASM_RISCV_SET_MEMORY_H #define _ASM_RISCV_SET_MEMORY_H +#ifndef __ASSEMBLY__ /* * Functions to change memory attributes. */ @@ -24,4 +25,16 @@ static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } int set_direct_map_invalid_noflush(struct page *page); int set_direct_map_default_noflush(struct page *page); +#endif /* __ASSEMBLY__ */ + +#ifdef CONFIG_ARCH_HAS_STRICT_KERNEL_RWX +#ifdef CONFIG_64BIT +#define SECTION_ALIGN (1 << 21) +#else +#define SECTION_ALIGN (1 << 22) +#endif +#else /* !CONFIG_ARCH_HAS_STRICT_KERNEL_RWX */ +#define SECTION_ALIGN L1_CACHE_BYTES +#endif /* CONFIG_ARCH_HAS_STRICT_KERNEL_RWX */ + #endif /* _ASM_RISCV_SET_MEMORY_H */ diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S index 02e948b620af..ef87deea0350 100644 --- a/arch/riscv/kernel/vmlinux.lds.S +++ b/arch/riscv/kernel/vmlinux.lds.S @@ -9,6 +9,7 @@ #include #include #include +#include OUTPUT_ARCH(riscv) ENTRY(_start) @@ -36,6 +37,7 @@ SECTIONS PERCPU_SECTION(L1_CACHE_BYTES) __init_end = .; + . = ALIGN(SECTION_ALIGN); .text : { _text = .; _stext = .; @@ -53,13 +55,14 @@ SECTIONS /* Start of data section */ _sdata = .; - RO_DATA(L1_CACHE_BYTES) + RO_DATA(SECTION_ALIGN) .srodata : { *(.srodata*) } EXCEPTION_TABLE(0x10) + . = ALIGN(SECTION_ALIGN); _data = .; RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE) From patchwork Tue Apr 7 14:46:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11478259 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E72D4174A for ; Tue, 7 Apr 2020 14:47:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C4F83208FE for ; Tue, 7 Apr 2020 14:47:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="sj/U3q62"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="TtvzXxSJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4F83208FE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=E4WIiXeGbKDxs9UbyyfZOjVgyCJTWA/AduHG+mw3Ozg=; b=sj/U3q62ktZ5Zn tk1NuMjGncoaTw3ckgchyZRQ/z5DLYxa3Uc7Doc8ogodfNCPc+bMx6aAhSQxRUrFexhYMtGKaRWbf 9z7w9IKml+H8IN0niQNw1EGLVnxQDAvw81tP+x1lpdUVODX/wl/JubD1n/JgJ/IIhUsigUdd2HbTU s2Um0z800hUPh33WBEwiGBJFIcYy6Xt0JE1iGY/IaLV5DoNGf3wd2RuyMLK0BZAYGBnGsXfKfLseh uWWOW0iGASqwUDdhWoVWsuVPK6RGaT0Af5G9ATsURUtH1Ra/oArC+Wa0zRCLmXCRTbL/cOR3uic6P EcEh26y6IN05mZaCSCxA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVu-000709-DT; Tue, 07 Apr 2020 14:47:42 +0000 Received: from mail-pj1-x1044.google.com ([2607:f8b0:4864:20::1044]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jLpVW-0006TJ-Lr for linux-riscv@lists.infradead.org; Tue, 07 Apr 2020 14:47:20 +0000 Received: by mail-pj1-x1044.google.com with SMTP id nu11so858575pjb.1 for ; Tue, 07 Apr 2020 07:47:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E4WIiXeGbKDxs9UbyyfZOjVgyCJTWA/AduHG+mw3Ozg=; b=TtvzXxSJ0vpaB0ZG++G2fkPig6AtFEsttCjqY2Xvv9MaF0/H7Z/pUc+9b4ouCtetSj 4120O3V+N+4RGONMM6wYETKEPPwBodh7xtC8Aeat9tSSrhoRmifL+6stENNVtQwsGgn4 4qdDIhygGHzmwFk2y8DpRVDHh5WpmIDnqHRLSWxY83gDpKpJjWbZh7S7P9kq1Ks1kXxI b3Qe2gvh2ejACDVV9xe+XJV40SBNBlygiUBuFR/PU+9KbIRTWi5fg15zt3eVXvMTByU+ xzW7/AEiuPvSp7AYkSo4O/lwLVRR/E/a40w/7WqVXe6vnpHdE8iaICEgw4NRtXQ3zqjW W6Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E4WIiXeGbKDxs9UbyyfZOjVgyCJTWA/AduHG+mw3Ozg=; b=ePyE7nQTEDHiFUQ47bV/d3ywFie2QbMDgeStfQsktC8L2Lhz0dnQo0I4fDLezgPo8x oNcfhp6jLsXqlfQ8R9FPPc+lS/fUphe+/Icl35c76tQb9KrLRjYWOcHLmI5VPXJHPCzE 4wQSJzNdhOZ7cq3ynGEG1jpvquna48qyd85VMwhWi8SKZZ5a7IutafvCCJYo+cTErXHA GEaIQ5MIFgChwq41P/vxRXK6xb+nDJ+zlkfk5blLO9Z5OoUc0FhfAqp/cSd8S5Acu9f0 I+L5mH/OgiQHopPiLrQ84dvQPkP3UgiszqFbKAvoX6r4wwcCeJdJW6NBexg4YdsCEpcp 1Z1w== X-Gm-Message-State: AGi0Pub/KPc0DOIEhvn1Ycpwps4qup2wQ1sH/8h5NM0K/9H60Zczm/Rk kza9UYMN5USMBFRFyEGugUFRSQ== X-Google-Smtp-Source: APiQypJUKWF94J6ayuHjup8iitIs5Pg3Mufg8IUFflYhlm4L0XY+598UDR5xW3cZS1qS1UmEjpbxIw== X-Received: by 2002:a17:90a:c207:: with SMTP id e7mr3190940pjt.117.1586270837136; Tue, 07 Apr 2020 07:47:17 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (220-132-236-182.HINET-IP.hinet.net. [220.132.236.182]) by smtp.gmail.com with ESMTPSA id y15sm14190093pfc.206.2020.04.07.07.47.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Apr 2020 07:47:16 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 9/9] riscv: add STRICT_KERNEL_RWX support Date: Tue, 7 Apr 2020 22:46:54 +0800 Message-Id: <933a02ac01c1ac2a5c174474d46da12e8db96d63.1586265122.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200407_074718_761570_33B285CF X-CRM114-Status: GOOD ( 10.53 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:1044 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org The commit contains that make text section as non-writable, rodata section as read-only, and data section as non-executable. The init section should be changed to non-executable. Signed-off-by: Zong Li --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/set_memory.h | 8 ++++++ arch/riscv/mm/init.c | 44 +++++++++++++++++++++++++++++ 3 files changed, 53 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 1e1efc998baf..58b556167d59 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -61,6 +61,7 @@ config RISCV select ARCH_HAS_GIGANTIC_PAGE select ARCH_HAS_SET_DIRECT_MAP select ARCH_HAS_SET_MEMORY + select ARCH_HAS_STRICT_KERNEL_RWX select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select SPARSEMEM_STATIC if 32BIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h index 4c5bae7ca01c..c38df4771c09 100644 --- a/arch/riscv/include/asm/set_memory.h +++ b/arch/riscv/include/asm/set_memory.h @@ -22,6 +22,14 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; } static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } #endif +#ifdef CONFIG_STRICT_KERNEL_RWX +void set_kernel_text_ro(void); +void set_kernel_text_rw(void); +#else +static inline void set_kernel_text_ro(void) { } +static inline void set_kernel_text_rw(void) { } +#endif + int set_direct_map_invalid_noflush(struct page *page); int set_direct_map_default_noflush(struct page *page); diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index fab855963c73..b55be44ff9bd 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -477,6 +478,17 @@ static void __init setup_vm_final(void) csr_write(CSR_SATP, PFN_DOWN(__pa_symbol(swapper_pg_dir)) | SATP_MODE); local_flush_tlb_all(); } + +void free_initmem(void) +{ + unsigned long init_begin = (unsigned long)__init_begin; + unsigned long init_end = (unsigned long)__init_end; + + /* Make the region as non-execuatble. */ + set_memory_nx(init_begin, (init_end - init_begin) >> PAGE_SHIFT); + free_initmem_default(POISON_FREE_INITMEM); +} + #else asmlinkage void __init setup_vm(uintptr_t dtb_pa) { @@ -488,6 +500,38 @@ static inline void setup_vm_final(void) } #endif /* CONFIG_MMU */ +#ifdef CONFIG_STRICT_KERNEL_RWX +void set_kernel_text_rw(void) +{ + unsigned long text_start = (unsigned long)_text; + unsigned long text_end = (unsigned long)_etext; + + set_memory_rw(text_start, (text_end - text_start) >> PAGE_SHIFT); +} + +void set_kernel_text_ro(void) +{ + unsigned long text_start = (unsigned long)_text; + unsigned long text_end = (unsigned long)_etext; + + set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT); +} + +void mark_rodata_ro(void) +{ + unsigned long text_start = (unsigned long)_text; + unsigned long text_end = (unsigned long)_etext; + unsigned long rodata_start = (unsigned long)__start_rodata; + unsigned long data_start = (unsigned long)_data; + unsigned long max_low = (unsigned long)(__va(PFN_PHYS(max_low_pfn))); + + set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT); + set_memory_ro(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT); + set_memory_nx(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT); + set_memory_nx(data_start, (max_low - data_start) >> PAGE_SHIFT); +} +#endif + void __init paging_init(void) { setup_vm_final();