From patchwork Wed Apr 8 07:56:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479541 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B16481 for ; Wed, 8 Apr 2020 07:57:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 187FE20747 for ; Wed, 8 Apr 2020 07:57:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="hguDn0jT"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="Kn66xOjs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 187FE20747 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tOQsL1UwAyeSD7aVc86VJnzoqXGbBrzuC9r/s6opFAQ=; b=hguDn0jTLi72lT Ss52iNDE4aQkhhRBJSjyJm4VcKiKsVfWuEdrFAjokSBYbPZwuywUN8ie5WPwL5NUMSayZXYXKd0bq wduvya4k9jGQ6XqeQyvLk4G8lRWaEWf8bsJ1wvwLjejKb31geIGBy3RUBp6Rg8yNs4Yith0elhudC s3eV1pwsG+Hq1c3YfOog2ZtLW16a2KD74HhZJkuqMsS/NHz601YnzSHZi1IrLhiW7rmGr4Kr8dlg9 5+eH98h69gKerREnVlMKSVLBRoGfcepH2YAdH3qB6MwST5OY5IaTaWv7uN85lK66BeQfiVIxtXA++ /E3Tf4KS0VP1Ejj+NxfQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aE-0007P5-Gk; Wed, 08 Apr 2020 07:57:14 +0000 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aB-0007Nj-U5 for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:13 +0000 Received: by mail-pg1-x544.google.com with SMTP id s4so2961502pgk.10 for ; Wed, 08 Apr 2020 00:57:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tOQsL1UwAyeSD7aVc86VJnzoqXGbBrzuC9r/s6opFAQ=; b=Kn66xOjs81SGng0oCT/+Z7hx0FVYo/eEbSiAPSkG9YVazysZSGW4dQ90RJVw3fhxTU LjJTuf3qNq70mPiPOSxgd3h57DEvXCZ4aaKn8N0CWCciCzKED0k7HF0Tu2fNtAMQ4T0k 4c6XsSZEpyV80iSmIYTvIqO9MXT32jyPMkSRuqmIC1LEYBF6OB3BZ+ajDJ4YboAcW1DI 7NKm61fnwUrEFJkjyStI7+n8KexOl+3V3WwgZQYS/GQpoeqSunEHf249g+eF8J4tP0PX cJCST86Qnj/SwsTN+DWIWo6oRiD/mhGFKz4rr2flD9ipTpcUO41ef2MS2ymx9Vevq3uT D9Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tOQsL1UwAyeSD7aVc86VJnzoqXGbBrzuC9r/s6opFAQ=; b=i6QKaKRFJjfEGR+hmEOjrpJ1cmbATZviin2c8ClbZF3kZFNqGyDxprGxaJV0/0MuNk wviR1Xn4MtGJca+4BwGdWN5nocDLbFYuRV7yvHhuSrw8MbkBTrTu0QNvUUO/WUj+HQF6 5BjOtZ1mZFaxxo4wua5dD9yKtId2wZDmS1doOet+xvDEv0dW3qNFxlGmiwK6gA862Y/i z1QQE+yOzk3Ush4vEcc1yYOXxsB/JjEL0hnqYK7HC/yno7k7aGyE+sUjWwrTpZ+21pBm dPV4m4yFh841LNMtPdFx0O/mZtCQkOH8wzkZmegs8dhKifiRY4TatIVVeLbyC8Zd3ekZ IQeg== X-Gm-Message-State: AGi0PuYH1DESi1VK64ha9y8dkltaUF/QcMJoprq/FbBS3Sx+djuq/IHm Hrs+YEWYXhcBGDZrnM6u9z1/Uw== X-Google-Smtp-Source: APiQypKDfW2TB8LOB4CYBluerKzQ+zb+Jp1k53ujx1q7dkVM3x7cUFvX9tSZRYulxwzoPAbZJn1KxQ== X-Received: by 2002:a05:6a00:2cf:: with SMTP id b15mr6388378pft.174.1586332631240; Wed, 08 Apr 2020 00:57:11 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:10 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 1/9] riscv: add macro to get instruction length Date: Wed, 8 Apr 2020 15:56:56 +0800 Message-Id: <621303b9cdea215af57329b401b15750c1f683ab.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005711_963358_ACA5F44E X-CRM114-Status: UNSURE ( 9.34 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:544 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Extract the calculation of instruction length for common use. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/bug.h | 8 ++++++++ arch/riscv/kernel/traps.c | 3 ++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/bug.h b/arch/riscv/include/asm/bug.h index 75604fec1b1b..d6f1ec08d97b 100644 --- a/arch/riscv/include/asm/bug.h +++ b/arch/riscv/include/asm/bug.h @@ -19,6 +19,14 @@ #define __BUG_INSN_32 _UL(0x00100073) /* ebreak */ #define __BUG_INSN_16 _UL(0x9002) /* c.ebreak */ +#define GET_INSN_LENGTH(insn) \ +({ \ + unsigned long __len; \ + __len = ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? \ + 4UL : 2UL; \ + __len; \ +}) + typedef u32 bug_insn_t; #ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index ffb3d94bf0cc..a4d136355f78 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -118,7 +118,8 @@ static inline unsigned long get_break_insn_length(unsigned long pc) if (probe_kernel_address((bug_insn_t *)pc, insn)) return 0; - return (((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? 4UL : 2UL); + + return GET_INSN_LENGTH(insn); } asmlinkage __visible void do_trap_break(struct pt_regs *regs) From patchwork Wed Apr 8 07:56:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479545 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 80F7381 for ; Wed, 8 Apr 2020 07:57:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5F7F020747 for ; Wed, 8 Apr 2020 07:57:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Zvo7r6ag"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="DUUM9esi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F7F020747 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gzfFvJg7bfdrO4wN3wqu7wI1vDUb7LOnXkYNhBz170c=; b=Zvo7r6agZVoZ2E WxHeZ1aL0TA1LN59DHg0DHrtfXij7HQN2sUzXUW4Rjg0jze+5Qqq3p3MApPwb61kqlIz1UFlpQAxR a5vitMtZGZ8KQcsHCm7iPumkpNkXA+JrkrSdqqQZVD0MEE49bt0goAq48Ik2d3N4b325+jFFL8Pvg Urv+Okxv+hiOd7hRvG8tkar3+Fn/oRvpuNkq1sgr+DHpSBrMHk+ba0SDg4+S4sLuYzqWCWfGEVmlb G66PVRrGziB/cuohbMBtVrdQooloPQmsfqfGT8iseobMCW6MmKtGEhLtShpmEeJ9QqeglCfYLkMuR iIkQ1c7qJSqvSVr6twsA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aG-0007T3-Or; Wed, 08 Apr 2020 07:57:16 +0000 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aD-0007O8-Ok for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:15 +0000 Received: by mail-pl1-x642.google.com with SMTP id d24so2232460pll.8 for ; Wed, 08 Apr 2020 00:57:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gzfFvJg7bfdrO4wN3wqu7wI1vDUb7LOnXkYNhBz170c=; b=DUUM9esi5yDTXSCeZzowtJQEir93e17UxiKCRjrYo4mnv6nxqFylmAPtvD3smQejcm lJpb21k6+4md7Y8bwt5nrlCkyWISZ7kvJP8Dtg1E0ikmPiZkOFj+Z4Yjp0cIHCeuiePf +CrC+38+UO3/R2H/7o626s/xRNvv36sbC7++I8mt1Tb0GMqngN1tgAd97+UdUFC9jMfo X/n+qWxYwSR7rbY53FTcocyp9VjKWoMlFng8AL9MCieS8w7ASDULXLGOp4epJeWz/u5e IihrodIXip3NOHRh/k+cgJwRmtDESlr/uAxqOqmsmFl1OSVszWioEhBr03nrYtyEIjrj nERw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gzfFvJg7bfdrO4wN3wqu7wI1vDUb7LOnXkYNhBz170c=; b=AS4SqiBrOg5MxBBu0IseWxtSU1kCGdapADrrjoyEEfntg+rVws9S+5uC5+45DLQk9p dxzNzxnnuc6uGFxkTBlCNdTubt2cVhPECYAj6XKoLEIbDGYfK8grlkxj7Lqg54tQ3dd1 efH5z6WncK1bU4O9FBC3cU9ha6qRQZ2kEQiYbVmR1pHvz7G2M+aM3FFWYZwWPXBUBVHD XjXSpRDIjBBm/k75vkEVU9C1B3Y3seRAyS/P7Av/zsoodz9+RYN+a6wdJ/OgT4sij+bw 4AmmE3xvuVJV4wl+Calrin0HxyofVrcGDYEAJ4WC1/7CTQnZdZkAZ5fNbbxcbb5kVAtl U+kQ== X-Gm-Message-State: AGi0Pua7EF31FGdhiKdwExgnOqR96TNFe7YKi+Lo4Wyr3dUy8jEC8uu1 +uOrmLN7uHsOhCZip6xFpXr9FQ== X-Google-Smtp-Source: APiQypJqeiJZAEat09X84l9XFrf+GSwJJrpEFWwrSRBtjsWIdCdkVPTxNOW60AsyGwm+ha4ww8leAg== X-Received: by 2002:a17:90a:b702:: with SMTP id l2mr3826593pjr.22.1586332633065; Wed, 08 Apr 2020 00:57:13 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:12 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 2/9] riscv: introduce interfaces to patch kernel code Date: Wed, 8 Apr 2020 15:56:57 +0800 Message-Id: <6f145470ffdf83b303ecd83db4f6e06477b61220.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005713_814590_13F0939E X-CRM114-Status: GOOD ( 17.04 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:642 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org On strict kernel memory permission, we couldn't patch code without writable permission. Preserve two holes in fixmap area, so we can map the kernel code temporarily to fixmap area, then patch the instructions. We need two pages here because we support the compressed instruction, so the instruction might be align to 2 bytes. When patching the 32-bit length instruction which is 2 bytes alignment, it will across two pages. Introduce two interfaces to patch kernel code: riscv_patch_text_nosync: - patch code without synchronization, it's caller's responsibility to synchronize all CPUs if needed. riscv_patch_text: - patch code and always synchronize with stop_machine() Signed-off-by: Zong Li Suggested-by: Masami Hiramatsu Reviewed-by: Masami Hiramatsu Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/fixmap.h | 2 + arch/riscv/include/asm/patch.h | 12 +++ arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/patch.c | 128 ++++++++++++++++++++++++++++++++ 4 files changed, 145 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/patch.h create mode 100644 arch/riscv/kernel/patch.c diff --git a/arch/riscv/include/asm/fixmap.h b/arch/riscv/include/asm/fixmap.h index 42d2c42f3cc9..2368d49eb4ef 100644 --- a/arch/riscv/include/asm/fixmap.h +++ b/arch/riscv/include/asm/fixmap.h @@ -27,6 +27,8 @@ enum fixed_addresses { FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1, FIX_PTE, FIX_PMD, + FIX_TEXT_POKE1, + FIX_TEXT_POKE0, FIX_EARLYCON_MEM_BASE, __end_of_fixed_addresses }; diff --git a/arch/riscv/include/asm/patch.h b/arch/riscv/include/asm/patch.h new file mode 100644 index 000000000000..9a7d7346001e --- /dev/null +++ b/arch/riscv/include/asm/patch.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2020 SiFive + */ + +#ifndef _ASM_RISCV_PATCH_H +#define _ASM_RISCV_PATCH_H + +int patch_text_nosync(void *addr, const void *insns, size_t len); +int patch_text(void *addr, u32 insn); + +#endif /* _ASM_RISCV_PATCH_H */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index f40205cb9a22..d189bd3d8501 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -4,7 +4,8 @@ # ifdef CONFIG_FTRACE -CFLAGS_REMOVE_ftrace.o = -pg +CFLAGS_REMOVE_ftrace.o = -pg +CFLAGS_REMOVE_patch.o = -pg endif extra-y += head.o @@ -26,6 +27,7 @@ obj-y += traps.o obj-y += riscv_ksyms.o obj-y += stacktrace.o obj-y += cacheinfo.o +obj-y += patch.o obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_M_MODE) += clint.o diff --git a/arch/riscv/kernel/patch.c b/arch/riscv/kernel/patch.c new file mode 100644 index 000000000000..f16466123947 --- /dev/null +++ b/arch/riscv/kernel/patch.c @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 SiFive + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +struct patch_insn_patch { + void *addr; + u32 insn; + atomic_t cpu_count; +}; + +#ifdef CONFIG_MMU +static void *patch_map(void *addr, int fixmap) +{ + uintptr_t uintaddr = (uintptr_t) addr; + struct page *page; + + if (core_kernel_text(uintaddr)) + page = phys_to_page(__pa_symbol(addr)); + else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX)) + page = vmalloc_to_page(addr); + else + return addr; + + BUG_ON(!page); + + return (void *)set_fixmap_offset(fixmap, page_to_phys(page) + + (uintaddr & ~PAGE_MASK)); +} +NOKPROBE_SYMBOL(patch_map); + +static void patch_unmap(int fixmap) +{ + clear_fixmap(fixmap); +} +NOKPROBE_SYMBOL(patch_unmap); + +static int patch_insn_write(void *addr, const void *insn, size_t len) +{ + void *waddr = addr; + bool across_pages = (((uintptr_t) addr & ~PAGE_MASK) + len) > PAGE_SIZE; + int ret; + + /* + * Before reaching here, it was expected to lock the text_mutex + * already, so we don't need to give another lock here and could + * ensure that it was safe between each cores. + */ + lockdep_assert_held(&text_mutex); + + if (across_pages) + patch_map(addr + len, FIX_TEXT_POKE1); + + waddr = patch_map(addr, FIX_TEXT_POKE0); + + ret = probe_kernel_write(waddr, insn, len); + + patch_unmap(FIX_TEXT_POKE0); + + if (across_pages) + patch_unmap(FIX_TEXT_POKE1); + + return ret; +} +NOKPROBE_SYMBOL(patch_insn_write); +#else +static int patch_insn_write(void *addr, const void *insn, size_t len) +{ + return probe_kernel_write(addr, insn, len); +} +NOKPROBE_SYMBOL(patch_insn_write); +#endif /* CONFIG_MMU */ + +int patch_text_nosync(void *addr, const void *insns, size_t len) +{ + u32 *tp = addr; + int ret; + + ret = patch_insn_write(tp, insns, len); + + if (!ret) + flush_icache_range((uintptr_t) tp, (uintptr_t) tp + len); + + return ret; +} +NOKPROBE_SYMBOL(patch_text_nosync); + +static int patch_text_cb(void *data) +{ + struct patch_insn_patch *patch = data; + int ret = 0; + + if (atomic_inc_return(&patch->cpu_count) == 1) { + ret = + patch_text_nosync(patch->addr, &patch->insn, + GET_INSN_LENGTH(patch->insn)); + atomic_inc(&patch->cpu_count); + } else { + while (atomic_read(&patch->cpu_count) <= num_online_cpus()) + cpu_relax(); + smp_mb(); + } + + return ret; +} +NOKPROBE_SYMBOL(patch_text_cb); + +int patch_text(void *addr, u32 insn) +{ + struct patch_insn_patch patch = { + .addr = addr, + .insn = insn, + .cpu_count = ATOMIC_INIT(0), + }; + + return stop_machine_cpuslocked(patch_text_cb, + &patch, cpu_online_mask); +} +NOKPROBE_SYMBOL(patch_text); From patchwork Wed Apr 8 07:56:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479547 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EAF5981 for ; Wed, 8 Apr 2020 07:57:24 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BC7BE20771 for ; Wed, 8 Apr 2020 07:57:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="dqXWl9dn"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="GoQ2lksF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BC7BE20771 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XkJsQ5wK/x/TlqEbA+mDNHgNHjaEj7vOian5h3Az/dI=; b=dqXWl9dnRLkWKo HPAORMPD7nclg/tKAEfrayTyGEQROL0ZEFwhq9G3mkEvLaacTvZlbQV/kpNAffmYT6oassl2If/yR dUQIFCaSTwEHA0FIltlP/isUW+/0bl02cebHxyfgh2DC1+iz3jLA01Z4jt+e0uLKu74Z1KHOjzm/4 NI9Cw5HbZ41uTG7CA+sd7u/+UYIY5u26DuZl2uXztqi2MrxiLudulNPTYALTSKCt1h6tIIj2rkjHq oAEvPGZOFhT/D7Oge9roP/bJT3czwgcsvKIwqB6nsJEzCdnMo3J9cZLu7qvUslFAkQVlCWQrO6UVf dcGx/GfcfBUnVt1Ssc1Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aJ-0007We-Aw; Wed, 08 Apr 2020 07:57:19 +0000 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aF-0007Qp-Mb for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:17 +0000 Received: by mail-pg1-x541.google.com with SMTP id m13so2970197pgd.8 for ; Wed, 08 Apr 2020 00:57:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XkJsQ5wK/x/TlqEbA+mDNHgNHjaEj7vOian5h3Az/dI=; b=GoQ2lksF14YYCaOtMuoKYM7vT3Yx9eYSzDJeBCT1onK+swzFQVmrxQNz8A0Tx1UV7w N4ukqiuGUz7US9jG6G1pNTsCLz0BIPllR5BjjArtNUhj2U6cj8SyMJrr0nyeieYkUBWS lRuu0Ksmv3TE6giCaCh0dC879/19i4O751Yez/1Q6bLNmfw/Avdp1LOLgu3G/M+9XGP2 OD2j9gih+Y+EHJJMt6O6mMQ8IEtjo/iu/nAwwpROg7jmT5q2oS8i+Fyi2h759BYMgIic FXmTCxNbmRzSoqCsLEcV0QdM/JF3rA1UzVR/48veI1V1jXkwkN5Aj5Sh4zsr73xcomL4 u3yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XkJsQ5wK/x/TlqEbA+mDNHgNHjaEj7vOian5h3Az/dI=; b=ZAIioFoDcwrb3AQr1EZNbxZOH6picyYZTRd44/yMDH69uYspBupbOy/QQ814kH7RzS 4z73sA0Ek6yqUq9QKfqN1FA09zPmZ4tKP7g5v/EVLi6brAWW1MReoVaW+HXUA1E2HSRX Ie4eJMnYzFIQy0R3hWEFyAdIHOMN7pv2FHjpQDiJo3bazRKvvyzGyDj2FZDu2fZ4OI5N HgZFWPgnyEK5xQDZ3Nb4UtsXo554Cflr7cIHIb7x3My0Q4AbAhJg4GhGNokYKyFZxQ80 0CBOBGECWdywZXm9mEboROJBSW3vbxYo8dOsQmGE+mhAT0iFzpg7AaZeCyEnrRMXe/3j tY2g== X-Gm-Message-State: AGi0PuY1JQe9h3I8UYkfxyNRFj5BogiEqIIMvLh6gt66+qY+6sSDISH+ cyOA9Uvvzs5sAxW3krz/V0voqw== X-Google-Smtp-Source: APiQypJAFMM/eXRWkh6FXOGZKFYt6ev58qcGDr+z+KrnA0/S0UYY70C6txy6deaM8WjT5DJ2UcB/OQ== X-Received: by 2002:a62:1e84:: with SMTP id e126mr6071812pfe.315.1586332634924; Wed, 08 Apr 2020 00:57:14 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:14 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 3/9] riscv: patch code by fixmap mapping Date: Wed, 8 Apr 2020 15:56:58 +0800 Message-Id: <27f31750f58cd0ad2805579b16b001aa88103688.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005715_755718_A775B000 X-CRM114-Status: GOOD ( 10.73 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:541 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org On strict kernel memory permission, the ftrace have to change the permission of text for dynamic patching the intructions. Use riscv_patch_text_nosync() to patch code instead of probe_kernel_write. Signed-off-by: Zong Li Suggested-by: Masami Hiramatsu Reviewed-by: Masami Hiramatsu Reviewed-by: Palmer Dabbelt --- arch/riscv/kernel/ftrace.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index c40fdcdeb950..08396614d6f4 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -7,9 +7,23 @@ #include #include +#include #include +#include #ifdef CONFIG_DYNAMIC_FTRACE +int ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) +{ + mutex_lock(&text_mutex); + return 0; +} + +int ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) +{ + mutex_unlock(&text_mutex); + return 0; +} + static int ftrace_check_current_call(unsigned long hook_pos, unsigned int *expected) { @@ -46,20 +60,14 @@ static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, { unsigned int call[2]; unsigned int nops[2] = {NOP4, NOP4}; - int ret = 0; make_call(hook_pos, target, call); - /* replace the auipc-jalr pair at once */ - ret = probe_kernel_write((void *)hook_pos, enable ? call : nops, - MCOUNT_INSN_SIZE); - /* return must be -EPERM on write error */ - if (ret) + /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */ + if (patch_text_nosync + ((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE)) return -EPERM; - smp_mb(); - flush_icache_range((void *)hook_pos, (void *)hook_pos + MCOUNT_INSN_SIZE); - return 0; } From patchwork Wed Apr 8 07:56:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479549 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F2B4B1744 for ; Wed, 8 Apr 2020 07:57:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A6B3220769 for ; Wed, 8 Apr 2020 07:57:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="l5qQwIYG"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="I34Uf3OW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A6B3220769 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bdvS3ImXktegIGIUSt7DM4X3H//vtPG1T5XbtOqEIxE=; b=l5qQwIYGdE7VUs TFOJDxm9diRHB27WSZkilqhBT5ysxfwy2t74ovC1sE1T0qXI4vqWGLnbmN31tQ8QDy8iVp5PdOHXm zrplMYP5yJE39MyZl8CRmJpfbEQAxZm0P5ZvD91aq5uEITd5D3sKPUKjnD+zqZEMdLz1odT8kw7IX /LzRFtIH/HL+P1MPFYAj/fgtP9iQ53/E1OB2IAFTCWuh0t+0YI4A/26mFw/t36zm1civGo24nwau8 BtrVawqBARiYpvdXiFZnLY3hsHdhylzmdqCIuscu2j9Dg5QhWVh8idpZW2LHnLWSUfLVLqfz+Xp85 zwWVc6pFh/dJBfJ9FtXA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aM-0007bR-5G; Wed, 08 Apr 2020 07:57:22 +0000 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aH-0007To-QP for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:19 +0000 Received: by mail-pl1-x643.google.com with SMTP id h11so2231471plk.7 for ; Wed, 08 Apr 2020 00:57:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bdvS3ImXktegIGIUSt7DM4X3H//vtPG1T5XbtOqEIxE=; b=I34Uf3OWOjly54a8zK1Z6P+uRbyGvbAmOXqDlhNGGylMwBHq7lfHMSXiQI9W7wTnTH j72/kWxb5DGM7slCb9wWWEtqXAXn3J7A74hTMWuOM0Apo/nA07YnfV+eCosDd/mO2kK+ zqrlOtqMeX036nlaXiEsqaq19xXo25Eonlu7QYg/gBLuUn8ry9YCStTkJscntjl1E5/I im0ZgOKBamzkCnP7kR6cRKyqokCQ+v026Li3BYsIiFKVSza+WSv7DGnFB4UD27x968pI PdNW7iPHAJK5Kn7ZQpRsWaWJxsN+c3ZTp6Ywxln5tYnNnXRmD6TRgY96QZjt8vkPKXV0 42uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bdvS3ImXktegIGIUSt7DM4X3H//vtPG1T5XbtOqEIxE=; b=Sj9aKIvJ1ySQHkeFuTAMdc9Le1deEINOG8d4EWEMwNwGBNVz4RbtsST/15g5fglRg1 yx4/5H0uSPP3CqCywiF54A//b9n6pjhX3Yr55YsgJ+ijnbyOFvXRGnUXRgnYdcysYpIo 8ppZNxX5CMc3kiLYgHOBG8NjzXeg9bAmeIH0JlsEUk+An/kC+hSIUaP/ewEa3yUqG5Cl 21rCRIVJUP2ovsJyDC3pTYkEvt7JwRkUxO9D5Ex7i6jPcScfOe6m8lFSRLfSmG38NoLO G6p6Jn7JhImjp6gF40IrC9oaAB6fIHFs5ODFVlymTLCICCPjouDvRHDEBC8QH0FMcyXc epnw== X-Gm-Message-State: AGi0PuaQoznkP0zlYsEfdBbDU+MfWumj9AUZ0IhhHuV5+SWbnm+w4p51 BQ25qP2cnDdDLs9gjf+PLWEa5Q== X-Google-Smtp-Source: APiQypIcP/jAUQL/r+m9eqvPJnOxTFkevN0rViU3kpPi2xUJOdriYl6OrqcAg/opP7nZzVezzNKuAA== X-Received: by 2002:a17:902:8bc3:: with SMTP id r3mr6124029plo.220.1586332636877; Wed, 08 Apr 2020 00:57:16 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:16 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 4/9] riscv: add ARCH_HAS_SET_MEMORY support Date: Wed, 8 Apr 2020 15:56:59 +0800 Message-Id: <1bb772acf62521b731305f16240e1f393fab839a.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005717_868574_5DAEBC37 X-CRM114-Status: GOOD ( 13.56 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:643 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Add set_memory_ro/rw/x/nx architecture hooks to change the page attribution. Use own set_memory.h rather than generic set_memory.h (i.e. include/asm-generic/set_memory.h), because we want to add other function prototypes here. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/set_memory.h | 24 +++++ arch/riscv/mm/Makefile | 2 +- arch/riscv/mm/pageattr.c | 150 ++++++++++++++++++++++++++++ 4 files changed, 176 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/set_memory.h create mode 100644 arch/riscv/mm/pageattr.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 39c93ddff0cb..9044e0dd95ea 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -59,6 +59,7 @@ config RISCV select HAVE_EBPF_JIT if 64BIT select EDAC_SUPPORT select ARCH_HAS_GIGANTIC_PAGE + select ARCH_HAS_SET_MEMORY select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select SPARSEMEM_STATIC if 32BIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h new file mode 100644 index 000000000000..79a810f0f38b --- /dev/null +++ b/arch/riscv/include/asm/set_memory.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2019 SiFive + */ + +#ifndef _ASM_RISCV_SET_MEMORY_H +#define _ASM_RISCV_SET_MEMORY_H + +/* + * Functions to change memory attributes. + */ +#ifdef CONFIG_MMU +int set_memory_ro(unsigned long addr, int numpages); +int set_memory_rw(unsigned long addr, int numpages); +int set_memory_x(unsigned long addr, int numpages); +int set_memory_nx(unsigned long addr, int numpages); +#else +static inline int set_memory_ro(unsigned long addr, int numpages) { return 0; } +static inline int set_memory_rw(unsigned long addr, int numpages) { return 0; } +static inline int set_memory_x(unsigned long addr, int numpages) { return 0; } +static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } +#endif + +#endif /* _ASM_RISCV_SET_MEMORY_H */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 814e16a8d68a..363ef01c30b1 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -7,7 +7,7 @@ endif obj-y += init.o obj-y += extable.o -obj-$(CONFIG_MMU) += fault.o +obj-$(CONFIG_MMU) += fault.o pageattr.o obj-y += cacheflush.o obj-y += context.o diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c new file mode 100644 index 000000000000..fcd59ef2835b --- /dev/null +++ b/arch/riscv/mm/pageattr.c @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2019 SiFive + */ + +#include +#include +#include +#include + +struct pageattr_masks { + pgprot_t set_mask; + pgprot_t clear_mask; +}; + +static unsigned long set_pageattr_masks(unsigned long val, struct mm_walk *walk) +{ + struct pageattr_masks *masks = walk->private; + unsigned long new_val = val; + + new_val &= ~(pgprot_val(masks->clear_mask)); + new_val |= (pgprot_val(masks->set_mask)); + + return new_val; +} + +static int pageattr_pgd_entry(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pgd_t val = READ_ONCE(*pgd); + + if (pgd_leaf(val)) { + val = __pgd(set_pageattr_masks(pgd_val(val), walk)); + set_pgd(pgd, val); + } + + return 0; +} + +static int pageattr_p4d_entry(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + p4d_t val = READ_ONCE(*p4d); + + if (p4d_leaf(val)) { + val = __p4d(set_pageattr_masks(p4d_val(val), walk)); + set_p4d(p4d, val); + } + + return 0; +} + +static int pageattr_pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pud_t val = READ_ONCE(*pud); + + if (pud_leaf(val)) { + val = __pud(set_pageattr_masks(pud_val(val), walk)); + set_pud(pud, val); + } + + return 0; +} + +static int pageattr_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pmd_t val = READ_ONCE(*pmd); + + if (pmd_leaf(val)) { + val = __pmd(set_pageattr_masks(pmd_val(val), walk)); + set_pmd(pmd, val); + } + + return 0; +} + +static int pageattr_pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + pte_t val = READ_ONCE(*pte); + + val = __pte(set_pageattr_masks(pte_val(val), walk)); + set_pte(pte, val); + + return 0; +} + +static int pageattr_pte_hole(unsigned long addr, unsigned long next, + int depth, struct mm_walk *walk) +{ + /* Nothing to do here */ + return 0; +} + +const static struct mm_walk_ops pageattr_ops = { + .pgd_entry = pageattr_pgd_entry, + .p4d_entry = pageattr_p4d_entry, + .pud_entry = pageattr_pud_entry, + .pmd_entry = pageattr_pmd_entry, + .pte_entry = pageattr_pte_entry, + .pte_hole = pageattr_pte_hole, +}; + +static int __set_memory(unsigned long addr, int numpages, pgprot_t set_mask, + pgprot_t clear_mask) +{ + int ret; + unsigned long start = addr; + unsigned long end = start + PAGE_SIZE * numpages; + struct pageattr_masks masks = { + .set_mask = set_mask, + .clear_mask = clear_mask + }; + + if (!numpages) + return 0; + + down_read(&init_mm.mmap_sem); + ret = walk_page_range_novma(&init_mm, start, end, &pageattr_ops, NULL, + &masks); + up_read(&init_mm.mmap_sem); + + flush_tlb_kernel_range(start, end); + + return ret; +} + +int set_memory_ro(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(_PAGE_READ), + __pgprot(_PAGE_WRITE)); +} + +int set_memory_rw(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(_PAGE_READ | _PAGE_WRITE), + __pgprot(0)); +} + +int set_memory_x(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(_PAGE_EXEC), __pgprot(0)); +} + +int set_memory_nx(unsigned long addr, int numpages) +{ + return __set_memory(addr, numpages, __pgprot(0), __pgprot(_PAGE_EXEC)); +} From patchwork Wed Apr 8 07:57:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479551 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D50D91744 for ; Wed, 8 Apr 2020 07:57:28 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B3D2720769 for ; Wed, 8 Apr 2020 07:57:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="O2WPeH/A"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="M8FsO9ur" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3D2720769 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1qkEQ73ghM61/BXRvlDJEs7SEg6hkPYIpkfLFdxJqkM=; b=O2WPeH/A+Xd5Wn VT4sQUiKhLHSvD3rF5EGR0qYhOt62e3P8FGEzgRxW1FFvkZwIolzk3ReJBNZCbLdwWV+ruQH2bIWa Pn9RiajIBiEIsBDFfq6aP+EiHhKNNa+Bv7KWS3VH3/8txjidGVUwoqQp+OU7LNlmbKqozEz0RX/QM 79wFhjM5qbRSafqoIRYK+BHDEgh3fa4bbf1onruRGl0aCS0U6pEkTFOhWmsUxeuxQdcTM3sAkFfvr BcfbEqtn2bKd5FaaxBds8ykAVU9ZzMU7cuvUj3roINseiI2F6dPCnGnwNZeXPlqOqcYm2Fw2JQSiQ sn4IYCI6b7lP1G5DHECw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aO-0007f5-P9; Wed, 08 Apr 2020 07:57:24 +0000 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aJ-0007Wo-D0 for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:20 +0000 Received: by mail-pl1-x642.google.com with SMTP id t4so2222005plq.12 for ; Wed, 08 Apr 2020 00:57:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1qkEQ73ghM61/BXRvlDJEs7SEg6hkPYIpkfLFdxJqkM=; b=M8FsO9urOLYfx+qNB72XqLl5IeK3k1rkWGB2QMaKDxaZ3GMZ9TkmwxK2X106mzWTZc h6SfGPemIf+TGa9RC0VllenRql97VDhVTSgiLjT7uRcU/sIo1L09xxkH4TJZbbTOdany KvIdpvhuZ7aZu4Hew/CRlD3Nu11OQ1e8+YCp1KVzmmD0In0eppM5yJwdWx6d1xyfhEF5 F7/KOZyVklGrHaTKN3apLnV+MT57NiaA9GWFMExs7czk8HzYqQxKqRwOpymkHHiB4R0b e94SlP/mbS0WGfoYQTh6cOJPSe609uoPJPYWGWI5B03/10hz2Cwb363yMLe7BTSN9DRM yFGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1qkEQ73ghM61/BXRvlDJEs7SEg6hkPYIpkfLFdxJqkM=; b=cMHL4UNkV5ELRRCc+aqcdUC2ot4nqXgVnEQqOdaH8Lh6gyTjhK2WbIpHb2TV7em+b6 A2ozWBohC72+ABU6A7pas6Kmb/l/n++50nT2SOio5FrpwVdOIP//Y8ka99qe842vjPWd IJvBAROAy6h04Msg84WSrUeeQqN61RROwvvn28A2K4K+2mQJT7lX/J5NP8/drrMbBt8b QO+Yy3bgRKktcgKebURwsnx75pD7X066c7uJPNykND2zPe2WWtZ59/9W8DKpVo8ANJgb eXnQSsX721m8HG0XjsjvAqoyzWZJ9DjHH4i1QUhxHcpu8a8Ia5vXjwsWrPj/ioeykKFw IuaQ== X-Gm-Message-State: AGi0Pua16Mvfq1N90RyJ5I0e/tH2A1BqAbtVcaxP3iSZevJAHRiR39/w cQbf6yGCm2dvDHn/BPnH07QkoA== X-Google-Smtp-Source: APiQypIIXeNxjIMGSbVinBL4trNwBKG9qqUy1Cb/noizBIcTE9mvQcsmUXs8oVjWTimn72tJBLN8Cw== X-Received: by 2002:a17:90a:33d1:: with SMTP id n75mr3776658pjb.167.1586332638821; Wed, 08 Apr 2020 00:57:18 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:18 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 5/9] riscv: add ARCH_HAS_SET_DIRECT_MAP support Date: Wed, 8 Apr 2020 15:57:00 +0800 Message-Id: <357ebe77f7f676dcc7e828436275ec096fd61a96.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005719_543424_55D24CFD X-CRM114-Status: UNSURE ( 9.84 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:642 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Add set_direct_map_*() functions for setting the direct map alias for the page to its default permissions and to an invalid state that cannot be cached in a TLB. (See d253ca0c ("x86/mm/cpa: Add set_direct_map_*() functions")) Add a similar implementation for RISC-V. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/set_memory.h | 3 +++ arch/riscv/mm/pageattr.c | 24 ++++++++++++++++++++++++ 3 files changed, 28 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 9044e0dd95ea..a94d0f064d9c 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -59,6 +59,7 @@ config RISCV select HAVE_EBPF_JIT if 64BIT select EDAC_SUPPORT select ARCH_HAS_GIGANTIC_PAGE + select ARCH_HAS_SET_DIRECT_MAP select ARCH_HAS_SET_MEMORY select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select SPARSEMEM_STATIC if 32BIT diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h index 79a810f0f38b..620d81c372d9 100644 --- a/arch/riscv/include/asm/set_memory.h +++ b/arch/riscv/include/asm/set_memory.h @@ -21,4 +21,7 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; } static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } #endif +int set_direct_map_invalid_noflush(struct page *page); +int set_direct_map_default_noflush(struct page *page); + #endif /* _ASM_RISCV_SET_MEMORY_H */ diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index fcd59ef2835b..7be6cd67e2ef 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -148,3 +148,27 @@ int set_memory_nx(unsigned long addr, int numpages) { return __set_memory(addr, numpages, __pgprot(0), __pgprot(_PAGE_EXEC)); } + +int set_direct_map_invalid_noflush(struct page *page) +{ + unsigned long start = (unsigned long)page_address(page); + unsigned long end = start + PAGE_SIZE; + struct pageattr_masks masks = { + .set_mask = __pgprot(0), + .clear_mask = __pgprot(_PAGE_PRESENT) + }; + + return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks); +} + +int set_direct_map_default_noflush(struct page *page) +{ + unsigned long start = (unsigned long)page_address(page); + unsigned long end = start + PAGE_SIZE; + struct pageattr_masks masks = { + .set_mask = PAGE_KERNEL, + .clear_mask = __pgprot(0) + }; + + return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks); +} From patchwork Wed Apr 8 07:57:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479553 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24CB681 for ; Wed, 8 Apr 2020 07:57:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F414920747 for ; Wed, 8 Apr 2020 07:57:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="LyZ0b7U8"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="U7G20s6L" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F414920747 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=v9zl9wdVM8GEYtYil3Mhwjs87sA3d02fBbOSD/V/2Io=; b=LyZ0b7U8XF+WNs XLerlZfFOEOAvYJH20Y5hF+IIgdYevM7hXI7SdBDsyPvs8YKdHhwyTQA+ursSWJisICu7ZXuf0jcQ NRbU/aI53Du+CsWvVvyQp86sVl8XfPvOJ2IjO61jl4fIypGIidIiM7+p5riQmUp5VYpgkf8uMc6Dd lIkRG1ntDbZPLvNVmITbYJgMZp1u8rq+kpQdln9rL3d1RnjTHNYW29hUflHPE1Tcik7DSgZcQT+C9 vCeAk1d+H6ZjLw5jTGCnCU/HXhJagJ+Lv9wefT2nCQ3U/iYBEmF/ANXN7dPoNbYDzmbRKmfW3iZXp hpTjPbi8syOZQyOler5w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aT-0007lW-3g; Wed, 08 Apr 2020 07:57:29 +0000 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aL-0007a0-B0 for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:22 +0000 Received: by mail-pl1-x644.google.com with SMTP id v2so860541plp.9 for ; Wed, 08 Apr 2020 00:57:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v9zl9wdVM8GEYtYil3Mhwjs87sA3d02fBbOSD/V/2Io=; b=U7G20s6LmTf2LuYfnrXDNYEjXHb9WEsy956inSxRQMo/VA6l9qWdgAo0dEbehYjLYD f5E2WmGKWxTZ/YFKy5cJ2nJCMcOHestOXgrl8JHnYc6uOLdJMD66pzAkhMoz0lXMJa1a q6Sd93eJ5SmSFiJQR2GnbVljhf0cUURmRBg2OIVfpRGKqhHY6oA0yOci57cIypYhU94D KQViqVTTVemVaSYG2zoGoljdEnt4pnHr8w+Avu8mP8JIeNAYm12aAVJ+P5t/mLZGbxTk ky/UBHaVirsmaCNTa+FVw3OV3jfW5Xa2HRerKbZmZibpXXzOtY/bXT0OyuTQyV6liYO2 rGmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v9zl9wdVM8GEYtYil3Mhwjs87sA3d02fBbOSD/V/2Io=; b=sBHNqSA3HmnPzIRj4GhBlPJ3GmsuDfzSJ5Xx3zEgsTQxcOXg7fGbwsMWmfmazpVl1G 3f8KTj6dproJpTch7YCN8rk5V5Z+6RJRki3zAnZE2wooaRHgidkT502rbQU3rgK6YiG3 wavpGXeWedcgfJL7zMpc37DJsFxzwQFeVefI5bmahaQHbDc46mHGjUmUbbr4tTVXsajG 0Ln4qSSZmhFjI0kWpxi9A+Jk9lVborfEfvqxEVnLA6dbISrweshLJarL5E2+9ojgtUjs gLmoTM5wGppFxnzI3QkX4jaXKfywdR7rJj/s3K4cAkLyYi8Odf7gnrXCwHb0nb6A/Srw QeIg== X-Gm-Message-State: AGi0Pub7m08OmvdWWN7h1DbejNhT/7a0AV20GTTxEJrr+wbknvKpG92g noL/3A7gHYFt5QlNj8dO6pjNqg== X-Google-Smtp-Source: APiQypLxV8BTgShiG5s/VFnuxWwrOSXsD17o88Vs6ZV9ud2Hzidb8dVTOvmyVJfp26rP9LzGVWdm7w== X-Received: by 2002:a17:90a:db54:: with SMTP id u20mr3902925pjx.140.1586332640815; Wed, 08 Apr 2020 00:57:20 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:20 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 6/9] riscv: add ARCH_SUPPORTS_DEBUG_PAGEALLOC support Date: Wed, 8 Apr 2020 15:57:01 +0800 Message-Id: <15fce3374de4978c4541beaab8216bf0f84ca72d.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005721_378551_A5AC3058 X-CRM114-Status: UNSURE ( 8.12 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:644 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org ARCH_SUPPORTS_DEBUG_PAGEALLOC provides a hook to map and unmap pages for debugging purposes. Implement the __kernel_map_pages functions to fill the poison pattern. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/Kconfig | 3 +++ arch/riscv/mm/pageattr.c | 13 +++++++++++++ 2 files changed, 16 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index a94d0f064d9c..1e1efc998baf 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -132,6 +132,9 @@ config ARCH_SELECT_MEMORY_MODEL config ARCH_WANT_GENERAL_HUGETLB def_bool y +config ARCH_SUPPORTS_DEBUG_PAGEALLOC + def_bool y + config SYS_SUPPORTS_HUGETLBFS def_bool y diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index 7be6cd67e2ef..728759eb530a 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -172,3 +172,16 @@ int set_direct_map_default_noflush(struct page *page) return walk_page_range(&init_mm, start, end, &pageattr_ops, &masks); } + +void __kernel_map_pages(struct page *page, int numpages, int enable) +{ + if (!debug_pagealloc_enabled()) + return; + + if (enable) + __set_memory((unsigned long)page_address(page), numpages, + __pgprot(_PAGE_PRESENT), __pgprot(0)); + else + __set_memory((unsigned long)page_address(page), numpages, + __pgprot(0), __pgprot(_PAGE_PRESENT)); +} From patchwork Wed Apr 8 07:57:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479557 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C0221744 for ; Wed, 8 Apr 2020 07:57:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6463620769 for ; Wed, 8 Apr 2020 07:57:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="L+khQVf/"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="mPZ6G/Dv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6463620769 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Quhy5/QZPA8/P4q/9fnjufmjHKuWjkZaQYF9/wabbVw=; b=L+khQVf/aHmpJg ZESn/MgzMFyM7cLIb+so4ro6NFA0oGDHZiQBfFjR1I/yI/hQI3LHjQKRFKJauUyJT2B7okqxu+lLZ VJHBFLhwltg3D7/UlhyPZTGii5kwYadns5nfBNtDtSIJvQrzzTJcQ1dJ9H4VpOYmeg7vqmrPYcOo6 cVIVG7CKJQDL1qPZeahNWxpNUAoNlytdcWNpdSvg0uJ6Pl231ysNpHXPxh0Yj3pC3GTvrB6/SzGSu Y77cXwx8Otk3Cu9Bm+QM/UsjrY4CoQjRWcM70ZYiS0EYacdoxPzB/i+bSS/v3mN6TuidC78Pv/GlX /6PsI9BMu2mE5bbvOrGg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aX-0007sr-V4; Wed, 08 Apr 2020 07:57:33 +0000 Received: from mail-pj1-x1041.google.com ([2607:f8b0:4864:20::1041]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aN-0007ct-Jc for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:24 +0000 Received: by mail-pj1-x1041.google.com with SMTP id kx8so840832pjb.5 for ; Wed, 08 Apr 2020 00:57:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Quhy5/QZPA8/P4q/9fnjufmjHKuWjkZaQYF9/wabbVw=; b=mPZ6G/DvSUkmvJrsHi7YzN51egMqhQiZtrVAWVMnop5ZSHgLEIBO5VaJpn2sJ8xXc5 ceGghSnnkHeQjGqVbbP2GSy0IaxBEEDD6Uf8jv9x8gIZlTQBljt2bYXg6NVDSFCdlCZu 8816ZszTSt5DuxQO6UH5wDNaGBBzdvNleauumy+j4WmLKypcrS3dk46ERhxQb0XuNgpI ZF9HkUjimJRUSoU1j4w7FGVsVl7LPc7yXlvcMju/o9Dfbc/lTI3GZnewD5hzYZvIjtsT F2Kr+fHNYLd4hVK2lKG/xFoBanJbWknoy5vYCfoT2CFkMrgBE5EeLDHzF4N5HwLlEfYh 7lyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Quhy5/QZPA8/P4q/9fnjufmjHKuWjkZaQYF9/wabbVw=; b=SdfCgFfc2tbUC05BEIAHvaDYvVcitO2/i2IUSejAj4jgNNvBOMnIDLkU4KAwBN16OI JrG+XN4u0QUWnkCEpGg4KadyG+iWc5Qtj7to4SoZFr9ixIHTIa4tHqzagAhesMRJdDn5 hrk8pwBjZDPOyMlCv995/puN4vHuKi4P6MrlUST/vya+ARjUO8AP7nbaGbynjyewGOr0 4Jsfuq4Wmptcq34pmcodgB3BoC1oVRsjwOaMIMiBrTwQztj40+vWnQtEzXkykA5wtv5A 3z3GCvcl3WT/XAXCT60D8Teh8Bk27DHQXjSl8Qmqy0AtsjHBWFyczTwzixLXMUs/v/tU 6Jzg== X-Gm-Message-State: AGi0PubdWBCryzm+njf6ZLlUFb3nIWffSuH1clPU3rXq31vIzK2S4bAb oa1SCiRfv/hp/NEUlqhZZhSZNA== X-Google-Smtp-Source: APiQypJXFX/0Lnb7UwEs543bcGfXQmbM061XO3WoufDW75ChPwGGHilbdAdbTBnLkBOrX4EDrVWkHw== X-Received: by 2002:a17:90a:65c5:: with SMTP id i5mr2480742pjs.18.1586332642779; Wed, 08 Apr 2020 00:57:22 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:22 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 7/9] riscv: move exception table immediately after RO_DATA Date: Wed, 8 Apr 2020 15:57:02 +0800 Message-Id: <6dba29ff224e253bcacc7891caecadb5666d8059.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005723_647527_FBD00449 X-CRM114-Status: UNSURE ( 9.71 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:1041 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org Move EXCEPTION_TABLE immediately after RO_DATA. Make it easy to set the attribution of the sections which should be read-only at a time. Add _data to specify the start of data section with write permission. This patch is prepared for STRICT_KERNEL_RWX support. Signed-off-by: Zong Li Suggested-by: Palmer Dabbelt --- arch/riscv/kernel/vmlinux.lds.S | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S index 1e0193ded420..02e948b620af 100644 --- a/arch/riscv/kernel/vmlinux.lds.S +++ b/arch/riscv/kernel/vmlinux.lds.S @@ -58,6 +58,10 @@ SECTIONS *(.srodata*) } + EXCEPTION_TABLE(0x10) + + _data = .; + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE) .sdata : { __global_pointer$ = . + 0x800; @@ -69,8 +73,6 @@ SECTIONS BSS_SECTION(PAGE_SIZE, PAGE_SIZE, 0) - EXCEPTION_TABLE(0x10) - .rel.dyn : { *(.rel.dyn*) } From patchwork Wed Apr 8 07:57:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479555 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9BAE51744 for ; Wed, 8 Apr 2020 07:57:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7BE7A20769 for ; Wed, 8 Apr 2020 07:57:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="nfidC9TY"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="S5QTx2CF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7BE7A20769 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=geDMvpdyp4eREUjGenxPiMAgszPzsf8hdgTrRUcyDVI=; b=nfidC9TYl14okL 9r0yRics/fqbGsaaeAIfypVzwuAORsmplrqja0cKARXEQYx/63IXtSmpUD851Qf6Itm7Cb4Lj3tti 0zNhNDM2BN5YaogSespZuOjMrYH9S1qHejNq2yzqfIVoVPT+TaWREgW7DgKmbZl/MUEDGuJnI2SEp paVdi7mXd5toKISgk2SPNsbR56PR6wsgNPLXfOxzbwKNZnO6S8NNPZVb3xpcMWcVnzkxPvFl1+TR2 BgnM/vz37166FgCUHqnFjKyGLG1kLcHTQiR3s02U6lDlkoz9LeLl/GHgf0I8RzW55NqDkjyeWFyao fbWfMdLdNJVOriAqQeBw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aV-0007pT-Rs; Wed, 08 Apr 2020 07:57:31 +0000 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aP-0007fw-9g for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:26 +0000 Received: by mail-pl1-x643.google.com with SMTP id d24so2232668pll.8 for ; Wed, 08 Apr 2020 00:57:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=geDMvpdyp4eREUjGenxPiMAgszPzsf8hdgTrRUcyDVI=; b=S5QTx2CFt98Mt0i/C+3txkabVRmjPH9b1pHlRJMR81mtdW58kYh3qyK64Fxh3p+xSf aAW3UKDWwCxdsOUs10tw5Ieny0e/pYI/NixKjBtIvtsme6hq16mKn0ItpbINHGIfLrgl ahuZAlc1+dGCD+d7DCjrGl9v4lTLPWjkH+eMkZ2iyLMgWUlGt93OeiLRRj5jrflGGofw K+B3X0TIF3SIYc19ebpPfRZ0nORpsN0mnJSbba8ZifizsOIGjzaTmsoSA6e1xANjcnWr C2IZ5SKoe/ACv7Ijz+iMtf02S2kQM754MH/kuVyZkMTV6803o5EHrqrVFleBcaWMTS8a lPDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=geDMvpdyp4eREUjGenxPiMAgszPzsf8hdgTrRUcyDVI=; b=chsQdDlkAJSlgt6f5slA8JY75W9BqNq5zKtjABbu/wyOdLzgrarFsFrhyLnrkeukD1 34bP8AKHQ1jPYdw54dfjcVbur8yXSQhnqaF6VOqa0JWxVRuet9xUJgaFpNuBtrm5pwKL 7FPjJ/ppxkoFSyLROx1qg6Al8sZhOThr164D3Vxmbpllp0n2ACsj1qp5b/0fF2d4KSoS +i2SXgoG9Nb8HEjPp0XcuM/RstShqWbXn81b8lYDSALr5TaigNjvt0cEVDiQZDMGoSFS v/lNTwzazsB7tgkP/rzx0lqfa5YIn1Lx5lorvns4A0ZVIySrSR/c8OX3cNykQFECoRlu cg/A== X-Gm-Message-State: AGi0Pubu2hm/x9DmtDXIKABhdOSuXeYMr2E/XMImnpx9ruYR1hnkrKnB R0Jh/1Vbug132yc4yumsen9wxA== X-Google-Smtp-Source: APiQypI5ixhPVxiaOZYKBPMsxFPDPz2i1VrQzUF3bngypFXRTDQ8GJzp3IxjnJPFrgBfCklRUsc91A== X-Received: by 2002:a17:90a:1b6b:: with SMTP id q98mr3859791pjq.107.1586332644756; Wed, 08 Apr 2020 00:57:24 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:24 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 8/9] riscv: add alignment for text, rodata and data sections Date: Wed, 8 Apr 2020 15:57:03 +0800 Message-Id: X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005725_360140_B896A741 X-CRM114-Status: GOOD ( 12.10 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:643 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org The kernel mapping will tried to optimize its mapping by using bigger size. In rv64, it tries to use PMD_SIZE, and tryies to use PGDIR_SIZE in rv32. To ensure that the start address of these sections could fit the mapping entry size, make them align to the biggest alignment. Define a macro SECTION_ALIGN because the HPAGE_SIZE or PMD_SIZE, etc., are invisible in linker script. This patch is prepared for STRICT_KERNEL_RWX support. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/set_memory.h | 13 +++++++++++++ arch/riscv/kernel/vmlinux.lds.S | 5 ++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h index 620d81c372d9..4c5bae7ca01c 100644 --- a/arch/riscv/include/asm/set_memory.h +++ b/arch/riscv/include/asm/set_memory.h @@ -6,6 +6,7 @@ #ifndef _ASM_RISCV_SET_MEMORY_H #define _ASM_RISCV_SET_MEMORY_H +#ifndef __ASSEMBLY__ /* * Functions to change memory attributes. */ @@ -24,4 +25,16 @@ static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } int set_direct_map_invalid_noflush(struct page *page); int set_direct_map_default_noflush(struct page *page); +#endif /* __ASSEMBLY__ */ + +#ifdef CONFIG_ARCH_HAS_STRICT_KERNEL_RWX +#ifdef CONFIG_64BIT +#define SECTION_ALIGN (1 << 21) +#else +#define SECTION_ALIGN (1 << 22) +#endif +#else /* !CONFIG_ARCH_HAS_STRICT_KERNEL_RWX */ +#define SECTION_ALIGN L1_CACHE_BYTES +#endif /* CONFIG_ARCH_HAS_STRICT_KERNEL_RWX */ + #endif /* _ASM_RISCV_SET_MEMORY_H */ diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S index 02e948b620af..ef87deea0350 100644 --- a/arch/riscv/kernel/vmlinux.lds.S +++ b/arch/riscv/kernel/vmlinux.lds.S @@ -9,6 +9,7 @@ #include #include #include +#include OUTPUT_ARCH(riscv) ENTRY(_start) @@ -36,6 +37,7 @@ SECTIONS PERCPU_SECTION(L1_CACHE_BYTES) __init_end = .; + . = ALIGN(SECTION_ALIGN); .text : { _text = .; _stext = .; @@ -53,13 +55,14 @@ SECTIONS /* Start of data section */ _sdata = .; - RO_DATA(L1_CACHE_BYTES) + RO_DATA(SECTION_ALIGN) .srodata : { *(.srodata*) } EXCEPTION_TABLE(0x10) + . = ALIGN(SECTION_ALIGN); _data = .; RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE) From patchwork Wed Apr 8 07:57:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zong Li X-Patchwork-Id: 11479559 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85C2A81 for ; Wed, 8 Apr 2020 07:57:39 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6431F20747 for ; Wed, 8 Apr 2020 07:57:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="IwGDa8hM"; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="TOZYTN0Z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6431F20747 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=E4WIiXeGbKDxs9UbyyfZOjVgyCJTWA/AduHG+mw3Ozg=; b=IwGDa8hMbArYAv jElh/cwS297LhR0injtc9o/6yyLkFXJHbV7BfFbNuSrNRCeSWP4e9z0KZNXeY8MheVLZh0/mtCRqQ hBDkTBW1dNEztVkouruFNGqZMscMSZ/Md5GkKkvM2wEVj5HUWzhYhO1DjAaWZelrMXwzVDqhqAzhC uGjXdaqsEXY83k7hUd9KuiIfzVNfRwrbLYIgR6jZyrLxO7e3zHOEs4+ZK0lW2lThb4MxEUzsxUQoB vI7ezuh5Fk8lc3vzp+jVV5ocfqIWr6syILwQtnkhlZNUuISmmyfTzpGJo1/G0b/IIkndscVpqE1bW niQJ0ekrNCY5wQ8Hu5vw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aa-0007w6-2I; Wed, 08 Apr 2020 07:57:36 +0000 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jM5aR-0007iW-91 for linux-riscv@lists.infradead.org; Wed, 08 Apr 2020 07:57:28 +0000 Received: by mail-pl1-x641.google.com with SMTP id s23so2218089plq.13 for ; Wed, 08 Apr 2020 00:57:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E4WIiXeGbKDxs9UbyyfZOjVgyCJTWA/AduHG+mw3Ozg=; b=TOZYTN0ZmTo7tkdrjh0d1AhqQRXNkDEiNfrJu4HSuIaIiOkXzgv343+X9uduChlxuH 5GPD/mmxsXBoBB14GTJKfsYkmdbX+iUL+dCVfbt+u75yXYegqNi1SrMLOcTYAi4T88lv qtT4EPWIizz8lTpXFwCmAj5VEUJfA30aJ5B+himyfvXRvcepspXbnZEigEMTMyxBRsI1 rCfbOUbOO2fuhdV2fehBFtONQJoNk61FccNMvakWX+2TCphrgANZ1IFSPA42eCeWa9OG M+RbsCIz3G7evFBUt61IGLqkg9rI5gPYiJuMTj5URYLiZTZW/IrnKhHTcL/h5ex2W4c1 2vRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E4WIiXeGbKDxs9UbyyfZOjVgyCJTWA/AduHG+mw3Ozg=; b=fCEaTQqjO7tb0x3uj0he3oC9RigFQyMsbJFnAwjIq5Z0y3IiqMrChYVuPmE2iKoyP3 y7hyyRAxLVka40PWtPQ+cf+WuBDBlNbECwN8ran2FOsGIjpq4E+Qs5VZZiXX6gsS5tlA 5iOnSiNDXbSCL7oHS64dRonhCObqlrBwGRqNkx7zi4LeWlDV5cNHcaH4ImXoohPT6yqk tk55qI318kFXoMvNwSrU2EWH5Ucf2b0Xv340mZAZC/EiqA7cHo9LytmSiZeXDCY2CcEW keMuShZLMpXR9IFSGTjc1mtJFZlLYSCaOPR8ROW944cP18XU3a0uNR1soHtylyGYoP0f UDDw== X-Gm-Message-State: AGi0PuZ3RTH+EmJ7bSoSidlG7C8BXpzqKNLBjZhcOP1U+vdS+Wk8PjKV qrUt9phQa7UabDeFNPT4mha/PWiBpW0w1A== X-Google-Smtp-Source: APiQypIgaMKX49GV/U1HjE4l3Up0jhdZCobmLhyfkuxFpjq9sh+MzskfqRAcr1tN8QgE7gggnLep2w== X-Received: by 2002:a17:90a:714b:: with SMTP id g11mr3741642pjs.17.1586332646601; Wed, 08 Apr 2020 00:57:26 -0700 (PDT) Received: from hsinchu02.internal.sifive.com (114-34-229-221.HINET-IP.hinet.net. [114.34.229.221]) by smtp.gmail.com with ESMTPSA id d85sm485599pfd.157.2020.04.08.00.57.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Apr 2020 00:57:26 -0700 (PDT) From: Zong Li To: palmer@dabbelt.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, mhiramat@kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 9/9] riscv: add STRICT_KERNEL_RWX support Date: Wed, 8 Apr 2020 15:57:04 +0800 Message-Id: <100e739c5fd722a96fcc640c8ee0c82fe34fcb6a.1586332296.git.zong.li@sifive.com> X-Mailer: git-send-email 2.26.0 In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200408_005727_332797_EC408AB7 X-CRM114-Status: GOOD ( 10.82 ) X-Spam-Score: -0.2 (/) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-0.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:641 listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zong Li Sender: "linux-riscv" Errors-To: linux-riscv-bounces+patchwork-linux-riscv=patchwork.kernel.org@lists.infradead.org The commit contains that make text section as non-writable, rodata section as read-only, and data section as non-executable. The init section should be changed to non-executable. Signed-off-by: Zong Li Reviewed-by: Palmer Dabbelt --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/set_memory.h | 8 ++++++ arch/riscv/mm/init.c | 44 +++++++++++++++++++++++++++++ 3 files changed, 53 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 1e1efc998baf..58b556167d59 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -61,6 +61,7 @@ config RISCV select ARCH_HAS_GIGANTIC_PAGE select ARCH_HAS_SET_DIRECT_MAP select ARCH_HAS_SET_MEMORY + select ARCH_HAS_STRICT_KERNEL_RWX select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select SPARSEMEM_STATIC if 32BIT select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h index 4c5bae7ca01c..c38df4771c09 100644 --- a/arch/riscv/include/asm/set_memory.h +++ b/arch/riscv/include/asm/set_memory.h @@ -22,6 +22,14 @@ static inline int set_memory_x(unsigned long addr, int numpages) { return 0; } static inline int set_memory_nx(unsigned long addr, int numpages) { return 0; } #endif +#ifdef CONFIG_STRICT_KERNEL_RWX +void set_kernel_text_ro(void); +void set_kernel_text_rw(void); +#else +static inline void set_kernel_text_ro(void) { } +static inline void set_kernel_text_rw(void) { } +#endif + int set_direct_map_invalid_noflush(struct page *page); int set_direct_map_default_noflush(struct page *page); diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index fab855963c73..b55be44ff9bd 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -477,6 +478,17 @@ static void __init setup_vm_final(void) csr_write(CSR_SATP, PFN_DOWN(__pa_symbol(swapper_pg_dir)) | SATP_MODE); local_flush_tlb_all(); } + +void free_initmem(void) +{ + unsigned long init_begin = (unsigned long)__init_begin; + unsigned long init_end = (unsigned long)__init_end; + + /* Make the region as non-execuatble. */ + set_memory_nx(init_begin, (init_end - init_begin) >> PAGE_SHIFT); + free_initmem_default(POISON_FREE_INITMEM); +} + #else asmlinkage void __init setup_vm(uintptr_t dtb_pa) { @@ -488,6 +500,38 @@ static inline void setup_vm_final(void) } #endif /* CONFIG_MMU */ +#ifdef CONFIG_STRICT_KERNEL_RWX +void set_kernel_text_rw(void) +{ + unsigned long text_start = (unsigned long)_text; + unsigned long text_end = (unsigned long)_etext; + + set_memory_rw(text_start, (text_end - text_start) >> PAGE_SHIFT); +} + +void set_kernel_text_ro(void) +{ + unsigned long text_start = (unsigned long)_text; + unsigned long text_end = (unsigned long)_etext; + + set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT); +} + +void mark_rodata_ro(void) +{ + unsigned long text_start = (unsigned long)_text; + unsigned long text_end = (unsigned long)_etext; + unsigned long rodata_start = (unsigned long)__start_rodata; + unsigned long data_start = (unsigned long)_data; + unsigned long max_low = (unsigned long)(__va(PFN_PHYS(max_low_pfn))); + + set_memory_ro(text_start, (text_end - text_start) >> PAGE_SHIFT); + set_memory_ro(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT); + set_memory_nx(rodata_start, (data_start - rodata_start) >> PAGE_SHIFT); + set_memory_nx(data_start, (max_low - data_start) >> PAGE_SHIFT); +} +#endif + void __init paging_init(void) { setup_vm_final();