From patchwork Mon Oct 7 06:28:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 13824103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 172B2CFB442 for ; Mon, 7 Oct 2024 06:29:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B5F36B00F9; Mon, 7 Oct 2024 02:29:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C70B6B00FA; Mon, 7 Oct 2024 02:29:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6402D6B00FB; Mon, 7 Oct 2024 02:29:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3DAED6B00F9 for ; Mon, 7 Oct 2024 02:29:27 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EC233ABC17 for ; Mon, 7 Oct 2024 06:29:26 +0000 (UTC) X-FDA: 82645829532.19.8D92B0F Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf20.hostedemail.com (Postfix) with ESMTP id 5EA4D1C0012 for ; Mon, 7 Oct 2024 06:29:24 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mAvpd3li; spf=pass (imf20.hostedemail.com: domain of rppt@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728282496; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=dmIgbpb6z/o+zx5bxBhj9UIRV4gMhFqgOuFTs+9FqXw=; b=3D3etrGjZs6YbcU3xSxZT0KvwiZ+A+I1RGZXj3gHilSAIk/D3TY50FIFzLIFbo9SYvtJUC EoSGER0rIWto9DbVy/lJyM89SI63DCTZ/86Hk5SkbjLtcsEWya4fupvg/vTfn4qa9OzzmT JaTeeqQc0FXpQFWiBvPnwcs4hv7cZts= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mAvpd3li; spf=pass (imf20.hostedemail.com: domain of rppt@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728282496; a=rsa-sha256; cv=none; b=fN6UAwMN4pMv00yTdVVAuIihtBrspLK6DM3CKekH8Fa67o5bZOBqlmVcyt8JWfh9iSpjQD 6u08YQdwOWcmf3N+OXfWFWum6mYyDuKjoG0j3ulANb59o+V5IzEvPMh55juG+n3g/u+0sr qjtcZEHoW91OMnJ4BFVW29HRaM30pUE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id A8B86A41431; Mon, 7 Oct 2024 06:29:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3B94C4CEC6; Mon, 7 Oct 2024 06:29:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728282562; bh=GYz9jJeJwdTzMuunM08gf0B0slcAjxJN/5ca8sCoryM=; h=From:To:Cc:Subject:Date:From; b=mAvpd3li61HdTUdxFJIe7eyvRfePLEvKecXQdexCLVpbBEa8HxHGH4u3IK8jVju4x ZXgoWx2/G65gU8/jWAV0QcRmBNwVxuebK3LnO7N6jsJxAiKtM3WlATwassj9ysIxRf JMkkeWiqetONDupxedDcqAYSK6K+hWFncl2o2LE1/LXb6kyCjzd2wzXgw7Yk9EJGAz bMWr/AwedePV7iVNtLs5+ErNIgVz90TWToU4sfjPGxTZUPTZwiKvYcEPtVhBTKYZL0 nHAhBcqyxEcAbP8TanB1vMyYdueNcpnycESXgCpWvBxKeH9MKkSWS4f/IP5DbcrXTz xYawM/9Xr6WPA== From: Mike Rapoport To: Andrew Morton Cc: Andreas Larsson , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Brian Cain , Catalin Marinas , Christoph Hellwig , Christophe Leroy , Dave Hansen , Dinh Nguyen , Geert Uytterhoeven , Guo Ren , Helge Deller , Huacai Chen , Ingo Molnar , Johannes Berg , John Paul Adrian Glaubitz , Kent Overstreet , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Matt Turner , Max Filippov , Michael Ellerman , Michal Simek , Mike Rapoport , Oleg Nesterov , Palmer Dabbelt , Peter Zijlstra , Richard Weinberger , Russell King , Song Liu , Stafford Horne , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Uladzislau Rezki , Vineet Gupta , Will Deacon , bpf@vger.kernel.org, linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-sh@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-trace-kernel@vger.kernel.org, linux-um@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v4 0/8] x86/module: use large ROX pages for text allocations Date: Mon, 7 Oct 2024 09:28:50 +0300 Message-ID: <20241007062858.44248-1-rppt@kernel.org> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 5EA4D1C0012 X-Stat-Signature: xzmsaw19uccyypjdtu6s5kjosc7s6ggk X-HE-Tag: 1728282564-47564 X-HE-Meta: U2FsdGVkX1+22w8gpSbts4EMLYJ7UGSaZXzEYsUPQQOF6v5c1vBKkMfA2QPH8MVFDsTnChctcTrYX1tgEJfs1AmhoAMu8cb926+sV7S+viduToIgJs+5gZr+mnFZAGXH1oWDO1oD78oB2TEZWeHnZaB91sPbCTQqVvV+90GSLD9/3Gn2vlu5hhL8YPONb3qaeMcFjH4e+i9jEE/qO4KwxHnfh5X2wFomDv60Tl8Z7mSJScmvvtyxIMgyOr2f/fIVuil6XjxNjMJBOPw5zE+42xVW9PCXsRVONq0Vghk3IZ1KIL6eHAZq4VlPL9kLhrbzbWsASLXZtxmLdRpKPZuORwnEmA9AFIR6WKJi47fbNfVZnwqG52T/ben0+zTg9MB0pJ0DxE2/7i0zWvZAl5Od1lS7+JfkkkzI1CJUCInQI76Sz+pjqI1CjU15TIbkqfcy2ClNRhlPnK+x4aXoy7mWQ3ioHEtqzDhs5gV5XeoOC7+Oo4cTu/pMWW6xsVIcnaYOwpUbSu9rd74ke3dnK28+CvFvru5/mgG8fq8I75SEMVOpVa0/wCoxnrhSjD0WGY15ae159kqkgqnNzeiIoPnk0Cinx8MNRxbVim2XcsLTOjfqT1/KitgGcYyAzjCo3QtdQTIxT05lcGzVJFVINSq0irFqwknSG5DnpINNiwOTNQpa4D2r5hyo657nauutST3ZJAzVHkKqcKV3faZM2SjMWV0n0B92XlfrZNQF2HjDPULEcbRVTKBgnYM+iYhwfkv8FXvO/J5AYLoEu0nq+HWZJ7fBAtXuYNnIeRF8BqMK5l65vymOpry1JfKkXMP7bQz8rGVez+fUVid6qoYZyyjXBslTQwgy9r6sKBiGHJH9VZQQ9S+6spDlnxbf9qLACrQQhFPrfcm8WLC40pjq2xqEb2E25fFa+Cly9NJ/MfV6b+uLpVOPi1v+0BN/CB6raSb62kP0Cv13ZwCsT3EF4qr IrhQhazR DraAwpiAovxWcbiPTVXRq1MeqX7skW0uip6otLUTvb5jsGhwZxah7qntdZb8SD92L3LFTJQ3wLBsNJWXyZyVLj52ZApdqJNhOY8byKnDxVjCNNSEotS50FBRiUu14TV3kuBnLvBrATgZvOmXzWmHZaPxSZM7uCK3+Nv62D57G29DLMTwA77PGmv8o59yDdWyYOP4MFsj8smlDDDzfBXbCyt8vlTJhzMn0C8AkPiDmoRFsS/npT0bz23OU9pprPer5a1mncWIwIEBEwgLCwoGHnxg1K3s/udLAv+af X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" Hi, These patches add support for using large ROX pages for allocations of executable memory on x86. They address Andy's comments [1] about having executable mappings for code that was not completely formed. The approach taken is to allocate ROX memory along with writable but not executable memory and use the writable copy to perform relocations and alternatives patching. After the module text gets into its final shape, the contents of the writable memory is copied into the actual ROX location using text poking. The allocations of the ROX memory use vmalloc(VMAP_ALLOW_HUGE_MAP) to allocate PMD aligned memory, fill that memory with invalid instructions and in the end remap it as ROX. Portions of these large pages are handed out to execmem_alloc() callers without any changes to the permissions. When the memory is freed with execmem_free() it is invalidated again so that it won't contain stale instructions. The module memory allocation, x86 code dealing with relocations and alternatives patching take into account the existence of the two copies, the writable memory and the ROX memory at the actual allocated virtual address. The patches are available at git: https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=execmem/x86-rox/v2 [1] https://lore.kernel.org/all/a17c65c6-863f-4026-9c6f-a04b659e9ab4@app.fastmail.com v3: https://lore.kernel.org/all/20240909064730.3290724-1-rppt@kernel.org * Drop ftrace_swap_func(). It is not needed because mcount array lives in a data section (Peter) * Update maple_tree usage (Liam) * Set ->fill_trapping_insns pointer on init (Ard) * Instead of using VM_FLUSH_RESET_PERMS for execmem cache, completely remove it from the direct map v2: https://lore.kernel.org/all/20240826065532.2618273-1-rppt@kernel.org * add comment why ftrace_swap_func() is needed (Steve) Since RFC: https://lore.kernel.org/all/20240411160526.2093408-1-rppt@kernel.org * update changelog about HUGE_VMAP allocations (Christophe) * move module_writable_address() from x86 to modules core (Ingo) * rename execmem_invalidate() to execmem_fill_trapping_insns() (Peter) * call alternatives_smp_unlock() after module text in-place is up to date (Nadav) Mike Rapoport (Microsoft) (8): mm: vmalloc: group declarations depending on CONFIG_MMU together mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations asm-generic: introduce text-patching.h module: prepare to handle ROX allocations for text arch: introduce set_direct_map_valid_noflush() x86/module: perpare module loading for ROX allocations of text execmem: add support for cache of large ROX pages x86/module: enable ROX caches for module text arch/alpha/include/asm/Kbuild | 1 + arch/arc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/arm/kernel/ftrace.c | 2 +- arch/arm/kernel/jump_label.c | 2 +- arch/arm/kernel/kgdb.c | 2 +- arch/arm/kernel/patch.c | 2 +- arch/arm/probes/kprobes/core.c | 2 +- arch/arm/probes/kprobes/opt-arm.c | 2 +- arch/arm64/include/asm/set_memory.h | 1 + .../asm/{patching.h => text-patching.h} | 0 arch/arm64/kernel/ftrace.c | 2 +- arch/arm64/kernel/jump_label.c | 2 +- arch/arm64/kernel/kgdb.c | 2 +- arch/arm64/kernel/patching.c | 2 +- arch/arm64/kernel/probes/kprobes.c | 2 +- arch/arm64/kernel/traps.c | 2 +- arch/arm64/mm/pageattr.c | 10 + arch/arm64/net/bpf_jit_comp.c | 2 +- arch/csky/include/asm/Kbuild | 1 + arch/hexagon/include/asm/Kbuild | 1 + arch/loongarch/include/asm/Kbuild | 1 + arch/loongarch/include/asm/set_memory.h | 1 + arch/loongarch/mm/pageattr.c | 21 ++ arch/m68k/include/asm/Kbuild | 1 + arch/microblaze/include/asm/Kbuild | 1 + arch/mips/include/asm/Kbuild | 1 + arch/nios2/include/asm/Kbuild | 1 + arch/openrisc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/parisc/kernel/ftrace.c | 2 +- arch/parisc/kernel/jump_label.c | 2 +- arch/parisc/kernel/kgdb.c | 2 +- arch/parisc/kernel/kprobes.c | 2 +- arch/parisc/kernel/patch.c | 2 +- arch/powerpc/include/asm/kprobes.h | 2 +- .../asm/{code-patching.h => text-patching.h} | 0 arch/powerpc/kernel/crash_dump.c | 2 +- arch/powerpc/kernel/epapr_paravirt.c | 2 +- arch/powerpc/kernel/jump_label.c | 2 +- arch/powerpc/kernel/kgdb.c | 2 +- arch/powerpc/kernel/kprobes.c | 2 +- arch/powerpc/kernel/module_32.c | 2 +- arch/powerpc/kernel/module_64.c | 2 +- arch/powerpc/kernel/optprobes.c | 2 +- arch/powerpc/kernel/process.c | 2 +- arch/powerpc/kernel/security.c | 2 +- arch/powerpc/kernel/setup_32.c | 2 +- arch/powerpc/kernel/setup_64.c | 2 +- arch/powerpc/kernel/static_call.c | 2 +- arch/powerpc/kernel/trace/ftrace.c | 2 +- arch/powerpc/kernel/trace/ftrace_64_pg.c | 2 +- arch/powerpc/lib/code-patching.c | 2 +- arch/powerpc/lib/feature-fixups.c | 2 +- arch/powerpc/lib/test-code-patching.c | 2 +- arch/powerpc/lib/test_emulate_step.c | 2 +- arch/powerpc/mm/book3s32/mmu.c | 2 +- arch/powerpc/mm/book3s64/hash_utils.c | 2 +- arch/powerpc/mm/book3s64/slb.c | 2 +- arch/powerpc/mm/kasan/init_32.c | 2 +- arch/powerpc/mm/mem.c | 2 +- arch/powerpc/mm/nohash/44x.c | 2 +- arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +- arch/powerpc/mm/nohash/tlb.c | 2 +- arch/powerpc/mm/nohash/tlb_64e.c | 2 +- arch/powerpc/net/bpf_jit_comp.c | 2 +- arch/powerpc/perf/8xx-pmu.c | 2 +- arch/powerpc/perf/core-book3s.c | 2 +- arch/powerpc/platforms/85xx/smp.c | 2 +- arch/powerpc/platforms/86xx/mpc86xx_smp.c | 2 +- arch/powerpc/platforms/cell/smp.c | 2 +- arch/powerpc/platforms/powermac/smp.c | 2 +- arch/powerpc/platforms/powernv/idle.c | 2 +- arch/powerpc/platforms/powernv/smp.c | 2 +- arch/powerpc/platforms/pseries/smp.c | 2 +- arch/powerpc/xmon/xmon.c | 2 +- arch/riscv/errata/andes/errata.c | 2 +- arch/riscv/errata/sifive/errata.c | 2 +- arch/riscv/errata/thead/errata.c | 2 +- arch/riscv/include/asm/set_memory.h | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/riscv/include/asm/uprobes.h | 2 +- arch/riscv/kernel/alternative.c | 2 +- arch/riscv/kernel/cpufeature.c | 3 +- arch/riscv/kernel/ftrace.c | 2 +- arch/riscv/kernel/jump_label.c | 2 +- arch/riscv/kernel/patch.c | 2 +- arch/riscv/kernel/probes/kprobes.c | 2 +- arch/riscv/mm/pageattr.c | 15 + arch/riscv/net/bpf_jit_comp64.c | 2 +- arch/riscv/net/bpf_jit_core.c | 2 +- arch/s390/include/asm/set_memory.h | 1 + arch/s390/mm/pageattr.c | 11 + arch/sh/include/asm/Kbuild | 1 + arch/sparc/include/asm/Kbuild | 1 + arch/um/kernel/um_arch.c | 16 +- arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +- arch/x86/include/asm/set_memory.h | 1 + arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 160 +++++---- arch/x86/kernel/ftrace.c | 30 +- arch/x86/kernel/module.c | 45 ++- arch/x86/mm/init.c | 26 +- arch/x86/mm/pat/set_memory.c | 8 + arch/xtensa/include/asm/Kbuild | 1 + include/asm-generic/text-patching.h | 5 + include/linux/execmem.h | 25 ++ include/linux/module.h | 9 + include/linux/moduleloader.h | 4 + include/linux/set_memory.h | 6 + include/linux/text-patching.h | 15 + include/linux/vmalloc.h | 60 ++-- kernel/module/main.c | 77 +++- kernel/module/strict_rwx.c | 3 + mm/execmem.c | 328 +++++++++++++++++- mm/internal.h | 1 + mm/vmalloc.c | 14 +- 118 files changed, 831 insertions(+), 235 deletions(-) rename arch/arm/include/asm/{patch.h => text-patching.h} (100%) rename arch/arm64/include/asm/{patching.h => text-patching.h} (100%) rename arch/parisc/include/asm/{patch.h => text-patching.h} (100%) rename arch/powerpc/include/asm/{code-patching.h => text-patching.h} (100%) rename arch/riscv/include/asm/{patch.h => text-patching.h} (100%) create mode 100644 include/asm-generic/text-patching.h create mode 100644 include/linux/text-patching.h base-commit: 9852d85ec9d492ebef56dc5f229416c925758edc