From patchwork Wed Oct 16 12:24:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 13838288 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 068FBD1AD50 for ; Wed, 16 Oct 2024 12:24:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E7066B007B; Wed, 16 Oct 2024 08:24:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 896676B0082; Wed, 16 Oct 2024 08:24:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 736346B0083; Wed, 16 Oct 2024 08:24:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 564DD6B007B for ; Wed, 16 Oct 2024 08:24:53 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DF4AD8020A for ; Wed, 16 Oct 2024 12:24:43 +0000 (UTC) X-FDA: 82679384172.24.0BF1E1D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf07.hostedemail.com (Postfix) with ESMTP id AE47040016 for ; Wed, 16 Oct 2024 12:24:38 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aWqOyxpU; spf=pass (imf07.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729081346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=o+PxARH1y+lh68rv/QHakJekt17WUXfUlamPRQ+Zru8=; b=nFO0g4P4li7jOdVC17Ultz1NsgrExjUhfmD1W1+SrmDdp1yQwbcXS67OP+D5gsG8YxWvhQ YSxbP5I/c4im65v5FmsmwKttGUbtbx8icAhpq+bcJ7HzUZAxwh9Y1gO97YUmCgUjBqXorN N5+UCjQgpXWQ7uHp6KRq1G9NbOKeJI0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729081346; a=rsa-sha256; cv=none; b=grdkBvwrfL1B5nwkjaQO5hZ4AhPqfrm9FbpCciKD5qEjxV9ZoIc/+FEBqXrAmTrdECbr4i rIiRtn5TdJBIkfuGlRUUsSNVwak8VRgJO8wsZKQpaqrs2mUqcrEmrseV7zcGsHSrkDzWL1 YI+MhyqvlrPyYz82VXsBWvkN3w17oRM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=aWqOyxpU; spf=pass (imf07.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0D7385C4079; Wed, 16 Oct 2024 12:24:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1679AC4CEC5; Wed, 16 Oct 2024 12:24:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729081489; bh=4N2yBcPU+CqIrBqJtLGRM1iXFl9m4h3NwwUOC60mErs=; h=From:To:Cc:Subject:Date:From; b=aWqOyxpUj1gfcj49U3mSIzH/yjlvaY2K6EaC5E1zyyr27ClwKNSE9g9ljhhOVHnRX yZsw0i7w3XASaQBNaWSV2ANJiXGaVfWXt0o/pSaMsAHu42ifXz822zdt7tbZzbtvdR 5kh2b563Wd172MmkXAd0utcPpZjjYIoT7sEEzlZu3URTzU0XAmpB2N+odQmdJ5wRrd xtZG1TTsWOtaPPlCmX39nPmTnHR5EiPfIWXv9Wpds2twF8IXWMnAfI3VF19bJxRlaL APqiM6ovq/Qy/PhgpW97mZHw9EXg5x26F+OIZuuZvRJuLmOks9xDajoQpYUqi4spdz ZE00LsHAnP9KQ== From: Mike Rapoport To: Andrew Morton , Luis Chamberlain Cc: Andreas Larsson , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Brian Cain , Catalin Marinas , Christoph Hellwig , Christophe Leroy , Dave Hansen , Dinh Nguyen , Geert Uytterhoeven , Guo Ren , Helge Deller , Huacai Chen , Ingo Molnar , Johannes Berg , John Paul Adrian Glaubitz , Kent Overstreet , "Liam R. Howlett" , Mark Rutland , Masami Hiramatsu , Matt Turner , Max Filippov , Michael Ellerman , Michal Simek , Mike Rapoport , Oleg Nesterov , Palmer Dabbelt , Peter Zijlstra , Richard Weinberger , Russell King , Song Liu , Stafford Horne , Steven Rostedt , Suren Baghdasaryan , Thomas Bogendoerfer , Thomas Gleixner , Uladzislau Rezki , Vineet Gupta , Will Deacon , bpf@vger.kernel.org, linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-sh@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-trace-kernel@vger.kernel.org, linux-um@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v6 0/8] x86/module: use large ROX pages for text allocations Date: Wed, 16 Oct 2024 15:24:16 +0300 Message-ID: <20241016122424.1655560-1-rppt@kernel.org> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Stat-Signature: 15aiy3dp6bk4rb75pqgejsabm1fmurzq X-Rspamd-Queue-Id: AE47040016 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1729081478-232130 X-HE-Meta: U2FsdGVkX19x2JP2btdQtpv3Wc8KPAWee+0hSZLG3q+hH3k2ZcMln9G86ektbD9X/peLTsJ3JVs55LWimh44PYU1k+hmddXizca4PU/H6v82Ezjn+/gLjPU5dG+PbB2Qe9rVVTuB3UcfCfjSwZSdICK6tpmWK3Vb1RrLEYfRX4O6MxqMdoarMzDenBKqSg2ExSee3b7LmmurkFtHjxj3LMHKlGRy8QMjrlbeutTWJ6ZWU2IgnXlKhr9qzsJy3xp9AD+fkSOs/ZKewKUXxwSj0Qy//lYS0D31S0P7unDb+QbPbWSvy5aJ1BvxMGO9IFz+Ee47z43eaPxcTV8oho1FnhnJz1HzmRuLu5yVrHVAcNiw8JjLne16Lpu/rUtwTkAX+YEn5QulFLuS/C+0spz1gH98F4V5jhyTA/44Yj0tiMmgfRs/yyJLxWvSrumozpxBRyXiRy6geYm8ffmtdu9q4d5iEFo0ko/fbUmAsct20ZgZnn1C/kamLVIAo1S8Koya1z7iaRoGlUOTkt1NV5LS9ofrJMLk82SPcsBpUbLbewsUITvOXpvOhQCUQWWmCLlCqrktknY1A+4ljofV5c6M/+lX7F05PKFh7VPMXrr9Dy8c6XBmcqUF6Vn68I+ISx5eyAWbGHeH5MbgjRpXQfQuIqn7DOQDyGyGf4Xf7yZMV6ksVn8ZEFadbh6gCaTpTdV8rT5onLTqe/7N0p7zQHcbJksHsTZQ68yYefia+k2VEZ+uiyh7kQPNta+bWXxuyw4r7tg18o62LdOad5fSI++8CQ+M5N4e5NR2nzQ7t6KKpXsuZ0eXBd4c5N+CCuwPp2z1TbN2WUeWelpvFRMXZSWh+s6l7uHZoHGhxvMWbtN63DgJ11C3OceuIFtj5/4I3QyUB+4IPKrr6DnwTggayEp4yoC4v47NPcSZdPShNFuPC2D+pxASc0hQmm1nvx3CnnHJuoX5zM0K6BpyGVDml8C wVPE7mVV fmswlJjqlciEbR+aasbcCtB0CnfSi2/IfPfMdqmw4LzeL+klKzwyBk2RP0oF+rMJfGqo0hVrloWdjqlPZYfxep/euOCJBOoowNxa//0rxa3mOHfFRlQV8zT5NnnPJcM4waYkL4zCERleoPi6+3wr95EiVaMv8mVqXAfhusce1Z+Ap/8CBbYagT4MbA9NmEhLQySEzy8lbp50R5EDQ8rYhla2SSf+RACbjZ/475CCt2jACCgUXcwW/wANFzYiPTtXXxcM4xc7cVtkaBuSSUFQCyes/NXuYtBTLhU+6JraQTt6NXbTpD4eJXBdjGQMWLYFsphS4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" Hi, This is an updated version of execmem ROX caches. Andrew, Luis, there is a conflict with Suren's "page allocation tag compression" patches: https://lore.kernel.org/all/20241014203646.1952505-1-surenb@google.com Probably taking this via mmotm would be more convenient. v5: https://lore.kernel.org/all/20241009180816.83591-1-rppt@kernel.org * Droped check for !area in mas_for_each() loop (Kees Bakker) * Droped externs in include/linux/vmalloc.h (Christoph) * Fixed handling of alternatives for CFI-enabled configs (Nathan) * Fixed interaction with kmemleak (Sergey). It looks like execmem and kmemleak interaction should be improved further, but it's out of scope of this series. * Added ARCH_HAS_EXECMEM_ROX configuration option to arch/Kconfig. The option serves two purposes: - make sure architecture that uses ROX caches implements execmem_fill_trapping_insns() callback (Christoph) - make sure entire physical memory is mapped in the direct map (Dave) v4: https://lore.kernel.org/all/20241007062858.44248-1-rppt@kernel.org * Fix copy/paste error in looongarch (Huacai) v3: https://lore.kernel.org/all/20240909064730.3290724-1-rppt@kernel.org * Drop ftrace_swap_func(). It is not needed because mcount array lives in a data section (Peter) * Update maple_tree usage (Liam) * Set ->fill_trapping_insns pointer on init (Ard) * Instead of using VM_FLUSH_RESET_PERMS for execmem cache, completely remove it from the direct map v2: https://lore.kernel.org/all/20240826065532.2618273-1-rppt@kernel.org * add comment why ftrace_swap_func() is needed (Steve) Since RFC: https://lore.kernel.org/all/20240411160526.2093408-1-rppt@kernel.org * update changelog about HUGE_VMAP allocations (Christophe) * move module_writable_address() from x86 to modules core (Ingo) * rename execmem_invalidate() to execmem_fill_trapping_insns() (Peter) * call alternatives_smp_unlock() after module text in-place is up to date (Nadav) = Original cover letter = These patches add support for using large ROX pages for allocations of executable memory on x86. They address Andy's comments [1] about having executable mappings for code that was not completely formed. The approach taken is to allocate ROX memory along with writable but not executable memory and use the writable copy to perform relocations and alternatives patching. After the module text gets into its final shape, the contents of the writable memory is copied into the actual ROX location using text poking. The allocations of the ROX memory use vmalloc(VMAP_ALLOW_HUGE_MAP) to allocate PMD aligned memory, fill that memory with invalid instructions and in the end remap it as ROX. Portions of these large pages are handed out to execmem_alloc() callers without any changes to the permissions. When the memory is freed with execmem_free() it is invalidated again so that it won't contain stale instructions. The module memory allocation, x86 code dealing with relocations and alternatives patching take into account the existence of the two copies, the writable memory and the ROX memory at the actual allocated virtual address. The patches are available at git: https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=execmem/x86-rox/v6 [1] https://lore.kernel.org/all/a17c65c6-863f-4026-9c6f-a04b659e9ab4@app.fastmail.com Mike Rapoport (Microsoft) (8): mm: vmalloc: group declarations depending on CONFIG_MMU together mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations asm-generic: introduce text-patching.h module: prepare to handle ROX allocations for text arch: introduce set_direct_map_valid_noflush() x86/module: prepare module loading for ROX allocations of text execmem: add support for cache of large ROX pages x86/module: enable ROX caches for module text on 64 bit arch/Kconfig | 8 + arch/alpha/include/asm/Kbuild | 1 + arch/arc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/arm/kernel/ftrace.c | 2 +- arch/arm/kernel/jump_label.c | 2 +- arch/arm/kernel/kgdb.c | 2 +- arch/arm/kernel/patch.c | 2 +- arch/arm/probes/kprobes/core.c | 2 +- arch/arm/probes/kprobes/opt-arm.c | 2 +- arch/arm64/include/asm/set_memory.h | 1 + .../asm/{patching.h => text-patching.h} | 0 arch/arm64/kernel/ftrace.c | 2 +- arch/arm64/kernel/jump_label.c | 2 +- arch/arm64/kernel/kgdb.c | 2 +- arch/arm64/kernel/patching.c | 2 +- arch/arm64/kernel/probes/kprobes.c | 2 +- arch/arm64/kernel/traps.c | 2 +- arch/arm64/mm/pageattr.c | 10 + arch/arm64/net/bpf_jit_comp.c | 2 +- arch/csky/include/asm/Kbuild | 1 + arch/hexagon/include/asm/Kbuild | 1 + arch/loongarch/include/asm/Kbuild | 1 + arch/loongarch/include/asm/set_memory.h | 1 + arch/loongarch/mm/pageattr.c | 19 + arch/m68k/include/asm/Kbuild | 1 + arch/microblaze/include/asm/Kbuild | 1 + arch/mips/include/asm/Kbuild | 1 + arch/nios2/include/asm/Kbuild | 1 + arch/openrisc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/parisc/kernel/ftrace.c | 2 +- arch/parisc/kernel/jump_label.c | 2 +- arch/parisc/kernel/kgdb.c | 2 +- arch/parisc/kernel/kprobes.c | 2 +- arch/parisc/kernel/patch.c | 2 +- arch/powerpc/include/asm/kprobes.h | 2 +- .../asm/{code-patching.h => text-patching.h} | 0 arch/powerpc/kernel/crash_dump.c | 2 +- arch/powerpc/kernel/epapr_paravirt.c | 2 +- arch/powerpc/kernel/jump_label.c | 2 +- arch/powerpc/kernel/kgdb.c | 2 +- arch/powerpc/kernel/kprobes.c | 2 +- arch/powerpc/kernel/module_32.c | 2 +- arch/powerpc/kernel/module_64.c | 2 +- arch/powerpc/kernel/optprobes.c | 2 +- arch/powerpc/kernel/process.c | 2 +- arch/powerpc/kernel/security.c | 2 +- arch/powerpc/kernel/setup_32.c | 2 +- arch/powerpc/kernel/setup_64.c | 2 +- arch/powerpc/kernel/static_call.c | 2 +- arch/powerpc/kernel/trace/ftrace.c | 2 +- arch/powerpc/kernel/trace/ftrace_64_pg.c | 2 +- arch/powerpc/lib/code-patching.c | 2 +- arch/powerpc/lib/feature-fixups.c | 2 +- arch/powerpc/lib/test-code-patching.c | 2 +- arch/powerpc/lib/test_emulate_step.c | 2 +- arch/powerpc/mm/book3s32/mmu.c | 2 +- arch/powerpc/mm/book3s64/hash_utils.c | 2 +- arch/powerpc/mm/book3s64/slb.c | 2 +- arch/powerpc/mm/kasan/init_32.c | 2 +- arch/powerpc/mm/mem.c | 2 +- arch/powerpc/mm/nohash/44x.c | 2 +- arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +- arch/powerpc/mm/nohash/tlb.c | 2 +- arch/powerpc/mm/nohash/tlb_64e.c | 2 +- arch/powerpc/net/bpf_jit_comp.c | 2 +- arch/powerpc/perf/8xx-pmu.c | 2 +- arch/powerpc/perf/core-book3s.c | 2 +- arch/powerpc/platforms/85xx/smp.c | 2 +- arch/powerpc/platforms/86xx/mpc86xx_smp.c | 2 +- arch/powerpc/platforms/cell/smp.c | 2 +- arch/powerpc/platforms/powermac/smp.c | 2 +- arch/powerpc/platforms/powernv/idle.c | 2 +- arch/powerpc/platforms/powernv/smp.c | 2 +- arch/powerpc/platforms/pseries/smp.c | 2 +- arch/powerpc/xmon/xmon.c | 2 +- arch/riscv/errata/andes/errata.c | 2 +- arch/riscv/errata/sifive/errata.c | 2 +- arch/riscv/errata/thead/errata.c | 2 +- arch/riscv/include/asm/set_memory.h | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/riscv/include/asm/uprobes.h | 2 +- arch/riscv/kernel/alternative.c | 2 +- arch/riscv/kernel/cpufeature.c | 3 +- arch/riscv/kernel/ftrace.c | 2 +- arch/riscv/kernel/jump_label.c | 2 +- arch/riscv/kernel/patch.c | 2 +- arch/riscv/kernel/probes/kprobes.c | 2 +- arch/riscv/mm/pageattr.c | 15 + arch/riscv/net/bpf_jit_comp64.c | 2 +- arch/riscv/net/bpf_jit_core.c | 2 +- arch/s390/include/asm/set_memory.h | 1 + arch/s390/mm/pageattr.c | 11 + arch/sh/include/asm/Kbuild | 1 + arch/sparc/include/asm/Kbuild | 1 + arch/um/kernel/um_arch.c | 16 +- arch/x86/Kconfig | 1 + arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +- arch/x86/include/asm/set_memory.h | 1 + arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 175 +++++---- arch/x86/kernel/ftrace.c | 30 +- arch/x86/kernel/module.c | 45 ++- arch/x86/mm/init.c | 37 +- arch/x86/mm/pat/set_memory.c | 8 + arch/xtensa/include/asm/Kbuild | 1 + include/asm-generic/text-patching.h | 5 + include/linux/execmem.h | 37 ++ include/linux/module.h | 9 + include/linux/moduleloader.h | 4 + include/linux/set_memory.h | 6 + include/linux/text-patching.h | 15 + include/linux/vmalloc.h | 60 ++-- kernel/module/debug_kmemleak.c | 3 +- kernel/module/main.c | 77 +++- kernel/module/strict_rwx.c | 3 + mm/execmem.c | 336 +++++++++++++++++- mm/internal.h | 1 + mm/vmalloc.c | 14 +- 121 files changed, 878 insertions(+), 244 deletions(-) rename arch/arm/include/asm/{patch.h => text-patching.h} (100%) rename arch/arm64/include/asm/{patching.h => text-patching.h} (100%) rename arch/parisc/include/asm/{patch.h => text-patching.h} (100%) rename arch/powerpc/include/asm/{code-patching.h => text-patching.h} (100%) rename arch/riscv/include/asm/{patch.h => text-patching.h} (100%) create mode 100644 include/asm-generic/text-patching.h create mode 100644 include/linux/text-patching.h base-commit: 9852d85ec9d492ebef56dc5f229416c925758edc