From patchwork Sat Feb 18 21:13:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13145639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED4F4C636CC for ; Sat, 18 Feb 2023 21:16:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D357B6B0072; Sat, 18 Feb 2023 16:16:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CE5586B0073; Sat, 18 Feb 2023 16:16:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5EB96B0074; Sat, 18 Feb 2023 16:16:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A63356B0072 for ; Sat, 18 Feb 2023 16:16:02 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6D8A91204CA for ; Sat, 18 Feb 2023 21:16:02 +0000 (UTC) X-FDA: 80481670164.29.DB33020 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf25.hostedemail.com (Postfix) with ESMTP id 89C88A000D for ; Sat, 18 Feb 2023 21:15:59 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=BydQXJvx; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf25.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676754960; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=6ag/KpHcjQtLoUDcA0B/Dksx5GjrpM5/YluaDoG8g1A=; b=ikb4j7QwqTQx7ap040QnBJXRo5bX52dOgDoEAIaaa3YMEfgVz+M+TAgamwqOD/XpNzJxw8 k1ydaLYVjxY3G+C2y+mAyj70qP+4psxLkAelq18iRwPdlAEeVbQ825pNXtA7CpJ8gq9+mq 4SYvOo2VHgYzhTe3kFLa6jh1wfe5ZjA= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=BydQXJvx; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf25.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676754960; a=rsa-sha256; cv=none; b=lWWKWgy3htNI8QBcnorLFD4q1q9z8ATjcBf1V9k1KOP3thICiZCynolQDand/E8GU7pN7o kwK5IjOucR9zapDpbfmxin+puGoOUrmFQ02V3vdAyqvTLAu6cDJLgYR1y47qU2cCt6I43k XF+ZJxXwAJ8PRXikygkctoPwjF5A99Y= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676754959; x=1708290959; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Eg+f7Jyu/wEAqAGui0h6Em+F7HIJYdbqfOZIHZ4d7ks=; b=BydQXJvxgE0OhOXqyGpRQTmSZOcNmEThAeTG+11TZ1PSG6/bnG6IxuAB UfyaNLbFTBy88+dNdv3qAgZBPjCySolzvqDrr21B82UQUEnKbyHt0Iaxo C7pziOQAx9x921tU8PeQi8GgVaGxj8oeIXg/3UIjWkyM4oBXG561hh8pu ol0X25YrxwipcPaEv1SxVaPYii5F1p9ABEv3iNpkB3x4RA1jSL5ZyDdJk fJnTZG6weYtwRxDTyaazVqfPJapNcK28SWwfL9kG2dEAd4MDiFf/GDuvg YNcB4PhnMbLcmu3/FsJjA2Bf2FJwyrndI5cvKOx7QWRNRdszP2O7qkDon Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10625"; a="418427079" X-IronPort-AV: E=Sophos;i="5.97,309,1669104000"; d="scan'208";a="418427079" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2023 13:15:57 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10625"; a="664241566" X-IronPort-AV: E=Sophos;i="5.97,309,1669104000"; d="scan'208";a="664241566" Received: from adityava-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.209.80.223]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2023 13:15:56 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com Cc: rick.p.edgecombe@intel.com Subject: [PATCH v6 00/41] Shadow stacks for userspace Date: Sat, 18 Feb 2023 13:13:52 -0800 Message-Id: <20230218211433.26859-1-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: 89C88A000D X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: mrsaceec7ck9ciokf1h7qs6gaoiizwjs X-HE-Tag: 1676754959-224589 X-HE-Meta: U2FsdGVkX18CpaK3MjWfpCazgSFVhQKebsbCdX+0pvcqDYNb8WP7GO7sT+6vHKs/458KEuWsLSOH8kc6TMjQg0DPxiMknS42MZ5FarlwrZrPKLfk9YpfffI2xnkzess8qwNjjd8B3b3p9DbM4uoYu1nr0c+Bkye8XyD4xJrGWf1nu4Tagyh/C6vmKPlO1tMLvpd2Jq4RFG9+Qff6Hj53ZVRXz3LxRAQqdEWkeUhVFW97znkMrBNwHX3lb81ewCp7OdQX2dydXZ+e/Nf3OYVF3o9p0dJgOY8nU0mmA2Yn6zcVwG9Aw8EsgRvGKSqBOLbeaM4P6Zx24n4QBdKPGFHgvRQdiB47UCPvifirZc9SndpfqWp3Pyr1bORRfK7k1yFs0IH5pt6EYX4p9aXJG0xAPSReUhk0GQAXNkfe1BN3oEwp6Br8u3kGNEjUOGPTonihbU9hVAaItV8Slw8a4nDWo93qANAXGJn7aNf6EYWs6fI/NHDkBLAaJUeRVy50FHwn5Ga6aIqaLrBOu75gW3bw54IST9Qo9iR+0KC8rzjF8eYDiA8HsAG6sWtVCEF/LIQaL+NsF0jeXJPiFk4CXB2haM53KjnEaiS41sZbqzlv3ycNQ4Yp+310dYDXLdrHNMY8vqm7CPxp/WxTbhsVN0b+5BT18pAVg6wrqJhAgIMpPe4atfQqmXM55v6tvjGL3+fe26AqcVxRGioixT0j14ExsXqr0NsC2SRW+MfzeeOBQnKNWLUB3xGx6sH5uX4cQRJIS7TOmfhM/LfAHKuKw2qNE4fyx+GfZk10jhM2GIk7qP5/EdkVRcsJ1Vi0/oBlumZKugvmkIqas3+7PKLEemaelh7cowGk6eA0NDbc0eccWua+HPmbA4KE0CiEFMvXnB/Zfy68EeXRIXi8dACsp90KwknVJEkawp4WJeE+DUSu/tCFkcKuDXIOhzwxJz+XG4RmyMsT7LYk621SE9eBnR4 9+o1pTzP LXLRdScaoMr0k2b0wJVKLWjJH86b0XQ6wppVmweegPfr2UUoVttaWUfHAZ7TPGBFdm3x8JhH4aHVV+ag2haU7ZJN1+K4ylHgczTuTlr8rQIId3fMay2AlyPFUfEROY90Yu0yDhqR416PpgxYX5WEQAJXb45zRa1Roek/cpxbYLYjeohNSfDi60T6pCggo+/jHrlQyulgvv7rskJd7t4btgamkXnG1i1oZPIvDeAB1nG2IFio26E7WKTSGTjeV280fTJ6yVn6fEoE4k5U6JZ5/NgPbdOOad0pc23lmPer9m7AqJlGIozl0ffmiew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, This series implements Shadow Stacks for userspace using x86's Control-flow Enforcement Technology (CET). CET consists of two related security features: shadow stacks and indirect branch tracking. This series implements just the shadow stack part of this feature, and just for userspace. The main use case for shadow stack is providing protection against return oriented programming attacks. It works by maintaining a secondary (shadow) stack using a special memory type that has protections against modification. When executing a CALL instruction, the processor pushes the return address to both the normal stack and to the special permission shadow stack. Upon RET, the processor pops the shadow stack copy and compares it to the normal stack copy. For more details, see the coverletter from v1 [0]. The main changes in this version are the MM suggestions by David Hildenbrand to have pte_mkwrite() take a vma, and rename _PAGE_COW. The former is split over three patches: mm: Introduce pte_mkwrite_kernel() s390/mm: Introduce pmd_mkwrite_kernel() mm: Make pte_mkwrite() take a VMA With these changes, and an adjustment to "mm: Warn on shadow stack memory in wrong vma", references to "shstk" are now only in x86 arch code, hopefully addressing Andrew Morton's concerns. There are still a couple VM_SHADOW_STACK references, which seems to be in keeping with the treatment of other VM_HIGH_ARCH flags. If other shadow stack implementations end up with identical logic, it can easily be refactored at that point. There was also some more feedback from Boris which was incorporated. I left tested-by tags in place per discussion with testers. Testers, please retest. Previous version [1]. Thanks, Rick [0] https://lore.kernel.org/lkml/20220130211838.8382-1-rick.p.edgecombe@intel.com/ [1] https://lore.kernel.org/lkml/20230119212317.8324-1-rick.p.edgecombe@intel.com/ Kirill A. Shutemov (1): x86: Introduce userspace API for shadow stack Mike Rapoport (1): x86/shstk: Add ARCH_SHSTK_UNLOCK Rick Edgecombe (19): x86/fpu: Add helper for modifying xstate x86: Move control protection handler to separate file mm: Introduce pte_mkwrite_kernel() s390/mm: Introduce pmd_mkwrite_kernel() mm: Make pte_mkwrite() take a VMA x86/mm: Introduce _PAGE_SAVED_DIRTY x86/mm: Start actually marking _PAGE_SAVED_DIRTY x86/mm: Teach pte_mkwrite() about stack memory mm: Don't allow write GUPs to shadow stack memory x86/mm: Introduce MAP_ABOVE4G mm: Warn on shadow stack memory in wrong vma x86/mm: Warn if create Write=0,Dirty=1 with raw prot x86/shstk: Introduce map_shadow_stack syscall x86/shstk: Support WRSS for userspace x86: Expose thread features in /proc/$PID/status x86/shstk: Wire in shadow stack interface selftests/x86: Add shadow stack test x86/fpu: Add helper for initing features x86/shstk: Add ARCH_SHSTK_STATUS Yu-cheng Yu (20): Documentation/x86: Add CET shadow stack description x86/shstk: Add Kconfig option for shadow stack x86/cpufeatures: Add CPU feature flags for shadow stacks x86/cpufeatures: Enable CET CR4 bit for shadow stack x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states x86/shstk: Add user control-protection fault handler x86/mm: Remove _PAGE_DIRTY from kernel RO pages x86/mm: Move pmd_write(), pud_write() up in the file x86/mm: Update ptep/pmdp_set_wrprotect() for _PAGE_SAVED_DIRTY mm: Move VM_UFFD_MINOR_BIT from 37 to 38 mm: Introduce VM_SHADOW_STACK for shadow stack memory x86/mm: Check shadow stack page fault errors mm: Add guard pages around a shadow stack. mm/mmap: Add shadow stack pages to memory accounting mm: Re-introduce vm_flags to do_mmap() x86/shstk: Add user-mode shadow stack support x86/shstk: Handle thread shadow stack x86/shstk: Introduce routines modifying shstk x86/shstk: Handle signals for shadow stack x86: Add PTRACE interface for shadow stack Documentation/filesystems/proc.rst | 1 + Documentation/mm/arch_pgtable_helpers.rst | 9 +- Documentation/x86/index.rst | 1 + Documentation/x86/shstk.rst | 176 +++++ arch/alpha/include/asm/pgtable.h | 6 +- arch/arc/include/asm/hugepage.h | 2 +- arch/arc/include/asm/pgtable-bits-arcv2.h | 7 +- arch/arm/include/asm/pgtable-3level.h | 7 +- arch/arm/include/asm/pgtable.h | 2 +- arch/arm/kernel/signal.c | 2 +- arch/arm64/include/asm/pgtable.h | 9 +- arch/arm64/kernel/signal.c | 2 +- arch/arm64/kernel/signal32.c | 2 +- arch/arm64/mm/trans_pgd.c | 4 +- arch/csky/include/asm/pgtable.h | 2 +- arch/hexagon/include/asm/pgtable.h | 2 +- arch/ia64/include/asm/pgtable.h | 2 +- arch/loongarch/include/asm/pgtable.h | 4 +- arch/m68k/include/asm/mcf_pgtable.h | 2 +- arch/m68k/include/asm/motorola_pgtable.h | 6 +- arch/m68k/include/asm/sun3_pgtable.h | 6 +- arch/microblaze/include/asm/pgtable.h | 2 +- arch/mips/include/asm/pgtable.h | 6 +- arch/nios2/include/asm/pgtable.h | 2 +- arch/openrisc/include/asm/pgtable.h | 2 +- arch/parisc/include/asm/pgtable.h | 6 +- arch/powerpc/include/asm/book3s/32/pgtable.h | 2 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 4 +- arch/powerpc/include/asm/nohash/32/pgtable.h | 2 +- arch/powerpc/include/asm/nohash/32/pte-8xx.h | 2 +- arch/powerpc/include/asm/nohash/64/pgtable.h | 2 +- arch/riscv/include/asm/pgtable.h | 6 +- arch/s390/include/asm/hugetlb.h | 4 +- arch/s390/include/asm/pgtable.h | 14 +- arch/s390/mm/pageattr.c | 4 +- arch/sh/include/asm/pgtable_32.h | 10 +- arch/sparc/include/asm/pgtable_32.h | 2 +- arch/sparc/include/asm/pgtable_64.h | 6 +- arch/sparc/kernel/signal32.c | 2 +- arch/sparc/kernel/signal_64.c | 2 +- arch/um/include/asm/pgtable.h | 2 +- arch/x86/Kconfig | 24 + arch/x86/Kconfig.assembler | 5 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/disabled-features.h | 16 +- arch/x86/include/asm/fpu/api.h | 9 + arch/x86/include/asm/fpu/regset.h | 7 +- arch/x86/include/asm/fpu/sched.h | 3 +- arch/x86/include/asm/fpu/types.h | 16 +- arch/x86/include/asm/fpu/xstate.h | 6 +- arch/x86/include/asm/idtentry.h | 2 +- arch/x86/include/asm/mmu_context.h | 2 + arch/x86/include/asm/msr.h | 11 + arch/x86/include/asm/pgtable.h | 322 ++++++++- arch/x86/include/asm/pgtable_types.h | 71 +- arch/x86/include/asm/processor.h | 8 + arch/x86/include/asm/shstk.h | 40 ++ arch/x86/include/asm/special_insns.h | 13 + arch/x86/include/asm/tlbflush.h | 3 +- arch/x86/include/asm/trap_pf.h | 2 + arch/x86/include/asm/traps.h | 12 + arch/x86/include/uapi/asm/mman.h | 4 + arch/x86/include/uapi/asm/prctl.h | 12 + arch/x86/kernel/Makefile | 4 + arch/x86/kernel/cet.c | 152 ++++ arch/x86/kernel/cpu/common.c | 35 +- arch/x86/kernel/cpu/cpuid-deps.c | 1 + arch/x86/kernel/cpu/proc.c | 23 + arch/x86/kernel/fpu/core.c | 59 +- arch/x86/kernel/fpu/regset.c | 86 +++ arch/x86/kernel/fpu/xstate.c | 148 ++-- arch/x86/kernel/fpu/xstate.h | 6 + arch/x86/kernel/idt.c | 2 +- arch/x86/kernel/process.c | 18 +- arch/x86/kernel/process_64.c | 9 +- arch/x86/kernel/ptrace.c | 12 + arch/x86/kernel/shstk.c | 491 +++++++++++++ arch/x86/kernel/signal.c | 1 + arch/x86/kernel/signal_32.c | 2 +- arch/x86/kernel/signal_64.c | 8 +- arch/x86/kernel/sys_x86_64.c | 6 +- arch/x86/kernel/traps.c | 87 --- arch/x86/mm/fault.c | 38 + arch/x86/mm/pat/set_memory.c | 4 +- arch/x86/mm/pgtable.c | 38 + arch/x86/xen/enlighten_pv.c | 2 +- arch/x86/xen/mmu_pv.c | 2 +- arch/x86/xen/xen-asm.S | 2 +- arch/xtensa/include/asm/pgtable.h | 2 +- fs/aio.c | 2 +- fs/proc/array.c | 6 + fs/proc/task_mmu.c | 3 + include/asm-generic/hugetlb.h | 4 +- include/linux/mm.h | 46 +- include/linux/mman.h | 4 + include/linux/pgtable.h | 14 + include/linux/proc_fs.h | 2 + include/linux/syscalls.h | 1 + include/uapi/asm-generic/siginfo.h | 3 +- include/uapi/asm-generic/unistd.h | 2 +- include/uapi/linux/elf.h | 2 + ipc/shm.c | 2 +- kernel/sys_ni.c | 1 + mm/debug_vm_pgtable.c | 16 +- mm/gup.c | 2 +- mm/huge_memory.c | 7 +- mm/hugetlb.c | 4 +- mm/memory.c | 5 +- mm/migrate_device.c | 2 +- mm/mmap.c | 12 +- mm/mprotect.c | 2 +- mm/nommu.c | 4 +- mm/userfaultfd.c | 2 +- mm/util.c | 2 +- tools/testing/selftests/x86/Makefile | 4 +- .../testing/selftests/x86/test_shadow_stack.c | 676 ++++++++++++++++++ 117 files changed, 2671 insertions(+), 324 deletions(-) create mode 100644 Documentation/x86/shstk.rst create mode 100644 arch/x86/include/asm/shstk.h create mode 100644 arch/x86/kernel/cet.c create mode 100644 arch/x86/kernel/shstk.c create mode 100644 tools/testing/selftests/x86/test_shadow_stack.c Tested-by: Kees Cook Acked-by: Mike Rapoport (IBM) Tested-by: John Allen