From patchwork Mon Dec 12 18:53:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 13071390 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADA15C4167B for ; Mon, 12 Dec 2022 18:54:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00DA68E0005; Mon, 12 Dec 2022 13:54:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EB16F8E0002; Mon, 12 Dec 2022 13:54:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB76D8E0005; Mon, 12 Dec 2022 13:54:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B5EE98E0002 for ; Mon, 12 Dec 2022 13:54:20 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 71435140CB1 for ; Mon, 12 Dec 2022 18:54:20 +0000 (UTC) X-FDA: 80234554680.04.6A872DE Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf30.hostedemail.com (Postfix) with ESMTP id CFC0080017 for ; Mon, 12 Dec 2022 18:54:18 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=kOiSXGdK; spf=pass (imf30.hostedemail.com: domain of "SRS0=Tsvw=4K=zx2c4.com=Jason@kernel.org" designates 145.40.68.75 as permitted sender) smtp.mailfrom="SRS0=Tsvw=4K=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670871259; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JDnIaEdxmFpwDp34HQjcaj4AkiYeEvsGKnBz84Pn62I=; b=GuPPTzfQVCvtsu1Jgl8zZR/bBSvY+FeEM2PStgqwDhUtwi5NoqgVxP7yoiYP1VRRiHqsyR gVL+qc7c+lDriksL2KeLVIcyTM+AXSvfTaAOw5ldRrxXKQTYHrkKSPehhJmlIjLswzmoib OQMWYS3HMVLRPc0fhpLI/gL6P4CgPuc= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=zx2c4.com header.s=20210105 header.b=kOiSXGdK; spf=pass (imf30.hostedemail.com: domain of "SRS0=Tsvw=4K=zx2c4.com=Jason@kernel.org" designates 145.40.68.75 as permitted sender) smtp.mailfrom="SRS0=Tsvw=4K=zx2c4.com=Jason@kernel.org"; dmarc=pass (policy=quarantine) header.from=zx2c4.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670871259; a=rsa-sha256; cv=none; b=GSD0AhJapBDRFYNj7ICkiIvRDGqirDEGbZQUIHEF0YEqm+Ln1vqAFFymq46cxIT6QCGES9 O66qHA+rTCwOG4ja/g4qLDsoOy0mtHHuSB3sxF5k1hFQsBdHO/PHK7MxhzyZt/dpu1Fg/r KjHyZYkfQOl0146yDYydK72oJCNLbNw= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 57DB0B80DE5; Mon, 12 Dec 2022 18:54:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 92CBDC433F0; Mon, 12 Dec 2022 18:54:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zx2c4.com; s=20210105; t=1670871253; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JDnIaEdxmFpwDp34HQjcaj4AkiYeEvsGKnBz84Pn62I=; b=kOiSXGdKAuVtQwPSnP+N4oa+AsFI5Vcf7q+FQraZSoGQa1/SmvKpJgbOdOi+DN39qNoy9l FreY9SMWteCbSsG8y/mam6UYgbtM+LthUwnRVRY3IL1FMiy+zIr24cFzt4OrFnp3cpihxT N+TcMfOBgYggh3tDP9wY6rW6z7O8JsI= Received: by mail.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id 4335dacd (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 12 Dec 2022 18:54:13 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org, patches@lists.linux.dev, tglx@linutronix.de Cc: "Jason A. Donenfeld" , linux-crypto@vger.kernel.org, linux-api@vger.kernel.org, x86@kernel.org, Greg Kroah-Hartman , Adhemerval Zanella Netto , Carlos O'Donell , Florian Weimer , Arnd Bergmann , Jann Horn , Christian Brauner , linux-mm@kvack.org Subject: [PATCH RFC v12 2/6] x86: mm: Skip faulting instruction for VM_DROPPABLE faults Date: Mon, 12 Dec 2022 11:53:43 -0700 Message-Id: <20221212185347.1286824-3-Jason@zx2c4.com> In-Reply-To: <20221212185347.1286824-1-Jason@zx2c4.com> References: <20221212185347.1286824-1-Jason@zx2c4.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CFC0080017 X-Stat-Signature: apja7z8bnbtw8xym9ftg7qjsc5ztqqri X-Rspam-User: X-HE-Tag: 1670871258-307193 X-HE-Meta: U2FsdGVkX1/G2EANtEN9Zn0K9kzwAMCzbadl1Ox+fGRYTO014+bbb6PKGyn9XlkhFdkVVZyvuaCi2PEMIYlwsAMh6tU8Sb5Iq0P8r+eg9SVkGBGMspfreEERgCGa6LgEuL5gogSFX0UVA5pIQmnn6kFq41JV63rOKXZFEqpQ+2RzJSaBTvweyEOfl0qFTanzQZY/IwcXbJA2nwm37fcZAop5JLBUnyA7SLzV2tPB5CyKalFJ2X6+59ta39uJ+n/gluwLnjZ+9UsUWLaX5ZtIXiTCe+CD/86/lmPgjyvGP6flvj6m2ZjSLERxMBlhP7x1XVfZZcpBlMNBvfobe+Ms105uxa0RWtEmVbwKbR6YHVL5UviZpkg0vXHT6W4cWNYxUa2znf3M7Z+h/nz4bp5mUH9YmLyrw2tZ2Qns09s2S3ctgjXW3Eoqdk61YTfgT2pxEPc5OQnVLzI73ATtQrjM6nVb5H81culJJQPJUbQLvgT0CxNT526geAJ4YlEn4W5jd5WVR1VLgIGniC5g04a++QIrZwjuOIHpJv1+u8Qh+4WYHyz+lZCtoHLSnv/aRDQ+jmpt1VyGcAKxJ3dWfg0MIPhjcY3oXTiYACx4V2066Vw5fcbe8FoD7NP3xKHUARtE1Ck92y1To8rzZZ0VqAJI7896E7eSge9ugTNgDGPhOXXebSiPuS/8fsOFYE5O4VBrxk8Bx+/FcmpdLxfMXBqp+b9uCnDKRu87RS/ik3HsqnX6WJPhpMgtY0SzC+HYeHQuF9nu4qesCEhd8VX6ss4RW5WUkMw/upcv58yPjLhN8Kf7Wyiyc7MB/F+WKQTtdmfhBguDlnkWDO+llyzq+GSk1LKDigpfbFB/Q59jTYFg9Wljjfxt1CtszKH2+ML7xxJWu+zGgDGlqzUCJTIgUxDBw3wyoQAXGEq8Ee2P4MtmHMKThMVJdbyPfPq9NPYIIF81/oXy0PRKBWURRGDUDIk 0f/hmXz5 bofm2+2iRdsUHd7bdFlCfre8aq+3lNjDYnEJnqtaCWhUZRguEF6Mvf0+uAcCXhththQRoOt2B2YlgpFweKiHsYH+r28HlndO3qkUmFuzmheqQvvyXcAQZFcq4aQ14/VPhWq/XFfrXCfvpD/qvhY5E399Bqgfqc4dBm7s4WQWp6nsyL9wg16yzB5ICjESfpf7kL9IBOMvsJvIfWkNaxpCLgJSJoQUfpAgBnwmiwtgQLdxKINpLWg0IsKmnPYekZJcIz0v+oEG1YRPlU/1+DU7wcUdVhTbO/Ug0C3KXPtzXhdWPbhY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The prior commit introduced VM_DROPPABLE, but in a limited form where the faulting instruction was retried instead of skipped. Finish that up with the platform-specific aspect of skipping the actual instruction. This works by copying userspace's %rip to a stack buffer of size MAX_INSN_SIZE, decoding it, and then adding the length of the decoded instruction to userspace's %rip. In the event any of these fail, just fallback to not advancing %rip and trying again. Cc: linux-mm@kvack.org Signed-off-by: Jason A. Donenfeld --- arch/x86/mm/fault.c | 19 +++++++++++++++++++ include/linux/mm_types.h | 5 ++++- mm/memory.c | 4 +++- 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 7b0d4ab894c8..e5328073f8e0 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -33,6 +33,8 @@ #include /* kvm_handle_async_pf */ #include /* fixup_vdso_exception() */ #include +#include /* insn_decode() */ +#include /* in_32bit_syscall() */ #define CREATE_TRACE_POINTS #include @@ -1454,6 +1456,23 @@ void do_user_addr_fault(struct pt_regs *regs, } mmap_read_unlock(mm); + + if (fault & VM_FAULT_SKIP_INSN) { + u8 insn_buf[MAX_INSN_SIZE]; + struct insn insn; + size_t len; + + len = sizeof(insn_buf) - copy_from_user(insn_buf, (void *)regs->ip, sizeof(insn_buf)); + if (!len) + return; + + if (insn_decode(&insn, insn_buf, len, in_32bit_syscall() ? INSN_MODE_32 : INSN_MODE_64) < 0) + return; + + regs->ip += insn.length; + return; + } + if (likely(!(fault & VM_FAULT_ERROR))) return; diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 500e536796ca..4def1051499b 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -861,6 +861,7 @@ typedef __bitwise unsigned int vm_fault_t; * fsync() to complete (for synchronous page faults * in DAX) * @VM_FAULT_COMPLETED: ->fault completed, meanwhile mmap lock released + * @VM_FAULT_SKIP_INSN: ->handle the fault by skipping faulting instruction * @VM_FAULT_HINDEX_MASK: mask HINDEX value * */ @@ -879,6 +880,7 @@ enum vm_fault_reason { VM_FAULT_DONE_COW = (__force vm_fault_t)0x001000, VM_FAULT_NEEDDSYNC = (__force vm_fault_t)0x002000, VM_FAULT_COMPLETED = (__force vm_fault_t)0x004000, + VM_FAULT_SKIP_INSN = (__force vm_fault_t)0x008000, VM_FAULT_HINDEX_MASK = (__force vm_fault_t)0x0f0000, }; @@ -903,7 +905,8 @@ enum vm_fault_reason { { VM_FAULT_RETRY, "RETRY" }, \ { VM_FAULT_FALLBACK, "FALLBACK" }, \ { VM_FAULT_DONE_COW, "DONE_COW" }, \ - { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" } + { VM_FAULT_NEEDDSYNC, "NEEDDSYNC" }, \ + { VM_FAULT_SKIP_INSN, "SKIP_INSN" } struct vm_special_mapping { const char *name; /* The name, e.g. "[vdso]". */ diff --git a/mm/memory.c b/mm/memory.c index 72403585e1a5..8834a7c1580f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5220,8 +5220,10 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, lru_gen_exit_fault(); /* If the mapping is droppable, then errors due to OOM aren't fatal. */ - if (vma->vm_flags & VM_DROPPABLE) + if ((ret & VM_FAULT_OOM) && (vma->vm_flags & VM_DROPPABLE)) { ret &= ~VM_FAULT_OOM; + ret |= VM_FAULT_SKIP_INSN; + } if (flags & FAULT_FLAG_USER) { mem_cgroup_exit_user_fault();