From patchwork Thu Jul 13 09:53:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41205EB64DD for ; Thu, 13 Jul 2023 09:40:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E7488D0003; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 997578D0001; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7EB0C8D0003; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 643FB8D0001 for ; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3C08414015A for ; Thu, 13 Jul 2023 09:40:31 +0000 (UTC) X-FDA: 81006093462.06.003AA74 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf11.hostedemail.com (Postfix) with ESMTP id 9A8504000F for ; Thu, 13 Jul 2023 09:40:28 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241229; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=apbewmKGP8blQ1CiiHysDpzU5DYCinKGt5o+N+K+bcg=; b=IsTZollCZDtmqF6/96HkGod55MgPaFjgqWN42OtYjdEDV0tUJiHqrU8TkHe1LYuhMFlp76 4sKfe/FUP5GSlK8/FMllCDhS7yO3NIb/DeIs1/KUQJrmX1oCyTG3pJ6bzcdx8zIwaiytha /dbbS4q3AcDl+vjQJTN88ei3akBb7dQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241229; a=rsa-sha256; cv=none; b=nld+LiT+IVHGelapUQCRrw1G0Q0z+21430QKa0DDS127P8WPOYc2w++CeWo+LcRjX5+ISn y6hpX5+ITc3sq4vBZ5W3w2r/nLxHHyvzG94aF0GLXOEx4geJxsuP8YUnX6V57H3K0Ehmbl 5x6lYeJFL3e8x5Jl6daUKZQ95ws8BSc= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4R1qMR24J5zrRlb; Thu, 13 Jul 2023 17:39:47 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:22 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 01/10] mm: add a generic VMA lock-based page fault handler Date: Thu, 13 Jul 2023 17:53:29 +0800 Message-ID: <20230713095339.189715-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 9A8504000F X-Rspam-User: X-Stat-Signature: awiq693g3hbykgafaotq98ohot31hn43 X-Rspamd-Server: rspam03 X-HE-Tag: 1689241228-884174 X-HE-Meta: U2FsdGVkX19814nneLRSo0b+jPhA/b1sJ0BMerEJ73vKoaKSWWJ5Wjavr0417hgirbTXaHcIMCE9QXJ1S+6gRmEoQLIWOzJwNwiRGSyox13raJKMcagQ3EufOgKlYZQa7YI4FRA6QqNMy38eW9y7t2Yj33VG/fMuDkCM62iRPr0Z/Kun303CcjgzklaeMu5jYulXnfeIK1yOBu8U7WLQb/4UUTvMmD90tzjl0p/3Qw8NoV9njCO05+THqd1vo6evIpe4Eb7viUAxsnaF/yQ7Y1XR/udfVBLoBn9DswfM96onua+OfcOVkZryD9/qRUvqDi0iNhnWCWx6XqaaTrgYNY9rqF4ZgA/u/f+8k5nugizmZ2SRcoVMh6YD1j7KaJraA2GVH52/Kwd6jWhrKFKnJDg28OokcMLlkof8lYjA4vuBNo/TJSgWcEcA6HylVVe6qmE8AFRa5gH0A1fhqaO7YvaBNGoiIvY8X/CpGvgW6k/LGt5KQIqY2XDa7ka0fKCyP9M9oW/JR1zfjK++zAKkTFT/usgLfEOIPVYdVaOX8dHQQVArPfzUp1XNSpNYTzONklbZGBkwdVVIC35MkpMuVmitURhlbC5qxUlMk5D3bLqLnKU8aWTqp4a6DERs5nhWRiZwjm+lFbGSSZ40kraizraEdF6Babh4bqGcdUyq6Ee27xuCElv8B/8vQ68/IE7ZXI2fE6ywxLXpOk5va7j2UF/Rx8/ZvYFx1hgKqvN/fRjKZyhW2Y0G9YTvSQ+KZW2DcXD+jk8Kgme2Oj8bZLMAgbph8SHc0l9aaClcuUarPNJGUZwMa0ROCMkqq7L8itE9skuZxR7vYPfqKgvUCbtzAFmpb8mcYrde4/+/0TuVbic+t8F4bA+OK7kbKdqbkRlERl9GyXNY/qkhSucMkkv+6Xfb7EDXWep3Uvutb8IUynAePe6JGKXrBIyB6UkgSWP6eUbqLsB096RwxHfevcz 7NF/oUix +ZX9J5LIsu0tI/MWchqBgtCdJIpP/dkuk06UcuV7S94S5gCGu0WunfTWZDBJK+lsM7goky9FDcLVgq2y63Ov1iAoKEqLQPw375CrYNxM9uO57mixRE8LpNwVpAdEiH5RItoFn4HGI6QJuV7FLCUNiXQ8BHw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are more and more architectures enabled ARCH_SUPPORTS_PER_VMA_LOCK, eg, x86, arm64, powerpc and s390, and riscv, those implementation are very similar which results in some duplicated codes, let's add a generic VMA lock-based page fault handler to eliminate them, and which also make it easy to support this feature on new architectures. Signed-off-by: Kefeng Wang --- include/linux/mm.h | 28 ++++++++++++++++++++++++++++ mm/memory.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index c7886784832b..cba1b7b19c9d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -633,6 +633,15 @@ static inline void vma_numab_state_init(struct vm_area_struct *vma) {} static inline void vma_numab_state_free(struct vm_area_struct *vma) {} #endif /* CONFIG_NUMA_BALANCING */ +struct vm_locked_fault { + struct mm_struct *mm; + unsigned long address; + unsigned int fault_flags; + unsigned long vm_flags; + struct pt_regs *regs; + unsigned long fault_code; +}; + #ifdef CONFIG_PER_VMA_LOCK /* * Try to read-lock a vma. The function is allowed to occasionally yield false @@ -733,6 +742,19 @@ static inline void assert_fault_locked(struct vm_fault *vmf) struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm, unsigned long address); +#define VM_LOCKED_FAULT_INIT(_name, _mm, _address, _fault_flags, _vm_flags, _regs, _fault_code) \ + _name.mm = _mm; \ + _name.address = _address; \ + _name.fault_flags = _fault_flags; \ + _name.vm_flags = _vm_flags; \ + _name.regs = _regs; \ + _name.fault_code = _fault_code + +int __weak arch_vma_check_access(struct vm_area_struct *vma, + struct vm_locked_fault *vmlf); + +int try_vma_locked_page_fault(struct vm_locked_fault *vmlf, vm_fault_t *ret); + #else /* CONFIG_PER_VMA_LOCK */ static inline bool vma_start_read(struct vm_area_struct *vma) @@ -742,6 +764,12 @@ static inline void vma_start_write(struct vm_area_struct *vma) {} static inline void vma_assert_write_locked(struct vm_area_struct *vma) {} static inline void vma_mark_detached(struct vm_area_struct *vma, bool detached) {} +#define VM_LOCKED_FAULT_INIT(_name, _mm, _address, _fault_flags, _vm_flags, _regs, _fault_code) +static inline int try_vma_locked_page_fault(struct vm_locked_fault *vmlf, + vm_fault_t *ret) +{ + return -EINVAL; +} static inline void release_fault_lock(struct vm_fault *vmf) { diff --git a/mm/memory.c b/mm/memory.c index ad790394963a..d3f5d1270e7a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5449,6 +5449,48 @@ struct vm_area_struct *lock_vma_under_rcu(struct mm_struct *mm, count_vm_vma_lock_event(VMA_LOCK_ABORT); return NULL; } + +int __weak arch_vma_check_access(struct vm_area_struct *vma, + struct vm_locked_fault *vmlf) +{ + if (!(vma->vm_flags & vmlf->vm_flags)) + return -EINVAL; + return 0; +} + +int try_vma_locked_page_fault(struct vm_locked_fault *vmlf, vm_fault_t *ret) +{ + struct vm_area_struct *vma; + vm_fault_t fault; + + if (!(vmlf->fault_flags & FAULT_FLAG_USER)) + return -EINVAL; + + vma = lock_vma_under_rcu(vmlf->mm, vmlf->address); + if (!vma) + return -EINVAL; + + if (arch_vma_check_access(vma, vmlf)) { + vma_end_read(vma); + return -EINVAL; + } + + fault = handle_mm_fault(vma, vmlf->address, + vmlf->fault_flags | FAULT_FLAG_VMA_LOCK, + vmlf->regs); + *ret = fault; + + if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) + vma_end_read(vma); + + if ((fault & VM_FAULT_RETRY)) + count_vm_vma_lock_event(VMA_LOCK_RETRY); + else + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + + return 0; +} + #endif /* CONFIG_PER_VMA_LOCK */ #ifndef __PAGETABLE_P4D_FOLDED From patchwork Thu Jul 13 09:53:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84C60C0015E for ; Thu, 13 Jul 2023 09:40:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ECA868D0005; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA0688D0001; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1B4D8D0005; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id BCF368D0001 for ; Thu, 13 Jul 2023 05:40:31 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8E84314015A for ; Thu, 13 Jul 2023 09:40:31 +0000 (UTC) X-FDA: 81006093462.04.ACD0F3E Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf19.hostedemail.com (Postfix) with ESMTP id EF0F21A0019 for ; Thu, 13 Jul 2023 09:40:28 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf19.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241229; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FeYFc6zSdM9ygR4Vi91p1lIFddSoVzdmv76+Wud4OVw=; b=MwOA1aa1x6F4U67ytYnLDoeINYK0tAJuFjuipQq3ShRSF4HsoDlYjj3j4f5yHhJeviGw1w IYZJy3blTdo+y5AyLq36cjLdTBRuuM0ITEEg6L7NtzxAa5tM5AnYN5pIAKbfR1NwpFMvzp DrdyRLmHGtBPS3LBTbYycy+6gvR5ZRg= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf19.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241229; a=rsa-sha256; cv=none; b=WkAasZQvjevJJBsXNTJUqcIDeYgInPQVWyguE58QTU4iIZPCwY3IZQD8q6sPT3zA1B75sx IvMP1wLDZVEQEYoN15p+6NwqZ4FP+RwCKAr0rWHwDPd8zmhtgA42VHBtGh/R7varV+crnQ W74U2fT6TIrQkh3UFfwEb+QPPegEu34= Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4R1qMT1xcPz18MB1; Thu, 13 Jul 2023 17:39:49 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:23 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 02/10] x86: mm: use try_vma_locked_page_fault() Date: Thu, 13 Jul 2023 17:53:30 +0800 Message-ID: <20230713095339.189715-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: EF0F21A0019 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 15b95u7ygzqcuesgx973ad8tfg1mcy4m X-HE-Tag: 1689241228-718664 X-HE-Meta: U2FsdGVkX1+qz/xGp7D4eIXd5WWL7cGc2nIsHFEuNcb+Lk43quv/viDa3cGSv6/HCG6Qk+Ro4pq7h7JOgbJZsvABQ3yopcfTYyqdradBjbV/bUBu7HFViiNbnnxoB+W6+7ZiKGtPZpHGF6Gt7wqfG9UNHXXRODJqD6aFrBNITmf8wlMddZyMtj2dKqLVHs/105sZyCG0n5gkYBl/wGY5T0w/BuE6GZWi+AYzdQzUbL5WcS/7xLs+HFNRwAnA8n5Z44rbFEleNCo1NffGmPOnfkJF85Ubgq8TsR6zeTn32Gk9jwk5r3OKScJ4C2nbvunxYogl1oo4MkDLG+fx6RGoWhdg5ifR2oVbuygOmmKyKK0jzBylFtKJMRRDhmQ46kfm8vY8Sbk6PFS2elVhQcp9+16ry0nzNEBokaPZSvRgHhiIbkA1EqFD5lktBJC/h41CYVr+bOv4jgZMzwicla/Zhz5HVymxMJJbDrCwQ9xfDff/IHBNEN7cENU05jHq811ubUdepo3Ay2Zuxg53Sm9QixBSKJoNrsl1OlHyp3BPI+r30kxJu32qJj5a94iS41COtPfTtRSYUW8h97qHCvPspoAmojE5sVDOq0cw8Uk6/j2HCVxrsk3RWxEoOuMI+GaAI+IU8WRBZ3gUMBQQdOcu1ebCvIeKfZxop3rjSKx+vPauVNkLCgQdnUtdiQq4CL9hZmyU/amObyfh7TrukbgDO5f7o0DbSpa+/A6R3tlHmKV1ARYKOZ/OInKMj7ScsuHNOihBOeuYm3tKdWbX6ffSj4BkwnrAo+Zq1ECCT0Xrk9Ye7ALPLk2jt36sbfKu5uGmbeBfkedStpAG79bSCBmKkb8NJ1rS8mECAH1fDYj3RtwZJa4oZbUBLauwfXR4HtgR/mXaoXG1HMLQLe2c9ImtsfmSCFN8gFp1Jgw/LrfaKwGxpJ7RDza+UBEohJCfUN2iGWyoCf049IMRZq6mhdA FJPtXeYz UvwqiEfV/UaUBwL2hRhiZlE17mV68FDvEvOwZmhHcA+lfcIBHm5TN17z87kYDi9u/nK7h3IGgKV8DVmiIZLfv09Q3vg7Efp48xkJBMDc7/qhgmn2jxVM+CROldkpdGL9wXwxHMNXnVmoX1zzyyX2rKTL0dGhK+QXO1YsyAMNDEwa1Yjl5LbPR0X/AYk+gOHp1/iLb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use new try_vma_locked_page_fault() helper to simplify code. No functional change intended. Signed-off-by: Kefeng Wang --- arch/x86/mm/fault.c | 39 +++++++++++++++------------------------ 1 file changed, 15 insertions(+), 24 deletions(-) diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 56b4f9faf8c4..3f3b8b0a87de 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1213,6 +1213,16 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code, } NOKPROBE_SYMBOL(do_kern_addr_fault); +#ifdef CONFIG_PER_VMA_LOCK +int arch_vma_check_access(struct vm_area_struct *vma, + struct vm_locked_fault *vmlf) +{ + if (unlikely(access_error(vmlf->fault_code, vma))) + return -EINVAL; + return 0; +} +#endif + /* * Handle faults in the user portion of the address space. Nothing in here * should check X86_PF_USER without a specific justification: for almost @@ -1231,6 +1241,7 @@ void do_user_addr_fault(struct pt_regs *regs, struct mm_struct *mm; vm_fault_t fault; unsigned int flags = FAULT_FLAG_DEFAULT; + struct vm_locked_fault vmlf; tsk = current; mm = tsk->mm; @@ -1328,27 +1339,11 @@ void do_user_addr_fault(struct pt_regs *regs, } #endif -#ifdef CONFIG_PER_VMA_LOCK - if (!(flags & FAULT_FLAG_USER)) - goto lock_mmap; - - vma = lock_vma_under_rcu(mm, address); - if (!vma) - goto lock_mmap; - - if (unlikely(access_error(error_code, vma))) { - vma_end_read(vma); - goto lock_mmap; - } - fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); - if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) - vma_end_read(vma); - - if (!(fault & VM_FAULT_RETRY)) { - count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + VM_LOCKED_FAULT_INIT(vmlf, mm, address, flags, 0, regs, error_code); + if (try_vma_locked_page_fault(&vmlf, &fault)) + goto retry; + else if (!(fault | VM_FAULT_RETRY)) goto done; - } - count_vm_vma_lock_event(VMA_LOCK_RETRY); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { @@ -1358,8 +1353,6 @@ void do_user_addr_fault(struct pt_regs *regs, ARCH_DEFAULT_PKEY); return; } -lock_mmap: -#endif /* CONFIG_PER_VMA_LOCK */ retry: vma = lock_mm_and_find_vma(mm, address, regs); @@ -1419,9 +1412,7 @@ void do_user_addr_fault(struct pt_regs *regs, } mmap_read_unlock(mm); -#ifdef CONFIG_PER_VMA_LOCK done: -#endif if (likely(!(fault & VM_FAULT_ERROR))) return; From patchwork Thu Jul 13 09:53:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DECB2C0015E for ; Thu, 13 Jul 2023 09:40:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BEF08D0001; Thu, 13 Jul 2023 05:40:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 244D98D0008; Thu, 13 Jul 2023 05:40:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10F278D0001; Thu, 13 Jul 2023 05:40:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E74F78D0008 for ; Thu, 13 Jul 2023 05:40:35 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C2C34B0020 for ; Thu, 13 Jul 2023 09:40:35 +0000 (UTC) X-FDA: 81006093630.09.A7ED5B9 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf12.hostedemail.com (Postfix) with ESMTP id 681F84001A for ; Thu, 13 Jul 2023 09:40:31 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241233; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AtOkhhiw09TdDExhZx6wTJwpnMZwYVrrYxu7zFjNWQw=; b=5DZmmXbyQywAe9vOK35feat7Resyf19N3yarj5x5jerupP1nGDYPtfEkI2GqQQUTdDlez3 5R0dtWimiPLoQWV3NrDh79QT2Bm/N5e8mWEjuwjAya4AylD61TpDDXsvMQVuVZAwReHkfg abPcGpwe1VO0hNaRS6c4CWItIXjzjxM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241233; a=rsa-sha256; cv=none; b=ZTKWpYoZfBFnUbioFMl1dyGvjcxgoEuvAG49duGdv5R44v4zmEIHS7vk64G4DWSQz+DJ82 pihIT4hddkj2WbSCcw9lDeOmFjSl/MdXjs1z+a78P6a+eNq0OgGmT1A8EnCtPhUl+M82Ko KqcQkO35oCh3lCwXx1YiFLP/cQWmmTw= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; spf=pass (imf12.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.57]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4R1qJk00S7ztRVh; Thu, 13 Jul 2023 17:37:25 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:25 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 03/10] arm64: mm: use try_vma_locked_page_fault() Date: Thu, 13 Jul 2023 17:53:31 +0800 Message-ID: <20230713095339.189715-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 681F84001A X-Rspam-User: X-Stat-Signature: wx5d5fo8pqiik9iwnuw7zy4oxwcm4gii X-Rspamd-Server: rspam03 X-HE-Tag: 1689241231-844163 X-HE-Meta: U2FsdGVkX18utWEpInfD9FC7qPNAYYFGLZUVaR0fHcwskN+4j8v1yhbLqgekgSIwzxvENAsvQsxPAgyt0HNpW7inFAiyMEBOy3vIRjaoLPwqg1qG1mp1vCvFdKxJ/Gol8b9nfA4PLAUObkDGvAqqGitJghDs2vWE8StkK74REVAdNCT9oaLDTifUT2YD5IYkpuw8eYrQdNIZ60o2t5XMia/ccLjtr9bCXYFi5TzkLEtMQGATWOEB2GNP7yhA90WC/Wzdqma5c/8m92Z5nXxdfA8FSKf4188P/DZEGAVcxfmuszfPr9N0mIYnFltnr7pDh7grGJal6+Q1S/AybdnHhug1ebc/pofRJXJssJYKrWzLypYEXDMjpICobplXS8rdqgq3RdTJ9CGB56urpUpEOdkuqwSCyrAX29ANzh0FuXYGqc0hYqiDb+NiaJG855QNIvXT749N7baVqDUJmcS+ZMh7LZkyW7AZrOeuumnAM37t68c3ZIfRk9jiaUVXxF88M7pYb1YFoz5k38wjtF/lRuhpo5rxh6ZQE8EqKnLGZEYCnPaBGWZBcHHO5FP5yqXwrOy8DZlBgmJC74F1LuNTMuBzWH3K8slpurjc9SCRN9u46rK9mInvwAEr7dInvKllXPi1EXV6sbAoQC+nqzXmfG5uGtVmKGw2MaK5y7fQubSUJpwYllTDMRwHEFnaGGyaZg3qnGfzA0knJhw8QYNEt1Y7cVAuvizQBrSj8yhBZh0OgDs41S5oD2IvSqijHqgfL5v0JIjgTHhHlCT4DML5/I+qvR3GjzQs9azKderDY+EzQwq2uyOKjzaKLSsAi9CcWsHTwkv/LZ55X8m1SzLhI6fHJHgkCfHY0EC3eyCiYUAjWZ0H/k7yN0StUzKR54w37Z5H3vVWlaVZnWTHjApWqFTKvzCZq9m5Htvj1R+Tt5svn6vzGrkjUbr0292DFGEp0tkF4zTgWME1c1xeTb7 sfwykqkN x4jR2k3rPozv8rz8Ggk75sZhbBdviXqC/5mwDXWyRhpREqwCvQ5nOUZ7f8xraE5ARnphy84yRLlsNRdWYKloLJHJnWtdk8Z7d3PQpMPxpEjYPRjgNsxNPamgS/beUr6vPYxoSWJK3p0DiduOiTQ3abU6hisLQ4gqAPIj3pCTlAApoWlOzTicfShuygnz/zIW2/KsQxYXsc14v+GE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use new try_vma_locked_page_fault() helper to simplify code. No functional change intended. Signed-off-by: Kefeng Wang --- arch/arm64/mm/fault.c | 28 +++++----------------------- 1 file changed, 5 insertions(+), 23 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index b8c80f7b8a5f..614bb53fc1bc 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -537,6 +537,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, unsigned int mm_flags = FAULT_FLAG_DEFAULT; unsigned long addr = untagged_addr(far); struct vm_area_struct *vma; + struct vm_locked_fault vmlf; if (kprobe_page_fault(regs, esr)) return 0; @@ -587,27 +588,11 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr); -#ifdef CONFIG_PER_VMA_LOCK - if (!(mm_flags & FAULT_FLAG_USER)) - goto lock_mmap; - - vma = lock_vma_under_rcu(mm, addr); - if (!vma) - goto lock_mmap; - - if (!(vma->vm_flags & vm_flags)) { - vma_end_read(vma); - goto lock_mmap; - } - fault = handle_mm_fault(vma, addr, mm_flags | FAULT_FLAG_VMA_LOCK, regs); - if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) - vma_end_read(vma); - - if (!(fault & VM_FAULT_RETRY)) { - count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + VM_LOCKED_FAULT_INIT(vmlf, mm, addr, mm_flags, vm_flags, regs, esr); + if (try_vma_locked_page_fault(&vmlf, &fault)) + goto retry; + else if (!(fault | VM_FAULT_RETRY)) goto done; - } - count_vm_vma_lock_event(VMA_LOCK_RETRY); /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { @@ -615,9 +600,6 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto no_context; return 0; } -lock_mmap: -#endif /* CONFIG_PER_VMA_LOCK */ - retry: vma = lock_mm_and_find_vma(mm, addr, regs); if (unlikely(!vma)) { From patchwork Thu Jul 13 09:53:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A84C7EB64DD for ; Thu, 13 Jul 2023 09:59:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 266D28D0002; Thu, 13 Jul 2023 05:59:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 217658D0001; Thu, 13 Jul 2023 05:59:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 106508D0002; Thu, 13 Jul 2023 05:59:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 012B48D0001 for ; Thu, 13 Jul 2023 05:59:04 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BD45FAFB94 for ; Thu, 13 Jul 2023 09:59:04 +0000 (UTC) X-FDA: 81006140208.11.DD55307 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf23.hostedemail.com (Postfix) with ESMTP id 625E014000B for ; Thu, 13 Jul 2023 09:58:59 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689242342; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PgqE83exuykN0WeeFpdKu52slDj3sRyE7QOTEX+Z6zU=; b=sXYXB+jcnH/hs5KA/V5ydlSzLKDZrAZj541Rck6FeHDV3KPZcSPGX1Q/gmkrIYEwZfA8hv hb7LZI8vJoZuFdk0HievD/g4T9cDgTqsnwF2aH/MSC6H1APN2tgmSGodYyXwNmXGmuELew jRZXLGMCfLkIz+1MlvMHHy8LcdB1pVY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689242342; a=rsa-sha256; cv=none; b=K7cDSRgJmniXotpJmRXIpUWYaTw4aOnIwQED8T5gRio3NKYby7MyQZcPmtcFfmSR7YgGr/ o+nw4rI8/8XW8yp6RSP1Vs57MZS+V74g8T47V93Y8gXNuIjxZx2yWRDqRswR0K1WuyBMIg VDQxLWBsrY2fjgM3DjVbfnohFpU0flY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4R1qJl1fsnztRTt; Thu, 13 Jul 2023 17:37:27 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:26 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 04/10] s390: mm: use try_vma_locked_page_fault() Date: Thu, 13 Jul 2023 17:53:32 +0800 Message-ID: <20230713095339.189715-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 625E014000B X-Rspam-User: X-Stat-Signature: abic1uuu9b3oaptgbhz14j513z44w1kb X-Rspamd-Server: rspam03 X-HE-Tag: 1689242339-920951 X-HE-Meta: U2FsdGVkX18CJDsnX6mOArTBUfUGL5e2kdWDkR1SNg3BfjfdKXG4uskWWXVFg3kWKdNR2txis7EZ9vgP8T+zD4nF4+yD/pC6NZvms4q4jmr4OaMuRug8YdhDxBnO62t5oe0XxplUYZcUhAqXoWEOXMY+3J/euDVK+UwRrVYlIPlhINnP9UwFr0g6A3TNcSgmoZu2+vnYY6BHgAOcjNNj+2C/FhrILitQIY3sXFIxTSN5kLFl5CXmtey15inXu2PZ/4ie0CbvDK/5K9WhrI2fC3t6meDnqyIt83Rm7o0xFUVZCgFaz0eUv0AvxvQh5WeFZvbMckruQv9iyocPSRXuabYAUcmoIhxn8GF56j/EbWRsVHJ6yRDj6K2UcTMwna7KRRXDyEeZoEfZIog56H+B8gVMo4TCJFBVVAg66nLLodMDbbgFKirE0bE61KjTjz0MnXO3RzCNPyOk/xdQUmMdHbOcKE+Mt1MxXwa9Ahy8sEyn81FrgYTL+EjwxggM6byvxdPrBEmzo8UCPMpWWqNQZmgSxzteYWTPXO7Fje4bLI7/MSfAYZJMX/oiohSUeFFGTa+Fb3IE5ViFNH3x9ssUg4xAq6iAYqeK3TIgnnxkpT2I8Z/tcX0BjjpjVBCeiTdke4jcgiGLylwoYBABVj1n/Sprt9phAuZRFwaivd/dzNS0FEqUYQqK2/gkY0aBtigWt4/UBNLnyJ73RBWv0b3Iu9JuL4YAb73/E3IG43mFNykklPjm9aIelDs/eJfRnJ+7MxTlzhnMCXQoyB5kqzjsAvPlAfY5xPrSaAMjd7sLzR4RR5WLbIW/hqZWd4OTo6VYhlysZ+ZCsxibKQiFe4Wb0lbGCn65sbMenPD8Qp/dzrTnfflqEOFupXn8Wfs1J4Uv78CgHwUg4oxmzyyE3uhmqrND8jw9i3vmctXwQJXznORzh+ACridzMg1b2GTBIuegBHM/ygrXY4DAIh3v5/a j1AMHTz1 ClCxe5+H+tLm7w16BKHPT6IkYmf0eF7AveOga6rlfeVLOaEgPq/lSoFjqimHXXQDH/eqJemgKcx7L29g6XDtW4w4BDwvaP5UtbTiQnZgwrfku8gulfYpHCDd/7+XJTT/owP5DSdGq3Gif+Ml8TR19CSAJ9mfD126XJxVkF3/sClYfu2uGQ72UbFW2z8o7noSnW6tb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use new try_vma_locked_page_fault() helper to simplify code. No functional change intended. Signed-off-by: Kefeng Wang --- arch/s390/mm/fault.c | 23 ++++++----------------- 1 file changed, 6 insertions(+), 17 deletions(-) diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index 40a71063949b..97e511690352 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -362,6 +362,7 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access) struct task_struct *tsk; struct mm_struct *mm; struct vm_area_struct *vma; + struct vm_locked_fault vmlf; enum fault_type type; unsigned long address; unsigned int flags; @@ -407,31 +408,19 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access) access = VM_WRITE; if (access == VM_WRITE) flags |= FAULT_FLAG_WRITE; -#ifdef CONFIG_PER_VMA_LOCK - if (!(flags & FAULT_FLAG_USER)) - goto lock_mmap; - vma = lock_vma_under_rcu(mm, address); - if (!vma) - goto lock_mmap; - if (!(vma->vm_flags & access)) { - vma_end_read(vma); + + VM_LOCKED_FAULT_INIT(vmlf, mm, address, flags, access, regs, 0); + if (try_vma_locked_page_fault(&vmlf, &fault)) goto lock_mmap; - } - fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); - if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) - vma_end_read(vma); - if (!(fault & VM_FAULT_RETRY)) { - count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + else if (!(fault | VM_FAULT_RETRY)) goto out; - } - count_vm_vma_lock_event(VMA_LOCK_RETRY); + /* Quick path to respond to signals */ if (fault_signal_pending(fault, regs)) { fault = VM_FAULT_SIGNAL; goto out; } lock_mmap: -#endif /* CONFIG_PER_VMA_LOCK */ mmap_read_lock(mm); gmap = NULL; From patchwork Thu Jul 13 09:53:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DF3AC001E0 for ; Thu, 13 Jul 2023 09:40:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB8288D0007; Thu, 13 Jul 2023 05:40:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D68018D0001; Thu, 13 Jul 2023 05:40:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2FE18D0007; Thu, 13 Jul 2023 05:40:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B4D008D0001 for ; Thu, 13 Jul 2023 05:40:35 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 76373801A7 for ; Thu, 13 Jul 2023 09:40:35 +0000 (UTC) X-FDA: 81006093630.04.6ED33DE Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf29.hostedemail.com (Postfix) with ESMTP id 3B50F12001D for ; Thu, 13 Jul 2023 09:40:31 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241233; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CULSRs+EGVOzWStogtsujbJIkl95CPb3awE35pF/W1s=; b=2UKrZbHKl9xedKfFAcPc2+8LnbKyuAo58Zy7+cBRLPZBNuIIi4IKoW0I60XLbLiKw6B6wg d+UZplWIFK9Whm7kCWUfWT9YN22sLBal0VkbDAFWkJ0QDJBuRWJAEssENU7e4E3w4M1LFO YIQYTP3AxevUboBW0uQruIheWWCB5pc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf29.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241233; a=rsa-sha256; cv=none; b=eIvzBALFw8cbtUFKSfjGGUG5NSWVDgzXzlT5ygGSVt0UhkcbrwpI0RIBKFaxwn8Uteyi83 oyB+as7RJ53TtwB/8N15Nx/G2EDQv1nncG/6dRnwP897jpeGWToTVcc6Tf+MpBV93an5dl GMu0i70x5ql+2Q5X7xLuHCgWYL2spBY= Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4R1qLp2H4wzVjg5; Thu, 13 Jul 2023 17:39:14 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:27 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 05/10] powerpc: mm: use try_vma_locked_page_fault() Date: Thu, 13 Jul 2023 17:53:33 +0800 Message-ID: <20230713095339.189715-6-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3B50F12001D X-Stat-Signature: shpz6hcpkz4b5oazn7ng47c84nj3tcx4 X-Rspam-User: X-HE-Tag: 1689241231-372652 X-HE-Meta: U2FsdGVkX1/oySPS0W20WEOSSCp4ZG7yLzyG4mk4DQyYUbsdtNkhcQZgYkskfYPeZ39c4ddd3QNtyKpVdwydhakv9abkCoBvB5R+6B2a+JoAOxf3P2dLv3/ehbOmXomu2/GtZipHSoc10tvVNaAEZPCgkEpmk9iuTTgRmm5Wu5Tr0QZhHiWi2+MFc5m0Bbhy7vCSGCv3L7eORn+dQtF+WEHHz+TgnVeB9KWhYuH0c5y3ysO/kXvilyUZiNP0FZ3B3wiVK6sSe+l3CLkIqqVGUV7C4T4zMzniXhvAqGhkNdxfb4FTukLdC203RUS0i4UJ2LOIvtVjLpyCw/xlKNwPCOVCLqUoWWV9iQb+cJEPTqzQcZzQAgOQbWyuSYkgBxqOMUS7wXu8gNb+/shTO6Ug3e5oo0A8Nsm52zlQKO3CGvOSGd5Hn7FZLoAoSQSusX2VQqHC/MEIpbNd8TLORsBd7s/9MUQqoBwKeWwaB3KbNxIRe8ZKeqd9mwyz8uXnjeKjN8mucJsHSg4GWBmjK7h73boPTN7OXkChYtEarpsMvIMwhf4IIYhX4T2L74cMe7QYjOGKfyNOSKlozb49+HeVwpzaT5VHF4eafMytWd/8Q/Dvev8C3/PiCPmUDj+uOBMoGeYNxdc+BKRZtZ5TqU2BZUqWJZIbzGvA/7w5a2zzqR+ToXmjr/Wqvai8NRt4wO5AdwgsbE6aDgYwZjvBuwX+kU+FcYFnV6Lsb7NnfgKO9G2psIupNPz8FLUEQmAc+Xx0lyxkf3lHojVeHhg5hnR5VyG3Y8L2z6AHl/tMX3x49M3aHf0LM2FJqtMSYQceC71vZwIujPOBBdz51HMcqAncvVKgEo4rSLm7AKapsb1OO7MtrLypMXKhnGQXpA52BeDro3/uDgzisKDwKWuedh19MYP5U1lkSZ1omQRs/6FZ5t4IRlliOX+ezqRufxnJziNGW+5b2B1K8t1CJy7/6bj lNZDsH/w VDrEZ8C1tnAT1mvXgl13OY1PnDLPTiq0afcgEwUsjhN1GW3dTtkAVUZ3KWZEfXVqmlM+iJOA/tdnRqvJxNDxKDL1HLBqngrdAFn2+SsaKbupnkxg51mht1NLuhDDv26vpe64q8huzxj/BeCWoVOwcaNfZi4h5+mKWH43AmSp8MO7DwIsbbypsFOz/bTQHPH90CQZH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use new try_vma_locked_page_fault() helper to simplify code. No functional change intended. Signed-off-by: Kefeng Wang --- arch/powerpc/mm/fault.c | 54 +++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 32 deletions(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 82954d0e6906..dd4832a3cf10 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -391,6 +391,23 @@ static int page_fault_is_bad(unsigned long err) #define page_fault_is_bad(__err) ((__err) & DSISR_BAD_FAULT_32S) #endif +#ifdef CONFIG_PER_VMA_LOCK +int arch_vma_check_access(struct vm_area_struct *vma, + struct vm_locked_fault *vmlf) +{ + int is_exec = TRAP(vmlf->regs) == INTERRUPT_INST_STORAGE; + int is_write = page_fault_is_write(vmlf->fault_code); + + if (unlikely(access_pkey_error(is_write, is_exec, + (vmlf->fault_code & DSISR_KEYFAULT), vma))) + return -EINVAL; + + if (unlikely(access_error(is_write, is_exec, vma))) + return -EINVAL; + return 0; +} +#endif + /* * For 600- and 800-family processors, the error_code parameter is DSISR * for a data fault, SRR1 for an instruction fault. @@ -413,6 +430,7 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, int is_write = page_fault_is_write(error_code); vm_fault_t fault, major = 0; bool kprobe_fault = kprobe_page_fault(regs, 11); + struct vm_locked_fault vmlf; if (unlikely(debugger_fault_handler(regs) || kprobe_fault)) return 0; @@ -469,41 +487,15 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, if (is_exec) flags |= FAULT_FLAG_INSTRUCTION; -#ifdef CONFIG_PER_VMA_LOCK - if (!(flags & FAULT_FLAG_USER)) - goto lock_mmap; - - vma = lock_vma_under_rcu(mm, address); - if (!vma) - goto lock_mmap; - - if (unlikely(access_pkey_error(is_write, is_exec, - (error_code & DSISR_KEYFAULT), vma))) { - vma_end_read(vma); - goto lock_mmap; - } - - if (unlikely(access_error(is_write, is_exec, vma))) { - vma_end_read(vma); - goto lock_mmap; - } - - fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); - if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) - vma_end_read(vma); - - if (!(fault & VM_FAULT_RETRY)) { - count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + VM_LOCKED_FAULT_INIT(vmlf, mm, address, flags, 0, regs, error_code); + if (try_vma_locked_page_fault(&vmlf, &fault)) + goto retry; + else if (!(fault | VM_FAULT_RETRY)) goto done; - } - count_vm_vma_lock_event(VMA_LOCK_RETRY); if (fault_signal_pending(fault, regs)) return user_mode(regs) ? 0 : SIGBUS; -lock_mmap: -#endif /* CONFIG_PER_VMA_LOCK */ - /* When running in the kernel we expect faults to occur only to * addresses in user space. All other faults represent errors in the * kernel and should generate an OOPS. Unfortunately, in the case of an @@ -552,9 +544,7 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, mmap_read_unlock(current->mm); -#ifdef CONFIG_PER_VMA_LOCK done: -#endif if (unlikely(fault & VM_FAULT_ERROR)) return mm_fault_error(regs, address, fault); From patchwork Thu Jul 13 09:53:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBA92C0015E for ; Thu, 13 Jul 2023 09:40:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E3948D000A; Thu, 13 Jul 2023 05:40:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 293348D0008; Thu, 13 Jul 2023 05:40:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15BFF8D000A; Thu, 13 Jul 2023 05:40:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 076098D0008 for ; Thu, 13 Jul 2023 05:40:38 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B125416016A for ; Thu, 13 Jul 2023 09:40:37 +0000 (UTC) X-FDA: 81006093714.19.45467D4 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf13.hostedemail.com (Postfix) with ESMTP id 0CC8F20002 for ; Thu, 13 Jul 2023 09:40:34 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf13.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241235; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FkTnQQkkp7c5OfI+DN0NqR3ZKn27H8ODcUopLpWusuw=; b=WuoqyDoD81V8PnX8ZVBwiibz2wYB9xbd2gMbJ3KyVv6mehLsJxT6z8BwbYUF/PSyDsEmRf wUQqjl8FzzbQ6aQyo4wy4KIqGL0mn41ARhsUIqZpdLB1VhP1+UNc0FqzvWN16c0di5+73k EkbZ27Vb6kxoBkiOhWGU1ryShhnsZwI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf13.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241235; a=rsa-sha256; cv=none; b=JV52e4aL4IhGBCOUimWHg2pjEPOhk5GokSDn2Zi+AdrhMZ0IR4lEUSiqN/SPiA8cAegVJB TE0z7YAcwu1R3aj6YoZLWAq0Bo+U4iI4dXaNNahIWvUDgdxlqEAntBfz8hL3DBGvPIMKVP a2PxQi4pK75tD7HE4l7ZefHrmFzDq+s= Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4R1qLq3yDQzVjXt; Thu, 13 Jul 2023 17:39:15 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:28 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 06/10] riscv: mm: use try_vma_locked_page_fault() Date: Thu, 13 Jul 2023 17:53:34 +0800 Message-ID: <20230713095339.189715-7-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 0CC8F20002 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 6t6eos8o3uh9y9i1cagn67z6bdzjyehf X-HE-Tag: 1689241234-980587 X-HE-Meta: U2FsdGVkX1/xCxgyLE+d9s27ki+RGdTUgglZ4xeBrVJlBaCFRicmeds8hLBYX0LSJGXWS4rWkaEvB+LKvPKsPrYkpNjs4aFn+W8Jgeom3ZYhn3/Tlorx5mNILxUBWImuUxgPBL8HyxAZ7819D40eV6jE2PIgtcH1kNPyaQwNWn2M+U1pze1gRIGcivxDQCNRvtWccGq/8XuQVT/r9/4vc3Exv5/n+WogzejG8IebpvcK0hbjLyytHYmOWs4/6mugWuifnD3wV0hK/Otk3tkwwHgcRyKE+Mk6tYHU9/LzKbitUZ8NBVRRA707eCmVCKLqda1hEzZCg9x/i/+3ZHWaRsU+Re2ngwPX6nX5+YUE79jcGE1iKTaS7MXqK/eNnmzpKVNPmwDVvlV3Uett0l9NNFJ0VIsKYgaXgtIx7MQknljSsF7oNH6NjHOTDzBK4JP1FhPauQHgwPGmay5vdmKldbHg8ktvNLQKv0JuoxrZkiFi5nixnYBjYP7ZvTJw/gUt2TXbfZ0Wz3Ow8+SnI/LZdSGaXARD0bDlX+mFjbrjcUXx9cl3tuPz91G5Nz2Cb2ujlKin+2akJXcieR0qg7b/4a6YQMeQhVnK5h3Ga+rQaaoXPCLQw3gvl7/kSbnOApmZ4PaSIFr0ZTv4vAR4l6UzQrmh/SIlSs4VouXc5y9ALXHT480W+unVxvKJXBhta/dQW6wQkkI5Cy0+GlINId7quLcbJYiO7M6l3kvZbCx283Rq8BjDEeuckKOcwDMQ+7qXhRdCoNK7l/ByJdY20OeZwdvSXW4y+8WR4yzj8VOLNujcDhfAhscg7zHaycLsA97dWjGo3CMdnKH0lWhs1m+SYRELwrPKiexGe5RGCPQjoQDL1akgdSu9LYGpEC5LzLQ/JLv4L6ROXBeS5Vk/EbO20ngXfADpAMj4KOnDVzMvdjS0qnIXXpt8dJVoE4rEe80wBf8IHsJNVd11DnO+RVx SKrdnl42 VurMzoWCWnrFHxixW790/89eij10aA+nJ4qDM+hHoQIirkzINgG1qFgdYA66HOqAXJ3JBcGT5vkPZZ+AczS8Zf+I+xTed9U3MscQWOJvMFIDB0dk+VR43VmZkez8Dl0JWJLpbEMZEnfgwEiZFqIqKLZ1bveYiXFDrq4RBdPsynfmj35O0wCNBJ487X48XdqyhFngX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use new try_vma_locked_page_fault() helper to simplify code. No functional change intended. Signed-off-by: Kefeng Wang --- arch/riscv/mm/fault.c | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 6ea2cce4cc17..13bc60370b5c 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -215,6 +215,16 @@ static inline bool access_error(unsigned long cause, struct vm_area_struct *vma) return false; } +#ifdef CONFIG_PER_VMA_LOCK +int arch_vma_check_access(struct vm_area_struct *vma, + struct vm_locked_fault *vmlf) +{ + if (unlikely(access_error(vmlf->fault_code, vma))) + return -EINVAL; + return 0; +} +#endif + /* * This routine handles page faults. It determines the address and the * problem, and then passes it off to one of the appropriate routines. @@ -228,6 +238,7 @@ void handle_page_fault(struct pt_regs *regs) unsigned int flags = FAULT_FLAG_DEFAULT; int code = SEGV_MAPERR; vm_fault_t fault; + struct vm_locked_fault vmlf; cause = regs->cause; addr = regs->badaddr; @@ -283,35 +294,18 @@ void handle_page_fault(struct pt_regs *regs) flags |= FAULT_FLAG_WRITE; else if (cause == EXC_INST_PAGE_FAULT) flags |= FAULT_FLAG_INSTRUCTION; -#ifdef CONFIG_PER_VMA_LOCK - if (!(flags & FAULT_FLAG_USER)) - goto lock_mmap; - vma = lock_vma_under_rcu(mm, addr); - if (!vma) - goto lock_mmap; - - if (unlikely(access_error(cause, vma))) { - vma_end_read(vma); - goto lock_mmap; - } - - fault = handle_mm_fault(vma, addr, flags | FAULT_FLAG_VMA_LOCK, regs); - vma_end_read(vma); - - if (!(fault & VM_FAULT_RETRY)) { - count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + VM_LOCKED_FAULT_INIT(vmlf, mm, addr, flags, 0, regs, cause); + if (try_vma_locked_page_fault(&vmlf, &fault)) + goto retry; + else if (!(fault | VM_FAULT_RETRY)) goto done; - } - count_vm_vma_lock_event(VMA_LOCK_RETRY); if (fault_signal_pending(fault, regs)) { if (!user_mode(regs)) no_context(regs, addr); return; } -lock_mmap: -#endif /* CONFIG_PER_VMA_LOCK */ retry: vma = lock_mm_and_find_vma(mm, addr, regs); @@ -368,9 +362,7 @@ void handle_page_fault(struct pt_regs *regs) mmap_read_unlock(mm); -#ifdef CONFIG_PER_VMA_LOCK done: -#endif if (unlikely(fault & VM_FAULT_ERROR)) { tsk->thread.bad_cause = cause; mm_fault_error(regs, addr, fault); From patchwork Thu Jul 13 09:53:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8AFBC001DC for ; Thu, 13 Jul 2023 09:40:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7EC518D0009; Thu, 13 Jul 2023 05:40:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 726E88D0008; Thu, 13 Jul 2023 05:40:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 553648D0009; Thu, 13 Jul 2023 05:40:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3EE548D0008 for ; Thu, 13 Jul 2023 05:40:37 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 05793A01C4 for ; Thu, 13 Jul 2023 09:40:37 +0000 (UTC) X-FDA: 81006093714.23.555D80D Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf22.hostedemail.com (Postfix) with ESMTP id 696BFC0009 for ; Thu, 13 Jul 2023 09:40:33 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241235; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IV4DNkdW4Hd8ZP157oL1HKEHrRYXoRV6592fFT5pGzE=; b=iS4Qt5FDByrln1PfNXkTeKiTeR3EqRCjO4S2nHvzM7madT9SNPWrDOQWU4zvyzy+DlUdZ8 +BTGCegd9MxQzukoonU+Zv/+Hqg35Q7y2uGcslmt+OC1uTXdml/X7wTnpSEY+qhFcolTUe JvPOWB3WapplV9mg3baIXeW269uurFM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf22.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241235; a=rsa-sha256; cv=none; b=rgYTI7Mo/yXaNRDuafunMCmih3F8hjQP8BXfAueTM7hUnyoVF3jhauwkm5B4MUJS7nHj7G fePAInVUJQ0zSryqnD5zrLK0+VLjo5juV/yCQJ7dHHySCEjlaTt4PEAypOZi6VuoGblZyn BAjbfucWm1C1k9tZhIOSpVB6nE8XUn4= Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4R1qMZ56yBzrRn7; Thu, 13 Jul 2023 17:39:54 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:30 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 07/10] ARM: mm: try VMA lock-based page fault handling first Date: Thu, 13 Jul 2023 17:53:35 +0800 Message-ID: <20230713095339.189715-8-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 696BFC0009 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: sw3duq7zbyywix6pbtout6e6p57kakgr X-HE-Tag: 1689241233-931896 X-HE-Meta: U2FsdGVkX19JPFAhele5VEyiSuY09fKshxAIeJf/MAzayo7lCgxNJecKM1TtSLs8QBQxh/kQE1t/JCKbj8isD9YthcT1EanMJG0BY7Ec/S03+7OcsQFrFLIjZfjuVPA/d4xx7zCf9Wlk8W8z3bWDQdUaaaE08+38ho4gQ4+nc0Fxz7LTvu2r71QgXlYLBLw5Xs0+Oa4q5vrF0tfUjtgIyOF5hXiKO0lSyfq9nLNT85yIJgT7MXA5ZngcppRoG0/9kpc4P2ZgD73K6g/MFcTglrQK63JeVcKPhINcj7yHDkss/A4J3IcWlnCTXr7e/NQba6ldvG2vXdSSUQmgW5nttIeEQf/GIb/+E5tNhN1AO8c6t3jmqEVicpWRKDGVJA6hzRv88VtyNAIOF2vz1fmczkfcFEYfOxIv/Q2MFBATc9daRk938aCplrx1cM5rsMuiDmqk4pbn7wh4s1bWQqak+x+YG3toMjJakNDb2gowuCMN24z06I1ctC3vxUEYVZh5a8vcVpKQrC4zzlS4qtTTZWE/urK5HehdSG2LZ/SVc3EiS9bRDD39S6BLWeMHyJzguTGmuF9GGV1mQkJCWXAsaomUE8deMk+tUIu3VjnlakzhJrkReW3hteU7ZrBKgqZ9aoJ5m3x8JwNU/voIvQ6NapJL5x+yT1KEQnIw2W7B0v87d3zJ13lBih+7DyEJwRfaEZCFYfOQD/KiElhM1UuW0nctfLiDbB8g1G3bE6H0TnQiIjk+0JNLnFgIoOIeHrFqVeuzgJZURWFyDsYg7e/K5r51/qvQ6x0/OWwGQV4K5+TZWtgzjikH/NIGvY86rdPoyPANE0AV70NLjh0wxfExyB44C631bmOBW9UUFb2c5onPgNzveobWugR5fEIj7yXwbDQFtHq0Hd9naF15XAyLOeqYUB88e61rHYVOdWH0jx9M+PeB0p0do6ZAjRFuLgnqBaY+xX4qvmpzhd+HB8D Jqowt0r1 HRb7RUJA/tSeIPPIR94w+GuJ6VoxIAMFqi/UYJ6cQcN4sHuDaIJuVBKbFjWBkabbNqyBbsa1p8spXk5LNnDYpsS+TN5zBGLnMMLm6zcvNGCJFV+omKOdBoMWJeZtOEY3AB7x6+ikHEW4tlaOs2DuSKm2TBA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Signed-off-by: Kefeng Wang --- arch/arm/Kconfig | 1 + arch/arm/mm/fault.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 1a6a6eb48a15..8b6d4507ccee 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -34,6 +34,7 @@ config ARM select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT if CPU_V7 select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_HUGETLBFS if ARM_LPAE + select ARCH_SUPPORTS_PER_VMA_LOCK select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_MEMTEST diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index fef62e4a9edd..c44b83841e36 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -244,6 +244,7 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) vm_fault_t fault; unsigned int flags = FAULT_FLAG_DEFAULT; unsigned long vm_flags = VM_ACCESS_FLAGS; + struct vm_locked_fault vmlf; if (kprobe_page_fault(regs, fsr)) return 0; @@ -278,6 +279,18 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, addr); + VM_LOCKED_FAULT_INIT(vmlf, mm, addr, flags, vm_flags, regs, fsr); + if (try_vma_locked_page_fault(&vmlf, &fault)) + goto retry; + else if (!(fault | VM_FAULT_RETRY)) + goto done; + + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + goto no_context; + return 0; + } + retry: vma = lock_mm_and_find_vma(mm, addr, regs); if (unlikely(!vma)) { @@ -316,7 +329,7 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) } mmap_read_unlock(mm); - +done: /* * Handle the "normal" case first - VM_FAULT_MAJOR */ From patchwork Thu Jul 13 09:53:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3EB0C001E0 for ; Thu, 13 Jul 2023 09:40:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 83B0E8D000C; Thu, 13 Jul 2023 05:40:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E9CA8D0008; Thu, 13 Jul 2023 05:40:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68B0A8D000C; Thu, 13 Jul 2023 05:40:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 50AD98D0008 for ; Thu, 13 Jul 2023 05:40:39 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1AA5F120194 for ; Thu, 13 Jul 2023 09:40:39 +0000 (UTC) X-FDA: 81006093798.05.E65856B Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf11.hostedemail.com (Postfix) with ESMTP id 920C64001B for ; Thu, 13 Jul 2023 09:40:36 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241237; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=58o4MR5l0Nei2gEa3JbIAdGHk1zCv0Ec6/i0uRr1oCU=; b=W30LoIfRYj9C9BLiXwoFURXn5ste3MMmkEjY/WBdDWW3+fJOIa9Wb4yjiJ+0JHe28ixaGV ir6nBfIvAw5bE4qnuye/bWybCVETGJRTJnGYBdGqzzYcfIyDtRgrTHC2wq6chCUv3cSgnn em+MpzR/YVM9K0BXRe2c2tA7w9uAOzA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241237; a=rsa-sha256; cv=none; b=3se6zbT9A6sOpMxE8kka5dI2xgtyTQ2qyj7vI4kSC0TwjVNWHNJRIkq2P/ktLizoQBVe4E m60rdE4Q3ZrxDhm0M+nvsp8EVMfdMyYbLVCxYRL/XeDt2dvNeCRBCWqOq8729vutHeSQiu Lq7naMPyvD+NSe9uj/0SFUdMc1YtpgI= Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4R1qMc4wfvz18Lt7; Thu, 13 Jul 2023 17:39:56 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:31 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 08/10] loongarch: mm: cleanup __do_page_fault() Date: Thu, 13 Jul 2023 17:53:36 +0800 Message-ID: <20230713095339.189715-9-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 920C64001B X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 384zxy5momwz7mcksmd846pzpb3i7nnx X-HE-Tag: 1689241236-589738 X-HE-Meta: U2FsdGVkX1+bFGd62XL5Oxbxb0qmadHuAoZkQ8HKeT9VHeINLbDriFfJS0fiZQXRCWaXiZyZ3LfMKSzbn33rgMjGcYrxjela5OPfveRX6BLR1ikXC4hpXKepz7E44bf6j2gy0pxkU8wAFD8d1PK6V9DR/6wpguSshjhnOLcXQZXiNrxxtMxxMNrxRHhi5+nLRctUTT58lIkD9uB/xooigqqlDyOp6qFhYSkcq4KJJQU7VxFt6ZaB90Ay4l3rwGFlClKkpda5DsMtGrkKdAHqCyig200k9xbt1RUxcT5HtqkUAaW2zsd4Tfz/3Znbf6MZdemiTm+haYeqKwJ2wYP0v3YTD9XCZSbTY2nihah0S1jNpKpAyrL0AeaVqkCSTKcIwoW8z1q1RQjkRZIe/SGT+XCtCvkg27wO/+G8qzmF+zxSer9EUnjnyz8tNBTYqCzXdxxPWWL8hgJsJfP+Xf6jUTjEDYIzPSmsrxp5+b4PZVgWmcvxq1IIkQoOIX8Gq5tRufHFKZRdCgpNVAA4b9k92Ts4d2zxovwiNB6qnci1nL6AVZmi5BjcqFg+cS46Fxn1xiLsmMXfhER7xUnKffADrJTLdpWdl25sPuCoC5jRXm4lyAVkG38tAlu9zs+2618TAKOxJ8r18Il9/0Lmkzv2h5ghKYJzWK4/wQTdg95MUz6MRdUFWB6KBO36f7VT/lLggfqEfN+S4fIK4XKrsjvh6Ja2Cz9NASYgtf174cNec068GBkCtUy99oisx6xVBdM088DqVIeNtxEctPMBj7H0AfIEc9xoznZ23s1v/0x9wXrFRippu68iybdJNc779nzJDrpcjLLDvXumtijwzviJgYKJIqWppA3tuf4SE6EFlQfoulKiLarB5GJnm/QzJq3Ha6ubHfqR/aJ5VkJVNl+tWWBwlyQynkwrvkSj7a1VUxMcRq0yEweCKMRmJLu7x2cbYYK2cj/CF67JPvLkCww rvl4Ssub VmnNiY2cqXL7OdFVteU7i/523u9TzM6in1xtY3DD3+uWdheJQ+gqWW4KK/TWZjTux6TPm9KykE8EBkEkSWme5xqD/jeIm0OQ5hKgcYzM3ycSb9SGuesSbfTAUSjmSJeDB8B4KqQLULE5aesCpQjNdJTNOu1Tr6Q1FpwvycsExmUYhTX5MUbqrKhx+83SvWzn2xX7B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Cleanup __do_page_fault() by reuse bad_area_nosemaphore and bad_area label. Signed-off-by: Kefeng Wang --- arch/loongarch/mm/fault.c | 36 +++++++++++------------------------- 1 file changed, 11 insertions(+), 25 deletions(-) diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c index da5b6d518cdb..03d06ee184da 100644 --- a/arch/loongarch/mm/fault.c +++ b/arch/loongarch/mm/fault.c @@ -151,18 +151,15 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, if (!user_mode(regs)) no_context(regs, address); else - do_sigsegv(regs, write, address, si_code); - return; + goto bad_area_nosemaphore; } /* * If we're in an interrupt or have no user * context, we must not take the fault.. */ - if (faulthandler_disabled() || !mm) { - do_sigsegv(regs, write, address, si_code); - return; - } + if (faulthandler_disabled() || !mm) + goto bad_area_nosemaphore; if (user_mode(regs)) flags |= FAULT_FLAG_USER; @@ -172,23 +169,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, vma = lock_mm_and_find_vma(mm, address, regs); if (unlikely(!vma)) goto bad_area_nosemaphore; - goto good_area; - -/* - * Something tried to access memory that isn't in our memory map.. - * Fix it, but check if it's kernel or user first.. - */ -bad_area: - mmap_read_unlock(mm); -bad_area_nosemaphore: - do_sigsegv(regs, write, address, si_code); - return; -/* - * Ok, we have a good vm_area for this memory access, so - * we can handle it.. - */ -good_area: si_code = SEGV_ACCERR; if (write) { @@ -229,14 +210,15 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, */ goto retry; } + + mmap_read_unlock(mm); + if (unlikely(fault & VM_FAULT_ERROR)) { - mmap_read_unlock(mm); if (fault & VM_FAULT_OOM) { do_out_of_memory(regs, address); return; } else if (fault & VM_FAULT_SIGSEGV) { - do_sigsegv(regs, write, address, si_code); - return; + goto bad_area_nosemaphore; } else if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|VM_FAULT_HWPOISON_LARGE)) { do_sigbus(regs, write, address, si_code); return; @@ -244,7 +226,11 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, BUG(); } + return; +bad_area: mmap_read_unlock(mm); +bad_area_nosemaphore: + do_sigsegv(regs, write, address, si_code); } asmlinkage void __kprobes do_page_fault(struct pt_regs *regs, From patchwork Thu Jul 13 09:53:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCB38C001DC for ; Thu, 13 Jul 2023 09:40:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 183798D000D; Thu, 13 Jul 2023 05:40:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 134EB8D0008; Thu, 13 Jul 2023 05:40:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EEF208D000D; Thu, 13 Jul 2023 05:40:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E03398D0008 for ; Thu, 13 Jul 2023 05:40:39 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id ADAB31201BD for ; Thu, 13 Jul 2023 09:40:39 +0000 (UTC) X-FDA: 81006093798.01.3DEDEA5 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf24.hostedemail.com (Postfix) with ESMTP id 2D8A6180019 for ; Thu, 13 Jul 2023 09:40:36 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf24.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241237; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=smGocFmrYZVxcmC4HyPQleNsFVdy0YNcqxX8ihatQWA=; b=HKHs1eJu7Hzk6kNmhvulfsktO5LQpj+L5Y+LiM4FdU0OGFmt1U0w2u7nIPmbXut54NT+e2 sVwphYpJzy7XqeqtproVedY4w85UFKENxo7QfMWPKh/nIr3SeWJ1rFcxDE2FvynHz/LyOx ULYBi6VWMZlLJYEFVXZOA/ugfYn/4Ks= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf24.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241237; a=rsa-sha256; cv=none; b=T2mwpN/yf2uSVJWzlCtc5/zTgJmMroYRRoROIoqLpJHFIhe3bhwnPu4qFXtQ3MksyN1t1X cAgnMPf6YNqvehwcaZvtaJsD1MANOPGLF1ylpw0+Nzus0aLmB38n8yWZWvTAfLUTjb6WZH kexUx5nXMtxZbVcEz7xYPYfwMzoY83k= Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4R1qLv1v51zVjYP; Thu, 13 Jul 2023 17:39:19 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:32 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 09/10] loongarch: mm: add access_error() helper Date: Thu, 13 Jul 2023 17:53:37 +0800 Message-ID: <20230713095339.189715-10-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: xmbqf4wkzhuu7fmkteqc4h96ucortgw8 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 2D8A6180019 X-HE-Tag: 1689241236-698654 X-HE-Meta: U2FsdGVkX1/nF+hitJFb49UFP53BfxU2hTjMy5aVIJ4ysk71T/P8xqTHoeU+iIs4hcOyP5BhtNqTDdmkMm/V3zylvIgjcFiX5TbTIVNrjhFMNv4IfOnLpe1DicT+d73vbz0/gbpRLhS13AYE+tSFhGcJG0tVVE2ubMosdYak8Y02+WlEl/rhZH39TQIc6WrI+kMAuQL2q0p2T7KYwBIbDN/VV/f59Wdjn2GAboG9ZrCkn/Hy6mnuojJp+XTPdHhBfcfDHJOUu8QGYodXXJNtHOImy1zzo/4BxYD3eERft3tDm/bBhQvg9gdGd2XlWg9LoZDREVS9pQOrNwxiPzSpaaS7JZ8vndkHRWN4YWK7JqeurcSV3r21a/hYVv6V7jIAwHS4mYZwizP/SfIKZ9eUtEZxJ7I5soihcqPKzkllDbxwOFnShot9ExrBrsuzN0gqqxamVZzoGOQW/u3Ui/9wOHbBLM7dF4ykKtNm6qP+DqdJiWUMMdubh5fDN2Ie0opgD5l5QLhoqKPbcRNhFixZUCjDFlfovg5iu89JQ4DOvHhoYJ6Xi/pm1mPyYlZqmQQPv1EtN4e/TL9IWiiipcWphraPHbJYykmmZQCYHBzdpxNZRI3Wqy1j7om6fwSadvzZ8L3kCuLeT6xQCJSfBbupgBfhtz0QaGIleWH0L72dEzqyBGDrkjVgLN0xDbtOm1udBVGIfkvlXq6O0bs1T122yT9vyDewMEJ2HhvWCfdgrpMBF0y+qmB7vz4ZblzIiZA0xjTCAe43qMBJW1Few6ZgijRpqIP4QhhZV1V27FRA02KmAaE5FQjkNu4barrCkGEBbHSKEVnJx4WmtmONzQdYtZcsr7PGB/S8pH78QUox2Xd66ww46JHyZ1t1p+P039oTtX+KNdbUdPFQY0piKtCHL2ZYAaWGbohYDnD/26iWeAU1TWIWdmXCqb7ONc3+v+j8NN3LbWmJJYoQb18Y4ei 0nE4MfhK byTa5Y2Kd52g/l3eGLmPALYUB6zBazD9K84b49Ad8ivYIxqaxdATD9hR0+BkvUUsRks5JsQIXzkq95p4po2F4PiUOwaNLpzDXlFYwIWXpOLq9V0yHpdOf5IcK2DtgkfqFPnj1wJVXO/9g3Ji1NRtGby17yBTKdQB6+5v3a2eUscLIFL9yV+5urGArIoWtUdsdlMew X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add access_error() to check whether vma could be accessible or not, which will be used __do_page_fault() and later vma locked based page fault. Signed-off-by: Kefeng Wang --- arch/loongarch/mm/fault.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c index 03d06ee184da..cde2ea0119fa 100644 --- a/arch/loongarch/mm/fault.c +++ b/arch/loongarch/mm/fault.c @@ -120,6 +120,22 @@ static void __kprobes do_sigsegv(struct pt_regs *regs, force_sig_fault(SIGSEGV, si_code, (void __user *)address); } +static inline bool access_error(unsigned int flags, struct pt_regs *regs, + unsigned long addr, struct vm_area_struct *vma) +{ + if (flags & FAULT_FLAG_WRITE) { + if (!(vma->vm_flags & VM_WRITE)) + return true; + } else { + if (!(vma->vm_flags & VM_READ) && addr != exception_era(regs)) + return true; + if (!(vma->vm_flags & VM_EXEC) && addr == exception_era(regs)) + return true; + } + + return false; +} + /* * This routine handles page faults. It determines the address, * and the problem, and then passes it off to one of the appropriate @@ -163,6 +179,8 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, if (user_mode(regs)) flags |= FAULT_FLAG_USER; + if (write) + flags |= FAULT_FLAG_WRITE; perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address); retry: @@ -172,16 +190,8 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, si_code = SEGV_ACCERR; - if (write) { - flags |= FAULT_FLAG_WRITE; - if (!(vma->vm_flags & VM_WRITE)) - goto bad_area; - } else { - if (!(vma->vm_flags & VM_READ) && address != exception_era(regs)) - goto bad_area; - if (!(vma->vm_flags & VM_EXEC) && address == exception_era(regs)) - goto bad_area; - } + if (access_error(flags, regs, vma)) + goto bad_area; /* * If for any reason at all we couldn't handle the fault, From patchwork Thu Jul 13 09:53:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13311667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26F0BC04A6A for ; Thu, 13 Jul 2023 09:40:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A6C68D000E; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3566A8D0008; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F7118D000E; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0B06B8D0008 for ; Thu, 13 Jul 2023 05:40:41 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BE2CB1401B6 for ; Thu, 13 Jul 2023 09:40:40 +0000 (UTC) X-FDA: 81006093840.25.BF178D8 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf03.hostedemail.com (Postfix) with ESMTP id 0CDEF20014 for ; Thu, 13 Jul 2023 09:40:37 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689241238; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kw+TYNhPpY/abAWXtmhBQbSP/ks5Qd9nSeDRwWdiv9o=; b=ez79Sab+tyOikervhcZY0YSJ1qSmoZw6JroL2NHYnf/FWRSpb0T3ymH048F2GtqWFFOg+c HS+Pyq/DOvQT9osfZ/UXb4WAEAC4fxe8io2xRnAEEB5+7/bNPYhJVgwP9es4Is4xvHHqIy MmfZZRj134AEZ1vwpM225omAcCwaN+M= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689241238; a=rsa-sha256; cv=none; b=2YWJ9CpjbdzTsYXlOotL8TSn4dKH01iOq4mgl80+VNzZHhEW6Wb+b8KCEE+xFRW0mi+ljk g81WSc/+FcnOd/lFoaDPU/TFnOdOBByWNR5WrFwo/xBQ/dO2gk6Fsu2khiluH7IWptVyTM P6dFDkEaJ3j/zoopevsIpCW7S1IJDvg= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from dggpemm500001.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4R1qMg1LkQz18Lnh; Thu, 13 Jul 2023 17:39:59 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm500001.china.huawei.com (7.185.36.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 13 Jul 2023 17:40:33 +0800 From: Kefeng Wang To: , Andrew Morton , CC: Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , , , , , , , , Kefeng Wang Subject: [PATCH rfc -next 10/10] loongarch: mm: try VMA lock-based page fault handling first Date: Thu, 13 Jul 2023 17:53:38 +0800 Message-ID: <20230713095339.189715-11-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230713095339.189715-1-wangkefeng.wang@huawei.com> References: <20230713095339.189715-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500001.china.huawei.com (7.185.36.107) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 0CDEF20014 X-Rspam-User: X-Stat-Signature: 3pgpst815xw9bxirhaia5tu1apahya9i X-Rspamd-Server: rspam03 X-HE-Tag: 1689241237-637535 X-HE-Meta: U2FsdGVkX1/2ykratl6neh6znNkJ3jbQ0umQC1gKDHhlCu+UDmF1zInIGnaXHkXjMXoCjzFVH1Ao6UN4F4zC6wT/tqN/U9bi5BgR9YU7souD/c9IeKBIDBepYLlG9Dz6clq+J5H11wbas47L/V8HqdGul7rIO3CHh8Nfva+BH/fa9mryzQxYw6HJ/YeFr4+7mkskwajnphOMXXkMg5AtgSxzQa/KdkMXa9E6DGuaMIWm5IQFcHtFhHEGUfFVPiKoJxQEsPGPZ0r8Q7UxY5TRsWJGUob7TKNLUAaAMCOnXCPRPdhwnIVhVc+lcEy/jxFalEQleO45A9X0l4sdEIuPGTnCjjGlBwhI0e/ss3ApivH/My0axVBDSLltsF2h+x4o+Y2+rdr9WWzFiIG4TIm6t01uJZmvzoymJQ2LeGMNYSEaz1W4UuMrf76w6Rvm1N8YT6h4CIIv+OxiLAwWlEWB6xMhgDWPK0MBYY6vvmaDICEMl0W/Y6IPvIN8fUEek/fo/xF/nzkRHd+Inq2Cvx4wc9hDCFqksOFDwZQITqr3KJVoQlp+8huK7/b2YUqaceLmG+tfv4w7SxDiTpc7BT0jb47nfL1mnK/Zl3TF3hkSp6pfRlsYDxq7hOSp3eZ+1rbNSWuUq3irJNwhrVPoyzhpYenSlfhyd84PbXMLFiq8GsuvJCeJAxgDw7Q2VZnErtZXTPV4Vd6gYgFZPP9+bf6KUkX61SAU/MtmTnfez8qTXAEDog7MTICCqBBtTgn6BgxJUVNahYzhQt8bIymKbKt2HHMnOG+B0HzfMo74CF5ajy0LaBbHYGKgGxMjOo+QV6hCsxFILvc0wOhMFaPXnzvjd2wJedckIvF6LpR13RzGyt22wDIHIc3yvH9AEEO7/8ssE4wgTLsXEWAjcFD/Bq+B1yuUt20KKkJf4M73NXZItyfMCW6vlDKoBCMKEnk720YK3SEBFySL6byb8oQvKvx TTgLUlxt /yJvrSumbV+P35vN8As7zsNMNTSgNVjgoARTbS3Lyw7VdUpy40y50zodeQKViJOWYtDBE8parTZZecfKmB+Jj8RMDKx5Q61NfvAc0J8pR2t5Fom/EsrpdJE6nHlcozf4Y8SNtOI7hIhenvXY1RKGcBREjTQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Signed-off-by: Kefeng Wang --- arch/loongarch/Kconfig | 1 + arch/loongarch/mm/fault.c | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 397203e18800..afb0ccabab97 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -53,6 +53,7 @@ config LOONGARCH select ARCH_SUPPORTS_LTO_CLANG select ARCH_SUPPORTS_LTO_CLANG_THIN select ARCH_SUPPORTS_NUMA_BALANCING + select ARCH_SUPPORTS_PER_VMA_LOCK select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF select ARCH_USE_QUEUED_RWLOCKS diff --git a/arch/loongarch/mm/fault.c b/arch/loongarch/mm/fault.c index cde2ea0119fa..7e54bc48813e 100644 --- a/arch/loongarch/mm/fault.c +++ b/arch/loongarch/mm/fault.c @@ -136,6 +136,17 @@ static inline bool access_error(unsigned int flags, struct pt_regs *regs, return false; } +#ifdef CONFIG_PER_VMA_LOCK +int arch_vma_check_access(struct vm_area_struct *vma, + struct vm_locked_fault *vmlf) +{ + if (unlikely(access_error(vmlf->fault_flags, vmlf->regs, vmlf->address, + vma))) + return -EINVAL; + return 0; +} +#endif + /* * This routine handles page faults. It determines the address, * and the problem, and then passes it off to one of the appropriate @@ -149,6 +160,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, struct task_struct *tsk = current; struct mm_struct *mm = tsk->mm; struct vm_area_struct *vma = NULL; + struct vm_locked_fault vmlf; vm_fault_t fault; if (kprobe_page_fault(regs, current->thread.trap_nr)) @@ -183,6 +195,19 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, flags |= FAULT_FLAG_WRITE; perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address); + + VM_LOCKED_FAULT_INIT(vmlf, mm, address, flags, 0, regs, 0); + if (try_vma_locked_page_fault(&vmlf, &fault)) + goto retry; + else if (!(fault | VM_FAULT_RETRY)) + goto done; + + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + no_context(regs, address); + return; + } + retry: vma = lock_mm_and_find_vma(mm, address, regs); if (unlikely(!vma)) @@ -223,6 +248,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, mmap_read_unlock(mm); +done: if (unlikely(fault & VM_FAULT_ERROR)) { if (fault & VM_FAULT_OOM) { do_out_of_memory(regs, address);