From patchwork Fri Jan 24 23:54:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 13950034 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5C26C02181 for ; Fri, 24 Jan 2025 23:56:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1BFC32800BA; Fri, 24 Jan 2025 18:56:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 16F812800B0; Fri, 24 Jan 2025 18:56:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F05C72800BA; Fri, 24 Jan 2025 18:56:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D0A562800B0 for ; Fri, 24 Jan 2025 18:56:04 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8A5364B92C for ; Fri, 24 Jan 2025 23:56:04 +0000 (UTC) X-FDA: 83044006248.25.8AD9424 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf21.hostedemail.com (Postfix) with ESMTP id 954A41C0012 for ; Fri, 24 Jan 2025 23:56:02 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=XeEkIkrW; spf=pass (imf21.hostedemail.com: domain of anthony.yznaga@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=anthony.yznaga@oracle.com; dmarc=pass (policy=reject) header.from=oracle.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737762962; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nCdkv5pMD/BW8EJZHRB98QYPFY1ik8038tE9HN9GQDQ=; b=HBdj1OnpYNC9RdYkBF5HpHLtOL1GLETD2mUTgJ2HtpQXI3p9wPtktk8+dlynPQYmH4fwJp QtRzmeUO1CBTPVje60BiJ7yLAR0+oEgKAMzjsvjWzP4NbAZOlLWgxYoHQUggAMrZ0nPptJ UrrhQNFkVCeNltQO2Np1qRpqZIF+aG4= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=XeEkIkrW; spf=pass (imf21.hostedemail.com: domain of anthony.yznaga@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=anthony.yznaga@oracle.com; dmarc=pass (policy=reject) header.from=oracle.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737762962; a=rsa-sha256; cv=none; b=hG4+gp1Xro+7ebVY4eEnQAyCgHh1nz071EFJbMWcWa6tc+6cMd3ytO92/5HxAiHWXqf224 C/DtFWtEz9nRM+AqhmR+FKH0DifVSb2avDc6mrq3YgofZHbPcmzbRp1rcUtRDELjd/XYdo 9+gJcNEEqiWnmjVrWRCMK/ufTI5e/io= Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50OIeKEb031124; Fri, 24 Jan 2025 23:55:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=nCdkv 5pMD/BW8EJZHRB98QYPFY1ik8038tE9HN9GQDQ=; b=XeEkIkrWDwXm5HtebbhBD p3lABo0GMrvK4ZvGeh56K+Ji9vVwuOJ6CfP9WQgDNnIg/fylBvrEnda76gj1hlaH djlWZy7JPMD3YWvJ07uS/u2FQf/WNtgihIgs+ahOJw7m476tTNkDwMXzkkvgVlMb WaAhY1K9BmKIjf9cBYWQCBii9KnDOWEQ86r5Jgcb+DoKoPYGmvQzlxO/2s1/HjSE 2X8wi2WkZkemqVoSzjBSlGdAQCPFca5ePjZctgTNBHixFaMq6YMIwXCF0STwxFVX 1V8aXMFIhRqshW3vli+id7PjxsJ0s6TGm5KFe5oka0l703yelWNl+uHD03Bpstfy Q== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4485nsmvn0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Jan 2025 23:55:50 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 50OMD09a036431; Fri, 24 Jan 2025 23:55:50 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 44917u4ahx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Jan 2025 23:55:49 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 50ONsxQ8018051; Fri, 24 Jan 2025 23:55:48 GMT Received: from localhost.us.oracle.com (dhcp-10-65-130-174.vpn.oracle.com [10.65.130.174]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 44917u49ww-14; Fri, 24 Jan 2025 23:55:48 +0000 From: Anthony Yznaga To: akpm@linux-foundation.org, willy@infradead.org, markhemm@googlemail.com, viro@zeniv.linux.org.uk, david@redhat.com, khalid@kernel.org Cc: anthony.yznaga@oracle.com, jthoughton@google.com, corbet@lwn.net, dave.hansen@intel.com, kirill@shutemov.name, luto@kernel.org, brauner@kernel.org, arnd@arndb.de, ebiederm@xmission.com, catalin.marinas@arm.com, mingo@redhat.com, peterz@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, vbabka@suse.cz, jannh@google.com, hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, shakeel.butt@linux.dev, muchun.song@linux.dev, tglx@linutronix.de, cgroups@vger.kernel.org, x86@kernel.org, linux-doc@vger.kernel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mhiramat@kernel.org, rostedt@goodmis.org, vasily.averin@linux.dev, xhao@linux.alibaba.com, pcc@google.com, neilb@suse.de, maz@kernel.org Subject: [PATCH 13/20] x86/mm: enable page table sharing Date: Fri, 24 Jan 2025 15:54:47 -0800 Message-ID: <20250124235454.84587-14-anthony.yznaga@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250124235454.84587-1-anthony.yznaga@oracle.com> References: <20250124235454.84587-1-anthony.yznaga@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-24_10,2025-01-23_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 malwarescore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2501240162 X-Proofpoint-GUID: m7wXHyW69pLXZ8gl0E-xcyVHCx0Mq1Za X-Proofpoint-ORIG-GUID: m7wXHyW69pLXZ8gl0E-xcyVHCx0Mq1Za X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 954A41C0012 X-Stat-Signature: 9acmpjot6pip3pywaa4miw33kmu73ouy X-Rspam-User: X-HE-Tag: 1737762962-583143 X-HE-Meta: U2FsdGVkX18a3e3SgiVw/iyzWFec9KlNlGHekbbdGgLlOQcMfJqGz42ZgRqVMoIp0OHiG1CU71+5q78WC1SYCutonbC/r3c8HBkiEY4YtPHFrp2WVkdPwCRboGeQjPbY3JihCJ2UYe5azvv2JvcOUxfjhTHgwXTZzEXhopplDPZWCxcYSPIMuR7rKHRiN5kAIrKYrZRdhM9pETOp96lOUOp+G+hieITECMiq/3RMjBzYNSCu9D2A7AorVz5AXy8nXVIWXaPoO0TSu+YnJr49eXRLP809qG36ngVdbhHqATH7/B903M8NC9I9NMQ5m/kDZwKgDQ6jO9TNc4Xt6KveQqxbRhUPUEc2vV/wwBsrbaQQYjI2wDjbbUS7ygQfg/5Q6kBBLgi50XazwSh9e7dhd3FF/IOZqQXV+CKyJPl4gzgIzvnVJ+C/9zZhPSA3tgdk6fmGkscSuWr7Wdx2NmBBvIHPNgxRhyqQUZ3z5RCWIAbJb6AYVLRVJWayShAaKLpfQ9LE3HwVNNGjdQPWU9Ei/irNBRnpA0bAnso6v0CpTN3haM53aRs/VPi+Kd/rb7X9KyFySuT9Zlex7yiOCOMFNlkzcPTwn6rVgsyBcqhhtgD4aTVOX9uAIU8CStIxgKyIF97Y9rc6+ibYUnTQ0vOqO8z5+7DWx8hl9Mek4HcQClBhCy9wrD5R35Y0yBwa+1itTFWVEBL4iNL4UcXp3yc0C0UfWhWt9BivzG973YG//JHQheNvN7F8/T0vF3aqe25YYIPxz8qu6cQyEe/uSGa8WfmVW6zTABCa8/SN8S34K/waaFwcpBkK80HIx4dSW8lqdWfe7ZlJhMS6E3EK8NAP375GfoHpeJjphrhGoW5sjtAssuQ4iqjaKTd5FzjCJh0toK5aVJ1SDpthu1/UcddWlbBdBVu8lOeIMnBPMOHQlMIxD9Iq2/85q5bniJ+uobSxcoCtPhU4uk54mNTGsWh KSgcG4W8 X6Iei3kSU+hLut1TWGKrQjtiNRF60FrVTWF3m30BxITQkCGpC4LXtbETrsLe858tEZuFKwD6g7vV/Dbsu6SH2Yqfn+dSmq2EPO3d4QjyQyvKJDl7ml/YK4p05VceaITbsvV/mz4Bp1zpPQxwzRpXLF+dvoamUiqbi8FTFjyaQ+HsjiTd/zFFucCkNXpIegTokToQEPnWDDzzBMuYIf3UQNobWUf3q65yEAlyDF0RXOvY0DgE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Enable x86 support for handling page faults in an mshare region by redirecting page faults to operate on the mshare mm_struct and vmas contained in it. Some permissions checks are done using vma flags in architecture-specfic fault handling code so the actual vma needed to complete the handling is acquired before calling handle_mm_fault(). Because of this an ARCH_SUPPORTS_MSHARE config option is added. Signed-off-by: Anthony Yznaga --- arch/Kconfig | 3 +++ arch/x86/Kconfig | 1 + arch/x86/mm/fault.c | 37 ++++++++++++++++++++++++++++++++++++- mm/Kconfig | 2 +- 4 files changed, 41 insertions(+), 2 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 6682b2a53e34..32474cdcb882 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1640,6 +1640,9 @@ config HAVE_ARCH_PFN_VALID config ARCH_SUPPORTS_DEBUG_PAGEALLOC bool +config ARCH_SUPPORTS_MSHARE + bool + config ARCH_SUPPORTS_PAGE_TABLE_CHECK bool diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 2e1a3e4386de..453a39098dfa 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -120,6 +120,7 @@ config X86 select ARCH_SUPPORTS_ACPI select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_DEBUG_PAGEALLOC + select ARCH_SUPPORTS_MSHARE if X86_64 select ARCH_SUPPORTS_PAGE_TABLE_CHECK if X86_64 select ARCH_SUPPORTS_NUMA_BALANCING if X86_64 select ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP if NR_CPUS <= 4096 diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index e6c469b323cc..4b55ade61a01 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1217,6 +1217,8 @@ void do_user_addr_fault(struct pt_regs *regs, struct mm_struct *mm; vm_fault_t fault; unsigned int flags = FAULT_FLAG_DEFAULT; + bool is_shared_vma; + unsigned long addr; tsk = current; mm = tsk->mm; @@ -1330,6 +1332,12 @@ void do_user_addr_fault(struct pt_regs *regs, if (!vma) goto lock_mmap; + /* mshare does not support per-VMA locks yet */ + if (vma_is_mshare(vma)) { + vma_end_read(vma); + goto lock_mmap; + } + if (unlikely(access_error(error_code, vma))) { bad_area_access_error(regs, error_code, address, NULL, vma); count_vm_vma_lock_event(VMA_LOCK_SUCCESS); @@ -1358,17 +1366,38 @@ void do_user_addr_fault(struct pt_regs *regs, lock_mmap: retry: + addr = address; + is_shared_vma = false; vma = lock_mm_and_find_vma(mm, address, regs); if (unlikely(!vma)) { bad_area_nosemaphore(regs, error_code, address); return; } + if (unlikely(vma_is_mshare(vma))) { + fault = find_shared_vma(&vma, &addr); + + if (fault) { + mmap_read_unlock(mm); + goto done; + } + + if (!vma) { + mmap_read_unlock(mm); + bad_area_nosemaphore(regs, error_code, address); + return; + } + + is_shared_vma = true; + } + /* * Ok, we have a good vm_area for this memory access, so * we can handle it.. */ if (unlikely(access_error(error_code, vma))) { + if (unlikely(is_shared_vma)) + mmap_read_unlock(vma->vm_mm); bad_area_access_error(regs, error_code, address, mm, vma); return; } @@ -1386,7 +1415,11 @@ void do_user_addr_fault(struct pt_regs *regs, * userland). The return to userland is identified whenever * FAULT_FLAG_USER|FAULT_FLAG_KILLABLE are both set in flags. */ - fault = handle_mm_fault(vma, address, flags, regs); + fault = handle_mm_fault(vma, addr, flags, regs); + + if (unlikely(is_shared_vma) && ((fault & VM_FAULT_COMPLETED) || + (fault & VM_FAULT_RETRY) || fault_signal_pending(fault, regs))) + mmap_read_unlock(mm); if (fault_signal_pending(fault, regs)) { /* @@ -1414,6 +1447,8 @@ void do_user_addr_fault(struct pt_regs *regs, goto retry; } + if (unlikely(is_shared_vma)) + mmap_read_unlock(vma->vm_mm); mmap_read_unlock(mm); done: if (likely(!(fault & VM_FAULT_ERROR))) diff --git a/mm/Kconfig b/mm/Kconfig index ba3dbe31f86a..4fc056bb5643 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1360,7 +1360,7 @@ config PT_RECLAIM config MSHARE bool "Mshare" - depends on MMU + depends on MMU && ARCH_SUPPORTS_MSHARE help Enable msharefs: A ram-based filesystem that allows multiple processes to share page table entries for shared pages. A file