From patchwork Fri Jan 27 19:41:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13119318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3086CC54EAA for ; Fri, 27 Jan 2023 20:03:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=vqVwsdtt0Y761LqGRSioLjRZjclFE8m296e30FqdZH8=; b=V611MfBu9n/SKVEdn3r+1eTyA8 Aed9GHD+LzSdK1N1o9Bv90LKdtDsBEnNfvjpcPJxw5J0zYoZwNSQ6frVnQJ8kyZkqgBUpG1nIgwgH jY5eaFKkC3DEUajv5kYDYYpo5cv6Ij6y1httDBIsmnEIsyzauH5qwviLTR/xwEYYQ2hIVwkmkvEh6 FiXAMrfoQiZOjbbPVtEBfsk7a4TksYVo/LtoFrF6YbYoxVgoFxFkadUKhpHWGtnHo9+pUHnPEf+hk xcW67fa2To3h8BB/7MG69UAlAYj9pifMu8iC0rJX8PZT/mFBUV76hzWdDNhgplXUR5d5e7pDBJeYx kh65r5+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pLUvd-00GGwn-TJ; Fri, 27 Jan 2023 20:02:30 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pLUvb-00GGui-R0 for linux-arm-kernel@bombadil.infradead.org; Fri, 27 Jan 2023 20:02:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=kzxeHm8qCtyYeQADp+m05TsFOqarDXtVrIERGy2x4bE=; b=HzpLwR61iYZxqZzIQGFlNvPQmx HmNhNOi8jeTETFKP8087D0BG35GgunHf71B59y9M3fJ3cJwnExnZG2+8fnh9NO+uzahqEwBUPp2U1 m17xURi48ktaQcE2efWofwi7Cerb7Cl1dSsko8tLooQd0qLZW/0+yjGFSiSBWdyKAs2yQggwME13L Bsbp8G0zXJ8a02qs7PbvJl8TF0DPD9PQ/zso2CY1+2JQvbqCgTiYwk72QWE8ZA19o6Jn/48JgoI15 FL1zIVzucA6JA99rFdCVNfxEf61R3HvbdKkr1zNBDPzGK+4rmDbMlCZFN8+0WGDFIg/RX57YYXDSE /3dcRixQ==; Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by casper.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pLUcI-007zTn-Aa for linux-arm-kernel@lists.infradead.org; Fri, 27 Jan 2023 19:42:32 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-4c11ae6ab25so65260057b3.8 for ; Fri, 27 Jan 2023 11:42:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kzxeHm8qCtyYeQADp+m05TsFOqarDXtVrIERGy2x4bE=; b=Qw/TzqObuxKUrxtkILJdFMJrnRXDl7IhHsfqWYsg15Fht0zJymOStyW6umwGn696mL /vz0L3L+AxNuyNMgLOdykFyo3ffxujr1NJ9PZ/3TCzeykyM3g1PjhFRGoo/e9FoO8fmJ 0gJOei12QFUXaa/Oa6hhy8ENMFQY5aEUa+SKaoSJiwpGU68x3ruM/ipRGHnS2jScV2XA EPWUY+rZwtRmlqA25G6oa8LHcJDQM3bZQ4diO+PFB7fofYZtcMVMGn5EMXtqF1YwdUnI Yur83PIWndlmZU2J2xMV4at0clCg0WXiP4a0wm/tFrIYOxaiWzueFhV10wBQ62TvKtYz 5Efg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kzxeHm8qCtyYeQADp+m05TsFOqarDXtVrIERGy2x4bE=; b=bjhWW+gGx1h8JmqsapwCGX3stqy/Q4b5xLq3JDSDAPBE1AZKrS6B/e9AI6MdlIgJ5T NbYlbfdZjMUqfKTlTWQPQNJ1FmRWDeAYF20dQtML5+y1y+N2lLpgoEBLHCpgA7L3xdwM c2G8zUnmeNRGCpaECd8ZtHfc0Zyd6XjZ3soWuyHE3TKnaD51vANXHx+yGaFT7UxO1KAg 3ionFHNaRf+6tEeeyY3U/Z7lQNp5lU7I6kZCa7+ySAm+A1sWbh8Wfi8TwVHL7uTQ1b8t eanLtsuoAXLIz0UeZRla6sRu26gfVlnGSEuWyUOoHcqu98iEreUuVAvBxlIeO4cdiAJA Uzhg== X-Gm-Message-State: AFqh2krInOJ4ydRsQoFQgR7dImk0V2yBGACiM3Bd26izvhTOFyClfVNa VOXmYmUvu61SUIonl5dwsPltlUl4PbQ= X-Google-Smtp-Source: AMrXdXvuIF+ftjjZVhAq6LLQlzBTotqsaAzARNS7Nb+n90q796c/W7VGIXChRGDMsXog66D2ETjcV07ygFs= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:200:4e19:be9:c5d0:8483]) (user=surenb job=sendgmr) by 2002:a81:1754:0:b0:4fd:ff18:de6c with SMTP id 81-20020a811754000000b004fdff18de6cmr2568712ywx.227.1674848547503; Fri, 27 Jan 2023 11:42:27 -0800 (PST) Date: Fri, 27 Jan 2023 11:41:08 -0800 In-Reply-To: <20230127194110.533103-1-surenb@google.com> Mime-Version: 1.0 References: <20230127194110.533103-1-surenb@google.com> X-Mailer: git-send-email 2.39.1.456.gfc5497dd1b-goog Message-ID: <20230127194110.533103-32-surenb@google.com> Subject: [PATCH v2 31/33] powerc/mm: try VMA lock-based page fault handling first From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, mingo@redhat.com, will@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, rppt@kernel.org, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, leewalsh@google.com, posk@google.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230127_194230_820696_7C172F93 X-CRM114-Status: GOOD ( 13.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Laurent Dufour Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. Copied from "x86/mm: try VMA lock-based page fault handling first" Signed-off-by: Laurent Dufour Signed-off-by: Suren Baghdasaryan --- arch/powerpc/mm/fault.c | 41 ++++++++++++++++++++++++++ arch/powerpc/platforms/powernv/Kconfig | 1 + arch/powerpc/platforms/pseries/Kconfig | 1 + 3 files changed, 43 insertions(+) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 2bef19cc1b98..cab229e75018 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -469,6 +469,44 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, if (is_exec) flags |= FAULT_FLAG_INSTRUCTION; +#ifdef CONFIG_PER_VMA_LOCK + if (!(flags & FAULT_FLAG_USER) || atomic_read(&mm->mm_users) == 1) + goto lock_mmap; + + vma = lock_vma_under_rcu(mm, address); + if (!vma) + goto lock_mmap; + + if (unlikely(access_pkey_error(is_write, is_exec, + (error_code & DSISR_KEYFAULT), vma))) { + int rc = bad_access_pkey(regs, address, vma); + + vma_end_read(vma); + return rc; + } + + if (unlikely(access_error(is_write, is_exec, vma))) { + int rc = bad_access(regs, address); + + vma_end_read(vma); + return rc; + } + + fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); + vma_end_read(vma); + + if (!(fault & VM_FAULT_RETRY)) { + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + goto done; + } + count_vm_vma_lock_event(VMA_LOCK_RETRY); + + if (fault_signal_pending(fault, regs)) + return user_mode(regs) ? 0 : SIGBUS; + +lock_mmap: +#endif /* CONFIG_PER_VMA_LOCK */ + /* When running in the kernel we expect faults to occur only to * addresses in user space. All other faults represent errors in the * kernel and should generate an OOPS. Unfortunately, in the case of an @@ -545,6 +583,9 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, mmap_read_unlock(current->mm); +#ifdef CONFIG_PER_VMA_LOCK +done: +#endif if (unlikely(fault & VM_FAULT_ERROR)) return mm_fault_error(regs, address, fault); diff --git a/arch/powerpc/platforms/powernv/Kconfig b/arch/powerpc/platforms/powernv/Kconfig index ae248a161b43..70a46acc70d6 100644 --- a/arch/powerpc/platforms/powernv/Kconfig +++ b/arch/powerpc/platforms/powernv/Kconfig @@ -16,6 +16,7 @@ config PPC_POWERNV select PPC_DOORBELL select MMU_NOTIFIER select FORCE_SMP + select ARCH_SUPPORTS_PER_VMA_LOCK default y config OPAL_PRD diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig index a3b4d99567cb..e036a04ff1ca 100644 --- a/arch/powerpc/platforms/pseries/Kconfig +++ b/arch/powerpc/platforms/pseries/Kconfig @@ -21,6 +21,7 @@ config PPC_PSERIES select HOTPLUG_CPU select FORCE_SMP select SWIOTLB + select ARCH_SUPPORTS_PER_VMA_LOCK default y config PARAVIRT