From patchwork Mon Aug 12 18:12:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13760908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E5C6C531DC for ; Mon, 12 Aug 2024 18:12:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57F8F6B009E; Mon, 12 Aug 2024 14:12:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 530DA6B009F; Mon, 12 Aug 2024 14:12:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E6A46B00A0; Mon, 12 Aug 2024 14:12:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0F44C6B009E for ; Mon, 12 Aug 2024 14:12:42 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 838E716035A for ; Mon, 12 Aug 2024 18:12:41 +0000 (UTC) X-FDA: 82444388922.19.66794A7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id EC9242001F for ; Mon, 12 Aug 2024 18:12:37 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hqTqk2Ug; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723486306; a=rsa-sha256; cv=none; b=MqDUOrEnwOX0DOCiy+M+uANqfua9JRkQV/AYUx+3KHeIF2yHTaCuI2GMHng8q2c3BUGpF7 qhfRH37UzzeluVS8g+r8iAMxtoY+BmebO/67S2Y2inWzIm0SXLWDGQk/OkUzTJJfNzgH/w Ew9ORRnSR2uROCPwwTzPzlGKy2plhGg= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hqTqk2Ug; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723486306; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wAu0M0+1UWqmiItjD8rL7YnlMQXKv8wW1V5JPENrD98=; b=KYVLRUd4AMHVoGLv3RnNT5SlnuQcyINXavkSrTP4MRa9K4j7E+7BGQF5fBNQ5QAj2PsFQf VDbR3PKCQLz7IoiDJ79A2c4tDdpNWXlnPS04P+q6eyqRjPr4koHyhL0ta9VZMBN2SaPq2I gzIFxMjb7LccJJhMytKmF89PCwfj+H4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723486357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wAu0M0+1UWqmiItjD8rL7YnlMQXKv8wW1V5JPENrD98=; b=hqTqk2UgY/UVvg9dsRH6hq38kv1/HPOoepA1N9p8s3VIpUOE4hdPDxj9O10Ok+4XJAj4Wo 03HvrK4wPpN6IyrOxQt2MdVS3OuD99NXBxrWLa3zP/5VAeFjtPIjurcMjdu4/J7+Im7wQW kxdKbjPMvaub9uUtXigMUrzFZRfELKo= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-682-ZhZR4BwvOUqvLag7JIzsVQ-1; Mon, 12 Aug 2024 14:12:36 -0400 X-MC-Unique: ZhZR4BwvOUqvLag7JIzsVQ-1 Received: by mail-qk1-f199.google.com with SMTP id af79cd13be357-7a1d3c02fcfso18662385a.0 for ; Mon, 12 Aug 2024 11:12:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723486356; x=1724091156; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wAu0M0+1UWqmiItjD8rL7YnlMQXKv8wW1V5JPENrD98=; b=U3SUfTQT4I4VExgESo0voAwzSTwr5FE9GSYMBsE9Zp/ntPM+6CppLkVo0weIpcdxDv rbsMM2TvtBiG39USHdc4xpekQtUdNsA1MwEpMriIwPTOK1UxpZsfl5xwnH+efN/WOQiX YEiM9avnXyYEU7Yar9rdo6YS8fz188SQlfJoJdNwgWVLrNcGhfb+YPERvs5PkhkdR5Mx opUVVHbmfqrFU44mKkamvDZMdv3O1mZeEvWkV06F2OsoCq+Dx530pAf8Gldye6nkj61m zLgVtTCnItSoUUXmcwF8IYyWzBGOomSMF9Z7VA10lg4lseWwuGg+YbcGiCJuqoYLdNiI zszg== X-Forwarded-Encrypted: i=1; AJvYcCVJhzYj7dLax2LpQ6Hj39N2oUw9HU31cGijuC+xSbcq1BqWbWcGJ+iXsH+7bbnG6vZfvoaZWzeD3Ldls5a6Bi8sL6Y= X-Gm-Message-State: AOJu0Yz3E8O4DesW75czdSc12DxLUU50VmBa0B1Y+6yyNX2KsFik8TyO Yxo0PRKFpkZfuprELF5HSM+8tCrKUfggRz98BhkI7cQQcKJsZVDcFw4ia30Qfck+ivB11NjABTf 1Nb/liMFhtevPuTEBHBelhqxRkDXT4CWA5TTZGIMTCX95t++F X-Received: by 2002:a05:620a:4005:b0:79f:84f:80b1 with SMTP id af79cd13be357-7a4e15d46a6mr74880985a.7.1723486355729; Mon, 12 Aug 2024 11:12:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFpNsU7cY2l6XFvfi/NYSnYqbYktrA9n3rcHeLoBtv5kBl17wu9V8UOiRcR1CmKk37Q2l0U8w== X-Received: by 2002:a05:620a:4005:b0:79f:84f:80b1 with SMTP id af79cd13be357-7a4e15d46a6mr74878085a.7.1723486355340; Mon, 12 Aug 2024 11:12:35 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a4c7dee013sm268663985a.84.2024.08.12.11.12.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Aug 2024 11:12:34 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: "Kirill A . Shutemov" , Nicholas Piggin , David Hildenbrand , Matthew Wilcox , Andrew Morton , James Houghton , Huang Ying , "Aneesh Kumar K . V" , peterx@redhat.com, Vlastimil Babka , Rick P Edgecombe , Hugh Dickins , Borislav Petkov , Christophe Leroy , Michael Ellerman , Rik van Riel , Dan Williams , Mel Gorman , x86@kernel.org, Ingo Molnar , linuxppc-dev@lists.ozlabs.org, Dave Hansen , Dave Jiang , Oscar Salvador , Thomas Gleixner Subject: [PATCH v5 3/7] mm/powerpc: Add missing pud helpers Date: Mon, 12 Aug 2024 14:12:21 -0400 Message-ID: <20240812181225.1360970-4-peterx@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240812181225.1360970-1-peterx@redhat.com> References: <20240812181225.1360970-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: EC9242001F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 1c9dytq6urigfry8j1phaw977hjdq1kj X-HE-Tag: 1723486357-821458 X-HE-Meta: U2FsdGVkX19mH6C7TeXI6ZEKCJI6oWBo0rKxDqijk8d2SNvURf2+fII4oOeY2/tzb+TKfqsJOIjoDc6JCROOnZ6iKUdsP37Iar32m0ttVNtzLM5ecY0ky2GR/bQDp+IaTkEqNXUFJ1z8l+J7ZLASvGTYJNlCeo1icMDye1MYo/QWwAnS/c4IHfEC8utueziZ6FNI4SsF+eHyEG3vI6YVEjxnp5j72UNtMVyfGvyfJZbsZEXwbuvygRfMkcCFAsoyj9vQQAfFmzeGlbGiVl9/dxiy7NMJQq3Jy8vnhOR1OmsxNPhvUq6VqAUAg8g1pISUgDDVsaGKa7aquC8qR1rBGPsgH/sB7yHxBmguX1uYQtVJqVbFBI74POq1BLKUOVolMIYMegDBby3VMB/Whhy2GPT76WF3F732CDW1axhBYjG7zaLDkFQc/3DUIZWgZ8Om87+82jT1z317k8slHZlJ1kUcTR92lJVpil2ZpwmL7KCRB9lMjCBrY/3/hJqgXAHCL/Vi8i7Y0bNDdlxytV0V1amogWQLWId3+Fc/QoRPxGCeOcbVD1D4XmjdKuQQaIU0B90csWzAU46qk0b8Fyn0nRz/xH9IetybeGQbAY/AzxW5rGls+XAU8tetxxjx9iqspUAE7tCR+Y8tHD8nNVBbs+yqjzR3WnlXKoM1p0uGggtbKoMux41IvHeJzSjtFo2mc+Dpn4ce0BClHgTNtgUKQRzVr28mgWS/8Usfm/mv0U7Z/BygXDVnhDDIWdWgCmMp5N9BONCm4Ql6WWY1lOwnYLgUifXEdxC/7m3tk2Bwo3Tc6hDO2hOcR/NjNaWxufwLMI8WmyuL6FWq76brrrc5hqiINa8QEwyo6/lCPgYn3dUsjYGwCpYcfZebtdtvGh9aEaZAWuXeyqe/4AMoIdJ5THIC/NHrmO0lROxkAwpm4b0xIBlkEU69e0ijWrM8HZOuUWriMID67q81VD833n3 Iqh4dKVQ t8FoDpDEr3JkCDYg5IIowpg9QgXYHuK2WSSbBQj0q6cQlb7FK8zCzb2kEH/c/dM0sskIxp0aE3X3bFGD5Ogm5gBKn3EUndtzO5ZGfTzBkuGoSwJQBh4tXOmbocDq9hFyHeOtJVRZ0QEWt6tXElS3M9BZ1QUMz/uKp/iaHStY0pb31D+CSVjS9bKL2QSLAEvAArc2bLEV5vrQDG+EtTlNSsMldO6Q5dIBM1kZnuMmdZWdWFSxkXuQ6+/OV79mDENwKmxoZHBCLn2XfunTnvNVQNTUZ/Ogq+qnYk581LDHCoV3fv+QXmF6GAE6/zOhd2vLwPnT74R+JM4UFbGbAUqDpOrORmGvzV49p4GqP+xBNPXBXSUOCQR8rwIq2X9hHTgtduQqbZauYWa50+S7JvJgy6VUS2cYOq5M78GSn7IbaWaw7f2C3v/zqWuenwdul5UvENcQgEPYkq8OsDwinPc2FfUYRAcZQcRZhpN61G/rW6kYW6io/IZwPS+tPKbstDZotUPBHifzASGwg31TKZzU1TnUrMpOsHsNxZjhTvieIE/2CKLl+cjrTZMEfEFjuVPDKyP8F/pbJ3ERc86jrQAN/0y0moQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some new helpers will be needed for pud entry updates soon. Introduce these helpers by referencing the pmd ones. Namely: - pudp_invalidate(): this helper invalidates a huge pud before a split happens, so that the invalidated pud entry will make sure no race will happen (either with software, like a concurrent zap, or hardware, like a/d bit lost). - pud_modify(): this helper applies a new pgprot to an existing huge pud mapping. For more information on why we need these two helpers, please refer to the corresponding pmd helpers in the mprotect() code path. Cc: Michael Ellerman Cc: Nicholas Piggin Cc: Christophe Leroy Cc: linuxppc-dev@lists.ozlabs.org Cc: Aneesh Kumar K.V Signed-off-by: Peter Xu --- arch/powerpc/include/asm/book3s/64/pgtable.h | 3 +++ arch/powerpc/mm/book3s64/pgtable.c | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 519b1743a0f4..5da92ba68a45 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1124,6 +1124,7 @@ extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot); extern pud_t pfn_pud(unsigned long pfn, pgprot_t pgprot); extern pmd_t mk_pmd(struct page *page, pgprot_t pgprot); extern pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot); +extern pud_t pud_modify(pud_t pud, pgprot_t newprot); extern void set_pmd_at(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, pmd_t pmd); extern void set_pud_at(struct mm_struct *mm, unsigned long addr, @@ -1384,6 +1385,8 @@ static inline pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, #define __HAVE_ARCH_PMDP_INVALIDATE extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, pmd_t *pmdp); +extern pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp); #define pmd_move_must_withdraw pmd_move_must_withdraw struct spinlock; diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c index f4d8d3c40e5c..5a4a75369043 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -176,6 +176,17 @@ pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, return __pmd(old_pmd); } +pud_t pudp_invalidate(struct vm_area_struct *vma, unsigned long address, + pud_t *pudp) +{ + unsigned long old_pud; + + VM_WARN_ON_ONCE(!pud_present(*pudp)); + old_pud = pud_hugepage_update(vma->vm_mm, address, pudp, _PAGE_PRESENT, _PAGE_INVALID); + flush_pud_tlb_range(vma, address, address + HPAGE_PUD_SIZE); + return __pud(old_pud); +} + pmd_t pmdp_huge_get_and_clear_full(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmdp, int full) { @@ -259,6 +270,15 @@ pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot) pmdv &= _HPAGE_CHG_MASK; return pmd_set_protbits(__pmd(pmdv), newprot); } + +pud_t pud_modify(pud_t pud, pgprot_t newprot) +{ + unsigned long pudv; + + pudv = pud_val(pud); + pudv &= _HPAGE_CHG_MASK; + return pud_set_protbits(__pud(pudv), newprot); +} #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /* For use by kexec, called with MMU off */