From patchwork Tue Mar 8 21:34:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 12774395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8713DC433EF for ; Tue, 8 Mar 2022 21:35:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A2EC8D0009; Tue, 8 Mar 2022 16:35:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 12B8C8D0001; Tue, 8 Mar 2022 16:35:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBFF18D0009; Tue, 8 Mar 2022 16:34:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id DC3968D0001 for ; Tue, 8 Mar 2022 16:34:59 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8DBBCA759C for ; Tue, 8 Mar 2022 21:34:59 +0000 (UTC) X-FDA: 79222524318.28.3D2EA3F Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf03.hostedemail.com (Postfix) with ESMTP id F2E512000C for ; Tue, 8 Mar 2022 21:34:58 +0000 (UTC) Received: by mail-pf1-f201.google.com with SMTP id 127-20020a620685000000b004f6eaf868easo265436pfg.22 for ; Tue, 08 Mar 2022 13:34:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=DbXdz/QNU2FalrGjiQ9AEtwD/eVZwmYdtIsI67Y/vrQ=; b=H0+9KgUA7b5DaGQ5trxXk7aCYPyhuPtPM/vrIRcW7jDStpS1S6cbo569qrOF+RBBCu ycp7o9uYddCPrjflOfFtltQ8OnIM85gaYtIyKNpei4nwL90hD4u2EFFZSM2L3t0D6sMN zTtmOix6EO44ju5jLs6OxLt5cZ8wpaFJ+Pv7Erui1glEdgscjz3TvrXZYhRe1wi+rtba a1nxEv8oOKbiq2gYJ9iCbO21s1i8Jrf43i2dhl69qRy3+umHuSU4Vx+WHnF92DtcVHzP SddqZy6CTJyC2+fII5NDMQftuZ1dBwAUINS3l2TPxps5lGH1y+xbUUPPva2SYvB5Cdla LSig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DbXdz/QNU2FalrGjiQ9AEtwD/eVZwmYdtIsI67Y/vrQ=; b=r5FQeMHkrFpdAojaN64afAMjEH3tHRYr0NVN5USHM+mJ8e7vNFSFnkoMk6hMNJ6rsO VFRsuFKXjVIH/HBZ2G8RVifbsVVUHus8tH/IHFhK8nn8c4JTdMAxYo3TqIXhtdEc4g4I ziOMc6z6aF3XPgy7YSgLundCI1GaunwhP4PJJvbhcGqGCWrLSs9cL+6Z2e4iIhovX2VB T1Cl23ibISS6mFWbaFQNsfbFkQtvrRQpsYJLq8uarMDk4RBG7o6zy+I6WGCpD4VhmDOW PoPdDKbKUFuNibOxXmRRQ+rZpK2RnRZMCdGuDNjNoRFtPExMH6BHWb46IzM3453nbGri 0jYQ== X-Gm-Message-State: AOAM53082V+KUz99/FhuN2YSIICcls1NNVP+c/+e1q6v+LfbL+8L9nwl S36St5kHFX7yo54dtQygbVHIECC6H1AA X-Google-Smtp-Source: ABdhPJy7RCgUr3XsGlQcMcqkd2uK7AmGUdabvjDfGVr64UpdBPBh0ZjDJR2D+X2PKEILFF+7oqFgKTkpthHA X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:90b:1e10:b0:1bf:6c78:54a9 with SMTP id pg16-20020a17090b1e1000b001bf6c7854a9mr927850pjb.1.1646775297572; Tue, 08 Mar 2022 13:34:57 -0800 (PST) Date: Tue, 8 Mar 2022 13:34:09 -0800 In-Reply-To: <20220308213417.1407042-1-zokeefe@google.com> Message-Id: <20220308213417.1407042-7-zokeefe@google.com> Mime-Version: 1.0 References: <20220308213417.1407042-1-zokeefe@google.com> X-Mailer: git-send-email 2.35.1.616.g0bdcbb4464-goog Subject: [RFC PATCH 06/14] mm/khugepaged: add hugepage_vma_revalidate_pmd_count() From: "Zach O'Keefe" To: Alex Shi , David Hildenbrand , David Rientjes , Michal Hocko , Pasha Tatashin , SeongJae Park , Song Liu , Vlastimil Babka , Zi Yan , linux-mm@kvack.org Cc: Andrea Arcangeli , Andrew Morton , Arnd Bergmann , Axel Rasmussen , Chris Kennelly , Chris Zankel , Helge Deller , Hugh Dickins , Ivan Kokshaysky , "James E.J. Bottomley" , Jens Axboe , "Kirill A. Shutemov" , Matthew Wilcox , Matt Turner , Max Filippov , Miaohe Lin , Minchan Kim , Patrick Xia , Pavel Begunkov , Peter Xu , Richard Henderson , Thomas Bogendoerfer , Yang Shi , "Zach O'Keefe" X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: F2E512000C X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=H0+9KgUA; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of 3AcwnYgcKCFAH62wwxwy66y3w.u64305CF-442Dsu2.69y@flex--zokeefe.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3AcwnYgcKCFAH62wwxwy66y3w.u64305CF-442Dsu2.69y@flex--zokeefe.bounces.google.com X-Stat-Signature: gbzpc89qcw5ohnbp47kbapyum3xgtsm9 X-HE-Tag: 1646775298-541287 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: madvise collapse context operates on pmds in batch. We will want to be able to revalidate a region that spans multiple pmds in the same vma. Add hugepage_vma_revalidate_pmd_count() which extends hugepage_vma_revalidate() with number of pmds to revalidate. hugepage_vma_revalidate() now calls through this. Signed-off-by: Zach O'Keefe --- mm/khugepaged.c | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 56f2ef7146c7..1d20be47bcea 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -964,18 +964,17 @@ khugepaged_alloc_page(struct page **hpage, gfp_t gfp, int node) #endif /* - * If mmap_lock temporarily dropped, revalidate vma - * before taking mmap_lock. - * Return 0 if succeeds, otherwise return none-zero - * value (scan code). + * Revalidate a vma's eligibility to collapse nr hugepages. */ - -static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, - struct vm_area_struct **vmap) +static int hugepage_vma_revalidate_pmd_count(struct mm_struct *mm, + unsigned long address, int nr, + struct vm_area_struct **vmap) { struct vm_area_struct *vma; unsigned long hstart, hend; + mmap_assert_locked(mm); + if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -985,7 +984,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; hend = vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (address < hstart || (address + nr * HPAGE_PMD_SIZE) > hend) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; @@ -995,6 +994,17 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, return 0; } +/* + * If mmap_lock temporarily dropped, revalidate vma before taking mmap_lock. + * Return 0 if succeeds, otherwise return none-zero value (scan code). + */ + +static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, + struct vm_area_struct **vmap) +{ + return hugepage_vma_revalidate_pmd_count(mm, address, 1, vmap); +} + /* * Bring missing pages in from swap, to complete THP collapse. * Only done if khugepaged_scan_pmd believes it is worthwhile.