From patchwork Wed Jul 20 14:06:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 12924027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29BF5C43334 for ; Wed, 20 Jul 2022 14:06:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E42F6B0073; Wed, 20 Jul 2022 10:06:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 794168E0001; Wed, 20 Jul 2022 10:06:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 635BA6B0075; Wed, 20 Jul 2022 10:06:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 530EE6B0073 for ; Wed, 20 Jul 2022 10:06:22 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 300991205B1 for ; Wed, 20 Jul 2022 14:06:22 +0000 (UTC) X-FDA: 79707653004.04.0AE7F26 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf15.hostedemail.com (Postfix) with ESMTP id AAD10A008D for ; Wed, 20 Jul 2022 14:06:21 +0000 (UTC) Received: by mail-pj1-f73.google.com with SMTP id pg9-20020a17090b1e0900b001f076f7d15eso1270981pjb.5 for ; Wed, 20 Jul 2022 07:06:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=eS/0TDZ57j7zzZRooTjg5w5BNN07jR34xInjcUrqOSk=; b=sHOjDbjyuIQdDcTE8Obi8hnGKxuPEk2x/fZ9V413IAVimFiMjgOt4Mfkg2wYlz9Z6W c7uaSLW8wOF4YKCmqaxbNne7KBBtZapmQPYM3Fy2TvpVwm1cbzC2x+qMtyffSNyR4bdk lO5FRYjG9XQiKczz96mv96vBRT8ZraI4F5sTQrWR/gHS3Ykh/JOXpb89EnYgU4ZhO1Ie 7j3fCjF+C3nFp+ahikRYuPf87uuMVHnOYnwoFl/AHSmgCTnEG7pXMJZ3kA6FIizoygYu AGL44aOXGdum2qYpMUpdTB0y6myUh+l4SIvuFFDJ58NBt/i6BeDwu/VtO1MD4yLd/bVG Sfbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=eS/0TDZ57j7zzZRooTjg5w5BNN07jR34xInjcUrqOSk=; b=UNUq0+/ooX2D4IJD4kz+jFDrjBTrh7SPQh2DwHvHAXt+1riXQVww1jgYuqt/bFCZIZ WLmuf1pZ95/GYMLpXCznqhf005+dV8kqfNRSrLM2OxI4HdJdEh3qfE0wMxfBx2R9rXOu z5+z1z49VuutyCGsBEZ2dTSFAQcF+vSygGeNlBtihINSIpvyfsgowp1N3kTR9L4Soijp EBdarzcCMGwhPXXvBbI9m7A4XrJkMfmCt2CRm3t/2yqsadcYcfc6QUy6vF5lF5GB82V2 bOb0ztLNLLvx7D6njYt/9oevbTAPkP3GDj4S+TJHVcF72p9wlQHirFzi2vBDaSEKJm0P WjDg== X-Gm-Message-State: AJIora95xyKke/PE89bDiy5XNOIGWFG50BTAc6wvdMYO9ORayks+Zrng ZlZT8fhWfa3MVVorTPUOO19OSFM16SLu X-Google-Smtp-Source: AGRyM1vUJiy+ekO60jWK2XBEm40biWIDU7lblvBsX8GwFInAgpAo9b7s6iC67yR9zxZ+yJHm/Oy7Nq1KFFpJ X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:902:b198:b0:16c:1c13:cd8d with SMTP id s24-20020a170902b19800b0016c1c13cd8dmr38712413plr.92.1658325980700; Wed, 20 Jul 2022 07:06:20 -0700 (PDT) Date: Wed, 20 Jul 2022 07:06:00 -0700 In-Reply-To: <20220720140603.1958773-1-zokeefe@google.com> Message-Id: <20220720140603.1958773-2-zokeefe@google.com> Mime-Version: 1.0 References: <20220720140603.1958773-1-zokeefe@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH mm-unstable 1/4] mm/khugepaged: Use minimal bits to store num page < HPAGE_PMD_NR From: "Zach O'Keefe" To: Andrew Morton Cc: linux-mm@kvack.org, Hugh Dickins , Yang Shi , Miaohe Lin , "Zach O'Keefe" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658325981; a=rsa-sha256; cv=none; b=YIPCXdYXOYuqnQlEuuEu4xry8IVhvXRH207KGfVhUDKWzcfBlwM6bZuciZzhkmIU+xjK/v Vby/6GPji78NtiCkCAZDF6KtT6c8rA/PMVuEDTmR2VMCoL4WtYr2xmFLCpkbcDiz2xhBjt 2tkkI62Np6wXf+htz47d2DL6Rl0TPjE= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sHOjDbjy; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf15.hostedemail.com: domain of 33AvYYgcKCHEodZTTUTVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--zokeefe.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=33AvYYgcKCHEodZTTUTVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--zokeefe.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658325981; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eS/0TDZ57j7zzZRooTjg5w5BNN07jR34xInjcUrqOSk=; b=3SMWKjegtd4VgT1ckojQ/LopZF64fCmXfEcrMItkme6fDr4w5SWUeCDCeDJv4tAF633Pgg C2VPfnUl0fDKVpAZ1Z2NEy8O7RtMDS29FcJ+hyVXZiEHe1R32bA3jVjCcqLKz3mg18YSw+ uYiWA3J7oI1rP8gM+RZcC599AFYG2jA= X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sHOjDbjy; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf15.hostedemail.com: domain of 33AvYYgcKCHEodZTTUTVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--zokeefe.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=33AvYYgcKCHEodZTTUTVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--zokeefe.bounces.google.com X-Stat-Signature: ygd1asa4t47k9x4a8dupf1cq7ta4dczh X-Rspamd-Queue-Id: AAD10A008D X-Rspamd-Server: rspam02 X-HE-Tag: 1658325981-254083 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Minimally, node_load[] entries just need to be able to hold the maximum value of HPAGE_PMD_NR, which is compile-time defined per-arch based on PMD_SHIFT and PAGE_SHIFT. node_load[] is only written either via memset(), or with via post-increment. struct collapse_control may be allocated via kmalloc() in other collapse contexts, and MAX_NUMNODES may be arbitrarily large. #define the underlying type of node_load[] based off HPAGE_PMD_NR to avoid excessive memory allocated for this struct. Fixes: 3b07f3bb225a ("mm/khugepaged: add struct collapse_control") Link: https://lore.kernel.org/linux-mm/Ys2CeIm%2FQmQwWh9a@google.com/ Signed-off-by: Zach O'Keefe Reviewed-by: Yang Shi Acked-by: David Rientjes --- mm/khugepaged.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 69990dacde14..ecd28bfeab60 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -92,8 +92,11 @@ struct collapse_control { bool is_khugepaged; /* Num pages scanned per node */ - int node_load[MAX_NUMNODES]; - +#if HPAGE_PMD_ORDER < 16 + u16 node_load[MAX_NUMNODES]; +#else + u32 node_load[MAX_NUMNODES]; +#endif /* Last target selected in hpage_collapse_find_target_node() */ int last_target_node; }; From patchwork Wed Jul 20 14:06:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 12924028 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3402C43334 for ; Wed, 20 Jul 2022 14:06:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B4DB6B0074; Wed, 20 Jul 2022 10:06:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17A138E0001; Wed, 20 Jul 2022 10:06:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F37046B0078; Wed, 20 Jul 2022 10:06:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E4A656B0074 for ; Wed, 20 Jul 2022 10:06:24 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C12C140601 for ; Wed, 20 Jul 2022 14:06:24 +0000 (UTC) X-FDA: 79707653088.08.17099E6 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf04.hostedemail.com (Postfix) with ESMTP id 4E73D40067 for ; Wed, 20 Jul 2022 14:06:23 +0000 (UTC) Received: by mail-pl1-f201.google.com with SMTP id c18-20020a170903235200b0016c37f6d48cso10513359plh.19 for ; Wed, 20 Jul 2022 07:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=nvt0g38UPWQARVq2jIGTphYoZFkzkKWnTTwKEnIQhfQ=; b=kOfoDGHB1Q9BQsPHDaoPg/ATquyXpM9RonSrVrlCbITcTc52Ft5gfFhXSXVsKGtpT8 PhvYIniMH9RTaBk+gFr5ZlU2V0cu2Da5rBmxfSBpmP02wNvp/9j+KcFAu5np5Rt+i/Kd xYjSTLqlMYENAD1F1oFsJF6+xoXfLlg6UuEm39SQfb5lwoDhe9BW8yYDRHaZBgUBHvNd QWxM+NZKxTEhRO5sM5TXM1BMUri6U6YuRtfi8/Gtpzwg7XjNg3FC/H13Wxipk8+iRh0O XHcLbUy8sAMMw5fe/phEkIW90Z+SF7FeYMKlWrFpvUeFVL0ZZggSwwEfDwLwDDuhB6/i uImA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=nvt0g38UPWQARVq2jIGTphYoZFkzkKWnTTwKEnIQhfQ=; b=kWpr30ATvO1MAOREiW21rtH5ow8/tgbxYSJIZB4wXYCBUwiCulDMoeBYjCzt96JGT1 3C8pg3EDiiIXBESEOEjC7DqbKXsXpN4sxRZusSDkYxfsCJFpAcWZ6krB5LFJEcBWJVpp JNYuimCl7e+qSsWtVB/o3H3EOw7vNcLZ7NMEH5d18hXSXXI/CnnR0K6+XXEFUgzCYzhH EBFjluR8t/mNbCXZS8+iWU0YlI8Y7vLH0JoJ4azvVJFhGwMVTeNey/DecT4wzRWpm0FL eWUsHa2+nJVzxICuBpB3cfNyyJC+aSKzGtN2H/XcQpdQeHghL7zot0P2+tenkHgtT5Id i6wQ== X-Gm-Message-State: AJIora+cr1wDPFROzLoVI+BiTAtepQlnD0w2Qte04KaOddtiT0MLTNql FH0wLiyzKvNJqjpTYzMaMQPUQslZgdRQ X-Google-Smtp-Source: AGRyM1t2zdRRt9DeLnkDsuT/2qqMN51waS6QwunsePs8o1uAuGPXyJWhmN0DJLPTbGeYSCqe/mK6qeN6fKZD X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:902:ab95:b0:16d:12a8:2af1 with SMTP id f21-20020a170902ab9500b0016d12a82af1mr7385488plr.164.1658325982447; Wed, 20 Jul 2022 07:06:22 -0700 (PDT) Date: Wed, 20 Jul 2022 07:06:01 -0700 In-Reply-To: <20220720140603.1958773-1-zokeefe@google.com> Message-Id: <20220720140603.1958773-3-zokeefe@google.com> Mime-Version: 1.0 References: <20220720140603.1958773-1-zokeefe@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH mm-unstable 2/4] mm/khugepaged: consistently order cc->is_khugepaged and pte_* checks From: "Zach O'Keefe" To: Andrew Morton Cc: linux-mm@kvack.org, Hugh Dickins , Yang Shi , Miaohe Lin , "Zach O'Keefe" ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kOfoDGHB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 33gvYYgcKCHMqfbVVWVXffXcV.TfdcZelo-ddbmRTb.fiX@flex--zokeefe.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=33gvYYgcKCHMqfbVVWVXffXcV.TfdcZelo-ddbmRTb.fiX@flex--zokeefe.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658325983; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nvt0g38UPWQARVq2jIGTphYoZFkzkKWnTTwKEnIQhfQ=; b=ekBUW7EnL1JSdnr36lxPyVDwVnT8T7YBrn+1NK2PWaZ8n0XENrhWayTwmdzX+bqDxYsAU9 2mAP0keO0WfvO/MRqJFYfcw6CUrcdRCa7AJlTiHc8gz3NNTA+lClIhsaI15nfZXNosTkuD 6UdW4ZfzyfhQah70x5LNTvvJSQ7TnJs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658325983; a=rsa-sha256; cv=none; b=rk1oR9Sj3t1Hw4AmH+cCMxpvbJe6utBIyXNXus8h4PI60dHCWhEPn625mdYpq0I0S2GK8X jokTz9l7WBrNjjCmeSxjzJF030jh+BwpbwuqD2pDX22dGbINxtAvruYitFz81U1fFZaH/0 b4l61zZoTkWbpe5974zVh4us2gKAXuM= X-Rspamd-Queue-Id: 4E73D40067 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kOfoDGHB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 33gvYYgcKCHMqfbVVWVXffXcV.TfdcZelo-ddbmRTb.fiX@flex--zokeefe.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=33gvYYgcKCHMqfbVVWVXffXcV.TfdcZelo-ddbmRTb.fiX@flex--zokeefe.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: db8py9to9mtrs1ost74wmfrm7rzx518a X-HE-Tag: 1658325983-877853 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: cc->is_khugepaged is used to predicate the khugepaged-only behavior of enforcing khugepaged heuristics limited by the sysfs knobs khugepaged_max_ptes_[none|swap|shared]. In branches where khugepaged_max_ptes_* is checked, consistently check cc->is_khugepaged first. Also, local counters (for comparison vs khugepaged_max_ptes_* limits) were previously incremented in the comparison expression. Some of these counters (unmapped) are additionally used outside of khugepaged_max_ptes_* enforcement, and all counters are communicated in tracepoints. Move the correct accounting of these counters before branching statements to avoid future errors due to C's short-circuiting evaluation. Fixes: 9fab4752a181 ("mm/khugepaged: add flag to predicate khugepaged-only behavior") Link: https://lore.kernel.org/linux-mm/Ys2qJm6FaOQcxkha@google.com/ Signed-off-by: Zach O'Keefe Reviewed-by: Yang Shi Acked-by: David Rientjes --- mm/khugepaged.c | 49 +++++++++++++++++++++++++++++-------------------- 1 file changed, 29 insertions(+), 20 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ecd28bfeab60..290422577172 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -574,9 +574,10 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, pte_t pteval = *_pte; if (pte_none(pteval) || (pte_present(pteval) && is_zero_pfn(pte_pfn(pteval)))) { + ++none_or_zero; if (!userfaultfd_armed(vma) && - (++none_or_zero <= khugepaged_max_ptes_none || - !cc->is_khugepaged)) { + (!cc->is_khugepaged || + none_or_zero <= khugepaged_max_ptes_none)) { continue; } else { result = SCAN_EXCEED_NONE_PTE; @@ -596,11 +597,14 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma, VM_BUG_ON_PAGE(!PageAnon(page), page); - if (cc->is_khugepaged && page_mapcount(page) > 1 && - ++shared > khugepaged_max_ptes_shared) { - result = SCAN_EXCEED_SHARED_PTE; - count_vm_event(THP_SCAN_EXCEED_SHARED_PTE); - goto out; + if (page_mapcount(page) > 1) { + ++shared; + if (cc->is_khugepaged && + shared > khugepaged_max_ptes_shared) { + result = SCAN_EXCEED_SHARED_PTE; + count_vm_event(THP_SCAN_EXCEED_SHARED_PTE); + goto out; + } } if (PageCompound(page)) { @@ -1170,8 +1174,9 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, _pte++, _address += PAGE_SIZE) { pte_t pteval = *_pte; if (is_swap_pte(pteval)) { - if (++unmapped <= khugepaged_max_ptes_swap || - !cc->is_khugepaged) { + ++unmapped; + if (!cc->is_khugepaged || + unmapped <= khugepaged_max_ptes_swap) { /* * Always be strict with uffd-wp * enabled swap entries. Please see @@ -1189,9 +1194,10 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, } } if (pte_none(pteval) || is_zero_pfn(pte_pfn(pteval))) { + ++none_or_zero; if (!userfaultfd_armed(vma) && - (++none_or_zero <= khugepaged_max_ptes_none || - !cc->is_khugepaged)) { + (!cc->is_khugepaged || + none_or_zero <= khugepaged_max_ptes_none)) { continue; } else { result = SCAN_EXCEED_NONE_PTE; @@ -1221,12 +1227,14 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, goto out_unmap; } - if (cc->is_khugepaged && - page_mapcount(page) > 1 && - ++shared > khugepaged_max_ptes_shared) { - result = SCAN_EXCEED_SHARED_PTE; - count_vm_event(THP_SCAN_EXCEED_SHARED_PTE); - goto out_unmap; + if (page_mapcount(page) > 1) { + ++shared; + if (cc->is_khugepaged && + shared > khugepaged_max_ptes_shared) { + result = SCAN_EXCEED_SHARED_PTE; + count_vm_event(THP_SCAN_EXCEED_SHARED_PTE); + goto out_unmap; + } } page = compound_head(page); @@ -1961,8 +1969,9 @@ static int khugepaged_scan_file(struct mm_struct *mm, struct file *file, continue; if (xa_is_value(page)) { + ++swap; if (cc->is_khugepaged && - ++swap > khugepaged_max_ptes_swap) { + swap > khugepaged_max_ptes_swap) { result = SCAN_EXCEED_SWAP_PTE; count_vm_event(THP_SCAN_EXCEED_SWAP_PTE); break; @@ -2013,8 +2022,8 @@ static int khugepaged_scan_file(struct mm_struct *mm, struct file *file, rcu_read_unlock(); if (result == SCAN_SUCCEED) { - if (present < HPAGE_PMD_NR - khugepaged_max_ptes_none && - cc->is_khugepaged) { + if (cc->is_khugepaged && + present < HPAGE_PMD_NR - khugepaged_max_ptes_none) { result = SCAN_EXCEED_NONE_PTE; count_vm_event(THP_SCAN_EXCEED_NONE_PTE); } else { From patchwork Wed Jul 20 14:06:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 12924029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD8FFC433EF for ; Wed, 20 Jul 2022 14:06:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB1D86B0075; Wed, 20 Jul 2022 10:06:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D609E6B0078; Wed, 20 Jul 2022 10:06:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C013E6B007B; Wed, 20 Jul 2022 10:06:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A72FE6B0075 for ; Wed, 20 Jul 2022 10:06:25 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8200E160570 for ; Wed, 20 Jul 2022 14:06:25 +0000 (UTC) X-FDA: 79707653130.20.EB6FABC Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf11.hostedemail.com (Postfix) with ESMTP id 0C4044009A for ; Wed, 20 Jul 2022 14:06:24 +0000 (UTC) Received: by mail-pg1-f201.google.com with SMTP id q12-20020a632a0c000000b00419b66851e8so8248608pgq.3 for ; Wed, 20 Jul 2022 07:06:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7NJSO06s3TRWUOwwWKGEkKcgu74469OKtlVGQrWK7PU=; b=lihTN5FIZmm4BKH9BW6zoLNDSypmXTI4lx2hMxm4pmCPYvViIOStDc9VYW2ctlkHzk baXAgJ6fJvws8YtpoQpl5l+2qLPcPwIyU3UJqG1KKYFuViVXMrdGx21hBe0+BXWqE/BL NfYpFXRmCD9rKSju+yVFpY+RhqPA9n0UG/ta1HPKWHDrT7yD8lxyXeam/LZw/8cr+bTT cL9JwIZj26vybK7WW9R/Z8kwO80aXIwYHkWpEP10iMljbVJdB1qTmUK7LOouyn8cgFFN SbmfkhBilIoTKcB8RYVed0N0bn1Gbl9FoeeAXBRq3o90+9NnuDx5LDmhj0sTg3/8Q6Pa GzxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7NJSO06s3TRWUOwwWKGEkKcgu74469OKtlVGQrWK7PU=; b=tFYnUAhhsJb7Nbq8jTDEFDjiBNSUug7JcHPrZpU7beBEyyL7b27LpPaClQQRRo+N4K Nx3YZEP3DM2vseNY7NbdCjduIf+Oj2+BdTkMici+bS0Rr0wUG8UGpPY2IT+SORT9dlbz W/zLUC+IZ/kPrlHu6eX8hVzmFypOsy6uI1NEoRqdPfpTnHH47AJsnMeW0QE+kTkfXH8k iv9AeDTz42rfRVqtFZKvPv7FFGhhV1wXR1vcQ2mcw1gioxTCkflRLbPA1OklsZ8AYBq1 I+2pcsEOqQf6bbYs6O6pI7VDigim4VvXytjGnp1uIzipoO6AHFqyiMhk7MFAdFPMvcXe 4ECQ== X-Gm-Message-State: AJIora81beytuQO8ZaxPwbhiGM+Ox/3MIUZa3jsVFD+vXTjzjlsGqw5Q kpx18aFMykEURCf1V2j1/iC3ec85USXa X-Google-Smtp-Source: AGRyM1sAiGqSywWL4I1MBvo8Kmqz6aP+4UdYwbHe5/QZXCJHTBV7tn+UKtPve20i2aFgq52Z8z9uEPrabnep X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a17:902:da8e:b0:16c:56d7:e00e with SMTP id j14-20020a170902da8e00b0016c56d7e00emr39720943plx.91.1658325983976; Wed, 20 Jul 2022 07:06:23 -0700 (PDT) Date: Wed, 20 Jul 2022 07:06:02 -0700 In-Reply-To: <20220720140603.1958773-1-zokeefe@google.com> Message-Id: <20220720140603.1958773-4-zokeefe@google.com> Mime-Version: 1.0 References: <20220720140603.1958773-1-zokeefe@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH mm-unstable 3/4] mm/khugepaged: delay computation of hpage boundaries until use From: "Zach O'Keefe" To: Andrew Morton Cc: linux-mm@kvack.org, Hugh Dickins , Yang Shi , Miaohe Lin , "Zach O'Keefe" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658325985; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7NJSO06s3TRWUOwwWKGEkKcgu74469OKtlVGQrWK7PU=; b=qBawD4lA7WUHcrvwowi/CI2bj7fHGKGr0nbI7pHXcJGLbM0vcdDLglhojP6qfkf7T8i3EZ oFDjSHhBVMY+ij6av0IKGb9EPkmwKle2zgpikJWqFqjysRxEimJMcceSlnO+qraagt75NV ZlXykL5O8cVES6poUAwWFiP3Qm2cD/k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658325985; a=rsa-sha256; cv=none; b=xqo+1JAxJVYnnUQ7Jnadhlx7jLFm6Y6hJgCx9nBMIKvOTcEKSXLa8QZXYUzKpJV09iN9os Zd9774cB1Gqy2wvfa6IyOhJ1rkdH71ze3tYSkPg3e2SW0QFCM/kI+j0AvqxGBNNC3SHxdL JIRouEAjY0AL/pAVzP6FnC5yJ8NtWdk= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lihTN5FI; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 33wvYYgcKCHQrgcWWXWYggYdW.Ugedafmp-eecnSUc.gjY@flex--zokeefe.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=33wvYYgcKCHQrgcWWXWYggYdW.Ugedafmp-eecnSUc.gjY@flex--zokeefe.bounces.google.com X-Rspamd-Queue-Id: 0C4044009A Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lihTN5FI; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 33wvYYgcKCHQrgcWWXWYggYdW.Ugedafmp-eecnSUc.gjY@flex--zokeefe.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=33wvYYgcKCHQrgcWWXWYggYdW.Ugedafmp-eecnSUc.gjY@flex--zokeefe.bounces.google.com X-Rspamd-Server: rspam12 X-Rspam-User: X-Stat-Signature: pjg86xoqn5pamisz6p8gx9k5ukteirun X-HE-Tag: 1658325984-742226 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Only compute hstart/hend once we've passed all checks that would cause early return in madvise_collapse(). Fixes: c9d968ffd9ba ("mm/madvise: introduce MADV_COLLAPSE sync hugepage collapse") Signed-off-by: Zach O'Keefe Reviewed-by: Yang Shi Acked-by: David Rientjes --- mm/khugepaged.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 290422577172..70e9d9950415 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2417,9 +2417,6 @@ int madvise_collapse(struct vm_area_struct *vma, struct vm_area_struct **prev, if (!vma->anon_vma || !vma_is_anonymous(vma)) return -EINVAL; - hstart = (start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = end & HPAGE_PMD_MASK; - if (!hugepage_vma_check(vma, vma->vm_flags, false, false, false)) return -EINVAL; @@ -2432,6 +2429,9 @@ int madvise_collapse(struct vm_area_struct *vma, struct vm_area_struct **prev, mmgrab(mm); lru_add_drain_all(); + hstart = (start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; + hend = end & HPAGE_PMD_MASK; + for (addr = hstart; addr < hend; addr += HPAGE_PMD_SIZE) { int result = SCAN_FAIL; From patchwork Wed Jul 20 14:06:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 12924030 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4F1EC43334 for ; Wed, 20 Jul 2022 14:06:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7950B6B0078; Wed, 20 Jul 2022 10:06:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 746526B007B; Wed, 20 Jul 2022 10:06:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54ABC6B007D; Wed, 20 Jul 2022 10:06:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 460336B0078 for ; Wed, 20 Jul 2022 10:06:27 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 24188140545 for ; Wed, 20 Jul 2022 14:06:27 +0000 (UTC) X-FDA: 79707653214.13.E2526C1 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf11.hostedemail.com (Postfix) with ESMTP id 9FE72400AE for ; Wed, 20 Jul 2022 14:06:26 +0000 (UTC) Received: by mail-pf1-f201.google.com with SMTP id b11-20020aa7810b000000b0052aeb125cb3so5852846pfi.11 for ; Wed, 20 Jul 2022 07:06:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MiQGLn0ozPDq7MLM//IzrZBoVTKI/HOaHFEX3CjUgQg=; b=jVg0Dm01NAz27++UGQ3UXHVO9ybrNwu5lN2ejCDWHcMnNKZkWsR9Zh9t4XTaNLKILI Gq0drMEzzIQlknQ+DVPwBEZj5dmm9TDeHsYhKBSD1caYXA1b0XV6cF6Tehkf6P2XtiNT 6eAxUWc/9JkZjOpnUBfgqfHDemcI+IiHFWPL4yjfg6/iiAJC88d3ju6EBPxIXTY8WG+M JxmMiCuaxHxj+jDlB2h2u0rHaVetglXfmmQhNlWBhSDH9FRTCveS5/z+jxAi7wyWAm/J oodbOLFrSVJBghOg0XtXwRVNOXF/o3XzJbcnUTsVx3JihDptWbc5cPW/OaMA3m9QWRs7 mBKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MiQGLn0ozPDq7MLM//IzrZBoVTKI/HOaHFEX3CjUgQg=; b=Ha4bKpJiRLwu4xqTSStJlMlYze2SehlBisTrIzNMjt/oZU74DzCbh6GSbj3KbfCrCt CTlToFL9RIkxHohKSskY1TP96U3TBLawJggTQDITJQka3yoJnbx4QKjmxK6Dp4gpt7vt K9F5gjab/Fud8okbYJC+OIx1d4OPnciVeDxz0utWIw1pQ0ojfCP3+NaQsQx8+pIBqC4N WC56NIWtXEIXqvzVhe7066uhICfUqZ5xlYYy5IjgLi0c6PY5mg40lhh4tvnKlWVOCiOi qk2QHPPWl9efIKRb3SDY/UkU/t14m97fHlIwUCfDbPrUYKkr1Pla6aV21Z2XCHVAaE+e bsWw== X-Gm-Message-State: AJIora8PgajgHyu574OF8iDiSY71uHeHrIZrsnQHWEeK3Mq2Rv1PdifV uhQAbhqzZ+i/MS1ITfZ22l5rNC59ZB2F X-Google-Smtp-Source: AGRyM1uTPV9Q11QoL+UpkBS7D/Dw7Q8pIpTSQMSA3oZYUyXM6y9YkmQPGz+cvhzirteZzpWhyMkQFgtDRV+n X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a05:6a00:2410:b0:528:be6e:8c2e with SMTP id z16-20020a056a00241000b00528be6e8c2emr39214970pfh.31.1658325985602; Wed, 20 Jul 2022 07:06:25 -0700 (PDT) Date: Wed, 20 Jul 2022 07:06:03 -0700 In-Reply-To: <20220720140603.1958773-1-zokeefe@google.com> Message-Id: <20220720140603.1958773-5-zokeefe@google.com> Mime-Version: 1.0 References: <20220720140603.1958773-1-zokeefe@google.com> X-Mailer: git-send-email 2.37.0.170.g444d1eabd0-goog Subject: [PATCH mm-unstable 4/4] Revert "mm/madvise: add huge_memory:mm_madvise_collapse tracepoint" From: "Zach O'Keefe" To: Andrew Morton Cc: linux-mm@kvack.org, Hugh Dickins , Yang Shi , Miaohe Lin , "Zach O'Keefe" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658325986; a=rsa-sha256; cv=none; b=EY3ln36sz2vuXn3g8kMF/wETG2oDVsDzKhRLj22Fx8dhhXBhvMepF8X/215ku+V8rQ2gxU QZCzT/+XPxXUvOojo4VgfuIxXhSbTfd1muXISuhfvtfheV9CoBGJx80CZMswrBoLbjkwap HUsQgB9hJyEZppH6h4VhBzH1zbGiZaA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=jVg0Dm01; spf=pass (imf11.hostedemail.com: domain of 34QvYYgcKCHYtieYYZYaiiafY.Wigfchor-ggepUWe.ila@flex--zokeefe.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=34QvYYgcKCHYtieYYZYaiiafY.Wigfchor-ggepUWe.ila@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658325986; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MiQGLn0ozPDq7MLM//IzrZBoVTKI/HOaHFEX3CjUgQg=; b=8BXu1Y6D4XqFn6RWrec7XlATQ/NvUX+KzqlppV7KJiexlOMii6ofChNAyWSztJ8HZ/CY0t dt+MtQEaNnI/aN/wVzVLB3y0U9YdALOurC+cD/8mAvUAxa9U0Fr2URFEJjdtSrh/ij5BAV ouksZKd9hRTRE3alphbZfBJH/AyjzvA= X-Stat-Signature: 7hm1pikmmdj5136jmxbh9wdpb4sghybd X-Rspamd-Queue-Id: 9FE72400AE Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=jVg0Dm01; spf=pass (imf11.hostedemail.com: domain of 34QvYYgcKCHYtieYYZYaiiafY.Wigfchor-ggepUWe.ila@flex--zokeefe.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=34QvYYgcKCHYtieYYZYaiiafY.Wigfchor-ggepUWe.ila@flex--zokeefe.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1658325986-106427 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In the anonymous collapse path, huge_memory:mm_khugepaged_scan_pmd can be used to get roughly the same information as this proposed tracepoint. Remove it. Fixes: 0fff8a0de881 ("mm/madvise: add huge_memory:mm_madvise_collapse tracepoint") Link: https://lore.kernel.org/linux-mm/Ys2vzYyVFmljt+B8@google.com/ Signed-off-by: Zach O'Keefe Reviewed-by: Yang Shi Acked-by: David Rientjes --- include/trace/events/huge_memory.h | 22 ---------------------- mm/khugepaged.c | 2 -- 2 files changed, 24 deletions(-) diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge_memory.h index 38d339ffdb16..55392bf30a03 100644 --- a/include/trace/events/huge_memory.h +++ b/include/trace/events/huge_memory.h @@ -167,27 +167,5 @@ TRACE_EVENT(mm_collapse_huge_page_swapin, __entry->ret) ); -TRACE_EVENT(mm_madvise_collapse, - - TP_PROTO(struct mm_struct *mm, unsigned long addr, int result), - - TP_ARGS(mm, addr, result), - - TP_STRUCT__entry(__field(struct mm_struct *, mm) - __field(unsigned long, addr) - __field(int, result) - ), - - TP_fast_assign(__entry->mm = mm; - __entry->addr = addr; - __entry->result = result; - ), - - TP_printk("mm=%p addr=%#lx result=%s", - __entry->mm, - __entry->addr, - __print_symbolic(__entry->result, SCAN_STATUS)) -); - #endif /* __HUGE_MEMORY_H */ #include diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 70e9d9950415..28cb8429dad4 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2452,8 +2452,6 @@ int madvise_collapse(struct vm_area_struct *vma, struct vm_area_struct **prev, if (!mmap_locked) *prev = NULL; /* Tell caller we dropped mmap_lock */ - trace_mm_madvise_collapse(mm, addr, result); - switch (result) { case SCAN_SUCCEED: case SCAN_PMD_MAPPED: