From patchwork Tue Aug 13 12:02:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13761885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3ACE3C52D7B for ; Tue, 13 Aug 2024 12:03:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B53806B009E; Tue, 13 Aug 2024 08:03:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADCDB6B009F; Tue, 13 Aug 2024 08:03:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92E9F6B00A0; Tue, 13 Aug 2024 08:03:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6FF0E6B009E for ; Tue, 13 Aug 2024 08:03:48 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1653EA6FB1 for ; Tue, 13 Aug 2024 12:03:48 +0000 (UTC) X-FDA: 82447088136.16.8386F11 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf14.hostedemail.com (Postfix) with ESMTP id 33E3E100035 for ; Tue, 13 Aug 2024 12:03:45 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=R78+EhzQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723550613; a=rsa-sha256; cv=none; b=s4bSGh86YzDdKHUt25CMMrd/CMyIMN84NuCS0DjlQKQWBuskwZX86QSXFwpnjanSki4spP 0q3sWkN8F7RJe5JLsADiH+SMaEU2T1XzpikSgnT1NSIEW2HTs+KvI+Ox9zd60sg/KrmPI6 X8AFQYLvbm8VcJi1ARki9tm5ryJ034M= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=R78+EhzQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723550613; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0tRn9HldKKoz64Z505xjl60T+jy+qH2RDb/CoWbJ5vM=; b=Ey0CxSUvIjryWNFBQkLztcDTDQZeY7vpOQgwZh248w+EUgWIY9vcn/Io6mfcMonVS7MsdY eZw9kGDeL10BGuNYexLsQ2U+V8tlH8SqbD3hgdJHZ2KBfrG1XUBVQ4SA0b+VxRMU854PdE SfQDQ1qEMFER9AOfVMmmebNFFaF38Vg= Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-7a1d066a5daso339934985a.3 for ; Tue, 13 Aug 2024 05:03:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1723550624; x=1724155424; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0tRn9HldKKoz64Z505xjl60T+jy+qH2RDb/CoWbJ5vM=; b=R78+EhzQi/X99hWIBGSDAsm3p5jtLr9cGPj3J0xqFwzXEieJpDBDKtv4HME8Bp9mM1 xA5hnuBTTEPVrgQXoUixZoj0JyTAYUxcit2aFwRF1fUEEssDK1B6mr0XUKUy88k7D9rP Ob++fLgGkoiAh5X2MQ9tuh+3UJFCCVMziGOPVFP4j2LghDdRjJaYAe4P6bx7ulod+/Dn oqHbXO+HX0Q7bQaXY6j2QdrWAg2xT1sJ+JsoTrBmabLglRwgPfvhaxC7Z6SUZMWP0QM+ Gtu7xu6Xd9Fca2dNHUOnE9NyqAqFGJCN2MF03nqdwW+zkKtIiyx5xbZ0mMJyCcfMmJux snRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723550624; x=1724155424; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0tRn9HldKKoz64Z505xjl60T+jy+qH2RDb/CoWbJ5vM=; b=sOjOwmeOhKRZqMztH2V76YUbRcNyq6gknPY7Z3uaoLjsAMNz1qgRKcdnVm7JG+gfx2 1KCT8UqeE6TohORIWE2OCIz7e4DYYBdlnhYOl3PgYqziDDX1P1d+C0qDFuKPS7vIOQcs oJNfXTq8V653HoSyfN3UuJM/vKegBZkrZxXBpoJ3yYRdL3jvUbxofqnnpLwmcqM/bHDd D6gAScHToXfpVBNnYxP7p4gDDvmClfTGlp8xYINoodjE7iudwMM/ILRFjPadsGFmpJSU ssbIo/otvXiJtyawt0JSk0Q5/vOB8p8YUPvC26erODcnkB/bRqcwZyAXvxSB5BcE/pRt lsvg== X-Forwarded-Encrypted: i=1; AJvYcCXalZZ3Kyv/xNIXwDgDiP1/XX2HK2mwOv6wyMYi1Dkjr5T7eVgFu3uuIX3nT+Cvlea30GtPsmF0QQ==@kvack.org X-Gm-Message-State: AOJu0YwdMKeiCyWIv0M567lssIWV5wu1vzOS0944jlehi2IzhSOlb1F0 puLj/0gcl62D4e+li/yz/l9IwV8e/h5jEskcjJZ4bGwcvd56bBuU X-Google-Smtp-Source: AGHT+IHCyk5c5SE6e3z0DoyqupXKxHAXubnY9Py1uh22TPysRE3LzZi4VhhYcHmcgCKHwTje7P8Lig== X-Received: by 2002:a05:620a:4091:b0:79d:9102:554a with SMTP id af79cd13be357-7a4e15011femr407319085a.14.1723550624171; Tue, 13 Aug 2024 05:03:44 -0700 (PDT) Received: from localhost (fwdproxy-ash-112.fbsv.net. [2a03:2880:20ff:70::face:b00c]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4531c26ec58sm31235081cf.60.2024.08.13.05.03.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 05:03:43 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, roman.gushchin@linux.dev, yuzhao@google.com, david@redhat.com, baohua@kernel.org, ryan.roberts@arm.com, rppt@kernel.org, willy@infradead.org, cerasuolodomenico@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kernel-team@meta.com, Shuang Zhai , Usama Arif Subject: [PATCH v3 1/6] mm: free zapped tail pages when splitting isolated thp Date: Tue, 13 Aug 2024 13:02:44 +0100 Message-ID: <20240813120328.1275952-2-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240813120328.1275952-1-usamaarif642@gmail.com> References: <20240813120328.1275952-1-usamaarif642@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 33E3E100035 X-Rspamd-Server: rspam01 X-Stat-Signature: b4st4c7jycwq7i1xs4wzgkawzn3nho9c X-HE-Tag: 1723550625-353487 X-HE-Meta: U2FsdGVkX1+tkCl/Uqe+Qah7s1Lx7NAD+sb20Sd+6U69ciwAfs2FyqsNEEgiFfGTBlRwcjxX8L6k4Ed92QcysjbeGt1HbQ3+NBfAqU8+VVGzbV9PoU7+PlHMKmaDMkipy+5U16KcceZhIFiyDvtuLzfK+50Lg95RmFGCEhnLy2tF6GEehmkRVShO3EQivCSLnKv3z5lvR6H4NrIm8gcf/EL25VtRFIrMrEPX8KP6YKp4Z24rA/C+GBLPlWxAU2gHPa5Ih72iMaF4dBd3+VYhaqfzceIhQ3VxqezDDNCKXL+/XeRlI95dbWErCrqlz3+RBSnM+GBDZrZK/vTq1Gpvk70MO6Tw15LApBxcX1q0CCX88Au5s2R3rEMxXT55IGdTZ1CjvzXAlraZ8Y6Lw3Qdu2A9CgxrHcMk2qHEfelzbJlOlDWEb2UyQ8ZTvbGgq53K49YLiY75Z9+YDzOQK9o9OuDHPa5FjCsLse5x6zo8Eq9ZtBH4FXe2OqMYFObzLyfO4sWc5TdugpuyzeFQZWPNXc+HYDsmKcp9hctrl+9lUxNmOt+F2Xt9lnpfodc3PiPhxy3LTHb/OaRj0U5ktT3IA1gOO4t5ykxorhsjqMAvrNxCCMW6qpgmvUzhqp5GWz28FaFMuaa7cR3w6k5vdfWKvdxRRbxl4OkqT4TeaudDj9tzowTiYjAK0m9K0l6wA4M1dsaBZmYhrRNcP2SNRjPvBUQAYjXJvxXL8ktGfiGYBICoI2Npt+eXu4KsOz+FVio3U1B/Zxhr2TVugGZ0qEQmUFDPYun/HhIHVsdqngkNh5wS2//Zjc1J8l0d7Q8cTS7x93Zx7QNydqwRT7Q8YmkIGuVPOGOw/lLorSG6u/TFFtFeg/tQSOn35PLDNjyw830O1zjrQ5rkKKdsFcnXfqSRAH6vKaZ3d0OQ0kgUNF+aWoU/jB8qqQXd+9xgmKZ/zUOnZ19uoUgZG7eHZ2Ol390 jM2JQ64K LI+IqRepDocWq2jirqfGNPpfzcT909+xNmY5vdehICTFEuugnp9k/9GBFWbGyz8dQ1VmXmQA9IWulGJb2GJ04jPAgyQ0XDNxHWOhPIsNjqpJy1CXlKS0QKeObGQd7hVRZYJ0v7bHv04xlY4wCMJdQ/Fns9djzfN6ywpRbHfUWiV1cLXW+7t5YrAqebS9bWliYS7FI4jWkCrK1Iqp+nB+VE1ntMacJkJxXRyQkplmU9aj/HnGjDOJTKwwP4+c2QU7Vjaie9I72FDQMhnYV9+RmLiKFbqKtYI2JoXt/JyQj4Y/iGdOnDiMStYTer0SZrc7ATPveyViVGe8Vijuk0xFjIQBD0mxWpVRGY8EQJC3oEa7L1NWChEZ65+je1cCC/LcH7dZSIrXaY+b477wciHo4dBnbOrZjre61y4LG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Yu Zhao If a tail page has only two references left, one inherited from the isolation of its head and the other from lru_add_page_tail() which we are about to drop, it means this tail page was concurrently zapped. Then we can safely free it and save page reclaim or migration the trouble of trying it. Signed-off-by: Yu Zhao Tested-by: Shuang Zhai Signed-off-by: Usama Arif Acked-by: Johannes Weiner --- mm/huge_memory.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 04ee8abd6475..85a424e954be 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3059,7 +3059,9 @@ static void __split_huge_page(struct page *page, struct list_head *list, unsigned int new_nr = 1 << new_order; int order = folio_order(folio); unsigned int nr = 1 << order; + struct folio_batch free_folios; + folio_batch_init(&free_folios); /* complete memcg works before add pages to LRU */ split_page_memcg(head, order, new_order); @@ -3143,6 +3145,26 @@ static void __split_huge_page(struct page *page, struct list_head *list, if (subpage == page) continue; folio_unlock(new_folio); + /* + * If a folio has only two references left, one inherited + * from the isolation of its head and the other from + * lru_add_page_tail() which we are about to drop, it means this + * folio was concurrently zapped. Then we can safely free it + * and save page reclaim or migration the trouble of trying it. + */ + if (list && folio_ref_freeze(new_folio, 2)) { + VM_WARN_ON_ONCE_FOLIO(folio_test_lru(new_folio), new_folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_large(new_folio), new_folio); + VM_WARN_ON_ONCE_FOLIO(folio_mapped(new_folio), new_folio); + + folio_clear_active(new_folio); + folio_clear_unevictable(new_folio); + if (!folio_batch_add(&free_folios, folio)) { + mem_cgroup_uncharge_folios(&free_folios); + free_unref_folios(&free_folios); + } + continue; + } /* * Subpages may be freed if there wasn't any mapping @@ -3153,6 +3175,11 @@ static void __split_huge_page(struct page *page, struct list_head *list, */ free_page_and_swap_cache(subpage); } + + if (free_folios.nr) { + mem_cgroup_uncharge_folios(&free_folios); + free_unref_folios(&free_folios); + } } /* Racy check whether the huge page can be split */