From patchwork Fri Aug 30 10:03:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Usama Arif X-Patchwork-Id: 13784867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BC63CA0EC5 for ; Fri, 30 Aug 2024 10:04:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B7466B00F3; Fri, 30 Aug 2024 06:04:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 268BD6B00F5; Fri, 30 Aug 2024 06:04:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BA3F6B00F6; Fri, 30 Aug 2024 06:04:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E04276B00F3 for ; Fri, 30 Aug 2024 06:04:45 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9083E121852 for ; Fri, 30 Aug 2024 10:04:45 +0000 (UTC) X-FDA: 82508477730.06.5B86F2C Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by imf23.hostedemail.com (Postfix) with ESMTP id A7B9E140010 for ; Fri, 30 Aug 2024 10:04:43 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eWxYxrd5; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725012262; a=rsa-sha256; cv=none; b=45nC5s4VPoSKpmp2g5sJWvIW+he5kvGXCZrCOo4wFGexylup5hosTkdvNwU80SLrS868j0 AR5ckL6ymznkdONGsubcRNMUGgiCqiq0TEZBgI51RUmzxi4Ubcy8o0cPUHDR5W+6trCj6A ++E1CbmxSYeq1lzGFLFy/Ty31aTSBV0= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eWxYxrd5; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725012262; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r5a45zgnKr1XZRxdreeDmqy+oLRtttPm8KTNMecFwc8=; b=vbGB5H+FX1sUrPbU8oz0Por1DzTe4DEPo3MmX3yCxQmbiv1YfiN2gI0B25qspqAQGEgXHY yf56/pSeXK/o3c8+yIV0IOT1sWcJcQEs5tcILgcbtiCHRwDAlGIhk9hCEnAcqxNy7HCmcj /w7mnMsEJfJWr/HdHkAtHBHX74mYYns= Received: by mail-qv1-f43.google.com with SMTP id 6a1803df08f44-6bf99fee82aso8752176d6.2 for ; Fri, 30 Aug 2024 03:04:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725012283; x=1725617083; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r5a45zgnKr1XZRxdreeDmqy+oLRtttPm8KTNMecFwc8=; b=eWxYxrd5ak+12Gtj7BKiL6kRCcNFwCV/II6Hp0lI6DHyWmAosBDR+GIal/nX/hT0t/ X6+PXTcT4yHEemQAipTwi9NHPp95mU8wvKkzvlxTbd3mmlq5Mzb3lgM/kTPeb9VMr/Tm ysXPuzmk4zqqHHkudMVdLDvzZTl/zAXFTqAxzcgHEgrxchyxH+n2SHI0FzOu1jlzKqrj JRicOtMC1HBkumKUanr4c3UFI+QAae4czkiaaGo/dJz7dOsxQHQ12egKtGPiadLhHvkV KW+5kl3LgTvtX+9R3LJZG5WKLpdIp5TXkA0Z7RPA+nioxLuysv4C0hZgmX7wDqKUdOVn chww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725012283; x=1725617083; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r5a45zgnKr1XZRxdreeDmqy+oLRtttPm8KTNMecFwc8=; b=WxaH4ujeCB8L8uKNwK0aANiDDspjZtg0GXOgk+Dj0QvbunKxthHYiqDo2Sj56080ro hVjy94yx0OPfCAlPjUMO/riGdWD7DFZC3NCs/XkZKYvwHmFnYNrj6mK4+3btkp6eBPeD BXiPfn1l5LEoagXfbGKbeqzy5Sid+byWKDUypr9cruDlGVwLJWTE30O07HsjSVGmwaTs BMuLQalf9eNxXShZGu7n1vL9cp+/Wd9rpCMQjAMuBtLrbHCeRD/q/61tF0b4mrJi8Bha Ok0p8JoNLDea50x0eEF/Uf+4VUcT7uHFsu2qmck/Jv2DssPyCt2FjNOZK65ye/VFa9Zp sv0Q== X-Forwarded-Encrypted: i=1; AJvYcCVU4AkKpKCNC05NjhiR/Nuc+SqbpckcBWGyBMWaM/DHdwxzLl2NhrzjVzf/ORf9zu7y7FxnJmbO8w==@kvack.org X-Gm-Message-State: AOJu0YwxXDz5ZfJQgwRFzhXE2mk+9qVTVBS/m8LXpxfVE8ODEUlfy+F+ imIcBDRIX9M5w4Z2aHL3T3DfuEUrT/VfsbbkLF/GHeodG60nnq6N X-Google-Smtp-Source: AGHT+IGgUTAqqh5BMFm/0McbIC4rNQXxf1Fjc0Krv4v/rvVrVCmsGivh9at8oTGIG8VkATlTayTR6g== X-Received: by 2002:a05:6214:5410:b0:6bf:6b15:a6d9 with SMTP id 6a1803df08f44-6c33e69f6ecmr68793496d6.51.1725012282619; Fri, 30 Aug 2024 03:04:42 -0700 (PDT) Received: from localhost (fwdproxy-ash-014.fbsv.net. [2a03:2880:20ff:e::face:b00c]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6c340c435dbsm13151326d6.71.2024.08.30.03.04.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Aug 2024 03:04:41 -0700 (PDT) From: Usama Arif To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, roman.gushchin@linux.dev, yuzhao@google.com, david@redhat.com, npache@redhat.com, baohua@kernel.org, ryan.roberts@arm.com, rppt@kernel.org, willy@infradead.org, cerasuolodomenico@gmail.com, ryncsn@gmail.com, corbet@lwn.net, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kernel-team@meta.com, Shuang Zhai , Usama Arif Subject: [PATCH v5 1/6] mm: free zapped tail pages when splitting isolated thp Date: Fri, 30 Aug 2024 11:03:35 +0100 Message-ID: <20240830100438.3623486-2-usamaarif642@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240830100438.3623486-1-usamaarif642@gmail.com> References: <20240830100438.3623486-1-usamaarif642@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: A7B9E140010 X-Rspamd-Server: rspam01 X-Stat-Signature: gk9gru41pr1u5jgnwc7dbzc8gxk6np38 X-HE-Tag: 1725012283-702886 X-HE-Meta: U2FsdGVkX18chWJKdfeo6yFsKRUD6Zu3qjgUb736oY3xUgM3HwveOWbPMyEHhqGXkqS0DEtMWkBG9a8aOdxwcI+N/yESUyFNDAZiG2xEGb0t/fm6IvGblC0giYxZj6gI3ikKteSNmQjf+X267pYB1mIg0haPTSujjTG68EOeF2AaFSA0vwFpSC/pw1yGghSlhgohkVu/zZKmT8eR35LJdLypvwmdpJ63ueiXklUVy6CgbKAvKgspel9unok+yYnzwDZn0cHUPujEhonoyKJJ8iXF6DIjXimUnWPeTuQ2zdUXWKMg2onzg+ttPaCINfCT0qYzRYpR/KqzOcNPnxv69+tfx4tAN3oy1MMC85c79gIz2vP1I21IS3ikpBkneGt0UJZe1C+RWzm9dodI6A7yOrCUMZYswNItwvGFDJU9QGECEXXpyeiMUHKQoexm/n0JQvYd+NImZsG5IG9s7PH2SNs/mSRHHQ4bYb/TRDNaGrAaKEA1TrdhpFf4uXxIlhyRc0zB4ax+KWHCy4n6GZnrM613XRXMAgKx0fBNE//ZLGn1xfYlooT3ai8kqegfZBg1QyVJJvX/v6gji7++Wafvxqg8VJz8duUGfifxnoPO0carn5/VCYZyIA6O9FMIlhFoAl/GmV7sxtHyY9ataDo1s19rNjgr5QANdFZL1LwOdstSjsYfat51K1uL+s++ucXnQo1LWKa/czZ5plt/jbOPfWO/rmDKNpAwcBy9f0BViTaN/zic6s2/0Y/HfYw0lu5sSUWmunBTMkl61ezglkSPOiVS5APjezFrCLMtd39Vx93pJRcS2tFwKY3d5jFobOBSQyNz43ybLJKEcAu+avHVmOGLJfkB0UMsKuzX11f+nu5oKI4spIhhruNWBi8NYLEwIdtBgDvLMRzQiRP1BNv8ZEnovJxKaIYrJJnWVaKSArxS6tApKgQcMT6iWQlU0NP0MblzISnrVIdSYLcq6W0 suhNVnrd HjQYfrhezdkUAac8NJYJIl/MgLdz0U/UxlnFo4DUBRYVKDdRMR4Ssjs8nXRdwELuJoBH2AHNcJ77y8GW+2UN47VLCa1iT9MEZDhx0XOhp+5f+gk6qzb+2T3TznkWB+KMPpM7+c8nnBpWc5tYhZT3u3FvodPqbWDTHRm6wngz8TpjOeefvOPAbk9DcJmEB+KtFA4flcKIauo3QJLy5EaMI4eFvAsYVDHdrLu4pMNDfWNyx1zMHA4uDL2+fiUL2wFOp/6VQUNlQzMUDyqa4bpyTnzOGnB2hIuHUra29eyr0RNhGjmQU4ZzKyxYhwL5oHAOKfhSPpPcPasRsGa4moIbXSruwIezkL909E7OBqm21T4E4h6ZqSbMT9Jd4vi5nZ2p1zp9ASDDBD0JoejgnUgCqpBqGqmxqSYDOSI0qBT1W45bC9F1UruKCTOBIJ/05sJrj84lhkIZV7vdMfZMiHtRrRrlb3P3Oggh8EKpLYwybMilneqvPDR+AYXXCiM6SpBfTYOIOy/t1cN7rYjLvUfBa7YUTQlK5Ab4HK+Lg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Yu Zhao If a tail page has only two references left, one inherited from the isolation of its head and the other from lru_add_page_tail() which we are about to drop, it means this tail page was concurrently zapped. Then we can safely free it and save page reclaim or migration the trouble of trying it. Signed-off-by: Yu Zhao Tested-by: Shuang Zhai Acked-by: Johannes Weiner Signed-off-by: Usama Arif --- mm/huge_memory.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 15418ffdd377..0c48806ccb9a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3170,7 +3170,9 @@ static void __split_huge_page(struct page *page, struct list_head *list, unsigned int new_nr = 1 << new_order; int order = folio_order(folio); unsigned int nr = 1 << order; + struct folio_batch free_folios; + folio_batch_init(&free_folios); /* complete memcg works before add pages to LRU */ split_page_memcg(head, order, new_order); @@ -3254,6 +3256,27 @@ static void __split_huge_page(struct page *page, struct list_head *list, if (subpage == page) continue; folio_unlock(new_folio); + /* + * If a folio has only two references left, one inherited + * from the isolation of its head and the other from + * lru_add_page_tail() which we are about to drop, it means this + * folio was concurrently zapped. Then we can safely free it + * and save page reclaim or migration the trouble of trying it. + */ + if (list && folio_ref_freeze(new_folio, 2)) { + VM_WARN_ON_ONCE_FOLIO(folio_test_lru(new_folio), new_folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_large(new_folio), new_folio); + VM_WARN_ON_ONCE_FOLIO(folio_mapped(new_folio), new_folio); + + folio_clear_active(new_folio); + folio_clear_unevictable(new_folio); + list_del(&new_folio->lru); + if (!folio_batch_add(&free_folios, new_folio)) { + mem_cgroup_uncharge_folios(&free_folios); + free_unref_folios(&free_folios); + } + continue; + } /* * Subpages may be freed if there wasn't any mapping @@ -3264,6 +3287,11 @@ static void __split_huge_page(struct page *page, struct list_head *list, */ free_page_and_swap_cache(subpage); } + + if (free_folios.nr) { + mem_cgroup_uncharge_folios(&free_folios); + free_unref_folios(&free_folios); + } } /* Racy check whether the huge page can be split */