From patchwork Thu Dec 19 20:32:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rik van Riel X-Patchwork-Id: 13915756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 193ECE77184 for ; Thu, 19 Dec 2024 20:33:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F32B6B007B; Thu, 19 Dec 2024 15:33:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A1726B0082; Thu, 19 Dec 2024 15:33:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1928F6B0083; Thu, 19 Dec 2024 15:33:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EE4376B007B for ; Thu, 19 Dec 2024 15:33:49 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6B494AE202 for ; Thu, 19 Dec 2024 20:33:49 +0000 (UTC) X-FDA: 82912858770.26.CB136A0 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) by imf05.hostedemail.com (Postfix) with ESMTP id F18B610000C for ; Thu, 19 Dec 2024 20:32:43 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734640405; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references; bh=0Ccq0U+3IOI/6eUxIxyV7Tfz5Kok9xBMqUuSAaMouNM=; b=cUr+VthX13DHsxJrlm8zlEuG7HU8UVXztQEt/GjPX9r/nmgX7BUm9w4QHXRLQWq/OSCULy e7xHAoQjIBQujLlig5PgkUkMeXVih/xE/QwWAolYRkisAGYRF0jG+UXVax1AcjGlHH7wrG dtrB9vNwvDZkqxyzz1Fp+iVPqOTQPwI= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734640405; a=rsa-sha256; cv=none; b=7GCLnUaIzOphp9jZphNrYYSJdhiYACtuFanUiOSgIquPyrB8zpZpFuGzfQZFWDpgcsUA6K tsaCUxnSOLRzlEuSn1Hl3EG8EyFHJdmRiBx6lIo05oFWiIS83rrI5dpyDI5Vu4cI8rkwVr I8wxmECw6wPbbecxBYhMQP76Xd9ajro= Received: from [2601:18c:9101:a8b6:82e7:cf5d:dfd9:50ef] (helo=fangorn) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1tONCY-000000002Ni-0at9; Thu, 19 Dec 2024 15:32:54 -0500 Date: Thu, 19 Dec 2024 15:32:53 -0500 From: Rik van Riel To: Shakeel Butt Cc: David Hildenbrand , Andrew Morton , Chris Li , Ryan Roberts , "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org Subject: [PATCH] mm: remove unnecessary calls to lru_add_drain Message-ID: <20241219153253.3da9e8aa@fangorn> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.43; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Stat-Signature: r799eok86s1fuxscnksmbwond59pifab X-Rspamd-Queue-Id: F18B610000C X-Rspam-User: X-HE-Tag: 1734640363-876383 X-HE-Meta: U2FsdGVkX1/pADtjc00qzOzoY7uE/EsdFhTWVs1XiyukcgUGlMu7vATMtAJ+54djQdbFVW60qK9gyqwKW6GqnUgUwALFtwOcFteaUEyUpTDw/bqrxKnzcXCTAGXP0gl35+7NaYer1CzzLy8PPBRBT7NFQuieryBDvg/FiGiaLvYDnzExigO6nfMroj0QFrQSy3VKlV7WbIxzTl5IiPGBLURKu8x5iLF379tNbjYiGBQpX9fE+b+W4B3nHywIDi4JNxbJtWY3PqSjT6dBmedWGBTu/jPk8Es3GVLgCi1tleYZzJpkcxCqpJ50Fz/vYYIZ4ZPFhlpbueQcmue/IuOz6Grd59HtLVnlqOMBwTUckaIUABXfk0PRxtifgMeVAeQOAJa4kA8d53/Q5MQl3QXy1KWNTaOAZhlNeioLPdmOzJvUvV1YBEIQkeOASI112vEFo3kMI3ZsdhnzLBD9HXDs4Rj6SLEqfh9hagdbhyvH1tpfDvqrxVAlOVQykTyJq3W7y+pNrY2sPchrlAznJflMw/erE0yjnTIivcJ8iq6FyyaE7nvCsKhRV+t3fp93uSc5v2ypQHW/rdlMDZUCJ4d0ancQMDJabvB78IgTi2r1DZm2YyTwYWoem8+xsMBkqc+XzZYmwyumYz+niLsKraTLI7hVJZjslZ3wvmVn8r7PMS9kyBi6aTto2fJ/IDtOb2G6R4j7ihoQVO7UYeSWSInunfZXIZ0uePyljYsdmuXLmITIYnNAP4ZfrTzbK04LclvLyxogtGnzua2aENDtbLfe5mI9EBRzzOtwwPVro1TUF0r2n/aBDN+zdSpQqdCiKlabzdwpUzvW3Fkxj+OXwkiH1SX4fYIyEPAwJJcv2JkIwfL+sCq4IHHqCZ9DjnO1vgW+aPgHwFd8TUIsTwgBq8ESPopMTp/bMfDY9bNjgaDpZwIotcu1K+ajHoHu1JzAiuYtcbZOF4+F9SeNHCT2DRd 4VDb1Caj U/lhyJog/VY12eZsu6A4DNqaQh2ldyOu8ooNRuCYJI2mMdIY5BvHsISv1xdeUG0Vr7eif4wgSy/LeuU1652jP6WnVZ35O7c6s0nqTXlgfk8LSNvvuoJobyTR6muS94MpI7/mkQtvmoqBK71BHJuTgoMbgNKUSt37p1AIziqrgfkKP/LVZbjWorN+TOaK1oOjNuygSQ/b6+Li6I9Gl5mWV3tit+/R3C0ypgH+Dn90PnyFdxoPyFLt5iVGrHBAAdWjmGWBfmS7HCqEqLiy5WU3xSnCf+t+FKzWX89kPrOOtCs2PaI/EH4DMVpgUQM/PGskQP3ck0vmWUtgyYu/Ma98mbi6uH0GN1/98vHfJLgWRNgwUecuWE77aqlsCl0W7MlCEGgkETWY83tV0Q6aboxCtCD930zo36zU+YHei X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There seem to be several categories of calls to lru_add_drain and lru_add_drain_all. The first are code paths that recently allocated, swapped in, or otherwise processed a batch of pages, and want them all on the LRU. These drain pages that were recently allocated, probably on the local CPU. A second category are code paths that are actively trying to reclaim, migrate, or offline memory. These often use lru_add_drain_all, to drain the caches on all CPUs. However, there also seem to be some other callers where we aren't really doing either. They are calling lru_add_drain(), despite operating on pages that may have been allocated long ago, and quite possibly on different CPUs. Those calls are not likely to be effective at anything but creating lock contention on the LRU locks. Remove the lru_add_drain calls in the latter category. Signed-off-by: Rik van Riel Suggested-by: David Hildenbrand Acked-by: Shakeel Butt Acked-by: David Hildenbrand --- mm/memory.c | 1 - mm/mmap.c | 2 -- mm/swap_state.c | 1 - mm/vma.c | 2 -- 4 files changed, 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 75c2dfd04f72..95ce298dc254 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1935,7 +1935,6 @@ void zap_page_range_single(struct vm_area_struct *vma, unsigned long address, struct mmu_notifier_range range; struct mmu_gather tlb; - lru_add_drain(); mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma->vm_mm, address, end); hugetlb_zap_begin(vma, &range.start, &range.end); diff --git a/mm/mmap.c b/mm/mmap.c index d32b7e701058..ef57488f1020 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1660,7 +1660,6 @@ void exit_mmap(struct mm_struct *mm) goto destroy; } - lru_add_drain(); flush_cache_mm(mm); tlb_gather_mmu_fullmm(&tlb, mm); /* update_hiwater_rss(mm) here? but nobody should be looking */ @@ -2103,7 +2102,6 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift) vma, new_start, length, false, true)) return -ENOMEM; - lru_add_drain(); tlb_gather_mmu(&tlb, mm); next = vma_next(&vmi); if (new_end > old_start) { diff --git a/mm/swap_state.c b/mm/swap_state.c index e0c0321b8ff7..ca42b2be64d9 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -317,7 +317,6 @@ void free_pages_and_swap_cache(struct encoded_page **pages, int nr) struct folio_batch folios; unsigned int refs[PAGEVEC_SIZE]; - lru_add_drain(); folio_batch_init(&folios); for (int i = 0; i < nr; i++) { struct folio *folio = page_folio(encoded_page_ptr(pages[i])); diff --git a/mm/vma.c b/mm/vma.c index 8e31b7e25aeb..d84e5ef6d15b 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -398,7 +398,6 @@ void unmap_region(struct ma_state *mas, struct vm_area_struct *vma, struct mm_struct *mm = vma->vm_mm; struct mmu_gather tlb; - lru_add_drain(); tlb_gather_mmu(&tlb, mm); update_hiwater_rss(mm); unmap_vmas(&tlb, mas, vma, vma->vm_start, vma->vm_end, vma->vm_end, @@ -1130,7 +1129,6 @@ static inline void vms_clear_ptes(struct vma_munmap_struct *vms, * were isolated before we downgraded mmap_lock. */ mas_set(mas_detach, 1); - lru_add_drain(); tlb_gather_mmu(&tlb, vms->vma->vm_mm); update_hiwater_rss(vms->vma->vm_mm); unmap_vmas(&tlb, mas_detach, vms->vma, vms->start, vms->end,