From patchwork Mon May 16 07:56:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 12850364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D08CC433EF for ; Mon, 16 May 2022 07:56:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F1A666B0072; Mon, 16 May 2022 03:56:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA3AF6B0073; Mon, 16 May 2022 03:56:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D44D46B0075; Mon, 16 May 2022 03:56:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C36276B0072 for ; Mon, 16 May 2022 03:56:24 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 926482190F for ; Mon, 16 May 2022 07:56:24 +0000 (UTC) X-FDA: 79470848688.15.964BD13 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf30.hostedemail.com (Postfix) with ESMTP id 30BC6800B1 for ; Mon, 16 May 2022 07:56:02 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id m136-20020a25268e000000b0064b233e03d1so12453897ybm.14 for ; Mon, 16 May 2022 00:56:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=AkzbZUUJbKkKZF8OCLuIxnaK4yhhmDEBAMVwQQZIjPw=; b=NHb73BqEQQrOLYwOniqH+I1KrZOGNOQlLtgMcagSkYUSWDy6GPOzxTw7tEKSlK2J3N FR54JD6olOOTL2vnRrP48GyP8UtZGMbg2uZS6ZK31kbD65Rwi1MmwobJYt4XF6bDT45N MkZvFxTloWynDzi7URUQCiKC112K+j2qIf1uRoQpmYLqbcjcS9EHTOzECCiTGM7jqIoP X9V58uSAJjXUTQv31yTK45lW/bIhqPTIVXc7063Q1rAPzZ9hULPMnuAeGQEl3DnHO3N3 zF1GXUi+2rVHEJwDxlzfwGpDsI/apcxtXOY7QYv3qlM6SUbPtxAneseB5rQOd7rWf7/s mf/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=AkzbZUUJbKkKZF8OCLuIxnaK4yhhmDEBAMVwQQZIjPw=; b=SbH8Nz8i3ozqZN608FrERDAPJgt8tCDy/ZLS4EZQ1QuLeznv2Iie15km0hBC/Ojkwi WZGSo8FykcPLTXtO6hy/Ih4so+1Gz0Pi8qm/0dy+hdXcEUkXOEM0tJMfW+bL7ayssWJQ IFKaIEqDq6VhbPfTos1xbTFXYPRlcc0WwqAlp23m8KACv/IgfLOnXiYY6in9eTMiDJ50 +jOtFyRQMjEfv7HwLYTr3tiuz9ugnpN/2X/tE9DCgogTJ1uQ3p9M5qRQ0oppGpIOiPga MyuD7DzaiqLEYQjnuojrEpuzh8WTDCQ+Ry+aYUVadDLBDAYZEgtBO0cQB6LeA9EQONW0 Jm/g== X-Gm-Message-State: AOAM530xznPmYQPmdfZ6UB6JgGcLBIBmA5nZlUMZ+oys+zf0j3XCLLmD 3HtDVgace7CPEB6QPAFYsH5k3knCA5s= X-Google-Smtp-Source: ABdhPJyBBCojxk0seMo3izlq0YIsm5tU550dSAqNmlgmosNUvnP68lSg1jhby26PHmTgALdpi3FuEyIMnak= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:200:97b7:3e1e:26a3:690f]) (user=surenb job=sendgmr) by 2002:a0d:eb53:0:b0:2fe:b6bb:aa38 with SMTP id u80-20020a0deb53000000b002feb6bbaa38mr15064368ywe.377.1652687783377; Mon, 16 May 2022 00:56:23 -0700 (PDT) Date: Mon, 16 May 2022 00:56:18 -0700 Message-Id: <20220516075619.1277152-1-surenb@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.36.0.550.gb090851708-goog Subject: [PATCH v2 1/2] mm: drop oom code from exit_mmap From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: mhocko@suse.com, rientjes@google.com, willy@infradead.org, hannes@cmpxchg.org, guro@fb.com, minchan@kernel.org, kirill@shutemov.name, aarcange@redhat.com, brauner@kernel.org, hch@infradead.org, oleg@redhat.com, david@redhat.com, jannh@google.com, shakeelb@google.com, peterx@redhat.com, jhubbard@nvidia.com, shuah@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Stat-Signature: efisjxm538zt5sgx4znrcj3tjns5ykc4 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 30BC6800B1 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=NHb73BqE; spf=pass (imf30.hostedemail.com: domain of 3pwOCYgYKCNAEGD09x2AA270.yA8749GJ-886Hwy6.AD2@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3pwOCYgYKCNAEGD09x2AA270.yA8749GJ-886Hwy6.AD2@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-HE-Tag: 1652687762-911313 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The primary reason to invoke the oom reaper from the exit_mmap path used to be a prevention of an excessive oom killing if the oom victim exit races with the oom reaper (see [1] for more details). The invocation has moved around since then because of the interaction with the munlock logic but the underlying reason has remained the same (see [2]). Munlock code is no longer a problem since [3] and there shouldn't be any blocking operation before the memory is unmapped by exit_mmap so the oom reaper invocation can be dropped. The unmapping part can be done with the non-exclusive mmap_sem and the exclusive one is only required when page tables are freed. Remove the oom_reaper from exit_mmap which will make the code easier to read. This is really unlikely to make any observable difference although some microbenchmarks could benefit from one less branch that needs to be evaluated even though it almost never is true. [1] 212925802454 ("mm: oom: let oom_reap_task and exit_mmap run concurrently") [2] 27ae357fa82b ("mm, oom: fix concurrent munlock and oom reaper unmap, v3") [3] a213e5cf71cb ("mm/munlock: delete munlock_vma_pages_all(), allow oomreap") Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko Reviewed-by: Shuah Khan --- include/linux/oom.h | 2 -- mm/mmap.c | 31 ++++++++++++------------------- mm/oom_kill.c | 2 +- 3 files changed, 13 insertions(+), 22 deletions(-) diff --git a/include/linux/oom.h b/include/linux/oom.h index 2db9a1432511..6cdf0772dbae 100644 --- a/include/linux/oom.h +++ b/include/linux/oom.h @@ -106,8 +106,6 @@ static inline vm_fault_t check_stable_address_space(struct mm_struct *mm) return 0; } -bool __oom_reap_task_mm(struct mm_struct *mm); - long oom_badness(struct task_struct *p, unsigned long totalpages); diff --git a/mm/mmap.c b/mm/mmap.c index 313b57d55a63..ded42150e706 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3105,30 +3105,13 @@ void exit_mmap(struct mm_struct *mm) /* mm's last user has gone, and its about to be pulled down */ mmu_notifier_release(mm); - if (unlikely(mm_is_oom_victim(mm))) { - /* - * Manually reap the mm to free as much memory as possible. - * Then, as the oom reaper does, set MMF_OOM_SKIP to disregard - * this mm from further consideration. Taking mm->mmap_lock for - * write after setting MMF_OOM_SKIP will guarantee that the oom - * reaper will not run on this mm again after mmap_lock is - * dropped. - * - * Nothing can be holding mm->mmap_lock here and the above call - * to mmu_notifier_release(mm) ensures mmu notifier callbacks in - * __oom_reap_task_mm() will not block. - */ - (void)__oom_reap_task_mm(mm); - set_bit(MMF_OOM_SKIP, &mm->flags); - } - - mmap_write_lock(mm); + mmap_read_lock(mm); arch_exit_mmap(mm); vma = mm->mmap; if (!vma) { /* Can happen if dup_mmap() received an OOM */ - mmap_write_unlock(mm); + mmap_read_unlock(mm); return; } @@ -3138,6 +3121,16 @@ void exit_mmap(struct mm_struct *mm) /* update_hiwater_rss(mm) here? but nobody should be looking */ /* Use -1 here to ensure all VMAs in the mm are unmapped */ unmap_vmas(&tlb, vma, 0, -1); + mmap_read_unlock(mm); + + /* + * Set MMF_OOM_SKIP to hide this task from the oom killer/reaper + * because the memory has been already freed. Do not bother checking + * mm_is_oom_victim because setting a bit unconditionally is cheaper. + */ + set_bit(MMF_OOM_SKIP, &mm->flags); + + mmap_write_lock(mm); free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING); tlb_finish_mmu(&tlb); diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 49d7df39b02d..36355b162727 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -509,7 +509,7 @@ static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait); static struct task_struct *oom_reaper_list; static DEFINE_SPINLOCK(oom_reaper_lock); -bool __oom_reap_task_mm(struct mm_struct *mm) +static bool __oom_reap_task_mm(struct mm_struct *mm) { struct vm_area_struct *vma; bool ret = true; From patchwork Mon May 16 07:56:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 12850365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CA24C433FE for ; Mon, 16 May 2022 07:56:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E7EC46B0073; Mon, 16 May 2022 03:56:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E07836B0075; Mon, 16 May 2022 03:56:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C829B6B0078; Mon, 16 May 2022 03:56:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B6A9C6B0073 for ; Mon, 16 May 2022 03:56:26 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 9D5758115A for ; Mon, 16 May 2022 07:56:26 +0000 (UTC) X-FDA: 79470848772.26.44D1AB3 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf01.hostedemail.com (Postfix) with ESMTP id A5933400C8 for ; Mon, 16 May 2022 07:56:11 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id 135-20020a25058d000000b0064dd6bc9cfdso422207ybf.23 for ; Mon, 16 May 2022 00:56:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=bbA0qY1A4n3sQ32XOHCQYzNEBpxFCtTkAwZ65vjYfT4=; b=EzUcTngDzwso/L64fcVNakp/QNp2XxIVJx7Dc64hdmuiHc5AH2OFv0w8gdLxmwWPWS 0lQp7y6srQf/7QpKJATmstgTbdAePXt4McKCxanrSy9x4wAMtIKXHDH2UV3TEr2bdzhs rcv9dZx45zGWQpgfxEMwDy/ft01xVUILxl+wSJ46UXteeV8JbEzRu/DYN25yCr87q4Et WJkOZgXv3btbIB15dVs5CPFaVIHanllDdUmXZbSSbxLomlDB7QFoAqNJIf/80XLX4OVN X3x+FbDeBVcrGj1t4b2xdzKv/JYHrwX6M/n8h17yd3Z388IDgEuKg4e5FgyujLjSJRUk /O1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=bbA0qY1A4n3sQ32XOHCQYzNEBpxFCtTkAwZ65vjYfT4=; b=KoGKZAwyoRXDmk7aVIpztB2bDsn0fXk0y9t/QIwvUW1v+iBeTvLqjZcDLaFqkCipOF RPe1kqSILfJoAWNfO4uVTpqIx2PeNS0unxkuDc9nW5hqPwtwd4gjiAbgZmFXmKy9Lz1f i5xEfCHIJzmiVlWeXUXB07SvUAUzcbWf1cgUEA1SDgUT1lDPUaO2YFjCXY8YUG+uZo4Y q9k33tPtzVoHM+7SODWlgliIOlbNYs4iMGFQqDs6mu/OKF6vm88fBaeP+pIX09L0UQur kjPD6DK7kgs97QpE0AJsd5zCZ9FuTHzKoooLs6z0SZwImEXFnPXLYHAsSSgvcwpOyInI h4kw== X-Gm-Message-State: AOAM532MUHO+Q/KYyfk4Xk0QwJHTtwXDnSAtGxSjlJ8ggupAvhzxliIF oT6BDJdbLjBgJqdXCkKUQUd6rWk/bj8= X-Google-Smtp-Source: ABdhPJxRNFyBKRteFpyB2k4NehlzATJkvZrFhQcwM3ufUJXDyGPuUAk/55A2q7MGyZ2oCIfTir1jgdNC7NA= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:200:97b7:3e1e:26a3:690f]) (user=surenb job=sendgmr) by 2002:a25:b6c9:0:b0:64a:9f32:5ac4 with SMTP id f9-20020a25b6c9000000b0064a9f325ac4mr15077774ybm.131.1652687785373; Mon, 16 May 2022 00:56:25 -0700 (PDT) Date: Mon, 16 May 2022 00:56:19 -0700 In-Reply-To: <20220516075619.1277152-1-surenb@google.com> Message-Id: <20220516075619.1277152-2-surenb@google.com> Mime-Version: 1.0 References: <20220516075619.1277152-1-surenb@google.com> X-Mailer: git-send-email 2.36.0.550.gb090851708-goog Subject: [PATCH v2 2/2] mm: delete unused MMF_OOM_VICTIM flag From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: mhocko@suse.com, rientjes@google.com, willy@infradead.org, hannes@cmpxchg.org, guro@fb.com, minchan@kernel.org, kirill@shutemov.name, aarcange@redhat.com, brauner@kernel.org, hch@infradead.org, oleg@redhat.com, david@redhat.com, jannh@google.com, shakeelb@google.com, peterx@redhat.com, jhubbard@nvidia.com, shuah@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A5933400C8 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=EzUcTngD; spf=pass (imf01.hostedemail.com: domain of 3qQOCYgYKCNIGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3qQOCYgYKCNIGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Stat-Signature: yf5nasikcewrpurdxc7b3173ktsnmb13 X-HE-Tag: 1652687771-124493 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With the last usage of MMF_OOM_VICTIM in exit_mmap gone, this flag is now unused and can be removed. Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko Reviewed-by: Shuah Khan --- include/linux/oom.h | 9 --------- include/linux/sched/coredump.h | 7 +++---- mm/oom_kill.c | 4 +--- 3 files changed, 4 insertions(+), 16 deletions(-) diff --git a/include/linux/oom.h b/include/linux/oom.h index 6cdf0772dbae..25990e9d9e15 100644 --- a/include/linux/oom.h +++ b/include/linux/oom.h @@ -77,15 +77,6 @@ static inline bool tsk_is_oom_victim(struct task_struct * tsk) return tsk->signal->oom_mm; } -/* - * Use this helper if tsk->mm != mm and the victim mm needs a special - * handling. This is guaranteed to stay true after once set. - */ -static inline bool mm_is_oom_victim(struct mm_struct *mm) -{ - return test_bit(MMF_OOM_VICTIM, &mm->flags); -} - /* * Checks whether a page fault on the given mm is still reliable. * This is no longer true if the oom reaper started to reap the diff --git a/include/linux/sched/coredump.h b/include/linux/sched/coredump.h index 4d9e3a656875..5318c791ad39 100644 --- a/include/linux/sched/coredump.h +++ b/include/linux/sched/coredump.h @@ -70,9 +70,8 @@ static inline int get_dumpable(struct mm_struct *mm) #define MMF_UNSTABLE 22 /* mm is unstable for copy_from_user */ #define MMF_HUGE_ZERO_PAGE 23 /* mm has ever used the global huge zero page */ #define MMF_DISABLE_THP 24 /* disable THP for all VMAs */ -#define MMF_OOM_VICTIM 25 /* mm is the oom victim */ -#define MMF_OOM_REAP_QUEUED 26 /* mm was queued for oom_reaper */ -#define MMF_MULTIPROCESS 27 /* mm is shared between processes */ +#define MMF_OOM_REAP_QUEUED 25 /* mm was queued for oom_reaper */ +#define MMF_MULTIPROCESS 26 /* mm is shared between processes */ /* * MMF_HAS_PINNED: Whether this mm has pinned any pages. This can be either * replaced in the future by mm.pinned_vm when it becomes stable, or grow into @@ -80,7 +79,7 @@ static inline int get_dumpable(struct mm_struct *mm) * pinned pages were unpinned later on, we'll still keep this bit set for the * lifecycle of this mm, just for simplicity. */ -#define MMF_HAS_PINNED 28 /* FOLL_PIN has run, never cleared */ +#define MMF_HAS_PINNED 27 /* FOLL_PIN has run, never cleared */ #define MMF_DISABLE_THP_MASK (1 << MMF_DISABLE_THP) #define MMF_INIT_MASK (MMF_DUMPABLE_MASK | MMF_DUMP_FILTER_MASK |\ diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 36355b162727..11291b99599f 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -732,10 +732,8 @@ static void mark_oom_victim(struct task_struct *tsk) return; /* oom_mm is bound to the signal struct life time. */ - if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) { + if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) mmgrab(tsk->signal->oom_mm); - set_bit(MMF_OOM_VICTIM, &mm->flags); - } /* * Make sure that the task is woken up from uninterruptible sleep