From patchwork Thu Oct 25 08:24:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 10655593 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23ADA14DE for ; Thu, 25 Oct 2018 08:24:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1194C2B22F for ; Thu, 25 Oct 2018 08:24:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 05EB52B237; Thu, 25 Oct 2018 08:24:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 72BFD2B22F for ; Thu, 25 Oct 2018 08:24:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F04D6B026D; Thu, 25 Oct 2018 04:24:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2A3B76B026F; Thu, 25 Oct 2018 04:24:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CFCD6B0270; Thu, 25 Oct 2018 04:24:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id A16426B026D for ; Thu, 25 Oct 2018 04:24:15 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id x44-v6so4273084edd.17 for ; Thu, 25 Oct 2018 01:24:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=e/av0pB8zJCwcJxHsePcmzWL1U7Z3G4tDndbbOHv8Og=; b=LmshDs6KsG+Z8LJSg0XD64vnXLNqPZZXbDFYXj5JpZ2ILlrGm3mIjaQ4DtQ0vJNoVF mypF6B1vxwvRzZJKiejqlOdVKaxMJb6PxoaeDEYOmdpQZqu+Fq1yeiXLCV3QJsZ66zFW LcAYVNqCSmCAugHvgZZ/WC1aLqthuxFTtm/P/PcYiN9Oga80hTHKfJorWfYggh0QYqfV 8MlrF/TxXxCrf7eASh0qiEO1CMANx6Q7HRzVK3RlCMwfs88iueKHi/rIRwD2H0wcHTiT FJ5y0vwA1XKWG6IlneXRQM2TvY5wjy6fnJBLgeUHv0GxXnlGqItPcigJ4cUhDNqDKhSA AEgA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Gm-Message-State: AGRZ1gI4iGZIeaChDE3rjB+v9uHf6o4CXgf0UPaAg8SKdSVJsc4Gocju XUR1bfEb6tZuBcSL8Eia+9ZYrSC7jmduwZGu6NPDwkLkOf78RSzVLcMl6VtctFs8r8Sb2AchkcE DvPxVH1/HGfn0r9hJN3UuheQx6TU/iPXyOPl3Kccz8xmXJnmdSz3Zwxov48sT4J7brpctO9r0V1 UX5pz2CmpZ/8S09Uam/biFlewOLDz5gLJ1ZvAh1Ilj7nDrcPUWm9t1xDDpeHIaZynYrR3OYhSMA YOYyX1RAhlRMceOln62ZiOgX+PcnGkSN/FTVdrF0jAVPIvxkiEaxmVxscCDDbYv5MTT+aXY5lB0 i2fKwdQFYn0erMxoepRnj3G3QWu+M27hmaG7KsVjGTjOMKgnNvDbl5EljqrQl8WD0ZEJu/qa/A= = X-Received: by 2002:a17:906:c15:: with SMTP id s21-v6mr326535ejf.140.1540455855081; Thu, 25 Oct 2018 01:24:15 -0700 (PDT) X-Received: by 2002:a17:906:c15:: with SMTP id s21-v6mr326503ejf.140.1540455853935; Thu, 25 Oct 2018 01:24:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540455853; cv=none; d=google.com; s=arc-20160816; b=KTrV63xaJB8hWprfK4YqKlK/oZdotYN8DY0FnruPpQBGWVm3V9WD6DnkmQyZeDtABv HGeESHgj7SlYM6YnkEae+oIy+kUKsUWa/AKKfXQ1VtFhFxdmlFfckaN/6CF4QkjMzCwS DGrLQMI7Vv6WtyMVG9bQWujFAW49tq16BARYZHrVDxZRE2oayIFIgTDiVxOEreDURwJ4 q6aTDhyfbrc8JVc50ezDDhdJumZL4k7VhhVdvcpf0XH8Zdegxwa+MH91u/pTeyJrPWfZ Z6QxlPPDAaxPAwIo7N4J2CnIgm8qdXZ4o1wMjz94xM4Q0EVv5pgYj41zMXGpW9EpcCXh QT4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=e/av0pB8zJCwcJxHsePcmzWL1U7Z3G4tDndbbOHv8Og=; b=i6HINSnn23iIX0mSTzEZpVhsocryWLxudNEWWI1FX1O7FcQAZYWRtMIFTQDX6uVaKq W7znuBS+keDLn4gveC6dgcP6uheg9BqdYIGAZeDZYRDhyI4jQaD1RXQuc2iu5KeIjaBw GMPvGi4H0OXLaXMl8zzWhke0knwR4p35TmpnukK9rj/l8QcrVQz4D8IcfgBlp00Cwu1B sUPU/ldsLb8EyoBDH64khA2uyE9j4SuKsEJx74+oRhlhsf6X3BvYXIl50Z9REmnx/7up EV+Wtp1vd0st98WKnYt1kjPdEyb9UScqss3o08aliEM6NGaCnQi+bnNq1Brq77G0kF4Z Qvgg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y52-v6sor5617975edb.19.2018.10.25.01.24.13 for (Google Transport Security); Thu, 25 Oct 2018 01:24:13 -0700 (PDT) Received-SPF: pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Google-Smtp-Source: AJdET5e0QqkEcd4GBDzyoq1FkDzf2Fs/LbjX69f2r2oP9Z/TgnX0nNGmzCcvRPQgpQdJdzfGWCNp1A== X-Received: by 2002:aa7:d4c9:: with SMTP id t9-v6mr741268edr.256.1540455853240; Thu, 25 Oct 2018 01:24:13 -0700 (PDT) Received: from tiehlicka.suse.cz (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id m24-v6sm2628277edd.31.2018.10.25.01.24.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Oct 2018 01:24:12 -0700 (PDT) From: Michal Hocko To: Cc: Tetsuo Handa , Roman Gushchin , David Rientjes , Andrew Morton , LKML , Michal Hocko Subject: [RFC PATCH v2 2/3] mm, oom: keep retrying the oom_reap operation as long as there is substantial memory left Date: Thu, 25 Oct 2018 10:24:02 +0200 Message-Id: <20181025082403.3806-3-mhocko@kernel.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181025082403.3806-1-mhocko@kernel.org> References: <20181025082403.3806-1-mhocko@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko oom_reaper is not able to reap all types of memory. E.g. mlocked mappings or page tables. In some cases this might be a lot of memory and we do rely on exit_mmap to release that memory. Yet we cannot rely on exit_mmap to set MMF_OOM_SKIP right now because there are several places when sleeping locks are taken. This patch adds a simple heuristic to check for the amount of memory the mm is sitting on after oom_reaper is done with it. If this is still a considerable amount of the original memory (more than 1/4 of the original badness) then simply keep retrying oom_reap_task_mm. The portion is quite arbitrary and a subject of future tuning based on real life usecases but it should catch some outliars when the vast portion of the memory is consumed by non-reapable memory. Changes since RFC - do not use hardcoded threshold for retry and rather use a portion of the original badness instead Signed-off-by: Michal Hocko --- mm/oom_kill.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/mm/oom_kill.c b/mm/oom_kill.c index b3b2c2bbd8ab..ab42717661dc 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -189,6 +189,16 @@ static bool is_dump_unreclaim_slabs(void) return (global_node_page_state(NR_SLAB_UNRECLAIMABLE) > nr_lru); } +/* + * Rough memory consumption of the given mm which should be theoretically freed + * when the mm is removed. + */ +static unsigned long oom_badness_pages(struct mm_struct *mm) +{ + return get_mm_rss(mm) + get_mm_counter(mm, MM_SWAPENTS) + + mm_pgtables_bytes(mm) / PAGE_SIZE; +} + /** * oom_badness - heuristic function to determine which candidate task to kill * @p: task struct of which task we should calculate @@ -230,8 +240,7 @@ unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg, * The baseline for the badness score is the proportion of RAM that each * task's rss, pagetable and swap space use. */ - points = get_mm_rss(p->mm) + get_mm_counter(p->mm, MM_SWAPENTS) + - mm_pgtables_bytes(p->mm) / PAGE_SIZE; + points = oom_badness_pages(p->mm); task_unlock(p); /* Normalize to oom_score_adj units */ @@ -488,7 +497,7 @@ static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait); static struct task_struct *oom_reaper_list; static DEFINE_SPINLOCK(oom_reaper_lock); -static bool __oom_reap_task_mm(struct mm_struct *mm) +static bool __oom_reap_task_mm(struct mm_struct *mm, unsigned long original_badness) { struct vm_area_struct *vma; bool ret = true; @@ -532,6 +541,16 @@ static bool __oom_reap_task_mm(struct mm_struct *mm) } } + /* + * If we still sit on a noticeable amount of memory even after successfully + * reaping the address space then keep retrying until exit_mmap makes some + * further progress. + * TODO: add a flag for a stage when the exit path doesn't block anymore + * and hand over MMF_OOM_SKIP handling there in that case + */ + if (oom_badness_pages(mm) > (original_badness >> 2)) + ret = false; + return ret; } @@ -541,7 +560,7 @@ static bool __oom_reap_task_mm(struct mm_struct *mm) * Returns true on success and false if none or part of the address space * has been reclaimed and the caller should retry later. */ -static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) +static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm, unsigned long original_badness) { bool ret = true; @@ -564,7 +583,7 @@ static bool oom_reap_task_mm(struct task_struct *tsk, struct mm_struct *mm) trace_start_task_reaping(tsk->pid); /* failed to reap part of the address space. Try again later */ - ret = __oom_reap_task_mm(mm); + ret = __oom_reap_task_mm(mm, original_badness); if (!ret) goto out_finish; @@ -586,9 +605,10 @@ static void oom_reap_task(struct task_struct *tsk) { int attempts = 0; struct mm_struct *mm = tsk->signal->oom_mm; + unsigned long original_badness = oom_badness_pages(mm); /* Retry the down_read_trylock(mmap_sem) a few times */ - while (attempts++ < MAX_OOM_REAP_RETRIES && !oom_reap_task_mm(tsk, mm)) + while (attempts++ < MAX_OOM_REAP_RETRIES && !oom_reap_task_mm(tsk, mm, original_badness)) schedule_timeout_idle(HZ/10); if (attempts <= MAX_OOM_REAP_RETRIES ||