From patchwork Sun Jan 20 03:30:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiongchun Duan X-Patchwork-Id: 10772285 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 650E813B5 for ; Sun, 20 Jan 2019 03:31:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 527A52AFD0 for ; Sun, 20 Jan 2019 03:31:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 469812AFD2; Sun, 20 Jan 2019 03:31:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9C4D2AFD0 for ; Sun, 20 Jan 2019 03:31:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D9BB8E0007; Sat, 19 Jan 2019 22:31:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9621D8E0002; Sat, 19 Jan 2019 22:31:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DD0E8E0007; Sat, 19 Jan 2019 22:31:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 385BB8E0002 for ; Sat, 19 Jan 2019 22:31:08 -0500 (EST) Received: by mail-pl1-f199.google.com with SMTP id e68so10654789plb.3 for ; Sat, 19 Jan 2019 19:31:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=/zsbjQO9YDL29rXU1zSaBifmoCx2btuFtC325tJUsuQ=; b=uAD1p/ZJ4CNB8iwLqc40tVQhK03iKFHBi8oIxm47dF7deTWmwxS4YA3YyrptZ99e3E Omk3S5Ra2avQjYNI00G9QiOvvXedwlp7+ruTotknbQQKLfaFGVH1fO/tufM67nTaiz/l N/5ocfFKUwKltJKk/thvQI4b93Cmk5SiqW/RKKv6lhb5+nplV/Biz6S+2kFhcsSawu4f 8R7WH6KSv661F6+YdDaDaW/6lRGg5ohWwFhmZWXIwSTcZvFb2JOJA8W4fzZI3GSEF/bP opTD8zEIr9E9+22/n74+8kWPGllq3nJdcdIJGcb1+WFoUuCxYy3hP59BLORjRCD8OoDC mHQw== X-Gm-Message-State: AJcUukekOu2hPNozipU3XZ9tgPJI5ZI5tCg8JlMBdt3CWu8hHvq1+UM+ g9f8rsaMMfZh2FV2njfVAiTpPBdKJXcf62DOcCGREIz1FcIa/luf/xIsShMytAUOKXvJE1zBgJA UGa1VMM7ZxQFhS0Nm9MbH/BmleKGdPxnWJs0pkIURtuxlcha+q72PqW81P/myKjGIlIHfNz4CiW dI35iSgeIbuufXXJHOe3RPwiYEzHnNax8u9c3krM/ugTU0gs/LFyAvOF4nfnRfMrNXeEjo1OVZr Fo8hMoQHSPlDRHUKEVbySXrORlNKvI8coS8mqsdUOBu5mPLcnHv3wByAXCsf8pi00OAf5nSWNDV CsoWtLH+NQQa0LZuJXRKFa2fWnjGI0JT3keK0wkMTIIJw46FPn5x2eDj812wG3RbvORG4BnwJ3/ n X-Received: by 2002:a62:6408:: with SMTP id y8mr25023061pfb.202.1547955067827; Sat, 19 Jan 2019 19:31:07 -0800 (PST) X-Received: by 2002:a62:6408:: with SMTP id y8mr25023026pfb.202.1547955066898; Sat, 19 Jan 2019 19:31:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547955066; cv=none; d=google.com; s=arc-20160816; b=Ytr/s19gOjmKYAo36TvVjHavdQ0CVmPsFsGDukwDJEH45EWNZxWxH0zvoJoDDESZuX Cb7FtPTJKnztOqkDKc6KEdS4kz0UyPV/F4knByqlneZswCf5X9GiZofUYm+/QnMUhFqK TdAbtJ73IZIt/rGFJ8r2/MRlmgyDHxsSlb+7BFgCH3a2Ja5L5kvzEdaVavS2ySjBQA2E NwugJGqZ7JCegPRhHndv5Yr7XUk8GcrtmaqwpsRbybzgv4Ua39IR/ATKsWizMwcd8OAu Ughrnim/0186zHeLrS0BJ9nRVc7nXHoZUPuckDvqdwFH0zDHaRVkFZluCiAv9MttxpNs DlUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=/zsbjQO9YDL29rXU1zSaBifmoCx2btuFtC325tJUsuQ=; b=ibsiWvMtS4qaWs86gyflwa4G0xmW998BtXWtULSiJxcc+hZ19roX6Fw8bZYlz3Tg9U EhqiF43ZdfwZFSRETXjRQ89TsxdmuNRz8+yUsx/uEXqcSwQy9rkq0d0POu4jRXZ7l85e TreRiTMUagDHLzTtNIKn7z1O3r3pbmRGFE6LNO1hoUvooFENM9AQO59Q32rTqKYba6cf tetGr2nut6KeEiu4AhQPHI9/7Mc/VN//R54aL4udfSdLf2+NfqdFbhEd//m+dZrzafX6 onBdaOolyO0hsBG2IU5iM1czsULLMQ6mZUehERBUhnaHbNV1WlquGDIa4vitz0tkjU1Q xRzQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20150623.gappssmtp.com header.s=20150623 header.b=oeSSmuan; spf=pass (google.com: domain of duanxiongchun@bytedance.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=duanxiongchun@bytedance.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 43sor12796381plc.28.2019.01.19.19.31.06 for (Google Transport Security); Sat, 19 Jan 2019 19:31:06 -0800 (PST) Received-SPF: pass (google.com: domain of duanxiongchun@bytedance.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20150623.gappssmtp.com header.s=20150623 header.b=oeSSmuan; spf=pass (google.com: domain of duanxiongchun@bytedance.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=duanxiongchun@bytedance.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=bytedance.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/zsbjQO9YDL29rXU1zSaBifmoCx2btuFtC325tJUsuQ=; b=oeSSmuanGf+0tYWi62PjqBsZm8a2wMP7eUXJSjjaiGWQZh9GD5xP7leLps0sWCJe3U wXfgXv/s0FG1m3VAcVhui8/QVbFs4oRRuBkRVzHXHLSmYOjhH4mWMmjSB2xt9eAyQADE CI2QiCjwBjIdrRaUxe6G9yTR7VeAnwpu47Wpf41ZNPfu4mtzJnhgfn6fWI8Fis/27wO9 nmSnVrDeVbWVr1zq5cFSXy0GnyAjWfJnpqPXOvBYI+TR1pNLPU6IvTsM8hLcZrNjNsA5 QOOCMetn4ghz3OneldvBgvzHPoMRe73JWp+UYXTTXtjcLBp74t0UBkjZYkf7FkzqAReX Q1xg== X-Google-Smtp-Source: ALg8bN5ubSZjZUGBdxmFZbqIdKpxZ26RE4n1Wa0Y2fWsb0a9AAVX1+z59rPyfhoIEhT5jr1XN8VAIg== X-Received: by 2002:a17:902:7e4f:: with SMTP id a15mr24447024pln.149.1547955066536; Sat, 19 Jan 2019 19:31:06 -0800 (PST) Received: from localhost.localdomain.localdomain ([61.120.150.71]) by smtp.gmail.com with ESMTPSA id s130sm16022421pgc.60.2019.01.19.19.31.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Jan 2019 19:31:06 -0800 (PST) From: Xiongchun Duan To: cgroups@vger.kernel.org, linux-mm@kvack.org Cc: shy828301@gmail.com, mhocko@kernel.org, tj@kernel.org, hannes@cmpxchg.org, zhangyongsu@bytedance.com, liuxiaozhou@bytedance.com, zhengfeiran@bytedance.com, wangdongdong.6@bytedance.com, Xiongchun Duan Subject: [PATCH 4/5] Memcgroup:Implement force empty work function Date: Sat, 19 Jan 2019 22:30:20 -0500 Message-Id: <1547955021-11520-5-git-send-email-duanxiongchun@bytedance.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1547955021-11520-1-git-send-email-duanxiongchun@bytedance.com> References: <1547955021-11520-1-git-send-email-duanxiongchun@bytedance.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Implement force empty work function and add trigger by global work. force_empty_list : offline cgroup wait for trigger force empty. empty_fail_list: offline cgroup which had been trigger for too many time will not auto retrigger. Signed-off-by: Xiongchun Duan --- include/linux/memcontrol.h | 4 +++ mm/memcontrol.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 85 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 0a29f7f..064192e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -315,6 +315,10 @@ struct mem_cgroup { int current_retry; unsigned long timer_jiffies; + struct list_head force_empty_node; + struct list_head empty_fail_node; + struct work_struct force_empty_work; + struct mem_cgroup_per_node *nodeinfo[0]; /* WARNING: nodeinfo must be the last member here */ }; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index fad1aae..21b4432 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -86,6 +86,10 @@ struct mem_cgroup *root_mem_cgroup __read_mostly; +static DEFINE_MUTEX(offline_cgroup_mutex); +static LIST_HEAD(force_empty_list); +static LIST_HEAD(empty_fail_list); + #define MEM_CGROUP_RECLAIM_RETRIES 5 /* Socket memory accounting disabled? */ @@ -2939,9 +2943,52 @@ static ssize_t mem_cgroup_force_empty_write(struct kernfs_open_file *of, static void add_force_empty_list(struct mem_cgroup *memcg) { + struct list_head *pos, *n; + struct mem_cgroup *pos_memcg; + unsigned long tmp = memcg->timer_jiffies; + + mutex_lock(&offline_cgroup_mutex); + list_for_each_safe(pos, n, &force_empty_list) { + pos_memcg = container_of(pos, + struct mem_cgroup, force_empty_node); + if (time_after(tmp, pos_memcg->timer_jiffies)) + tmp = pos_memcg->timer_jiffies; + if (time_after(pos_memcg->timer_jiffies, memcg->timer_jiffies)) + break; + } + list_add_tail(&memcg->force_empty_node, pos); + mutex_unlock(&offline_cgroup_mutex); + mod_timer(&empty_trigger, tmp); } +static void mem_cgroup_force_empty_delay(struct work_struct *work) +{ + unsigned int order; + struct mem_cgroup *memcg = container_of(work, + struct mem_cgroup, force_empty_work); + + if (page_counter_read(&memcg->memory)) { + mem_cgroup_force_empty(memcg); + memcg->current_retry += 1; + if (page_counter_read(&memcg->memory)) { + if (memcg->current_retry >= memcg->max_retry) { + if (list_empty(&memcg->empty_fail_node)) { + mutex_lock(&offline_cgroup_mutex); + list_add(&memcg->empty_fail_node, + &empty_fail_list); + mutex_unlock(&offline_cgroup_mutex); + } + } else { + order = 1 << (memcg->current_retry - 1); + memcg->timer_jiffies = jiffies + HZ * order; + add_force_empty_list(memcg); + } + } + } + css_put(&memcg->css); +} + static u64 mem_cgroup_hierarchy_read(struct cgroup_subsys_state *css, struct cftype *cft) { @@ -4545,6 +4592,9 @@ static struct mem_cgroup *mem_cgroup_alloc(void) if (cgroup_subsys_on_dfl(memory_cgrp_subsys) && !cgroup_memory_nosocket) static_branch_inc(&memcg_sockets_enabled_key); + + INIT_LIST_HEAD(&memcg->force_empty_node); + INIT_LIST_HEAD(&memcg->empty_fail_node); memcg->max_retry = sysctl_cgroup_default_retry; memcg->current_retry = 0; @@ -4577,7 +4627,26 @@ static int mem_cgroup_css_online(struct cgroup_subsys_state *css) static void trigger_force_empty(struct work_struct *work) { + struct list_head *pos, *n; + struct mem_cgroup *memcg; + mutex_lock(&offline_cgroup_mutex); + list_for_each_safe(pos, n, &force_empty_list) { + memcg = container_of(pos, struct mem_cgroup, + force_empty_node); + if (time_after(jiffies, memcg->timer_jiffies)) { + if (atomic_long_add_unless(&memcg->css.refcnt.count, + 1, 0) == 0) { + continue; + } else if (!queue_work(memcg_force_empty_wq, + &memcg->force_empty_work)) { + css_put(&memcg->css); + } else { + list_del_init(&memcg->force_empty_node); + } + } + } + mutex_unlock(&offline_cgroup_mutex); } static void empty_timer_trigger(struct timer_list *t) @@ -4595,6 +4664,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) mem_cgroup_force_empty(memcg); if (page_counter_read(&memcg->memory) && memcg->max_retry != 1) { + INIT_WORK(&memcg->force_empty_work, + mem_cgroup_force_empty_delay); memcg->timer_jiffies = jiffies + HZ; add_force_empty_list(memcg); } @@ -4626,6 +4697,16 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) static void mem_cgroup_css_released(struct cgroup_subsys_state *css) { struct mem_cgroup *memcg = mem_cgroup_from_css(css); + if (!list_empty(&memcg->force_empty_node)) { + mutex_lock(&offline_cgroup_mutex); + list_del_init(&memcg->force_empty_node); + mutex_unlock(&offline_cgroup_mutex); + } + if (!list_empty(&memcg->empty_fail_node)) { + mutex_lock(&offline_cgroup_mutex); + list_del_init(&memcg->empty_fail_node); + mutex_unlock(&offline_cgroup_mutex); + } invalidate_reclaim_iterators(memcg); }