From patchwork Tue Aug 28 17:22:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 10578891 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B0495A4 for ; Tue, 28 Aug 2018 17:23:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E6C9286C0 for ; Tue, 28 Aug 2018 17:23:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 024A92A579; Tue, 28 Aug 2018 17:23:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4F1B1286C0 for ; Tue, 28 Aug 2018 17:23:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B3416B473C; Tue, 28 Aug 2018 13:23:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7632A6B473E; Tue, 28 Aug 2018 13:23:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C7176B4740; Tue, 28 Aug 2018 13:23:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f69.google.com (mail-yw1-f69.google.com [209.85.161.69]) by kanga.kvack.org (Postfix) with ESMTP id 0C5B36B473C for ; Tue, 28 Aug 2018 13:23:29 -0400 (EDT) Received: by mail-yw1-f69.google.com with SMTP id w23-v6so978639ywg.11 for ; Tue, 28 Aug 2018 10:23:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=rYqRJD/Nn/mjyDEIlHij4Nq2ZQJyQDLbJcRnxwRjma8=; b=A0zJZxDDsXHtGFvMladlXdhjdmBbju4Q4Snw/fYC4syKsCq15Df4xnTyB4WKBINnim e1PGmhw9+78opZpfm/cac2G3jbWvLKDHw4NKWkxWs7IuEAu4UPfzK7DWh5ZgTV8WE90d ogQXQDkgLbo9yoQudXyZqvlcxi7XkNT7wZTh5lY+hggodKiYc5jQh+1GOyYeRJ2qUbgV cVPttMlaEU5rzWpbvq3LZVyPyjsIpPUDdj+myeUDFFgw/N2pqpLQuS5kV0oFzoMEZ3zN AxL1uPy1ctrwVacXIAacbpk70LNO3PAJO4HlSc56+PQKFR0jRi/roQR/tuwi+EWzP8XZ UkVQ== X-Gm-Message-State: APzg51A1y8QWAii2vINMZAT0Olb11HGs3aY1WaspdGYSwijpNJqOYVqe IwEHRbqLwUxrz7Pz+rP7sFiy16qsE3shYz5gWaXHop0Ua7lDPKcN4rruRmB5RH2g6lPFNKdWmg3 x/3uLk2zxuzrmEEBgKhokPH8JF9EXLQE//gn42x1v8+rQRLfUp+4APqKMcIErsFkvVFAwgZA73a bB3HlV9XLLwy72we1SiKh1pMx9vP9+rNJ7IUxisvI/9M4kTdS4Xq4zWg34tPdmmDnuAEfbNlp0F Kwf5AmLbbckBXmTxSV96UxfM/vN4HdHjuRsrBFbGv8t46fjioBvLKO1lv9NKtGZkzexPHqWW81F 70l69ajEtLcMzeAx6Y+0uyFS/Fq3uEdQOrY/N6UR0Q7vQO69yTzQko1YTjv5WV7wxdI3RdYyBhd X X-Received: by 2002:a25:2984:: with SMTP id p126-v6mr1402441ybp.514.1535477008791; Tue, 28 Aug 2018 10:23:28 -0700 (PDT) X-Received: by 2002:a25:2984:: with SMTP id p126-v6mr1402403ybp.514.1535477007848; Tue, 28 Aug 2018 10:23:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535477007; cv=none; d=google.com; s=arc-20160816; b=wKeGFJDrd9VWCKRBG1vAjnBFT7GOEp6AJOhP9gf2BpF/7Z1omJEVH1R87TnpyeUmwB UY/RhjnSpbctW3T6f6GxNxOQ5nGqFaWAKZbK7vFXNN8FJSFeZBYFznECSZcmo0v3vdLS xqHwtyYxDmjPwYID4zAJma/o+vg61kVfrFQ4nO2EEYBGSzbCrc8ZgPmQCM09+qc9Ibgg tE05UIoixPbo1/2NHVC67oStXZBsVvJp3zVm2wD/3EluZzsEnTuJky2ZmtXfx0+CzwPE rbhPrB0BtFw2w0asikbXNDwuFibAB7bqm9iKVbu6g4Iwx/HJf0mngeIMWopJxa+L3AaR s2RA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=rYqRJD/Nn/mjyDEIlHij4Nq2ZQJyQDLbJcRnxwRjma8=; b=QMx7vP5oUsZakYMojVE5kzj6sc59OofO2HDOr9aPv6gOpRll16bcR5qVG0r4jPHxWY HAIuZQEHsJ8S/ZA6EX4hFDtcMgHMVB3QO0p/2ikgb/EK6EEhWoGV1eFiVkHDLeFyJ3c7 jY/tfT/thwtg350q/jig/0GgWfPP8tvIEVNoe6hbjJ97kddAyTX8Az8pReaFblHiG/ov stIb756iQ1/Rtl164V8v8fVUEEfOsoEUFWxCgMZy39eGmKlWGaiCwEHk8WGeBVw2iUmX 2qRj/xFYPQKOcyuteHeu113zj/HKG1xVHlPsSx8u1ToqvO2iswXgtmKQ4qkJTe0l4pw5 L01w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@cmpxchg-org.20150623.gappssmtp.com header.s=20150623 header.b=rv6b+n5M; spf=pass (google.com: domain of hannes@cmpxchg.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 185-v6sor430549ybr.4.2018.08.28.10.23.25 for (Google Transport Security); Tue, 28 Aug 2018 10:23:25 -0700 (PDT) Received-SPF: pass (google.com: domain of hannes@cmpxchg.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20150623.gappssmtp.com header.s=20150623 header.b=rv6b+n5M; spf=pass (google.com: domain of hannes@cmpxchg.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rYqRJD/Nn/mjyDEIlHij4Nq2ZQJyQDLbJcRnxwRjma8=; b=rv6b+n5M5/s6153mcQJP8vb8uQTvc0ZkerEzMVY/d84AUjep2LMXmRZe1FWs1qjyXF s0y0Ha0UgpGEJTASxmj542o50DJGU1z1eH3au+YA4fExeolwZJJ5hJBuH4Tau6DKmtxx SB/JZUuFL9anXIHerqONyisv6lsGmSnn8vJfn9deTIyTgidwM+3hGJm/z0JA81qWhsLf 4dQgA3HjMEuD/3l6gkLH4TFX8U1/eJUq+MNxqI91DS/OI9iF6jUKjBz/OpVN5l1ziDrn Kfirj7wk4y8q/udRMCGkLXNyJgUVZKh3tY/O/DLXkJkib73qxfeUKXRcAxGoSLAH2RbV hRgg== X-Google-Smtp-Source: ANB0VdYcmdWXjf09BCqTnjZc3KIcNh2/avH9YSJ2USJIxkxAPyEpIv0mY9KySK7FjbOWOTftW3VZAQ== X-Received: by 2002:a25:103:: with SMTP id 3-v6mr1411840ybb.421.1535477005285; Tue, 28 Aug 2018 10:23:25 -0700 (PDT) Received: from localhost ([2620:10d:c091:200::1:de86]) by smtp.gmail.com with ESMTPSA id c126-v6sm1609448ywa.104.2018.08.28.10.23.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 28 Aug 2018 10:23:24 -0700 (PDT) From: Johannes Weiner To: Ingo Molnar , Peter Zijlstra , Andrew Morton , Linus Torvalds Cc: Tejun Heo , Suren Baghdasaryan , Daniel Drake , Vinayak Menon , Christopher Lameter , Peter Enderborg , Shakeel Butt , Mike Galbraith , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 3/9] delayacct: track delays from thrashing cache pages Date: Tue, 28 Aug 2018 13:22:52 -0400 Message-Id: <20180828172258.3185-4-hannes@cmpxchg.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180828172258.3185-1-hannes@cmpxchg.org> References: <20180828172258.3185-1-hannes@cmpxchg.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Delay accounting already measures the time a task spends in direct reclaim and waiting for swapin, but in low memory situations tasks spend can spend a significant amount of their time waiting on thrashing page cache. This isn't tracked right now. To know the full impact of memory contention on an individual task, measure the delay when waiting for a recently evicted active cache page to read back into memory. Also update tools/accounting/getdelays.c: [hannes@computer accounting]$ sudo ./getdelays -d -p 1 print delayacct stats ON PID 1 CPU count real total virtual total delay total delay average 50318 745000000 847346785 400533713 0.008ms IO count delay total delay average 435 122601218 0ms SWAP count delay total delay average 0 0 0ms RECLAIM count delay total delay average 0 0 0ms THRASHING count delay total delay average 19 12621439 0ms Signed-off-by: Johannes Weiner --- include/linux/delayacct.h | 23 +++++++++++++++++++++++ include/uapi/linux/taskstats.h | 6 +++++- kernel/delayacct.c | 15 +++++++++++++++ mm/filemap.c | 11 +++++++++++ tools/accounting/getdelays.c | 8 +++++++- 5 files changed, 61 insertions(+), 2 deletions(-) diff --git a/include/linux/delayacct.h b/include/linux/delayacct.h index 31c865d1842e..577d1b25fccd 100644 --- a/include/linux/delayacct.h +++ b/include/linux/delayacct.h @@ -57,7 +57,12 @@ struct task_delay_info { u64 freepages_start; u64 freepages_delay; /* wait for memory reclaim */ + + u64 thrashing_start; + u64 thrashing_delay; /* wait for thrashing page */ + u32 freepages_count; /* total count of memory reclaim */ + u32 thrashing_count; /* total count of thrash waits */ }; #endif @@ -76,6 +81,8 @@ extern int __delayacct_add_tsk(struct taskstats *, struct task_struct *); extern __u64 __delayacct_blkio_ticks(struct task_struct *); extern void __delayacct_freepages_start(void); extern void __delayacct_freepages_end(void); +extern void __delayacct_thrashing_start(void); +extern void __delayacct_thrashing_end(void); static inline int delayacct_is_task_waiting_on_io(struct task_struct *p) { @@ -156,6 +163,18 @@ static inline void delayacct_freepages_end(void) __delayacct_freepages_end(); } +static inline void delayacct_thrashing_start(void) +{ + if (current->delays) + __delayacct_thrashing_start(); +} + +static inline void delayacct_thrashing_end(void) +{ + if (current->delays) + __delayacct_thrashing_end(); +} + #else static inline void delayacct_set_flag(int flag) {} @@ -182,6 +201,10 @@ static inline void delayacct_freepages_start(void) {} static inline void delayacct_freepages_end(void) {} +static inline void delayacct_thrashing_start(void) +{} +static inline void delayacct_thrashing_end(void) +{} #endif /* CONFIG_TASK_DELAY_ACCT */ diff --git a/include/uapi/linux/taskstats.h b/include/uapi/linux/taskstats.h index b7aa7bb2349f..5e8ca16a9079 100644 --- a/include/uapi/linux/taskstats.h +++ b/include/uapi/linux/taskstats.h @@ -34,7 +34,7 @@ */ -#define TASKSTATS_VERSION 8 +#define TASKSTATS_VERSION 9 #define TS_COMM_LEN 32 /* should be >= TASK_COMM_LEN * in linux/sched.h */ @@ -164,6 +164,10 @@ struct taskstats { /* Delay waiting for memory reclaim */ __u64 freepages_count; __u64 freepages_delay_total; + + /* Delay waiting for thrashing page */ + __u64 thrashing_count; + __u64 thrashing_delay_total; }; diff --git a/kernel/delayacct.c b/kernel/delayacct.c index ca8ac2824f0b..2a12b988c717 100644 --- a/kernel/delayacct.c +++ b/kernel/delayacct.c @@ -135,9 +135,12 @@ int __delayacct_add_tsk(struct taskstats *d, struct task_struct *tsk) d->swapin_delay_total = (tmp < d->swapin_delay_total) ? 0 : tmp; tmp = d->freepages_delay_total + tsk->delays->freepages_delay; d->freepages_delay_total = (tmp < d->freepages_delay_total) ? 0 : tmp; + tmp = d->thrashing_delay_total + tsk->delays->thrashing_delay; + d->thrashing_delay_total = (tmp < d->thrashing_delay_total) ? 0 : tmp; d->blkio_count += tsk->delays->blkio_count; d->swapin_count += tsk->delays->swapin_count; d->freepages_count += tsk->delays->freepages_count; + d->thrashing_count += tsk->delays->thrashing_count; raw_spin_unlock_irqrestore(&tsk->delays->lock, flags); return 0; @@ -169,3 +172,15 @@ void __delayacct_freepages_end(void) ¤t->delays->freepages_count); } +void __delayacct_thrashing_start(void) +{ + current->delays->thrashing_start = ktime_get_ns(); +} + +void __delayacct_thrashing_end(void) +{ + delayacct_end(¤t->delays->lock, + ¤t->delays->thrashing_start, + ¤t->delays->thrashing_delay, + ¤t->delays->thrashing_count); +} diff --git a/mm/filemap.c b/mm/filemap.c index 5e53424d9097..ca895ebe43ac 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -36,6 +36,7 @@ #include #include #include +#include #include "internal.h" #define CREATE_TRACE_POINTS @@ -1073,8 +1074,15 @@ static inline int wait_on_page_bit_common(wait_queue_head_t *q, { struct wait_page_queue wait_page; wait_queue_entry_t *wait = &wait_page.wait; + bool thrashing = false; int ret = 0; + if (bit_nr == PG_locked && !PageSwapBacked(page) && + !PageUptodate(page) && PageWorkingset(page)) { + delayacct_thrashing_start(); + thrashing = true; + } + init_wait(wait); wait->flags = lock ? WQ_FLAG_EXCLUSIVE : 0; wait->func = wake_page_function; @@ -1113,6 +1121,9 @@ static inline int wait_on_page_bit_common(wait_queue_head_t *q, finish_wait(q, wait); + if (thrashing) + delayacct_thrashing_end(); + /* * A signal could leave PageWaiters set. Clearing it here if * !waitqueue_active would be possible (by open-coding finish_wait), diff --git a/tools/accounting/getdelays.c b/tools/accounting/getdelays.c index 9f420d98b5fb..8cb504d30384 100644 --- a/tools/accounting/getdelays.c +++ b/tools/accounting/getdelays.c @@ -203,6 +203,8 @@ static void print_delayacct(struct taskstats *t) "SWAP %15s%15s%15s\n" " %15llu%15llu%15llums\n" "RECLAIM %12s%15s%15s\n" + " %15llu%15llu%15llums\n" + "THRASHING%12s%15s%15s\n" " %15llu%15llu%15llums\n", "count", "real total", "virtual total", "delay total", "delay average", @@ -222,7 +224,11 @@ static void print_delayacct(struct taskstats *t) "count", "delay total", "delay average", (unsigned long long)t->freepages_count, (unsigned long long)t->freepages_delay_total, - average_ms(t->freepages_delay_total, t->freepages_count)); + average_ms(t->freepages_delay_total, t->freepages_count), + "count", "delay total", "delay average", + (unsigned long long)t->thrashing_count, + (unsigned long long)t->thrashing_delay_total, + average_ms(t->thrashing_delay_total, t->thrashing_count)); } static void task_context_switch_counts(struct taskstats *t)