From patchwork Fri Jun 22 15:12:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Andrzej Siewior X-Patchwork-Id: 10482397 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 687A960230 for ; Fri, 22 Jun 2018 15:12:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 66A88289A2 for ; Fri, 22 Jun 2018 15:12:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5B86628BB0; Fri, 22 Jun 2018 15:12:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C6AF2289A2 for ; Fri, 22 Jun 2018 15:12:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCE6B6B000E; Fri, 22 Jun 2018 11:12:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B5DAB6B0010; Fri, 22 Jun 2018 11:12:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1E796B0266; Fri, 22 Jun 2018 11:12:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199]) by kanga.kvack.org (Postfix) with ESMTP id 46D666B000E for ; Fri, 22 Jun 2018 11:12:37 -0400 (EDT) Received: by mail-wr0-f199.google.com with SMTP id w9-v6so4704754wrl.13 for ; Fri, 22 Jun 2018 08:12:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=nFfnYJfXIuQW+z0mDhRbTVPyHFoLVkquOyV1uiwSf3w=; b=ppdqn/eC9oBVsLVIEdY0PpNF9+sjRTmeMkrNnOsYJvoBVC+ptsrIyowwCVjF8NkB44 0Iq8bQPqQ8cCcA3Tgo5SYwECN2OrJvoknp52OmD1kjyyUwew+aydGp5EFN4/btbo3lyc hMDMAETzW2seRmYDIlPcNj2zNEGlBEKcbLcuNXfcrMSLQRuDab8EbPF8dLWb0rfiP9Kn T/NFhjZt3UTDBeS0czf7OPzye1wxjQi0fsn1dPvWJkD8r0+62j9QrTcbcEhP4bOfqRkL 4Rjza+JPhZ0ckRbuqnHmchzhL7kQpsF12QhMkAfk0668shMKEQIwxTuAWzEfLmR+NxU3 xsMg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bigeasy@linutronix.de designates 2a01:7a0:2:106d:700::1 as permitted sender) smtp.mailfrom=bigeasy@linutronix.de X-Gm-Message-State: APt69E2nx9j4z2dHuBQsVfYXtPGFmydurb4TYr0fqF6M37Kx+nRpmc31 1scZUPX/XU8afLuYav9WzckPe0A/5U1vMLVwLswBJ3UrgxSAsU3MXmiTrVCBWyqXBUuWiqEzu2r ZopHI2L7RMw4CxSQqZN1DMV3oDe1TItdnxf9Ft1gdMPJbF4x2E7J/U7to1/UyN9+WFQ== X-Received: by 2002:adf:fc05:: with SMTP id i5-v6mr1942471wrr.157.1529680356848; Fri, 22 Jun 2018 08:12:36 -0700 (PDT) X-Google-Smtp-Source: AAOMgpcX+/MK+u2ivEqcuJijSgkMsyRMQ5Di2glPHpMijLNka0THSjpslqA7FyUX8MSHWs5f4Xr2 X-Received: by 2002:adf:fc05:: with SMTP id i5-v6mr1942417wrr.157.1529680355911; Fri, 22 Jun 2018 08:12:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529680355; cv=none; d=google.com; s=arc-20160816; b=OVsJkkq/v27+YwOwPdaqm4MW34Ifw+8t4Z/CrhgywX0YmP1ud6pT6VxpZGwl0SmSBI DfFG8gMTtYe143MZ7UP+sp1rYFeeuHjEIXW5DWQgRof1TsSFg6behmE4MPHvwFw1VUpt 0oT46Qqpfaci24wGDfaUfljFUoZN/yL02dVZngGh0rRavtrzmnws4TJF654PsD4U70yz Nou0PmnXjxnkiGeBI2V4VgZeNtnPVRRjIflAh7o+1naK6KkXNHzySMAiVwYgYdZEyxCW Cqlajrm45lXVmBkzbLK0xZDrgSxuo0SkgXnlhsY1lnTt4nPLaDyCnIqnOAwan7hBGqdM xD/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=nFfnYJfXIuQW+z0mDhRbTVPyHFoLVkquOyV1uiwSf3w=; b=oEXSxFibUdpQORr98Iet5Ms7uGZWOdifBVe/ln55oqQ23DPiFjmsmr7CW3QEaQoDQI +QCsfVfdUwi19FYDNrUlcBI+WQ+C3r0Nmbn7akBdy0Ei9xTtx41AdUoSmOjsl1RYQT6e ktX/xTYhOgN9QLMSbpdKkQKg1pcw+ehEAOy+EjWgJ13vElwgIwQcxioh/LR06ZqBeC2b hco9zw1wc4T63YrBVvgs+O9pVRFYzqQEIUQwSwwiteK5CZttm1DrEpmI3blqnbDKo58U Xnbtl/tArrEpCyndur1T3bD0/62r4DLzjH5jbHmov54RjVpAnhI5lfzPjqyX/hOkF7B+ epOQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of bigeasy@linutronix.de designates 2a01:7a0:2:106d:700::1 as permitted sender) smtp.mailfrom=bigeasy@linutronix.de Received: from Galois.linutronix.de (Galois.linutronix.de. [2a01:7a0:2:106d:700::1]) by mx.google.com with ESMTPS id b11-v6si676488wri.417.2018.06.22.08.12.35 for (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 22 Jun 2018 08:12:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bigeasy@linutronix.de designates 2a01:7a0:2:106d:700::1 as permitted sender) client-ip=2a01:7a0:2:106d:700::1; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bigeasy@linutronix.de designates 2a01:7a0:2:106d:700::1 as permitted sender) smtp.mailfrom=bigeasy@linutronix.de Received: from localhost ([127.0.0.1] helo=bazinga.breakpoint.cc) by Galois.linutronix.de with esmtp (Exim 4.80) (envelope-from ) id 1fWNjm-0007Dp-Bk; Fri, 22 Jun 2018 17:12:34 +0200 From: Sebastian Andrzej Siewior To: linux-mm@kvack.org Cc: tglx@linutronix.de, Andrew Morton , Sebastian Andrzej Siewior Subject: [PATCH 3/3] mm: list_lru: Add lock_irq member to __list_lru_init() Date: Fri, 22 Jun 2018 17:12:21 +0200 Message-Id: <20180622151221.28167-4-bigeasy@linutronix.de> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180622151221.28167-1-bigeasy@linutronix.de> References: <20180622151221.28167-1-bigeasy@linutronix.de> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP scan_shadow_nodes() is the only user of __list_lru_walk_one() which disables interrupts before invoking it. The reason is that nlru->lock is nesting inside IRQ-safe i_pages lock. Some functions unconditionally acquire the lock with the _irq() suffix. __list_lru_walk_one() can't acquire the lock unconditionally with _irq() suffix because it might invoke a callback which unlocks the nlru->lock and invokes a sleeping function without enabling interrupts. Add an argument to __list_lru_init() which identifies wheather the nlru->lock needs to be acquired with disabling interrupts or without. Signed-off-by: Sebastian Andrzej Siewior Reviewed-by: Vladimir Davydov --- include/linux/list_lru.h | 12 ++++++++---- mm/list_lru.c | 14 ++++++++++---- mm/workingset.c | 12 ++++-------- 3 files changed, 22 insertions(+), 16 deletions(-) diff --git a/include/linux/list_lru.h b/include/linux/list_lru.h index 96def9d15b1b..c2161c3a1809 100644 --- a/include/linux/list_lru.h +++ b/include/linux/list_lru.h @@ -51,18 +51,22 @@ struct list_lru_node { struct list_lru { struct list_lru_node *node; + bool lock_irq; #if defined(CONFIG_MEMCG) && !defined(CONFIG_SLOB) struct list_head list; #endif }; void list_lru_destroy(struct list_lru *lru); -int __list_lru_init(struct list_lru *lru, bool memcg_aware, +int __list_lru_init(struct list_lru *lru, bool memcg_aware, bool lock_irq, struct lock_class_key *key); -#define list_lru_init(lru) __list_lru_init((lru), false, NULL) -#define list_lru_init_key(lru, key) __list_lru_init((lru), false, (key)) -#define list_lru_init_memcg(lru) __list_lru_init((lru), true, NULL) +#define list_lru_init(lru) __list_lru_init((lru), false, false, \ + NULL) +#define list_lru_init_key(lru, key) __list_lru_init((lru), false, false, \ + (key)) +#define list_lru_init_memcg(lru) __list_lru_init((lru), true, false, \ + NULL) int memcg_update_all_list_lrus(int num_memcgs); void memcg_drain_all_list_lrus(int src_idx, int dst_idx); diff --git a/mm/list_lru.c b/mm/list_lru.c index fcfb6c89ed47..1c49d48078e4 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -204,7 +204,10 @@ __list_lru_walk_one(struct list_lru *lru, int nid, int memcg_idx, struct list_head *item, *n; unsigned long isolated = 0; - spin_lock(&nlru->lock); + if (lru->lock_irq) + spin_lock_irq(&nlru->lock); + else + spin_lock(&nlru->lock); l = list_lru_from_memcg_idx(nlru, memcg_idx); restart: list_for_each_safe(item, n, &l->list) { @@ -251,7 +254,10 @@ __list_lru_walk_one(struct list_lru *lru, int nid, int memcg_idx, } } - spin_unlock(&nlru->lock); + if (lru->lock_irq) + spin_unlock_irq(&nlru->lock); + else + spin_unlock(&nlru->lock); return isolated; } @@ -553,7 +559,7 @@ static void memcg_destroy_list_lru(struct list_lru *lru) } #endif /* CONFIG_MEMCG && !CONFIG_SLOB */ -int __list_lru_init(struct list_lru *lru, bool memcg_aware, +int __list_lru_init(struct list_lru *lru, bool memcg_aware, bool lock_irq, struct lock_class_key *key) { int i; @@ -580,7 +586,7 @@ int __list_lru_init(struct list_lru *lru, bool memcg_aware, lru->node = NULL; goto out; } - + lru->lock_irq = lock_irq; list_lru_register(lru); out: memcg_put_cache_ids(); diff --git a/mm/workingset.c b/mm/workingset.c index 529480c21f93..23ce00f48212 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -480,13 +480,8 @@ static enum lru_status shadow_lru_isolate(struct list_head *item, static unsigned long scan_shadow_nodes(struct shrinker *shrinker, struct shrink_control *sc) { - unsigned long ret; - - /* list_lru lock nests inside the IRQ-safe i_pages lock */ - local_irq_disable(); - ret = list_lru_shrink_walk(&shadow_nodes, sc, shadow_lru_isolate, NULL); - local_irq_enable(); - return ret; + return list_lru_shrink_walk(&shadow_nodes, sc, shadow_lru_isolate, + NULL); } static struct shrinker workingset_shadow_shrinker = { @@ -523,7 +518,8 @@ static int __init workingset_init(void) pr_info("workingset: timestamp_bits=%d max_order=%d bucket_order=%u\n", timestamp_bits, max_order, bucket_order); - ret = __list_lru_init(&shadow_nodes, true, &shadow_nodes_key); + /* list_lru lock nests inside the IRQ-safe i_pages lock */ + ret = __list_lru_init(&shadow_nodes, true, true, &shadow_nodes_key); if (ret) goto err; ret = register_shrinker(&workingset_shadow_shrinker);