From patchwork Thu Jan 21 13:19:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham Ananthakrishna X-Patchwork-Id: 12036145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6298C433DB for ; Thu, 21 Jan 2021 13:20:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5A64D239FD for ; Thu, 21 Jan 2021 13:20:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A64D239FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DE9C66B000E; Thu, 21 Jan 2021 08:20:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9BC36B0010; Thu, 21 Jan 2021 08:20:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CAF186B0012; Thu, 21 Jan 2021 08:20:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0081.hostedemail.com [216.40.44.81]) by kanga.kvack.org (Postfix) with ESMTP id B1E7D6B000E for ; Thu, 21 Jan 2021 08:20:50 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 790B4180AD81D for ; Thu, 21 Jan 2021 13:20:50 +0000 (UTC) X-FDA: 77729842260.28.river16_3d0e7c327563 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 4EE626D68 for ; Thu, 21 Jan 2021 13:20:50 +0000 (UTC) X-HE-Tag: river16_3d0e7c327563 X-Filterd-Recvd-Size: 8396 Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Jan 2021 13:20:49 +0000 (UTC) Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDF7ke095736; Thu, 21 Jan 2021 13:20:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=JAakYMUma2jefmdeRpGGqctUcknR7wWq0NyR3UJjT+E=; b=v1W1XZOLzb9S5EzQIarazmx4SbZsEjE8LFmnJ4cvqwpNjAEj6k46NUHtESgUhnAE/wZZ /Ns5T23sqLrT5yH5j11zY78mKe1Ba0+PivfGxmnH1PbK/jaf3MEVt/6ouD1WbQcoO0ur 2jruQ6s5MAjch+l9fya1dt4RAu4w/YpMV658p5UBgX6mw1y+IprfuDL8gH3ALi6DslG8 ODnLBPrG5Hn2pussl9DI/1Mra1cLTvZNFiH4eIYsMj6X3C17SNZQnrq5THjcyYOMT0NK 0Y+qmDZzwHgS5CSJD7LzST1UGLqcno8WdTRhua65XMPjXIN9ujzaqH2dEj47KrgFxbxq 4w== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 3668qrf9ns-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:20:48 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDFkAW106737; Thu, 21 Jan 2021 13:20:47 GMT Received: from pps.reinject (localhost [127.0.0.1]) by userp3030.oracle.com with ESMTP id 3668rexqhv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:20:47 +0000 Received: from userp3030.oracle.com (userp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 10LDKZJ9123118; Thu, 21 Jan 2021 13:20:47 GMT Received: from gmananth-linux.oraclecorp.com (dhcp-10-166-171-141.vpn.oracle.com [10.166.171.141]) by userp3030.oracle.com with ESMTP id 3668rexq88-2; Thu, 21 Jan 2021 13:20:47 +0000 From: Gautham Ananthakrishna To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, matthew.wilcox@oracle.com, khlebnikov@yandex-team.ru, gautham.ananthakrishna@oracle.com Subject: [PATCH RFC 1/6] dcache: sweep cached negative dentries to the end of list of siblings Date: Thu, 21 Jan 2021 18:49:40 +0530 Message-Id: <1611235185-1685-2-git-send-email-gautham.ananthakrishna@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> References: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9870 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 suspectscore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 spamscore=0 phishscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101210072 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov For disk filesystems result of every negative lookup is cached, content of directories is usually cached too. Production of negative dentries isn't limited with disk speed. It's really easy to generate millions of them if system has enough memory. Negative dentries are linked into siblings list along with normal positive dentries. Some operations walks dcache tree but looks only for positive dentries: most important is fsnotify/inotify. This patch moves negative dentries to the end of list at final dput() and marks with flag which tells that all following dentries are negative too. Reverse operation is required before instantiating negative dentry. Signed-off-by: Konstantin Khlebnikov Signed-off-by: Gautham Ananthakrishna --- fs/dcache.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++--- include/linux/dcache.h | 6 +++++ 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index ea04858..a506169 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -632,6 +632,48 @@ static inline struct dentry *lock_parent(struct dentry *dentry) return __lock_parent(dentry); } +/* + * Move cached negative dentry to the tail of parent->d_subdirs. + * This lets walkers skip them all together at first sight. + * Must be called at dput of negative dentry. + */ +static void sweep_negative(struct dentry *dentry) +{ + struct dentry *parent; + + if (!d_is_tail_negative(dentry)) { + parent = lock_parent(dentry); + if (!parent) + return; + + if (!d_count(dentry) && d_is_negative(dentry) && + !d_is_tail_negative(dentry)) { + dentry->d_flags |= DCACHE_TAIL_NEGATIVE; + list_move_tail(&dentry->d_child, &parent->d_subdirs); + } + + spin_unlock(&parent->d_lock); + } +} + +/* + * Undo sweep_negative() and move to the head of parent->d_subdirs. + * Must be called before converting negative dentry into positive. + */ +static void recycle_negative(struct dentry *dentry) +{ + struct dentry *parent; + + spin_lock(&dentry->d_lock); + parent = lock_parent(dentry); + dentry->d_flags &= ~DCACHE_TAIL_NEGATIVE; + if (parent) { + list_move(&dentry->d_child, &parent->d_subdirs); + spin_unlock(&parent->d_lock); + } + spin_unlock(&dentry->d_lock); +} + static inline bool retain_dentry(struct dentry *dentry) { WARN_ON(d_in_lookup(dentry)); @@ -737,7 +779,7 @@ static struct dentry *dentry_kill(struct dentry *dentry) static inline bool fast_dput(struct dentry *dentry) { int ret; - unsigned int d_flags; + unsigned int d_flags, required; /* * If we have a d_op->d_delete() operation, we sould not @@ -785,6 +827,8 @@ static inline bool fast_dput(struct dentry *dentry) * a 'delete' op, and it's referenced and already on * the LRU list. * + * Cached negative dentry must be swept to the tail. + * * NOTE! Since we aren't locked, these values are * not "stable". However, it is sufficient that at * some point after we dropped the reference the @@ -796,10 +840,15 @@ static inline bool fast_dput(struct dentry *dentry) */ smp_rmb(); d_flags = READ_ONCE(dentry->d_flags); - d_flags &= DCACHE_REFERENCED | DCACHE_LRU_LIST | DCACHE_DISCONNECTED; + + required = DCACHE_REFERENCED | DCACHE_LRU_LIST | + (d_flags_negative(d_flags) ? DCACHE_TAIL_NEGATIVE : 0); + + d_flags &= DCACHE_REFERENCED | DCACHE_LRU_LIST | + DCACHE_DISCONNECTED | DCACHE_TAIL_NEGATIVE; /* Nothing to do? Dropping the reference was all we needed? */ - if (d_flags == (DCACHE_REFERENCED | DCACHE_LRU_LIST) && !d_unhashed(dentry)) + if (d_flags == required && !d_unhashed(dentry)) return true; /* @@ -871,6 +920,8 @@ void dput(struct dentry *dentry) rcu_read_unlock(); if (likely(retain_dentry(dentry))) { + if (d_is_negative(dentry)) + sweep_negative(dentry); spin_unlock(&dentry->d_lock); return; } @@ -1970,6 +2021,8 @@ void d_instantiate(struct dentry *entry, struct inode * inode) { BUG_ON(!hlist_unhashed(&entry->d_u.d_alias)); if (inode) { + if (d_is_tail_negative(entry)) + recycle_negative(entry); security_d_instantiate(entry, inode); spin_lock(&inode->i_lock); __d_instantiate(entry, inode); diff --git a/include/linux/dcache.h b/include/linux/dcache.h index 6f95c33..5f4ce3a 100644 --- a/include/linux/dcache.h +++ b/include/linux/dcache.h @@ -219,6 +219,7 @@ struct dentry_operations { #define DCACHE_PAR_LOOKUP 0x10000000 /* being looked up (with parent locked shared) */ #define DCACHE_DENTRY_CURSOR 0x20000000 #define DCACHE_NORCU 0x40000000 /* No RCU delay for freeing */ +#define DCACHE_TAIL_NEGATIVE 0x80000000 /* All following siblings are negative */ extern seqlock_t rename_lock; @@ -495,6 +496,11 @@ static inline int simple_positive(const struct dentry *dentry) return d_really_is_positive(dentry) && !d_unhashed(dentry); } +static inline bool d_is_tail_negative(const struct dentry *dentry) +{ + return unlikely(dentry->d_flags & DCACHE_TAIL_NEGATIVE); +} + extern void d_set_fallthru(struct dentry *dentry); static inline bool d_is_fallthru(const struct dentry *dentry) From patchwork Thu Jan 21 13:19:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham Ananthakrishna X-Patchwork-Id: 12036147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F86AC433DB for ; Thu, 21 Jan 2021 13:21:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8D0D3239FD for ; Thu, 21 Jan 2021 13:21:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D0D3239FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 18DA06B0010; Thu, 21 Jan 2021 08:21:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 117206B0012; Thu, 21 Jan 2021 08:21:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 005286B0022; Thu, 21 Jan 2021 08:21:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0003.hostedemail.com [216.40.44.3]) by kanga.kvack.org (Postfix) with ESMTP id DBC226B0010 for ; Thu, 21 Jan 2021 08:21:14 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A0C638249980 for ; Thu, 21 Jan 2021 13:21:14 +0000 (UTC) X-FDA: 77729843268.25.twist15_1e1614e27563 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 792E91804E3B7 for ; Thu, 21 Jan 2021 13:21:14 +0000 (UTC) X-HE-Tag: twist15_1e1614e27563 X-Filterd-Recvd-Size: 4706 Received: from userp2130.oracle.com (userp2130.oracle.com [156.151.31.86]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Jan 2021 13:21:13 +0000 (UTC) Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDDfln043163; Thu, 21 Jan 2021 13:21:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=m2knT8uSI6MLG69Ij4awUn0WhOK6WWXgzwRlQjg273U=; b=NUNs4uOJ1YNaFJUN18BDLqeYKsrNxZ2U0j9SnM1jH08gmuKtV/Llw6L1qOOzqUG8PccB sTYIdaGy7B6hnYUOhoP2PRQeLOV4OJVjQwHuSDB1+4SJO/T5m8cThriPn0XQsGDYT6ZO d2GFjVmel3tjP1XDYA5ZzwcJBuYeO+4R3jGEC+H7z2KFET7ph+5bFF0K0kgYAZ03HFjn cm6diN/JDzuIFtdXLWr6189cWN68e+djsIhcsoUf/AH7avReVXRJF0cuzYRfa3PblbhX uNgY8rH3oVjqOxNYHFpnI2LZyjrZfY7wdbBG0DzaIq+sCtY6ruYbtg0d6uMk2aT9Puwe pg== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 3668qaf9jc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:21:12 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDFkhP106711; Thu, 21 Jan 2021 13:21:12 GMT Received: from pps.reinject (localhost [127.0.0.1]) by userp3030.oracle.com with ESMTP id 3668rexr2m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:21:12 +0000 Received: from userp3030.oracle.com (userp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 10LDKZJB123118; Thu, 21 Jan 2021 13:21:12 GMT Received: from gmananth-linux.oraclecorp.com (dhcp-10-166-171-141.vpn.oracle.com [10.166.171.141]) by userp3030.oracle.com with ESMTP id 3668rexq88-3; Thu, 21 Jan 2021 13:21:12 +0000 From: Gautham Ananthakrishna To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, matthew.wilcox@oracle.com, khlebnikov@yandex-team.ru, gautham.ananthakrishna@oracle.com Subject: [PATCH RFC 2/6] fsnotify: stop walking child dentries if remaining tail is negative Date: Thu, 21 Jan 2021 18:49:41 +0530 Message-Id: <1611235185-1685-3-git-send-email-gautham.ananthakrishna@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> References: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9870 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 impostorscore=0 mlxscore=0 priorityscore=1501 phishscore=0 mlxlogscore=973 lowpriorityscore=0 malwarescore=0 adultscore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101210072 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000022, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov When notification starts/stops listening events from inode's children it have to update dentry->d_flags of all positive child dentries. Scanning may took a long time if directory has a lot of negative child dentries. This is main beneficiary of sweeping cached negative dentries to the end. Before patch: nr_dentry = 24172597 24.2M nr_buckets = 8388608 2.9 avg nr_unused = 24158110 99.9% nr_negative = 24142810 99.9% inotify time: 0.507182 seconds After patch: nr_dentry = 24562747 24.6M nr_buckets = 8388608 2.9 avg nr_unused = 24548714 99.9% nr_negative = 24543867 99.9% inotify time: 0.000010 seconds Negative dentries no longer slow down inotify op at parent directory. Signed-off-by: Konstantin Khlebnikov Signed-off-by: Gautham Ananthakrishna --- fs/notify/fsnotify.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/notify/fsnotify.c b/fs/notify/fsnotify.c index 8d3ad5e..4ccb59d 100644 --- a/fs/notify/fsnotify.c +++ b/fs/notify/fsnotify.c @@ -127,8 +127,12 @@ void __fsnotify_update_child_dentry_flags(struct inode *inode) * original inode) */ spin_lock(&alias->d_lock); list_for_each_entry(child, &alias->d_subdirs, d_child) { - if (!child->d_inode) + if (!child->d_inode) { + /* all remaining children are negative */ + if (d_is_tail_negative(child)) + break; continue; + } spin_lock_nested(&child->d_lock, DENTRY_D_LOCK_NESTED); if (watched) From patchwork Thu Jan 21 13:19:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham Ananthakrishna X-Patchwork-Id: 12036211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC4B7C433DB for ; Thu, 21 Jan 2021 13:23:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4000D23118 for ; Thu, 21 Jan 2021 13:23:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4000D23118 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8425B6B000D; Thu, 21 Jan 2021 08:23:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 819036B000E; Thu, 21 Jan 2021 08:23:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72F196B0010; Thu, 21 Jan 2021 08:23:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0055.hostedemail.com [216.40.44.55]) by kanga.kvack.org (Postfix) with ESMTP id 5B1AC6B000D for ; Thu, 21 Jan 2021 08:23:20 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 255998249980 for ; Thu, 21 Jan 2021 13:23:20 +0000 (UTC) X-FDA: 77729848560.13.lock62_3a07a0427563 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 05C0F18140B70 for ; Thu, 21 Jan 2021 13:23:20 +0000 (UTC) X-HE-Tag: lock62_3a07a0427563 X-Filterd-Recvd-Size: 4639 Received: from userp2130.oracle.com (userp2130.oracle.com [156.151.31.86]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Jan 2021 13:23:18 +0000 (UTC) Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDDw4d043265; Thu, 21 Jan 2021 13:23:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=umojX43O9UL+jYD0gI/4zhN6/zL41ZdveCdmKKXIwBM=; b=pqEGV2pIOoYPSPnybD/fuGsJki4qKy0xpc1E9Y+8q/QFTbNTeUfUSqndtJYovCU+vWGz MigODdMtjCkXqF9MbTJs2RCTr5U+YK0AJUELQEADtvNq3VE/SSoCUnUCGSC+fHVugx82 6HbvlcIGUMeK8uLEFlquZF9Ip+Hce3IAN0BDFw14tXbRjcS7YyKGL9brCe7AKgOgcCqd g0l9z2Yh5oSkdes/btHbn1FIagJ9/jhvks5IzVgpZsNJ22YgRVbqTmbcJxYe7BWovCR3 wyG9mNYlyGRfPInXuSr7EHOFWbLZ1PatmjZvm4lWvEXwoE6Y7zZWdQuaJc9leLQGXLK/ 8A== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 3668qaf9tk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:23:17 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDFktF106733; Thu, 21 Jan 2021 13:21:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by userp3030.oracle.com with ESMTP id 3668rexr63-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:21:16 +0000 Received: from userp3030.oracle.com (userp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 10LDKZJD123118; Thu, 21 Jan 2021 13:21:16 GMT Received: from gmananth-linux.oraclecorp.com (dhcp-10-166-171-141.vpn.oracle.com [10.166.171.141]) by userp3030.oracle.com with ESMTP id 3668rexq88-4; Thu, 21 Jan 2021 13:21:16 +0000 From: Gautham Ananthakrishna To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, matthew.wilcox@oracle.com, khlebnikov@yandex-team.ru, gautham.ananthakrishna@oracle.com Subject: [PATCH RFC 3/6] dcache: add action D_WALK_SKIP_SIBLINGS to d_walk() Date: Thu, 21 Jan 2021 18:49:42 +0530 Message-Id: <1611235185-1685-4-git-send-email-gautham.ananthakrishna@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> References: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9870 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 impostorscore=0 mlxscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 lowpriorityscore=0 malwarescore=0 adultscore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101210072 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov This lets skip remaining siblings at seeing d_is_tail_negative(). Signed-off-by: Konstantin Khlebnikov Signed-off-by: Gautham Ananthakrishna --- fs/dcache.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/dcache.c b/fs/dcache.c index a506169..894e6da 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1320,12 +1320,14 @@ void shrink_dcache_sb(struct super_block *sb) * @D_WALK_QUIT: quit walk * @D_WALK_NORETRY: quit when retry is needed * @D_WALK_SKIP: skip this dentry and its children + * @D_WALK_SKIP_SIBLINGS: skip siblings and their children */ enum d_walk_ret { D_WALK_CONTINUE, D_WALK_QUIT, D_WALK_NORETRY, D_WALK_SKIP, + D_WALK_SKIP_SIBLINGS, }; /** @@ -1356,6 +1358,7 @@ static void d_walk(struct dentry *parent, void *data, break; case D_WALK_QUIT: case D_WALK_SKIP: + case D_WALK_SKIP_SIBLINGS: goto out_unlock; case D_WALK_NORETRY: retry = false; @@ -1387,6 +1390,9 @@ static void d_walk(struct dentry *parent, void *data, case D_WALK_SKIP: spin_unlock(&dentry->d_lock); continue; + case D_WALK_SKIP_SIBLINGS: + spin_unlock(&dentry->d_lock); + goto skip_siblings; } if (!list_empty(&dentry->d_subdirs)) { @@ -1398,6 +1404,7 @@ static void d_walk(struct dentry *parent, void *data, } spin_unlock(&dentry->d_lock); } +skip_siblings: /* * All done at this level ... ascend and resume the search. */ From patchwork Thu Jan 21 13:19:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham Ananthakrishna X-Patchwork-Id: 12036213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E84EAC433DB for ; Thu, 21 Jan 2021 13:23:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 72DE423118 for ; Thu, 21 Jan 2021 13:23:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 72DE423118 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD3FB6B000E; Thu, 21 Jan 2021 08:23:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C86AF6B0010; Thu, 21 Jan 2021 08:23:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B73FA6B0012; Thu, 21 Jan 2021 08:23:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id A27CD6B000E for ; Thu, 21 Jan 2021 08:23:23 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5F0113621 for ; Thu, 21 Jan 2021 13:23:23 +0000 (UTC) X-FDA: 77729848686.12.iron03_2e0e87627563 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 3D325180033C2 for ; Thu, 21 Jan 2021 13:23:23 +0000 (UTC) X-HE-Tag: iron03_2e0e87627563 X-Filterd-Recvd-Size: 5152 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Jan 2021 13:23:22 +0000 (UTC) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDEVIh072889; Thu, 21 Jan 2021 13:23:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=ZE8IsY7P4ozut4mYRdoD4bNb9GcUENrj7M6f6C6UVV4=; b=0DwwfQ1uPbgN3FIQaCYnbaoDbzrD1UuDk1zmCF24XNxgCBh40W7jNisxbNIiuv0db0IM Nf0a3GxIufyjt9uEEGHxu3MShho9TgJAtYaorOzTzqz5iRoaHrIrgpFF3xX7FNvHzP4h 5gydTY/HBF7BZH4da9l+RiXHSPOBSQUaTtDclMMKbPGWgCxZ0wIjx2HNY5Lik6pHQgS7 AuvyiFlQc5t7UXPYjSc4xGrREnlbXISTiFMucvRg42zMBo5d6qiZZ6pictA55JCpxfEa li/+f/TgSDRrpIOcJ5WibZIc9C74OYBvQX/l0GudHYcxKG6Rn4PqpFfZti5+LgH9xiKX ag== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 3668qmy98t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:23:21 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDFjEG106643; Thu, 21 Jan 2021 13:21:21 GMT Received: from pps.reinject (localhost [127.0.0.1]) by userp3030.oracle.com with ESMTP id 3668rexr9n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:21:21 +0000 Received: from userp3030.oracle.com (userp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 10LDKZJF123118; Thu, 21 Jan 2021 13:21:20 GMT Received: from gmananth-linux.oraclecorp.com (dhcp-10-166-171-141.vpn.oracle.com [10.166.171.141]) by userp3030.oracle.com with ESMTP id 3668rexq88-5; Thu, 21 Jan 2021 13:21:20 +0000 From: Gautham Ananthakrishna To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, matthew.wilcox@oracle.com, khlebnikov@yandex-team.ru, gautham.ananthakrishna@oracle.com Subject: [PATCH RFC 4/6] dcache: stop walking siblings if remaining dentries all negative Date: Thu, 21 Jan 2021 18:49:43 +0530 Message-Id: <1611235185-1685-5-git-send-email-gautham.ananthakrishna@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> References: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9870 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 priorityscore=1501 adultscore=0 impostorscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 phishscore=0 clxscore=1015 bulkscore=0 mlxscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101210072 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov Most walkers are interested only in positive dentries. Changes in simple_* libfs helpers are mostly cosmetic: it shouldn't cache negative dentries unless uses d_delete other than always_delete_dentry(). Signed-off-by: Konstantin Khlebnikov Signed-off-by: Gautham Ananthakrishna --- fs/dcache.c | 9 +++++++++ fs/libfs.c | 3 +++ 2 files changed, 12 insertions(+) diff --git a/fs/dcache.c b/fs/dcache.c index 894e6da..492a42f 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1459,6 +1459,8 @@ static enum d_walk_ret path_check_mount(void *data, struct dentry *dentry) struct check_mount *info = data; struct path path = { .mnt = info->mnt, .dentry = dentry }; + if (d_is_tail_negative(dentry)) + return D_WALK_SKIP_SIBLINGS; if (likely(!d_mountpoint(dentry))) return D_WALK_CONTINUE; if (__path_is_mountpoint(&path)) { @@ -1705,6 +1707,10 @@ void shrink_dcache_for_umount(struct super_block *sb) static enum d_walk_ret find_submount(void *_data, struct dentry *dentry) { struct dentry **victim = _data; + + if (d_is_tail_negative(dentry)) + return D_WALK_SKIP_SIBLINGS; + if (d_mountpoint(dentry)) { __dget_dlock(dentry); *victim = dentry; @@ -3174,6 +3180,9 @@ static enum d_walk_ret d_genocide_kill(void *data, struct dentry *dentry) { struct dentry *root = data; if (dentry != root) { + if (d_is_tail_negative(dentry)) + return D_WALK_SKIP_SIBLINGS; + if (d_unhashed(dentry) || !dentry->d_inode) return D_WALK_SKIP; diff --git a/fs/libfs.c b/fs/libfs.c index 7124c2e..15d5ecf 100644 --- a/fs/libfs.c +++ b/fs/libfs.c @@ -410,6 +410,9 @@ int simple_empty(struct dentry *dentry) spin_lock(&dentry->d_lock); list_for_each_entry(child, &dentry->d_subdirs, d_child) { + if (d_is_tail_negative(child)) + break; + spin_lock_nested(&child->d_lock, DENTRY_D_LOCK_NESTED); if (simple_positive(child)) { spin_unlock(&child->d_lock); From patchwork Thu Jan 21 13:19:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham Ananthakrishna X-Patchwork-Id: 12036215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7784C433DB for ; Thu, 21 Jan 2021 13:23:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7563D206F7 for ; Thu, 21 Jan 2021 13:23:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7563D206F7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EABC06B0010; Thu, 21 Jan 2021 08:23:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E82A36B0012; Thu, 21 Jan 2021 08:23:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D48E86B0022; Thu, 21 Jan 2021 08:23:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id BC7296B0010 for ; Thu, 21 Jan 2021 08:23:27 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7F50B181AEF3F for ; Thu, 21 Jan 2021 13:23:27 +0000 (UTC) X-FDA: 77729848854.23.nut44_6311c2427563 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 4BBDD37609 for ; Thu, 21 Jan 2021 13:23:27 +0000 (UTC) X-HE-Tag: nut44_6311c2427563 X-Filterd-Recvd-Size: 4504 Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Jan 2021 13:23:26 +0000 (UTC) Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDEw0A095496; Thu, 21 Jan 2021 13:23:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=ql0wC9LxRsGgSrOHu3fRQsna06B8LdR/VErJMQR/XfM=; b=JVod1dEdB5Z5+ycpndwCZHxyDUxY9WQL3XSTTZ+5wrAhWMOZCFaciabNZYgMoUUhgNZz SQ3NFYPq1BPWMjAgygYYSSLakocigr5f8SAz/kSqoHYt4veUYuBNasCKKQzX+LtYjTGa IRV7mav+nTjZBVW4X246ZbHtUvx9oAj7TgXNAvw+KhR/C03KDPhxZfdS4n/Ox8Gnen15 bfX0gwQ6KcipvqE0w/kloUhJOnXpSLJ4e8lOXhYV7LAevEYx7pPVVMj6HR4nH9SIvuR7 MMXEV9iyLT8wV2VjuKcJOT8dJb8NMM8jHwjh+O2d+Vm4spaxhkiMfTsJQ7sj5JcDe4IU hg== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 3668qrfa03-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:23:25 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDFjEK106643; Thu, 21 Jan 2021 13:21:25 GMT Received: from pps.reinject (localhost [127.0.0.1]) by userp3030.oracle.com with ESMTP id 3668rexrcc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:21:25 +0000 Received: from userp3030.oracle.com (userp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 10LDKZJH123118; Thu, 21 Jan 2021 13:21:24 GMT Received: from gmananth-linux.oraclecorp.com (dhcp-10-166-171-141.vpn.oracle.com [10.166.171.141]) by userp3030.oracle.com with ESMTP id 3668rexq88-6; Thu, 21 Jan 2021 13:21:24 +0000 From: Gautham Ananthakrishna To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, matthew.wilcox@oracle.com, khlebnikov@yandex-team.ru, gautham.ananthakrishna@oracle.com Subject: [PATCH RFC 5/6] dcache: push releasing dentry lock into sweep_negative Date: Thu, 21 Jan 2021 18:49:44 +0530 Message-Id: <1611235185-1685-6-git-send-email-gautham.ananthakrishna@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> References: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9870 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 suspectscore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 spamscore=0 phishscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101210072 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov Release the dentry lock inside the sweep_negative() function. This is in preparation for a follow up patch and doesn't change runtime behavior. Signed-off-by: Konstantin Khlebnikov Signed-off-by: Gautham Ananthakrisha --- fs/dcache.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 492a42f..22c990b 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -638,13 +638,14 @@ static inline struct dentry *lock_parent(struct dentry *dentry) * Must be called at dput of negative dentry. */ static void sweep_negative(struct dentry *dentry) + __releases(dentry->d_lock) { struct dentry *parent; if (!d_is_tail_negative(dentry)) { parent = lock_parent(dentry); if (!parent) - return; + goto out; if (!d_count(dentry) && d_is_negative(dentry) && !d_is_tail_negative(dentry)) { @@ -654,6 +655,8 @@ static void sweep_negative(struct dentry *dentry) spin_unlock(&parent->d_lock); } +out: + spin_unlock(&dentry->d_lock); } /* @@ -922,7 +925,8 @@ void dput(struct dentry *dentry) if (likely(retain_dentry(dentry))) { if (d_is_negative(dentry)) sweep_negative(dentry); - spin_unlock(&dentry->d_lock); + else + spin_unlock(&dentry->d_lock); return; } From patchwork Thu Jan 21 13:19:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham Ananthakrishna X-Patchwork-Id: 12036149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A05CC433E0 for ; Thu, 21 Jan 2021 13:21:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1C518239FD for ; Thu, 21 Jan 2021 13:21:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C518239FD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A40316B0012; Thu, 21 Jan 2021 08:21:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CA196B0022; Thu, 21 Jan 2021 08:21:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B95E6B0023; Thu, 21 Jan 2021 08:21:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0117.hostedemail.com [216.40.44.117]) by kanga.kvack.org (Postfix) with ESMTP id 7484B6B0012 for ; Thu, 21 Jan 2021 08:21:31 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4178D181AEF1F for ; Thu, 21 Jan 2021 13:21:31 +0000 (UTC) X-FDA: 77729843982.16.level42_531371427563 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 17812100E690B for ; Thu, 21 Jan 2021 13:21:31 +0000 (UTC) X-HE-Tag: level42_531371427563 X-Filterd-Recvd-Size: 7192 Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Jan 2021 13:21:30 +0000 (UTC) Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDEq8m095432; Thu, 21 Jan 2021 13:21:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=QBObwOkC+BCQ8yX5EsfVIOGtrp1YnjhtZHLKG9PfSg8=; b=ZLX20WzQQ/bo7PLkOdw15YtVaBo7IiJKDMmC2uFMUemQwoBdEE/5cIYQijHL0FmAaxdm wmmSLdDxID++3AVb28cgltLX62yIhusODAt98dB+0Ix/Mvmytq6P03KaarmV8pVshsP6 J8l9Hrxxz6r5i6TYX9VHM1EEj0Jciy6/QUFsO2Z9yPU35wv9eDAt0O7Rq0NsG3UzQoy6 89j1QUSK+z42B4eQVoOIsZUZgrlgpxI+nACcDa68BgxSXhR2mAbsEMqWlLAp1i63aVbg KPigXqfCiMGR5ZoPPp5b3+MsEv2C9YYRY6cV6ND1f+o8zddwCltPyxeSajmnbRK1gdms IQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 3668qrf9qy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:21:29 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10LDFkUG106713; Thu, 21 Jan 2021 13:21:28 GMT Received: from pps.reinject (localhost [127.0.0.1]) by userp3030.oracle.com with ESMTP id 3668rexrf4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Jan 2021 13:21:28 +0000 Received: from userp3030.oracle.com (userp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 10LDKZJJ123118; Thu, 21 Jan 2021 13:21:28 GMT Received: from gmananth-linux.oraclecorp.com (dhcp-10-166-171-141.vpn.oracle.com [10.166.171.141]) by userp3030.oracle.com with ESMTP id 3668rexq88-7; Thu, 21 Jan 2021 13:21:28 +0000 From: Gautham Ananthakrishna To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: viro@zeniv.linux.org.uk, matthew.wilcox@oracle.com, khlebnikov@yandex-team.ru, gautham.ananthakrishna@oracle.com Subject: [PATCH RFC 6/6] dcache: prevent flooding with negative dentries Date: Thu, 21 Jan 2021 18:49:45 +0530 Message-Id: <1611235185-1685-7-git-send-email-gautham.ananthakrishna@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> References: <1611235185-1685-1-git-send-email-gautham.ananthakrishna@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9870 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=321 mlxscore=0 suspectscore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 spamscore=0 phishscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101210072 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov Without memory pressure count of negative dentries isn't bounded. They could consume all memory and drain all other inactive caches. Typical scenario is an idle system where some process periodically creates temporary files and removes them. After some time, memory will be filled with negative dentries for these random file names. Reclaiming them took some time because slab frees pages only when all related objects are gone. Time of dentry lookup is usually unaffected because hash table grows along with size of memory. Unless somebody especially crafts hash collisions. Simple lookup of random names also generates negative dentries very fast. This patch implements heuristic which detects such scenarios and prevents unbounded growth of completely unneeded negative dentries. It keeps up to three latest negative dentry in each bucket unless they were referenced. At first dput of negative dentry when it swept to the tail of siblings we'll also clear it's reference flag and look at next dentries in chain. Then kill third in series of negative, unused and unreferenced denries. This way each hash bucket will preserve three negative dentry to let them get reference and survive. Adding positive or used dentry into hash chain also protects few recent negative dentries. In result total size of dcache asymptotically limited by count of buckets and positive or used dentries. Before patch: tool 'dcache_stress' could fill entire memory with dentries. nr_dentry = 104913261 104.9M nr_buckets = 8388608 12.5 avg nr_unused = 104898729 100.0% nr_negative = 104883218 100.0% After this patch count of dentries saturates at around 3 per bucket: nr_dentry = 24619259 24.6M nr_buckets = 8388608 2.9 avg nr_unused = 24605226 99.9% nr_negative = 24600351 99.9% This heuristic isn't bulletproof and solves only most practical case. It's easy to deceive: just touch same random name twice. Signed-off-by: Konstantin Khlebnikov Signed-off-by: Gautham Ananthakrishna --- fs/dcache.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/fs/dcache.c b/fs/dcache.c index 22c990b..6281938 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -633,6 +633,58 @@ static inline struct dentry *lock_parent(struct dentry *dentry) } /* + * Called at first dput of each negative dentry. + * Prevents filling cache with never reused negative dentries. + * + * This clears reference and then looks at following dentries in hash chain. + * If they are negative, unused and unreferenced then keep two and kill third. + */ +static void trim_negative(struct dentry *dentry) + __releases(dentry->d_lock) +{ + struct dentry *victim, *parent; + struct hlist_bl_node *next; + int keep = 2; + + rcu_read_lock(); + + dentry->d_flags &= ~DCACHE_REFERENCED; + spin_unlock(&dentry->d_lock); + + next = rcu_dereference_raw(dentry->d_hash.next); + while (1) { + victim = hlist_bl_entry(next, struct dentry, d_hash); + + if (!next || d_count(victim) || !d_is_negative(victim) || + (victim->d_flags & DCACHE_REFERENCED)) { + rcu_read_unlock(); + return; + } + + if (!keep--) + break; + + next = rcu_dereference_raw(next->next); + } + + spin_lock(&victim->d_lock); + parent = lock_parent(victim); + + rcu_read_unlock(); + + if (d_count(victim) || !d_is_negative(victim) || + (victim->d_flags & DCACHE_REFERENCED)) { + if (parent) + spin_unlock(&parent->d_lock); + spin_unlock(&victim->d_lock); + return; + } + + __dentry_kill(victim); + dput(parent); +} + +/* * Move cached negative dentry to the tail of parent->d_subdirs. * This lets walkers skip them all together at first sight. * Must be called at dput of negative dentry. @@ -654,6 +706,8 @@ static void sweep_negative(struct dentry *dentry) } spin_unlock(&parent->d_lock); + + return trim_negative(dentry); } out: spin_unlock(&dentry->d_lock);