From patchwork Thu Aug 30 21:55:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10582913 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99BFB5A4 for ; Thu, 30 Aug 2018 21:55:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89B982BE90 for ; Thu, 30 Aug 2018 21:55:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D4E32C24B; Thu, 30 Aug 2018 21:55:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB1372BE90 for ; Thu, 30 Aug 2018 21:55:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFEB16B5393; Thu, 30 Aug 2018 17:55:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A841F6B5395; Thu, 30 Aug 2018 17:55:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94D986B5396; Thu, 30 Aug 2018 17:55:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id 571F36B5394 for ; Thu, 30 Aug 2018 17:55:29 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id d1-v6so10445870qth.21 for ; Thu, 30 Aug 2018 14:55:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=CCBS3Wuve8RiXTWHGrwTznKM/VN762uDCXQIsdJF4Ag=; b=AuPY+Swc7W4M43bpA5hrjmrvetnffQHurHdWIcZQkn86GRnF1sWRzTXP1RuzK1JvBy sJSWBCC/jana6Z2EheShu3A7XUTQ7n+4GKL+F0gZgELxJVzOsfzoH20kEjIctUM5giOg 6dUBpvNedZZdDu3Ry/nLqBfWUel96DoOmQytxdE3CJ6P5Lshc0Y8vIswbc2bLtFuz0Bf mS7809xP3JiBotdFDJPGQrnrNzDyAiIIbytNgrmHOrrjb3o70/2lbu7/b3FFEidPSQ7M bynkgFiw5bfotuURK8jrtgAz0DS9nTCc3bauQIqgqUYgBMKcWs8ZwuzheiiY42QEPgzt Qjaw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51Dil9IvlVsbm7lEszBXM8XebL2sxkpQ/orLGO2DS0wxrpobRysE xrStNgylobVP1OLVU8FQ1PxhaRX/J+KQxhAp/wtgeft4bmXy0GolO3Ib+eH7bn/WmNCf4VpeIRa CI8hxgHFyniLppFR+NjtpQMx/xdDhQi//YTL/nXlmza5HGtbqtj8HYeGFkY8L3LL1oA== X-Received: by 2002:a37:b12:: with SMTP id 18-v6mr13271576qkl.167.1535666129117; Thu, 30 Aug 2018 14:55:29 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYACnBmPD7d/eg9BxvDSiARGhVBxaiudFlBnfZk2DETu1PCBQQIYrtcJrSDYgzHVsocY8ji X-Received: by 2002:a37:b12:: with SMTP id 18-v6mr13271558qkl.167.1535666128529; Thu, 30 Aug 2018 14:55:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535666128; cv=none; d=google.com; s=arc-20160816; b=GLyEskvIVBQryZDln78OhCCKoiXi0ZBtMinCewE/VzGZaiKn5mv5CafxifsDOo3j4v zIqS9KQaZgggNW6cc+3BLPZMvcGCWsu89ycLEfz8w+F32xVhh5JDTTqmKa8MM4guW+o2 GmdxnpdguIgBGUdphyx5oqQ3ofPcenSA8CYnzU1xcUWC8kML6oIM/uIAyJUxcoDXYe33 nX1MFqwOgnUgL5+KLtzYNyCNsg4lI1ivKIIbr3o84OQHl8ZmgHdySSknHZIVTd7YkKKK 625k/xhcNPFxeGRg7W0ho94T4aP0fUEFnWW0Pjv1dQW07o/VLhZMHVlHMy500PJUbgJ0 VLjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=CCBS3Wuve8RiXTWHGrwTznKM/VN762uDCXQIsdJF4Ag=; b=Biv/H8rcsQb69SA/HXZ6WlcWHwWNNNBadvrpLAuXM6L2Nur5MKNNamChARJ1qgN1hT M6Y/sxhnVbIoUItr2wyXt/yoo2tf0MPtsx1VBB/HMQEx/A6AYi8NbyE4znoHUIoxt1oJ erUuwVz0Oi4kkknOQFcWoaaDfMFl6YkdzozxSE+l/1U8jsvlsgO+Ruj6gjju+Y2bn4xZ pEBAfa4HzynNIAJby4LzHb9nLk0GpklnZXkhKn9F4Dvs1gLSqG5v0OoYeHulafB+SKrY aBUliMvPCT+ARc7yluEbgKNGcuNdPquKAX5B4ylPNVh9ZD9tpcBu64IW2kwGhAE/u7Yr MNmA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id m22-v6si1051503qtg.95.2018.08.30.14.55.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Aug 2018 14:55:28 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C1FFA7788D; Thu, 30 Aug 2018 21:55:27 +0000 (UTC) Received: from llong.com (dhcp-17-8.bos.redhat.com [10.18.17.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id 325FD2157F49; Thu, 30 Aug 2018 21:55:27 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v2 1/4] fs/dcache: Fix incorrect nr_dentry_unused accounting in shrink_dcache_sb() Date: Thu, 30 Aug 2018 17:55:04 -0400 Message-Id: <1535666107-25699-2-git-send-email-longman@redhat.com> In-Reply-To: <1535666107-25699-1-git-send-email-longman@redhat.com> References: <1535666107-25699-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Thu, 30 Aug 2018 21:55:28 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Thu, 30 Aug 2018 21:55:28 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The nr_dentry_unused per-cpu counter tracks dentries in both the LRU lists and the shrink lists where the DCACHE_LRU_LIST bit is set. The shrink_dcache_sb() function moves dentries from the LRU list to a shrink list and subtracts the dentry count from nr_dentry_unused. This is incorrect as the nr_dentry_unused count Will also be decremented in shrink_dentry_list() via d_shrink_del(). To fix this double decrement, the decrement in the shrink_dcache_sb() function is taken out. Fixes: 4e717f5c1083 ("list_lru: remove special case function list_lru_dispose_all." Signed-off-by: Waiman Long --- fs/dcache.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 2e7e8d8..cb515f1 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1202,15 +1202,11 @@ static enum lru_status dentry_lru_isolate_shrink(struct list_head *item, */ void shrink_dcache_sb(struct super_block *sb) { - long freed; - do { LIST_HEAD(dispose); - freed = list_lru_walk(&sb->s_dentry_lru, + list_lru_walk(&sb->s_dentry_lru, dentry_lru_isolate_shrink, &dispose, 1024); - - this_cpu_sub(nr_dentry_unused, freed); shrink_dentry_list(&dispose); } while (list_lru_count(&sb->s_dentry_lru) > 0); } From patchwork Thu Aug 30 21:55:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10582915 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 904F113AC for ; Thu, 30 Aug 2018 21:55:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8102E2BE90 for ; Thu, 30 Aug 2018 21:55:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 74F812C24B; Thu, 30 Aug 2018 21:55:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 164932BE90 for ; Thu, 30 Aug 2018 21:55:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 188876B5394; Thu, 30 Aug 2018 17:55:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 10D896B5395; Thu, 30 Aug 2018 17:55:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC9FC6B5396; Thu, 30 Aug 2018 17:55:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id 98F7B6B5394 for ; Thu, 30 Aug 2018 17:55:29 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id y54-v6so10408340qta.8 for ; Thu, 30 Aug 2018 14:55:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=4B/Wm4vURCyVHVLH/q3jpiYPWlPN176DKvy63T1eH4Y=; b=uCDtDg1LGWxFg5kELTZwcTKS0XuhSRSBAoIwNWxe0PiQCqwkuIIAaotpZ0fg49fhd1 il5fYutn2gWd6xFNk2fGEqj1Tl7d+Kensm1UEkDU06pXrkws25tHtfaEbbZnW2dSF+lA LREeZLh1qsfopx+qH8lekduNo2shmf3brf/Pqesqrf+51VAAsgJiehYl6/2kssMyv/qV Se+qcwy8nHmCKl8fwb0D2b3/2j4DuBf/3h4RjxHvXrk1F5osw/zYM/urdfNkFKJ4/YSo R2Eo3lZK+HJtzF/d9aB6yudcFqYeBnf055JCb+xUgq5MdzyCYenzEAucv1/QX01d85+v JhtQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51CKyMLD/mLPeTCHnrD6G2knmfY92gEWTJaZOZ/4IU7uhR64ejKE KcNtaKnA4Cd8Mt6NMWEB2wk1nCjj2gIKUJ+JL7AUoFozd9+7bEi8st3rt/DMFMCia/BdLo5amVg q7+TTfmQFp3g6C4fjvpbZgVxxGvwzR+DNKD+DJEHnTnXf5ecyhWW57mmsfP8pm0G6QA== X-Received: by 2002:a37:a0c2:: with SMTP id j185-v6mr13345464qke.67.1535666129389; Thu, 30 Aug 2018 14:55:29 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZ3bsP5hpL9uDmGuaIeheGePhUmZASixU/sJ5+3nkG5TJNfsKJwA+okf7thmOdJf7ZJpKDw X-Received: by 2002:a37:a0c2:: with SMTP id j185-v6mr13345443qke.67.1535666128888; Thu, 30 Aug 2018 14:55:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535666128; cv=none; d=google.com; s=arc-20160816; b=kp/HkURYR/cxeUBFGWs5AR7Q5L03e+T+MviIH5mGoC97Rj54e5D/3fgv8LS91wgTZT vsS37uZx6abmpulEsVXrLOQ15xFNyLgvgCuW0NVBCDRAGf128Au7+T+F8ndYIhhzin8T DIIy8v2jUgSLPoD4C2O2bIHOlVaqu9D3hQTZ4Xuq4YrPKbwITzGwH9KEQatKRlmUUxDl SLPR0MP6AZ9LXH40PVjf7v6gsaOWNE2Xq+fkXMj55NddLcnhNVb8vQTHfwab99TwGqIe 1qvNI5hSn4eygK1WFqCo+zmLpC7uKFJ0igcTTRJDXP6cq9R1wWz27X0f1UIcPODKclWI luNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=4B/Wm4vURCyVHVLH/q3jpiYPWlPN176DKvy63T1eH4Y=; b=D+ygVxa0HnlYP/Y+hMU3kO4/f9QVF+/puyGZYrRdUl94DrwfBF6Zu1pUuPLW2mOnDp SMbmaL94Am6zSK3rVSXurcHqlR1rTjPno7coi2vptYXUftnd9OmqlXEZCxg3FQB6rwKt A8d1j0QR8SjUc8Xk9MsqtbDD4RcfDPWx+0YJiAVfzAaPFwFH0QLGxCbNScBKZmjsUazj o37tpuTBvAGxvByeM2SkEnUt0XKulW63pUZMxRSZej8i6gWII8U1h1DiMXRnoSbiVIa8 rXTlqWsdnVC6LEXbHcB30H26yw3QYUertyN5iPrcS937TumkNFGiBJxxiM0jpG1aDpVS 2Cvg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id 14-v6si7913924qvj.188.2018.08.30.14.55.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Aug 2018 14:55:28 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5A91640216E7; Thu, 30 Aug 2018 21:55:28 +0000 (UTC) Received: from llong.com (dhcp-17-8.bos.redhat.com [10.18.17.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id BE6BD2157F4A; Thu, 30 Aug 2018 21:55:27 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v2 2/4] fs: Don't need to put list_lru into its own cacheline Date: Thu, 30 Aug 2018 17:55:05 -0400 Message-Id: <1535666107-25699-3-git-send-email-longman@redhat.com> In-Reply-To: <1535666107-25699-1-git-send-email-longman@redhat.com> References: <1535666107-25699-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 30 Aug 2018 21:55:28 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 30 Aug 2018 21:55:28 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The list_lru structure is essentially just a pointer to a table of per-node LRU lists. Even if CONFIG_MEMCG_KMEM is defined, the list field is just used for LRU list registration and shrinker_id is set at initialization. Those fields won't need to be touched that often. So there is no point to make the list_lru structures to sit in their own cachelines. Signed-off-by: Waiman Long --- include/linux/fs.h | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 3332270..fd4cd8a 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1443,11 +1443,12 @@ struct super_block { struct user_namespace *s_user_ns; /* - * Keep the lru lists last in the structure so they always sit on their - * own individual cachelines. + * The list_lru structure is essentially just a pointer to a table + * of per-node lru lists, each of which has its own spinlock. + * There is no need to put them into separate cachelines. */ - struct list_lru s_dentry_lru ____cacheline_aligned_in_smp; - struct list_lru s_inode_lru ____cacheline_aligned_in_smp; + struct list_lru s_dentry_lru; + struct list_lru s_inode_lru; struct rcu_head rcu; struct work_struct destroy_work; From patchwork Thu Aug 30 21:55:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10582919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BA3F5A4 for ; Thu, 30 Aug 2018 21:55:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4C62C2BE90 for ; Thu, 30 Aug 2018 21:55:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3FE0B2C24B; Thu, 30 Aug 2018 21:55:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93F4F2C24A for ; Thu, 30 Aug 2018 21:55:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4D276B5395; Thu, 30 Aug 2018 17:55:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5D496B5397; Thu, 30 Aug 2018 17:55:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB15C6B5398; Thu, 30 Aug 2018 17:55:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f199.google.com (mail-qt0-f199.google.com [209.85.216.199]) by kanga.kvack.org (Postfix) with ESMTP id 6C6516B5395 for ; Thu, 30 Aug 2018 17:55:30 -0400 (EDT) Received: by mail-qt0-f199.google.com with SMTP id e14-v6so10379427qtp.17 for ; Thu, 30 Aug 2018 14:55:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=pDthHmUayMvAyYA01Z0SVkDMqF47g0eC4R9HaxByrIs=; b=VyxOZllAJu2tKjAJ8z5w+iiTiUVPfFIUNPHTdLL/uAIMJAFqekgfQFo7Hb4YM58GDM VJHUe9pFdakpWuC+LsaYs1cN5AMiZOLC/9ViDVe+eFnE/FjogN1JzV7vpgi14XTmBYbj smwLZeJGLfAYobvmKnL87XLIBe+0/3GS/A9fXW/EFp1ODqL04117KnSyd/VeaIS0mEnd hLPXNCboKMqT/Y3U2EyfrATB7yD1igjfCtWolInkuGSkcUhAkRQEC2+EJUakhV+GT6x2 0r6zUEbhfmuTTR8V7zLHa51g1ETEfulzxdlQ+1CneIx0CW5wnJveiCXGgSHjDLtex6eO gBkg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51BGaVeBPlEh5l9iJw0D9+MJpa2TJCHOZKF/cn5aLkNEGnPVRYSd DGbBIJgRPjrnmxZUpOB63Fr+1wqaZIjTJNkbfh7mbK9pAWJO1b3Dq4D6pIu3DKT3yuvRNZ7dmfK TkLFSP7qZOL6Q/EEoDm0tfyY7nft4n/UqKYnUtprWlJYx/caHxnnG4Vjn26p1msFJ5Q== X-Received: by 2002:ac8:2204:: with SMTP id o4-v6mr13394189qto.332.1535666130205; Thu, 30 Aug 2018 14:55:30 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZeoZ7E3gbh+l+p+u4JqX4UBdhQ4/oJFKBjjQb+BUwpFrUKE/0PEjDHzfAQy9dIW0KIZGgX X-Received: by 2002:ac8:2204:: with SMTP id o4-v6mr13394158qto.332.1535666129343; Thu, 30 Aug 2018 14:55:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535666129; cv=none; d=google.com; s=arc-20160816; b=mV2gdlN5b2wLHOzQANdsSj7Zi/jbhhNHKMnamHb+bWjwPUPR7/P9R7O6i7wUYJfn9L fN3QDh97yCXDyTwdDFLDsJixeP2fiEIIRqvAlV8BH7KweAybRuFnIJOSu6xjsWQE+T/C 0Q7HtYCKG6f4b1N1sXk/nMYMyAgSIsOdZ4aLoUN29Eo9b3TaPjPcd2/q7LTamfQD3BYk cimtKysRzfAhehdml8pczw6Ioz6CImNbP+zi8ir0pu9WwiSLMCx2dvxHeyoGaeQcJUkH QR7yHFURXXxCQE6QkJPSWVs5dX9UCuMrnNl5kjkFFgxbgHVZGk9ffqssNXaqF35OU/RD g1Fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=pDthHmUayMvAyYA01Z0SVkDMqF47g0eC4R9HaxByrIs=; b=GJg/aQOrlySoVTwvKq/69xAbSTcGCXCZnzUj53qCWp1s2lFJcFxZUoEoKlIsd5Um1w OQWBUiRKhWf3osbTIfH5FEUwyTPi2/aF1sjhSUyzL0iqKdQec9wY1u6ciribeL6CWuMv rV7KR5dKTgZd8Fxt8GYRLlsaR7yss2picDCGLCsEjWrVDblYueq1yMHHgTO70wFm0xXF y4EEifO1aBEbINrfdUvAGgTlxfMDhIKauezt4FWc/LO+in4qKZYkvZOp9hhDcE4cQ0iW 7mfSuMHaZxUHD4XiDa99wkXk7bJ+gXfKDsTuPRJWgXYZnOS+Umw5oHa11xu+rUIAEiri dRkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id z74-v6si6504328qkg.148.2018.08.30.14.55.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Aug 2018 14:55:29 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E44BE8151D46; Thu, 30 Aug 2018 21:55:28 +0000 (UTC) Received: from llong.com (dhcp-17-8.bos.redhat.com [10.18.17.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5FD892157F4B; Thu, 30 Aug 2018 21:55:28 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v2 3/4] fs/dcache: Track & report number of negative dentries Date: Thu, 30 Aug 2018 17:55:06 -0400 Message-Id: <1535666107-25699-4-git-send-email-longman@redhat.com> In-Reply-To: <1535666107-25699-1-git-send-email-longman@redhat.com> References: <1535666107-25699-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 30 Aug 2018 21:55:29 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Thu, 30 Aug 2018 21:55:29 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The current dentry number tracking code doesn't distinguish between positive & negative dentries. It just reports the total number of dentries in the LRU lists. As excessive number of negative dentries can have an impact on system performance, it will be wise to track the number of positive and negative dentries separately. This patch adds tracking for the total number of negative dentries in the system LRU lists and reports it in the 7th field in the /proc/sys/fs/dentry-state file. The number, however, does not include negative dentries that are in flight but not in the LRU yet as well as those in the shrinker lists. The number of positive dentries in the LRU lists can be roughly found by subtracting the number of negative dentries from the unused count. Signed-off-by: Waiman Long --- Documentation/sysctl/fs.txt | 22 ++++++++++++++-------- fs/dcache.c | 31 +++++++++++++++++++++++++++++++ include/linux/dcache.h | 5 +++-- kernel/sysctl.c | 2 +- 4 files changed, 49 insertions(+), 11 deletions(-) diff --git a/Documentation/sysctl/fs.txt b/Documentation/sysctl/fs.txt index 819caf8..21679d0 100644 --- a/Documentation/sysctl/fs.txt +++ b/Documentation/sysctl/fs.txt @@ -56,26 +56,32 @@ of any kernel data structures. dentry-state: -From linux/fs/dentry.c: +From linux/include/linux/dcache.h: -------------------------------------------------------------- -struct { +struct dentry_stat_t dentry_stat { int nr_dentry; int nr_unused; int age_limit; /* age in seconds */ int want_pages; /* pages requested by system */ int dummy[2]; -} dentry_stat = {0, 0, 45, 0,}; --------------------------------------------------------------- + int nr_negative; /* # of unused negative dentries */ +}; +-------------------------------------------------------------- + +Dentries are dynamically allocated and deallocated. + +nr_dentry shows the total number of dentries allocated (active ++ unused). nr_unused shows the number of dentries that are not +actively used, but are saved in the LRU list for future reuse. -Dentries are dynamically allocated and deallocated, and -nr_dentry seems to be 0 all the time. Hence it's safe to -assume that only nr_unused, age_limit and want_pages are -used. Nr_unused seems to be exactly what its name says. Age_limit is the age in seconds after which dcache entries can be reclaimed when memory is short and want_pages is nonzero when shrink_dcache_pages() has been called and the dcache isn't pruned yet. +nr_negative shows the number of unused dentries that are also +negative dentries which do not mapped to actual files. + ============================================================== dquot-max & dquot-nr: diff --git a/fs/dcache.c b/fs/dcache.c index cb515f1..c1cc956 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -119,6 +119,7 @@ struct dentry_stat_t dentry_stat = { static DEFINE_PER_CPU(long, nr_dentry); static DEFINE_PER_CPU(long, nr_dentry_unused); +static DEFINE_PER_CPU(long, nr_dentry_negative); #if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS) @@ -152,11 +153,22 @@ static long get_nr_dentry_unused(void) return sum < 0 ? 0 : sum; } +static long get_nr_dentry_negative(void) +{ + int i; + long sum = 0; + + for_each_possible_cpu(i) + sum += per_cpu(nr_dentry_negative, i); + return sum < 0 ? 0 : sum; +} + int proc_nr_dentry(struct ctl_table *table, int write, void __user *buffer, size_t *lenp, loff_t *ppos) { dentry_stat.nr_dentry = get_nr_dentry(); dentry_stat.nr_unused = get_nr_dentry_unused(); + dentry_stat.nr_negative = get_nr_dentry_negative(); return proc_doulongvec_minmax(table, write, buffer, lenp, ppos); } #endif @@ -331,6 +343,8 @@ static inline void __d_clear_type_and_inode(struct dentry *dentry) flags &= ~(DCACHE_ENTRY_TYPE | DCACHE_FALLTHRU); WRITE_ONCE(dentry->d_flags, flags); dentry->d_inode = NULL; + if (dentry->d_flags & DCACHE_LRU_LIST) + this_cpu_inc(nr_dentry_negative); } static void dentry_free(struct dentry *dentry) @@ -385,6 +399,10 @@ static void dentry_unlink_inode(struct dentry * dentry) * The per-cpu "nr_dentry_unused" counters are updated with * the DCACHE_LRU_LIST bit. * + * The per-cpu "nr_dentry_negative" counters are only updated + * when deleted or added to the per-superblock LRU list, not + * on the shrink list. + * * These helper functions make sure we always follow the * rules. d_lock must be held by the caller. */ @@ -394,6 +412,8 @@ static void d_lru_add(struct dentry *dentry) D_FLAG_VERIFY(dentry, 0); dentry->d_flags |= DCACHE_LRU_LIST; this_cpu_inc(nr_dentry_unused); + if (d_is_negative(dentry)) + this_cpu_inc(nr_dentry_negative); WARN_ON_ONCE(!list_lru_add(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -402,6 +422,8 @@ static void d_lru_del(struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); + if (d_is_negative(dentry)) + this_cpu_dec(nr_dentry_negative); WARN_ON_ONCE(!list_lru_del(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -432,6 +454,8 @@ static void d_lru_isolate(struct list_lru_one *lru, struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); + if (d_is_negative(dentry)) + this_cpu_dec(nr_dentry_negative); list_lru_isolate(lru, &dentry->d_lru); } @@ -440,6 +464,8 @@ static void d_lru_shrink_move(struct list_lru_one *lru, struct dentry *dentry, { D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags |= DCACHE_SHRINK_LIST; + if (d_is_negative(dentry)) + this_cpu_dec(nr_dentry_negative); list_lru_isolate_move(lru, &dentry->d_lru, list); } @@ -1836,6 +1862,11 @@ static void __d_instantiate(struct dentry *dentry, struct inode *inode) WARN_ON(d_in_lookup(dentry)); spin_lock(&dentry->d_lock); + /* + * Decrement negative dentry count if it was in the LRU list. + */ + if (dentry->d_flags & DCACHE_LRU_LIST) + this_cpu_dec(nr_dentry_negative); hlist_add_head(&dentry->d_u.d_alias, &inode->i_dentry); raw_write_seqcount_begin(&dentry->d_seq); __d_set_inode_and_type(dentry, inode, add_flags); diff --git a/include/linux/dcache.h b/include/linux/dcache.h index ef4b70f..619a249 100644 --- a/include/linux/dcache.h +++ b/include/linux/dcache.h @@ -62,9 +62,10 @@ struct qstr { struct dentry_stat_t { long nr_dentry; long nr_unused; - long age_limit; /* age in seconds */ - long want_pages; /* pages requested by system */ + long age_limit; /* age in seconds */ + long want_pages; /* pages requested by system */ long dummy[2]; + long nr_negative; /* # of unused negative dentries */ }; extern struct dentry_stat_t dentry_stat; diff --git a/kernel/sysctl.c b/kernel/sysctl.c index cc02050..77ff61c 100644 --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -1708,7 +1708,7 @@ static int sysrq_sysctl_handler(struct ctl_table *table, int write, { .procname = "dentry-state", .data = &dentry_stat, - .maxlen = 6*sizeof(long), + .maxlen = 7*sizeof(long), .mode = 0444, .proc_handler = proc_nr_dentry, }, From patchwork Thu Aug 30 21:55:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10582921 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F06613AC for ; Thu, 30 Aug 2018 21:55:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F3A552BE90 for ; Thu, 30 Aug 2018 21:55:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E73572C24B; Thu, 30 Aug 2018 21:55:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 654DD2BE90 for ; Thu, 30 Aug 2018 21:55:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 533E56B5397; Thu, 30 Aug 2018 17:55:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4D2AD6B5398; Thu, 30 Aug 2018 17:55:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30C146B5399; Thu, 30 Aug 2018 17:55:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id CD8CE6B5398 for ; Thu, 30 Aug 2018 17:55:30 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id a15-v6so10559885qtj.15 for ; Thu, 30 Aug 2018 14:55:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=jWlOFtELmXfxC3ynjpZuAA8uJ4Rm+lS3xewNX5/reIA=; b=nfG9lt8zW1I8j18xi/RSYm7qIpXpha79e+VL9LyU+CZqlAykAae28At9M+dcEvYXen hMq9l29EDUA5TiIV+02ByttLGc47DyexIqVe/XNKOZDKicgxXi4ZRzaV7NGvfk50VO0R dzXm8T0XyF3wDl5OiRkBGBDgE+i0q8WYMTbK4Xfpv+4xg/hhTycwhBe5ZA1s5lR1ipxa PRvvAlTgjXxFSmGgAVIpYa6LK8jmlD9cPcIRpQT4yJNtw2wPejLmrUoACB7nZf2mrNVP 3zRp93amBlSavD87hZObTwXM63hab6WFvKNKD4uvXzxEwoEg0Ad7K6aCEWAx+xVabFsn +7ZA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51DbcCUF/vw4soOyE2YgXKJrEKKrSykbGaMW+co9snl1Qt7wGp1/ EMNBcnWy4bPKUztjA4p9/SB9obwcMg6pv1DrjPC9Vi3rSwoDFeXBEDhXaSmalAZE9TwZqsVeNCB 3MR+DGro85C5BoKa93SzE3varIGjs5bbGSitTZqa8WklbuOUG1f86uoxownhOOiT21g== X-Received: by 2002:a0c:aa81:: with SMTP id f1-v6mr13373781qvb.80.1535666130581; Thu, 30 Aug 2018 14:55:30 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaJbwehBXlRcsgUU/DwrQmT/upx5+aABIEd6/wofkjLnmNMOhQaSJvuQapPJ2CK3704+9Yz X-Received: by 2002:a0c:aa81:: with SMTP id f1-v6mr13373754qvb.80.1535666129984; Thu, 30 Aug 2018 14:55:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535666129; cv=none; d=google.com; s=arc-20160816; b=jrZs7tzpY5MrH4CW2DUNUVixXImHMIsPkBtQGb1rBNOw+MMeOWSsntxs9vWRyPumxZ 4WArJH3BgltqlLul59DuSYZ+TExmsXVZwTiUwSu4WdoJr9s+NO+lmI4fwqYopLjQsahI SVGchk5Ui/0MTsYC6XSyAi/gctBCErLohuRwml5Bd3WO/Md3w8ZgamI63skVeUqDlgXS P590l6z6iWiSBstf2B93h3WcFUkUyefKqxYf0i4Mv0QoA6f1v74gMIDdZLcU7AligTT+ Uyw3vMjToRiF74sIwecFxVw3/q5S5OnXRiz0zgVNXWdFRP1z0ajkxpos6bKPsOQZdh1/ uwPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=jWlOFtELmXfxC3ynjpZuAA8uJ4Rm+lS3xewNX5/reIA=; b=dbN//49faUqhoqiGMNNmyLAt/Y7bHbGxPi/mh6r/Cg9QrlAKFYuYtqhI4HvuzchtU6 vzMjkxn7bpqjwmY2NLGqxUgvvxmXzGIsjWI6gNTEoh7Q6Yk9wH8R+iWUVhLlXFGMGc2M 3lHKF6JIfk0M5ZKhBzuELBwRL8AxR3lvmE1DeqenQqgfdHBNlWvDme9UammZjxwk0X6W wlRpRVh8nM5y4PlhjOjZJPsZlu5s3d/I3sq60D0LyIBbeup6vjIFyPm55vfla1Ps5LVt 0+A4UNkirbSgsNIOXgDBad/XjKlQI+zTmnAqohAsR3LCpQvI74zO+VNUsxhCCP4pvOFg 2GOg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id x195-v6si1097481qkx.393.2018.08.30.14.55.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Aug 2018 14:55:29 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 80BB67DAC2; Thu, 30 Aug 2018 21:55:29 +0000 (UTC) Received: from llong.com (dhcp-17-8.bos.redhat.com [10.18.17.8]) by smtp.corp.redhat.com (Postfix) with ESMTP id EAFEE2157F4A; Thu, 30 Aug 2018 21:55:28 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v2 4/4] fs/dcache: Eliminate branches in nr_dentry_negative accounting Date: Thu, 30 Aug 2018 17:55:07 -0400 Message-Id: <1535666107-25699-5-git-send-email-longman@redhat.com> In-Reply-To: <1535666107-25699-1-git-send-email-longman@redhat.com> References: <1535666107-25699-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 30 Aug 2018 21:55:29 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Thu, 30 Aug 2018 21:55:29 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Because the accounting of nr_dentry_negative depends on whether a dentry is a negative one or not, branch instructions are introduced to handle the accounting conditionally. That may potentially slow down the task by a noticeable amount if that introduces sizeable amount of additional branch mispredictions. To avoid that, the accounting code is now modified to use conditional move instructions instead, if supported by the architecture. Signed-off-by: Waiman Long --- fs/dcache.c | 41 +++++++++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index c1cc956..dfd5628 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -171,6 +171,29 @@ int proc_nr_dentry(struct ctl_table *table, int write, void __user *buffer, dentry_stat.nr_negative = get_nr_dentry_negative(); return proc_doulongvec_minmax(table, write, buffer, lenp, ppos); } + +/* + * Increment/Decrement nr_dentry_negative if the condition is true. + * For architectures that support some kind of conditional move, compiler + * should be able generate code to inc/dec negative dentry counter + * without any branch instruction. + */ +static inline void cond_negative_dentry_inc(bool cond) +{ + int val = !!cond; + + this_cpu_add(nr_dentry_negative, val); +} + +static inline void cond_negative_dentry_dec(bool cond) +{ + int val = !!cond; + + this_cpu_sub(nr_dentry_negative, val); +} +#else +static inline void cond_negative_dentry_inc(bool cond) { } +static inline void cond_negative_dentry_dec(bool cond) { } #endif /* @@ -343,8 +366,7 @@ static inline void __d_clear_type_and_inode(struct dentry *dentry) flags &= ~(DCACHE_ENTRY_TYPE | DCACHE_FALLTHRU); WRITE_ONCE(dentry->d_flags, flags); dentry->d_inode = NULL; - if (dentry->d_flags & DCACHE_LRU_LIST) - this_cpu_inc(nr_dentry_negative); + cond_negative_dentry_inc(dentry->d_flags & DCACHE_LRU_LIST); } static void dentry_free(struct dentry *dentry) @@ -412,8 +434,7 @@ static void d_lru_add(struct dentry *dentry) D_FLAG_VERIFY(dentry, 0); dentry->d_flags |= DCACHE_LRU_LIST; this_cpu_inc(nr_dentry_unused); - if (d_is_negative(dentry)) - this_cpu_inc(nr_dentry_negative); + cond_negative_dentry_inc(d_is_negative(dentry)); WARN_ON_ONCE(!list_lru_add(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -422,8 +443,7 @@ static void d_lru_del(struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); - if (d_is_negative(dentry)) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(d_is_negative(dentry)); WARN_ON_ONCE(!list_lru_del(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -454,8 +474,7 @@ static void d_lru_isolate(struct list_lru_one *lru, struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); - if (d_is_negative(dentry)) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(d_is_negative(dentry)); list_lru_isolate(lru, &dentry->d_lru); } @@ -464,8 +483,7 @@ static void d_lru_shrink_move(struct list_lru_one *lru, struct dentry *dentry, { D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags |= DCACHE_SHRINK_LIST; - if (d_is_negative(dentry)) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(d_is_negative(dentry)); list_lru_isolate_move(lru, &dentry->d_lru, list); } @@ -1865,8 +1883,7 @@ static void __d_instantiate(struct dentry *dentry, struct inode *inode) /* * Decrement negative dentry count if it was in the LRU list. */ - if (dentry->d_flags & DCACHE_LRU_LIST) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(dentry->d_flags & DCACHE_LRU_LIST); hlist_add_head(&dentry->d_u.d_alias, &inode->i_dentry); raw_write_seqcount_begin(&dentry->d_seq); __d_set_inode_and_type(dentry, inode, add_flags);