From patchwork Tue Sep 11 19:18:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10596191 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 811F014F9 for ; Tue, 11 Sep 2018 19:18:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78EED29BD9 for ; Tue, 11 Sep 2018 19:18:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6C2C429D13; Tue, 11 Sep 2018 19:18:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4AA029D0A for ; Tue, 11 Sep 2018 19:18:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E69598E0002; Tue, 11 Sep 2018 15:18:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E330C8E0001; Tue, 11 Sep 2018 15:18:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9A938E0003; Tue, 11 Sep 2018 15:18:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 916408E0002 for ; Tue, 11 Sep 2018 15:18:36 -0400 (EDT) Received: by mail-qk1-f198.google.com with SMTP id w142-v6so21547771qkw.8 for ; Tue, 11 Sep 2018 12:18:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=CCBS3Wuve8RiXTWHGrwTznKM/VN762uDCXQIsdJF4Ag=; b=ELeTSBjgMvbTlW0Bg+46/ZdfSJsinXaKosjIYOzQJMwaoQFso/5voNc9iNShX9zyPp v/5XqK0ZNW4C6Op4vd7aZtJXnolS6uAj5BOeEDyUCi/6Hi2POBUxQo+d/jhykqMD9Dgw Y4ElX2TaWZxnjY2ymVAeF6D1CK6UdEjqtZmYFfJ1ZrcW+YEnjIKDZyhfXdO0UMqsRdd4 MJzs7Dn3FS83papEFHnpEJeHXo+N9iE90Hq795PQJ+2yAcdoJ9grmEdDbSdC4Fy2AYGq Va+MWBtxo5hJOO8JxKxBRGFdMLboV7L0iK4DkrY4HZ5xuzTuaAb7q7qf5yQHYjLneAfJ g3kg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51BLI55VRUZRQVQqXZoyxxh2nGb9F+cM9LWSickpzSSbqTbg0cBn v+Z/HVFZaAqOT0yhSsn2sD3Y3d/UZNMmEwjeu5LoOkqYbeP21SnXoZwVN6qMleoX90dId19txsA bXKdkH5u+ANMo3NrU5kt8qz/ZhT/a5W9qGN0+LmE5R0WBi0CN425tfbUEj1Z/6PYpuQ== X-Received: by 2002:a37:5e03:: with SMTP id s3-v6mr20358350qkb.339.1536693516358; Tue, 11 Sep 2018 12:18:36 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYJdB11OHPMtYd48FuQoKV9oeXPhqcXXoKkrHAzJjPnJnhFO7O4PTudQoUraESCqmrcpvhy X-Received: by 2002:a37:5e03:: with SMTP id s3-v6mr20358323qkb.339.1536693515830; Tue, 11 Sep 2018 12:18:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536693515; cv=none; d=google.com; s=arc-20160816; b=gPV/Hpsat2LWixXT2lPhicoAoXW6WQEYix2/OWHfVebtmRGy8UEX7uh2WQ7S1eO+5m iehUr3POiC4LTcBrKe+j8DXQyetY8EJMQnG+xHXHh8trasZEpnsrulfoyN/kr5fRhmmD jz49nwwUEytdYVxf5pXkzjoMrZFru0mDTpLwGverIOFDYNSiKCtWJ/Dgt5obkC7zcWhN ZuGX0tRQPax4DagTE8vMrda2kRHC2PPz4sE+JaNUjNrWH5pXenY6f//+Am4xmvqnOtb9 qHr8dW8pkwoop2sZ9h1Jn7tniBIzh3ChzYQbP0VVM4mWf0BjVsdmTJ/csG6N5uPbp7tb GdcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=CCBS3Wuve8RiXTWHGrwTznKM/VN762uDCXQIsdJF4Ag=; b=P94qqrLa9291kKWB39aaSAhEeeYoz84GN9LaFwGSG9XERMGedy72fiPG/m15h4fDsJ KhI56y7ldlfMSXgKuuYYM7d2nOUO6Bzo0LCR+nXZMLG7co1ZeQxmJm0QoyQWadgtDyFs Y/obereE5q1JIbr2iAfFwfXk11DTvsM9u//DDZ6zM8zmuilu6j9NQxNR8pHkRNPkZeY0 oymW7WNXzE9hGHKefuDGxxf917yTncoe+MedzM8LlTtDm7eJ8sVx+n2YPdt00mhox8wT 9yH1UT9SYmKVYgELhujGtbsEFMDsZ+r0JBhuxvRSNJFMKYKShtb7j5IDoHCvh3UAKtHl oefA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id o53-v6si251233qtk.150.2018.09.11.12.18.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 12:18:35 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6591E804B4DB; Tue, 11 Sep 2018 19:18:35 +0000 (UTC) Received: from llong.com (ovpn-123-172.rdu2.redhat.com [10.10.123.172]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B3092157F4A; Tue, 11 Sep 2018 19:18:34 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v3 1/4] fs/dcache: Fix incorrect nr_dentry_unused accounting in shrink_dcache_sb() Date: Tue, 11 Sep 2018 15:18:23 -0400 Message-Id: <1536693506-11949-2-git-send-email-longman@redhat.com> In-Reply-To: <1536693506-11949-1-git-send-email-longman@redhat.com> References: <1536693506-11949-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 11 Sep 2018 19:18:35 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 11 Sep 2018 19:18:35 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The nr_dentry_unused per-cpu counter tracks dentries in both the LRU lists and the shrink lists where the DCACHE_LRU_LIST bit is set. The shrink_dcache_sb() function moves dentries from the LRU list to a shrink list and subtracts the dentry count from nr_dentry_unused. This is incorrect as the nr_dentry_unused count Will also be decremented in shrink_dentry_list() via d_shrink_del(). To fix this double decrement, the decrement in the shrink_dcache_sb() function is taken out. Fixes: 4e717f5c1083 ("list_lru: remove special case function list_lru_dispose_all." Signed-off-by: Waiman Long Reviewed-by: Dave Chinner --- fs/dcache.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 2e7e8d8..cb515f1 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -1202,15 +1202,11 @@ static enum lru_status dentry_lru_isolate_shrink(struct list_head *item, */ void shrink_dcache_sb(struct super_block *sb) { - long freed; - do { LIST_HEAD(dispose); - freed = list_lru_walk(&sb->s_dentry_lru, + list_lru_walk(&sb->s_dentry_lru, dentry_lru_isolate_shrink, &dispose, 1024); - - this_cpu_sub(nr_dentry_unused, freed); shrink_dentry_list(&dispose); } while (list_lru_count(&sb->s_dentry_lru) > 0); } From patchwork Tue Sep 11 19:18:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10596195 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 982AA14BD for ; Tue, 11 Sep 2018 19:18:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9054229BD9 for ; Tue, 11 Sep 2018 19:18:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 849B429D0D; Tue, 11 Sep 2018 19:18:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A72429BD9 for ; Tue, 11 Sep 2018 19:18:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 13DD28E0003; Tue, 11 Sep 2018 15:18:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 115FE8E0001; Tue, 11 Sep 2018 15:18:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 02BCD8E0003; Tue, 11 Sep 2018 15:18:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id CA7F38E0001 for ; Tue, 11 Sep 2018 15:18:37 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id q26-v6so25770803qtj.14 for ; Tue, 11 Sep 2018 12:18:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=4B/Wm4vURCyVHVLH/q3jpiYPWlPN176DKvy63T1eH4Y=; b=uSivED7rHVk4fmwidsnPwM/ExHEPlmAhkRRa3cZ39a2QR6DhuFY67HZEyiZ9zXKgHI Yaxgohr23LYDu+9DKRenf7BO83aRdr2XF707cjdGCSA6qIKB3Q9HPNdF9NQIPPC2N45/ YXoUWUYs4J7YaalgRdkDpM1+ioODZhVEkbKS3Kpc6ozeXmS6cxDc0NY9sgF2fLyVfXg6 rsIlHl9wujadjSJs8tHJuQLvrjxCFyagjnnOc/rvy2SY+R0GAgZWEE809oIsRBtN3TZ9 p95GLjX9ZA6kRXFFZPYmtlRJLcv6pbu0OYVe75T0+q7Je/R2V81gnRORXQPMm9+6w9+D YfzQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51Co64gkOwgkALtNZtu9U4ERO8ceiLET0BpS49cHSy9SlbWVzM+S dpYbMN6YiwJd26ype50evmcCZ78YzHg45ONGNs3RlB+YvGjujubzkUGp2Nq2gdVbmbQyV7GkIbC E0rV8QotkIZT8SeCAZh4xzWp9zwEpIbKlmH7ghcJBbHDpfUfSrsj9E7Yo6BHCqZjTVA== X-Received: by 2002:a0c:e792:: with SMTP id x18-v6mr19458518qvn.143.1536693517538; Tue, 11 Sep 2018 12:18:37 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZWh0DLLo4G3H54/NfI84WfbqFDRzNSwdi5sTyoalqSQAaPRi7Iv5WW6YuL3m+9RaTE79+S X-Received: by 2002:a0c:e792:: with SMTP id x18-v6mr19458493qvn.143.1536693516990; Tue, 11 Sep 2018 12:18:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536693516; cv=none; d=google.com; s=arc-20160816; b=BNSJ7fTLgHM0KpOok/pW6IsFZ64ili9Fwo1KmI5cj2L4Silm2WbLDjPpDywzUINvBo J8/fCIztcMzB9ZkXTO56WHxIjXX3oU4xNCAy36RK7yerWUlKbl2odNAz5OSVjLksIWDp O071CtcMK085NA67whwNzMN1YAZgLssZCIBhDJ+ZeYTfIMikd3M6NVlf4Zk67bX4ZzJy EflesxG0BmIioBPjFCx/XTxiMG+oR6bStIfnI1m+agCHW31i3qG/QeXChNOqTfo11uIN 9V5VikGyFpfhGBcuI2q8Vvm0fKi1VAQp7QuGK42qo3nH2UMjaa1byqMUn/sURjk0njB+ sx+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=4B/Wm4vURCyVHVLH/q3jpiYPWlPN176DKvy63T1eH4Y=; b=SPGDLrAbCH65qMCU1To6lx0gk+fXDUzjHez6hwkODUvIwFlhjZdoYkk1GWXa+XzgAV 7KIKuiCGzzRg6f3khjEz0caonCL1JD9jygX21uZ5uZsnQBwoe6+ktD6TVTi47T4gIRs2 cHPWht3t+o1Pib+pg5vOWw5LAmqp1aGldivyNOhA4rMQBgsl19NfsUG0D4S+XpybhFI3 sFehMuub+lJhPKQZWcvOWL3PfVqFjws13NUNaP5fhYUpMFfZDnRslsC9GnGHTiqGn97Z 5QDwDAoA3GpKm3ZVDTMqN9HWfgnZ3EQBGG9bOWfzqz4Zt9PDLsi8+o2J14BUGruOEuzv 0QyA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id k42-v6si8512915qtf.286.2018.09.11.12.18.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 12:18:36 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8DEC74023ECC; Tue, 11 Sep 2018 19:18:36 +0000 (UTC) Received: from llong.com (ovpn-123-172.rdu2.redhat.com [10.10.123.172]) by smtp.corp.redhat.com (Postfix) with ESMTP id 773B82166BA3; Tue, 11 Sep 2018 19:18:35 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v3 2/4] fs: Don't need to put list_lru into its own cacheline Date: Tue, 11 Sep 2018 15:18:24 -0400 Message-Id: <1536693506-11949-3-git-send-email-longman@redhat.com> In-Reply-To: <1536693506-11949-1-git-send-email-longman@redhat.com> References: <1536693506-11949-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 11 Sep 2018 19:18:36 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 11 Sep 2018 19:18:36 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The list_lru structure is essentially just a pointer to a table of per-node LRU lists. Even if CONFIG_MEMCG_KMEM is defined, the list field is just used for LRU list registration and shrinker_id is set at initialization. Those fields won't need to be touched that often. So there is no point to make the list_lru structures to sit in their own cachelines. Signed-off-by: Waiman Long Reviewed-by: Dave Chinner --- include/linux/fs.h | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 3332270..fd4cd8a 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1443,11 +1443,12 @@ struct super_block { struct user_namespace *s_user_ns; /* - * Keep the lru lists last in the structure so they always sit on their - * own individual cachelines. + * The list_lru structure is essentially just a pointer to a table + * of per-node lru lists, each of which has its own spinlock. + * There is no need to put them into separate cachelines. */ - struct list_lru s_dentry_lru ____cacheline_aligned_in_smp; - struct list_lru s_inode_lru ____cacheline_aligned_in_smp; + struct list_lru s_dentry_lru; + struct list_lru s_inode_lru; struct rcu_head rcu; struct work_struct destroy_work; From patchwork Tue Sep 11 19:18:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10596197 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D45414F9 for ; Tue, 11 Sep 2018 19:18:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 762EB29BD9 for ; Tue, 11 Sep 2018 19:18:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 69D2C29D10; Tue, 11 Sep 2018 19:18:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5EB729BD9 for ; Tue, 11 Sep 2018 19:18:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97BFA8E0005; Tue, 11 Sep 2018 15:18:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 92D5A8E0001; Tue, 11 Sep 2018 15:18:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8410E8E0005; Tue, 11 Sep 2018 15:18:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id 5B2BD8E0001 for ; Tue, 11 Sep 2018 15:18:39 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id d18-v6so25657270qtj.20 for ; Tue, 11 Sep 2018 12:18:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=MV7sweeXZCYt1FY8IUx5TOm44xr8ZjSGlGVv/JPWaXM=; b=sQvoZ1es7/jN9w2kzR8ByWBS5ndXq7+vSPmbZWAW4Me2YOGfPecxvqkD/Rxh3A3Gn1 CQF3xEa3dOfrCSnAz8uk2L3oRXxojul1XYBklpVllAe/Z9q2C7pbn/dVfF9U/wR6um81 QWIxwdWoPhHNq3EQ2GwnOXqoCVDgJVzP989cDRbqbCJZZgI9AhQ3sNrHw6ebi8kB2lX1 fvJ20qmFc9TYcWfVzWQ4TrZBTFqr7H5SwTVjYaR13/ypC1bsdWvpgPEwqgYKU8glV73V pZKt9ukaolyISt1m9rkdmE1OHjnKbOMYOW5lwGioTVnZ3RvJx3A1VyiEFOj57Kf4PwQF mBrw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51DkVB19+YcJ+5sVCHgxfNzLS/F/XcqKtYfN+blZ2Yj1qTbDwQHz KhfDDFfeAIbzOvI/ODIPAES6mF6m/ug0zVvXEwdq/SJ2DF4Jb8fc3DG0wDWPRCOH8jJBdn4XjnI I0pHiZRWRiOLfly+Gn+P/k1AnXfyxAHbPNVf8vaEbia0OAzaVOcQeXWAoAQgPOTgKsg== X-Received: by 2002:ac8:24d0:: with SMTP id t16-v6mr20935322qtt.116.1536693519113; Tue, 11 Sep 2018 12:18:39 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZasF69OOQPc5AczSZIJitxlnfNLGNRq6ULhSxRfycNty6ddG5JSE6f7Cnjmq6bm4MA3syA X-Received: by 2002:ac8:24d0:: with SMTP id t16-v6mr20935279qtt.116.1536693518184; Tue, 11 Sep 2018 12:18:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536693518; cv=none; d=google.com; s=arc-20160816; b=U0Vt8f4r3e9e9+uJQF5bZGkDH+ykdq43XT3VBuK1HBjv5cK/LvAZ0oWzLSVCwLjkqz cutL7Tsfmib+wvffyQjMIVObACBdz/lWcDjtb5FKqxaz4aQyES1QiblRYICkayL7x3J0 XPnCUeBtJsV0RETg/+plpndBbdhR14twt/IyqtHvHd1EVFoLDJnfAaP44HDq8GIZPjnA Y2csn4TYDW541DXh7uNT6r8+/Rg2spDynUeMoq7m9scHx3bSwo1qBBVf7pIcdOEv7xtA 5UyptPV7G+seyed/haynki7MvJ7MxMbh0ef7MBTpiAz3pTglKIaKZkgy6zedJg7b6DxM rdCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=MV7sweeXZCYt1FY8IUx5TOm44xr8ZjSGlGVv/JPWaXM=; b=BDu7aiDsimpNKtJz89fO2n7LW1RDixQjxkfOH1bYp2JiaKYRRkLA8hzA8JjFQrglkW 822S2O6BSFf3UT3EUZok/IazY+D4fm9Jc7QH+8gl49WiUUfI9+MWn782Y8KYyuaMRPQm osQR8V1eoHLjJoBYsxXFaixqA0Od1aUopHsRC/pVYI0czoJuVS3Q9h6MPquOe6MAUkkO 4g4nbG6pxSWIJmONoxiX0a8AAn1vO0ZFe1UEqXFPLMNmP1me+PaTxZcwe/LvmvSF0gP5 71jlNRAxN01EukgzHtriYixbUt3O4K39Bed3VcTyBYOxyq5ctqInJiWNFL9UXWoIuVGd NpKQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id 21-v6si4226748qtp.70.2018.09.11.12.18.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 12:18:38 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B794C804B9F4; Tue, 11 Sep 2018 19:18:37 +0000 (UTC) Received: from llong.com (ovpn-123-172.rdu2.redhat.com [10.10.123.172]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9FCF82157F4A; Tue, 11 Sep 2018 19:18:36 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v3 3/4] fs/dcache: Track & report number of negative dentries Date: Tue, 11 Sep 2018 15:18:25 -0400 Message-Id: <1536693506-11949-4-git-send-email-longman@redhat.com> In-Reply-To: <1536693506-11949-1-git-send-email-longman@redhat.com> References: <1536693506-11949-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 11 Sep 2018 19:18:37 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 11 Sep 2018 19:18:37 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The current dentry number tracking code doesn't distinguish between positive & negative dentries. It just reports the total number of dentries in the LRU lists. As excessive number of negative dentries can have an impact on system performance, it will be wise to track the number of positive and negative dentries separately. This patch adds tracking for the total number of negative dentries in the system LRU lists and reports it in the 7th field in the /proc/sys/fs/dentry-state file. The number, however, does not include negative dentries that are in flight but not in the LRU yet as well as those in the shrinker lists. The number of positive dentries in the LRU lists can be roughly found by subtracting the number of negative dentries from the unused count. Matthew Wilcox had confirmed that since the introduction of the dentry_stat structure in 2.1.60, the dummy array was there, probably for future extension. They were not replacements of pre-existing fields. So no sane applications that read the value of /proc/sys/fs/dentry-state will do dummy thing if the last 2 fields of the sysctl parameter are not zero. IOW, it will be safe to use one of the dummy array entry for negative dentry count. Signed-off-by: Waiman Long --- Documentation/sysctl/fs.txt | 26 ++++++++++++++++---------- fs/dcache.c | 31 +++++++++++++++++++++++++++++++ include/linux/dcache.h | 7 ++++--- 3 files changed, 51 insertions(+), 13 deletions(-) diff --git a/Documentation/sysctl/fs.txt b/Documentation/sysctl/fs.txt index 819caf8..3b4f441 100644 --- a/Documentation/sysctl/fs.txt +++ b/Documentation/sysctl/fs.txt @@ -56,26 +56,32 @@ of any kernel data structures. dentry-state: -From linux/fs/dentry.c: +From linux/include/linux/dcache.h: -------------------------------------------------------------- -struct { +struct dentry_stat_t dentry_stat { int nr_dentry; int nr_unused; int age_limit; /* age in seconds */ int want_pages; /* pages requested by system */ - int dummy[2]; -} dentry_stat = {0, 0, 45, 0,}; --------------------------------------------------------------- - -Dentries are dynamically allocated and deallocated, and -nr_dentry seems to be 0 all the time. Hence it's safe to -assume that only nr_unused, age_limit and want_pages are -used. Nr_unused seems to be exactly what its name says. + int nr_negative; /* # of unused negative dentries */ + int dummy; /* Reserved */ +}; +-------------------------------------------------------------- + +Dentries are dynamically allocated and deallocated. + +nr_dentry shows the total number of dentries allocated (active ++ unused). nr_unused shows the number of dentries that are not +actively used, but are saved in the LRU list for future reuse. + Age_limit is the age in seconds after which dcache entries can be reclaimed when memory is short and want_pages is nonzero when shrink_dcache_pages() has been called and the dcache isn't pruned yet. +nr_negative shows the number of unused dentries that are also +negative dentries which do not mapped to actual files. + ============================================================== dquot-max & dquot-nr: diff --git a/fs/dcache.c b/fs/dcache.c index cb515f1..c1cc956 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -119,6 +119,7 @@ struct dentry_stat_t dentry_stat = { static DEFINE_PER_CPU(long, nr_dentry); static DEFINE_PER_CPU(long, nr_dentry_unused); +static DEFINE_PER_CPU(long, nr_dentry_negative); #if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS) @@ -152,11 +153,22 @@ static long get_nr_dentry_unused(void) return sum < 0 ? 0 : sum; } +static long get_nr_dentry_negative(void) +{ + int i; + long sum = 0; + + for_each_possible_cpu(i) + sum += per_cpu(nr_dentry_negative, i); + return sum < 0 ? 0 : sum; +} + int proc_nr_dentry(struct ctl_table *table, int write, void __user *buffer, size_t *lenp, loff_t *ppos) { dentry_stat.nr_dentry = get_nr_dentry(); dentry_stat.nr_unused = get_nr_dentry_unused(); + dentry_stat.nr_negative = get_nr_dentry_negative(); return proc_doulongvec_minmax(table, write, buffer, lenp, ppos); } #endif @@ -331,6 +343,8 @@ static inline void __d_clear_type_and_inode(struct dentry *dentry) flags &= ~(DCACHE_ENTRY_TYPE | DCACHE_FALLTHRU); WRITE_ONCE(dentry->d_flags, flags); dentry->d_inode = NULL; + if (dentry->d_flags & DCACHE_LRU_LIST) + this_cpu_inc(nr_dentry_negative); } static void dentry_free(struct dentry *dentry) @@ -385,6 +399,10 @@ static void dentry_unlink_inode(struct dentry * dentry) * The per-cpu "nr_dentry_unused" counters are updated with * the DCACHE_LRU_LIST bit. * + * The per-cpu "nr_dentry_negative" counters are only updated + * when deleted or added to the per-superblock LRU list, not + * on the shrink list. + * * These helper functions make sure we always follow the * rules. d_lock must be held by the caller. */ @@ -394,6 +412,8 @@ static void d_lru_add(struct dentry *dentry) D_FLAG_VERIFY(dentry, 0); dentry->d_flags |= DCACHE_LRU_LIST; this_cpu_inc(nr_dentry_unused); + if (d_is_negative(dentry)) + this_cpu_inc(nr_dentry_negative); WARN_ON_ONCE(!list_lru_add(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -402,6 +422,8 @@ static void d_lru_del(struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); + if (d_is_negative(dentry)) + this_cpu_dec(nr_dentry_negative); WARN_ON_ONCE(!list_lru_del(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -432,6 +454,8 @@ static void d_lru_isolate(struct list_lru_one *lru, struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); + if (d_is_negative(dentry)) + this_cpu_dec(nr_dentry_negative); list_lru_isolate(lru, &dentry->d_lru); } @@ -440,6 +464,8 @@ static void d_lru_shrink_move(struct list_lru_one *lru, struct dentry *dentry, { D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags |= DCACHE_SHRINK_LIST; + if (d_is_negative(dentry)) + this_cpu_dec(nr_dentry_negative); list_lru_isolate_move(lru, &dentry->d_lru, list); } @@ -1836,6 +1862,11 @@ static void __d_instantiate(struct dentry *dentry, struct inode *inode) WARN_ON(d_in_lookup(dentry)); spin_lock(&dentry->d_lock); + /* + * Decrement negative dentry count if it was in the LRU list. + */ + if (dentry->d_flags & DCACHE_LRU_LIST) + this_cpu_dec(nr_dentry_negative); hlist_add_head(&dentry->d_u.d_alias, &inode->i_dentry); raw_write_seqcount_begin(&dentry->d_seq); __d_set_inode_and_type(dentry, inode, add_flags); diff --git a/include/linux/dcache.h b/include/linux/dcache.h index ef4b70f..73ff9f0 100644 --- a/include/linux/dcache.h +++ b/include/linux/dcache.h @@ -62,9 +62,10 @@ struct qstr { struct dentry_stat_t { long nr_dentry; long nr_unused; - long age_limit; /* age in seconds */ - long want_pages; /* pages requested by system */ - long dummy[2]; + long age_limit; /* age in seconds */ + long want_pages; /* pages requested by system */ + long nr_negative; /* # of unused negative dentries */ + long dummy; /* Reserved */ }; extern struct dentry_stat_t dentry_stat; From patchwork Tue Sep 11 19:18:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10596201 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4638F14BD for ; Tue, 11 Sep 2018 19:18:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3FFF929BD9 for ; Tue, 11 Sep 2018 19:18:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3372029D0D; Tue, 11 Sep 2018 19:18:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8A9629BD9 for ; Tue, 11 Sep 2018 19:18:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4F328E0007; Tue, 11 Sep 2018 15:18:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD6098E0001; Tue, 11 Sep 2018 15:18:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE9908E0007; Tue, 11 Sep 2018 15:18:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 8C2A38E0001 for ; Tue, 11 Sep 2018 15:18:40 -0400 (EDT) Received: by mail-qk1-f197.google.com with SMTP id n23-v6so21907647qkn.19 for ; Tue, 11 Sep 2018 12:18:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=jWlOFtELmXfxC3ynjpZuAA8uJ4Rm+lS3xewNX5/reIA=; b=o9glMdZFTzuh60EeNfWc4M3DW8oKY8qxEx6RWksbayYYFx98H6gfJf9LQpluAEy/3e H13hMGipOQINhWunFpPoHDO/a9tmOs6MrcDDEc8GTF8RtowXKmxfv8wkgGN3um6PGxp8 XreEn4r/pXddl38utaPLMm0kMi6nSSSYTIZMB9PAFuD1lVM7TnH2TXHhVhDKZj7ZtYfZ lATiZYZqnXzM2sule/e1NbJPMOGA2dSOLrKk6RoDIsm0SgckfGOcqnVIoLL06GAURf68 uKaq1gqABjmZpmRGoW8pKZ2S8rhoSCB6W9Xi2Wj3Ny3qj7kJU0SveLPbZeUOoWZAAAc3 UM7Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APzg51BzkxlS6EkpBARvZ0CpjkiLfsOkiEvx4tU/b5TAn8eVuNMEkT9O HpVr4rYtcDXdpddaXDqNdJYc/FevYly/muEcxGiwhX6+tPBIXM4mWwkuDjpZVY7LriAgpJ6c6Fh ianicVfidM+8OAvGItxzHz6E5Z+GHwE/I7q/Z5pRCkJJBcBVrLizDb4gwFS05TOimJQ== X-Received: by 2002:a0c:d135:: with SMTP id a50-v6mr19352683qvh.112.1536693520306; Tue, 11 Sep 2018 12:18:40 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbNm4zC0mY/qhwxKrf3dNwuFWMhk0UOqzQ1GaHhCP6heJKoBi285MBxpT5eEz1YZb8+Pdf9 X-Received: by 2002:a0c:d135:: with SMTP id a50-v6mr19352644qvh.112.1536693519398; Tue, 11 Sep 2018 12:18:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536693519; cv=none; d=google.com; s=arc-20160816; b=C4ADSy/o1oWp5JTZyxGtMNUUxso4mItfYn/tuENR8X+unHC2k/Dwv+iYX3k+vCx1rt q7uaC1O0CHq5YTgLRDALfnehPu5CGUm/vxG8HI7GJnh5iIk26PXbI4ZYC1yWyQp/aPyk DePK6a5gbOvENxoKEsgOZeS5ITP8LnKj2VRvEqKKtfhhcPr+NsvVEr9mHE1dgp1SH4Zn t3ZGehNg2Z02D34vbqEpXrQUtGwC3gNB2VeWweJXepf8GJKzYdjJa98qOJIOjnGmw05x D/6EWyh3r/UPAAFAhPX/51uIgZ/r/ZwaFdgShwFUh4krynJyfqZkx5umusoKVG0GcnXb wqTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=jWlOFtELmXfxC3ynjpZuAA8uJ4Rm+lS3xewNX5/reIA=; b=JZF4VWJEv0su82SksTmJdHC4aD7tHrd+lmzK+gS/leEDwxbTmIbSfWAtmmP3SRLqlg PpYkb94DkEyJVijiqcE/XTY5lHANoP791S0zkhvoZcR133dWXYueUFcRO8wQwKzb/NTy 0lbm84qLzbuKmjNDQRENrM0t0V9KbwBMSrLvjpJY+fnRORKjV4CcEv2amnpPt5lie6RK SJSE2g6yUazmVw2eltnj+Z8KEVtdpsUJbjRONwgbv7/LfFXATbTuM3HmZSKnTnDqUQjp s4m3UrSBYVY1fzYOf7WaoYp7aur1MVPh7WPk8iQKhQl9c4TEoPgSUULXR6Hiq0L9aLip aGIg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id o7-v6si403246qtb.200.2018.09.11.12.18.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 12:18:39 -0700 (PDT) Received-SPF: pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of longman@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E1DC440241C7; Tue, 11 Sep 2018 19:18:38 +0000 (UTC) Received: from llong.com (ovpn-123-172.rdu2.redhat.com [10.10.123.172]) by smtp.corp.redhat.com (Postfix) with ESMTP id CA3542156899; Tue, 11 Sep 2018 19:18:37 +0000 (UTC) From: Waiman Long To: Alexander Viro , Jonathan Corbet Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, "Luis R. Rodriguez" , Kees Cook , Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Matthew Wilcox , Larry Woodman , James Bottomley , "Wangkai (Kevin C)" , Michal Hocko , Waiman Long Subject: [PATCH v3 4/4] fs/dcache: Eliminate branches in nr_dentry_negative accounting Date: Tue, 11 Sep 2018 15:18:26 -0400 Message-Id: <1536693506-11949-5-git-send-email-longman@redhat.com> In-Reply-To: <1536693506-11949-1-git-send-email-longman@redhat.com> References: <1536693506-11949-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 11 Sep 2018 19:18:39 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 11 Sep 2018 19:18:39 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Because the accounting of nr_dentry_negative depends on whether a dentry is a negative one or not, branch instructions are introduced to handle the accounting conditionally. That may potentially slow down the task by a noticeable amount if that introduces sizeable amount of additional branch mispredictions. To avoid that, the accounting code is now modified to use conditional move instructions instead, if supported by the architecture. Signed-off-by: Waiman Long --- fs/dcache.c | 41 +++++++++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index c1cc956..dfd5628 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -171,6 +171,29 @@ int proc_nr_dentry(struct ctl_table *table, int write, void __user *buffer, dentry_stat.nr_negative = get_nr_dentry_negative(); return proc_doulongvec_minmax(table, write, buffer, lenp, ppos); } + +/* + * Increment/Decrement nr_dentry_negative if the condition is true. + * For architectures that support some kind of conditional move, compiler + * should be able generate code to inc/dec negative dentry counter + * without any branch instruction. + */ +static inline void cond_negative_dentry_inc(bool cond) +{ + int val = !!cond; + + this_cpu_add(nr_dentry_negative, val); +} + +static inline void cond_negative_dentry_dec(bool cond) +{ + int val = !!cond; + + this_cpu_sub(nr_dentry_negative, val); +} +#else +static inline void cond_negative_dentry_inc(bool cond) { } +static inline void cond_negative_dentry_dec(bool cond) { } #endif /* @@ -343,8 +366,7 @@ static inline void __d_clear_type_and_inode(struct dentry *dentry) flags &= ~(DCACHE_ENTRY_TYPE | DCACHE_FALLTHRU); WRITE_ONCE(dentry->d_flags, flags); dentry->d_inode = NULL; - if (dentry->d_flags & DCACHE_LRU_LIST) - this_cpu_inc(nr_dentry_negative); + cond_negative_dentry_inc(dentry->d_flags & DCACHE_LRU_LIST); } static void dentry_free(struct dentry *dentry) @@ -412,8 +434,7 @@ static void d_lru_add(struct dentry *dentry) D_FLAG_VERIFY(dentry, 0); dentry->d_flags |= DCACHE_LRU_LIST; this_cpu_inc(nr_dentry_unused); - if (d_is_negative(dentry)) - this_cpu_inc(nr_dentry_negative); + cond_negative_dentry_inc(d_is_negative(dentry)); WARN_ON_ONCE(!list_lru_add(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -422,8 +443,7 @@ static void d_lru_del(struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); - if (d_is_negative(dentry)) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(d_is_negative(dentry)); WARN_ON_ONCE(!list_lru_del(&dentry->d_sb->s_dentry_lru, &dentry->d_lru)); } @@ -454,8 +474,7 @@ static void d_lru_isolate(struct list_lru_one *lru, struct dentry *dentry) D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags &= ~DCACHE_LRU_LIST; this_cpu_dec(nr_dentry_unused); - if (d_is_negative(dentry)) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(d_is_negative(dentry)); list_lru_isolate(lru, &dentry->d_lru); } @@ -464,8 +483,7 @@ static void d_lru_shrink_move(struct list_lru_one *lru, struct dentry *dentry, { D_FLAG_VERIFY(dentry, DCACHE_LRU_LIST); dentry->d_flags |= DCACHE_SHRINK_LIST; - if (d_is_negative(dentry)) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(d_is_negative(dentry)); list_lru_isolate_move(lru, &dentry->d_lru, list); } @@ -1865,8 +1883,7 @@ static void __d_instantiate(struct dentry *dentry, struct inode *inode) /* * Decrement negative dentry count if it was in the LRU list. */ - if (dentry->d_flags & DCACHE_LRU_LIST) - this_cpu_dec(nr_dentry_negative); + cond_negative_dentry_dec(dentry->d_flags & DCACHE_LRU_LIST); hlist_add_head(&dentry->d_u.d_alias, &inode->i_dentry); raw_write_seqcount_begin(&dentry->d_seq); __d_set_inode_and_type(dentry, inode, add_flags);