From patchwork Fri Jan 29 19:30:46 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 8167011 Return-Path: X-Original-To: patchwork-linux-fsdevel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id F26989F440 for ; Fri, 29 Jan 2016 19:31:19 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1292220383 for ; Fri, 29 Jan 2016 19:31:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1831C2035D for ; Fri, 29 Jan 2016 19:31:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756810AbcA2TbL (ORCPT ); Fri, 29 Jan 2016 14:31:11 -0500 Received: from g1t6225.austin.hp.com ([15.73.96.126]:60220 "EHLO g1t6225.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752420AbcA2TbJ (ORCPT ); Fri, 29 Jan 2016 14:31:09 -0500 Received: from g2t4623.austin.hp.com (g2t4623.austin.hp.com [15.73.212.78]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by g1t6225.austin.hp.com (Postfix) with ESMTPS id 4B3842AF2 for ; Fri, 29 Jan 2016 19:31:08 +0000 (UTC) Received: from g1t6215.austin.hpicorp.net (g1t6215.austin.hpicorp.net [15.67.1.191]) by g2t4623.austin.hp.com (Postfix) with ESMTP id 873EB9A; Fri, 29 Jan 2016 19:31:06 +0000 (UTC) Received: from RHEL65.localdomain (unknown [16.214.195.24]) by g1t6215.austin.hpicorp.net (Postfix) with ESMTP id 7287D87; Fri, 29 Jan 2016 19:31:05 +0000 (UTC) From: Waiman Long To: Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Alexander Viro Cc: linux-fsdevel@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, Peter Zijlstra , Andi Kleen , Scott J Norton , Douglas Hatch , Waiman Long Subject: [PATCH v2 3/3] vfs: Enable list batching for the superblock's inode list Date: Fri, 29 Jan 2016 14:30:46 -0500 Message-Id: <1454095846-19628-4-git-send-email-Waiman.Long@hpe.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1454095846-19628-1-git-send-email-Waiman.Long@hpe.com> References: <1454095846-19628-1-git-send-email-Waiman.Long@hpe.com> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The inode_sb_list_add() and inode_sb_list_del() functions in the vfs layer just perform list addition and deletion under lock. So they can use the new list batching facility to speed up the list operations when many CPUs are trying to do it simultaneously. In particular, the inode_sb_list_del() function can be a performance bottleneck when large applications with many threads and associated inodes exit. With an exit microbenchmark that creates a large number of threads, attachs many inodes to them and then exits. The runtimes of that microbenchmark with 1000 threads before and after the patch on a 4-socket Intel E7-4820 v3 system (48 cores, 96 threads) were as follows: Kernel Elapsed Time System Time ------ ------------ ----------- Vanilla 4.4 65.29s 82m14s Patched 4.4 45.69s 49m44s The elapsed time and the reported system time were reduced by 30% and 40% respectively. Signed-off-by: Waiman Long --- fs/inode.c | 13 +++++-------- fs/super.c | 1 + include/linux/fs.h | 2 ++ 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 9f62db3..870de8c 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -424,19 +424,16 @@ static void inode_lru_list_del(struct inode *inode) */ void inode_sb_list_add(struct inode *inode) { - spin_lock(&inode->i_sb->s_inode_list_lock); - list_add(&inode->i_sb_list, &inode->i_sb->s_inodes); - spin_unlock(&inode->i_sb->s_inode_list_lock); + do_list_batch(&inode->i_sb->s_inode_list_lock, lb_cmd_add, + &inode->i_sb->s_list_batch, &inode->i_sb_list); } EXPORT_SYMBOL_GPL(inode_sb_list_add); static inline void inode_sb_list_del(struct inode *inode) { - if (!list_empty(&inode->i_sb_list)) { - spin_lock(&inode->i_sb->s_inode_list_lock); - list_del_init(&inode->i_sb_list); - spin_unlock(&inode->i_sb->s_inode_list_lock); - } + if (!list_empty(&inode->i_sb_list)) + do_list_batch(&inode->i_sb->s_inode_list_lock, lb_cmd_del_init, + &inode->i_sb->s_list_batch, &inode->i_sb_list); } static unsigned long hash(struct super_block *sb, unsigned long hashval) diff --git a/fs/super.c b/fs/super.c index 1182af8..b0e8540 100644 --- a/fs/super.c +++ b/fs/super.c @@ -206,6 +206,7 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags) mutex_init(&s->s_sync_lock); INIT_LIST_HEAD(&s->s_inodes); spin_lock_init(&s->s_inode_list_lock); + list_batch_init(&s->s_list_batch, &s->s_inodes); if (list_lru_init_memcg(&s->s_dentry_lru)) goto fail; diff --git a/include/linux/fs.h b/include/linux/fs.h index 1a20462..11d8b77 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -1403,6 +1404,7 @@ struct super_block { /* s_inode_list_lock protects s_inodes */ spinlock_t s_inode_list_lock ____cacheline_aligned_in_smp; struct list_head s_inodes; /* all inodes */ + struct list_batch s_list_batch; }; extern struct timespec current_fs_time(struct super_block *sb);