From patchwork Wed Aug 6 20:32:18 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 4688511 Return-Path: X-Original-To: patchwork-ocfs2-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id AF367C0338 for ; Wed, 6 Aug 2014 20:36:10 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B5F9120170 for ; Wed, 6 Aug 2014 20:36:09 +0000 (UTC) Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 91FA420136 for ; Wed, 6 Aug 2014 20:36:08 +0000 (UTC) Received: from acsinet21.oracle.com (acsinet21.oracle.com [141.146.126.237]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id s76KZuho020901 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 6 Aug 2014 20:35:57 GMT Received: from oss.oracle.com (oss-external.oracle.com [137.254.96.51]) by acsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id s76KZt1f010126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 6 Aug 2014 20:35:55 GMT Received: from localhost ([127.0.0.1] helo=oss.oracle.com) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1XF7t8-0003rE-0P; Wed, 06 Aug 2014 13:32:46 -0700 Received: from acsinet21.oracle.com ([141.146.126.237]) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1XF7t5-0003qt-GL for ocfs2-devel@oss.oracle.com; Wed, 06 Aug 2014 13:32:43 -0700 Received: from aserp1030.oracle.com (aserp1030.oracle.com [141.146.126.68]) by acsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id s76KWhFb001353 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 6 Aug 2014 20:32:43 GMT Received: from aserp2040.oracle.com (aserp2040.oracle.com [141.146.126.75]) by aserp1030.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id s76KWgos012298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 6 Aug 2014 20:32:42 GMT Received: from pps.filterd (aserp2040.oracle.com [127.0.0.1]) by aserp2040.oracle.com (8.14.7/8.14.7) with SMTP id s76KWgGE018085 for ; Wed, 6 Aug 2014 20:32:42 GMT Received: from mail-ig0-f201.google.com (mail-ig0-f201.google.com [209.85.213.201]) by aserp2040.oracle.com with ESMTP id 1nknw4kaf2-1 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for ; Wed, 06 Aug 2014 20:32:42 +0000 Received: by mail-ig0-f201.google.com with SMTP id h3so615492igd.4 for ; Wed, 06 Aug 2014 13:32:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:user-agent :mime-version:content-type:content-transfer-encoding; bh=rbvp4Zqwl7nHhCTquEhD0whNsOi6F6gtmz4LJTdAMsM=; b=G20KwXQYO7awAhhdGV2GfHMAvyYHUZI9XA6qs/Q4tkGqlYZjFfS0p3jrykdZTcrG2X J2EUQw1DCKvL5QgAYesbQ/wYiQ0UQQXJtfCKkEbP8wUiCigFPWamul725rR46lMLLivB p22CDDjidweVKndk3ut/EC9CNXNHx7l1kogY6qJ3UNRZwHgU3EkCxLzpnQycLhSp3Ebz j+wrXAqluZW8C0qh99gQ3jvuQPK7NlWIXK2b6yXbBi3wlPoMR5i7kdpS9SWZFuZ8cerL x68S1TSilAPJhXEFfAZofvo8N7UDF98iJ84SwVFogg9P/BoO2GkNQWvw+iVhxx71/kVA zzEA== X-Gm-Message-State: ALoCoQlqzQRmWJUcyagueghkXIOu+pcCMFJCBreFDcdlxlUNDk6hgPJMnaGvHC3bB6gU8HxLffll X-Received: by 10.50.80.111 with SMTP id q15mr7530203igx.0.1407357138627; Wed, 06 Aug 2014 13:32:18 -0700 (PDT) Received: from corp2gmr1-2.hot.corp.google.com (corp2gmr1-2.hot.corp.google.com [172.24.189.93]) by gmr-mx.google.com with ESMTPS id a66si137562yhg.7.2014.08.06.13.32.18 for (version=TLSv1.1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 06 Aug 2014 13:32:18 -0700 (PDT) Received: from akpm3.mtv.corp.google.com (akpm3.mtv.corp.google.com [172.17.131.127]) by corp2gmr1-2.hot.corp.google.com (Postfix) with ESMTP id 6818B5A4586; Wed, 6 Aug 2014 13:32:18 -0700 (PDT) Received: by akpm3.mtv.corp.google.com (Postfix, from userid 25780) id 2B9C31A0536; Wed, 6 Aug 2014 13:32:18 -0700 (PDT) Date: Wed, 06 Aug 2014 13:32:18 -0700 From: akpm@linux-foundation.org To: jlbec@evilplan.org, mfasheh@suse.com, ocfs2-devel@oss.oracle.com, akpm@linux-foundation.org, junxiao.bi@oracle.com, wen.gang.wang@oracle.com Message-ID: <53e290d2.LYQ748n3n5Uvamp1%akpm@linux-foundation.org> User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5600 definitions=7522 signatures=670497 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=2 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1408060245 Subject: [Ocfs2-devel] [patch 09/10] ocfs2: reflink: fix slow unlink for refcounted file X-BeenThere: ocfs2-devel@oss.oracle.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ocfs2-devel-bounces@oss.oracle.com Errors-To: ocfs2-devel-bounces@oss.oracle.com X-Source-IP: acsinet21.oracle.com [141.146.126.237] X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Junxiao Bi Subject: ocfs2: reflink: fix slow unlink for refcounted file When running ocfs2 test suite multiple nodes reflink stress test, for a 4 nodes cluster, every unlink() for refcounted file needs about 700s. The slow unlink is caused by the contention of refcount tree lock since all nodes are unlink files using the same refcount tree. When the unlinking file have many extents(over 1600 in our test), most of the extents has refcounted flag set. In ocfs2_commit_truncate(), it will execute the following call trace for every extents. This means it needs get and released refcount tree lock about 1600 times. And when several nodes are do this at the same time, the performance will be very low. ocfs2_remove_btree_range() ----ocfs2_lock_refcount_tree() ------ocfs2_refcount_lock() --------__ocfs2_cluster_lock() ocfs2_refcount_lock() is costly, move it to ocfs2_commit_truncate() to do lock/unlock once can improve a lot performance. Signed-off-by: Junxiao Bi Cc: Wengang Cc: Mark Fasheh Cc: Joel Becker Signed-off-by: Andrew Morton --- fs/ocfs2/alloc.c | 28 +++++++++++++++++++++------- fs/ocfs2/alloc.h | 2 +- fs/ocfs2/dir.c | 2 +- fs/ocfs2/file.c | 2 +- 4 files changed, 24 insertions(+), 10 deletions(-) diff -puN fs/ocfs2/alloc.c~ocfs2-reflink-fix-slow-unlink-for-refcounted-file fs/ocfs2/alloc.c --- a/fs/ocfs2/alloc.c~ocfs2-reflink-fix-slow-unlink-for-refcounted-file +++ a/fs/ocfs2/alloc.c @@ -5662,7 +5662,7 @@ int ocfs2_remove_btree_range(struct inod struct ocfs2_extent_tree *et, u32 cpos, u32 phys_cpos, u32 len, int flags, struct ocfs2_cached_dealloc_ctxt *dealloc, - u64 refcount_loc) + u64 refcount_loc, bool refcount_tree_locked) { int ret, credits = 0, extra_blocks = 0; u64 phys_blkno = ocfs2_clusters_to_blocks(inode->i_sb, phys_cpos); @@ -5676,11 +5676,13 @@ int ocfs2_remove_btree_range(struct inod BUG_ON(!(OCFS2_I(inode)->ip_dyn_features & OCFS2_HAS_REFCOUNT_FL)); - ret = ocfs2_lock_refcount_tree(osb, refcount_loc, 1, - &ref_tree, NULL); - if (ret) { - mlog_errno(ret); - goto bail; + if (!refcount_tree_locked) { + ret = ocfs2_lock_refcount_tree(osb, refcount_loc, 1, + &ref_tree, NULL); + if (ret) { + mlog_errno(ret); + goto bail; + } } ret = ocfs2_prepare_refcount_change_for_del(inode, @@ -7021,6 +7023,7 @@ int ocfs2_commit_truncate(struct ocfs2_s u64 refcount_loc = le64_to_cpu(di->i_refcount_loc); struct ocfs2_extent_tree et; struct ocfs2_cached_dealloc_ctxt dealloc; + struct ocfs2_refcount_tree *ref_tree = NULL; ocfs2_init_dinode_extent_tree(&et, INODE_CACHE(inode), di_bh); ocfs2_init_dealloc_ctxt(&dealloc); @@ -7130,9 +7133,18 @@ start: phys_cpos = ocfs2_blocks_to_clusters(inode->i_sb, blkno); + if ((flags & OCFS2_EXT_REFCOUNTED) && trunc_len && !ref_tree) { + status = ocfs2_lock_refcount_tree(osb, refcount_loc, 1, + &ref_tree, NULL); + if (status) { + mlog_errno(status); + goto bail; + } + } + status = ocfs2_remove_btree_range(inode, &et, trunc_cpos, phys_cpos, trunc_len, flags, &dealloc, - refcount_loc); + refcount_loc, true); if (status < 0) { mlog_errno(status); goto bail; @@ -7147,6 +7159,8 @@ start: goto start; bail: + if (ref_tree) + ocfs2_unlock_refcount_tree(osb, ref_tree, 1); ocfs2_schedule_truncate_log_flush(osb, 1); diff -puN fs/ocfs2/alloc.h~ocfs2-reflink-fix-slow-unlink-for-refcounted-file fs/ocfs2/alloc.h --- a/fs/ocfs2/alloc.h~ocfs2-reflink-fix-slow-unlink-for-refcounted-file +++ a/fs/ocfs2/alloc.h @@ -142,7 +142,7 @@ int ocfs2_remove_btree_range(struct inod struct ocfs2_extent_tree *et, u32 cpos, u32 phys_cpos, u32 len, int flags, struct ocfs2_cached_dealloc_ctxt *dealloc, - u64 refcount_loc); + u64 refcount_loc, bool refcount_tree_locked); int ocfs2_num_free_extents(struct ocfs2_super *osb, struct ocfs2_extent_tree *et); diff -puN fs/ocfs2/dir.c~ocfs2-reflink-fix-slow-unlink-for-refcounted-file fs/ocfs2/dir.c --- a/fs/ocfs2/dir.c~ocfs2-reflink-fix-slow-unlink-for-refcounted-file +++ a/fs/ocfs2/dir.c @@ -4477,7 +4477,7 @@ int ocfs2_dx_dir_truncate(struct inode * p_cpos = ocfs2_blocks_to_clusters(dir->i_sb, blkno); ret = ocfs2_remove_btree_range(dir, &et, cpos, p_cpos, clen, 0, - &dealloc, 0); + &dealloc, 0, false); if (ret) { mlog_errno(ret); goto out; diff -puN fs/ocfs2/file.c~ocfs2-reflink-fix-slow-unlink-for-refcounted-file fs/ocfs2/file.c --- a/fs/ocfs2/file.c~ocfs2-reflink-fix-slow-unlink-for-refcounted-file +++ a/fs/ocfs2/file.c @@ -1804,7 +1804,7 @@ static int ocfs2_remove_inode_range(stru ret = ocfs2_remove_btree_range(inode, &et, trunc_cpos, phys_cpos, trunc_len, flags, - &dealloc, refcount_loc); + &dealloc, refcount_loc, false); if (ret < 0) { mlog_errno(ret); goto out;