From patchwork Tue Feb 3 10:45:38 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liu Bo X-Patchwork-Id: 5767481 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 0FF269F302 for ; Tue, 3 Feb 2015 10:46:00 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 28D5420C3E for ; Tue, 3 Feb 2015 10:45:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 30AE520C3D for ; Tue, 3 Feb 2015 10:45:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754898AbbBCKpt (ORCPT ); Tue, 3 Feb 2015 05:45:49 -0500 Received: from userp1040.oracle.com ([156.151.31.81]:45031 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753162AbbBCKps (ORCPT ); Tue, 3 Feb 2015 05:45:48 -0500 Received: from ucsinet22.oracle.com (ucsinet22.oracle.com [156.151.31.94]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id t13AjlLk020380 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 3 Feb 2015 10:45:47 GMT Received: from aserz7021.oracle.com (aserz7021.oracle.com [141.146.126.230]) by ucsinet22.oracle.com (8.14.5+Sun/8.14.5) with ESMTP id t13AjkH5029512 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 3 Feb 2015 10:45:46 GMT Received: from abhmp0004.oracle.com (abhmp0004.oracle.com [141.146.116.10]) by aserz7021.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id t13AjjkB003226; Tue, 3 Feb 2015 10:45:45 GMT Received: from localhost.jp.oracle.com (/10.191.2.91) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 03 Feb 2015 02:45:45 -0800 From: Liu Bo To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org Subject: [PATCH] Btrfs: use asynchronous submit for large DIO io in single profile Date: Tue, 3 Feb 2015 18:45:38 +0800 Message-Id: <1422960338-12187-1-git-send-email-bo.li.liu@oracle.com> X-Mailer: git-send-email 1.8.1.4 X-Source-IP: ucsinet22.oracle.com [156.151.31.94] Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Commit 1ae399382512 ("Btrfs: do not use async submit for small DIO io's") benefits small DIO io's. However, if we're owning the SINGLE profile, this also affects large DIO io's since in that case, map_length is (chunk_length - bio's offset_in_chunk), it's farily large so that it's very likely to be larger than a large bio's size, which avoids asynchronous submit. For instance, if we have a 512k bio, the efforts of calculating (512k/4k=128) checksums will be taken by the DIO task. Test results with fio (tested on a hard disk, not tested on ssd, 4cpu, 8g memory) bs async sync async sync bw bw(KB/S) iops iop 4k 115312 115480 28827.6 28869.6 8k 114381 115586 14297.4 14447.6 16k 115393 116290 7211.4 7267.6 32k 114268 116589 3570.4 3643 64k 115421 113417 1803 1771.8 <-----ASYNC wins here 128k 115545 112585 902 879 256k 115178 111521 449.2 435 512k 115874 111620 226 217.6 This adds a limit 'BTRFS_STRIPE_LEN(64k)' to decide if it's small enough to avoid asynchronous submit. Still, in this case we don't need to split the bio and can submit it directly. Signed-off-by: Liu Bo --- The job in the test, [global] rw=write ioengine=libaio direct=1 iodepth=64 iodepth_batch=64 iodepth_batch_complete=64 iodepth_low=64 bs=4k size=8g sync=0 group_reporting fallocate=posix invalidate=1 runtime=30 [dio] directory=/mnt/btrfs filename=foobar fs/btrfs/inode.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index e687bb0..c640d7e 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -7792,6 +7792,7 @@ static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, int nr_pages = 0; int ret; int async_submit = 0; + u64 alloc_profile; map_length = orig_bio->bi_iter.bi_size; ret = btrfs_map_block(root->fs_info, rw, start_sector << 9, @@ -7799,15 +7800,26 @@ static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, if (ret) return -EIO; + alloc_profile = btrfs_get_alloc_profile(root, 1); + if (map_length >= orig_bio->bi_iter.bi_size) { bio = orig_bio; dip->flags |= BTRFS_DIO_ORIG_BIO_SUBMITTED; + + /* + * In the case of 'single' profile, the above check is very + * likely to be true as map_length is (chunk_length - offset), + * so checking BTRFS_STRIPE_LEN here. + */ + if ((alloc_profile & BTRFS_BLOCK_GROUP_PROFILE_MASK) == 0 && + orig_bio->bi_iter.bi_size >= BTRFS_STRIPE_LEN) + async_submit = 1; + goto submit; } /* async crcs make it difficult to collect full stripe writes. */ - if (btrfs_get_alloc_profile(root, 1) & - (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6)) + if (alloc_profile & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6)) async_submit = 0; else async_submit = 1;