From patchwork Wed Jan 11 01:21:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hou Tao X-Patchwork-Id: 9509011 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C79CB6075C for ; Wed, 11 Jan 2017 01:23:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8A1D2859F for ; Wed, 11 Jan 2017 01:23:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AD63C285B8; Wed, 11 Jan 2017 01:23:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CEE0C2859F for ; Wed, 11 Jan 2017 01:23:43 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v0B1MHsg012245; Tue, 10 Jan 2017 20:22:17 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v0B1MGb6031171 for ; Tue, 10 Jan 2017 20:22:16 -0500 Received: from mx1.redhat.com (ext-mx05.extmail.prod.ext.phx2.redhat.com [10.5.110.29]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v0B1MGoB029313 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 10 Jan 2017 20:22:16 -0500 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [119.145.14.66]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 160E23DE3E; Wed, 11 Jan 2017 01:22:10 +0000 (UTC) Received: from 172.24.1.47 (EHLO szxeml427-hub.china.huawei.com) ([172.24.1.47]) by szxrg03-dlp.huawei.com (MOS 4.4.3-GA FastPath queued) with ESMTP id CNU97594; Wed, 11 Jan 2017 09:22:05 +0800 (CST) Received: from [127.0.0.1] (10.177.31.14) by szxeml427-hub.china.huawei.com (10.82.67.182) with Microsoft SMTP Server id 14.3.235.1; Wed, 11 Jan 2017 09:22:01 +0800 To: Vivek Goyal References: <86d82254-b548-ae88-ddfe-e60af2cc009a@huawei.com> <20170110194259.GH23108@redhat.com> From: Hou Tao Message-ID: <546a1fdf-6c85-6cef-7572-337cbdcf6492@huawei.com> Date: Wed, 11 Jan 2017 09:21:56 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20170110194259.GH23108@redhat.com> X-Originating-IP: [10.177.31.14] X-CFilter-Loop: Reflected X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 199 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 11 Jan 2017 01:22:13 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 11 Jan 2017 01:22:13 +0000 (UTC) for IP:'119.145.14.66' DOMAIN:'szxga03-in.huawei.com' HELO:'szxga03-in.huawei.com' FROM:'houtao1@huawei.com' RCPT:'' X-RedHat-Spam-Score: -3.62 (BAYES_80, DCC_REPUT_13_19, RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RP_MATCHES_RCVD, SPF_PASS) 119.145.14.66 szxga03-in.huawei.com 119.145.14.66 szxga03-in.huawei.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.29 X-loop: dm-devel@redhat.com Cc: dm-devel@redhat.com, Mike Snitzer , Alasdair Kergon Subject: Re: [dm-devel] question about block-throttle on data device of dm-thin pool X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Virus-Scanned: ClamAV using ClamSMTP On 2017/1/11 3:42, Vivek Goyal wrote: > On Tue, Jan 10, 2017 at 02:47:02PM +0800, Hou Tao wrote: >> Hi, all. >> >> I am trying to test block-throttle on dm-thin devices. I find the throttling >> on dm-thin device is OK, but the throttling doesn't work for the data device >> of dm-thin pool. >> >> The following is my test case: >> #!/bin/sh >> >> dmsetup create pool --table '0 41943040 thin-pool /dev/vdb /dev/vda \ >> 128 6553 1 skip_block_zeroing >> dmsetup message /dev/mapper/pool 0 'create_thin 1' >> dmsetup create thin_1 --table '0 41943040 thin /dev/mapper/pool 1' >> >> mp=/thin_1 >> mkfs.xfs /dev/mapper/thin_1 >> mount /dev/mapper/thin_1 $mp >> >> cg=/sys/fs/cgroup/blkio/test >> mkdir -p $cg >> # get the block device id of the data device >> data_dev=$(dmsetup table /dev/mapper/pool | awk '{print $5}') >> echo "${data_dev} 1048576" > $cg/blkio.throttle.write_bps_device >> echo $$ > $cg/cgroup.procs >> dd if=/dev/zero of=$mp/zero bs=1M count=1 oflag=direct >> >> I read the dm-thin code roughly and find out that most bios are submitted >> by the workqueue of thin pool instead of the dd process which initiates the >> O_DIRECT write operations. The bios belong to the block cgroup "blkcg_root" >> instead of the created block cgroup "test" in test case, so the write >> limitation of blkcg "test" doesn't work. >> >> In order to make the throttling work out, can we save the original block >> cgroup info of the deferred bios and use the saved block cgroup info to >> submit the bios ? Is the method reasonable and is there a better way to >> complete the throttling on the data device of the thin pool ? > > I thought we had a patches where bio_clone_bioset() also retained > cgroup information of original bio. > > commit 20bd723ec6a3261df5e02250cd3a1fbb09a343f2 > Author: Paolo Valente > Date: Wed Jul 27 07:22:05 2016 +0200 > > block: add missing group association in bio-cloning functions > > Are you running new enough kernel. If not, may be there are still > some places either in generic code or dm code where cloned/newly > created bios are attributed to root cgroup and not to the cgroup > of bio which caused creation of that new bio. > > Vivek > Hi, Vivek. Thanks for your reply. The version of the used linux kernel is 4.10-rc3. bio_clone_blkcg_association() works only when the bi_css of source bio has been initializaed before. In my test case, the bios submitted to dm-thin device will not by throttled, so the bi_css is NULL. Maybe we need assign the bi_css field of bio by usingbio_associate_current() when the bi_css field is NULL and the target may defer the submit of the bio. Something likes the following patch does: --- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c index d1c05c1..97392a9 100644 --- a/drivers/md/dm-thin.c +++ b/drivers/md/dm-thin.c @@ -4177,6 +4177,7 @@ static int thin_ctr(struct dm_target *ti, unsigned argc, char **argv) static int thin_map(struct dm_target *ti, struct bio *bio) { bio->bi_iter.bi_sector = dm_target_offset(ti, bio->bi_iter.bi_sector); + dm_retain_bio_blkcg(bio); return thin_bio_map(ti, bio); } diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 3086da5..f129ca4 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1290,9 +1290,10 @@ static blk_qc_t dm_make_request(struct request_queue *q, struct bio *bio) if (unlikely(test_bit(DMF_BLOCK_IO_FOR_SUSPEND, &md->flags))) { dm_put_live_table(md, srcu_idx); - if (!(bio->bi_opf & REQ_RAHEAD)) + if (!(bio->bi_opf & REQ_RAHEAD)) { + dm_retain_bio_blkcg(bio); queue_io(md, bio); - else + } else bio_io_error(bio); return BLK_QC_T_NONE; } diff --git a/drivers/md/dm.h b/drivers/md/dm.h index f0aad08..e86c74f 100644 --- a/drivers/md/dm.h +++ b/drivers/md/dm.h @@ -214,4 +214,10 @@ void dm_free_md_mempools(struct dm_md_mempools *pools); */ unsigned dm_get_reserved_bio_based_ios(void); +static inline void dm_retain_bio_blkcg(struct bio *bio) +{ + if (!bio->bi_css) + bio_associate_current(bio); +} + #endif