From patchwork Fri Oct 26 01:42:48 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Junichi Nomura X-Patchwork-Id: 1649751 Return-Path: X-Original-To: patchwork-dm-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by patchwork2.kernel.org (Postfix) with ESMTP id D2B9DDF2AB for ; Fri, 26 Oct 2012 01:48:49 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id q9Q1huDm027462; Thu, 25 Oct 2012 21:43:59 -0400 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id q9Q1hsxM012879 for ; Thu, 25 Oct 2012 21:43:54 -0400 Received: from mx1.redhat.com (ext-mx15.extmail.prod.ext.phx2.redhat.com [10.5.110.20]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id q9Q1hn8w024821; Thu, 25 Oct 2012 21:43:49 -0400 Received: from tyo202.gate.nec.co.jp (TYO202.gate.nec.co.jp [210.143.35.52]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q9Q1hlZH003644; Thu, 25 Oct 2012 21:43:48 -0400 Received: from mailgate3.nec.co.jp ([10.7.69.197]) by tyo202.gate.nec.co.jp (8.13.8/8.13.4) with ESMTP id q9Q1hdi0021239; Fri, 26 Oct 2012 10:43:39 +0900 (JST) Received: (from root@localhost) by mailgate3.nec.co.jp (8.11.7/3.7W-MAILGATE-NEC) id q9Q1hdG00146; Fri, 26 Oct 2012 10:43:39 +0900 (JST) Received: from mail02.kamome.nec.co.jp (mail02.kamome.nec.co.jp [10.25.43.5]) by mailsv3.nec.co.jp (8.13.8/8.13.4) with ESMTP id q9Q1hccG012888; Fri, 26 Oct 2012 10:43:39 +0900 (JST) Received: from kaishu.jp.nec.com ([10.26.220.5] [10.26.220.5]) by mail02.kamome.nec.co.jp with ESMTP id BT-MMP-2192310; Fri, 26 Oct 2012 10:42:49 +0900 Received: from xzibit.linux.bs1.fc.nec.co.jp ([10.34.125.175] [10.34.125.175]) by mail.jp.nec.com with ESMTP; Fri, 26 Oct 2012 10:42:48 +0900 Message-ID: <5089EA98.9070004@ce.jp.nec.com> Date: Fri, 26 Oct 2012 10:42:48 +0900 From: "Jun'ichi Nomura" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20120717 Thunderbird/14.0 MIME-Version: 1.0 To: "linux-kernel@vger.kernel.org" , device-mapper development References: <50890937.7010809@ce.jp.nec.com> In-Reply-To: <50890937.7010809@ce.jp.nec.com> X-RedHat-Spam-Score: -1.902 (BAYES_00,SPF_HELO_PASS,SPF_PASS) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Scanned-By: MIMEDefang 2.68 on 10.5.110.20 X-loop: dm-devel@redhat.com Cc: Tejun Heo , Jens Axboe , Alasdair G Kergon , Vivek Goyal Subject: Re: [dm-devel] [PATCH 2/2] dm: stay in blk_queue_bypass until queue becomes initialized X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk Reply-To: device-mapper development List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com On 10/25/12 18:41, Jun'ichi Nomura wrote: > With 749fefe677 ("block: lift the initial queue bypass mode on > blk_register_queue() instead of blk_init_allocated_queue()"), > add_disk() eventually calls blk_queue_bypass_end(). > This change invokes the following warning when multipath is used. ... > The warning means during queue initialization blk_queue_bypass_start() > calls sleeping function (synchronize_rcu) while dm holds md->type_lock. > > dm device initialization basically includes the following 3 steps: > 1. create ioctl, allocates queue and call add_disk() > 2. table load ioctl, determines device type and initialize queue > if request-based > 3. resume ioctl, device becomes functional > > So it is better to have dm's queue stay in bypass mode until > the initialization completes in table load ioctl. > > The effect of additional blk_queue_bypass_start(): > > 3.7-rc2 (plain) > # time for n in $(seq 1000); do dmsetup create --noudevsync --notable a; \ > dmsetup remove a; done > > real 0m15.434s > user 0m0.423s > sys 0m7.052s > > 3.7-rc2 (with this patch) > # time for n in $(seq 1000); do dmsetup create --noudevsync --notable a; \ > dmsetup remove a; done > real 0m19.766s > user 0m0.442s > sys 0m6.861s > > If this additional cost is not negligible, we need a variant of add_disk() > that does not end bypassing. Or call blk_queue_bypass_start() before add_disk(): --- If the patch is modified like above, we could fix the issue without incurring additional cost on dm device creation. # time for n in $(seq 1000); do dmsetup create --noudevsync --notable a; \ dmsetup remove a; done real 0m15.684s user 0m0.404s sys 0m7.181s diff --git a/drivers/md/dm.c b/drivers/md/dm.c index ad02761..d14639b 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1868,9 +1868,9 @@ static struct mapped_device *alloc_dev(int minor) md->disk->queue = md->queue; md->disk->private_data = md; sprintf(md->disk->disk_name, "dm-%d", minor); - add_disk(md->disk); /* Until md type is determined, put the queue in bypass mode */ blk_queue_bypass_start(md->queue); + add_disk(md->disk); format_dev_t(md->name, MKDEV(_major, minor)); md->wq = alloc_workqueue("kdmflush",