From patchwork Thu Dec 6 16:41:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716351 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C88717DB for ; Thu, 6 Dec 2018 16:42:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B8AF2F14E for ; Thu, 6 Dec 2018 16:42:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 602EF2F154; Thu, 6 Dec 2018 16:42:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id ED8192F14E for ; Thu, 6 Dec 2018 16:42:24 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D6A5307D973; Thu, 6 Dec 2018 16:42:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EA1025D786; Thu, 6 Dec 2018 16:42:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id AA2993F7D0; Thu, 6 Dec 2018 16:42:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6Gfg66030179 for ; Thu, 6 Dec 2018 11:41:42 -0500 Received: by smtp.corp.redhat.com (Postfix) id 6DE986A522; Thu, 6 Dec 2018 16:41:42 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx17.extmail.prod.ext.phx2.redhat.com [10.5.110.46]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 623745C730; Thu, 6 Dec 2018 16:41:42 +0000 (UTC) Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 33A5931256BF; Thu, 6 Dec 2018 16:41:41 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id i7so1192267qtj.10; Thu, 06 Dec 2018 08:41:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Lcg2zF1imfNyveimZfa7nNlz5mO8GsZUtILSOxC+mSg=; b=bXXWMjxtYPwSge+21FnBLq8KO5ZuuQnl4xQSczn6L5OjEEC8qcPX8nn0ETibu4DwIT ijqXOD9NpANlMbYeM0U7kT6c0zF+3BBNzb2t5R3qxoSChdDx+heSQObG11T/QWA0m1DW kSqCdocM4v2YeaswKVleIVd2w4Na7vcGCCbZjcI+ck1XxUeBPqruoBlJTYdfpqIj7tq3 FBd/PWF7Y5RhoByV7ioopLqnJAICSGx0LTgFA/sbZf88vqdMcdJx2QHntl+d3ld1DFzo vL6NsqvLGcQEKs9JbcY0FcJdI7kPrs/Nt7KWI3lbzrsZlBEQwMvaswwkghhjotCGdE5N LpyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Lcg2zF1imfNyveimZfa7nNlz5mO8GsZUtILSOxC+mSg=; b=ViVQTX+eVMJhJL0ITfsiWQJI0PtNbd0zCcmqpbr4ygr76LlyJvK6/3m2qttAeS8rnn NjzBfTp4e4KcT/lSnwaPX0YBMRFxA9wUm4iMv0LfDRE4ftkoxRDtWhnVhz29FipFV3jt 83PWPGqjiVDPHjO0557yyjx1v0jelhn6qMs2pTsP5686SZwkuSrQPiMKnlm5l+4H6Ode 0hx9l7AaFMWAaQtPqCvGlHrsbWkXw/7jcRTGts09Pkeaej+Tnmr8wAIlx52QRMkquuqD w9dsbHuE3tjzSGU3WIyoQk8mcPlbeN1tXuz67g6FLdhkkl9Dosq//D7QJGUYiiF2ufzM Q2nA== X-Gm-Message-State: AA+aEWb0riGFA30/Kyjrwe+8Zi09317XW17ML8cw6KF2w5qm70PGyZLP e0XaYWIOUeRl4bpY1uMN5pE+fW1v X-Google-Smtp-Source: AFSGD/WwTggLZMmPm8zEhSJJVdCoUyGlt5qRP5wQbg2YwXwbLz8oLBORAbWBA+NfsVGNU/w9Xu/glg== X-Received: by 2002:a0c:96b5:: with SMTP id a50mr28051792qvd.33.1544114500336; Thu, 06 Dec 2018 08:41:40 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id f201sm373039qke.0.2018.12.06.08.41.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:39 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:22 -0500 Message-Id: <20181206164122.2166-8-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 06 Dec 2018 16:41:41 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 06 Dec 2018 16:41:41 +0000 (UTC) for IP:'209.85.160.193' DOMAIN:'mail-qt1-f193.google.com' HELO:'mail-qt1-f193.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.072 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS) 209.85.160.193 mail-qt1-f193.google.com 209.85.160.193 mail-qt1-f193.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.84 on 10.5.110.46 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 7/7] dm: remove the pending IO accounting X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Thu, 06 Dec 2018 16:42:23 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP From: Mikulas Patocka Remove the "pending" atomic counters, that duplicate block-core's in_flight counters, and update md_in_flight() to look at percpu in_flight counters. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 -- drivers/md/dm.c | 34 +++++++++++++++------------------- 2 files changed, 15 insertions(+), 21 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index 224d44503a06..6fe883fac471 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -65,7 +65,6 @@ struct mapped_device { */ struct work_struct work; wait_queue_head_t wait; - atomic_t pending[2]; spinlock_t deferred_lock; struct bio_list deferred; @@ -119,7 +118,6 @@ struct mapped_device { struct srcu_struct io_barrier; }; -int md_in_flight(struct mapped_device *md); void disable_write_same(struct mapped_device *md); void disable_write_zeroes(struct mapped_device *md); diff --git a/drivers/md/dm.c b/drivers/md/dm.c index a8ae7931bce7..d4bc4237b09a 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -646,25 +646,30 @@ static void free_tio(struct dm_target_io *tio) bio_put(&tio->clone); } -int md_in_flight(struct mapped_device *md) +static bool md_in_flight(struct mapped_device *md) { - return atomic_read(&md->pending[READ]) + - atomic_read(&md->pending[WRITE]); + int cpu; + struct hd_struct *part = &dm_disk(md)->part0; + + for_each_possible_cpu(cpu) { + if (part_stat_local_read_cpu(part, in_flight[0], cpu) || + part_stat_local_read_cpu(part, in_flight[1], cpu)) + return true; + } + + return false; } static void start_io_acct(struct dm_io *io) { struct mapped_device *md = io->md; struct bio *bio = io->orig_bio; - int rw = bio_data_dir(bio); io->start_time = jiffies; generic_start_io_acct(md->queue, bio_op(bio), bio_sectors(bio), &dm_disk(md)->part0); - atomic_inc(&md->pending[rw]); - if (unlikely(dm_stats_used(&md->stats))) dm_stats_account_io(&md->stats, bio_data_dir(bio), bio->bi_iter.bi_sector, bio_sectors(bio), @@ -676,8 +681,6 @@ static void end_io_acct(struct dm_io *io) struct mapped_device *md = io->md; struct bio *bio = io->orig_bio; unsigned long duration = jiffies - io->start_time; - int pending; - int rw = bio_data_dir(bio); generic_end_io_acct(md->queue, bio_op(bio), &dm_disk(md)->part0, io->start_time); @@ -687,16 +690,11 @@ static void end_io_acct(struct dm_io *io) bio->bi_iter.bi_sector, bio_sectors(bio), true, duration, &io->stats_aux); - /* - * After this is decremented the bio must not be touched if it is - * a flush. - */ - pending = atomic_dec_return(&md->pending[rw]); - pending += atomic_read(&md->pending[rw^0x1]); - /* nudge anyone waiting on suspend queue */ - if (!pending) - wake_up(&md->wait); + if (unlikely(waitqueue_active(&md->wait))) { + if (!md_in_flight(md)) + wake_up(&md->wait); + } } /* @@ -1904,8 +1902,6 @@ static struct mapped_device *alloc_dev(int minor) if (!md->disk) goto bad; - atomic_set(&md->pending[0], 0); - atomic_set(&md->pending[1], 0); init_waitqueue_head(&md->wait); INIT_WORK(&md->work, dm_wq_work); init_waitqueue_head(&md->eventq);