From patchwork Thu Dec 6 16:41:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716341 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1CC114E2 for ; Thu, 6 Dec 2018 16:42:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF8E92F0DC for ; Thu, 6 Dec 2018 16:42:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ADC322F14D; Thu, 6 Dec 2018 16:42:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3FEA72F0DE for ; Thu, 6 Dec 2018 16:42:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D2AD9A1F6A; Thu, 6 Dec 2018 16:42:00 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A2083226E8; Thu, 6 Dec 2018 16:42:00 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 594FA3F7CD; Thu, 6 Dec 2018 16:42:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6GfUw5030127 for ; Thu, 6 Dec 2018 11:41:30 -0500 Received: by smtp.corp.redhat.com (Postfix) id 4D70A3782; Thu, 6 Dec 2018 16:41:30 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx16.extmail.prod.ext.phx2.redhat.com [10.5.110.45]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3E4CC662D1; Thu, 6 Dec 2018 16:41:29 +0000 (UTC) Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0757330014B1; Thu, 6 Dec 2018 16:41:29 +0000 (UTC) Received: by mail-qt1-f196.google.com with SMTP id r14so1245400qtp.1; Thu, 06 Dec 2018 08:41:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=VS4iuAD6sE8maCiA4jBHJmimohyGnKGLtLR2nbbQu/g=; b=TbSEz4BgTiPstcqTMLG9oDGKXzhfwvHm6sWD7VyVQEjLqNNlJRmhbDd6htvJXV+JVe Iu5TEoTJSjjk8d7YR2Dfk5Uy+tL99kFaYPgRETp323IVm3FGWbgom4ZV9R5Swxt9zRS/ dGZ8+15zYZXLE9ECUsNwVo9QBCThG36mZXBuzBGycMABvrEJCjR3oro9+14Ter79+qBj r8vEjCN1EaB52EzTLMhO8azQBsTmgjDHuG+oXzgOzd3GwO+SHSNl/JTaNbgF3Xw/elwk 7GSgmHD8tf7LLF6b75zfIDlKwoeeoGZn1nNR8J5cQGkWtxvB79y9HHbUoWE2+bUXd6ai bA9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=VS4iuAD6sE8maCiA4jBHJmimohyGnKGLtLR2nbbQu/g=; b=p2dm696AQoRqkNa76ppwWOGComkSrTxlicAC/jl64Itl2P3J4uin0gSiHM8FD2lTTs Ota6/h5G0d5hSEurxk9WBvEcUxAaqAdNkHiiGMV0mpllN8KShMD7T+DaKdQFDlRB+o64 u4IWYPJlllQgOjZ81a7e1BWxJg2jBzAX7Td89n2sZ0wzpRwbelYcKV/6xMgz7xCfy7PM XFUfd1MIEViAN2awKisWZdcwfd2axZ20Hic3HqxmgWIVVtAJRacGmnddYJ7bsS2DuE+x Z1dW3pF6HI6CyRC3E6Gu8u8NvjOTBuHa+axfmObpKwgEhkrddff62e/HdiNpnw38qdKD GpJQ== X-Gm-Message-State: AA+aEWYobkE4bRpcVoyr70kj1pdVvDHln3ODGUKtvIFk6+XGjXBLPGHu T9iNkBt+xtEV6VsdcNf4Az+jqMIR X-Google-Smtp-Source: AFSGD/U7OJtFzzyUJYyYgohoXBmchJhF9ty6DRKXPvOs7nbzkiDXBDMJrKu+1GUSlk7NGtAanpCnmw== X-Received: by 2002:ac8:3a64:: with SMTP id w91mr28264753qte.70.1544114488189; Thu, 06 Dec 2018 08:41:28 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id y2sm585898qtb.88.2018.12.06.08.41.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:27 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:16 -0500 Message-Id: <20181206164122.2166-2-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 06 Dec 2018 16:41:29 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 06 Dec 2018 16:41:29 +0000 (UTC) for IP:'209.85.160.196' DOMAIN:'mail-qt1-f196.google.com' HELO:'mail-qt1-f196.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.072 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS) 209.85.160.196 mail-qt1-f196.google.com 209.85.160.196 mail-qt1-f196.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.84 on 10.5.110.45 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 1/7] dm: dont rewrite dm_disk(md)->part0.in_flight X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 06 Dec 2018 16:42:01 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP From: Mikulas Patocka generic_start_io_acct and generic_end_io_acct already update the variable in_flight using atomic operations, so we don't have to overwrite them again. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index a733e4c920af..a8ae7931bce7 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -663,8 +663,7 @@ static void start_io_acct(struct dm_io *io) generic_start_io_acct(md->queue, bio_op(bio), bio_sectors(bio), &dm_disk(md)->part0); - atomic_set(&dm_disk(md)->part0.in_flight[rw], - atomic_inc_return(&md->pending[rw])); + atomic_inc(&md->pending[rw]); if (unlikely(dm_stats_used(&md->stats))) dm_stats_account_io(&md->stats, bio_data_dir(bio), @@ -693,7 +692,6 @@ static void end_io_acct(struct dm_io *io) * a flush. */ pending = atomic_dec_return(&md->pending[rw]); - atomic_set(&dm_disk(md)->part0.in_flight[rw], pending); pending += atomic_read(&md->pending[rw^0x1]); /* nudge anyone waiting on suspend queue */ From patchwork Thu Dec 6 16:41:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716347 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11DF214E2 for ; Thu, 6 Dec 2018 16:42:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F3A5E2F0F4 for ; Thu, 6 Dec 2018 16:42:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F1BC52F12D; Thu, 6 Dec 2018 16:42:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 984E02F142 for ; Thu, 6 Dec 2018 16:42:18 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8F3F1307DA35; Thu, 6 Dec 2018 16:42:17 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 67EBF1001F5B; Thu, 6 Dec 2018 16:42:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 31985181B9EA; Thu, 6 Dec 2018 16:42:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6GfYNr030148 for ; Thu, 6 Dec 2018 11:41:34 -0500 Received: by smtp.corp.redhat.com (Postfix) id 1EC5A5E1A2; Thu, 6 Dec 2018 16:41:34 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx09.extmail.prod.ext.phx2.redhat.com [10.5.110.38]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 13AF55FC02; Thu, 6 Dec 2018 16:41:32 +0000 (UTC) Received: from mail-qt1-f194.google.com (mail-qt1-f194.google.com [209.85.160.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 05B71A4056; Thu, 6 Dec 2018 16:41:31 +0000 (UTC) Received: by mail-qt1-f194.google.com with SMTP id p17so1223502qtl.5; Thu, 06 Dec 2018 08:41:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Km3DVg7kPkPAtrwtLRbiZLclidyo2CRUUEynAEX5xBM=; b=UN3U5ouoIo7M5Ur/+YUgbw0A8Qv2l5hRlWwji2YMu7la0KY9AprDSU2AkhZ7EPKx82 E6Tb4DVe74VN/yuMPLIZN30Nw/UNqSw2AAWBtHm4my8UgFotdUQbuSNchqgmA+pF5ZpQ 5wxKKM6Cq+NGDmb6r+kjPwjmBkF/NFHV4kbrY/qGgfNwC8ak8Ac7NkPAZ/b1iIO8kLbL z47RgZgFK1Fc4y+bUGxY1J+4nCGrEipEh6tBL+WYGBl9dCI48yqQEVR5t4iB7b2N3tYb hWdfYtZstEK757WW6qxoy9nbggAUiXZGEIks3G/B8+nTOiYiak09gNcvK7CO2tEAoZ64 UtuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Km3DVg7kPkPAtrwtLRbiZLclidyo2CRUUEynAEX5xBM=; b=cOdrVAaPNNlFHlyM9BiIrPeN5yzT+Ga5JF7XjCwcz8eVGnnlf9ggySRCYytcHX2lL4 DsXaz2aA2isKQnv6YF+kdh/4soggVcQwWJ3lsRYCqrnyDBfYFQiGZwgdG5jqUebvHaQ6 kWQOBolnI75meWOof/8WvRSrtzTh4G0rG/ixGHhI+yg+6+gFeX8SF8/HspV1ftR2A9NR 49piT5T+JHJVQlT0r9r+nZPjPuGxQnw4YG9rmFXJHg1CssMdbuwalC617ICG8ZgDfjer 0Ah7PEMiGINRbOhakmu/4RpSfm7ovIw31jEO98ihvLdjWLIzjjiC4uOa59voSbsDzepX qWrg== X-Gm-Message-State: AA+aEWb0f00ni45/2iITFSp6ma7VSRPtwCZ2FGnUa8f84Djz7Pjt1YyR Rs51tvwc9pchEuQ3jUZ/wQ0= X-Google-Smtp-Source: AFSGD/XnxQ5VTVVYmF/iDAsXyyPE+2EEFFq4m7bO6etb5EAVApSCPju368BTobbxnlV1wyT2jx3aeg== X-Received: by 2002:a0c:f053:: with SMTP id b19mr28101301qvl.29.1544114490165; Thu, 06 Dec 2018 08:41:30 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id v53sm586744qtb.6.2018.12.06.08.41.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:29 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:17 -0500 Message-Id: <20181206164122.2166-3-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 06 Dec 2018 16:41:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 06 Dec 2018 16:41:31 +0000 (UTC) for IP:'209.85.160.194' DOMAIN:'mail-qt1-f194.google.com' HELO:'mail-qt1-f194.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.072 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS) 209.85.160.194 mail-qt1-f194.google.com 209.85.160.194 mail-qt1-f194.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.78 on 10.5.110.38 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 2/7] dm rq: leverage blk_mq_queue_busy() to check for outstanding IO X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Thu, 06 Dec 2018 16:42:18 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP Now that request-based dm-multipath only supports blk-mq, make use of the newly introduced blk_mq_queue_busy() to check for outstanding IO -- rather than (ab)using the block core's in_flight counters. Signed-off-by: Mike Snitzer --- drivers/md/dm-rq.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/drivers/md/dm-rq.c b/drivers/md/dm-rq.c index 1f1fe9a618ea..d2397d8fcbd1 100644 --- a/drivers/md/dm-rq.c +++ b/drivers/md/dm-rq.c @@ -130,11 +130,11 @@ static void rq_end_stats(struct mapped_device *md, struct request *orig) */ static void rq_completed(struct mapped_device *md, int rw, bool run_queue) { - atomic_dec(&md->pending[rw]); - /* nudge anyone waiting on suspend queue */ - if (!md_in_flight(md)) - wake_up(&md->wait); + if (unlikely(waitqueue_active(&md->wait))) { + if (!blk_mq_queue_busy(md->queue)) + wake_up(&md->wait); + } /* * dm_put() must be at the end of this function. See the comment above @@ -436,7 +436,6 @@ ssize_t dm_attr_rq_based_seq_io_merge_deadline_store(struct mapped_device *md, static void dm_start_request(struct mapped_device *md, struct request *orig) { blk_mq_start_request(orig); - atomic_inc(&md->pending[rq_data_dir(orig)]); if (unlikely(dm_stats_used(&md->stats))) { struct dm_rq_target_io *tio = tio_from_request(orig); From patchwork Thu Dec 6 16:41:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716353 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E07114E2 for ; Thu, 6 Dec 2018 16:42:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B4692F159 for ; Thu, 6 Dec 2018 16:42:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1010A2F15A; Thu, 6 Dec 2018 16:42:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 564832F156 for ; Thu, 6 Dec 2018 16:42:27 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3783C300207A; Thu, 6 Dec 2018 16:42:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0661D17C28; Thu, 6 Dec 2018 16:42:26 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C5BFA181B9EB; Thu, 6 Dec 2018 16:42:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6GfgwE030185 for ; Thu, 6 Dec 2018 11:41:42 -0500 Received: by smtp.corp.redhat.com (Postfix) id AC3A25D96F; Thu, 6 Dec 2018 16:41:42 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx08.extmail.prod.ext.phx2.redhat.com [10.5.110.32]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9F6424F9AB; Thu, 6 Dec 2018 16:41:40 +0000 (UTC) Received: from mail-qk1-f195.google.com (mail-qk1-f195.google.com [209.85.222.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 284CAC04B2F3; Thu, 6 Dec 2018 16:41:33 +0000 (UTC) Received: by mail-qk1-f195.google.com with SMTP id q70so687166qkh.6; Thu, 06 Dec 2018 08:41:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=85coUe5jj/aEgGb9sq6PLEn2E7N+OcXpkyJUZd5up8g=; b=dy8ba+YEMlUWrdDOY7IG2NTCbIgkWE87YxoMwjJ2pG8VYKGv7Afz1StZgMNek+tAy2 IenTxIqIBBXsL6lchbVk166bGQ2IYvj2Td9N+tVRxeMsJYjE3AuNyIsMbTGR6ibZTu2M aE1WIkky2XB8SuQYekNNfvY0D/pCuEGY43kf3DA7nVVlaeG3VP2s2AQaIIoU1LLEXRcT N8Xgl8v8svhi/fx5KtiyJ/g+atUTJ27nBA20NGx9iiZVlL8Kn0iaC9dMammSfErOWNME qBWf8wIb5MZqHXAhU64ZpOzZI1Ubup7D1d8UjAji37TQlNnTAfICGnnUs5izFP9p1jhH uUTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=85coUe5jj/aEgGb9sq6PLEn2E7N+OcXpkyJUZd5up8g=; b=ff1Zd/cewBgg/ofsfqY3HyLmXkZpclQXck4M3pR9L+FqruGwiUW+HzuQnj4O7a6dbA Kj9/CG+8CczrrO9UXFWhSFHBe+YexYt6RGkOtxoVXvIY5YRcZieBZ0isRnXQZDaIwUpW mkU3Nz4ZI5Cv257I6dmtsy7hQpR2gGPsATVFmi45BNJvTUnNeo9qmjR+KSQejuoSM3gR iYC0F6LX3dHCSdHtoABQ/TPQ9sf49IiiO53L7KztQU5qMmyY5B8TAuCV2yLegjau0wf/ J9ieoVOeLN65ymO1MJv6utUkh8jz2dQk0KD+FX/LstOOEgkgomInfxXRfy0frBXTrEC9 4CSg== X-Gm-Message-State: AA+aEWbKH18yklfBrDj3P6ynV9xxbeEmEXTRSk9rAa4KVnX9Tn3/W6HL 6T5p6QAgV/sgg/PVrpiCmkdKIe8T X-Google-Smtp-Source: AFSGD/WYWt0tHxQtDo6ZUGwLGPj/R9kz9OgisWdphiMW4Eryy+WA2+OOqP8kVQPvI6xxsTvRsR325g== X-Received: by 2002:a37:3b47:: with SMTP id i68mr26961444qka.277.1544114492204; Thu, 06 Dec 2018 08:41:32 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id p31sm763892qkh.86.2018.12.06.08.41.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:31 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:18 -0500 Message-Id: <20181206164122.2166-4-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 06 Dec 2018 16:41:33 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 06 Dec 2018 16:41:33 +0000 (UTC) for IP:'209.85.222.195' DOMAIN:'mail-qk1-f195.google.com' HELO:'mail-qk1-f195.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.091 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS) 209.85.222.195 mail-qk1-f195.google.com 209.85.222.195 mail-qk1-f195.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.78 on 10.5.110.32 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 3/7] block: stop passing 'cpu' to all percpu stats methods X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Thu, 06 Dec 2018 16:42:26 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP All of part_stat_* and related methods are used with preempt disabled, so there is no need to pass cpu around to allow of them. Just call smp_processor_id() as needed. Suggested-by: Jens Axboe Signed-off-by: Mike Snitzer --- block/bio.c | 16 +++++++++------- block/blk-core.c | 34 +++++++++++++++------------------- block/blk-merge.c | 5 ++--- block/genhd.c | 5 ++--- block/partition-generic.c | 5 ++--- drivers/md/md.c | 7 +++---- include/linux/genhd.h | 26 +++++++++++++------------- 7 files changed, 46 insertions(+), 52 deletions(-) diff --git a/block/bio.c b/block/bio.c index 03895cc0d74a..91e398ba57f1 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1667,11 +1667,12 @@ void generic_start_io_acct(struct request_queue *q, int op, unsigned long sectors, struct hd_struct *part) { const int sgrp = op_stat_group(op); - int cpu = part_stat_lock(); - part_round_stats(q, cpu, part); - part_stat_inc(cpu, part, ios[sgrp]); - part_stat_add(cpu, part, sectors[sgrp], sectors); + part_stat_lock(); + + part_round_stats(q, part); + part_stat_inc(part, ios[sgrp]); + part_stat_add(part, sectors[sgrp], sectors); part_inc_in_flight(q, part, op_is_write(op)); part_stat_unlock(); @@ -1683,10 +1684,11 @@ void generic_end_io_acct(struct request_queue *q, int req_op, { unsigned long duration = jiffies - start_time; const int sgrp = op_stat_group(req_op); - int cpu = part_stat_lock(); - part_stat_add(cpu, part, nsecs[sgrp], jiffies_to_nsecs(duration)); - part_round_stats(q, cpu, part); + part_stat_lock(); + + part_stat_add(part, nsecs[sgrp], jiffies_to_nsecs(duration)); + part_round_stats(q, part); part_dec_in_flight(q, part, op_is_write(req_op)); part_stat_unlock(); diff --git a/block/blk-core.c b/block/blk-core.c index ad59102ee30a..734b768c9d9d 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -584,14 +584,14 @@ struct request *blk_get_request(struct request_queue *q, unsigned int op, } EXPORT_SYMBOL(blk_get_request); -static void part_round_stats_single(struct request_queue *q, int cpu, +static void part_round_stats_single(struct request_queue *q, struct hd_struct *part, unsigned long now, unsigned int inflight) { if (inflight) { - __part_stat_add(cpu, part, time_in_queue, + __part_stat_add(part, time_in_queue, inflight * (now - part->stamp)); - __part_stat_add(cpu, part, io_ticks, (now - part->stamp)); + __part_stat_add(part, io_ticks, (now - part->stamp)); } part->stamp = now; } @@ -599,7 +599,6 @@ static void part_round_stats_single(struct request_queue *q, int cpu, /** * part_round_stats() - Round off the performance stats on a struct disk_stats. * @q: target block queue - * @cpu: cpu number for stats access * @part: target partition * * The average IO queue length and utilisation statistics are maintained @@ -613,7 +612,7 @@ static void part_round_stats_single(struct request_queue *q, int cpu, * /proc/diskstats. This accounts immediately for all queue usage up to * the current jiffies and restarts the counters again. */ -void part_round_stats(struct request_queue *q, int cpu, struct hd_struct *part) +void part_round_stats(struct request_queue *q, struct hd_struct *part) { struct hd_struct *part2 = NULL; unsigned long now = jiffies; @@ -635,9 +634,9 @@ void part_round_stats(struct request_queue *q, int cpu, struct hd_struct *part) part_in_flight(q, part, inflight); if (stats & 2) - part_round_stats_single(q, cpu, part2, now, inflight[1]); + part_round_stats_single(q, part2, now, inflight[1]); if (stats & 1) - part_round_stats_single(q, cpu, part, now, inflight[0]); + part_round_stats_single(q, part, now, inflight[0]); } EXPORT_SYMBOL_GPL(part_round_stats); @@ -1362,11 +1361,10 @@ void blk_account_io_completion(struct request *req, unsigned int bytes) if (blk_do_io_stat(req)) { const int sgrp = op_stat_group(req_op(req)); struct hd_struct *part; - int cpu; - cpu = part_stat_lock(); + part_stat_lock(); part = req->part; - part_stat_add(cpu, part, sectors[sgrp], bytes >> 9); + part_stat_add(part, sectors[sgrp], bytes >> 9); part_stat_unlock(); } } @@ -1381,14 +1379,13 @@ void blk_account_io_done(struct request *req, u64 now) if (blk_do_io_stat(req) && !(req->rq_flags & RQF_FLUSH_SEQ)) { const int sgrp = op_stat_group(req_op(req)); struct hd_struct *part; - int cpu; - cpu = part_stat_lock(); + part_stat_lock(); part = req->part; - part_stat_inc(cpu, part, ios[sgrp]); - part_stat_add(cpu, part, nsecs[sgrp], now - req->start_time_ns); - part_round_stats(req->q, cpu, part); + part_stat_inc(part, ios[sgrp]); + part_stat_add(part, nsecs[sgrp], now - req->start_time_ns); + part_round_stats(req->q, part); part_dec_in_flight(req->q, part, rq_data_dir(req)); hd_struct_put(part); @@ -1400,16 +1397,15 @@ void blk_account_io_start(struct request *rq, bool new_io) { struct hd_struct *part; int rw = rq_data_dir(rq); - int cpu; if (!blk_do_io_stat(rq)) return; - cpu = part_stat_lock(); + part_stat_lock(); if (!new_io) { part = rq->part; - part_stat_inc(cpu, part, merges[rw]); + part_stat_inc(part, merges[rw]); } else { part = disk_map_sector_rcu(rq->rq_disk, blk_rq_pos(rq)); if (!hd_struct_try_get(part)) { @@ -1424,7 +1420,7 @@ void blk_account_io_start(struct request *rq, bool new_io) part = &rq->rq_disk->part0; hd_struct_get(part); } - part_round_stats(rq->q, cpu, part); + part_round_stats(rq->q, part); part_inc_in_flight(rq->q, part, rw); rq->part = part; } diff --git a/block/blk-merge.c b/block/blk-merge.c index 4431da69a5cf..a120d59b9705 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -685,12 +685,11 @@ static void blk_account_io_merge(struct request *req) { if (blk_do_io_stat(req)) { struct hd_struct *part; - int cpu; - cpu = part_stat_lock(); + part_stat_lock(); part = req->part; - part_round_stats(req->q, cpu, part); + part_round_stats(req->q, part); part_dec_in_flight(req->q, part, rq_data_dir(req)); hd_struct_put(part); diff --git a/block/genhd.c b/block/genhd.c index 0145bcb0cc76..2fe00cf32b93 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1326,7 +1326,6 @@ static int diskstats_show(struct seq_file *seqf, void *v) struct hd_struct *hd; char buf[BDEVNAME_SIZE]; unsigned int inflight[2]; - int cpu; /* if (&disk_to_dev(gp)->kobj.entry == block_class.devices.next) @@ -1338,8 +1337,8 @@ static int diskstats_show(struct seq_file *seqf, void *v) disk_part_iter_init(&piter, gp, DISK_PITER_INCL_EMPTY_PART0); while ((hd = disk_part_iter_next(&piter))) { - cpu = part_stat_lock(); - part_round_stats(gp->queue, cpu, hd); + part_stat_lock(); + part_round_stats(gp->queue, hd); part_stat_unlock(); part_in_flight(gp->queue, hd, inflight); seq_printf(seqf, "%4d %7d %s " diff --git a/block/partition-generic.c b/block/partition-generic.c index 5f8db5c5140f..7e663cfb1487 100644 --- a/block/partition-generic.c +++ b/block/partition-generic.c @@ -121,10 +121,9 @@ ssize_t part_stat_show(struct device *dev, struct hd_struct *p = dev_to_part(dev); struct request_queue *q = part_to_disk(p)->queue; unsigned int inflight[2]; - int cpu; - cpu = part_stat_lock(); - part_round_stats(q, cpu, p); + part_stat_lock(); + part_round_stats(q, p); part_stat_unlock(); part_in_flight(q, p, inflight); return sprintf(buf, diff --git a/drivers/md/md.c b/drivers/md/md.c index fc488cb30a94..9a0a1e0934d5 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -334,7 +334,6 @@ static blk_qc_t md_make_request(struct request_queue *q, struct bio *bio) const int sgrp = op_stat_group(bio_op(bio)); struct mddev *mddev = q->queuedata; unsigned int sectors; - int cpu; blk_queue_split(q, &bio); @@ -359,9 +358,9 @@ static blk_qc_t md_make_request(struct request_queue *q, struct bio *bio) md_handle_request(mddev, bio); - cpu = part_stat_lock(); - part_stat_inc(cpu, &mddev->gendisk->part0, ios[sgrp]); - part_stat_add(cpu, &mddev->gendisk->part0, sectors[sgrp], sectors); + part_stat_lock(); + part_stat_inc(&mddev->gendisk->part0, ios[sgrp]); + part_stat_add(&mddev->gendisk->part0, sectors[sgrp], sectors); part_stat_unlock(); return BLK_QC_T_NONE; diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 0c5ee17b4d88..1677cd2a4c4e 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -295,8 +295,8 @@ extern struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, #define part_stat_lock() ({ rcu_read_lock(); get_cpu(); }) #define part_stat_unlock() do { put_cpu(); rcu_read_unlock(); } while (0) -#define __part_stat_add(cpu, part, field, addnd) \ - (per_cpu_ptr((part)->dkstats, (cpu))->field += (addnd)) +#define __part_stat_add(part, field, addnd) \ + (per_cpu_ptr((part)->dkstats, smp_processor_id())->field += (addnd)) #define part_stat_read(part, field) \ ({ \ @@ -333,7 +333,7 @@ static inline void free_part_stats(struct hd_struct *part) #define part_stat_lock() ({ rcu_read_lock(); 0; }) #define part_stat_unlock() rcu_read_unlock() -#define __part_stat_add(cpu, part, field, addnd) \ +#define __part_stat_add(part, field, addnd) \ ((part)->dkstats.field += addnd) #define part_stat_read(part, field) ((part)->dkstats.field) @@ -362,19 +362,19 @@ static inline void free_part_stats(struct hd_struct *part) part_stat_read(part, field[STAT_WRITE]) + \ part_stat_read(part, field[STAT_DISCARD])) -#define part_stat_add(cpu, part, field, addnd) do { \ - __part_stat_add((cpu), (part), field, addnd); \ +#define part_stat_add(part, field, addnd) do { \ + __part_stat_add((part), field, addnd); \ if ((part)->partno) \ - __part_stat_add((cpu), &part_to_disk((part))->part0, \ + __part_stat_add(&part_to_disk((part))->part0, \ field, addnd); \ } while (0) -#define part_stat_dec(cpu, gendiskp, field) \ - part_stat_add(cpu, gendiskp, field, -1) -#define part_stat_inc(cpu, gendiskp, field) \ - part_stat_add(cpu, gendiskp, field, 1) -#define part_stat_sub(cpu, gendiskp, field, subnd) \ - part_stat_add(cpu, gendiskp, field, -subnd) +#define part_stat_dec(gendiskp, field) \ + part_stat_add(gendiskp, field, -1) +#define part_stat_inc(gendiskp, field) \ + part_stat_add(gendiskp, field, 1) +#define part_stat_sub(gendiskp, field, subnd) \ + part_stat_add(gendiskp, field, -subnd) void part_in_flight(struct request_queue *q, struct hd_struct *part, unsigned int inflight[2]); @@ -399,7 +399,7 @@ static inline void free_part_info(struct hd_struct *part) } /* block/blk-core.c */ -extern void part_round_stats(struct request_queue *q, int cpu, struct hd_struct *part); +extern void part_round_stats(struct request_queue *q, struct hd_struct *part); /* block/genhd.c */ extern void device_add_disk(struct device *parent, struct gendisk *disk, From patchwork Thu Dec 6 16:41:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716343 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E219E14E2 for ; Thu, 6 Dec 2018 16:42:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CFA712F0D5 for ; Thu, 6 Dec 2018 16:42:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C3DEB2F13B; Thu, 6 Dec 2018 16:42:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B10D62F120 for ; Thu, 6 Dec 2018 16:42:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2375830DDBC3; Thu, 6 Dec 2018 16:42:05 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EDC911057048; Thu, 6 Dec 2018 16:42:04 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B48B7181B9E7; Thu, 6 Dec 2018 16:42:04 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6GfdNb030159 for ; Thu, 6 Dec 2018 11:41:39 -0500 Received: by smtp.corp.redhat.com (Postfix) id 20C7B5D77E; Thu, 6 Dec 2018 16:41:39 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 150D45D6A6; Thu, 6 Dec 2018 16:41:36 +0000 (UTC) Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 31BC2C0586D3; Thu, 6 Dec 2018 16:41:35 +0000 (UTC) Received: by mail-qk1-f169.google.com with SMTP id q70so687234qkh.6; Thu, 06 Dec 2018 08:41:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Iw1pxVal0P9iW2FYwEShmYf7++g/Hc6pwU/St/Unn+E=; b=Jhm0lkFPku52yv/ydwT9kt6hMkiCNiL4QCNcGxo+kbHwgAw1Q6zp0yrlzNAwnVFoCc 7PGncfiJ8u2TQB5ggkGbdr/8IH43vu6P3awwhuG2sCeYLc13diqcml2c6sGDEGDctkvo h158omwn9wpGQLkgk+ot6rC6mEFA/Cgaqag1POgXu7JuiZsXgSshZajr1uhGYHh6JG90 8Ur5+VVdUlOJlJMYbVWqAI64iN/EgsRE6H3nlj/OH1iTcIffSnrETaeCgTrr6JtwJl3I /gWj//q5aQMO3TMoBKD0eTGwcRhqE1pPpUyXyxP9ftc6JUaKsDXyJPY8amIVj0bI37jk pP4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Iw1pxVal0P9iW2FYwEShmYf7++g/Hc6pwU/St/Unn+E=; b=I71mGQdkReXtd5r3D88UpzNTRnX12Hq4hMaxzAYtCsfVXiSvU0LE/ZMzvp1YTtS22e h3K+2gVs+a1Cc4zdXPqDhqMQfGhppzcEEu/iK2YrcSuaq7sE0mCgKLeeqN4Ra/1H0D1n Z+7VlTrgEU6t0vHMX2JtJ8odsl11vyWzxhrZMbT5/JjPhr4g+yutgogt5goOqbitt8OR QhxZVQUeKkoLLQIXf9ASdKDq8E5JKUcVepFOZzJiQ9ziSCYO+7FYQ3vDGsCyNQ3+ib5o Z/2La/xNwCt6ekMlZVPpxIv3GmzbAA+1yHMP50UmPZ+s/PAiEhK6DNHlqlL2AZLQMRf6 0v8Q== X-Gm-Message-State: AA+aEWY25SeUo/32tyxOZCoXUGCTWJyIqqZ2gyaFd9V/8KD56mVbLJBX pVlq50qv/Vqlw3Yj/zXTb/JJLUau X-Google-Smtp-Source: AFSGD/WJ6tocTP1JdRSp7eRm9dZcgVxLYECUibvsnMTtzJl4J9OIL6VOOE0kgk2DDk+zBZTe+wyyEA== X-Received: by 2002:a37:a9cf:: with SMTP id s198mr25268156qke.274.1544114494277; Thu, 06 Dec 2018 08:41:34 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id p42sm509546qte.8.2018.12.06.08.41.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:33 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:19 -0500 Message-Id: <20181206164122.2166-5-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 06 Dec 2018 16:41:35 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 06 Dec 2018 16:41:35 +0000 (UTC) for IP:'209.85.222.169' DOMAIN:'mail-qk1-f169.google.com' HELO:'mail-qk1-f169.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.091 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS) 209.85.222.169 mail-qk1-f169.google.com 209.85.222.169 mail-qk1-f169.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 4/7] block: delete part_round_stats and switch to less precise counting X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Thu, 06 Dec 2018 16:42:06 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP From: Mikulas Patocka We want to convert to per-cpu in_flight counters. The function part_round_stats needs the in_flight counter every jiffy, it would be too costly to sum all the percpu variables every jiffy, so it must be deleted. part_round_stats is used to calculate two counters - time_in_queue and io_ticks. time_in_queue can be calculated without part_round_stats, by adding the duration of the I/O when the I/O ends (the value is almost as exact as the previously calculated value, except that time for in-progress I/Os is not counted). io_ticks can be approximated by increasing the value when I/O is started or ended and the jiffies value has changed. If the I/Os take less than a jiffy, the value is as exact as the previously calculated value. If the I/Os take more than a jiffy, io_ticks can drift behind the previously calculated value. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/bio.c | 24 +++++++++++++++--- block/blk-core.c | 62 +++-------------------------------------------- block/blk-merge.c | 1 - block/genhd.c | 3 --- block/partition-generic.c | 3 --- include/linux/genhd.h | 3 +-- 6 files changed, 26 insertions(+), 70 deletions(-) diff --git a/block/bio.c b/block/bio.c index 91e398ba57f1..0c2208a5446d 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1663,6 +1663,22 @@ void bio_check_pages_dirty(struct bio *bio) } EXPORT_SYMBOL_GPL(bio_check_pages_dirty); +void update_io_ticks(struct hd_struct *part, unsigned long now) +{ + unsigned long stamp; +again: + stamp = READ_ONCE(part->stamp); + if (unlikely(stamp != now)) { + if (likely(cmpxchg(&part->stamp, stamp, now) == stamp)) { + __part_stat_add(part, io_ticks, 1); + } + } + if (part->partno) { + part = &part_to_disk(part)->part0; + goto again; + } +} + void generic_start_io_acct(struct request_queue *q, int op, unsigned long sectors, struct hd_struct *part) { @@ -1670,7 +1686,7 @@ void generic_start_io_acct(struct request_queue *q, int op, part_stat_lock(); - part_round_stats(q, part); + update_io_ticks(part, jiffies); part_stat_inc(part, ios[sgrp]); part_stat_add(part, sectors[sgrp], sectors); part_inc_in_flight(q, part, op_is_write(op)); @@ -1682,13 +1698,15 @@ EXPORT_SYMBOL(generic_start_io_acct); void generic_end_io_acct(struct request_queue *q, int req_op, struct hd_struct *part, unsigned long start_time) { - unsigned long duration = jiffies - start_time; + unsigned long now = jiffies; + unsigned long duration = now - start_time; const int sgrp = op_stat_group(req_op); part_stat_lock(); + update_io_ticks(part, now); part_stat_add(part, nsecs[sgrp], jiffies_to_nsecs(duration)); - part_round_stats(q, part); + part_stat_add(part, time_in_queue, duration); part_dec_in_flight(q, part, op_is_write(req_op)); part_stat_unlock(); diff --git a/block/blk-core.c b/block/blk-core.c index 734b768c9d9d..268d2b8e9843 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -584,62 +584,6 @@ struct request *blk_get_request(struct request_queue *q, unsigned int op, } EXPORT_SYMBOL(blk_get_request); -static void part_round_stats_single(struct request_queue *q, - struct hd_struct *part, unsigned long now, - unsigned int inflight) -{ - if (inflight) { - __part_stat_add(part, time_in_queue, - inflight * (now - part->stamp)); - __part_stat_add(part, io_ticks, (now - part->stamp)); - } - part->stamp = now; -} - -/** - * part_round_stats() - Round off the performance stats on a struct disk_stats. - * @q: target block queue - * @part: target partition - * - * The average IO queue length and utilisation statistics are maintained - * by observing the current state of the queue length and the amount of - * time it has been in this state for. - * - * Normally, that accounting is done on IO completion, but that can result - * in more than a second's worth of IO being accounted for within any one - * second, leading to >100% utilisation. To deal with that, we call this - * function to do a round-off before returning the results when reading - * /proc/diskstats. This accounts immediately for all queue usage up to - * the current jiffies and restarts the counters again. - */ -void part_round_stats(struct request_queue *q, struct hd_struct *part) -{ - struct hd_struct *part2 = NULL; - unsigned long now = jiffies; - unsigned int inflight[2]; - int stats = 0; - - if (part->stamp != now) - stats |= 1; - - if (part->partno) { - part2 = &part_to_disk(part)->part0; - if (part2->stamp != now) - stats |= 2; - } - - if (!stats) - return; - - part_in_flight(q, part, inflight); - - if (stats & 2) - part_round_stats_single(q, part2, now, inflight[1]); - if (stats & 1) - part_round_stats_single(q, part, now, inflight[0]); -} -EXPORT_SYMBOL_GPL(part_round_stats); - void blk_put_request(struct request *req) { blk_mq_free_request(req); @@ -1383,9 +1327,10 @@ void blk_account_io_done(struct request *req, u64 now) part_stat_lock(); part = req->part; + update_io_ticks(part, jiffies); part_stat_inc(part, ios[sgrp]); part_stat_add(part, nsecs[sgrp], now - req->start_time_ns); - part_round_stats(req->q, part); + part_stat_add(part, time_in_queue, nsecs_to_jiffies64(now - req->start_time_ns)); part_dec_in_flight(req->q, part, rq_data_dir(req)); hd_struct_put(part); @@ -1420,11 +1365,12 @@ void blk_account_io_start(struct request *rq, bool new_io) part = &rq->rq_disk->part0; hd_struct_get(part); } - part_round_stats(rq->q, part); part_inc_in_flight(rq->q, part, rw); rq->part = part; } + update_io_ticks(part, jiffies); + part_stat_unlock(); } diff --git a/block/blk-merge.c b/block/blk-merge.c index a120d59b9705..9da5629d0887 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -689,7 +689,6 @@ static void blk_account_io_merge(struct request *req) part_stat_lock(); part = req->part; - part_round_stats(req->q, part); part_dec_in_flight(req->q, part, rq_data_dir(req)); hd_struct_put(part); diff --git a/block/genhd.c b/block/genhd.c index 2fe00cf32b93..cdf174d7d329 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1337,9 +1337,6 @@ static int diskstats_show(struct seq_file *seqf, void *v) disk_part_iter_init(&piter, gp, DISK_PITER_INCL_EMPTY_PART0); while ((hd = disk_part_iter_next(&piter))) { - part_stat_lock(); - part_round_stats(gp->queue, hd); - part_stat_unlock(); part_in_flight(gp->queue, hd, inflight); seq_printf(seqf, "%4d %7d %s " "%lu %lu %lu %u " diff --git a/block/partition-generic.c b/block/partition-generic.c index 7e663cfb1487..42d6138ac876 100644 --- a/block/partition-generic.c +++ b/block/partition-generic.c @@ -122,9 +122,6 @@ ssize_t part_stat_show(struct device *dev, struct request_queue *q = part_to_disk(p)->queue; unsigned int inflight[2]; - part_stat_lock(); - part_round_stats(q, p); - part_stat_unlock(); part_in_flight(q, p, inflight); return sprintf(buf, "%8lu %8lu %8llu %8u " diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 1677cd2a4c4e..838c2a7a40c5 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -398,8 +398,7 @@ static inline void free_part_info(struct hd_struct *part) kfree(part->info); } -/* block/blk-core.c */ -extern void part_round_stats(struct request_queue *q, struct hd_struct *part); +void update_io_ticks(struct hd_struct *part, unsigned long now); /* block/genhd.c */ extern void device_add_disk(struct device *parent, struct gendisk *disk, From patchwork Thu Dec 6 16:41:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716349 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 02B5217DB for ; Thu, 6 Dec 2018 16:42:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E34522F13B for ; Thu, 6 Dec 2018 16:42:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E1B292F14C; Thu, 6 Dec 2018 16:42:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 63CC52F13B for ; Thu, 6 Dec 2018 16:42:21 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 65C4C307D982; Thu, 6 Dec 2018 16:42:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 345335D96E; Thu, 6 Dec 2018 16:42:20 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 023AE181B9EB; Thu, 6 Dec 2018 16:42:20 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6Gfel2030166 for ; Thu, 6 Dec 2018 11:41:40 -0500 Received: by smtp.corp.redhat.com (Postfix) id BCDED608C4; Thu, 6 Dec 2018 16:41:40 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx16.extmail.prod.ext.phx2.redhat.com [10.5.110.45]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AF0A918254; Thu, 6 Dec 2018 16:41:38 +0000 (UTC) Received: from mail-qk1-f193.google.com (mail-qk1-f193.google.com [209.85.222.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 600F230014CA; Thu, 6 Dec 2018 16:41:37 +0000 (UTC) Received: by mail-qk1-f193.google.com with SMTP id o125so694751qkf.3; Thu, 06 Dec 2018 08:41:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=whhiNG3tXZdK2IE35hgGnKIrXnWcdYSeBEeAqJquoA8=; b=TddTX+tdpNJOc6uYUE/SpPHjNGz71Uo0ch0Qz8/jqnFH6DODUqJ/rVtBj3MW2mqgLW s6P6IOQXN77a8oq8TqT4e0oFxMWDHGy3r/SvDJW7Ko0Rb2T3mMpTvwqAjIhgeavRssQY Hy2rGN3fYWzoJHbdqT8aq+u8UTGSRJLCTDkmJ89zBOEAlz1pGAlTCzmz/pVfvyc1ro6Q ZkiKZTLzSoowLgfH0xV6msFCRmc0KI3Sf04cvoepLyXeU+NvtGP4YbKzc5bUfHyUmc3B zphFCSRiR7y4gAmDKCCCwWbgUTtN9r3Djr2fsxXgMk1RHwvg4tERBBVn9icPaqJDs0JG gf9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=whhiNG3tXZdK2IE35hgGnKIrXnWcdYSeBEeAqJquoA8=; b=T1Hi0PdR+bjxt+n6PAI9X2BiTBnejlMrffWXVqJtrq0JFEkqFs4RVcZD/7Tla1IX0t HE+iQchUCga2hJgRaqVzRFbCwyzG/J8n8PPzFLU9Nn9Zm6GFBaetoK6C8HXWkx1YZHu3 IdZNw71MOQabcyhcCPmk+iZ99yyXrcdlD5y+wB5G4yEXoT7/5yEl/H0BOmsc1ZX1UY2b NMTgyxnT+1QxeIho60qZDgNLFAThx8L8jUnuI/U06j8lt3q5FKMrO6R+zVqpJyIkzmGM GUi/14z0JiC+Xjgd1s4GiRg2NvUgMcDazH+wYESefOLGteW049s2Z0DeTJ10naoY28DF HOmw== X-Gm-Message-State: AA+aEWY2f8wpOodFCyt3lQCBt2PlAZzSw+JvivRiTc2lp2lPpJI/qV4L zie4Yavm/W23wGcFPCvH7YM= X-Google-Smtp-Source: AFSGD/XLArqca/cAf/AAegjj3fhHG0wMQaLU6cfpCVH++fyfnFNDf3M2KY3J27iTuqn4CrOCJ3NEFw== X-Received: by 2002:a37:688c:: with SMTP id d134mr27297745qkc.57.1544114496426; Thu, 06 Dec 2018 08:41:36 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id u50sm903683qta.23.2018.12.06.08.41.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:35 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:20 -0500 Message-Id: <20181206164122.2166-6-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 06 Dec 2018 16:41:37 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Thu, 06 Dec 2018 16:41:37 +0000 (UTC) for IP:'209.85.222.193' DOMAIN:'mail-qk1-f193.google.com' HELO:'mail-qk1-f193.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.091 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_PASS) 209.85.222.193 mail-qk1-f193.google.com 209.85.222.193 mail-qk1-f193.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.84 on 10.5.110.45 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 5/7] block: switch to per-cpu in-flight counters X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Thu, 06 Dec 2018 16:42:21 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP From: Mikulas Patocka Now when part_round_stats is gone, we can switch to per-cpu in-flight counters. We use the local-atomic type local_t, so that if part_inc_in_flight or part_dec_in_flight is reentrantly called from an interrupt, the value will be correct. The other counters could be corrupted due to reentrant interrupt, but the corruption only results in slight counter skew - the in_flight counter must be exact, so it needs local_t. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/genhd.c | 43 +++++++++++++++++++++++++++++++++---------- include/linux/genhd.h | 29 ++++++++++++++++++++++------- 2 files changed, 55 insertions(+), 17 deletions(-) diff --git a/block/genhd.c b/block/genhd.c index cdf174d7d329..9827a2c05db7 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -50,9 +50,9 @@ void part_inc_in_flight(struct request_queue *q, struct hd_struct *part, int rw) if (queue_is_mq(q)) return; - atomic_inc(&part->in_flight[rw]); + part_stat_local_inc(part, in_flight[rw]); if (part->partno) - atomic_inc(&part_to_disk(part)->part0.in_flight[rw]); + part_stat_local_inc(&part_to_disk(part)->part0, in_flight[rw]); } void part_dec_in_flight(struct request_queue *q, struct hd_struct *part, int rw) @@ -60,38 +60,61 @@ void part_dec_in_flight(struct request_queue *q, struct hd_struct *part, int rw) if (queue_is_mq(q)) return; - atomic_dec(&part->in_flight[rw]); + part_stat_local_dec(part, in_flight[rw]); if (part->partno) - atomic_dec(&part_to_disk(part)->part0.in_flight[rw]); + part_stat_local_dec(&part_to_disk(part)->part0, in_flight[rw]); } void part_in_flight(struct request_queue *q, struct hd_struct *part, unsigned int inflight[2]) { + int cpu; + if (queue_is_mq(q)) { blk_mq_in_flight(q, part, inflight); return; } - inflight[0] = atomic_read(&part->in_flight[0]) + - atomic_read(&part->in_flight[1]); + inflight[0] = 0; + for_each_possible_cpu(cpu) { + inflight[0] += part_stat_local_read_cpu(part, in_flight[0], cpu) + + part_stat_local_read_cpu(part, in_flight[1], cpu); + } + if ((int)inflight[0] < 0) + inflight[0] = 0; + if (part->partno) { part = &part_to_disk(part)->part0; - inflight[1] = atomic_read(&part->in_flight[0]) + - atomic_read(&part->in_flight[1]); + inflight[1] = 0; + for_each_possible_cpu(cpu) { + inflight[1] += part_stat_local_read_cpu(part, in_flight[0], cpu) + + part_stat_local_read_cpu(part, in_flight[1], cpu); + } + if ((int)inflight[1] < 0) + inflight[1] = 0; } } void part_in_flight_rw(struct request_queue *q, struct hd_struct *part, unsigned int inflight[2]) { + int cpu; + if (queue_is_mq(q)) { blk_mq_in_flight_rw(q, part, inflight); return; } - inflight[0] = atomic_read(&part->in_flight[0]); - inflight[1] = atomic_read(&part->in_flight[1]); + inflight[0] = 0; + inflight[1] = 0; + for_each_possible_cpu(cpu) { + inflight[0] += part_stat_local_read_cpu(part, in_flight[0], cpu); + inflight[1] += part_stat_local_read_cpu(part, in_flight[1], cpu); + } + if ((int)inflight[0] < 0) + inflight[0] = 0; + if ((int)inflight[1] < 0) + inflight[1] = 0; } struct hd_struct *__disk_get_part(struct gendisk *disk, int partno) diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 838c2a7a40c5..636b4f687e35 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -17,6 +17,7 @@ #include #include #include +#include #ifdef CONFIG_BLOCK @@ -89,6 +90,7 @@ struct disk_stats { unsigned long merges[NR_STAT_GROUPS]; unsigned long io_ticks; unsigned long time_in_queue; + local_t in_flight[2]; }; #define PARTITION_META_INFO_VOLNAMELTH 64 @@ -122,7 +124,6 @@ struct hd_struct { int make_it_fail; #endif unsigned long stamp; - atomic_t in_flight[2]; #ifdef CONFIG_SMP struct disk_stats __percpu *dkstats; #else @@ -295,8 +296,11 @@ extern struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, #define part_stat_lock() ({ rcu_read_lock(); get_cpu(); }) #define part_stat_unlock() do { put_cpu(); rcu_read_unlock(); } while (0) -#define __part_stat_add(part, field, addnd) \ - (per_cpu_ptr((part)->dkstats, smp_processor_id())->field += (addnd)) +#define part_stat_get_cpu(part, field, cpu) \ + (per_cpu_ptr((part)->dkstats, (cpu))->field) + +#define part_stat_get(part, field) \ + part_stat_get_cpu(part, field, smp_processor_id()) #define part_stat_read(part, field) \ ({ \ @@ -333,10 +337,9 @@ static inline void free_part_stats(struct hd_struct *part) #define part_stat_lock() ({ rcu_read_lock(); 0; }) #define part_stat_unlock() rcu_read_unlock() -#define __part_stat_add(part, field, addnd) \ - ((part)->dkstats.field += addnd) - -#define part_stat_read(part, field) ((part)->dkstats.field) +#define part_stat_get(part, field) ((part)->dkstats.field) +#define part_stat_get_cpu(part, field, cpu) part_stat_get(part, field) +#define part_stat_read(part, field) part_stat_get(part, field) static inline void part_stat_set_all(struct hd_struct *part, int value) { @@ -362,6 +365,9 @@ static inline void free_part_stats(struct hd_struct *part) part_stat_read(part, field[STAT_WRITE]) + \ part_stat_read(part, field[STAT_DISCARD])) +#define __part_stat_add(part, field, addnd) \ + (part_stat_get(part, field) += (addnd)) + #define part_stat_add(part, field, addnd) do { \ __part_stat_add((part), field, addnd); \ if ((part)->partno) \ @@ -376,6 +382,15 @@ static inline void free_part_stats(struct hd_struct *part) #define part_stat_sub(gendiskp, field, subnd) \ part_stat_add(gendiskp, field, -subnd) +#define part_stat_local_dec(gendiskp, field) \ + local_dec(&(part_stat_get(gendiskp, field))) +#define part_stat_local_inc(gendiskp, field) \ + local_inc(&(part_stat_get(gendiskp, field))) +#define part_stat_local_read(gendiskp, field) \ + local_read(&(part_stat_get(gendiskp, field))) +#define part_stat_local_read_cpu(gendiskp, field, cpu) \ + local_read(&(part_stat_get_cpu(gendiskp, field, cpu))) + void part_in_flight(struct request_queue *q, struct hd_struct *part, unsigned int inflight[2]); void part_in_flight_rw(struct request_queue *q, struct hd_struct *part, From patchwork Thu Dec 6 16:41:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716345 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACA9914E2 for ; Thu, 6 Dec 2018 16:42:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9975B2F0D5 for ; Thu, 6 Dec 2018 16:42:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8E1682F145; Thu, 6 Dec 2018 16:42:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EBB462F130 for ; Thu, 6 Dec 2018 16:42:08 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DBB2430B8F97; Thu, 6 Dec 2018 16:42:07 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AE2FB662D1; Thu, 6 Dec 2018 16:42:07 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 76FA3181B9EA; Thu, 6 Dec 2018 16:42:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6GfgPN030184 for ; Thu, 6 Dec 2018 11:41:42 -0500 Received: by smtp.corp.redhat.com (Postfix) id AC22F5D96E; Thu, 6 Dec 2018 16:41:42 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx09.extmail.prod.ext.phx2.redhat.com [10.5.110.38]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9F5805D9D6; Thu, 6 Dec 2018 16:41:40 +0000 (UTC) Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 52F6DDC910; Thu, 6 Dec 2018 16:41:39 +0000 (UTC) Received: by mail-qt1-f196.google.com with SMTP id r14so1246140qtp.1; Thu, 06 Dec 2018 08:41:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=XfUbBH6Cr+GEY0pE8uAPo/0HeFQFgkb69YK480+bzXg=; b=Y2FIg8DEwly6TEvFuo/G76K96SJhY4FdFHHQKRvDlmqJ85zYeciMiqmUCJMMYOPvi8 2mVEql1C1m0i4rRzckcdAUtx2PzAtiuIsH1Ybab76heVaCgBSWYA5H4BLNbxrfgEE1Hu KUFAgFon1OTfVsmA/zFK6uA+c1l9J1F+DERCwQAsyUcouozudUMY5DUdn2YGKValmzJK 191VHqNKTdO7AGSn7Xzbs+R5ujYHhGfXU6VKNQheWue0KnRytB4X9v1Z8qozeBo84NP5 Lf1yWFAG4ypSKnzt1wUwqXmx52nqcvigQ6qXGBLjIOzPd6w0B07FSnSSVxVxCLOnjS6i Zu5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=XfUbBH6Cr+GEY0pE8uAPo/0HeFQFgkb69YK480+bzXg=; b=pVmKVjiiZDYGreVLEWhwHy7bvCPmUIC6xasXMgp2xI6F8aoQ+9yMHkArKbdHL1PDjW /vrOoohPAW3axpk8nEH557xU1ISRb717kPZ+Igw0sdl1Fos/hkC7xzUaUBYjZqq60flH /IZBtIqyUOFtQzBSaP4LH2JA+FTgp9tzDx/v+qa1ZpQ5UzGUoUrLo380ZMFYeqewo+2n rj0mxk11O7Xgs8O80daXQkDZLkaiRDqtiQ5qils3M670B8Si1mp1m02zVJ4ji0hNKjHz +mPqFMp2Xuu3/X6Ort1CBkGkZKWP04MM6MxW1ODvWZ8iFvRPmKFG4Rjc9d5TszWNDNaJ 8a+A== X-Gm-Message-State: AA+aEWZR4yeBt5Js3mioE+I/oqp6u1gH9mlZWik4orzyxq0bGMsMX5zw QcmShkskxhG7449NZQ6tGsQY2jAw X-Google-Smtp-Source: AFSGD/VQdyMZQbgHEZF5otRR3yNwFAIHNqn+ZAYgHVUUqyqJEIz7GelrXEimvj1Bbb281/16S/mbKg== X-Received: by 2002:ac8:354e:: with SMTP id z14mr28336024qtb.272.1544114498433; Thu, 06 Dec 2018 08:41:38 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id d193sm701279qka.91.2018.12.06.08.41.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:37 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:21 -0500 Message-Id: <20181206164122.2166-7-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 06 Dec 2018 16:41:39 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 06 Dec 2018 16:41:39 +0000 (UTC) for IP:'209.85.160.196' DOMAIN:'mail-qt1-f196.google.com' HELO:'mail-qt1-f196.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.072 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS) 209.85.160.196 mail-qt1-f196.google.com 209.85.160.196 mail-qt1-f196.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.78 on 10.5.110.38 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 6/7] block: return just one value from part_in_flight X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Thu, 06 Dec 2018 16:42:08 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP From: Mikulas Patocka The previous patches deleted all the code that needed the second value returned from part_in_flight - now the kernel only uses the first value. Consequently, part_in_flight (and blk_mq_in_flight) may be changed so that it only returns one value. This patch just refactors the code, there's no functional change. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- block/blk-mq.c | 12 +++++------- block/blk-mq.h | 3 +-- block/genhd.c | 34 ++++++++++++---------------------- block/partition-generic.c | 6 +++--- include/linux/genhd.h | 3 +-- 5 files changed, 22 insertions(+), 36 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 900550594651..c6d3101352f4 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -100,25 +100,23 @@ static bool blk_mq_check_inflight(struct blk_mq_hw_ctx *hctx, struct mq_inflight *mi = priv; /* - * index[0] counts the specific partition that was asked for. index[1] - * counts the ones that are active on the whole device, so increment - * that if mi->part is indeed a partition, and not a whole device. + * index[0] counts the specific partition that was asked for. */ if (rq->part == mi->part) mi->inflight[0]++; - if (mi->part->partno) - mi->inflight[1]++; return true; } -void blk_mq_in_flight(struct request_queue *q, struct hd_struct *part, - unsigned int inflight[2]) +unsigned int blk_mq_in_flight(struct request_queue *q, struct hd_struct *part) { + unsigned inflight[2]; struct mq_inflight mi = { .part = part, .inflight = inflight, }; inflight[0] = inflight[1] = 0; blk_mq_queue_tag_busy_iter(q, blk_mq_check_inflight, &mi); + + return inflight[0]; } static bool blk_mq_check_inflight_rw(struct blk_mq_hw_ctx *hctx, diff --git a/block/blk-mq.h b/block/blk-mq.h index a664ea44ffd4..0c9c9ea2fefe 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -187,8 +187,7 @@ static inline bool blk_mq_hw_queue_mapped(struct blk_mq_hw_ctx *hctx) return hctx->nr_ctx && hctx->tags; } -void blk_mq_in_flight(struct request_queue *q, struct hd_struct *part, - unsigned int inflight[2]); +unsigned int blk_mq_in_flight(struct request_queue *q, struct hd_struct *part); void blk_mq_in_flight_rw(struct request_queue *q, struct hd_struct *part, unsigned int inflight[2]); diff --git a/block/genhd.c b/block/genhd.c index 9827a2c05db7..1dd8fd6613b8 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -65,34 +65,24 @@ void part_dec_in_flight(struct request_queue *q, struct hd_struct *part, int rw) part_stat_local_dec(&part_to_disk(part)->part0, in_flight[rw]); } -void part_in_flight(struct request_queue *q, struct hd_struct *part, - unsigned int inflight[2]) +unsigned int part_in_flight(struct request_queue *q, struct hd_struct *part) { int cpu; + unsigned int inflight; if (queue_is_mq(q)) { - blk_mq_in_flight(q, part, inflight); - return; + return blk_mq_in_flight(q, part); } - inflight[0] = 0; + inflight = 0; for_each_possible_cpu(cpu) { - inflight[0] += part_stat_local_read_cpu(part, in_flight[0], cpu) + - part_stat_local_read_cpu(part, in_flight[1], cpu); + inflight += part_stat_local_read_cpu(part, in_flight[0], cpu) + + part_stat_local_read_cpu(part, in_flight[1], cpu); } - if ((int)inflight[0] < 0) - inflight[0] = 0; + if ((int)inflight < 0) + inflight = 0; - if (part->partno) { - part = &part_to_disk(part)->part0; - inflight[1] = 0; - for_each_possible_cpu(cpu) { - inflight[1] += part_stat_local_read_cpu(part, in_flight[0], cpu) + - part_stat_local_read_cpu(part, in_flight[1], cpu); - } - if ((int)inflight[1] < 0) - inflight[1] = 0; - } + return inflight; } void part_in_flight_rw(struct request_queue *q, struct hd_struct *part, @@ -1348,7 +1338,7 @@ static int diskstats_show(struct seq_file *seqf, void *v) struct disk_part_iter piter; struct hd_struct *hd; char buf[BDEVNAME_SIZE]; - unsigned int inflight[2]; + unsigned int inflight; /* if (&disk_to_dev(gp)->kobj.entry == block_class.devices.next) @@ -1360,7 +1350,7 @@ static int diskstats_show(struct seq_file *seqf, void *v) disk_part_iter_init(&piter, gp, DISK_PITER_INCL_EMPTY_PART0); while ((hd = disk_part_iter_next(&piter))) { - part_in_flight(gp->queue, hd, inflight); + inflight = part_in_flight(gp->queue, hd); seq_printf(seqf, "%4d %7d %s " "%lu %lu %lu %u " "%lu %lu %lu %u " @@ -1376,7 +1366,7 @@ static int diskstats_show(struct seq_file *seqf, void *v) part_stat_read(hd, merges[STAT_WRITE]), part_stat_read(hd, sectors[STAT_WRITE]), (unsigned int)part_stat_read_msecs(hd, STAT_WRITE), - inflight[0], + inflight, jiffies_to_msecs(part_stat_read(hd, io_ticks)), jiffies_to_msecs(part_stat_read(hd, time_in_queue)), part_stat_read(hd, ios[STAT_DISCARD]), diff --git a/block/partition-generic.c b/block/partition-generic.c index 42d6138ac876..8e596a8dff32 100644 --- a/block/partition-generic.c +++ b/block/partition-generic.c @@ -120,9 +120,9 @@ ssize_t part_stat_show(struct device *dev, { struct hd_struct *p = dev_to_part(dev); struct request_queue *q = part_to_disk(p)->queue; - unsigned int inflight[2]; + unsigned int inflight; - part_in_flight(q, p, inflight); + inflight = part_in_flight(q, p); return sprintf(buf, "%8lu %8lu %8llu %8u " "%8lu %8lu %8llu %8u " @@ -137,7 +137,7 @@ ssize_t part_stat_show(struct device *dev, part_stat_read(p, merges[STAT_WRITE]), (unsigned long long)part_stat_read(p, sectors[STAT_WRITE]), (unsigned int)part_stat_read_msecs(p, STAT_WRITE), - inflight[0], + inflight, jiffies_to_msecs(part_stat_read(p, io_ticks)), jiffies_to_msecs(part_stat_read(p, time_in_queue)), part_stat_read(p, ios[STAT_DISCARD]), diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 636b4f687e35..06c0fd594097 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -391,8 +391,7 @@ static inline void free_part_stats(struct hd_struct *part) #define part_stat_local_read_cpu(gendiskp, field, cpu) \ local_read(&(part_stat_get_cpu(gendiskp, field, cpu))) -void part_in_flight(struct request_queue *q, struct hd_struct *part, - unsigned int inflight[2]); +unsigned int part_in_flight(struct request_queue *q, struct hd_struct *part); void part_in_flight_rw(struct request_queue *q, struct hd_struct *part, unsigned int inflight[2]); void part_dec_in_flight(struct request_queue *q, struct hd_struct *part, From patchwork Thu Dec 6 16:41:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10716351 X-Patchwork-Delegate: snitzer@redhat.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C88717DB for ; Thu, 6 Dec 2018 16:42:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B8AF2F14E for ; Thu, 6 Dec 2018 16:42:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 602EF2F154; Thu, 6 Dec 2018 16:42:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id ED8192F14E for ; Thu, 6 Dec 2018 16:42:24 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1D6A5307D973; Thu, 6 Dec 2018 16:42:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EA1025D786; Thu, 6 Dec 2018 16:42:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id AA2993F7D0; Thu, 6 Dec 2018 16:42:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id wB6Gfg66030179 for ; Thu, 6 Dec 2018 11:41:42 -0500 Received: by smtp.corp.redhat.com (Postfix) id 6DE986A522; Thu, 6 Dec 2018 16:41:42 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx17.extmail.prod.ext.phx2.redhat.com [10.5.110.46]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 623745C730; Thu, 6 Dec 2018 16:41:42 +0000 (UTC) Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 33A5931256BF; Thu, 6 Dec 2018 16:41:41 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id i7so1192267qtj.10; Thu, 06 Dec 2018 08:41:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Lcg2zF1imfNyveimZfa7nNlz5mO8GsZUtILSOxC+mSg=; b=bXXWMjxtYPwSge+21FnBLq8KO5ZuuQnl4xQSczn6L5OjEEC8qcPX8nn0ETibu4DwIT ijqXOD9NpANlMbYeM0U7kT6c0zF+3BBNzb2t5R3qxoSChdDx+heSQObG11T/QWA0m1DW kSqCdocM4v2YeaswKVleIVd2w4Na7vcGCCbZjcI+ck1XxUeBPqruoBlJTYdfpqIj7tq3 FBd/PWF7Y5RhoByV7ioopLqnJAICSGx0LTgFA/sbZf88vqdMcdJx2QHntl+d3ld1DFzo vL6NsqvLGcQEKs9JbcY0FcJdI7kPrs/Nt7KWI3lbzrsZlBEQwMvaswwkghhjotCGdE5N LpyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Lcg2zF1imfNyveimZfa7nNlz5mO8GsZUtILSOxC+mSg=; b=ViVQTX+eVMJhJL0ITfsiWQJI0PtNbd0zCcmqpbr4ygr76LlyJvK6/3m2qttAeS8rnn NjzBfTp4e4KcT/lSnwaPX0YBMRFxA9wUm4iMv0LfDRE4ftkoxRDtWhnVhz29FipFV3jt 83PWPGqjiVDPHjO0557yyjx1v0jelhn6qMs2pTsP5686SZwkuSrQPiMKnlm5l+4H6Ode 0hx9l7AaFMWAaQtPqCvGlHrsbWkXw/7jcRTGts09Pkeaej+Tnmr8wAIlx52QRMkquuqD w9dsbHuE3tjzSGU3WIyoQk8mcPlbeN1tXuz67g6FLdhkkl9Dosq//D7QJGUYiiF2ufzM Q2nA== X-Gm-Message-State: AA+aEWb0riGFA30/Kyjrwe+8Zi09317XW17ML8cw6KF2w5qm70PGyZLP e0XaYWIOUeRl4bpY1uMN5pE+fW1v X-Google-Smtp-Source: AFSGD/WwTggLZMmPm8zEhSJJVdCoUyGlt5qRP5wQbg2YwXwbLz8oLBORAbWBA+NfsVGNU/w9Xu/glg== X-Received: by 2002:a0c:96b5:: with SMTP id a50mr28051792qvd.33.1544114500336; Thu, 06 Dec 2018 08:41:40 -0800 (PST) Received: from localhost (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id f201sm373039qke.0.2018.12.06.08.41.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 08:41:39 -0800 (PST) From: Mike Snitzer To: Jens Axboe Date: Thu, 6 Dec 2018 11:41:22 -0500 Message-Id: <20181206164122.2166-8-snitzer@redhat.com> In-Reply-To: <20181206164122.2166-1-snitzer@redhat.com> References: <20181206164122.2166-1-snitzer@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 06 Dec 2018 16:41:41 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 06 Dec 2018 16:41:41 +0000 (UTC) for IP:'209.85.160.193' DOMAIN:'mail-qt1-f193.google.com' HELO:'mail-qt1-f193.google.com' FROM:'snitzer@gmail.com' RCPT:'' X-RedHat-Spam-Score: 0.072 (DKIM_SIGNED, DKIM_VALID, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_PASS) 209.85.160.193 mail-qt1-f193.google.com 209.85.160.193 mail-qt1-f193.google.com X-RedHat-Possible-Forgery: Mike Snitzer X-Scanned-By: MIMEDefang 2.84 on 10.5.110.46 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-loop: dm-devel@redhat.com Cc: linux-block@vger.kernel.org, dm-devel@redhat.com, Mikulas Patocka Subject: [dm-devel] [PATCH v4 7/7] dm: remove the pending IO accounting X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Thu, 06 Dec 2018 16:42:23 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP From: Mikulas Patocka Remove the "pending" atomic counters, that duplicate block-core's in_flight counters, and update md_in_flight() to look at percpu in_flight counters. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 -- drivers/md/dm.c | 34 +++++++++++++++------------------- 2 files changed, 15 insertions(+), 21 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index 224d44503a06..6fe883fac471 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -65,7 +65,6 @@ struct mapped_device { */ struct work_struct work; wait_queue_head_t wait; - atomic_t pending[2]; spinlock_t deferred_lock; struct bio_list deferred; @@ -119,7 +118,6 @@ struct mapped_device { struct srcu_struct io_barrier; }; -int md_in_flight(struct mapped_device *md); void disable_write_same(struct mapped_device *md); void disable_write_zeroes(struct mapped_device *md); diff --git a/drivers/md/dm.c b/drivers/md/dm.c index a8ae7931bce7..d4bc4237b09a 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -646,25 +646,30 @@ static void free_tio(struct dm_target_io *tio) bio_put(&tio->clone); } -int md_in_flight(struct mapped_device *md) +static bool md_in_flight(struct mapped_device *md) { - return atomic_read(&md->pending[READ]) + - atomic_read(&md->pending[WRITE]); + int cpu; + struct hd_struct *part = &dm_disk(md)->part0; + + for_each_possible_cpu(cpu) { + if (part_stat_local_read_cpu(part, in_flight[0], cpu) || + part_stat_local_read_cpu(part, in_flight[1], cpu)) + return true; + } + + return false; } static void start_io_acct(struct dm_io *io) { struct mapped_device *md = io->md; struct bio *bio = io->orig_bio; - int rw = bio_data_dir(bio); io->start_time = jiffies; generic_start_io_acct(md->queue, bio_op(bio), bio_sectors(bio), &dm_disk(md)->part0); - atomic_inc(&md->pending[rw]); - if (unlikely(dm_stats_used(&md->stats))) dm_stats_account_io(&md->stats, bio_data_dir(bio), bio->bi_iter.bi_sector, bio_sectors(bio), @@ -676,8 +681,6 @@ static void end_io_acct(struct dm_io *io) struct mapped_device *md = io->md; struct bio *bio = io->orig_bio; unsigned long duration = jiffies - io->start_time; - int pending; - int rw = bio_data_dir(bio); generic_end_io_acct(md->queue, bio_op(bio), &dm_disk(md)->part0, io->start_time); @@ -687,16 +690,11 @@ static void end_io_acct(struct dm_io *io) bio->bi_iter.bi_sector, bio_sectors(bio), true, duration, &io->stats_aux); - /* - * After this is decremented the bio must not be touched if it is - * a flush. - */ - pending = atomic_dec_return(&md->pending[rw]); - pending += atomic_read(&md->pending[rw^0x1]); - /* nudge anyone waiting on suspend queue */ - if (!pending) - wake_up(&md->wait); + if (unlikely(waitqueue_active(&md->wait))) { + if (!md_in_flight(md)) + wake_up(&md->wait); + } } /* @@ -1904,8 +1902,6 @@ static struct mapped_device *alloc_dev(int minor) if (!md->disk) goto bad; - atomic_set(&md->pending[0], 0); - atomic_set(&md->pending[1], 0); init_waitqueue_head(&md->wait); INIT_WORK(&md->work, dm_wq_work); init_waitqueue_head(&md->eventq);