From patchwork Wed Jun 17 23:55:51 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 6631621 Return-Path: X-Original-To: patchwork-linux-nvdimm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id B1FDD9F1C1 for ; Wed, 17 Jun 2015 23:58:36 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id AAABF207E7 for ; Wed, 17 Jun 2015 23:58:34 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 883492066D for ; Wed, 17 Jun 2015 23:58:33 +0000 (UTC) Received: from ml01.vlan14.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7B17C1826AC; Wed, 17 Jun 2015 16:58:33 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ml01.01.org (Postfix) with ESMTP id 9BF6A182686 for ; Wed, 17 Jun 2015 16:58:32 -0700 (PDT) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP; 17 Jun 2015 16:58:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,636,1427785200"; d="scan'208";a="745483927" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.23.232.36]) by fmsmga002.fm.intel.com with ESMTP; 17 Jun 2015 16:58:32 -0700 Subject: [PATCH 12/15] libnvdimm: enable iostat From: Dan Williams To: axboe@kernel.dk, linux-nvdimm@lists.01.org Date: Wed, 17 Jun 2015 19:55:51 -0400 Message-ID: <20150617235551.12943.76656.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20150617235209.12943.24419.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20150617235209.12943.24419.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-8-g92dd MIME-Version: 1.0 Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, linux-acpi@vger.kernel.org, linux-fsdevel@vger.kernel.org, hch@lst.de X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is disabled by default as the overhead is prohibitive, but if the user takes the action to turn it on we'll oblige. Reviewed-by: Vishal Verma Signed-off-by: Dan Williams --- drivers/nvdimm/Kconfig | 14 ++++++++++++++ drivers/nvdimm/blk.c | 7 ++++++- drivers/nvdimm/btt.c | 7 ++++++- drivers/nvdimm/core.c | 31 +++++++++++++++++++++++++++++++ drivers/nvdimm/nd.h | 13 +++++++++++++ drivers/nvdimm/pmem.c | 5 +++++ 6 files changed, 75 insertions(+), 2 deletions(-) diff --git a/drivers/nvdimm/Kconfig b/drivers/nvdimm/Kconfig index 912cb36b8435..9d72085a67c9 100644 --- a/drivers/nvdimm/Kconfig +++ b/drivers/nvdimm/Kconfig @@ -73,4 +73,18 @@ config ND_MAX_REGIONS Leave the default of 64 if you are unsure. +config ND_IOSTAT + bool "Enable iostat by default" + default n + ---help--- + Persistent memory i/o has very low latency to the point + where the overhead to measure statistics can dramatically + impact the relative performance of the driver. Say y here + to trade off performance for statistics gathering that is + enabled by default. These statistics can always be + enabled/disabled at run time via the 'iostat' attribute of + the block device's queue in sysfs. + + If unsure, say N + endif diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c index 9d609ef95266..8a65e5a500d8 100644 --- a/drivers/nvdimm/blk.c +++ b/drivers/nvdimm/blk.c @@ -168,8 +168,10 @@ static void nd_blk_make_request(struct request_queue *q, struct bio *bio) struct bio_integrity_payload *bip; struct nd_blk_device *blk_dev; struct bvec_iter iter; + unsigned long start; struct bio_vec bvec; int err = 0, rw; + bool do_acct; if (unlikely(bio_end_sector(bio) > get_capacity(disk))) { err = -EIO; @@ -191,6 +193,7 @@ static void nd_blk_make_request(struct request_queue *q, struct bio *bio) blk_dev = disk->private_data; rw = bio_data_dir(bio); + do_acct = nd_iostat_start(bio, &start); bio_for_each_segment(bvec, bio, iter) { unsigned int len = bvec.bv_len; @@ -202,9 +205,11 @@ static void nd_blk_make_request(struct request_queue *q, struct bio *bio) "io error in %s sector %lld, len %d,\n", (rw == READ) ? "READ" : "WRITE", (unsigned long long) iter.bi_sector, len); - goto out; + break; } } + if (do_acct) + nd_iostat_end(bio, start); out: bio_endio(bio, err); diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c index 83b798dd2e68..67484633c322 100644 --- a/drivers/nvdimm/btt.c +++ b/drivers/nvdimm/btt.c @@ -1178,8 +1178,10 @@ static void btt_make_request(struct request_queue *q, struct bio *bio) struct block_device *bdev = bio->bi_bdev; struct btt *btt = q->queuedata; struct bvec_iter iter; + unsigned long start; struct bio_vec bvec; int err = 0, rw; + bool do_acct; if (bio_end_sector(bio) > get_capacity(bdev->bd_disk)) { err = -EIO; @@ -1199,6 +1201,7 @@ static void btt_make_request(struct request_queue *q, struct bio *bio) goto out; } + do_acct = nd_iostat_start(bio, &start); bio_for_each_segment(bvec, bio, iter) { unsigned int len = bvec.bv_len; @@ -1215,9 +1218,11 @@ static void btt_make_request(struct request_queue *q, struct bio *bio) "io error in %s sector %lld, len %d,\n", (rw == READ) ? "READ" : "WRITE", (unsigned long long) iter.bi_sector, len); - goto out; + break; } } + if (do_acct) + nd_iostat_end(bio, start); out: bio_endio(bio, err); diff --git a/drivers/nvdimm/core.c b/drivers/nvdimm/core.c index d27b13357873..99cf95af5f24 100644 --- a/drivers/nvdimm/core.c +++ b/drivers/nvdimm/core.c @@ -218,9 +218,40 @@ void nd_blk_queue_init(struct request_queue *q) { blk_queue_max_hw_sectors(q, UINT_MAX); blk_queue_bounce_limit(q, BLK_BOUNCE_ANY); + if (IS_ENABLED(CONFIG_ND_IOSTAT)) + queue_flag_set_unlocked(QUEUE_FLAG_IO_STAT, q); } EXPORT_SYMBOL(nd_blk_queue_init); +void __nd_iostat_start(struct bio *bio, unsigned long *start) +{ + struct gendisk *disk = bio->bi_bdev->bd_disk; + const int rw = bio_data_dir(bio); + int cpu = part_stat_lock(); + + *start = jiffies; + part_round_stats(cpu, &disk->part0); + part_stat_inc(cpu, &disk->part0, ios[rw]); + part_stat_add(cpu, &disk->part0, sectors[rw], bio_sectors(bio)); + part_inc_in_flight(&disk->part0, rw); + part_stat_unlock(); +} +EXPORT_SYMBOL(__nd_iostat_start); + +void nd_iostat_end(struct bio *bio, unsigned long start) +{ + struct gendisk *disk = bio->bi_bdev->bd_disk; + unsigned long duration = jiffies - start; + const int rw = bio_data_dir(bio); + int cpu = part_stat_lock(); + + part_stat_add(cpu, &disk->part0, ticks[rw], duration); + part_round_stats(cpu, &disk->part0); + part_dec_in_flight(&disk->part0, rw); + part_stat_unlock(); +} +EXPORT_SYMBOL(nd_iostat_end); + static ssize_t commands_show(struct device *dev, struct device_attribute *attr, char *buf) { diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h index 2f20d5dca028..3c4c8b6c64ec 100644 --- a/drivers/nvdimm/nd.h +++ b/drivers/nvdimm/nd.h @@ -13,6 +13,7 @@ #ifndef __ND_H__ #define __ND_H__ #include +#include #include #include #include @@ -172,5 +173,17 @@ struct resource *nvdimm_allocate_dpa(struct nvdimm_drvdata *ndd, resource_size_t n); int nd_blk_region_init(struct nd_region *nd_region); void nd_blk_queue_init(struct request_queue *q); +void __nd_iostat_start(struct bio *bio, unsigned long *start); +static inline bool nd_iostat_start(struct bio *bio, unsigned long *start) +{ + struct gendisk *disk = bio->bi_bdev->bd_disk; + + if (!blk_queue_io_stat(disk->queue)) + return false; + + __nd_iostat_start(bio, start); + return true; +} +void nd_iostat_end(struct bio *bio, unsigned long start); resource_size_t nd_namespace_blk_validate(struct nd_namespace_blk *nsblk); #endif /* __ND_H__ */ diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 0337b00f5409..3fd854a78f09 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -59,6 +59,8 @@ static void pmem_do_bvec(struct pmem_device *pmem, struct page *page, static void pmem_make_request(struct request_queue *q, struct bio *bio) { int err = 0; + bool do_acct; + unsigned long start; struct bio_vec bvec; struct bvec_iter iter; struct block_device *bdev = bio->bi_bdev; @@ -69,9 +71,12 @@ static void pmem_make_request(struct request_queue *q, struct bio *bio) goto out; } + do_acct = nd_iostat_start(bio, &start); bio_for_each_segment(bvec, bio, iter) pmem_do_bvec(pmem, bvec.bv_page, bvec.bv_len, bvec.bv_offset, bio_data_dir(bio), iter.bi_sector); + if (do_acct) + nd_iostat_end(bio, start); out: bio_endio(bio, err);