From patchwork Wed May 27 05:24:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571793 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42FFB739 for ; Wed, 27 May 2020 05:24:35 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 20DEF20776 for ; Wed, 27 May 2020 05:24:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="dU4IYFdG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 20DEF20776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 12A871225F262; Tue, 26 May 2020 22:20:26 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 35BDA1225F25E for ; Tue, 26 May 2020 22:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=MXWnqW/ECLPoEYUDhXXIoQmMb8HA67Uhf/bGGut9o84=; b=dU4IYFdGrGJ02LwrgMAzBuw1L1 Fy6vkaFecBS3RToEwf7wyyza28rPAHoYKub+Lgc/mCWG56GSqipT0RuaEj0Q3Bsv/oOaTp8awHgO3 ycKwAhNt3E896qAQ81wFLKOMXDV0IRvo6SCccdEKqNR3KaNYVutYI9WWM7om9Ala4Nxb9orrVQFio 9BkblfgFB8zU8FmA8QEII+uNUu/KeBS6qDaMfhRfoF2lZiGZnu0O9+UNYXVYz1H7yGazw97sNC8d5 40AVY/hKNnapSYEJs4sqG5UWtozjcA0jO3heikgJMjfu6F/msMA6/YAZCC/9r9FO3F38X4VeAkPms TIpnUibA==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYC-0000mh-7k; Wed, 27 May 2020 05:24:24 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 01/16] block: add disk/bio-based accounting helpers Date: Wed, 27 May 2020 07:24:04 +0200 Message-Id: <20200527052419.403583-2-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: FSYFISRKNIEMNW465ZEUTTGR3NDVOJJQ X-Message-ID-Hash: FSYFISRKNIEMNW465ZEUTTGR3NDVOJJQ X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Add two new helpers to simplify I/O accounting for bio based drivers. Currently these drivers use the generic_start_io_acct and generic_end_io_acct helpers which have very cumbersome calling conventions, don't actually return the time they started accounting, and try to deal with accounting for partitions, which can't happen for bio based drivers. The new helpers will be used to subsequently replace uses of the old helpers. The main API is the bio based wrappes in blkdev.h, but for zram which wants to account rw_page based I/O lower level routines are provided as well. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/blk-core.c | 34 ++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+) diff --git a/block/blk-core.c b/block/blk-core.c index 77e57c2e8d602..8973104f88d90 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1432,6 +1432,40 @@ void blk_account_io_start(struct request *rq, bool new_io) part_stat_unlock(); } +unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors, + unsigned int op) +{ + struct hd_struct *part = &disk->part0; + const int sgrp = op_stat_group(op); + unsigned long now = READ_ONCE(jiffies); + + part_stat_lock(); + update_io_ticks(part, now, false); + part_stat_inc(part, ios[sgrp]); + part_stat_add(part, sectors[sgrp], sectors); + part_stat_local_inc(part, in_flight[op_is_write(op)]); + part_stat_unlock(); + + return now; +} +EXPORT_SYMBOL(disk_start_io_acct); + +void disk_end_io_acct(struct gendisk *disk, unsigned int op, + unsigned long start_time) +{ + struct hd_struct *part = &disk->part0; + const int sgrp = op_stat_group(op); + unsigned long now = READ_ONCE(jiffies); + unsigned long duration = now - start_time; + + part_stat_lock(); + update_io_ticks(part, now, true); + part_stat_add(part, nsecs[sgrp], jiffies_to_nsecs(duration)); + part_stat_local_dec(part, in_flight[op_is_write(op)]); + part_stat_unlock(); +} +EXPORT_SYMBOL(disk_end_io_acct); + /* * Steal bios from a request and add them to a bio list. * The request must not have been partially completed before. diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 7d10f4e632325..6f7ff0fa8fcf8 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1892,4 +1892,32 @@ static inline void blk_wake_io_task(struct task_struct *waiter) wake_up_process(waiter); } +unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors, + unsigned int op); +void disk_end_io_acct(struct gendisk *disk, unsigned int op, + unsigned long start_time); + +#ifdef CONFIG_BLOCK +/** + * bio_start_io_acct - start I/O accounting for bio based drivers + * @bio: bio to start account for + * + * Returns the start time that should be passed back to bio_end_io_acct(). + */ +static inline unsigned long bio_start_io_acct(struct bio *bio) +{ + return disk_start_io_acct(bio->bi_disk, bio_sectors(bio), bio_op(bio)); +} + +/** + * bio_end_io_acct - end I/O accounting for bio based drivers + * @bio: bio to end account for + * @start: start time returned by bio_start_io_acct() + */ +static inline void bio_end_io_acct(struct bio *bio, unsigned long start_time) +{ + return disk_end_io_acct(bio->bi_disk, bio_op(bio), start_time); +} +#endif /* CONFIG_BLOCK */ + #endif From patchwork Wed May 27 05:24:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571795 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17403912 for ; Wed, 27 May 2020 05:24:37 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E9AD6207ED for ; Wed, 27 May 2020 05:24:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="G4zCBkuA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9AD6207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 3DD551225F27D; Tue, 26 May 2020 22:20:27 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 37A231225F262 for ; Tue, 26 May 2020 22:20:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=5LFbl+1u6fs2VTDnFYoFyNCkVgiJa2DLTkAp8608ZsA=; b=G4zCBkuASjNCa2c9rwAHr+qPGJ S54mjipGj/5FyoDo9F3/S2qB8tzO1JnuOooR4zK3y32G0eReFUe5pDoK80zFo0JDNXZ0HlA1LQ4a7 gXwOCnEZbPB0ddpfEt+vQd5KVgIB5e+bCSN1Y8Pwsw4ir+78kr02g38dlAPh99dzMbIEK2SmdrN9w IaaRhEnyO+MQjZuJjOCDkyHRnu9unuZ1J+rYFfYz8TKzmMzgjwGOnWIwyEWbAi2g41p3BIENpcpWr HQt+IB7TqWP18nEU295n/1n44IN/9n+QlWYvb+vfjTYxFrrxIla1qE6+k9WuPAIabcOFpL1QTqTyQ AZIAEM0Q==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYF-0000nC-4j; Wed, 27 May 2020 05:24:27 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 02/16] drbd: use bio_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:05 +0200 Message-Id: <20200527052419.403583-3-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: 6TRBBQEBKL3WPTA2TJRUMQOKWTZ33MQY X-Message-ID-Hash: 6TRBBQEBKL3WPTA2TJRUMQOKWTZ33MQY X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Switch drbd to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/block/drbd/drbd_req.c | 27 ++++----------------------- 1 file changed, 4 insertions(+), 23 deletions(-) diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c index 840c3aef3c5c9..c80a2f1c3c2a7 100644 --- a/drivers/block/drbd/drbd_req.c +++ b/drivers/block/drbd/drbd_req.c @@ -21,24 +21,6 @@ static bool drbd_may_do_local_read(struct drbd_device *device, sector_t sector, int size); -/* Update disk stats at start of I/O request */ -static void _drbd_start_io_acct(struct drbd_device *device, struct drbd_request *req) -{ - struct request_queue *q = device->rq_queue; - - generic_start_io_acct(q, bio_op(req->master_bio), - req->i.size >> 9, &device->vdisk->part0); -} - -/* Update disk stats when completing request upwards */ -static void _drbd_end_io_acct(struct drbd_device *device, struct drbd_request *req) -{ - struct request_queue *q = device->rq_queue; - - generic_end_io_acct(q, bio_op(req->master_bio), - &device->vdisk->part0, req->start_jif); -} - static struct drbd_request *drbd_req_new(struct drbd_device *device, struct bio *bio_src) { struct drbd_request *req; @@ -263,7 +245,7 @@ void drbd_req_complete(struct drbd_request *req, struct bio_and_error *m) start_new_tl_epoch(first_peer_device(device)->connection); /* Update disk stats */ - _drbd_end_io_acct(device, req); + bio_end_io_acct(req->master_bio, req->start_jif); /* If READ failed, * have it be pushed back to the retry work queue, @@ -1222,16 +1204,15 @@ drbd_request_prepare(struct drbd_device *device, struct bio *bio, unsigned long bio_endio(bio); return ERR_PTR(-ENOMEM); } - req->start_jif = start_jif; + + /* Update disk stats */ + req->start_jif = bio_start_io_acct(req->master_bio); if (!get_ldev(device)) { bio_put(req->private_bio); req->private_bio = NULL; } - /* Update disk stats */ - _drbd_start_io_acct(device, req); - /* process discards always from our submitter thread */ if (bio_op(bio) == REQ_OP_WRITE_ZEROES || bio_op(bio) == REQ_OP_DISCARD) From patchwork Wed May 27 05:24:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571797 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 670FD739 for ; Wed, 27 May 2020 05:24:41 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 44A9B207ED for ; Wed, 27 May 2020 05:24:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="au7HokE0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 44A9B207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 540661225F26A; Tue, 26 May 2020 22:20:32 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id C00681225F26A for ; Tue, 26 May 2020 22:20:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=BoE9LPw2oUO3NHvNjCZuG9y6ne8/lIjQVC7yN88KH00=; b=au7HokE0bZOtEUIYi9XKtdmXJh iILFYm6nSw4YwEgYmqWTci3irvj6mGEu0DIOyoOLaZBJyQYGhxSVuuKNsaWqlWe0RIKauXSfrRBqE IvNdBZ5NNmeuUaIMQvVMRdzITUPU9BltNAHbol/EHeS1+TUcfMNOnvcHBpK1UspVIum+bX8Oqnv8O HrovPzyj17VacYYIiWjIJmULkhYr6pOri2dEv837UAOnlQAcOPbfIF5YlFuuhCFf9GiNj0B2FTBss cVNMAZoff+zoSDx1GMWkM+iNBbm0jZnHF9MWPOhXzlZf21SnmYJ6THHJVZpm7FTr5Ks3pOKtiCp+X q0iuPy3A==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYI-0000nL-5P; Wed, 27 May 2020 05:24:30 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 03/16] rsxx: use bio_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:06 +0200 Message-Id: <20200527052419.403583-4-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: NRVPQIHIPO7NYPN7T7F3P3A4E56ZZWHU X-Message-ID-Hash: NRVPQIHIPO7NYPN7T7F3P3A4E56ZZWHU X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Switch rsxx to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/block/rsxx/dev.c | 19 ++----------------- 1 file changed, 2 insertions(+), 17 deletions(-) diff --git a/drivers/block/rsxx/dev.c b/drivers/block/rsxx/dev.c index 8ffa8260dcafe..3ba07ab30c84f 100644 --- a/drivers/block/rsxx/dev.c +++ b/drivers/block/rsxx/dev.c @@ -96,20 +96,6 @@ static const struct block_device_operations rsxx_fops = { .ioctl = rsxx_blkdev_ioctl, }; -static void disk_stats_start(struct rsxx_cardinfo *card, struct bio *bio) -{ - generic_start_io_acct(card->queue, bio_op(bio), bio_sectors(bio), - &card->gendisk->part0); -} - -static void disk_stats_complete(struct rsxx_cardinfo *card, - struct bio *bio, - unsigned long start_time) -{ - generic_end_io_acct(card->queue, bio_op(bio), - &card->gendisk->part0, start_time); -} - static void bio_dma_done_cb(struct rsxx_cardinfo *card, void *cb_data, unsigned int error) @@ -121,7 +107,7 @@ static void bio_dma_done_cb(struct rsxx_cardinfo *card, if (atomic_dec_and_test(&meta->pending_dmas)) { if (!card->eeh_state && card->gendisk) - disk_stats_complete(card, meta->bio, meta->start_time); + bio_end_io_acct(meta->bio, meta->start_time); if (atomic_read(&meta->error)) bio_io_error(meta->bio); @@ -167,10 +153,9 @@ static blk_qc_t rsxx_make_request(struct request_queue *q, struct bio *bio) bio_meta->bio = bio; atomic_set(&bio_meta->error, 0); atomic_set(&bio_meta->pending_dmas, 0); - bio_meta->start_time = jiffies; if (!unlikely(card->halt)) - disk_stats_start(card, bio); + bio_meta->start_time = bio_start_io_acct(bio); dev_dbg(CARD_TO_DEV(card), "BIO[%c]: meta: %p addr8: x%llx size: %d\n", bio_data_dir(bio) ? 'W' : 'R', bio_meta, From patchwork Wed May 27 05:24:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571801 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB5DC912 for ; Wed, 27 May 2020 05:24:43 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C98F02145D for ; Wed, 27 May 2020 05:24:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="reZVOaPv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C98F02145D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 945D7122679E0; Tue, 26 May 2020 22:20:34 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 786001226C87A for ; Tue, 26 May 2020 22:20:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=e95roO15e/1FE821nYZkZq6uV5Q2qk7WbmPP5rrCFyg=; b=reZVOaPvRB8icQRe/szMFIkyye usjZ7NefiuG+mac7tudHmqWEB3WHBZzuCPnIkxdt+0exVpCNe8nAHLBaV+VdKT3h8SCBCSq/Xqh+V JQmylzi3yz2nLrBADieUmV5Uk2IwZGpV3/sXRoMccQ6LxQnIMpJ4w/kFHIDPV6gHReSf1IOhCW5eF zUnLwosHq5sDNL7l8DeAsdCxgLYUIKZ5hZyK9IlYNVWj/OxP5y5isY2i54uuYRh9zW09lVZn+Iesl gh4R8GESkny6q/zgWbj73NhmA2oZIK0lUjDhuO84tkAj+QFxba+Q0hyzgsy9cmTxiVFIf2bGiaS/u t234V5+Q==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYL-0000nh-2n; Wed, 27 May 2020 05:24:33 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 04/16] lightnvm/pblk: use bio_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:07 +0200 Message-Id: <20200527052419.403583-5-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: WBRNT6CC4T5NJU47TUAIXOSP2WBKEPZU X-Message-ID-Hash: WBRNT6CC4T5NJU47TUAIXOSP2WBKEPZU X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Switch rsxx to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/lightnvm/pblk-cache.c | 8 +++----- drivers/lightnvm/pblk-read.c | 11 ++++------- 2 files changed, 7 insertions(+), 12 deletions(-) diff --git a/drivers/lightnvm/pblk-cache.c b/drivers/lightnvm/pblk-cache.c index 5c1034c22197c..f185f1a000083 100644 --- a/drivers/lightnvm/pblk-cache.c +++ b/drivers/lightnvm/pblk-cache.c @@ -21,16 +21,14 @@ void pblk_write_to_cache(struct pblk *pblk, struct bio *bio, unsigned long flags) { - struct request_queue *q = pblk->dev->q; struct pblk_w_ctx w_ctx; sector_t lba = pblk_get_lba(bio); - unsigned long start_time = jiffies; + unsigned long start_time; unsigned int bpos, pos; int nr_entries = pblk_get_secs(bio); int i, ret; - generic_start_io_acct(q, REQ_OP_WRITE, bio_sectors(bio), - &pblk->disk->part0); + start_time = bio_start_io_acct(bio); /* Update the write buffer head (mem) with the entries that we can * write. The write in itself cannot fail, so there is no need to @@ -79,7 +77,7 @@ void pblk_write_to_cache(struct pblk *pblk, struct bio *bio, pblk_rl_inserted(&pblk->rl, nr_entries); out: - generic_end_io_acct(q, REQ_OP_WRITE, &pblk->disk->part0, start_time); + bio_end_io_acct(bio, start_time); pblk_write_should_kick(pblk); if (ret == NVM_IO_DONE) diff --git a/drivers/lightnvm/pblk-read.c b/drivers/lightnvm/pblk-read.c index 8efd14e683dc4..140927ebf41e9 100644 --- a/drivers/lightnvm/pblk-read.c +++ b/drivers/lightnvm/pblk-read.c @@ -187,12 +187,11 @@ static void pblk_end_user_read(struct bio *bio, int error) static void __pblk_end_io_read(struct pblk *pblk, struct nvm_rq *rqd, bool put_line) { - struct nvm_tgt_dev *dev = pblk->dev; struct pblk_g_ctx *r_ctx = nvm_rq_to_pdu(rqd); struct bio *int_bio = rqd->bio; unsigned long start_time = r_ctx->start_time; - generic_end_io_acct(dev->q, REQ_OP_READ, &pblk->disk->part0, start_time); + bio_end_io_acct(int_bio, start_time); if (rqd->error) pblk_log_read_err(pblk, rqd); @@ -263,17 +262,15 @@ static void pblk_read_rq(struct pblk *pblk, struct nvm_rq *rqd, struct bio *bio, void pblk_submit_read(struct pblk *pblk, struct bio *bio) { - struct nvm_tgt_dev *dev = pblk->dev; - struct request_queue *q = dev->q; sector_t blba = pblk_get_lba(bio); unsigned int nr_secs = pblk_get_secs(bio); bool from_cache; struct pblk_g_ctx *r_ctx; struct nvm_rq *rqd; struct bio *int_bio, *split_bio; + unsigned long start_time; - generic_start_io_acct(q, REQ_OP_READ, bio_sectors(bio), - &pblk->disk->part0); + start_time = bio_start_io_acct(bio); rqd = pblk_alloc_rqd(pblk, PBLK_READ); @@ -283,7 +280,7 @@ void pblk_submit_read(struct pblk *pblk, struct bio *bio) rqd->end_io = pblk_end_io_read; r_ctx = nvm_rq_to_pdu(rqd); - r_ctx->start_time = jiffies; + r_ctx->start_time = start_time; r_ctx->lba = blba; if (pblk_alloc_rqd_meta(pblk, rqd)) { From patchwork Wed May 27 05:24:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571799 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 875D7739 for ; Wed, 27 May 2020 05:24:42 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6560520776 for ; Wed, 27 May 2020 05:24:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="fzu6cAW8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6560520776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 71B631226C87D; Tue, 26 May 2020 22:20:33 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 662041225F26A for ; Tue, 26 May 2020 22:20:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=nZGXLKg/fa4tsfyS2fhj2WVlRQXCuRwjRot6NDlQz5I=; b=fzu6cAW87l58ZwGqhqLgXVanPP 3esCDLRZY/DJ2+wD/8brXOpiPVWoMKjWh/EaYnrAMvLsXPBhtyQQRmf2cLX0SHIE87Y75sA6RfkFj ObWnsjh4dBw2E+aSS5UDyEIy6I3bYiw8jJBof4llS///1Ee22hp8OjcO3Mkg4ihREWtRdjstQwsNW nmZGy811NLj6jVsz/Prri/VWqQI4cgAzk9boY8Qb/Bii73ihJ+V3Hum0HnHoJMVG3WNJqgsZp9VG/ 6gMWXiAvZQ8QkodLABZyXP9mO6rxOGI5kSsfE5MFAaE3tJwqs8Mt5KVH0ykUxWmrWmPHYODFZtxzf XyfwtGXw==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYN-0000oT-Ry; Wed, 27 May 2020 05:24:36 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 05/16] bcache: use bio_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:08 +0200 Message-Id: <20200527052419.403583-6-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: 5ERGJX5A2GCHU65I4Y66OJZ6FMO7M7BC X-Message-ID-Hash: 5ERGJX5A2GCHU65I4Y66OJZ6FMO7M7BC X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org, Coly Li X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Switch bcache to use the nicer bio accounting helpers, and call the routines where we also sample the start time to give coherent accounting results. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov Acked-by: Coly Li --- drivers/md/bcache/request.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/drivers/md/bcache/request.c b/drivers/md/bcache/request.c index 77d1a26975174..22b483527176b 100644 --- a/drivers/md/bcache/request.c +++ b/drivers/md/bcache/request.c @@ -668,9 +668,7 @@ static void backing_request_endio(struct bio *bio) static void bio_complete(struct search *s) { if (s->orig_bio) { - generic_end_io_acct(s->d->disk->queue, bio_op(s->orig_bio), - &s->d->disk->part0, s->start_time); - + bio_end_io_acct(s->orig_bio, s->start_time); trace_bcache_request_end(s->d, s->orig_bio); s->orig_bio->bi_status = s->iop.status; bio_endio(s->orig_bio); @@ -730,7 +728,7 @@ static inline struct search *search_alloc(struct bio *bio, s->recoverable = 1; s->write = op_is_write(bio_op(bio)); s->read_dirty_data = 0; - s->start_time = jiffies; + s->start_time = bio_start_io_acct(bio); s->iop.c = d->c; s->iop.bio = NULL; @@ -1082,8 +1080,7 @@ static void detached_dev_end_io(struct bio *bio) bio->bi_end_io = ddip->bi_end_io; bio->bi_private = ddip->bi_private; - generic_end_io_acct(ddip->d->disk->queue, bio_op(bio), - &ddip->d->disk->part0, ddip->start_time); + bio_end_io_acct(bio, ddip->start_time); if (bio->bi_status) { struct cached_dev *dc = container_of(ddip->d, @@ -1108,7 +1105,7 @@ static void detached_dev_do_request(struct bcache_device *d, struct bio *bio) */ ddip = kzalloc(sizeof(struct detached_dev_io_private), GFP_NOIO); ddip->d = d; - ddip->start_time = jiffies; + ddip->start_time = bio_start_io_acct(bio); ddip->bi_end_io = bio->bi_end_io; ddip->bi_private = bio->bi_private; bio->bi_end_io = detached_dev_end_io; @@ -1190,11 +1187,6 @@ blk_qc_t cached_dev_make_request(struct request_queue *q, struct bio *bio) } } - generic_start_io_acct(q, - bio_op(bio), - bio_sectors(bio), - &d->disk->part0); - bio_set_dev(bio, dc->bdev); bio->bi_iter.bi_sector += dc->sb.data_offset; @@ -1311,8 +1303,6 @@ blk_qc_t flash_dev_make_request(struct request_queue *q, struct bio *bio) return BLK_QC_T_NONE; } - generic_start_io_acct(q, bio_op(bio), bio_sectors(bio), &d->disk->part0); - s = search_alloc(bio, d); cl = &s->cl; bio = &s->bio.bio; From patchwork Wed May 27 05:24:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1439912 for ; Wed, 27 May 2020 05:24:45 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9FF8A207ED for ; Wed, 27 May 2020 05:24:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="OhTmjQSR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9FF8A207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id AA18F122679E4; Tue, 26 May 2020 22:20:36 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 0F5641226C87A for ; Tue, 26 May 2020 22:20:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=vFOxaeomnXbJ8W08WAqr52lHk9RJsAZ/1N3x9Xkju10=; b=OhTmjQSRXVaxotYxGsOXSMHrQC amZN/Dp3D9XGupkYbCp1d9amADvB7j8K16cmCS/ZwM10M9U4hHXJxdCHCwWzOwiiXzsl3lMBrPUnB KPFAX6IE+Bqk2Sk3s1RJXUVzXR+iGPbtFC4DQwSMrrF+2BzArKyrjspmjB41GZ0DSyHvL62KbYhy7 8auUOJ0kW4ayFuHzKvodYui/yG6Kd2FTIMPuxZeKTLhpNHxc44kpLAErS4FPlGc/NDYSP1fr2XzEY WVa8w2i1LCBGZhYK0djo3Ca6ifRi/SoC4tR+8CmE/HwU913N5aDuwK1AXwXG+oLebgQVcmqgm+qYV 4kA00gHA==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYQ-0000oi-QS; Wed, 27 May 2020 05:24:39 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 06/16] dm: use bio_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:09 +0200 Message-Id: <20200527052419.403583-7-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: XDP7ZJK6T5DOTBIGNZL2VKXBP5YTNYYK X-Message-ID-Hash: XDP7ZJK6T5DOTBIGNZL2VKXBP5YTNYYK X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Switch dm to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/md/dm.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index f215b86664484..3f39fa1ac756e 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -681,11 +681,7 @@ static void start_io_acct(struct dm_io *io) struct mapped_device *md = io->md; struct bio *bio = io->orig_bio; - io->start_time = jiffies; - - generic_start_io_acct(md->queue, bio_op(bio), bio_sectors(bio), - &dm_disk(md)->part0); - + io->start_time = bio_start_io_acct(bio); if (unlikely(dm_stats_used(&md->stats))) dm_stats_account_io(&md->stats, bio_data_dir(bio), bio->bi_iter.bi_sector, bio_sectors(bio), @@ -698,8 +694,7 @@ static void end_io_acct(struct dm_io *io) struct bio *bio = io->orig_bio; unsigned long duration = jiffies - io->start_time; - generic_end_io_acct(md->queue, bio_op(bio), &dm_disk(md)->part0, - io->start_time); + bio_end_io_acct(bio, io->start_time); if (unlikely(dm_stats_used(&md->stats))) dm_stats_account_io(&md->stats, bio_data_dir(bio), From patchwork Wed May 27 05:24:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571811 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3BEC912 for ; Wed, 27 May 2020 05:24:51 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D2C62207ED for ; Wed, 27 May 2020 05:24:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="i0G7rCar" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D2C62207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id DA698122679E9; Tue, 26 May 2020 22:20:42 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id A8B171225F266 for ; Tue, 26 May 2020 22:20:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=guMNtkQLFS+Af7Jv6Zf+xm7qUVgM/uq3RWfopHGYO30=; b=i0G7rCar+EI/zuVSrOdLgUCeNZ ne3hIFUC0D2jmNCM17C1KKsdDtbG4deN9Um0xYyZC1IK8VW79NkAViUEmMzZzOpcBd4HbNETXIWPW soDzc/XuJhvJnoTeYHDWpjaIX/VnYCB3m+cUMKTxWp+EBTyvAEOqJJmbbxp4PKIPgD4GsjjzMuFI5 GsgWTiYhmEaCtCCC/CVfO4Cx7+qCVHBji7tHFv6MCOySws0wIUF2PEPHvghPDN5fshok1+PgLIOJN GKK8NCmg+4TbUgsdxyMn5hFFbiAsrYdY2DJaGyT/FeITQwP3OUNnPxy21wObSPi/QewTDJCmAgqUQ JHXlosww==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYT-0000pk-Is; Wed, 27 May 2020 05:24:41 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 07/16] nvdimm: use bio_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:10 +0200 Message-Id: <20200527052419.403583-8-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: JBVQPNUGU7ZSQ3V4LQBP4CVYURISUN47 X-Message-ID-Hash: JBVQPNUGU7ZSQ3V4LQBP4CVYURISUN47 X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Switch dm to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/nvdimm/blk.c | 6 ++++-- drivers/nvdimm/btt.c | 6 ++++-- drivers/nvdimm/nd.h | 19 ------------------- drivers/nvdimm/pmem.c | 6 ++++-- 4 files changed, 12 insertions(+), 25 deletions(-) diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c index 43751fab9d36a..036e23aef9b04 100644 --- a/drivers/nvdimm/blk.c +++ b/drivers/nvdimm/blk.c @@ -178,7 +178,9 @@ static blk_qc_t nd_blk_make_request(struct request_queue *q, struct bio *bio) bip = bio_integrity(bio); nsblk = q->queuedata; rw = bio_data_dir(bio); - do_acct = nd_iostat_start(bio, &start); + do_acct = blk_queue_io_stat(bio->bi_disk->queue); + if (do_acct) + start = bio_start_io_acct(bio); bio_for_each_segment(bvec, bio, iter) { unsigned int len = bvec.bv_len; @@ -195,7 +197,7 @@ static blk_qc_t nd_blk_make_request(struct request_queue *q, struct bio *bio) } } if (do_acct) - nd_iostat_end(bio, start); + bio_end_io_acct(bio, start); bio_endio(bio); return BLK_QC_T_NONE; diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c index 3b09419218d6f..90c0c4bbe77b9 100644 --- a/drivers/nvdimm/btt.c +++ b/drivers/nvdimm/btt.c @@ -1452,7 +1452,9 @@ static blk_qc_t btt_make_request(struct request_queue *q, struct bio *bio) if (!bio_integrity_prep(bio)) return BLK_QC_T_NONE; - do_acct = nd_iostat_start(bio, &start); + do_acct = blk_queue_io_stat(bio->bi_disk->queue); + if (do_acct) + start = bio_start_io_acct(bio); bio_for_each_segment(bvec, bio, iter) { unsigned int len = bvec.bv_len; @@ -1477,7 +1479,7 @@ static blk_qc_t btt_make_request(struct request_queue *q, struct bio *bio) } } if (do_acct) - nd_iostat_end(bio, start); + bio_end_io_acct(bio, start); bio_endio(bio); return BLK_QC_T_NONE; diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h index 85dbb2a322b9b..85c1ae813ea31 100644 --- a/drivers/nvdimm/nd.h +++ b/drivers/nvdimm/nd.h @@ -396,25 +396,6 @@ static inline int nvdimm_setup_pfn(struct nd_pfn *nd_pfn, #endif int nd_blk_region_init(struct nd_region *nd_region); int nd_region_activate(struct nd_region *nd_region); -void __nd_iostat_start(struct bio *bio, unsigned long *start); -static inline bool nd_iostat_start(struct bio *bio, unsigned long *start) -{ - struct gendisk *disk = bio->bi_disk; - - if (!blk_queue_io_stat(disk->queue)) - return false; - - *start = jiffies; - generic_start_io_acct(disk->queue, bio_op(bio), bio_sectors(bio), - &disk->part0); - return true; -} -static inline void nd_iostat_end(struct bio *bio, unsigned long start) -{ - struct gendisk *disk = bio->bi_disk; - - generic_end_io_acct(disk->queue, bio_op(bio), &disk->part0, start); -} static inline bool is_bad_pmem(struct badblocks *bb, sector_t sector, unsigned int len) { diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 2df6994acf836..97f948f8f4e62 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -202,7 +202,9 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio) if (bio->bi_opf & REQ_PREFLUSH) ret = nvdimm_flush(nd_region, bio); - do_acct = nd_iostat_start(bio, &start); + do_acct = blk_queue_io_stat(bio->bi_disk->queue); + if (do_acct) + start = bio_start_io_acct(bio); bio_for_each_segment(bvec, bio, iter) { if (op_is_write(bio_op(bio))) rc = pmem_do_write(pmem, bvec.bv_page, bvec.bv_offset, @@ -216,7 +218,7 @@ static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio) } } if (do_acct) - nd_iostat_end(bio, start); + bio_end_io_acct(bio, start); if (bio->bi_opf & REQ_FUA) ret = nvdimm_flush(nd_region, bio); From patchwork Wed May 27 05:24:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571807 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D601D739 for ; Wed, 27 May 2020 05:24:50 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B36A6207ED for ; Wed, 27 May 2020 05:24:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="b1nllpsV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B36A6207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id C0ED6122679E5; Tue, 26 May 2020 22:20:41 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8F3D71225F266 for ; Tue, 26 May 2020 22:20:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=tYO/gDhBfCcsr0X6BnTt3DLrPKvKaNJnAYUJw8Z986o=; b=b1nllpsV/a54uoCmOngroSJuQa VdtWl/+xyPwYLgDYCElFBCov9QFtKdlqgpSsTeZMZdY5tjWDspuLhwb8xH3IBfT7/Myrkba3SV1Rw UUk44G6PQQVmP5kjd1ipNp3S92jY0f+Q6JXbR+qWNvfdV2UxcZ6bBHLvCJWY3hPK144wze9fdkCxT eXMmmV38qrNHhPgrQn1yxU6gL80vU7ZrHx4mjzY0coEDkn53mAq9ldSkvl3ltLuGTUV4yyLzeqCeG AeCxqW0IMayHOek2jmwnuKPsJSo6TChjmopqBwqRpwaxblDwX9LQX7vt2fiNYQ0o8lXOJ+4Q5GGkK aVsO+GsQ==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYW-0000q0-6A; Wed, 27 May 2020 05:24:44 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 08/16] zram: nvdimm: use bio_{start,end}_io_acct and disk_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:11 +0200 Message-Id: <20200527052419.403583-9-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: CO2S7CQFULX2QGH35IBZ2TJBLPCWHZ3O X-Message-ID-Hash: CO2S7CQFULX2QGH35IBZ2TJBLPCWHZ3O X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Switch zram to use the nicer bio accounting helpers, and as part of that ensure each bio is counted as a single I/O request. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- drivers/block/zram/zram_drv.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index ebb234f36909c..6e2ad90b17a37 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1510,13 +1510,8 @@ static void zram_bio_discard(struct zram *zram, u32 index, static int zram_bvec_rw(struct zram *zram, struct bio_vec *bvec, u32 index, int offset, unsigned int op, struct bio *bio) { - unsigned long start_time = jiffies; - struct request_queue *q = zram->disk->queue; int ret; - generic_start_io_acct(q, op, bvec->bv_len >> SECTOR_SHIFT, - &zram->disk->part0); - if (!op_is_write(op)) { atomic64_inc(&zram->stats.num_reads); ret = zram_bvec_read(zram, bvec, index, offset, bio); @@ -1526,8 +1521,6 @@ static int zram_bvec_rw(struct zram *zram, struct bio_vec *bvec, u32 index, ret = zram_bvec_write(zram, bvec, index, offset, bio); } - generic_end_io_acct(q, op, &zram->disk->part0, start_time); - zram_slot_lock(zram, index); zram_accessed(zram, index); zram_slot_unlock(zram, index); @@ -1548,6 +1541,7 @@ static void __zram_make_request(struct zram *zram, struct bio *bio) u32 index; struct bio_vec bvec; struct bvec_iter iter; + unsigned long start_time; index = bio->bi_iter.bi_sector >> SECTORS_PER_PAGE_SHIFT; offset = (bio->bi_iter.bi_sector & @@ -1563,6 +1557,7 @@ static void __zram_make_request(struct zram *zram, struct bio *bio) break; } + start_time = bio_start_io_acct(bio); bio_for_each_segment(bvec, bio, iter) { struct bio_vec bv = bvec; unsigned int unwritten = bvec.bv_len; @@ -1571,8 +1566,10 @@ static void __zram_make_request(struct zram *zram, struct bio *bio) bv.bv_len = min_t(unsigned int, PAGE_SIZE - offset, unwritten); if (zram_bvec_rw(zram, &bv, index, offset, - bio_op(bio), bio) < 0) - goto out; + bio_op(bio), bio) < 0) { + bio->bi_status = BLK_STS_IOERR; + break; + } bv.bv_offset += bv.bv_len; unwritten -= bv.bv_len; @@ -1580,12 +1577,8 @@ static void __zram_make_request(struct zram *zram, struct bio *bio) update_position(&index, &offset, &bv); } while (unwritten); } - + bio_end_io_acct(bio, start_time); bio_endio(bio); - return; - -out: - bio_io_error(bio); } /* @@ -1633,6 +1626,7 @@ static int zram_rw_page(struct block_device *bdev, sector_t sector, u32 index; struct zram *zram; struct bio_vec bv; + unsigned long start_time; if (PageTransHuge(page)) return -ENOTSUPP; @@ -1651,7 +1645,9 @@ static int zram_rw_page(struct block_device *bdev, sector_t sector, bv.bv_len = PAGE_SIZE; bv.bv_offset = 0; + start_time = disk_start_io_acct(bdev->bd_disk, SECTORS_PER_PAGE, op); ret = zram_bvec_rw(zram, &bv, index, offset, op, NULL); + disk_end_io_acct(bdev->bd_disk, op, start_time); out: /* * If I/O fails, just return error(ie, non-zero) without From patchwork Wed May 27 05:24:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571813 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1608A912 for ; Wed, 27 May 2020 05:24:54 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E871820776 for ; Wed, 27 May 2020 05:24:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="qEEJ85Hr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E871820776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 04A25122679EB; Tue, 26 May 2020 22:20:45 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id A5DE5122679E7 for ; Tue, 26 May 2020 22:20:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=j/0fX6heLmqRakQA7qi+pziHK3sxFOca3+bsUnfP3dI=; b=qEEJ85HrX/zM1/Qp5PJTXizQe7 ZMMBhaQjqwnkthh3OZVxds1+2PrSQ796RMv/+Q/FfoufYwyx6FKNS3NBFKHmq9jozRy4mFuvz3Kfh ROIennB5z9rigcC/EwdEaFNyXpTXfL5xo+oQAncGXo4/RObBt0w1a9ONcbH7PGi1zPWYGpOd57xLh 6tqtFZtPokO3Cp6QWTR7CHKsOo3krfH97u71KzF0ZygcY2QXY6+u6k8CjL8HL18Eel3OnWtEnlWb/ gp2Xl4aA9IpMx6Jo5JubP6EhkoUpK8KNW4YLscSnBEkcO4+JtaoxmoPaojxK1dzCW8MGjuktoPlBY cFB700Uw==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYY-0000qw-W9; Wed, 27 May 2020 05:24:47 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 09/16] block: remove generic_{start,end}_io_acct Date: Wed, 27 May 2020 07:24:12 +0200 Message-Id: <20200527052419.403583-10-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: 7X53EIQIFGRRJUGQHPUK45GO7Z7XDO2H X-Message-ID-Hash: 7X53EIQIFGRRJUGQHPUK45GO7Z7XDO2H X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Remove these now unused functions. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/bio.c | 39 --------------------------------------- include/linux/bio.h | 6 ------ 2 files changed, 45 deletions(-) diff --git a/block/bio.c b/block/bio.c index 9c101a0572ca2..3e89c7b37855a 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1392,45 +1392,6 @@ void update_io_ticks(struct hd_struct *part, unsigned long now, bool end) } } -void generic_start_io_acct(struct request_queue *q, int op, - unsigned long sectors, struct hd_struct *part) -{ - const int sgrp = op_stat_group(op); - int rw = op_is_write(op); - - part_stat_lock(); - - update_io_ticks(part, jiffies, false); - part_stat_inc(part, ios[sgrp]); - part_stat_add(part, sectors[sgrp], sectors); - part_stat_local_inc(part, in_flight[rw]); - if (part->partno) - part_stat_local_inc(&part_to_disk(part)->part0, in_flight[rw]); - - part_stat_unlock(); -} -EXPORT_SYMBOL(generic_start_io_acct); - -void generic_end_io_acct(struct request_queue *q, int req_op, - struct hd_struct *part, unsigned long start_time) -{ - unsigned long now = jiffies; - unsigned long duration = now - start_time; - const int sgrp = op_stat_group(req_op); - int rw = op_is_write(req_op); - - part_stat_lock(); - - update_io_ticks(part, now, true); - part_stat_add(part, nsecs[sgrp], jiffies_to_nsecs(duration)); - part_stat_local_dec(part, in_flight[rw]); - if (part->partno) - part_stat_local_dec(&part_to_disk(part)->part0, in_flight[rw]); - - part_stat_unlock(); -} -EXPORT_SYMBOL(generic_end_io_acct); - static inline bool bio_remaining_done(struct bio *bio) { /* diff --git a/include/linux/bio.h b/include/linux/bio.h index 950c9dc44c4f2..941378ec5b39f 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -444,12 +444,6 @@ void bio_release_pages(struct bio *bio, bool mark_dirty); extern void bio_set_pages_dirty(struct bio *bio); extern void bio_check_pages_dirty(struct bio *bio); -void generic_start_io_acct(struct request_queue *q, int op, - unsigned long sectors, struct hd_struct *part); -void generic_end_io_acct(struct request_queue *q, int op, - struct hd_struct *part, - unsigned long start_time); - extern void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter, struct bio *src, struct bvec_iter *src_iter); extern void bio_copy_data(struct bio *dst, struct bio *src); From patchwork Wed May 27 05:24:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571815 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E3DE739 for ; Wed, 27 May 2020 05:24:57 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C7D2208E4 for ; Wed, 27 May 2020 05:24:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="b9lpto70" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C7D2208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 292E6122679F2; Tue, 26 May 2020 22:20:48 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 68A84122679EF for ; Tue, 26 May 2020 22:20:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=jAURZKlR3wABF7HVZ7jgwwb3T9+sEOkFONnpRICBtxk=; b=b9lpto70CQqiKdI3uzCEpMlbCf S0IgWN79A13NXEagh9OH9+Qh+JTDTOeMoPjVIuUNfJiAQ+P7WzUbKwDKQk3vPhbGqHsJrVSk4utl2 rE2/SU1j43BMRA/XFHmWBPxMeSz9zQ0oiXDhfvPgsGJAMrsPJXTrGLcxfBBf8qGIfJynA3Mobq3E+ mjfsUtnhu0yeIU4MY49hiV9ahnzRBQyET4zmj6BzzQpIjttHnvEwO+71DYXKl/DaMmQljzKYxd6aJ NND9YGXuZ3liozt0LHGnA/lee1fDLPc7pwEf9Q19tbT96+Z0qcD37iWSLoJ4ojAms6K5BNdcZ5fSX fU8Jbe+Q==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYb-0000s4-VW; Wed, 27 May 2020 05:24:50 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 10/16] block: move update_io_ticks to blk-core.c Date: Wed, 27 May 2020 07:24:13 +0200 Message-Id: <20200527052419.403583-11-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: FSK7CKZXLV5G3RGJHME3VDSPXGZ6W74E X-Message-ID-Hash: FSK7CKZXLV5G3RGJHME3VDSPXGZ6W74E X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: All callers are in blk-core.c, so move update_io_ticks over. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/bio.c | 16 ---------------- block/blk-core.c | 15 +++++++++++++++ block/blk.h | 1 - 3 files changed, 15 insertions(+), 17 deletions(-) diff --git a/block/bio.c b/block/bio.c index 3e89c7b37855a..5235da6434aab 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1376,22 +1376,6 @@ void bio_check_pages_dirty(struct bio *bio) schedule_work(&bio_dirty_work); } -void update_io_ticks(struct hd_struct *part, unsigned long now, bool end) -{ - unsigned long stamp; -again: - stamp = READ_ONCE(part->stamp); - if (unlikely(stamp != now)) { - if (likely(cmpxchg(&part->stamp, stamp, now) == stamp)) { - __part_stat_add(part, io_ticks, end ? now - stamp : 1); - } - } - if (part->partno) { - part = &part_to_disk(part)->part0; - goto again; - } -} - static inline bool bio_remaining_done(struct bio *bio) { /* diff --git a/block/blk-core.c b/block/blk-core.c index 8973104f88d90..c1675d43c2da0 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1381,6 +1381,21 @@ unsigned int blk_rq_err_bytes(const struct request *rq) } EXPORT_SYMBOL_GPL(blk_rq_err_bytes); +static void update_io_ticks(struct hd_struct *part, unsigned long now, bool end) +{ + unsigned long stamp; +again: + stamp = READ_ONCE(part->stamp); + if (unlikely(stamp != now)) { + if (likely(cmpxchg(&part->stamp, stamp, now) == stamp)) + __part_stat_add(part, io_ticks, end ? now - stamp : 1); + } + if (part->partno) { + part = &part_to_disk(part)->part0; + goto again; + } +} + static void blk_account_io_completion(struct request *req, unsigned int bytes) { if (req->part && blk_do_io_stat(req)) { diff --git a/block/blk.h b/block/blk.h index 5db4ec1e85f7b..bdf5e94467aa2 100644 --- a/block/blk.h +++ b/block/blk.h @@ -344,7 +344,6 @@ void blk_queue_free_zone_bitmaps(struct request_queue *q); static inline void blk_queue_free_zone_bitmaps(struct request_queue *q) {} #endif -void update_io_ticks(struct hd_struct *part, unsigned long now, bool end); struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sector); int blk_alloc_devt(struct hd_struct *part, dev_t *devt); From patchwork Wed May 27 05:24:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571819 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5845C912 for ; Wed, 27 May 2020 05:24:59 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 37029207ED for ; Wed, 27 May 2020 05:24:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="EPMjuf39" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 37029207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 3DB66122679F5; Tue, 26 May 2020 22:20:50 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 145A7122679E3 for ; Tue, 26 May 2020 22:20:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=iHue7do6hun8JuTKkSqk6A1cSmWyXJMC7S37LKm+NoE=; b=EPMjuf39RInO9H+7VH8ftwUsNo nBrbMYTAFmxnLpfrZuVpY1ScxzqreAWOBCfaA7t5yKouEBcGkF9zKhgo5jFl18sDpjYO43J7js0Vl Hb6NNOxtLRfEynBwLQdDay3+Um06Dr8ig1m32jrhdmQZtDRCwxnqoBSie14OkjGMbx+mZ0IV03DQP oUDXNo3i728smfUjqtKqSWR4ki3hlV882jwsTd8h58POGiLeciUCsShGrp6BgatfPJG/3XbAAcU4G BvKXGJA002dkPLiBeS7htj5e0awbv+zJCGVKvc8zdxfH/6a3hKVqayJ4HXCmIftDglf8J0L7QRaCO l8BfM76g==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYe-0000sT-VA; Wed, 27 May 2020 05:24:53 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 11/16] block: always use a percpu variable for disk stats Date: Wed, 27 May 2020 07:24:14 +0200 Message-Id: <20200527052419.403583-12-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: Q2ZQBMGW6BLMVTMWJDLNO7MVWQG2HN6L X-Message-ID-Hash: Q2ZQBMGW6BLMVTMWJDLNO7MVWQG2HN6L X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: percpu variables have a perfectly fine working stub implementation for UP kernels, so use that. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/blk.h | 2 +- block/genhd.c | 12 +++------ block/partitions/core.c | 5 ++-- include/linux/genhd.h | 13 --------- include/linux/part_stat.h | 55 ++++++++------------------------------- 5 files changed, 18 insertions(+), 69 deletions(-) diff --git a/block/blk.h b/block/blk.h index bdf5e94467aa2..0ecba2ab383d6 100644 --- a/block/blk.h +++ b/block/blk.h @@ -378,7 +378,7 @@ static inline void hd_struct_put(struct hd_struct *part) static inline void hd_free_part(struct hd_struct *part) { - free_part_stats(part); + free_percpu(part->dkstats); kfree(part->info); percpu_ref_exit(&part->ref); } diff --git a/block/genhd.c b/block/genhd.c index 094ed90964964..3e7df0a3e6bb0 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -92,7 +92,6 @@ const char *bdevname(struct block_device *bdev, char *buf) } EXPORT_SYMBOL(bdevname); -#ifdef CONFIG_SMP static void part_stat_read_all(struct hd_struct *part, struct disk_stats *stat) { int cpu; @@ -112,12 +111,6 @@ static void part_stat_read_all(struct hd_struct *part, struct disk_stats *stat) stat->io_ticks += ptr->io_ticks; } } -#else /* CONFIG_SMP */ -static void part_stat_read_all(struct hd_struct *part, struct disk_stats *stat) -{ - memcpy(stat, &part->dkstats, sizeof(struct disk_stats)); -} -#endif /* CONFIG_SMP */ static unsigned int part_in_flight(struct request_queue *q, struct hd_struct *part) @@ -1688,14 +1681,15 @@ struct gendisk *__alloc_disk_node(int minors, int node_id) disk = kzalloc_node(sizeof(struct gendisk), GFP_KERNEL, node_id); if (disk) { - if (!init_part_stats(&disk->part0)) { + disk->part0.dkstats = alloc_percpu(struct disk_stats); + if (!disk->part0.dkstats) { kfree(disk); return NULL; } init_rwsem(&disk->lookup_sem); disk->node_id = node_id; if (disk_expand_part_tbl(disk, 0)) { - free_part_stats(&disk->part0); + free_percpu(disk->part0.dkstats); kfree(disk); return NULL; } diff --git a/block/partitions/core.c b/block/partitions/core.c index 297004fd22648..78951e33b2d7c 100644 --- a/block/partitions/core.c +++ b/block/partitions/core.c @@ -387,7 +387,8 @@ static struct hd_struct *add_partition(struct gendisk *disk, int partno, if (!p) return ERR_PTR(-EBUSY); - if (!init_part_stats(p)) { + p->dkstats = alloc_percpu(struct disk_stats); + if (!p->dkstats) { err = -ENOMEM; goto out_free; } @@ -468,7 +469,7 @@ static struct hd_struct *add_partition(struct gendisk *disk, int partno, out_free_info: kfree(p->info); out_free_stats: - free_part_stats(p); + free_percpu(p->dkstats); out_free: kfree(p); return ERR_PTR(err); diff --git a/include/linux/genhd.h b/include/linux/genhd.h index a9384449465a3..f0d6d77309a54 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -39,15 +39,6 @@ extern struct class block_class; #include #include -struct disk_stats { - u64 nsecs[NR_STAT_GROUPS]; - unsigned long sectors[NR_STAT_GROUPS]; - unsigned long ios[NR_STAT_GROUPS]; - unsigned long merges[NR_STAT_GROUPS]; - unsigned long io_ticks; - local_t in_flight[2]; -}; - #define PARTITION_META_INFO_VOLNAMELTH 64 /* * Enough for the string representation of any kind of UUID plus NULL. @@ -72,11 +63,7 @@ struct hd_struct { seqcount_t nr_sects_seq; #endif unsigned long stamp; -#ifdef CONFIG_SMP struct disk_stats __percpu *dkstats; -#else - struct disk_stats dkstats; -#endif struct percpu_ref ref; sector_t alignment_offset; diff --git a/include/linux/part_stat.h b/include/linux/part_stat.h index ece607607a864..6644197980b92 100644 --- a/include/linux/part_stat.h +++ b/include/linux/part_stat.h @@ -4,19 +4,23 @@ #include +struct disk_stats { + u64 nsecs[NR_STAT_GROUPS]; + unsigned long sectors[NR_STAT_GROUPS]; + unsigned long ios[NR_STAT_GROUPS]; + unsigned long merges[NR_STAT_GROUPS]; + unsigned long io_ticks; + local_t in_flight[2]; +}; + /* * Macros to operate on percpu disk statistics: * - * {disk|part|all}_stat_{add|sub|inc|dec}() modify the stat counters - * and should be called between disk_stat_lock() and - * disk_stat_unlock(). + * {disk|part|all}_stat_{add|sub|inc|dec}() modify the stat counters and should + * be called between disk_stat_lock() and disk_stat_unlock(). * * part_stat_read() can be called at any time. - * - * part_stat_{add|set_all}() and {init|free}_part_stats are for - * internal use only. */ -#ifdef CONFIG_SMP #define part_stat_lock() ({ rcu_read_lock(); get_cpu(); }) #define part_stat_unlock() do { put_cpu(); rcu_read_unlock(); } while (0) @@ -44,43 +48,6 @@ static inline void part_stat_set_all(struct hd_struct *part, int value) sizeof(struct disk_stats)); } -static inline int init_part_stats(struct hd_struct *part) -{ - part->dkstats = alloc_percpu(struct disk_stats); - if (!part->dkstats) - return 0; - return 1; -} - -static inline void free_part_stats(struct hd_struct *part) -{ - free_percpu(part->dkstats); -} - -#else /* !CONFIG_SMP */ -#define part_stat_lock() ({ rcu_read_lock(); 0; }) -#define part_stat_unlock() rcu_read_unlock() - -#define part_stat_get(part, field) ((part)->dkstats.field) -#define part_stat_get_cpu(part, field, cpu) part_stat_get(part, field) -#define part_stat_read(part, field) part_stat_get(part, field) - -static inline void part_stat_set_all(struct hd_struct *part, int value) -{ - memset(&part->dkstats, value, sizeof(struct disk_stats)); -} - -static inline int init_part_stats(struct hd_struct *part) -{ - return 1; -} - -static inline void free_part_stats(struct hd_struct *part) -{ -} - -#endif /* CONFIG_SMP */ - #define part_stat_read_accum(part, field) \ (part_stat_read(part, field[STAT_READ]) + \ part_stat_read(part, field[STAT_WRITE]) + \ From patchwork Wed May 27 05:24:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571821 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6374D912 for ; Wed, 27 May 2020 05:25:01 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4120A207ED for ; Wed, 27 May 2020 05:25:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="SUn+0mPi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4120A207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 51192122679F4; Tue, 26 May 2020 22:20:52 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2E324122679F1 for ; Tue, 26 May 2020 22:20:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Bqhnde9dFznWgi2aGOgRLd+HtchsYvr9PHZmbOGgvm4=; b=SUn+0mPicK4jlHHCaQrGP7lCUz eT3XvXAXsDRDpJXPJEUby/j+oC3UGTdirGmUi6TC2Cd7lFP1MV88zP0O6VMiKiV+yOzO447xpvHKt DlXXSz/CHOrcp72jA6NKbyPfxG+FCO4loHdYZN1F63x3DmHWeRqA4JP2Cp3iOakS8r3S36YQrh6sV FwnDTk5VPPwtTjnuyzVtXb60vfuvPwrOXi/FKSkJuPp4utdu9IlQVGAo37+ujlprGhnIT7gfoEKTa +3oLWmWzUbLXDlWxTC5rwkx6YSumQtpDrEE9m0tGzua9FcBhDj9TV1JddoKDK0Wnh5Jpvl3zW09tc xATfUvRQ==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYh-0000tS-QQ; Wed, 27 May 2020 05:24:56 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 12/16] block: account merge of two requests Date: Wed, 27 May 2020 07:24:15 +0200 Message-Id: <20200527052419.403583-13-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: RR3RZG4MIBZJZQ633MWFRPZQDRMRB2NE X-Message-ID-Hash: RR3RZG4MIBZJZQ633MWFRPZQDRMRB2NE X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Konstantin Khlebnikov Also rename blk_account_io_merge() into blk_account_io_merge_request() to distinguish it from merging request and bio. Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- block/blk-merge.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 6a4538d39efd2..c3beae5c1be71 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -669,18 +669,16 @@ void blk_rq_set_mixed_merge(struct request *rq) rq->rq_flags |= RQF_MIXED_MERGE; } -static void blk_account_io_merge(struct request *req) +static void blk_account_io_merge_request(struct request *req) { if (blk_do_io_stat(req)) { - struct hd_struct *part; - part_stat_lock(); - part = req->part; - - hd_struct_put(part); + part_stat_inc(req->part, merges[op_stat_group(req_op(req))]); + hd_struct_put(req->part); part_stat_unlock(); } } + /* * Two cases of handling DISCARD merge: * If max_discard_segments > 1, the driver takes every bio @@ -792,7 +790,7 @@ static struct request *attempt_merge(struct request_queue *q, /* * 'next' is going away, so update stats accordingly */ - blk_account_io_merge(next); + blk_account_io_merge_request(next); /* * ownership of bio passed from next to req, return 'next' for From patchwork Wed May 27 05:24:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571823 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 919B2739 for ; Wed, 27 May 2020 05:25:05 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 70087207ED for ; Wed, 27 May 2020 05:25:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="PNKRWR+g" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70087207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 66887122679F6; Tue, 26 May 2020 22:20:56 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id BB3CB122679F7 for ; Tue, 26 May 2020 22:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=RoHwY3xPQkQOdyCje9JyMhmjLXPKYVZXj7/LksSFANA=; b=PNKRWR+gtDTWIiBva0Bd7tCn54 gY3Djeij0cExl4f8R+kq0GVRm/X0QVv2UarGsPXAMFdfwSvRS/9VJ2ibYDPGiZCFu25fj+/umYhDW +zm43Na/czKInqfPrjZw2054BbA02h6NTZKpYQLXbKA7NtGb4b8+T0HpSPfgxmmCovRHGEbAbp8C9 c5JaeAyMWU8PyHMrjodL5JXGjeTr7BvYVfv6di4t01MtnNoo+E1bZ6wqXs0mJV1a9JIYijMfuGoxi nY4IsqEJD7I2yt4p/5EDb0epwY0Wa/kTJ1pGc5c32jToB98sWj3VJnhJpzTejbTRPcbdnk4o3F2FU 4eDsfuwQ==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYk-0000tw-MF; Wed, 27 May 2020 05:24:59 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 13/16] block: add a blk_account_io_merge_bio helper Date: Wed, 27 May 2020 07:24:16 +0200 Message-Id: <20200527052419.403583-14-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: GKWKLZSFLGP37NZXJWODG4BSCJAKSNP7 X-Message-ID-Hash: GKWKLZSFLGP37NZXJWODG4BSCJAKSNP7 X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Konstantin Khlebnikov Move the non-"new_io" branch of blk_account_io_start() into separate function. Fix merge accounting for discards (they were counted as write merges). The new blk_account_io_merge_bio() doesn't call update_io_ticks() unlike blk_account_io_start(), as there is no reason for that. Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- block/blk-core.c | 25 ++++++++++++++++--------- block/blk-exec.c | 2 +- block/blk-mq.c | 2 +- block/blk.h | 2 +- 4 files changed, 19 insertions(+), 12 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index c1675d43c2da0..bf2f7d4bc0c1c 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -636,6 +636,16 @@ void blk_put_request(struct request *req) } EXPORT_SYMBOL(blk_put_request); +static void blk_account_io_merge_bio(struct request *req) +{ + if (!blk_do_io_stat(req)) + return; + + part_stat_lock(); + part_stat_inc(req->part, merges[op_stat_group(req_op(req))]); + part_stat_unlock(); +} + bool bio_attempt_back_merge(struct request *req, struct bio *bio, unsigned int nr_segs) { @@ -656,7 +666,7 @@ bool bio_attempt_back_merge(struct request *req, struct bio *bio, bio_crypt_free_ctx(bio); - blk_account_io_start(req, false); + blk_account_io_merge_bio(req); return true; } @@ -682,7 +692,7 @@ bool bio_attempt_front_merge(struct request *req, struct bio *bio, bio_crypt_do_front_merge(req, bio); - blk_account_io_start(req, false); + blk_account_io_merge_bio(req); return true; } @@ -704,7 +714,7 @@ bool bio_attempt_discard_merge(struct request_queue *q, struct request *req, req->__data_len += bio->bi_iter.bi_size; req->nr_phys_segments = segments + 1; - blk_account_io_start(req, false); + blk_account_io_merge_bio(req); return true; no_merge: req_set_nomerge(q, req); @@ -1329,7 +1339,7 @@ blk_status_t blk_insert_cloned_request(struct request_queue *q, struct request * return BLK_STS_IOERR; if (blk_queue_io_stat(q)) - blk_account_io_start(rq, true); + blk_account_io_start(rq); /* * Since we have a scheduler attached on the top device, @@ -1433,16 +1443,13 @@ void blk_account_io_done(struct request *req, u64 now) } } -void blk_account_io_start(struct request *rq, bool new_io) +void blk_account_io_start(struct request *rq) { if (!blk_do_io_stat(rq)) return; part_stat_lock(); - if (!new_io) - part_stat_inc(rq->part, merges[rq_data_dir(rq)]); - else - rq->part = disk_map_sector_rcu(rq->rq_disk, blk_rq_pos(rq)); + rq->part = disk_map_sector_rcu(rq->rq_disk, blk_rq_pos(rq)); update_io_ticks(rq->part, jiffies, false); part_stat_unlock(); } diff --git a/block/blk-exec.c b/block/blk-exec.c index e20a852ae432d..85324d53d072f 100644 --- a/block/blk-exec.c +++ b/block/blk-exec.c @@ -55,7 +55,7 @@ void blk_execute_rq_nowait(struct request_queue *q, struct gendisk *bd_disk, rq->rq_disk = bd_disk; rq->end_io = done; - blk_account_io_start(rq, true); + blk_account_io_start(rq); /* * don't check dying flag for MQ because the request won't diff --git a/block/blk-mq.c b/block/blk-mq.c index cac11945f6023..c606c74463ccd 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1822,7 +1822,7 @@ static void blk_mq_bio_to_request(struct request *rq, struct bio *bio, blk_rq_bio_prep(rq, bio, nr_segs); blk_crypto_rq_bio_prep(rq, bio, GFP_NOIO); - blk_account_io_start(rq, true); + blk_account_io_start(rq); } static blk_status_t __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, diff --git a/block/blk.h b/block/blk.h index 0ecba2ab383d6..428f7e5d70a86 100644 --- a/block/blk.h +++ b/block/blk.h @@ -185,7 +185,7 @@ bool bio_attempt_discard_merge(struct request_queue *q, struct request *req, bool blk_attempt_plug_merge(struct request_queue *q, struct bio *bio, unsigned int nr_segs, struct request **same_queue_rq); -void blk_account_io_start(struct request *req, bool new_io); +void blk_account_io_start(struct request *req); void blk_account_io_done(struct request *req, u64 now); /* From patchwork Wed May 27 05:24:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571825 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C9E7912 for ; Wed, 27 May 2020 05:25:08 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6B54D207ED for ; Wed, 27 May 2020 05:25:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="KE/N/QUU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B54D207ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7A940122679FA; Tue, 26 May 2020 22:20:59 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 0F2BE122679F7 for ; Tue, 26 May 2020 22:20:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=9XennrgwipSgDkcsh71JEE0YegAAd1Z7p+AJvSBp+LI=; b=KE/N/QUU7KEPg8KwFaByBxVm16 6UVgeALZheCCC/5WRZ7tD/5KMAmgLmAWR4UAgXFISqpEQFvvshPhR7xaovhoiNCsgEOICUgq0pXla ebdmddx8T6oBlWKG05+ZpmhAiA1fKGN3B2PN2YKYFQ7W5495exwJOoHhhJ/pgfdB8Vsm97bCpnmsR hDB7EsMPcvACPQ2anG207HXBzX2lnwEQQgZsH4wNylFqZniDHe630r5pJbabxrW60loCtpXU6DYjs Ymp8VCHiro/kNu/cbtZMrNpFWRf84UEIQ4RmCFOF2rf67FoNQRBRThaQQ6ruizueZCgzqGZ+jTU5z 5GvxG0/Q==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYn-0000vd-EV; Wed, 27 May 2020 05:25:01 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 14/16] block: remove rcu_read_lock() from part_stat_lock() Date: Wed, 27 May 2020 07:24:17 +0200 Message-Id: <20200527052419.403583-15-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: 627E5P2EC4WUYMMNLJDQICMPGIAWL5YZ X-Message-ID-Hash: 627E5P2EC4WUYMMNLJDQICMPGIAWL5YZ X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Konstantin Khlebnikov The RCU lock is required only in disk_map_sector_rcu() to lookup the partition. After that request holds reference to related hd_struct. Replace get_cpu() with preempt_disable() - returned cpu index is unused. Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- block/genhd.c | 11 ++++++++--- include/linux/part_stat.h | 4 ++-- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/block/genhd.c b/block/genhd.c index 3e7df0a3e6bb0..1a76593276644 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -321,11 +321,12 @@ struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sector) struct hd_struct *part; int i; + rcu_read_lock(); ptbl = rcu_dereference(disk->part_tbl); part = rcu_dereference(ptbl->last_lookup); if (part && sector_in_part(part, sector) && hd_struct_try_get(part)) - return part; + goto out_unlock; for (i = 1; i < ptbl->len; i++) { part = rcu_dereference(ptbl->part[i]); @@ -339,10 +340,14 @@ struct hd_struct *disk_map_sector_rcu(struct gendisk *disk, sector_t sector) if (!hd_struct_try_get(part)) break; rcu_assign_pointer(ptbl->last_lookup, part); - return part; + goto out_unlock; } } - return &disk->part0; + + part = &disk->part0; +out_unlock: + rcu_read_unlock(); + return part; } /** diff --git a/include/linux/part_stat.h b/include/linux/part_stat.h index 6644197980b92..a6b0938ce82e9 100644 --- a/include/linux/part_stat.h +++ b/include/linux/part_stat.h @@ -21,8 +21,8 @@ struct disk_stats { * * part_stat_read() can be called at any time. */ -#define part_stat_lock() ({ rcu_read_lock(); get_cpu(); }) -#define part_stat_unlock() do { put_cpu(); rcu_read_unlock(); } while (0) +#define part_stat_lock() preempt_disable() +#define part_stat_unlock() preempt_enable() #define part_stat_get_cpu(part, field, cpu) \ (per_cpu_ptr((part)->dkstats, (cpu))->field) From patchwork Wed May 27 05:24:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571829 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7123739 for ; Wed, 27 May 2020 05:25:10 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 95A97208B8 for ; Wed, 27 May 2020 05:25:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="HgOzNOGA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 95A97208B8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9D652122679FE; Tue, 26 May 2020 22:21:01 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id BE30A122679FD for ; Tue, 26 May 2020 22:20:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=TB9a5RlSCttsRGqQbKXaUpDsmIDKmbhQMWP6LpH5XS4=; b=HgOzNOGAvq1hieB8BGbaYM8Yd6 OtocAuujspm6nkP05lUCMyukmzw/shOAkGR7H6vwElNxY37f5Gf6FH74JCcPV0KXohlfK46c5CIgq uKVpylnDhN13Tfwbx+8sF3cFuxHddwTW4sfegH7VWgZTbI3FlQh0trOMNHvm8ki30lCUyYaGnLCl9 MsfPEcsFZXBs3k0ckvvXDZ647AgGZTPg7G83k4yVSoZ97ChO88vK5GMnthTVY5zd0qyih6qBW39zM +J5YSkq2cRdoLxs4Af8aG+/+nZJwGO5OCweoM1adeVpWWaTOk7YX6pdcaCtyhmBG59XLnYknz5mbr PtQs+nWA==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYq-000141-Dn; Wed, 27 May 2020 05:25:04 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 15/16] block: use __this_cpu_add() instead of access by smp_processor_id() Date: Wed, 27 May 2020 07:24:18 +0200 Message-Id: <20200527052419.403583-16-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: AXJNHLZMO7YKD36ITVIKMTZJ6VCM55Y4 X-Message-ID-Hash: AXJNHLZMO7YKD36ITVIKMTZJ6VCM55Y4 X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: Konstantin Khlebnikov Most architectures have fast path to access percpu for current cpu. The required preempt_disable() is provided by part_stat_lock(). Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- include/linux/part_stat.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/part_stat.h b/include/linux/part_stat.h index a6b0938ce82e9..24125778ef3ec 100644 --- a/include/linux/part_stat.h +++ b/include/linux/part_stat.h @@ -54,7 +54,7 @@ static inline void part_stat_set_all(struct hd_struct *part, int value) part_stat_read(part, field[STAT_DISCARD])) #define __part_stat_add(part, field, addnd) \ - (part_stat_get(part, field) += (addnd)) + __this_cpu_add((part)->dkstats->field, addnd) #define part_stat_add(part, field, addnd) do { \ __part_stat_add((part), field, addnd); \ From patchwork Wed May 27 05:24:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 11571831 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0B0D739 for ; Wed, 27 May 2020 05:25:14 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9EBE0208B8 for ; Wed, 27 May 2020 05:25:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="sI1L1kUC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9EBE0208B8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B0037122679FF; Tue, 26 May 2020 22:21:05 -0700 (PDT) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=batv+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org; receiver= Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 725B512267A00 for ; Tue, 26 May 2020 22:21:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=1Ebl3DKp/tyRvnVEw2r5k4+wt6t2CIu1LhAwR5zP94s=; b=sI1L1kUCVsKPT3PJDiwWVR//KL jpR+grwUgE85PGxX9y/CxdzT08cfmmLLkpwY5YlQ1IiRxtcz131I/RxckdEudatS8oOdk1rK5jDh1 4N1XRMES+pKUaE+FgT7VIoPDChsaXTQvq1fCfGM2iD5Apq7tcTUv9Qa8IenWs7dVYDcZgx/But2TH g0EHNNrdoYUfnXT7xVC8/gwbhF9aV68JEdtrKApVj8FIWF4l+omJrx+sKsgCmU7cLPh0KyJHs8jyZ vqoGzcdPSCpAIt5QuUGzIdQ+uMPXaKkO+ABgHPQIOmgcT81JGpFCMOxkWY++OObYkDqijGF3goPbQ qw9YdcQg==; Received: from [2001:4bb8:18c:5da7:8164:affc:3c20:853d] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jdoYt-0001mp-Fh; Wed, 27 May 2020 05:25:07 +0000 From: Christoph Hellwig To: Jens Axboe Subject: [PATCH 16/16] block: reduce part_stat_lock() scope Date: Wed, 27 May 2020 07:24:19 +0200 Message-Id: <20200527052419.403583-17-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200527052419.403583-1-hch@lst.de> References: <20200527052419.403583-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Message-ID-Hash: IWLTKEU5BW57JNWX4JSRF3URZV7EY5FE X-Message-ID-Hash: IWLTKEU5BW57JNWX4JSRF3URZV7EY5FE X-MailFrom: BATV+2c76b185713af36e7087+6121+infradead.org+hch@bombadil.srs.infradead.org X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: Konstantin Khlebnikov , Minchan Kim , Nitin Gupta , dm-devel@redhat.com, linux-block@vger.kernel.org, drbd-dev@lists.linbit.com, linux-bcache@vger.kernel.org, linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: We only need the stats lock (aka preempt_disable()) for updating the states, not for looking up or dropping the hd_struct reference. Signed-off-by: Christoph Hellwig Reviewed-by: Konstantin Khlebnikov --- block/blk-core.c | 5 +++-- block/blk-merge.c | 3 ++- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index bf2f7d4bc0c1c..a01fb2b508f0e 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1437,9 +1437,9 @@ void blk_account_io_done(struct request *req, u64 now) update_io_ticks(part, jiffies, true); part_stat_inc(part, ios[sgrp]); part_stat_add(part, nsecs[sgrp], now - req->start_time_ns); + part_stat_unlock(); hd_struct_put(part); - part_stat_unlock(); } } @@ -1448,8 +1448,9 @@ void blk_account_io_start(struct request *rq) if (!blk_do_io_stat(rq)) return; - part_stat_lock(); rq->part = disk_map_sector_rcu(rq->rq_disk, blk_rq_pos(rq)); + + part_stat_lock(); update_io_ticks(rq->part, jiffies, false); part_stat_unlock(); } diff --git a/block/blk-merge.c b/block/blk-merge.c index c3beae5c1be71..f0b0bae075a0c 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -674,8 +674,9 @@ static void blk_account_io_merge_request(struct request *req) if (blk_do_io_stat(req)) { part_stat_lock(); part_stat_inc(req->part, merges[op_stat_group(req_op(req))]); - hd_struct_put(req->part); part_stat_unlock(); + + hd_struct_put(req->part); } }