From patchwork Thu Feb 10 22:38:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB2C4C433F5 for ; Thu, 10 Feb 2022 22:38:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345021AbiBJWil (ORCPT ); Thu, 10 Feb 2022 17:38:41 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238352AbiBJWik (ORCPT ); Thu, 10 Feb 2022 17:38:40 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 849B32664 for ; Thu, 10 Feb 2022 14:38:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532719; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=Aelm5YgrbgtRcjh/7giXHAljK3Fp2NuUOh2H4+pquFs=; b=GIgTagNhNNphI2bEY0NykcUZQ1lHThK9mx4fz2vdZgSiGrYLudGJaQBR2gIsHPyNgGDM2K jzH+5MgxhusCDUczk4snc2aPp73Tz/ZYm+Ew8EYMLfysRK4dDehbHOfc1JeKVc1p6AijT6 TXkxMxF9zRx8WxZwGLFJ+SG5HsrZUr0= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-536-iZrtfFaZPY2dlw7hMjGONw-1; Thu, 10 Feb 2022 17:38:36 -0500 X-MC-Unique: iZrtfFaZPY2dlw7hMjGONw-1 Received: by mail-qv1-f71.google.com with SMTP id a12-20020a056214062c00b0042c2f3fca04so4592085qvx.21 for ; Thu, 10 Feb 2022 14:38:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Aelm5YgrbgtRcjh/7giXHAljK3Fp2NuUOh2H4+pquFs=; b=y/Cw5EeszRuFn1ezQie5sllrSa0uTXbdtI7vy3sgaQxR4U78g6rqmpuUXRXJSrgOeC xBAbG7A0dHfCxk/ydjsNKvKRMAgtpMKmu1AtLfKlveKYukDMZbWJpaypgg0iqLZjcE+u KU9s1uu8eCehqTvCrl0vM3hvgEKQDBzDpsaJQuMULqLpoH2WsvCAU4FGjTqFSPyGMyRE u2MwIFEUY6m+XMOuS6QXsF0u3s0OVdoGI1SliLUH+RnDnWaR5gLkUCCx5MfE2hEPMu5H HWwUeR3eDyM6F5aJi73N+aJMfCxksCsLpmANINQWZMm47XlvhXq3m/eKJ0OSAB+GKUWL D6jA== X-Gm-Message-State: AOAM531EpRrl1/DP6ySK1MdfP8PcFYCaJalNR+g23xR5Ns7mAvVqZ4WS 1DPIpjSXaMEPW8jVvRG9+w3lcx6EE6uMh5HH7vhtRsLOm/WvpNOyvwuovnojfjQQVz8/XlZCRNM F/3HqRBrm9FQR6eMj9G6PeQ== X-Received: by 2002:a37:aa08:: with SMTP id t8mr5051594qke.773.1644532715637; Thu, 10 Feb 2022 14:38:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJyQixOwmD12PZY5UkjFPZ06AUK+aTARpLaV0mdw92GohG9RThiH7FoXYWT3NRL1ycsq0RFu0w== X-Received: by 2002:a37:aa08:: with SMTP id t8mr5051585qke.773.1644532715338; Thu, 10 Feb 2022 14:38:35 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id w5sm10477933qko.34.2022.02.10.14.38.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:34 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 01/14] dm: rename split functions Date: Thu, 10 Feb 2022 17:38:19 -0500 Message-Id: <20220210223832.99412-2-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Rename __split_and_process_bio to dm_split_and_process_bio. Rename __split_and_process_non_flush to __split_and_process_bio. Also fix a stale comment and whitespace. Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index ab9cc91931f9..2cecb8832936 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1359,7 +1359,7 @@ static bool __process_abnormal_io(struct clone_info *ci, struct dm_target *ti, /* * Select the correct strategy for processing a non-flush bio. */ -static int __split_and_process_non_flush(struct clone_info *ci) +static int __split_and_process_bio(struct clone_info *ci) { struct dm_target *ti; unsigned len; @@ -1395,8 +1395,8 @@ static void init_clone_info(struct clone_info *ci, struct mapped_device *md, /* * Entry point to split a bio into clones and submit them to the targets. */ -static void __split_and_process_bio(struct mapped_device *md, - struct dm_table *map, struct bio *bio) +static void dm_split_and_process_bio(struct mapped_device *md, + struct dm_table *map, struct bio *bio) { struct clone_info ci; int error = 0; @@ -1409,19 +1409,19 @@ static void __split_and_process_bio(struct mapped_device *md, } else if (op_is_zone_mgmt(bio_op(bio))) { ci.bio = bio; ci.sector_count = 0; - error = __split_and_process_non_flush(&ci); + error = __split_and_process_bio(&ci); } else { ci.bio = bio; ci.sector_count = bio_sectors(bio); - error = __split_and_process_non_flush(&ci); + error = __split_and_process_bio(&ci); if (ci.sector_count && !error) { /* * Remainder must be passed to submit_bio_noacct() * so that it gets handled *after* bios already submitted * have been completely processed. * We take a clone of the original to store in - * ci.io->orig_bio to be used by end_io_acct() and - * for dec_pending to use for completion handling. + * ci.io->orig_bio to be used by end_io_acct() and for + * dm_io_dec_pending() to use for completion handling. */ struct bio *b = bio_split(bio, bio_sectors(bio) - ci.sector_count, GFP_NOIO, &md->queue->bio_split); @@ -1470,7 +1470,7 @@ static void dm_submit_bio(struct bio *bio) if (is_abnormal_io(bio)) blk_queue_split(&bio); - __split_and_process_bio(md, map, bio); + dm_split_and_process_bio(md, map, bio); out: dm_put_live_table(md, srcu_idx); } @@ -2283,11 +2283,11 @@ static int __dm_suspend(struct mapped_device *md, struct dm_table *map, /* * Here we must make sure that no processes are submitting requests * to target drivers i.e. no one may be executing - * __split_and_process_bio from dm_submit_bio. + * dm_split_and_process_bio from dm_submit_bio. * - * To get all processes out of __split_and_process_bio in dm_submit_bio, + * To get all processes out of dm_split_and_process_bio in dm_submit_bio, * we take the write lock. To prevent any process from reentering - * __split_and_process_bio from dm_submit_bio and quiesce the thread + * dm_split_and_process_bio from dm_submit_bio and quiesce the thread * (dm_wq_work), we set DMF_BLOCK_IO_FOR_SUSPEND and call * flush_workqueue(md->wq). */ From patchwork Thu Feb 10 22:38:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC40EC433FE for ; Thu, 10 Feb 2022 22:38:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345012AbiBJWij (ORCPT ); Thu, 10 Feb 2022 17:38:39 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238352AbiBJWij (ORCPT ); Thu, 10 Feb 2022 17:38:39 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A37E92664 for ; Thu, 10 Feb 2022 14:38:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=gASqrtvOszXul4kd0cd64/f9seYYh7m9E4wnSmr4aF8=; b=WWDbVzTFf03vSHz9eFdBbUtQJEm4iUXd/HnNl78QJXZLHciPBeDk5O0pYKhFB9hzM44P8f cc9l8EmHKHD3b/QBhGNBBtCg+7MCoUPHMLb2EmrJKtyvz6ZEFqF7l4Shd1DrChVJw/0P5w mMjPoliec/OeLmQY96H+c/oOCBuDSPI= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-582-XIfGEU_9N-6Fy6cx7umvDA-1; Thu, 10 Feb 2022 17:38:37 -0500 X-MC-Unique: XIfGEU_9N-6Fy6cx7umvDA-1 Received: by mail-qv1-f71.google.com with SMTP id du13-20020a05621409ad00b0042c2949e2c1so4990692qvb.19 for ; Thu, 10 Feb 2022 14:38:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gASqrtvOszXul4kd0cd64/f9seYYh7m9E4wnSmr4aF8=; b=q17LQf8obUKQebXVmcgNyv4QhIxZldKbFbqDXeZeVd2898yvF06+54x8fq4CzaLL3V uR1oeXEOyHQoYzuJjjiVxVe5pY5INgTkSFep+3P550Jv/7fNuAh5/n1aif2x6I3yedp5 mZTb2uX7cLki1b2ML0J2/CEIVhAH8iiVNf+mRO7uVfwWOhDUsFGqwl1xOtpmHXhBn+7/ ikNOoq4j1UM5taMCJ80KGGNFv7YfyKPd3sJ20Rlg6E8N12PoEqzLo8q1zwVf2fdxCLag n9qTGgR/Wgf63VV/OtW6bpQfPmH9FYRTSUCnclZDEgo9AMxyFn5RLO8Ueqg7EOMcv4iV twxg== X-Gm-Message-State: AOAM530Rzw7EL8bmMyF59EWuyqTCrPj8D7dBrDmJJcJ5nDa5p7cTIx0R WwQDq7byc8sslEwEwexBuZ08EfM6Eu/MH6HwDXFxXZbpOz+iroBtQHC4uGOe2XDx+opKq2Xzqyb PIpodQ5qYLL5j5wbyqK1z7Q== X-Received: by 2002:a05:6214:262a:: with SMTP id gv10mr6836043qvb.98.1644532717061; Thu, 10 Feb 2022 14:38:37 -0800 (PST) X-Google-Smtp-Source: ABdhPJzshop5rhMcKKQyyYkS5tpUkVizfBc4b1Us/DFjfwBMEjhEWnkmSha5lLdwwyHGYJ/E/eeeTw== X-Received: by 2002:a05:6214:262a:: with SMTP id gv10mr6836035qvb.98.1644532716895; Thu, 10 Feb 2022 14:38:36 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id y15sm10527083qko.133.2022.02.10.14.38.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:36 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 02/14] dm: fold __clone_and_map_data_bio into __split_and_process_bio Date: Thu, 10 Feb 2022 17:38:20 -0500 Message-Id: <20220210223832.99412-3-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Fold __clone_and_map_data_bio into its only caller. Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm.c | 30 ++++++++---------------------- 1 file changed, 8 insertions(+), 22 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 2cecb8832936..2f1942b61d48 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1188,25 +1188,6 @@ static void bio_setup_sector(struct bio *bio, sector_t sector, unsigned len) bio->bi_iter.bi_size = to_bytes(len); } -/* - * Creates a bio that consists of range of complete bvecs. - */ -static int __clone_and_map_data_bio(struct clone_info *ci, struct dm_target *ti, - sector_t sector, unsigned *len) -{ - struct bio *bio = ci->bio, *clone; - - clone = alloc_tio(ci, ti, 0, len, GFP_NOIO); - bio_advance(clone, to_bytes(sector - clone->bi_iter.bi_sector)); - clone->bi_iter.bi_size = to_bytes(*len); - - if (bio_integrity(bio)) - bio_integrity_trim(clone); - - __map_bio(clone); - return 0; -} - static void alloc_multiple_bios(struct bio_list *blist, struct clone_info *ci, struct dm_target *ti, unsigned num_bios, unsigned *len) @@ -1361,6 +1342,7 @@ static bool __process_abnormal_io(struct clone_info *ci, struct dm_target *ti, */ static int __split_and_process_bio(struct clone_info *ci) { + struct bio *clone; struct dm_target *ti; unsigned len; int r; @@ -1374,9 +1356,13 @@ static int __split_and_process_bio(struct clone_info *ci) len = min_t(sector_t, max_io_len(ti, ci->sector), ci->sector_count); - r = __clone_and_map_data_bio(ci, ti, ci->sector, &len); - if (r < 0) - return r; + clone = alloc_tio(ci, ti, 0, &len, GFP_NOIO); + bio_advance(clone, to_bytes(ci->sector - clone->bi_iter.bi_sector)); + clone->bi_iter.bi_size = to_bytes(len); + if (bio_integrity(clone)) + bio_integrity_trim(clone); + + __map_bio(clone); ci->sector += len; ci->sector_count -= len; From patchwork Thu Feb 10 22:38:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 417DEC4332F for ; Thu, 10 Feb 2022 22:38:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238352AbiBJWil (ORCPT ); Thu, 10 Feb 2022 17:38:41 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345014AbiBJWil (ORCPT ); Thu, 10 Feb 2022 17:38:41 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 8588D267F for ; Thu, 10 Feb 2022 14:38:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532720; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=Dmbgb58sJS1NVsv2MiEVE8minE/+DuIbbyXB20Z/ekE=; b=E/yt9Fb2ar5YRYAso2OWXKf9lhj2BK/UAkroMVEEBkCqSDHvtdRFqMar/RBTfdYOn7b2HS KngeCAV8dwDJXfNphN1odV32uDPeWNz4mVQjX4C0ea2ZGpIHp/jqB3+ALVStu3R4kTpsoW zRU6fFuxCl0xYqPNdTSzvBwNrKOQ1xw= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-561-UaIb7MkANWycg3Mbx4ek4w-1; Thu, 10 Feb 2022 17:38:39 -0500 X-MC-Unique: UaIb7MkANWycg3Mbx4ek4w-1 Received: by mail-qt1-f198.google.com with SMTP id j6-20020ac85f86000000b002d9c2505d01so4593302qta.15 for ; Thu, 10 Feb 2022 14:38:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Dmbgb58sJS1NVsv2MiEVE8minE/+DuIbbyXB20Z/ekE=; b=dsqcYTzna7cX3MIOFXGpJabFEP3Lxh1/axt7n1QyezGi57zGamY8tUSALQm2y5lT6P oJosl9HCkezy3nWSAOpVrLTiEalv9Rnukgz7y1Hs4fbHdxC8OXtVZgxycQIG+aZ6sGYz jE3PSY/C9qnMz9ciZm1VPEmZTTXFivCM4YgRkXxx6hBG1DLg9UZOlRH5YH6DfvAmZfeN OWFmHBgB/YmwrOBBuXZ+UqNzg7p15CISRqL4kIlP5PNzvh452fyUh7ZxwoeTnYD7gbmY DAxfmybnh+pfsIRCOXxEquXtji7mqi1TJoXD46JuSNz7lIoZRh25n3zhatePkDPn/1l0 vtSw== X-Gm-Message-State: AOAM5309oX1ocfh79Ysi2vifVEn2GvhqB582KFbTtV3Uvd/oGNQXp6B7 oGlsV9fs4SM6xMu1tfslvgsJuRiodyCv1Kpuvf1PtUSZZb74PmBSHl2fyd3OFNC37K5TmohhWOH SYExEJZCVJovIsUeAE8hF3A== X-Received: by 2002:a37:aac9:: with SMTP id t192mr4866677qke.118.1644532718298; Thu, 10 Feb 2022 14:38:38 -0800 (PST) X-Google-Smtp-Source: ABdhPJzTEaXKZxCVPUG1VrVNIHN3jO/XBixkQ3uTQnAxaqJnd5AV3P7OpyaOaqfg8/NSZ6NUsUbCKA== X-Received: by 2002:a37:aac9:: with SMTP id t192mr4866671qke.118.1644532718037; Thu, 10 Feb 2022 14:38:38 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id f4sm10753431qko.72.2022.02.10.14.38.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:37 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 03/14] dm: refactor dm_split_and_process_bio a bit Date: Thu, 10 Feb 2022 17:38:21 -0500 Message-Id: <20220210223832.99412-4-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Remove needless branching. Leaves code to catch malformed op_is_zone_mgmt bios (they shouldn't have a payload). Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm.c | 51 ++++++++++++++++++++++++++------------------------- 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 2f1942b61d48..56734aae718d 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1375,7 +1375,13 @@ static void init_clone_info(struct clone_info *ci, struct mapped_device *md, { ci->map = map; ci->io = alloc_io(md, bio); + ci->bio = bio; ci->sector = bio->bi_iter.bi_sector; + ci->sector_count = bio_sectors(bio); + + /* Shouldn't happen but sector_count was being set to 0 so... */ + if (WARN_ON_ONCE(op_is_zone_mgmt(bio_op(bio)) && ci->sector_count)) + ci->sector_count = 0; } /* @@ -1392,34 +1398,29 @@ static void dm_split_and_process_bio(struct mapped_device *md, if (bio->bi_opf & REQ_PREFLUSH) { error = __send_empty_flush(&ci); /* dm_io_dec_pending submits any data associated with flush */ - } else if (op_is_zone_mgmt(bio_op(bio))) { - ci.bio = bio; - ci.sector_count = 0; - error = __split_and_process_bio(&ci); - } else { - ci.bio = bio; - ci.sector_count = bio_sectors(bio); - error = __split_and_process_bio(&ci); - if (ci.sector_count && !error) { - /* - * Remainder must be passed to submit_bio_noacct() - * so that it gets handled *after* bios already submitted - * have been completely processed. - * We take a clone of the original to store in - * ci.io->orig_bio to be used by end_io_acct() and for - * dm_io_dec_pending() to use for completion handling. - */ - struct bio *b = bio_split(bio, bio_sectors(bio) - ci.sector_count, - GFP_NOIO, &md->queue->bio_split); - ci.io->orig_bio = b; + goto out; + } - bio_chain(b, bio); - trace_block_split(b, bio->bi_iter.bi_sector); - submit_bio_noacct(bio); - } + error = __split_and_process_bio(&ci); + if (ci.sector_count && !error) { + /* + * Remainder must be passed to submit_bio_noacct() + * so that it gets handled *after* bios already submitted + * have been completely processed. + * We take a clone of the original to store in + * ci.io->orig_bio to be used by end_io_acct() and for + * dm_io_dec_pending() to use for completion handling. + */ + struct bio *b = bio_split(bio, bio_sectors(bio) - ci.sector_count, + GFP_NOIO, &md->queue->bio_split); + ci.io->orig_bio = b; + + bio_chain(b, bio); + trace_block_split(b, bio->bi_iter.bi_sector); + submit_bio_noacct(bio); } +out: start_io_acct(ci.io); - /* drop the extra reference count */ dm_io_dec_pending(ci.io, errno_to_blk_status(error)); } From patchwork Thu Feb 10 22:38:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE89EC433EF for ; Thu, 10 Feb 2022 22:38:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345024AbiBJWin (ORCPT ); Thu, 10 Feb 2022 17:38:43 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345014AbiBJWim (ORCPT ); Thu, 10 Feb 2022 17:38:42 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id A2FFA267F for ; Thu, 10 Feb 2022 14:38:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532721; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=N1ZtWuQD/2MiqgkrjoL8yazkFH5KXeFDL5OjLo4UsiQ=; b=FrP/i5qqnH1DFSHSYlOSOsbfjdi2GhPsN7Cxw/bE7fNaLeP/KpI3LBBpEAYGz/re9yOr1U b+5SmRWL/bpHzn7niyCnc0rscvjbRDOIhoaHsfcaL/yh3jExkDY8zaM1qushN4ljxAACDv two9SkOWVag/pMBZr3DgsigvkF7AvAA= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-270-vl5mJ_c2OFOBoeAlKIvQSw-1; Thu, 10 Feb 2022 17:38:40 -0500 X-MC-Unique: vl5mJ_c2OFOBoeAlKIvQSw-1 Received: by mail-qv1-f70.google.com with SMTP id du13-20020a05621409ad00b0042c2949e2c1so4990773qvb.19 for ; Thu, 10 Feb 2022 14:38:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=N1ZtWuQD/2MiqgkrjoL8yazkFH5KXeFDL5OjLo4UsiQ=; b=tDR/402hd/4MIfdMbVh5qsKrcvNvfRSrpwV88M1QgRk/Ja8PFbY6P8boG+HbtVUedI Sroo+PSVGMl27ITg623fFOY/uB/nyxlwLlCa++IsPJ+wk/QHj/DRsR5eQ8cthqlHH0pS JVhuRxSWWj+Py/Ii+vOfxXLKFh4HM43tPqmXcDWP8HeJuhBdMVkR0CV4e7+2STNNaFwA axQk5Ps3A1hb0J6mMf1Eujhx3UqM/zutcA761EP3iGCw2Bkf0UPJTgux2/DfIhi429ks QzKFD8FYywokvdogK9dAraAE2UZ5ZEKFgifVFokCtKgSIUaOi0olw7jNAj41Pj73SsSB rUuQ== X-Gm-Message-State: AOAM5319VkcAnG64z2NIrDVNzR492NiVbbHEMZvob4k95eHaXXkzm5Yo jDKbX2AQO+JcD26RN+SlbtpmF61DlkDnHmgFxGKspUbGd6ilHkNhtMb52geS2NWRQa3IW4t99EV jGxIZnJewGSu6aEOm2J9QHQ== X-Received: by 2002:ac8:7771:: with SMTP id h17mr6568557qtu.454.1644532719569; Thu, 10 Feb 2022 14:38:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJxVQD4DjXjI2TI6RHlCNfSHCzJOeMiWv0AytqCY43O5jsmCNUHGzFo0EsTCPdXZui5wk8R+dw== X-Received: by 2002:ac8:7771:: with SMTP id h17mr6568552qtu.454.1644532719407; Thu, 10 Feb 2022 14:38:39 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id h11sm2556932qkp.89.2022.02.10.14.38.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:39 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 04/14] dm: reduce code duplication in __map_bio Date: Thu, 10 Feb 2022 17:38:22 -0500 Message-Id: <20220210223832.99412-5-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Error path code (for handling DM_MAPIO_REQUEUE and DM_MAPIO_KILL) is effectively identical. Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 56734aae718d..cc014e56252e 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1161,20 +1161,14 @@ static void __map_bio(struct bio *clone) submit_bio_noacct(clone); break; case DM_MAPIO_KILL: - if (unlikely(swap_bios_limit(ti, clone))) { - struct mapped_device *md = io->md; - up(&md->swap_bios_semaphore); - } - free_tio(clone); - dm_io_dec_pending(io, BLK_STS_IOERR); - break; case DM_MAPIO_REQUEUE: - if (unlikely(swap_bios_limit(ti, clone))) { - struct mapped_device *md = io->md; - up(&md->swap_bios_semaphore); - } + if (unlikely(swap_bios_limit(ti, clone))) + up(&io->md->swap_bios_semaphore); free_tio(clone); - dm_io_dec_pending(io, BLK_STS_DM_REQUEUE); + if (r == DM_MAPIO_KILL) + dm_io_dec_pending(io, BLK_STS_IOERR); + else + dm_io_dec_pending(io, BLK_STS_DM_REQUEUE); break; default: DMWARN("unimplemented target map return value: %d", r); From patchwork Thu Feb 10 22:38:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DFE3C433F5 for ; Thu, 10 Feb 2022 22:38:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345014AbiBJWin (ORCPT ); Thu, 10 Feb 2022 17:38:43 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345022AbiBJWim (ORCPT ); Thu, 10 Feb 2022 17:38:42 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 700BD2664 for ; Thu, 10 Feb 2022 14:38:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532722; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=zWCBM8YnYusetp0OPiRYym7xOF2Me0YViFrrQcnA0W4=; b=cigcee3WcCGboGunEVl1ZU3TPUyg/vFsNE80ECsrxzXRT0EFKQzCUegkM7PxzObeZOFeFw Bq/lCqt8YpGsSUEykKrCpRnPskAqmxYZ1kbHLmKGQG4EMf2VYoncGfSqKE/VsRZ6j8jQsA YunX+jHBQ1VPE9cqBg+WJ3MslvxCL8E= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-624-L4JXrh2OMrSa4We90k7Xuw-1; Thu, 10 Feb 2022 17:38:41 -0500 X-MC-Unique: L4JXrh2OMrSa4We90k7Xuw-1 Received: by mail-qk1-f199.google.com with SMTP id i26-20020a05620a075a00b0047ec29823c0so4579650qki.6 for ; Thu, 10 Feb 2022 14:38:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zWCBM8YnYusetp0OPiRYym7xOF2Me0YViFrrQcnA0W4=; b=eXpOPcZ1szAW4UcUSvNC+8PafdF+TTchRaJ4l7/8Tf4ZAak89je8UV7wSrc1PPp/3T /b04jrOO1lYAsgY4zbZ0Z8GQgABC7g8TZGbracTQ6pa0ETfj+xcESo98+9ldYDeMGgk2 Sg6Z6N28Sd+d5RQbGNYf2Xdqovw9Ms2sBHuiAE30gQnNZQ/nQsAoHaqVqbTPolZz0bnn nozzoc6nHcpDgWwnB9CIZ3GUVmo3/g0EFG11Q8uMxvwv5J7yWFeXC5G+TJaoH+8L77MG xgfMajRPR9z/77TAKMsO1wu0PnTNpcowQt3XU/ufGtd+hnIp24Kb6Fb+8LVoUYhSqKE6 IIuw== X-Gm-Message-State: AOAM533xj1uZiWGGRPXyCZIK5IahMns+vF7f8NXE8g/oqFHIfzU+2qv/ VG9kOEEj9e//kChDUP2t3irVQxO9cRipYiwdn8NGYZp+hlDQ2+tAQFuI0M2tx+D8xCQccv7Y1od LENxhf2NLYJLLE/d6S5TZyw== X-Received: by 2002:a05:620a:8cd:: with SMTP id z13mr4860474qkz.487.1644532721014; Thu, 10 Feb 2022 14:38:41 -0800 (PST) X-Google-Smtp-Source: ABdhPJyRAUZfBBxSBcOHqSs0cu835y/n502phOPI93DcwbFy1t2ge6HipGrPT3511ZI+QNnRn+BDmg== X-Received: by 2002:a05:620a:8cd:: with SMTP id z13mr4860468qkz.487.1644532720833; Thu, 10 Feb 2022 14:38:40 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id w3sm11965918qta.13.2022.02.10.14.38.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:40 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 05/14] dm: remove impossible BUG_ON in __send_empty_flush Date: Thu, 10 Feb 2022 17:38:23 -0500 Message-Id: <20220210223832.99412-6-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org The flush_bio in question was just initialized to be empty, so there is no way bio_has_data() will retrun true. So remove stale BUG_ON(). Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index cc014e56252e..1985fc3f2a95 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1255,7 +1255,6 @@ static int __send_empty_flush(struct clone_info *ci) ci->bio = &flush_bio; ci->sector_count = 0; - BUG_ON(bio_has_data(ci->bio)); while ((ti = dm_table_get_target(ci->map, target_nr++))) __send_duplicate_bios(ci, ti, ti->num_flush_bios, NULL); From patchwork Thu Feb 10 22:38:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF718C433FE for ; Thu, 10 Feb 2022 22:38:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345025AbiBJWio (ORCPT ); Thu, 10 Feb 2022 17:38:44 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57508 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345022AbiBJWio (ORCPT ); Thu, 10 Feb 2022 17:38:44 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id F22D82664 for ; Thu, 10 Feb 2022 14:38:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=TzXjPgdywpvS4JKTWEPuvG6S5tjQ0HqbosRT6X11Ckc=; b=bfCoOBquYIQUJyMZUlzTi5hQT+tOxYEa+ePqIz/x8TACjGpsjRne7EHM/yeraxPMeBZ1rI VsSmFJ+DvLJU44U+h1JCfEZ3soba3yAD4nMJvdAEHxY5LV+NhA0ZI+W3QRq15iF5bjd0aq BFDgBNi6OKUcVF4Z8SAiTxkOFCZYgtA= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-458-DSzhdcofMkKWGvNh4ULfAg-1; Thu, 10 Feb 2022 17:38:43 -0500 X-MC-Unique: DSzhdcofMkKWGvNh4ULfAg-1 Received: by mail-qk1-f198.google.com with SMTP id z205-20020a3765d6000000b0047db3b020dfso4521445qkb.22 for ; Thu, 10 Feb 2022 14:38:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TzXjPgdywpvS4JKTWEPuvG6S5tjQ0HqbosRT6X11Ckc=; b=yDa3waKWseD9XjwztGQSTLUAMVU+DNEWtp7SO+YHMNnC5U2GXaLHDJyTGF5fZZDC4D 8eCo9G0OD/vrvtsvqhD4tSkCeM3AufAWcHa8eEi/tA16CvV4JktEzaQ65OtoKKifI8+y 44WyvjGdGqL/g4NQ/pb8J7evY+7tLJu60arnY35Glw/FviF1JgxgQKZOYip0t+7MGrSW tygYeenWMQ1Dm6ty960pw4MKH+WxQnGTB79zDRb65VQtwnHR9YTRSX8MTxVJvVr6eAlJ uo3lhQJby/9ICB8zKo5Vec6ulSvj5vi8rFnkVuDapMsmLwTXPUtSVG989kOuDe+PG8B/ hwfw== X-Gm-Message-State: AOAM531HU3s1MeODq1VoPyqczTNKxdeEVS1QKNBwqfuH9qRT+HD8v7Gh VqM1pKiDhAljqXZPmjt0mAXt7nHJ0W9A0++S8HSSU007T3dW5qndUV2is7fP2cSmKi0QXNDgs51 BGLK/EeTMJ+Ah2XdfMZ2pwQ== X-Received: by 2002:a05:6214:5084:: with SMTP id kk4mr6496956qvb.87.1644532722332; Thu, 10 Feb 2022 14:38:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJwdUxlHbTKVegt7q96UOVkLZzblVev6PndTCx99qOFkY4DYSKF6tEc5hRZE8V6mEfZmaLcoVA== X-Received: by 2002:a05:6214:5084:: with SMTP id kk4mr6496949qvb.87.1644532722175; Thu, 10 Feb 2022 14:38:42 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id v14sm10322732qkl.128.2022.02.10.14.38.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:41 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 06/14] dm: remove unused mapped_device argument from free_tio Date: Thu, 10 Feb 2022 17:38:24 -0500 Message-Id: <20220210223832.99412-7-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 1985fc3f2a95..f091bbf8a8dc 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -539,7 +539,7 @@ static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio) return io; } -static void free_io(struct mapped_device *md, struct dm_io *io) +static void free_io(struct dm_io *io) { bio_put(&io->tio.clone); } @@ -825,7 +825,7 @@ void dm_io_dec_pending(struct dm_io *io, blk_status_t error) io_error = io->status; start_time = io->start_time; stats_aux = io->stats_aux; - free_io(md, io); + free_io(io); end_io_acct(md, bio, start_time, &stats_aux); if (io_error == BLK_STS_DM_REQUEUE) From patchwork Thu Feb 10 22:38:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F452C433F5 for ; Thu, 10 Feb 2022 22:38:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345028AbiBJWir (ORCPT ); Thu, 10 Feb 2022 17:38:47 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345022AbiBJWir (ORCPT ); Thu, 10 Feb 2022 17:38:47 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C32702664 for ; Thu, 10 Feb 2022 14:38:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532727; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=DEABWOQAG5wChjoRVdmD9+4tJqi002FLqoS+HHDd0E8=; b=Vr/VJqaOg1oqHZSK/mtUuA68HaMDLweWwMBzA8GswOzCrtrmGS+t+PS4Fl/LNLqDmHZKIl TEpMIPRa9ak4MGxjTn1sN479OSLupGcNjrMJtpHgnx09EgltwJKp33+xXNmJoDR1liXsH/ U118mib5CXtprjZy9c/W3CaN26CdsUs= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-74-wOoIMIQVOMWZymbz9910XQ-1; Thu, 10 Feb 2022 17:38:44 -0500 X-MC-Unique: wOoIMIQVOMWZymbz9910XQ-1 Received: by mail-qk1-f197.google.com with SMTP id p23-20020a05620a15f700b00506d8ec3749so4592689qkm.4 for ; Thu, 10 Feb 2022 14:38:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DEABWOQAG5wChjoRVdmD9+4tJqi002FLqoS+HHDd0E8=; b=dgXSl/7zni+ly+eB2bmUUIgGfDlaPUsB3TNXFtDRPJKi4R8eC59xuB4Lnudlj9dcRh Bscb8P0YhnGPjS/eOQ310Ie5ZIqJhrIAHnydkqEB/dwao3Ym9CSKgTEThaXf9L28nfEi fxysQs0I8UOg6D4913JPILPHJykkFeMbkdEzRuiSIZihcszeqvCsNPJ8EaZIgDtAAPqs qPKjj2Ro/mg0NwzESV8WjB+NUnRByLMJDcsUX9jspDmIDAoG10r3Wjik/434/6sv7oHx TKeg9KAt4Zk3p4AQrhLreR7aFwl+xms1WfXtUSX+WJ+8YRI5TfXRNqNhEk+YMxkq/kM7 50dw== X-Gm-Message-State: AOAM532u4fk+tWzBJ/ZAf+4ftpXTUFfdaL94Hq9rKoWLqubql9T1jdUk 25eWI8aYKsnmTWhVlqkbkMxQ9XOrr2YUg2cRbEs3HQC89EbHotGQXGo3G/md4eV/cNO9FStIMGL Xoy6BMK2QFAMXRiFFPP8Jcw== X-Received: by 2002:a05:620a:2954:: with SMTP id n20mr4949256qkp.644.1644532723815; Thu, 10 Feb 2022 14:38:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJwB/jwDj6FI1/7jk/S/HO15AQAsoYUPunogWnQegdbM0VkAmgE3KfBpXMwIwDBuvLYdBmPAtw== X-Received: by 2002:a05:620a:2954:: with SMTP id n20mr4949253qkp.644.1644532723624; Thu, 10 Feb 2022 14:38:43 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id n7sm11816185qta.78.2022.02.10.14.38.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:43 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 07/14] dm: remove code only needed before submit_bio recursion Date: Thu, 10 Feb 2022 17:38:25 -0500 Message-Id: <20220210223832.99412-8-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Commit 8615cb65bd63 ("dm: remove useless loop in __split_and_process_bio") showcased that we no longer loop. Remove the bio_advance() in __split_and_process_bio() that was only needed when looping was possible. Similarly there is no need to advance the bio, using ci->sector cursor, in __send_duplicate_bios(). Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm.c | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index f091bbf8a8dc..5950d518e544 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1176,12 +1176,6 @@ static void __map_bio(struct bio *clone) } } -static void bio_setup_sector(struct bio *bio, sector_t sector, unsigned len) -{ - bio->bi_iter.bi_sector = sector; - bio->bi_iter.bi_size = to_bytes(len); -} - static void alloc_multiple_bios(struct bio_list *blist, struct clone_info *ci, struct dm_target *ti, unsigned num_bios, unsigned *len) @@ -1224,14 +1218,14 @@ static void __send_duplicate_bios(struct clone_info *ci, struct dm_target *ti, case 1: clone = alloc_tio(ci, ti, 0, len, GFP_NOIO); if (len) - bio_setup_sector(clone, ci->sector, *len); + clone->bi_iter.bi_size = to_bytes(*len); __map_bio(clone); break; default: alloc_multiple_bios(&blist, ci, ti, num_bios, len); while ((clone = bio_list_pop(&blist))) { if (len) - bio_setup_sector(clone, ci->sector, *len); + clone->bi_iter.bi_size = to_bytes(*len); __map_bio(clone); } break; @@ -1350,7 +1344,6 @@ static int __split_and_process_bio(struct clone_info *ci) len = min_t(sector_t, max_io_len(ti, ci->sector), ci->sector_count); clone = alloc_tio(ci, ti, 0, &len, GFP_NOIO); - bio_advance(clone, to_bytes(ci->sector - clone->bi_iter.bi_sector)); clone->bi_iter.bi_size = to_bytes(len); if (bio_integrity(clone)) bio_integrity_trim(clone); From patchwork Thu Feb 10 22:38:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0C89C433EF for ; Thu, 10 Feb 2022 22:38:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345022AbiBJWis (ORCPT ); Thu, 10 Feb 2022 17:38:48 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345026AbiBJWir (ORCPT ); Thu, 10 Feb 2022 17:38:47 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 40146267F for ; Thu, 10 Feb 2022 14:38:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532727; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=IfXmk5i0i3prdNRYxcELXekeRQ6SpDfxxN4oTGtZnNk=; b=ZswXJv+i1jnRYtVHjd95iU7q1Xfdq1ijuJePIRMEzlVOw2/TDd8JV8dqI333T1yJJi/T4i 1+lbGMSstf0vFGKcrjiOCOMPODS/c1KSlM4AdExDTo7yP+8Ehwcn0MalIiSOW1n43/dvqz GAVPRVkqrEVcxd3UY83Dat1iV7B8U2g= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-86-Dwy0aWArOrCYaOrYS_eCKw-1; Thu, 10 Feb 2022 17:38:46 -0500 X-MC-Unique: Dwy0aWArOrCYaOrYS_eCKw-1 Received: by mail-qv1-f72.google.com with SMTP id eo11-20020ad4594b000000b0042151b7180aso5025197qvb.8 for ; Thu, 10 Feb 2022 14:38:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IfXmk5i0i3prdNRYxcELXekeRQ6SpDfxxN4oTGtZnNk=; b=L1rT5MV1gie9aksuGUu5ygh14Q/Jui76+2j+huYOGM9tvc4tQMH+zEVfGK8Bi+XpDY Al34C5S1mUdeTUEn1y+DpWjr/H9p87I5SLXgw0mqGlAmvORm1tRCVh0Yu5dVd/UPaLMm 2suf2RyJ78Mx3A6kGo82Jlig42By5kUDQHkkgHolK/eZgETA5O3Fgqu2SnBZ/RVCQFbV 7foSYHbnFucXSr3VAUvMOLS8WLB5M9i9lEKUBs8O1qZ440ZLH1XHF94n4gDcq/aL5vHT PaqurUtl2RNH0/JfZL79jg7822T5EPgMlVPKN7KwT2tQgTC9WVkuhR0yt587Zc0Lso6a jw2w== X-Gm-Message-State: AOAM530u8psdnxU4Bk5eFCR9Smx9p8z9b39xL1ekNhtsMSxM4qUzPE1e nrdXm8jBuKeulUByRx6zr5Z+46Tv+mfdVn/HMNFVE1lIKKfTJxZLtdVqQzrNW8r1XOkNyRi8BZ5 EerCiND/DubxspduzbERj5A== X-Received: by 2002:a05:620a:4012:: with SMTP id h18mr5121613qko.421.1644532725503; Thu, 10 Feb 2022 14:38:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJw/WuBM1MWf014PQYWdlKPzXr9gGf3iIEk91KFNZ30dmljK3pCiGA6TTM3Dqo0OPQkq2MwW1w== X-Received: by 2002:a05:620a:4012:: with SMTP id h18mr5121607qko.421.1644532725259; Thu, 10 Feb 2022 14:38:45 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id c3sm6575052qkp.39.2022.02.10.14.38.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:44 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 08/14] dm: record old_sector in dm_target_io before calling map function Date: Thu, 10 Feb 2022 17:38:26 -0500 Message-Id: <20220210223832.99412-9-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Prep for being able to defer trace_block_bio_remap() until when the bio is remapped and submitted by the DM target. Signed-off-by: Mike Snitzer Reviewed-by: Christoph Hellwig --- drivers/md/dm-core.h | 1 + drivers/md/dm.c | 7 ++++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index 72d18c3fbf1f..f40be01cca81 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -214,6 +214,7 @@ struct dm_target_io { unsigned int target_bio_nr; unsigned int *len_ptr; bool inside_dm_io; + sector_t old_sector; struct bio clone; }; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 5950d518e544..3bd872b0e891 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -567,6 +567,7 @@ static struct bio *alloc_tio(struct clone_info *ci, struct dm_target *ti, tio->ti = ti; tio->target_bio_nr = target_bio_nr; tio->len_ptr = len; + tio->old_sector = 0; return &tio->clone; } @@ -1120,7 +1121,6 @@ static void __map_bio(struct bio *clone) { struct dm_target_io *tio = clone_to_tio(clone); int r; - sector_t sector; struct dm_io *io = tio->io; struct dm_target *ti = tio->ti; @@ -1132,7 +1132,7 @@ static void __map_bio(struct bio *clone) * this io. */ dm_io_inc_pending(io); - sector = clone->bi_iter.bi_sector; + tio->old_sector = clone->bi_iter.bi_sector; if (unlikely(swap_bios_limit(ti, clone))) { struct mapped_device *md = io->md; @@ -1157,7 +1157,8 @@ static void __map_bio(struct bio *clone) break; case DM_MAPIO_REMAPPED: /* the bio has been remapped so dispatch it */ - trace_block_bio_remap(clone, bio_dev(io->orig_bio), sector); + trace_block_bio_remap(clone, bio_dev(io->orig_bio), + tio->old_sector); submit_bio_noacct(clone); break; case DM_MAPIO_KILL: From patchwork Thu Feb 10 22:38:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4302C433FE for ; Thu, 10 Feb 2022 22:38:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345029AbiBJWit (ORCPT ); Thu, 10 Feb 2022 17:38:49 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345026AbiBJWit (ORCPT ); Thu, 10 Feb 2022 17:38:49 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9E9BF2664 for ; Thu, 10 Feb 2022 14:38:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532728; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=NJDDntpX1bDz2HHZg4ERdrCl6N9td/COm3CwT3DXHyU=; b=ePhkOheqtvJLVchAgzAB6zuGalMH5BQu+swFoy8ewLziMmGVvnWnd8a1BWbbPTwU87rCcH EQkC1Y2QwouamgTwmFg+ndqVdfG4RdQ1M68xFePt3t6WhjNzrQDHY5Gd5iGw7sCW9ABNYH zM1YALZBblZYEd+8u1KDS2OKNNiOYNw= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-241-u28ktMO6OGKGctNmkogoSg-1; Thu, 10 Feb 2022 17:38:47 -0500 X-MC-Unique: u28ktMO6OGKGctNmkogoSg-1 Received: by mail-qt1-f200.google.com with SMTP id j6-20020ac85f86000000b002d9c2505d01so4593527qta.15 for ; Thu, 10 Feb 2022 14:38:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NJDDntpX1bDz2HHZg4ERdrCl6N9td/COm3CwT3DXHyU=; b=GYv6uHasZdNvNDKJuLBolSU+afpdGXoERUAaXuxG0pFR7aXDPdk8hhxD2ccBSyemKE uXUGT8NWMg+SdozTJn/PEVvrxyuv+m+CgML4t8TJBTQx6TiEIfn0X1LYI7X5C+YhanYq 1iYbhk0NqpRMb70k0kbWLGMPe4wMdvqB5WXDwIRDBF8Q+vDwIkN6cvCLM2IoSHT156GS wUws42cDW1fpld52uBhLraT/EoAAkA6X9hSfTc4yVCuPYQAaXCrs2qGCQW/inaZU3uvj MuyHrNmsTMO8EW+TY1n5qdFEIPg0pLJfNz1d11mkmntCNCqBJh+xl/mkfj2x6IYfW61f twSQ== X-Gm-Message-State: AOAM530/j1pexHkVv6km+vyXPolKisbVbSoIMVkF/jxSSo92Cs45Rsvs 8YlqbRbL2JXTKLRcqzNx422exfhFtgkbERh7sHEl7G7UbrYKC9pOfc5g0nzBrtjivm6KfRZ80KV ibqfw0S5/ampxuhZa5tc/+Q== X-Received: by 2002:a05:6214:2308:: with SMTP id gc8mr6697704qvb.108.1644532726814; Thu, 10 Feb 2022 14:38:46 -0800 (PST) X-Google-Smtp-Source: ABdhPJx/swioXpnZqj/ggEJ7xdDFKsnoeOBasBsyZ13Up+u3iQ8ukuq5/OcxD8Z9zGj3LPPEVFxxxw== X-Received: by 2002:a05:6214:2308:: with SMTP id gc8mr6697693qvb.108.1644532726554; Thu, 10 Feb 2022 14:38:46 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id z4sm12115514qtw.4.2022.02.10.14.38.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:46 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 09/14] dm: prep for following changes Date: Thu, 10 Feb 2022 17:38:27 -0500 Message-Id: <20220210223832.99412-10-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Rename dm_io struct's 'endio_lock' member to 'lock' to reuse spinlock to protect new member used to flag if IO accounting has been started. Also move kicking of the suspend queue out to dm_io_dec_pending (the only caller) since end_io_acct will soon only be called if IO accounting was started. Some comment tweaks and removal of local variables. No functional change. Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 +- drivers/md/dm.c | 32 ++++++++++++++------------------ 2 files changed, 15 insertions(+), 19 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index f40be01cca81..8dd196aec130 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -230,7 +230,7 @@ struct dm_io { atomic_t io_count; struct bio *orig_bio; unsigned long start_time; - spinlock_t endio_lock; + spinlock_t lock; struct dm_stats_aux stats_aux; /* last member of dm_target_io is 'struct bio' */ struct dm_target_io tio; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 3bd872b0e891..8c0e96b8e1a5 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -487,12 +487,12 @@ EXPORT_SYMBOL_GPL(dm_start_time_ns_from_clone); static void start_io_acct(struct dm_io *io) { - struct mapped_device *md = io->md; struct bio *bio = io->orig_bio; bio_start_io_acct_time(bio, io->start_time); - if (unlikely(dm_stats_used(&md->stats))) - dm_stats_account_io(&md->stats, bio_data_dir(bio), + + if (unlikely(dm_stats_used(&io->md->stats))) + dm_stats_account_io(&io->md->stats, bio_data_dir(bio), bio->bi_iter.bi_sector, bio_sectors(bio), false, 0, &io->stats_aux); } @@ -500,18 +500,12 @@ static void start_io_acct(struct dm_io *io) static void end_io_acct(struct mapped_device *md, struct bio *bio, unsigned long start_time, struct dm_stats_aux *stats_aux) { - unsigned long duration = jiffies - start_time; - bio_end_io_acct(bio, start_time); if (unlikely(dm_stats_used(&md->stats))) dm_stats_account_io(&md->stats, bio_data_dir(bio), bio->bi_iter.bi_sector, bio_sectors(bio), - true, duration, stats_aux); - - /* nudge anyone waiting on suspend queue */ - if (unlikely(wq_has_sleeper(&md->wait))) - wake_up(&md->wait); + true, jiffies - start_time, stats_aux); } static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio) @@ -532,7 +526,7 @@ static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio) atomic_set(&io->io_count, 1); io->orig_bio = bio; io->md = md; - spin_lock_init(&io->endio_lock); + spin_lock_init(&io->lock); io->start_time = jiffies; @@ -796,10 +790,10 @@ void dm_io_dec_pending(struct dm_io *io, blk_status_t error) /* Push-back supersedes any I/O errors */ if (unlikely(error)) { - spin_lock_irqsave(&io->endio_lock, flags); + spin_lock_irqsave(&io->lock, flags); if (!(io->status == BLK_STS_DM_REQUEUE && __noflush_suspending(md))) io->status = error; - spin_unlock_irqrestore(&io->endio_lock, flags); + spin_unlock_irqrestore(&io->lock, flags); } if (atomic_dec_and_test(&io->io_count)) { @@ -829,6 +823,10 @@ void dm_io_dec_pending(struct dm_io *io, blk_status_t error) free_io(io); end_io_acct(md, bio, start_time, &stats_aux); + /* nudge anyone waiting on suspend queue */ + if (unlikely(wq_has_sleeper(&md->wait))) + wake_up(&md->wait); + if (io_error == BLK_STS_DM_REQUEUE) return; @@ -1127,9 +1125,7 @@ static void __map_bio(struct bio *clone) clone->bi_end_io = clone_endio; /* - * Map the clone. If r == 0 we don't need to do - * anything, the target has assumed ownership of - * this io. + * Map the clone. */ dm_io_inc_pending(io); tio->old_sector = clone->bi_iter.bi_sector; @@ -1154,6 +1150,7 @@ static void __map_bio(struct bio *clone) switch (r) { case DM_MAPIO_SUBMITTED: + /* target has assumed ownership of this io */ break; case DM_MAPIO_REMAPPED: /* the bio has been remapped so dispatch it */ @@ -1301,10 +1298,9 @@ static bool is_abnormal_io(struct bio *bio) static bool __process_abnormal_io(struct clone_info *ci, struct dm_target *ti, int *result) { - struct bio *bio = ci->bio; unsigned num_bios = 0; - switch (bio_op(bio)) { + switch (bio_op(ci->bio)) { case REQ_OP_DISCARD: num_bios = ti->num_discard_bios; break; From patchwork Thu Feb 10 22:38:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7F73C433F5 for ; Thu, 10 Feb 2022 22:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345026AbiBJWiw (ORCPT ); Thu, 10 Feb 2022 17:38:52 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345030AbiBJWiw (ORCPT ); Thu, 10 Feb 2022 17:38:52 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 799D6267F for ; Thu, 10 Feb 2022 14:38:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532731; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=clg1j9AsWGf13nZ+cL57xlbQXn238yrRMLtUXnWXd9Q=; b=U8AyYiF5QJbPl4iNPt8CPHpfljTxFc6Q/p5P8htPqs32l+mzYtsNHwTksGR19tt+wWfRxw GuF39lGhgzGP4DjZOGUWnaT8Yt8dAc4j61sg4lHigaEsul0CbSMkACD8sB32plDD+NLyCi FlnpwH/H5EnWkBPV01pJuui33G1Be0w= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-584-bkBE7R7VN6ieUV5dBD8J-g-1; Thu, 10 Feb 2022 17:38:49 -0500 X-MC-Unique: bkBE7R7VN6ieUV5dBD8J-g-1 Received: by mail-qv1-f69.google.com with SMTP id t3-20020a0562140c6300b00424a0fd3721so5014019qvj.12 for ; Thu, 10 Feb 2022 14:38:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=clg1j9AsWGf13nZ+cL57xlbQXn238yrRMLtUXnWXd9Q=; b=cWVIh4Uz15TixlQaZ/fqxYNthQ3kvZnJpJDZoyKXPgjL06CLrtBvPGXBcrZWxIsTzl NKEFEW9vvB3BOGgr+nVvM2KCIZ5o2A3pBffY1fjjoGErbBH3RwWMBst1mg1hx5IYRmLf EjJHpEViZ77sKBFtD4k0fhBpXV7esMY1NnXKnr2UxRzcOCR5dmwF766ICieeeiiSAKbf Fr8/O/6Z4uMXtWaB+otxFYjUQJ+SIfFDMD5p28/T1fQjcYHqAUJxruAdEfwf3m/13BCD qIce21+d3+fXs5usDRocCcPTBfeMumwPlwl+0PIP26eQbFS8AbADLYUFYGER1ibnN+KO X3Sw== X-Gm-Message-State: AOAM53031Npa0hTV7+2ibTrg/lfJJc+NZ0Zyme9QPEy+ts9uYffmEQSw T2DUPQoJhWsbEsiEarttG4Ulh7GbeVfrxTmkabIED+uTkgHn7mL2/dFJ4SPQeQsM7Gq29rmj4Po YdzbTySqJBQuW5UZReylKNA== X-Received: by 2002:a37:61ca:: with SMTP id v193mr5007960qkb.177.1644532728446; Thu, 10 Feb 2022 14:38:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJz2IldN0gw3kZyxdPqUqit4+42CSSY2elFZp9qQdt60LjZzqSwEFWIWb3gl3fnkNs62ee8TQw== X-Received: by 2002:a37:61ca:: with SMTP id v193mr5007952qkb.177.1644532728122; Thu, 10 Feb 2022 14:38:48 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id w14sm12344783qtc.29.2022.02.10.14.38.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:47 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 10/14] dm: add dm_submit_bio_remap interface Date: Thu, 10 Feb 2022 17:38:28 -0500 Message-Id: <20220210223832.99412-11-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Switch from early bio-based IO accounting (at the time DM clones each incoming bio) to late IO accounting just before each remapped bio is issued to underlying device via submit_bio_noacct(). Allows more precise bio-based IO accounting for DM targets that use their own workqueues to perform additional processing of each bio in conjunction with their DM_MAPIO_SUBMITTED return from their map function. Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 1 + drivers/md/dm.c | 93 +++++++++++++++++++++++++++++++++++++++---- include/linux/device-mapper.h | 7 ++++ 3 files changed, 93 insertions(+), 8 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index 8dd196aec130..3ecd6f294f53 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -230,6 +230,7 @@ struct dm_io { atomic_t io_count; struct bio *orig_bio; unsigned long start_time; + unsigned long io_acct_time; spinlock_t lock; struct dm_stats_aux stats_aux; /* last member of dm_target_io is 'struct bio' */ diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 8c0e96b8e1a5..ad512f40716e 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -485,21 +485,54 @@ u64 dm_start_time_ns_from_clone(struct bio *bio) } EXPORT_SYMBOL_GPL(dm_start_time_ns_from_clone); -static void start_io_acct(struct dm_io *io) +static void __start_io_acct(struct dm_io *io, struct bio *bio) { - struct bio *bio = io->orig_bio; + unsigned long flags; - bio_start_io_acct_time(bio, io->start_time); + /* Ensure IO accounting is only ever started once */ + spin_lock_irqsave(&io->lock, flags); + if (smp_load_acquire(&io->io_acct_time)) { + spin_unlock_irqrestore(&io->lock, flags); + return; + } + smp_store_release(&io->io_acct_time, jiffies); + spin_unlock_irqrestore(&io->lock, flags); + bio_start_io_acct_time(bio, io->start_time); if (unlikely(dm_stats_used(&io->md->stats))) dm_stats_account_io(&io->md->stats, bio_data_dir(bio), bio->bi_iter.bi_sector, bio_sectors(bio), false, 0, &io->stats_aux); } +static void start_io_acct(struct dm_io *io, struct bio *bio) +{ + /* Only start_io_acct() once for this IO */ + if (smp_load_acquire(&io->io_acct_time)) + return; + + __start_io_acct(io, bio); +} + +static void clone_and_start_io_acct(struct dm_io *io, struct bio *bio) +{ + struct bio io_acct_clone; + + /* Only clone_and_start_io_acct() once for this IO */ + if (smp_load_acquire(&io->io_acct_time)) + return; + + bio_init_clone(io->orig_bio->bi_bdev, + &io_acct_clone, bio, GFP_NOIO); + __start_io_acct(io, &io_acct_clone); +} + static void end_io_acct(struct mapped_device *md, struct bio *bio, unsigned long start_time, struct dm_stats_aux *stats_aux) { + if (!start_time) + return; + bio_end_io_acct(bio, start_time); if (unlikely(dm_stats_used(&md->stats))) @@ -529,6 +562,7 @@ static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio) spin_lock_init(&io->lock); io->start_time = jiffies; + io->io_acct_time = 0; return io; } @@ -818,7 +852,8 @@ void dm_io_dec_pending(struct dm_io *io, blk_status_t error) } io_error = io->status; - start_time = io->start_time; + if (io->io_acct_time) + start_time = io->start_time; stats_aux = io->stats_aux; free_io(io); end_io_acct(md, bio, start_time, &stats_aux); @@ -1099,6 +1134,43 @@ void dm_accept_partial_bio(struct bio *bio, unsigned n_sectors) } EXPORT_SYMBOL_GPL(dm_accept_partial_bio); +/* + * @clone: clone bio that DM core passed to target's .map function + * @tgt_clone: bio that target needs to submit (after DM_MAPIO_SUBMITTED) + * + * Targets should use this interface to submit bios they take + * ownership of when returning DM_MAPIO_SUBMITTED. + * + * Target should also enable ti->accounts_remapped_io + */ +void dm_submit_bio_remap(struct bio *clone, struct bio *tgt_clone) +{ + struct dm_target_io *tio = clone_to_tio(clone); + struct dm_io *io = tio->io; + struct block_device *clone_bdev = clone->bi_bdev; + + /* establish bio that will get submitted */ + if (!tgt_clone) + tgt_clone = clone; + + /* + * account IO to DM device in terms of clone's + * payload to avoid concern about late bio splitting. + * - clone will reflect any dm_accept_partial_bio() + * - any bio splitting is ultimately reflected in + * io->orig_bio so there is no IO imbalance in + * end_io_acct(). + */ + clone->bi_bdev = io->orig_bio->bi_bdev; + start_io_acct(io, clone); + clone->bi_bdev = clone_bdev; + + trace_block_bio_remap(tgt_clone, bio_dev(io->orig_bio), + tio->old_sector); + submit_bio_noacct(tgt_clone); +} +EXPORT_SYMBOL_GPL(dm_submit_bio_remap); + static noinline void __set_swap_bios_limit(struct mapped_device *md, int latch) { mutex_lock(&md->swap_bios_lock); @@ -1151,12 +1223,18 @@ static void __map_bio(struct bio *clone) switch (r) { case DM_MAPIO_SUBMITTED: /* target has assumed ownership of this io */ + if (!ti->accounts_remapped_io) { + /* + * Any split isn't reflected in io->orig_bio yet. And bio + * cannot be modified because target is submitting it. + * Clone bio and account IO to DM device. + */ + clone_and_start_io_acct(io, clone); + } break; case DM_MAPIO_REMAPPED: /* the bio has been remapped so dispatch it */ - trace_block_bio_remap(clone, bio_dev(io->orig_bio), - tio->old_sector); - submit_bio_noacct(clone); + dm_submit_bio_remap(clone, NULL); break; case DM_MAPIO_KILL: case DM_MAPIO_REQUEUE: @@ -1403,7 +1481,6 @@ static void dm_split_and_process_bio(struct mapped_device *md, submit_bio_noacct(bio); } out: - start_io_acct(ci.io); /* drop the extra reference count */ dm_io_dec_pending(ci.io, errno_to_blk_status(error)); } diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index b26fecf6c8e8..a3e397155bc9 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -362,6 +362,12 @@ struct dm_target { * zone append operations using regular writes. */ bool emulate_zone_append:1; + + /* + * Set if the target will submit IO using dm_submit_bio_remap() + * after returning DM_MAPIO_SUBMITTED from its map function. + */ + bool accounts_remapped_io:1; }; void *dm_per_bio_data(struct bio *bio, size_t data_size); @@ -465,6 +471,7 @@ int dm_suspended(struct dm_target *ti); int dm_post_suspending(struct dm_target *ti); int dm_noflush_suspending(struct dm_target *ti); void dm_accept_partial_bio(struct bio *bio, unsigned n_sectors); +void dm_submit_bio_remap(struct bio *clone, struct bio *tgt_clone); union map_info *dm_get_rq_mapinfo(struct request *rq); #ifdef CONFIG_BLK_DEV_ZONED From patchwork Thu Feb 10 22:38:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64446C433EF for ; Thu, 10 Feb 2022 22:38:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345031AbiBJWiw (ORCPT ); Thu, 10 Feb 2022 17:38:52 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345026AbiBJWiv (ORCPT ); Thu, 10 Feb 2022 17:38:51 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2932D2664 for ; Thu, 10 Feb 2022 14:38:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532731; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=QSJblWqLgyZmlZxEVd+22+iLGNHpkwCdE5Y4iGn/nKk=; b=B75ltYWfx6YwdcMKvPiBLipoM4wJYMkJ9i9xcRjzBaUOtes2kh9BwRgFgpImH4tU/8Q9jV hdrFrD19U4zQ3f8UiJcirIPjhrprwB4hOwOi4biDLRTgb7GTe8eTtvNyok3DnZBvzflU/J mLoeQ2ASwPjAPC9bfj30PYT7sIThxY8= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-155--OMCaOwqPy29gx78bCWKuw-1; Thu, 10 Feb 2022 17:38:50 -0500 X-MC-Unique: -OMCaOwqPy29gx78bCWKuw-1 Received: by mail-qt1-f197.google.com with SMTP id d25-20020ac84e39000000b002d1cf849207so5513244qtw.19 for ; Thu, 10 Feb 2022 14:38:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QSJblWqLgyZmlZxEVd+22+iLGNHpkwCdE5Y4iGn/nKk=; b=rUBhUKdsCRcnt9kzfDGGqrEuPqo5gHQFR8Th3ofC2zFQZdtfFMYPuGLOXEUhOmVV+R IW7Lwi/nV4f9v5sWXrsriiiXyltgoWLmyyjHXfHEftFmjE+/tAVAPKFFgETC+p9JfjSy HkVXjKa+PMue1E0nM7VMei8/L6M1r0SXcEoMBakePJLPbZUNTTUrM9+z1lQywCAuoWTr NVEHWR1m/ov/kzKxVYx66pEb2GfqGnEbjjU1O+QxoHyZRGjMZJSSigA1p4/EnrepfuEm 4dKsb7Q6aVOkg3Pe7w+QKft9snOC8G0+xQovVn+6O+H1OC5PBSHSCY1zB+3TCgwO3Z6k grwg== X-Gm-Message-State: AOAM531ovQV2Z2rxYZGuFIrOmZJln+iexgnsTGWTvX2sWcLoSsyuTK66 QYRIfp0hr5MZPP3WHfsEEC79mYPx2PzSgrVfKIhTrNwzxx07iH8wI0KmAEf3oYk0xXb7Wydbcac TkLWibT5cEHRkTqSGqJGacQ== X-Received: by 2002:a05:622a:411:: with SMTP id n17mr6496115qtx.466.1644532729490; Thu, 10 Feb 2022 14:38:49 -0800 (PST) X-Google-Smtp-Source: ABdhPJzT+PtvXuOy9fF5SaJVeCa3zFUy2819JDuwEm2PCy3GAfUwDOgDHA08+EaJap4V4vdpA0o8KQ== X-Received: by 2002:a05:622a:411:: with SMTP id n17mr6496110qtx.466.1644532729298; Thu, 10 Feb 2022 14:38:49 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id j11sm11447294qtx.67.2022.02.10.14.38.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:48 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 11/14] dm crypt: use dm_submit_bio_remap Date: Thu, 10 Feb 2022 17:38:29 -0500 Message-Id: <20220210223832.99412-12-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Signed-off-by: Mike Snitzer --- drivers/md/dm-crypt.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index a5006cb6ee8a..9ea197de08c2 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -1855,7 +1855,7 @@ static int kcryptd_io_read(struct dm_crypt_io *io, gfp_t gfp) return 1; } - submit_bio_noacct(clone); + dm_submit_bio_remap(io->base_bio, clone); return 0; } @@ -1881,7 +1881,7 @@ static void kcryptd_io_write(struct dm_crypt_io *io) { struct bio *clone = io->ctx.bio_out; - submit_bio_noacct(clone); + dm_submit_bio_remap(io->base_bio, clone); } #define crypt_io_from_node(node) rb_entry((node), struct dm_crypt_io, rb_node) @@ -1960,7 +1960,7 @@ static void kcryptd_crypt_write_io_submit(struct dm_crypt_io *io, int async) if ((likely(!async) && test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags)) || test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags)) { - submit_bio_noacct(clone); + dm_submit_bio_remap(io->base_bio, clone); return; } @@ -3363,6 +3363,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) ti->num_flush_bios = 1; ti->limit_swap_bios = true; + ti->accounts_remapped_io = true; dm_audit_log_ctr(DM_MSG_PREFIX, ti, 1); return 0; From patchwork Thu Feb 10 22:38:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 436A6C433FE for ; Thu, 10 Feb 2022 22:38:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345030AbiBJWix (ORCPT ); Thu, 10 Feb 2022 17:38:53 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345032AbiBJWiw (ORCPT ); Thu, 10 Feb 2022 17:38:52 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 62BEC2664 for ; Thu, 10 Feb 2022 14:38:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532732; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=CRfDTYNpxGAOLZDzgeFhYGqT+avXL5fj+x7+6gQLvzM=; b=VPy9sfQBKJGIAaT7m/Mhn1n6s4m93/9ka6AIkjgYHRcbI8bxketT05uhGJm0yjI2+hyFMG Pw1l2B8Za5WC2rSq2469sJjgnm/OsrD+BZ9gRoKBFDHqFwqx384GeJRI8EgfXknAWb8oBt UPf7RqIxQ6hAxLVNffDtR7Dv+l1scgM= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-642-PYCwf096NJ6ySnd8exE2qw-1; Thu, 10 Feb 2022 17:38:51 -0500 X-MC-Unique: PYCwf096NJ6ySnd8exE2qw-1 Received: by mail-qv1-f71.google.com with SMTP id ge15-20020a05621427cf00b00421df9f8f23so4991218qvb.17 for ; Thu, 10 Feb 2022 14:38:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CRfDTYNpxGAOLZDzgeFhYGqT+avXL5fj+x7+6gQLvzM=; b=bswW8RoeP+H7YzEePxXtGNDiCt0uEuYerwnUD+t04uTdeTZqrpwCpouvXlpg8Q1qIa 9rN4m7vD9JlqjXipGH2ait4MeTyYO1rIuhZ7j8/YunXS0nytSOd/bPebiuHtpZu6qf9w jNmKrBMdnhz4K77yL6H7ocO4iCnUxlXgdJBKRKBlQA4YO2w+Np1EFEnv/cL/aImL6Kpk T6vJvP2fZE0GXjspqxwnoOAoaw0eOV4dzj8m3lpHk9dEPxHMDzme0hAikTsFq6F+0VCn jIBES5BrCLahgVlpbNx/wPptIWpOf8LnHdE5v63GRAW+K9DVK2JfCCKcE5Bl2A1GA2C1 fn4Q== X-Gm-Message-State: AOAM5336GepJGYxpTQWgzpl+9y289PrCU9io17MAe8SUusl2VsdwXMaI pHlkOzsFfri95kcD3cXwk9+mEmLz+4H8JUQ99Vdjew7otIIa5ZBFrq03cM5nkX8l3hYyOLFZ54S dXZi2muYw2iIHMnjc7JaVOg== X-Received: by 2002:a37:a607:: with SMTP id p7mr5034685qke.763.1644532730799; Thu, 10 Feb 2022 14:38:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJyxEVIGFiZwGx+fh0BLy5zq1Vh8eVtyQH52OipWCoMyOw9uzoENBF0MFCF33ybEO88rYgtWaw== X-Received: by 2002:a37:a607:: with SMTP id p7mr5034680qke.763.1644532730613; Thu, 10 Feb 2022 14:38:50 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id bj24sm10469903qkb.115.2022.02.10.14.38.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:50 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 12/14] dm delay: dm_submit_bio_remap Date: Thu, 10 Feb 2022 17:38:30 -0500 Message-Id: <20220210223832.99412-13-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Signed-off-by: Mike Snitzer --- drivers/md/dm-delay.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/md/dm-delay.c b/drivers/md/dm-delay.c index 59e51d285b0e..8235927a3912 100644 --- a/drivers/md/dm-delay.c +++ b/drivers/md/dm-delay.c @@ -72,7 +72,7 @@ static void flush_bios(struct bio *bio) while (bio) { n = bio->bi_next; bio->bi_next = NULL; - submit_bio_noacct(bio); + dm_submit_bio_remap(bio, NULL); bio = n; } } @@ -232,6 +232,7 @@ static int delay_ctr(struct dm_target *ti, unsigned int argc, char **argv) ti->num_flush_bios = 1; ti->num_discard_bios = 1; + ti->accounts_remapped_io = true; ti->per_io_data_size = sizeof(struct dm_delay_info); return 0; From patchwork Thu Feb 10 22:38:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D80D5C4332F for ; Thu, 10 Feb 2022 22:38:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345033AbiBJWiy (ORCPT ); Thu, 10 Feb 2022 17:38:54 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345032AbiBJWiy (ORCPT ); Thu, 10 Feb 2022 17:38:54 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id AFA932664 for ; Thu, 10 Feb 2022 14:38:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532733; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=+Mnt9RTq0AtjFUsH7/0mhHCJN7nnx7AZj3Cr8laEsbg=; b=VbKD4yuqpCse32BCJBrCdxkvn0Mfc00NSIUVKB74wfMk2Ugee/AKsFOvR87t0ccvwEeQtu BmoJBteVT2QrWthOOPbzZTqQ9Hma0DRX1pWU3/MzUCybvvK1AJ1qICM+stLYqv6iYRklwK yYcaGca0twul6hDHgENreNxViLTrvyU= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-616-349IyoPtNkSJ2j6Sw6h0cQ-1; Thu, 10 Feb 2022 17:38:52 -0500 X-MC-Unique: 349IyoPtNkSJ2j6Sw6h0cQ-1 Received: by mail-qk1-f200.google.com with SMTP id b18-20020a05620a089200b004e0e2f73f35so4533859qka.19 for ; Thu, 10 Feb 2022 14:38:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+Mnt9RTq0AtjFUsH7/0mhHCJN7nnx7AZj3Cr8laEsbg=; b=Mc/54WlLLQKHLOfvVvhY42J4A+Db0fMitZRXa+brlCtpOZhuVZ8YdPUUWfkiizUT2R LeOm6Qbx962cNFY1IJuRNuNNHhn+xBp6gVvt1bs9cAWPX1OvND29DaS9vIiYuW+dcKxS +070d0RqOI1NCqEBbTGXwWBPcJcbqxiqcNkf67cKJhqnNW6j9RaAbgfrwbJLs9FgZPBW xNEwBv6kmvaEjzNfICc6OI6fNZy5NlPQpqHsGB0ienNHulPl6Xi0IEBcpXGvQZ4EggLq a/TUNQm6Wl/bfEIlRiIgz5NEPvahrWEDRGOcoPOm44EuDObOTatKmLxAjHtama+CbvPv xFVA== X-Gm-Message-State: AOAM533V/6gHqdLb9dUOADJotb6zxWaP7pUcwb2RyNPdWkumGMKkhbZ9 AlHN05lz9/2rS3MqFx17K+yf9KsNwPZoJv5OAqGvH72veEXYsJ8BE6WUxB9vGS48bt5mfs0NPIP xfp/Ee8Vx/5h4V04Abqff7Q== X-Received: by 2002:a05:6214:2306:: with SMTP id gc6mr6868145qvb.63.1644532731983; Thu, 10 Feb 2022 14:38:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJyamo0E431a2/YSMBqGadwIEB+1DQaLXFNobXLpuclNidyN4EcKw4raCDde0o/OY6oGwrdnag== X-Received: by 2002:a05:6214:2306:: with SMTP id gc6mr6868140qvb.63.1644532731828; Thu, 10 Feb 2022 14:38:51 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id r3sm10312080qkm.56.2022.02.10.14.38.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:51 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 13/14] dm: improve correctness and efficiency of bio-based IO accounting Date: Thu, 10 Feb 2022 17:38:31 -0500 Message-Id: <20220210223832.99412-14-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Don't use jiffies as a glorified bool because jiffies can/will rollover to 0. Also use xchg(), instead of spin_lock_irq{save,restore} and smp_load_acquire/smp_store_release, to avoid performance impact of disabling and enabling interrupts. Suggested-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 +- drivers/md/dm.c | 34 ++++++++++++---------------------- 2 files changed, 13 insertions(+), 23 deletions(-) diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h index 3ecd6f294f53..d3c116866fd7 100644 --- a/drivers/md/dm-core.h +++ b/drivers/md/dm-core.h @@ -230,7 +230,7 @@ struct dm_io { atomic_t io_count; struct bio *orig_bio; unsigned long start_time; - unsigned long io_acct_time; + int was_accounted; spinlock_t lock; struct dm_stats_aux stats_aux; /* last member of dm_target_io is 'struct bio' */ diff --git a/drivers/md/dm.c b/drivers/md/dm.c index ad512f40716e..329f0be64523 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -487,17 +487,6 @@ EXPORT_SYMBOL_GPL(dm_start_time_ns_from_clone); static void __start_io_acct(struct dm_io *io, struct bio *bio) { - unsigned long flags; - - /* Ensure IO accounting is only ever started once */ - spin_lock_irqsave(&io->lock, flags); - if (smp_load_acquire(&io->io_acct_time)) { - spin_unlock_irqrestore(&io->lock, flags); - return; - } - smp_store_release(&io->io_acct_time, jiffies); - spin_unlock_irqrestore(&io->lock, flags); - bio_start_io_acct_time(bio, io->start_time); if (unlikely(dm_stats_used(&io->md->stats))) dm_stats_account_io(&io->md->stats, bio_data_dir(bio), @@ -507,8 +496,8 @@ static void __start_io_acct(struct dm_io *io, struct bio *bio) static void start_io_acct(struct dm_io *io, struct bio *bio) { - /* Only start_io_acct() once for this IO */ - if (smp_load_acquire(&io->io_acct_time)) + /* Ensure IO accounting is only ever started once */ + if (xchg(&io->was_accounted, 1) == 1) return; __start_io_acct(io, bio); @@ -518,8 +507,8 @@ static void clone_and_start_io_acct(struct dm_io *io, struct bio *bio) { struct bio io_acct_clone; - /* Only clone_and_start_io_acct() once for this IO */ - if (smp_load_acquire(&io->io_acct_time)) + /* Ensure IO accounting is only ever started once */ + if (xchg(&io->was_accounted, 1) == 1) return; bio_init_clone(io->orig_bio->bi_bdev, @@ -530,9 +519,6 @@ static void clone_and_start_io_acct(struct dm_io *io, struct bio *bio) static void end_io_acct(struct mapped_device *md, struct bio *bio, unsigned long start_time, struct dm_stats_aux *stats_aux) { - if (!start_time) - return; - bio_end_io_acct(bio, start_time); if (unlikely(dm_stats_used(&md->stats))) @@ -562,7 +548,7 @@ static struct dm_io *alloc_io(struct mapped_device *md, struct bio *bio) spin_lock_init(&io->lock); io->start_time = jiffies; - io->io_acct_time = 0; + io->was_accounted = 0; return io; } @@ -819,6 +805,7 @@ void dm_io_dec_pending(struct dm_io *io, blk_status_t error) blk_status_t io_error; struct bio *bio; struct mapped_device *md = io->md; + bool was_accounted = false; unsigned long start_time = 0; struct dm_stats_aux stats_aux; @@ -852,11 +839,14 @@ void dm_io_dec_pending(struct dm_io *io, blk_status_t error) } io_error = io->status; - if (io->io_acct_time) + if (io->was_accounted) { + was_accounted = true; start_time = io->start_time; - stats_aux = io->stats_aux; + stats_aux = io->stats_aux; + } free_io(io); - end_io_acct(md, bio, start_time, &stats_aux); + if (was_accounted) + end_io_acct(md, bio, start_time, &stats_aux); /* nudge anyone waiting on suspend queue */ if (unlikely(wq_has_sleeper(&md->wait))) From patchwork Thu Feb 10 22:38:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 12742493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 255E3C433F5 for ; Thu, 10 Feb 2022 22:38:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345036AbiBJWi4 (ORCPT ); Thu, 10 Feb 2022 17:38:56 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:57604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345032AbiBJWi4 (ORCPT ); Thu, 10 Feb 2022 17:38:56 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 42A272664 for ; Thu, 10 Feb 2022 14:38:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644532735; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=64mfGr+BkNDL/qyiDObOyXjA+jYS7+jce6ArkgUIKnM=; b=d+YDSjsKBTRiq53DFnXfC1ahIFeQZlR+rR2k+9C4buJhTewmWV1IcHnjUkbfY+9SsAuah9 f0boDPiZpMspj7hmv4ZRCXWO3ysJe9ObWHAzo3E/5yEYobuZ17+LWpjRDtbOZy5+8WPVyK jGiUorm2G4Ubep8V3ptn8NyEd4KQHoo= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-447-GYcVAw12PoW0NF11as0S7g-1; Thu, 10 Feb 2022 17:38:54 -0500 X-MC-Unique: GYcVAw12PoW0NF11as0S7g-1 Received: by mail-qv1-f71.google.com with SMTP id ge15-20020a05621427cf00b00421df9f8f23so4991287qvb.17 for ; Thu, 10 Feb 2022 14:38:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=64mfGr+BkNDL/qyiDObOyXjA+jYS7+jce6ArkgUIKnM=; b=Cpi4s6gX03EEd3pjXTqlOZqJ3Hk98x/faQ/n5vsXTnoMzMBelxZVWRIzxDUV01ZXoB jL+iF3PHsBEiZ3DAxp1Y6FhgoBo3V/rDxWmLusxDBPVbDA0PDnb6AqGNPoxrdpkpuHiX OFo32uiWiH/60ydeL0k0XTvH3Pu8+UfdyEg8d+zTDvbczTAb61vByciM+YxVXUcvaW4S gi5mSQo8CHMepbFh6HSKqziQxv32mnyZSaQEPOBqYMqO4YYCbABq9dhhNvQjIWu9LCb4 YUyX7zzGK7sNKnpxvEB3vlXTCbyQWmfFtPW0ICeEhlxlUJn3/EMOoGMiB9OaQ+awB09s hlFw== X-Gm-Message-State: AOAM532FEfc5rqRcmbyY/8OqE9J/TUqrVqn/KUMvwlCHSNrGFRLPZcR/ cFbrzZ8Fa+h3Z5I8NXbPbM4UxBUUXgEAv+2dRr3hE6iRKWZKycZuVTsp4DZLVDnY88XxREjsZt6 26U5guqPGjJBgGnygcCMtzQ== X-Received: by 2002:a05:622a:1303:: with SMTP id v3mr6633036qtk.294.1644532733425; Thu, 10 Feb 2022 14:38:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJyiq9A/dl8/a5+pxZQ8CSXaYG/GlW7bPYTGMKHlZVSuct7m0MfGG+eetHtVr60p3O6UF0//mw== X-Received: by 2002:a05:622a:1303:: with SMTP id v3mr6633025qtk.294.1644532733141; Thu, 10 Feb 2022 14:38:53 -0800 (PST) Received: from localhost (pool-68-160-176-52.bstnma.fios.verizon.net. [68.160.176.52]) by smtp.gmail.com with ESMTPSA id q12sm12007982qtx.51.2022.02.10.14.38.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 14:38:52 -0800 (PST) From: Mike Snitzer To: dm-devel@redhat.com Cc: linux-block@vger.kernel.org Subject: [PATCH 14/14] block: add bio_start_io_acct_remapped for the benefit of DM Date: Thu, 10 Feb 2022 17:38:32 -0500 Message-Id: <20220210223832.99412-15-snitzer@redhat.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20220210223832.99412-1-snitzer@redhat.com> References: <20220210223832.99412-1-snitzer@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org DM needs the ability to account a clone bio's IO to the original block_device. So add @orig_bdev argument to bio_start_io_acct_time. Rename bio_start_io_acct_time to bio_start_io_acct_remapped. Also, follow bio_end_io_acct and bio_end_io_acct_remapped pattern by moving bio_start_io_acct to blkdev.h and have it call bio_start_io_acct_remapped. Improve DM to no longer need to play games with swizzling a clone bio's bi_bdev (in dm_submit_bio_remap) and remove DM's clone_and_start_io_acct() interface. Signed-off-by: Mike Snitzer --- block/blk-core.c | 24 ++++++++---------------- drivers/md/dm.c | 41 ++++++++--------------------------------- include/linux/blkdev.h | 16 ++++++++++++++-- 3 files changed, 30 insertions(+), 51 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index be8812f5489d..8f23be96c737 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1077,29 +1077,21 @@ static unsigned long __part_start_io_acct(struct block_device *part, } /** - * bio_start_io_acct_time - start I/O accounting for bio based drivers + * bio_start_io_acct_remapped - start I/O accounting for bio based drivers * @bio: bio to start account for * @start_time: start time that should be passed back to bio_end_io_acct(). - */ -void bio_start_io_acct_time(struct bio *bio, unsigned long start_time) -{ - __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), - bio_op(bio), start_time); -} -EXPORT_SYMBOL_GPL(bio_start_io_acct_time); - -/** - * bio_start_io_acct - start I/O accounting for bio based drivers - * @bio: bio to start account for + * @orig_bdev: block device that I/O must be accounted to. * * Returns the start time that should be passed back to bio_end_io_acct(). */ -unsigned long bio_start_io_acct(struct bio *bio) +unsigned long bio_start_io_acct_remapped(struct bio *bio, + unsigned long start_time, + struct block_device *orig_bdev) { - return __part_start_io_acct(bio->bi_bdev, bio_sectors(bio), - bio_op(bio), jiffies); + return __part_start_io_acct(orig_bdev, bio_sectors(bio), + bio_op(bio), start_time); } -EXPORT_SYMBOL_GPL(bio_start_io_acct); +EXPORT_SYMBOL_GPL(bio_start_io_acct_remapped); unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors, unsigned int op) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 329f0be64523..e020f505e243 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -485,35 +485,19 @@ u64 dm_start_time_ns_from_clone(struct bio *bio) } EXPORT_SYMBOL_GPL(dm_start_time_ns_from_clone); -static void __start_io_acct(struct dm_io *io, struct bio *bio) -{ - bio_start_io_acct_time(bio, io->start_time); - if (unlikely(dm_stats_used(&io->md->stats))) - dm_stats_account_io(&io->md->stats, bio_data_dir(bio), - bio->bi_iter.bi_sector, bio_sectors(bio), - false, 0, &io->stats_aux); -} - static void start_io_acct(struct dm_io *io, struct bio *bio) { /* Ensure IO accounting is only ever started once */ if (xchg(&io->was_accounted, 1) == 1) return; - __start_io_acct(io, bio); -} + bio_start_io_acct_remapped(bio, io->start_time, + io->orig_bio->bi_bdev); -static void clone_and_start_io_acct(struct dm_io *io, struct bio *bio) -{ - struct bio io_acct_clone; - - /* Ensure IO accounting is only ever started once */ - if (xchg(&io->was_accounted, 1) == 1) - return; - - bio_init_clone(io->orig_bio->bi_bdev, - &io_acct_clone, bio, GFP_NOIO); - __start_io_acct(io, &io_acct_clone); + if (unlikely(dm_stats_used(&io->md->stats))) + dm_stats_account_io(&io->md->stats, bio_data_dir(bio), + bio->bi_iter.bi_sector, bio_sectors(bio), + false, 0, &io->stats_aux); } static void end_io_acct(struct mapped_device *md, struct bio *bio, @@ -1137,7 +1121,6 @@ void dm_submit_bio_remap(struct bio *clone, struct bio *tgt_clone) { struct dm_target_io *tio = clone_to_tio(clone); struct dm_io *io = tio->io; - struct block_device *clone_bdev = clone->bi_bdev; /* establish bio that will get submitted */ if (!tgt_clone) @@ -1151,9 +1134,7 @@ void dm_submit_bio_remap(struct bio *clone, struct bio *tgt_clone) * io->orig_bio so there is no IO imbalance in * end_io_acct(). */ - clone->bi_bdev = io->orig_bio->bi_bdev; start_io_acct(io, clone); - clone->bi_bdev = clone_bdev; trace_block_bio_remap(tgt_clone, bio_dev(io->orig_bio), tio->old_sector); @@ -1213,14 +1194,8 @@ static void __map_bio(struct bio *clone) switch (r) { case DM_MAPIO_SUBMITTED: /* target has assumed ownership of this io */ - if (!ti->accounts_remapped_io) { - /* - * Any split isn't reflected in io->orig_bio yet. And bio - * cannot be modified because target is submitting it. - * Clone bio and account IO to DM device. - */ - clone_and_start_io_acct(io, clone); - } + if (!ti->accounts_remapped_io) + start_io_acct(io, clone); break; case DM_MAPIO_REMAPPED: /* the bio has been remapped so dispatch it */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 3bfc75a2a450..31d055d4a17e 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1512,11 +1512,23 @@ unsigned long disk_start_io_acct(struct gendisk *disk, unsigned int sectors, void disk_end_io_acct(struct gendisk *disk, unsigned int op, unsigned long start_time); -void bio_start_io_acct_time(struct bio *bio, unsigned long start_time); -unsigned long bio_start_io_acct(struct bio *bio); +unsigned long bio_start_io_acct_remapped(struct bio *bio, + unsigned long start_time, + struct block_device *orig_bdev); void bio_end_io_acct_remapped(struct bio *bio, unsigned long start_time, struct block_device *orig_bdev); +/** + * bio_start_io_acct - start I/O accounting for bio based drivers + * @bio: bio to start account for + * + * Returns the start time that should be passed back to bio_end_io_acct(). + */ +static inline unsigned long bio_start_io_acct(struct bio *bio) +{ + return bio_start_io_acct_remapped(bio, jiffies, bio->bi_bdev); +} + /** * bio_end_io_acct - end I/O accounting for bio based drivers * @bio: bio to end account for