From patchwork Thu Jan 18 03:33:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Snitzer X-Patchwork-Id: 10172335 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F1319601E7 for ; Thu, 18 Jan 2018 03:33:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E45431FFF9 for ; Thu, 18 Jan 2018 03:33:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D6E10204C4; Thu, 18 Jan 2018 03:33:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7AFCE1FFF9 for ; Thu, 18 Jan 2018 03:33:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753279AbeARDdj (ORCPT ); Wed, 17 Jan 2018 22:33:39 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59162 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753189AbeARDdj (ORCPT ); Wed, 17 Jan 2018 22:33:39 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 47283C049D53; Thu, 18 Jan 2018 03:33:39 +0000 (UTC) Received: from localhost (ovpn-112-37.rdu2.redhat.com [10.10.112.37]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 82BC4177E5; Thu, 18 Jan 2018 03:33:36 +0000 (UTC) Date: Wed, 17 Jan 2018 22:33:35 -0500 From: Mike Snitzer To: Bart Van Assche , Laurence Oberman Cc: "axboe@kernel.dk" , "hch@lst.de" , "dm-devel@redhat.com" , "linux-block@vger.kernel.org" , "tom.leiming@gmail.com" Subject: [PATCH] blk mq: don't blk_mq_request_bypass_insert _and_ return BLK_STS_RESOURCE Message-ID: <20180118033334.GA30338@redhat.com> References: <20180117162558.28553-1-snitzer@redhat.com> <45f70a28-aeea-8278-0b8e-9601ea043595@kernel.dk> <20180117165832.GA5573@redhat.com> <1516231868.2820.84.camel@wdc.com> <1516232589.3805.2.camel@redhat.com> <1516233206.2820.103.camel@wdc.com> <20180118005452.GB7649@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180118005452.GB7649@redhat.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 18 Jan 2018 03:33:39 +0000 (UTC) Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Wed, Jan 17 2018 at 7:54P -0500, Mike Snitzer wrote: > But sure, I suppose there is something I missed when refactoring Ming's > change to get it acceptable for upstream. I went over the mechanical > nature of what I did many times (comparing Ming's v4 to my v5). And yes there is one subtlety that I missed. > The call to blk_mq_request_bypass_insert will only occur via > __blk_mq_fallback_to_insert. Which as the name implies this is not the > fast path. This will occur if the underlying blk-mq device cannot get > resources it needs in order to issue the request. Specifically: if/when > in __blk_mq_try_issue_directly() the hctx is stopped, or queue is > quiesced, or it cannot get the driver tag or dispatch_budget (in the > case of scsi-mq). > > The same fallback, via call to blk_mq_request_bypass_insert, occured > with Ming's v4 though. Turns out Ming's v4 doesn't fallback to insert for the "or it cannot get the driver tag or dispatch_budget" case. This patch should fix it (Laurence, please report back on if this fixes your list_add corruption, pretty sure it will): From: Mike Snitzer Date: Wed, 17 Jan 2018 22:02:07 -0500 Subject: [PATCH] blk mq: don't blk_mq_request_bypass_insert _and_ return BLK_STS_RESOURCE It isn't ever valid to call blk_mq_request_bypass_insert() and return BLK_STS_RESOURCE. Unfortunately after commit 396eaf21ee ("blk-mq: improve DM's blk-mq IO merging via blk_insert_cloned_request feedback") we do just that if blk_mq_request_direct_issue() cannot get the resources (driver_tag or dispatch_budget) needed to directly issue a request. This will lead to "list_add corruption" because blk-mq submits the IO but then reports that it didn't (BLK_STS_RESOURCE in this case). Fix this by simply returning BLK_STS_RESOURCE for this case. Fixes: 396eaf21ee ("blk-mq: improve DM's blk-mq IO merging via blk_insert_cloned_request feedback") Reported-by: Laurence Oberman Signed-off-by: Mike Snitzer --- block/blk-mq.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index c418858a60ef..8bee37239255 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1799,20 +1799,18 @@ static blk_status_t __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, if (q->elevator && !bypass_insert) goto insert; - if (!blk_mq_get_driver_tag(rq, NULL, false)) - goto insert; - - if (!blk_mq_get_dispatch_budget(hctx)) { + if (!blk_mq_get_driver_tag(rq, NULL, false) || + !blk_mq_get_dispatch_budget(hctx)) { + /* blk_mq_put_driver_tag() is idempotent */ blk_mq_put_driver_tag(rq); + if (bypass_insert) + return BLK_STS_RESOURCE; goto insert; } return __blk_mq_issue_directly(hctx, rq, cookie); insert: __blk_mq_fallback_to_insert(rq, run_queue, bypass_insert); - if (bypass_insert) - return BLK_STS_RESOURCE; - return BLK_STS_OK; }