From patchwork Sat Mar 8 16:23:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14007586 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F8351DEFEB for ; Sat, 8 Mar 2025 16:23:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451031; cv=none; b=jNXGM4QsGWdwH9WgKcr0QshS86cdZIGt/pApid4DMA9vHfg2eOi2f113wSmr6JOPUZvOJ+T1u1j6/OIr90WBUBs31zn/u9jqhiP50ZXqyl+40l8PVhPjwZgOrvq8cLL2056YXOuwNR3Xst2us3UU1aufvSmnuGRFxAwU0GEQmWA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451031; c=relaxed/simple; bh=uCNXI+GJ0vao/4PujxQcbbNp8M2U6z90czkJ9T5PyEU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TyqqyEMtOBS+F4Xo/aPNYM6BXX2D5fzp2eJGlZV66ZSBcH7jcwioBsQckTd9IeRSBcgFJiuSLX6bmibhFOnuZVdHCRSg45eVf2ju9CSTHt174otTFs6JEdm/loTfwsHUnBfaFLoAtqdbnv36aYQho8fg203VqZDWforlDfHKN0o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=O9Aw1bv5; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="O9Aw1bv5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741451029; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BUfmfH8sGxGCZFYEG3nJrljYQw9Ko3dmbRiQod3+HBk=; b=O9Aw1bv5XEYeh32Kj+l5xoqZMzpzpTOftKMbe8mqj6IrfoEx6FVzjKcTs+0yt8AYfp92Np 3bFPWrdlJLerUpVItPCI8dyysoc80HGa8X2OKHz/F/ZBki+YFWWCN3imwN4NdxafBPo6UQ Ng7VDeHQ4rxzT4lMv6n7jAHlURTyOCo= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-665-95hZ2LccPh6qyoa_3MkbiA-1; Sat, 08 Mar 2025 11:23:48 -0500 X-MC-Unique: 95hZ2LccPh6qyoa_3MkbiA-1 X-Mimecast-MFC-AGG-ID: 95hZ2LccPh6qyoa_3MkbiA_1741451027 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 53ADA1956083; Sat, 8 Mar 2025 16:23:47 +0000 (UTC) Received: from localhost (unknown [10.72.120.5]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D37D219560AB; Sat, 8 Mar 2025 16:23:45 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Ming Lei Subject: [RESEND PATCH 1/5] loop: remove 'rw' parameter from lo_rw_aio() Date: Sun, 9 Mar 2025 00:23:05 +0800 Message-ID: <20250308162312.1640828-2-ming.lei@redhat.com> In-Reply-To: <20250308162312.1640828-1-ming.lei@redhat.com> References: <20250308162312.1640828-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 lo_rw_aio() is only called for READ/WRITE operation, which can be figured out from request directly, so remove 'rw' parameter from lo_rw_aio(), meantime rename the local variable as 'dir' which matches the actual use more. Meantime merge lo_read_simple() and lo_write_simple() into lo_rw_simple(). Signed-off-by: Ming Lei --- drivers/block/loop.c | 48 ++++++++++++++++++-------------------------- 1 file changed, 19 insertions(+), 29 deletions(-) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 657bf53decf3..6bbbaa4aaf2c 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -239,31 +239,25 @@ static int lo_write_bvec(struct file *file, struct bio_vec *bvec, loff_t *ppos) return bw; } -static int lo_write_simple(struct loop_device *lo, struct request *rq, - loff_t pos) -{ - struct bio_vec bvec; - struct req_iterator iter; - int ret = 0; - - rq_for_each_segment(bvec, rq, iter) { - ret = lo_write_bvec(lo->lo_backing_file, &bvec, &pos); - if (ret < 0) - break; - cond_resched(); - } - - return ret; -} - -static int lo_read_simple(struct loop_device *lo, struct request *rq, - loff_t pos) +static int lo_rw_simple(struct loop_device *lo, struct request *rq, loff_t pos) { struct bio_vec bvec; struct req_iterator iter; struct iov_iter i; ssize_t len; + if (req_op(rq) == REQ_OP_WRITE) { + int ret; + + rq_for_each_segment(bvec, rq, iter) { + ret = lo_write_bvec(lo->lo_backing_file, &bvec, &pos); + if (ret < 0) + break; + cond_resched(); + } + return ret; + } + rq_for_each_segment(bvec, rq, iter) { iov_iter_bvec(&i, ITER_DEST, &bvec, 1, bvec.bv_len); len = vfs_iter_read(lo->lo_backing_file, &i, &pos, 0); @@ -400,13 +394,13 @@ static void lo_rw_aio_complete(struct kiocb *iocb, long ret) lo_rw_aio_do_completion(cmd); } -static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, - loff_t pos, int rw) +static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, loff_t pos) { struct iov_iter iter; struct req_iterator rq_iter; struct bio_vec *bvec; struct request *rq = blk_mq_rq_from_pdu(cmd); + int dir = (req_op(rq) == REQ_OP_READ) ? ITER_DEST : ITER_SOURCE; struct bio *bio = rq->bio; struct file *file = lo->lo_backing_file; struct bio_vec tmp; @@ -448,7 +442,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, } atomic_set(&cmd->ref, 2); - iov_iter_bvec(&iter, rw, bvec, nr_bvec, blk_rq_bytes(rq)); + iov_iter_bvec(&iter, dir, bvec, nr_bvec, blk_rq_bytes(rq)); iter.iov_offset = offset; cmd->iocb.ki_pos = pos; @@ -457,7 +451,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, cmd->iocb.ki_flags = IOCB_DIRECT; cmd->iocb.ki_ioprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_NONE, 0); - if (rw == ITER_SOURCE) + if (dir == ITER_SOURCE) ret = file->f_op->write_iter(&cmd->iocb, &iter); else ret = file->f_op->read_iter(&cmd->iocb, &iter); @@ -498,15 +492,11 @@ static int do_req_filebacked(struct loop_device *lo, struct request *rq) case REQ_OP_DISCARD: return lo_fallocate(lo, rq, pos, FALLOC_FL_PUNCH_HOLE); case REQ_OP_WRITE: - if (cmd->use_aio) - return lo_rw_aio(lo, cmd, pos, ITER_SOURCE); - else - return lo_write_simple(lo, rq, pos); case REQ_OP_READ: if (cmd->use_aio) - return lo_rw_aio(lo, cmd, pos, ITER_DEST); + return lo_rw_aio(lo, cmd, pos); else - return lo_read_simple(lo, rq, pos); + return lo_rw_simple(lo, rq, pos); default: WARN_ON_ONCE(1); return -EIO; From patchwork Sat Mar 8 16:23:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14007587 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44E6D81720 for ; Sat, 8 Mar 2025 16:23:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451038; cv=none; b=ZK5WYHT7E1yMBExEcDdkUjH7qTyY5tmNd/j2kt0brQ0Z3Em4KBGrZgzNz7PYp5iGHj6EpwYDtSuN0833w8cT9BI9uONV2gvpzJvio/nJoxvKjUz0SsbBjqsGV083mCKpwfnYWQHMndJz432aPFoMCrIXXsMwlpAYcI8b1fdVewY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451038; c=relaxed/simple; bh=KBgRzZAbBGIQ69PfZ+7dGozNCyjO2VGPL3YMzUSFfpQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kRbWwK2+58B7xyXl6D3jeJqn4n/hShChlaaU/2IeXc1xkIsRJrYTgg4gxhvLtQfiusA1g2PhMijL72qM7DL5xB6q36V4RRhEUH+a0Xri8jspPzCw6I5RaynKbFm9O+jAaWzGeSxRrLfeIX4MsSArJYLp0YFZdjqLUJzhKhZkvZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UdQL84gJ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UdQL84gJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741451036; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=exaflMSl1un4CsRdVEleWfiozXNaKVaxajlaSlzzCVU=; b=UdQL84gJGwLShwaNaOHJgaCAh8xtppoazhcTaPMk0q1ZtH6Sf8yU/NsQXs2ZXVRRe/vutp OSLlUI1tBIZwNlngEbhN7XrWla0kF73fx/0Za15n/0mYghrtVAPbNMfhQcYgfifyg5shvn LyU1NYlAUKuswdHQXf/2kPg+H92XTeI= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-577-P982ZrE2Md-kGLQnCqQWHg-1; Sat, 08 Mar 2025 11:23:52 -0500 X-MC-Unique: P982ZrE2Md-kGLQnCqQWHg-1 X-Mimecast-MFC-AGG-ID: P982ZrE2Md-kGLQnCqQWHg_1741451031 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D0C9F1801A1A; Sat, 8 Mar 2025 16:23:51 +0000 (UTC) Received: from localhost (unknown [10.72.120.5]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 84EA3180174F; Sat, 8 Mar 2025 16:23:50 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Ming Lei Subject: [RESEND PATCH 2/5] loop: cleanup lo_rw_aio() Date: Sun, 9 Mar 2025 00:23:06 +0800 Message-ID: <20250308162312.1640828-3-ming.lei@redhat.com> In-Reply-To: <20250308162312.1640828-1-ming.lei@redhat.com> References: <20250308162312.1640828-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Cleanup lo_rw_aio() a bit by refactoring it into three parts: - lo_cmd_nr_bvec(), for calculating how many bvecs in this request - lo_rw_aio_prep(), for preparing loop command, which need to be called once - lo_submit_rw_aio(), for submitting this lo command, which can be called multiple times Prepare for trying to handle loop command by NOWAIT read/write IO first. Signed-off-by: Ming Lei --- drivers/block/loop.c | 83 +++++++++++++++++++++++++++++--------------- 1 file changed, 55 insertions(+), 28 deletions(-) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 6bbbaa4aaf2c..eae38cd38b7b 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -394,24 +394,63 @@ static void lo_rw_aio_complete(struct kiocb *iocb, long ret) lo_rw_aio_do_completion(cmd); } -static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, loff_t pos) +static int lo_submit_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, + loff_t pos, int nr_bvec) { - struct iov_iter iter; - struct req_iterator rq_iter; - struct bio_vec *bvec; struct request *rq = blk_mq_rq_from_pdu(cmd); int dir = (req_op(rq) == REQ_OP_READ) ? ITER_DEST : ITER_SOURCE; - struct bio *bio = rq->bio; struct file *file = lo->lo_backing_file; - struct bio_vec tmp; + struct iov_iter iter; + struct bio_vec *bvec; unsigned int offset; - int nr_bvec = 0; int ret; + if (rq->bio != rq->biotail) { + bvec = cmd->bvec; + offset = 0; + } else { + struct bio *bio = rq->bio; + + offset = bio->bi_iter.bi_bvec_done; + bvec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter); + } + iov_iter_bvec(&iter, dir, bvec, nr_bvec, blk_rq_bytes(rq)); + iter.iov_offset = offset; + cmd->iocb.ki_pos = pos; + + atomic_set(&cmd->ref, 2); + if (dir == ITER_SOURCE) + ret = file->f_op->write_iter(&cmd->iocb, &iter); + else + ret = file->f_op->read_iter(&cmd->iocb, &iter); + lo_rw_aio_do_completion(cmd); + + return ret; +} + +static inline unsigned lo_cmd_nr_bvec(struct loop_cmd *cmd) +{ + struct req_iterator rq_iter; + struct request *rq = blk_mq_rq_from_pdu(cmd); + struct bio_vec tmp; + int nr_bvec = 0; + rq_for_each_bvec(tmp, rq, rq_iter) nr_bvec++; + return nr_bvec; +} + +static int lo_rw_aio_prep(struct loop_device *lo, struct loop_cmd *cmd, + unsigned nr_bvec) +{ + struct request *rq = blk_mq_rq_from_pdu(cmd); + struct file *file = lo->lo_backing_file; + if (rq->bio != rq->biotail) { + struct req_iterator rq_iter; + struct bio_vec *bvec; + struct bio_vec tmp; bvec = kmalloc_array(nr_bvec, sizeof(struct bio_vec), GFP_NOIO); @@ -429,35 +468,23 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, loff_t pos) *bvec = tmp; bvec++; } - bvec = cmd->bvec; - offset = 0; - } else { - /* - * Same here, this bio may be started from the middle of the - * 'bvec' because of bio splitting, so offset from the bvec - * must be passed to iov iterator - */ - offset = bio->bi_iter.bi_bvec_done; - bvec = __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter); } - atomic_set(&cmd->ref, 2); - - iov_iter_bvec(&iter, dir, bvec, nr_bvec, blk_rq_bytes(rq)); - iter.iov_offset = offset; - - cmd->iocb.ki_pos = pos; cmd->iocb.ki_filp = file; cmd->iocb.ki_complete = lo_rw_aio_complete; cmd->iocb.ki_flags = IOCB_DIRECT; cmd->iocb.ki_ioprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_NONE, 0); - if (dir == ITER_SOURCE) - ret = file->f_op->write_iter(&cmd->iocb, &iter); - else - ret = file->f_op->read_iter(&cmd->iocb, &iter); + return 0; +} - lo_rw_aio_do_completion(cmd); +static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, loff_t pos) +{ + unsigned int nr_bvec = lo_cmd_nr_bvec(cmd); + int ret = lo_rw_aio_prep(lo, cmd, nr_bvec); + if (ret < 0) + return ret; + ret = lo_submit_rw_aio(lo, cmd, pos, nr_bvec); if (ret != -EIOCBQUEUED) lo_rw_aio_complete(&cmd->iocb, ret); return 0; From patchwork Sat Mar 8 16:23:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14007588 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FEE281720 for ; Sat, 8 Mar 2025 16:24:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451042; cv=none; b=MfTtAupKe7u4fGnbsg2vSqwFbJplwDAA5fwIjeS/nobesr22e+Or5zZ8gydJBLryLIxcBXFDKKaZhfbq49VJfXsxB7hJuSAOua/vQdEtOcTwoA+iHEW/tGswiS4IL+ovBekCMS3X/YDw3XKa3rtY3rLsgfI8gmPRMqiMtZnGf7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451042; c=relaxed/simple; bh=AgJFbGTafhFf0iPcSfyG89cLeR4PwjpGsExlKjXUMxg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=J+aZv6r1YRB0pbgzCOvwGDhUUnh9JFA5JJgC+Drj/ycTWIthVWUSf6brKIeN7LX8AnrH/GMP63pCI2c72mT/I3Q39Rj1X5AQmmSUogWo+6KjGgMYVVQ7WuUYcJazz9hZni6KGTAhpOyKmY2rLDWSLbi2fRTtHOL8s0I/JPlz/6E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=WfxDvn8/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WfxDvn8/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741451040; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dhGu5sg45RRsStdrJgrPGy8ehpmuWSgeWuA7ya3t1PY=; b=WfxDvn8/pvV0a637xcfLH5C8aJ8n2U+9xbWQNwTimTb9zIh7w0g+TOp6jwDePzuspAK8eU 4tneMuQk2nzgxmDoU4rTninV+yywGUoujIJnGJrmXF8uPYwU8jcY/nWXqUTb4WhySYIg7f MxwTZF6Y1p9up4a1frCxI0EvJJZuAoM= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-562-AT8JrPCeOi6uiKfoVieO9Q-1; Sat, 08 Mar 2025 11:23:57 -0500 X-MC-Unique: AT8JrPCeOi6uiKfoVieO9Q-1 X-Mimecast-MFC-AGG-ID: AT8JrPCeOi6uiKfoVieO9Q_1741451036 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1FE9F1955BC1; Sat, 8 Mar 2025 16:23:56 +0000 (UTC) Received: from localhost (unknown [10.72.120.5]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 202AF18009AE; Sat, 8 Mar 2025 16:23:54 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Ming Lei Subject: [RESEND PATCH 3/5] loop: add helper loop_queue_work_prep Date: Sun, 9 Mar 2025 00:23:07 +0800 Message-ID: <20250308162312.1640828-4-ming.lei@redhat.com> In-Reply-To: <20250308162312.1640828-1-ming.lei@redhat.com> References: <20250308162312.1640828-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Add helper loop_queue_work_prep() for making loop_queue_rq() more readable. Signed-off-by: Ming Lei --- drivers/block/loop.c | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index eae38cd38b7b..9f8d32d2dc4d 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -859,6 +859,27 @@ static inline int queue_on_root_worker(struct cgroup_subsys_state *css) } #endif +static void loop_queue_work_prep(struct loop_cmd *cmd) +{ + struct request *rq = blk_mq_rq_from_pdu(cmd); + + /* always use the first bio's css */ + cmd->blkcg_css = NULL; + cmd->memcg_css = NULL; +#ifdef CONFIG_BLK_CGROUP + if (rq->bio) { + cmd->blkcg_css = bio_blkcg_css(rq->bio); +#ifdef CONFIG_MEMCG + if (cmd->blkcg_css) { + cmd->memcg_css = + cgroup_get_e_css(cmd->blkcg_css->cgroup, + &memory_cgrp_subsys); + } +#endif + } +#endif +} + static void loop_queue_work(struct loop_device *lo, struct loop_cmd *cmd) { struct rb_node **node, *parent = NULL; @@ -866,6 +887,8 @@ static void loop_queue_work(struct loop_device *lo, struct loop_cmd *cmd) struct work_struct *work; struct list_head *cmd_list; + loop_queue_work_prep(cmd); + spin_lock_irq(&lo->lo_work_lock); if (queue_on_root_worker(cmd->blkcg_css)) @@ -1903,21 +1926,6 @@ static blk_status_t loop_queue_rq(struct blk_mq_hw_ctx *hctx, break; } - /* always use the first bio's css */ - cmd->blkcg_css = NULL; - cmd->memcg_css = NULL; -#ifdef CONFIG_BLK_CGROUP - if (rq->bio) { - cmd->blkcg_css = bio_blkcg_css(rq->bio); -#ifdef CONFIG_MEMCG - if (cmd->blkcg_css) { - cmd->memcg_css = - cgroup_get_e_css(cmd->blkcg_css->cgroup, - &memory_cgrp_subsys); - } -#endif - } -#endif loop_queue_work(lo, cmd); return BLK_STS_OK; From patchwork Sat Mar 8 16:23:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14007589 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E84D581720 for ; Sat, 8 Mar 2025 16:24:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451045; cv=none; b=jJcSjx8eQjsiyMuZHEwUpDC+mWEuQexz5o+KyxYNbtm94n9YsZpFpM51LeyAfJCyulYoxPuqVBQJuAChJDo/UyVU1I8x6LoQvj9wBmiuaGWyd/QZKI687xCnJ6C6wC1mMOiCy8HBBzQUFVIrE0pyvhWKPw9VOttLX0dv4c7tj4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451045; c=relaxed/simple; bh=Yb+POqdLdDFhux8EjKtMBpJkkYf3LItT4iridE1Yyo0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=njk+pQyidiPG+B4NqDrHdHFPChF+/8Kab4IxfVdNXBF1cowLeG3EkMQujuEE65MmYI07C2mcqUwH14AK2NDGpmzEtdrAKaOodRuMUTaImM/UXvP+tawomwMgCRjqxvRddIoHGopjTVFW0drLpuKowAsyOy7JBtohYdIToQFEOzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AzsmBJr2; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AzsmBJr2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741451042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fQnzm1yJJEvXwxmfMQ+FJneaXetol1zINJDCTJAaLFk=; b=AzsmBJr2c54hgrpw1BJ7QU152vAt93XDQSu0s0L5NavfgPF6dtcLxXgrrWC7liOVhw9eiz h1BRY8mDb50UrTw3Zh81Peds05VL6WPyogSzdVwkgAZAhFAODbinuaj66fK9+gxmWsUcEs 3cm49oqDxrATD4zHpZuHK78UnrF0EuM= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-433-Hye7Ve0yMm2tQQyKQlNXCg-1; Sat, 08 Mar 2025 11:24:01 -0500 X-MC-Unique: Hye7Ve0yMm2tQQyKQlNXCg-1 X-Mimecast-MFC-AGG-ID: Hye7Ve0yMm2tQQyKQlNXCg_1741451040 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B6D6F1956080; Sat, 8 Mar 2025 16:24:00 +0000 (UTC) Received: from localhost (unknown [10.72.120.5]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A43E31956095; Sat, 8 Mar 2025 16:23:59 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Ming Lei Subject: [RESEND PATCH 4/5] loop: try to handle loop aio command via NOWAIT IO first Date: Sun, 9 Mar 2025 00:23:08 +0800 Message-ID: <20250308162312.1640828-5-ming.lei@redhat.com> In-Reply-To: <20250308162312.1640828-1-ming.lei@redhat.com> References: <20250308162312.1640828-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Try to handle loop aio command via NOWAIT IO first, then we can avoid to queue the aio command into workqueue. Fallback to workqueue in case of -EAGAIN. BLK_MQ_F_BLOCKING has to be set for calling into .read_iter() or .write_iter() which might sleep even though it is NOWAIT. Signed-off-by: Ming Lei --- drivers/block/loop.c | 47 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 9f8d32d2dc4d..46be0c8e75a6 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -92,6 +92,8 @@ struct loop_cmd { #define LOOP_IDLE_WORKER_TIMEOUT (60 * HZ) #define LOOP_DEFAULT_HW_Q_DEPTH 128 +static void loop_queue_work(struct loop_device *lo, struct loop_cmd *cmd); + static DEFINE_IDR(loop_index_idr); static DEFINE_MUTEX(loop_ctl_mutex); static DEFINE_MUTEX(loop_validate_mutex); @@ -380,8 +382,17 @@ static void lo_rw_aio_do_completion(struct loop_cmd *cmd) if (!atomic_dec_and_test(&cmd->ref)) return; + + if (cmd->ret == -EAGAIN) { + struct loop_device *lo = rq->q->queuedata; + + loop_queue_work(lo, cmd); + return; + } + kfree(cmd->bvec); cmd->bvec = NULL; + if (likely(!blk_should_fake_timeout(rq->q))) blk_mq_complete_request(rq); } @@ -478,16 +489,34 @@ static int lo_rw_aio_prep(struct loop_device *lo, struct loop_cmd *cmd, } static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd, loff_t pos) +{ + unsigned int nr_bvec = lo_cmd_nr_bvec(cmd); + int ret; + + cmd->iocb.ki_flags &= ~IOCB_NOWAIT; + ret = lo_submit_rw_aio(lo, cmd, pos, nr_bvec); + if (ret != -EIOCBQUEUED) + lo_rw_aio_complete(&cmd->iocb, ret); + return 0; +} + +static int lo_rw_aio_nowait(struct loop_device *lo, struct loop_cmd *cmd, loff_t pos) { unsigned int nr_bvec = lo_cmd_nr_bvec(cmd); int ret = lo_rw_aio_prep(lo, cmd, nr_bvec); if (ret < 0) return ret; + + cmd->iocb.ki_flags |= IOCB_NOWAIT; ret = lo_submit_rw_aio(lo, cmd, pos, nr_bvec); - if (ret != -EIOCBQUEUED) + if (ret == -EIOCBQUEUED) + return 0; + if (ret != -EAGAIN) { lo_rw_aio_complete(&cmd->iocb, ret); - return 0; + return 0; + } + return ret; } static int do_req_filebacked(struct loop_device *lo, struct request *rq) @@ -1926,6 +1955,17 @@ static blk_status_t loop_queue_rq(struct blk_mq_hw_ctx *hctx, break; } + if (cmd->use_aio) { + loff_t pos = ((loff_t) blk_rq_pos(rq) << 9) + lo->lo_offset; + int ret = lo_rw_aio_nowait(lo, cmd, pos); + + if (!ret) + return BLK_STS_OK; + if (ret != -EAGAIN) + return BLK_STS_IOERR; + /* fallback to workqueue for handling aio */ + } + loop_queue_work(lo, cmd); return BLK_STS_OK; @@ -2076,7 +2116,8 @@ static int loop_add(int i) lo->tag_set.queue_depth = hw_queue_depth; lo->tag_set.numa_node = NUMA_NO_NODE; lo->tag_set.cmd_size = sizeof(struct loop_cmd); - lo->tag_set.flags = BLK_MQ_F_STACKING | BLK_MQ_F_NO_SCHED_BY_DEFAULT; + lo->tag_set.flags = BLK_MQ_F_STACKING | BLK_MQ_F_NO_SCHED_BY_DEFAULT | + BLK_MQ_F_BLOCKING; lo->tag_set.driver_data = lo; err = blk_mq_alloc_tag_set(&lo->tag_set); From patchwork Sat Mar 8 16:23:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 14007590 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52B2E1DE3DE for ; Sat, 8 Mar 2025 16:24:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451052; cv=none; b=LjEazSzMshwJ3uewRJeyqnjfPI2KqDkIvGL0S8K3wsgEqVPDwE4RUoIJi3/iawpGFx5e63B+pqIYNu5JTjz57Z8Mm02MnyenccEG+5Dhg3oZnwBBfVWU1s9dZj/zg6zAMugSube+WAQ4dCtqYNrDr9XoXcQCY5tS1sCfpTph7AY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741451052; c=relaxed/simple; bh=gL9iMdxsmgDg8iKlpYQ/fFXgO4Fuoc5oKg164Ik7boQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lWX+OyLtdWNrV+Ef328NrofkYL/OikwRnHOA4p97uw7ULfDpuk6TMHvuOtk+eL+q3bynZfaJn4twogIoYmaqt95O3pkvkujvhFGBzFq4Uu5ULgGgtYRHiwbeNPGR6H+Hy7MDfQ8zKhXgJ3XbYCbb6i0ZEsoF9Q+X860LxA3Hq34= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SzdDUJSp; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SzdDUJSp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1741451050; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oe0IUktthmJLl/yJLl+QxisFcC59Da6KFwQUDP6NVbw=; b=SzdDUJSpkFxkuGki9RN5q52mu4WOx75W54okxxjDAez5t3zOXeFUTIUTgU85VLaVs41PYG PUna0zNpMtLqd7VCBWRKPQ9s/PHNJuz5Oh+reOKECgcwZbyQV2RfyXOgnvF4dj2FKHX0zp 0+cq7i7YIY9RC9r7h0xOSy5GuaqKenY= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-475-E139ezQoN-GmHTdeu736vQ-1; Sat, 08 Mar 2025 11:24:07 -0500 X-MC-Unique: E139ezQoN-GmHTdeu736vQ-1 X-Mimecast-MFC-AGG-ID: E139ezQoN-GmHTdeu736vQ_1741451046 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 28BC2180049D; Sat, 8 Mar 2025 16:24:06 +0000 (UTC) Received: from localhost (unknown [10.72.120.5]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0831819560AB; Sat, 8 Mar 2025 16:24:03 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Ming Lei Subject: [RESEND PATCH 5/5] loop: add module parameter of 'nr_hw_queues' Date: Sun, 9 Mar 2025 00:23:09 +0800 Message-ID: <20250308162312.1640828-6-ming.lei@redhat.com> In-Reply-To: <20250308162312.1640828-1-ming.lei@redhat.com> References: <20250308162312.1640828-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Add module parameter of 'nr_hw_queues' so that loop can support MQ, which may reduce contention in case of too many io jobs. Signed-off-by: Ming Lei --- drivers/block/loop.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 46be0c8e75a6..6378dfee6681 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -91,6 +91,7 @@ struct loop_cmd { #define LOOP_IDLE_WORKER_TIMEOUT (60 * HZ) #define LOOP_DEFAULT_HW_Q_DEPTH 128 +#define LOOP_DEFAULT_NR_HW_Q 1 static void loop_queue_work(struct loop_device *lo, struct loop_cmd *cmd); @@ -1928,6 +1929,26 @@ static const struct kernel_param_ops loop_hw_qdepth_param_ops = { device_param_cb(hw_queue_depth, &loop_hw_qdepth_param_ops, &hw_queue_depth, 0444); MODULE_PARM_DESC(hw_queue_depth, "Queue depth for each hardware queue. Default: " __stringify(LOOP_DEFAULT_HW_Q_DEPTH)); +static int nr_hw_queues = LOOP_DEFAULT_NR_HW_Q; +static int loop_set_nr_hw_queues(const char *s, const struct kernel_param *p) +{ + int nr, ret; + + ret = kstrtoint(s, 0, &nr); + if (ret < 0) + return ret; + if (nr < 1) + return -EINVAL; + nr_hw_queues = nr; + return 0; +} +static const struct kernel_param_ops loop_nr_hw_q_param_ops = { + .set = loop_set_nr_hw_queues, + .get = param_get_int, +}; +device_param_cb(nr_hw_queues, &loop_nr_hw_q_param_ops, &nr_hw_queues, 0444); +MODULE_PARM_DESC(nr_hw_queues, "number of hardware queues. Default: " __stringify(LOOP_DEFAULT_NR_HW_Q)); + MODULE_DESCRIPTION("Loopback device support"); MODULE_LICENSE("GPL"); MODULE_ALIAS_BLOCKDEV_MAJOR(LOOP_MAJOR); @@ -2112,7 +2133,7 @@ static int loop_add(int i) i = err; lo->tag_set.ops = &loop_mq_ops; - lo->tag_set.nr_hw_queues = 1; + lo->tag_set.nr_hw_queues = nr_hw_queues; lo->tag_set.queue_depth = hw_queue_depth; lo->tag_set.numa_node = NUMA_NO_NODE; lo->tag_set.cmd_size = sizeof(struct loop_cmd);