From patchwork Wed Dec 14 18:55:19 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Wheeler X-Patchwork-Id: 9474521 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CE322607EE for ; Wed, 14 Dec 2016 19:00:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD9CA286E7 for ; Wed, 14 Dec 2016 19:00:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B21FF2872D; Wed, 14 Dec 2016 19:00:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EC07286F1 for ; Wed, 14 Dec 2016 19:00:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753769AbcLNTAb (ORCPT ); Wed, 14 Dec 2016 14:00:31 -0500 Received: from mx.ewheeler.net ([66.155.3.69]:58630 "EHLO mail.ewheeler.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753728AbcLNTAa (ORCPT ); Wed, 14 Dec 2016 14:00:30 -0500 Received: from localhost (localhost [127.0.0.1]) by mail.ewheeler.net (Postfix) with ESMTP id F1298A0694 for ; Wed, 14 Dec 2016 11:00:29 -0800 (PST) X-Virus-Scanned: amavisd-new at ewheeler.net Received: from mail.ewheeler.net ([127.0.0.1]) by localhost (mail.ewheeler.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id Hzyx1I89Xb_3 for ; Wed, 14 Dec 2016 11:00:29 -0800 (PST) Received: from mx.ewheeler.net (mx.ewheeler.net [66.155.3.69]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.ewheeler.net (Postfix) with ESMTPSA id 34538A0415 for ; Wed, 14 Dec 2016 11:00:29 -0800 (PST) Date: Wed, 14 Dec 2016 10:55:19 -0800 (PST) From: Eric Wheeler X-X-Sender: lists@mail.ewheeler.net To: Mikulas Patocka , Alasdair Kergon , Mike Snitzer , Shaohua Li , Jens Axboe , linux-block@vger.kernel.org, dm-devel@redhat.com Subject: [PATCH] dm-crypt: fix lost ioprio when queuing crypto bios from task with ioprio Message-ID: User-Agent: Alpine 2.11 (LRH 23 2013-08-11) MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Since dm-crypt queues writes (and sometimes reads) to a different kernel thread (workqueue), the bios will dispatch from tasks with different io_context->ioprio settings than the submitting task, thus giving incorrect ioprio hints to the io scheduler. By assigning the ioprio to the bio before queuing to a workqueue, the original submitting task's io_context->ioprio setting can be retained through the life of the bio. We only set the bio's ioprio in dm-crypt if not already set (by somewhere else, higher in the stack). Signed-off-by: Eric Wheeler Cc: Mikulas Patocka Cc: Alasdair Kergon Cc: Mike Snitzer Cc: Shaohua Li Cc: Jens Axboe Cc: stable@vger.kernel.org --- block/bio.c | 1 + drivers/md/dm-crypt.c | 11 +++++++++-- include/linux/bio.h | 23 +++++++++++++++++++++++ 3 files changed, 33 insertions(+), 2 deletions(-) diff --git a/block/bio.c b/block/bio.c index db85c57..1a529ff 100644 --- a/block/bio.c +++ b/block/bio.c @@ -584,6 +584,7 @@ void __bio_clone_fast(struct bio *bio, struct bio *bio_src) bio->bi_iter = bio_src->bi_iter; bio->bi_io_vec = bio_src->bi_io_vec; + bio_set_prio(bio, bio_prio(bio_src)); bio_clone_blkcg_association(bio, bio_src); } EXPORT_SYMBOL(__bio_clone_fast); diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 9b99ee9..ea7e102 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -1133,6 +1133,7 @@ static void clone_init(struct dm_crypt_io *io, struct bio *clone) clone->bi_private = io; clone->bi_end_io = crypt_endio; clone->bi_bdev = cc->dev->bdev; + bio_set_prio(clone, bio_prio(io->base_bio)); bio_set_op_attrs(clone, bio_op(io->base_bio), bio_flags(io->base_bio)); } @@ -2070,10 +2071,16 @@ static int crypt_map(struct dm_target *ti, struct bio *bio) io->ctx.req = (struct skcipher_request *)(io + 1); if (bio_data_dir(io->base_bio) == READ) { - if (kcryptd_io_read(io, GFP_NOWAIT)) + if (kcryptd_io_read(io, GFP_NOWAIT)) { + if (!ioprio_valid(bio_prio(io->base_bio))) + bio_set_task_prio(io->base_bio, current); kcryptd_queue_read(io); - } else + } + } else { + if (!ioprio_valid(bio_prio(io->base_bio))) + bio_set_task_prio(io->base_bio, current); kcryptd_queue_crypt(io); + } return DM_MAPIO_SUBMITTED; } diff --git a/include/linux/bio.h b/include/linux/bio.h index 97cb48f..0b4f8bb 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -62,6 +62,29 @@ #define bio_sectors(bio) ((bio)->bi_iter.bi_size >> 9) #define bio_end_sector(bio) ((bio)->bi_iter.bi_sector + bio_sectors((bio))) +/* Set the bio's ioprio to that of the task's io_context->ioprio, if any. + * Always returns io_context->ioprio (or 0 if none); bio can be NULL. + */ +static inline unsigned short bio_set_task_prio( + struct bio *bio, struct task_struct *task) +{ + struct io_context *ioc; + unsigned short ioprio = 0; + + ioc = get_task_io_context(current, GFP_NOIO, NUMA_NO_NODE); + if (ioc) { + if (ioprio_valid(ioc->ioprio)) { + ioprio = ioc->ioprio; + put_io_context(ioc); + } + + if (bio != NULL && ioprio_valid(ioprio)) + bio_set_prio(bio, ioc->ioprio); + } + + return ioprio; +} + /* * Check whether this bio carries any data or not. A NULL bio is allowed. */