From patchwork Fri Mar 9 00:42:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Vagin X-Patchwork-Id: 10269461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E2DBF602BD for ; Fri, 9 Mar 2018 00:43:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D21E829B84 for ; Fri, 9 Mar 2018 00:43:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C675629BA4; Fri, 9 Mar 2018 00:43:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C1BE29B84 for ; Fri, 9 Mar 2018 00:43:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751025AbeCIAnK (ORCPT ); Thu, 8 Mar 2018 19:43:10 -0500 Received: from mail-qt0-f193.google.com ([209.85.216.193]:46783 "EHLO mail-qt0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750848AbeCIAnI (ORCPT ); Thu, 8 Mar 2018 19:43:08 -0500 Received: by mail-qt0-f193.google.com with SMTP id m13so8937742qtg.13; Thu, 08 Mar 2018 16:43:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id; bh=oDx0P99eM/CYuD8c+4l3IZ3UdKrHqF3wb9NJdyD9MA4=; b=eSctz7hqLPYuIttJvOea+eOPj6YFbOlUFCmro3aLXrSyAy27Pzn3u32oQ+8WOdcXV7 Xlv2nH+LbMDaonc1z73HkIGuIU+Yp2LPH8Ybg+o0Ocsvm1o2McIIJy+/ng16xoqN2kDG ENZZE+HcR3kk/D7VJ0pkktWcPKR0VS/s4t1NJf6S6QpIP9cF9CRWaIvsznfW611DCVtO dgr5LsWKXFXmIvmZkdXGvd+KX4O8iSVgkLso6V64g6oK+qLrWB8KUhl3R3Q7PYYgjn5n /EK6SvoWINwkI5PqDCfhNybJRDOhkNKO8gI398fndwgeiPwEM5PbOtHntBtnatcRSs4v roWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id; bh=oDx0P99eM/CYuD8c+4l3IZ3UdKrHqF3wb9NJdyD9MA4=; b=rQ9ZK1HqtJmX6MrpKw92jQrQiQI+2Tu0CzPm+LDGyhoY2NpMJbRDJGH9wSck0RaOPa pMDse+j4w3OochkiSFczptJndRH+VXRXV2bBU7QUQ2sxMbTA/3rCTuC4V6ByglZz1c1J yKVdfII45vG9W6H5ZmBrI+DilaKyMgWaIJ4hnMvaCCS1+2rnDtqPpJlCqtP+B5ZuwDdc jyM07tv1V2izhHyOogojpNP2SbASmRbVQrRVoDkNrEBEs8xVGr4utlBcIaE60FcZzEJX xKzsDCq+/KeoudkoTmEpwsC5OseTK45n+cawm17xhqzygPdvK985XLv1nDaF6SuhkOd+ ac/w== X-Gm-Message-State: AElRT7GELw49M3Y0TemtNJ1embZmCAiz01FOH1OtEAD4UpW/GCdCedTs gZn4q2hqZRpmi1t7x2Gq/tqa9ns8 X-Google-Smtp-Source: AG47ELv52JBq86uAZs3p0JG9h1vDfz+VotuHprzWy4cV0s/Wti4yLeYfecM12q3UWVujAA8zda6ehg== X-Received: by 10.237.33.228 with SMTP id m33mr44022667qtc.30.1520556187570; Thu, 08 Mar 2018 16:43:07 -0800 (PST) Received: from localhost.localdomain ([65.152.152.74]) by smtp.gmail.com with ESMTPSA id b125sm12738105qkc.47.2018.03.08.16.43.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 08 Mar 2018 16:43:06 -0800 (PST) From: Andrei Vagin To: "Nicholas A. Bellinger" Cc: linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, Andrei Vagin Subject: [PATCH] [RFC] target/file: add support of direct and async I/O Date: Thu, 8 Mar 2018 16:42:59 -0800 Message-Id: <20180309004259.16052-1-avagin@openvz.org> X-Mailer: git-send-email 2.13.6 Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Direct I/O allows to not affect the write-back cache, this is expected when a non-buffered mode is used. Async I/O allows to handle a few commands concurrently, so a target shows a better perfomance: Mode: O_DSYNC Async: 1 $ ./fio --bs=4K --direct=1 --rw=randwrite --ioengine=libaio --iodepth=64 --name=/dev/sda --runtime=20 --numjobs=2 WRITE: bw=45.9MiB/s (48.1MB/s), 21.9MiB/s-23.0MiB/s (22.0MB/s-25.2MB/s), io=919MiB (963MB), run=20002-20020msec Mode: O_DSYNC Async: 0 $ ./fio --bs=4K --direct=1 --rw=randwrite --ioengine=libaio --iodepth=64 --name=/dev/sdb --runtime=20 --numjobs=2 WRITE: bw=1607KiB/s (1645kB/s), 802KiB/s-805KiB/s (821kB/s-824kB/s), io=31.8MiB (33.4MB), run=20280-20295msec Known issue: DIF (PI) emulation doesn't work when a target uses async I/O, because DIF metadata is saved in a separate file, and it is another non-trivial task how to synchronize writing in two files, so that a following read operation always returns a consisten metadata for a specified block. Cc: "Nicholas A. Bellinger" Signed-off-by: Andrei Vagin Tested-by: Bryant G. Ly --- drivers/target/target_core_file.c | 124 ++++++++++++++++++++++++++++++++++++-- drivers/target/target_core_file.h | 1 + 2 files changed, 120 insertions(+), 5 deletions(-) diff --git a/drivers/target/target_core_file.c b/drivers/target/target_core_file.c index 9b2c0c773022..e8c07a0f7084 100644 --- a/drivers/target/target_core_file.c +++ b/drivers/target/target_core_file.c @@ -250,6 +250,96 @@ static void fd_destroy_device(struct se_device *dev) } } +struct target_core_file_cmd { + atomic_t ref; + unsigned long len; + long ret; + struct se_cmd *cmd; + struct kiocb iocb; + struct bio_vec bvec[0]; +}; + +static void cmd_rw_aio_do_completion(struct target_core_file_cmd *cmd) +{ + if (!atomic_dec_and_test(&cmd->ref)) + return; + + if (cmd->ret != cmd->len) + target_complete_cmd(cmd->cmd, SAM_STAT_CHECK_CONDITION); + else + target_complete_cmd(cmd->cmd, SAM_STAT_GOOD); + + kfree(cmd); +} + +static void cmd_rw_aio_complete(struct kiocb *iocb, long ret, long ret2) +{ + struct target_core_file_cmd *cmd; + + cmd = container_of(iocb, struct target_core_file_cmd, iocb); + + cmd->ret = ret; + cmd_rw_aio_do_completion(cmd); +} + +static int fd_do_aio_rw(struct se_cmd *cmd, struct fd_dev *fd_dev, + u32 block_size, struct scatterlist *sgl, + u32 sgl_nents, u32 data_length, int is_write) +{ + struct file *file = fd_dev->fd_file; + struct target_core_file_cmd *aio_cmd; + struct scatterlist *sg; + struct iov_iter iter = {}; + struct bio_vec *bvec; + ssize_t len = 0; + loff_t pos = (cmd->t_task_lba * block_size); + int ret = 0, i; + + aio_cmd = kmalloc(sizeof(struct target_core_file_cmd) + + sgl_nents * sizeof(struct bio_vec), + GFP_KERNEL | __GFP_ZERO); + if (!aio_cmd) + return -ENOMEM; + + bvec = aio_cmd->bvec; + + for_each_sg(sgl, sg, sgl_nents, i) { + bvec[i].bv_page = sg_page(sg); + bvec[i].bv_len = sg->length; + bvec[i].bv_offset = sg->offset; + + len += sg->length; + } + + iov_iter_bvec(&iter, ITER_BVEC | is_write, bvec, sgl_nents, len); + + atomic_set(&aio_cmd->ref, 2); + + aio_cmd->cmd = cmd; + aio_cmd->len = len; + aio_cmd->iocb.ki_pos = pos; + aio_cmd->iocb.ki_filp = file; + aio_cmd->iocb.ki_complete = cmd_rw_aio_complete; + aio_cmd->iocb.ki_flags = 0; + + if (!(fd_dev->fbd_flags & FDBD_HAS_BUFFERED_IO_WCE)) + aio_cmd->iocb.ki_flags |= IOCB_DIRECT; + if (is_write && (cmd->se_cmd_flags & SCF_FUA)) + aio_cmd->iocb.ki_flags |= IOCB_DSYNC; + + if (is_write) + ret = call_write_iter(file, &aio_cmd->iocb, &iter); + else + ret = call_read_iter(file, &aio_cmd->iocb, &iter); + + cmd_rw_aio_do_completion(aio_cmd); + + if (ret != -EIOCBQUEUED) + aio_cmd->iocb.ki_complete(&aio_cmd->iocb, ret, 0); + + return 0; +} + static int fd_do_rw(struct se_cmd *cmd, struct file *fd, u32 block_size, struct scatterlist *sgl, u32 sgl_nents, u32 data_length, int is_write) @@ -536,6 +626,7 @@ fd_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, struct file *pfile = fd_dev->fd_prot_file; sense_reason_t rc; int ret = 0; + int aio = fd_dev->fbd_flags & FDBD_HAS_ASYNC_IO; /* * We are currently limited by the number of iovecs (2048) per * single vfs_[writev,readv] call. @@ -550,7 +641,11 @@ fd_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, * Call vectorized fileio functions to map struct scatterlist * physical memory addresses to struct iovec virtual memory. */ - if (data_direction == DMA_FROM_DEVICE) { + if (aio) { + ret = fd_do_aio_rw(cmd, fd_dev, dev->dev_attrib.block_size, + sgl, sgl_nents, cmd->data_length, + !(data_direction == DMA_FROM_DEVICE)); + } else if (data_direction == DMA_FROM_DEVICE) { if (cmd->prot_type && dev->dev_attrib.pi_prot_type) { ret = fd_do_rw(cmd, pfile, dev->prot_length, cmd->t_prot_sg, cmd->t_prot_nents, @@ -616,18 +711,21 @@ fd_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, if (ret < 0) return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE; - target_complete_cmd(cmd, SAM_STAT_GOOD); + if (!aio) + target_complete_cmd(cmd, SAM_STAT_GOOD); return 0; } enum { - Opt_fd_dev_name, Opt_fd_dev_size, Opt_fd_buffered_io, Opt_err + Opt_fd_dev_name, Opt_fd_dev_size, Opt_fd_buffered_io, + Opt_fd_async_io, Opt_err }; static match_table_t tokens = { {Opt_fd_dev_name, "fd_dev_name=%s"}, {Opt_fd_dev_size, "fd_dev_size=%s"}, {Opt_fd_buffered_io, "fd_buffered_io=%d"}, + {Opt_fd_async_io, "fd_async_io=%d"}, {Opt_err, NULL} }; @@ -693,6 +791,21 @@ static ssize_t fd_set_configfs_dev_params(struct se_device *dev, fd_dev->fbd_flags |= FDBD_HAS_BUFFERED_IO_WCE; break; + case Opt_fd_async_io: + ret = match_int(args, &arg); + if (ret) + goto out; + if (arg != 1) { + pr_err("bogus fd_async_io=%d value\n", arg); + ret = -EINVAL; + goto out; + } + + pr_debug("FILEIO: Using async I/O" + " operations for struct fd_dev\n"); + + fd_dev->fbd_flags |= FDBD_HAS_ASYNC_IO; + break; default: break; } @@ -709,10 +822,11 @@ static ssize_t fd_show_configfs_dev_params(struct se_device *dev, char *b) ssize_t bl = 0; bl = sprintf(b + bl, "TCM FILEIO ID: %u", fd_dev->fd_dev_id); - bl += sprintf(b + bl, " File: %s Size: %llu Mode: %s\n", + bl += sprintf(b + bl, " File: %s Size: %llu Mode: %s Async: %d\n", fd_dev->fd_dev_name, fd_dev->fd_dev_size, (fd_dev->fbd_flags & FDBD_HAS_BUFFERED_IO_WCE) ? - "Buffered-WCE" : "O_DSYNC"); + "Buffered-WCE" : "O_DSYNC", + !!(fd_dev->fbd_flags & FDBD_HAS_ASYNC_IO)); return bl; } diff --git a/drivers/target/target_core_file.h b/drivers/target/target_core_file.h index 53be5ffd3261..929b1ecd544e 100644 --- a/drivers/target/target_core_file.h +++ b/drivers/target/target_core_file.h @@ -22,6 +22,7 @@ #define FBDF_HAS_PATH 0x01 #define FBDF_HAS_SIZE 0x02 #define FDBD_HAS_BUFFERED_IO_WCE 0x04 +#define FDBD_HAS_ASYNC_IO 0x08 #define FDBD_FORMAT_UNIT_SIZE 2048 struct fd_dev {