From patchwork Tue Jun 7 08:06:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Xu X-Patchwork-Id: 12871494 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2461EC433EF for ; Tue, 7 Jun 2022 08:06:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237880AbiFGIGh (ORCPT ); Tue, 7 Jun 2022 04:06:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231132AbiFGIGf (ORCPT ); Tue, 7 Jun 2022 04:06:35 -0400 Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB9E6B82CB; Tue, 7 Jun 2022 01:06:34 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1654589193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uIx0SLPeGNO2+cSM+ls+SeNXYoun+BONGh4wyuQHRtw=; b=a3HBkxlRlqMQTfPlckGvdhrBmFUX2Z/pzhR3IJlxhZX6Us0tFj7Jf/MwBh2Pb+KsdLpcTj ZICu6pbPegedOfJnnWJa6M83Tc40sC64UPF2b5p9izJBswggIJh5AOUAIfuxmXehfC+EnB tyqHrzauiJEvuN7d3nhr6XZ9poH2BUw= From: Hao Xu To: io-uring@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , linux-fsdevel@vger.kernel.org, Alexander Viro , Andrew Morton , Luis Chamberlain , Kuniyuki Iwashima Subject: [PATCH 1/5] io_uring: move sp->len check up for splice and tee Date: Tue, 7 Jun 2022 16:06:15 +0800 Message-Id: <20220607080619.513187-2-hao.xu@linux.dev> In-Reply-To: <20220607080619.513187-1-hao.xu@linux.dev> References: <20220607080619.513187-1-hao.xu@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org From: Hao Xu The traditional sync splice code return 0 if len is 0 at the beginning of syscall, similar thing for tee. So move up sp->len zero check so that it reaches quick ending when len is 0. Signed-off-by: Hao Xu --- io_uring/splice.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/io_uring/splice.c b/io_uring/splice.c index 0e19d6330345..b2cd1044c3ee 100644 --- a/io_uring/splice.c +++ b/io_uring/splice.c @@ -53,6 +53,9 @@ int io_tee(struct io_kiocb *req, unsigned int issue_flags) struct file *in; long ret = 0; + if (unlikely(!sp->len)) + goto done; + if (issue_flags & IO_URING_F_NONBLOCK) return -EAGAIN; @@ -65,8 +68,7 @@ int io_tee(struct io_kiocb *req, unsigned int issue_flags) goto done; } - if (sp->len) - ret = do_tee(in, out, sp->len, flags); + ret = do_tee(in, out, sp->len, flags); if (!(sp->flags & SPLICE_F_FD_IN_FIXED)) io_put_file(in); @@ -95,6 +97,9 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags) struct file *in; long ret = 0; + if (unlikely(!sp->len)) + goto done; + if (issue_flags & IO_URING_F_NONBLOCK) return -EAGAIN; @@ -110,8 +115,7 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags) poff_in = (sp->off_in == -1) ? NULL : &sp->off_in; poff_out = (sp->off_out == -1) ? NULL : &sp->off_out; - if (sp->len) - ret = do_splice(in, poff_in, out, poff_out, sp->len, flags); + ret = do_splice(in, poff_in, out, poff_out, sp->len, flags); if (!(sp->flags & SPLICE_F_FD_IN_FIXED)) io_put_file(in); From patchwork Tue Jun 7 08:06:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Xu X-Patchwork-Id: 12871495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A233C43334 for ; Tue, 7 Jun 2022 08:06:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238021AbiFGIGr (ORCPT ); Tue, 7 Jun 2022 04:06:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238000AbiFGIGo (ORCPT ); Tue, 7 Jun 2022 04:06:44 -0400 Received: from out1.migadu.com (out1.migadu.com [IPv6:2001:41d0:2:863f::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE2A8B82DA; Tue, 7 Jun 2022 01:06:39 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1654589198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=klVOdmR1nU7hPRaqNMeN8/4eylLm3n3KJl/eiNUtm0o=; b=HxqJGVj9AWEXmaTuJYwqAghzOqYCKWtZjzq5XUfX8XIfvgZzRHJak4kUBr4KLoGXwbNuVJ DqnaAk8Y4CnHz8rU7nit+jqCQnVERRBvVKhH0zyeFrVyLXchdffcqg8P2nzsDIkQYd6WOK 9BmlK2+vbR9mEjoN20otm7nkxGZmFds= From: Hao Xu To: io-uring@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , linux-fsdevel@vger.kernel.org, Alexander Viro , Andrew Morton , Luis Chamberlain , Kuniyuki Iwashima Subject: [PATCH 2/5] pipe: add trylock helpers for pipe lock Date: Tue, 7 Jun 2022 16:06:16 +0800 Message-Id: <20220607080619.513187-3-hao.xu@linux.dev> In-Reply-To: <20220607080619.513187-1-hao.xu@linux.dev> References: <20220607080619.513187-1-hao.xu@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org From: Hao Xu Add two helpers pipe_trylock() and pipe_double_trylock(), they are used in nonblock splice(pipe to pipe) scenario in next patches. Signed-off-by: Hao Xu --- fs/pipe.c | 29 +++++++++++++++++++++++++++++ include/linux/pipe_fs_i.h | 2 ++ 2 files changed, 31 insertions(+) diff --git a/fs/pipe.c b/fs/pipe.c index 74ae9fafd25a..033736eb61fb 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -98,6 +98,11 @@ void pipe_unlock(struct pipe_inode_info *pipe) } EXPORT_SYMBOL(pipe_unlock); +int pipe_trylock(struct pipe_inode_info *pipe) +{ + return mutex_trylock(&pipe->mutex); +} + static inline void __pipe_lock(struct pipe_inode_info *pipe) { mutex_lock_nested(&pipe->mutex, I_MUTEX_PARENT); @@ -122,6 +127,30 @@ void pipe_double_lock(struct pipe_inode_info *pipe1, } } +int pipe_double_trylock(struct pipe_inode_info *pipe1, + struct pipe_inode_info *pipe2) +{ + BUG_ON(pipe1 == pipe2); + + if (pipe1 < pipe2) { + if (!pipe_trylock(pipe1)) + return 0; + if (!pipe_trylock(pipe2)) { + pipe_unlock(pipe1); + return 0; + } + } else { + if (!pipe_trylock(pipe2)) + return 0; + if (!pipe_trylock(pipe1)) { + pipe_unlock(pipe2); + return 0; + } + } + + return 1; +} + static void anon_pipe_buf_release(struct pipe_inode_info *pipe, struct pipe_buffer *buf) { diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h index cb0fd633a610..78dc2c7c999f 100644 --- a/include/linux/pipe_fs_i.h +++ b/include/linux/pipe_fs_i.h @@ -235,8 +235,10 @@ static inline bool pipe_buf_try_steal(struct pipe_inode_info *pipe, /* Pipe lock and unlock operations */ void pipe_lock(struct pipe_inode_info *); +int pipe_trylock(struct pipe_inode_info *); void pipe_unlock(struct pipe_inode_info *); void pipe_double_lock(struct pipe_inode_info *, struct pipe_inode_info *); +int pipe_double_trylock(struct pipe_inode_info *, struct pipe_inode_info *); /* Wait for a pipe to be readable/writable while dropping the pipe lock */ void pipe_wait_readable(struct pipe_inode_info *); From patchwork Tue Jun 7 08:06:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Xu X-Patchwork-Id: 12871496 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92F27CCA47C for ; Tue, 7 Jun 2022 08:06:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238000AbiFGIGr (ORCPT ); Tue, 7 Jun 2022 04:06:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238024AbiFGIGq (ORCPT ); Tue, 7 Jun 2022 04:06:46 -0400 Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EB46B82F1; Tue, 7 Jun 2022 01:06:44 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1654589202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RkLnU8BX1gl8psHZPmN10zKlj7zdxAeaXWQrglTE4fE=; b=Cd3KYch3FJFW7iw6PqMlDjONThZ+ckBbn78xiRnrhCbsrx5zukS64v+DCg3maMbbHOq//G iOdXayAe0kWLBpTMQgkoL/E89JnbfCl1Wu+J2cDUDXjcgSdSS9XAEI0JVl6SYsf6GSxjqd asSsKCT9mpcXwRlAiRoz9jqAqL8yqH8= From: Hao Xu To: io-uring@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , linux-fsdevel@vger.kernel.org, Alexander Viro , Andrew Morton , Luis Chamberlain , Kuniyuki Iwashima Subject: [PATCH 3/5] splice: support nonblock for splice from pipe to pipe Date: Tue, 7 Jun 2022 16:06:17 +0800 Message-Id: <20220607080619.513187-4-hao.xu@linux.dev> In-Reply-To: <20220607080619.513187-1-hao.xu@linux.dev> References: <20220607080619.513187-1-hao.xu@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org From: Hao Xu When SPLICE_F_NONBLOCK is set, splice() still may be blocked by pipe lock in pipe to pipe scenario. Add trylock logic to make it more nonblock Signed-off-by: Hao Xu --- fs/splice.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 047b79db8eb5..b087e00ed079 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -1372,7 +1372,12 @@ static int ipipe_prep(struct pipe_inode_info *pipe, unsigned int flags) return 0; ret = 0; - pipe_lock(pipe); + if (flags & SPLICE_F_NONBLOCK) { + if (!pipe_trylock(pipe)) + return -EAGAIN; + } else { + pipe_lock(pipe); + } while (pipe_empty(pipe->head, pipe->tail)) { if (signal_pending(current)) { @@ -1408,7 +1413,12 @@ static int opipe_prep(struct pipe_inode_info *pipe, unsigned int flags) return 0; ret = 0; - pipe_lock(pipe); + if (flags & SPLICE_F_NONBLOCK) { + if (!pipe_trylock(pipe)) + return -EAGAIN; + } else { + pipe_lock(pipe); + } while (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { if (!pipe->readers) { @@ -1460,7 +1470,12 @@ static int splice_pipe_to_pipe(struct pipe_inode_info *ipipe, * grabbing by pipe info address. Otherwise two different processes * could deadlock (one doing tee from A -> B, the other from B -> A). */ - pipe_double_lock(ipipe, opipe); + if (flags & SPLICE_F_NONBLOCK) { + if (!pipe_double_trylock(ipipe, opipe)) + return -EAGAIN; + } else { + pipe_double_lock(ipipe, opipe); + } i_tail = ipipe->tail; i_mask = ipipe->ring_size - 1; From patchwork Tue Jun 7 08:06:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Xu X-Patchwork-Id: 12871497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A89B3C43334 for ; Tue, 7 Jun 2022 08:07:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238027AbiFGIGz (ORCPT ); Tue, 7 Jun 2022 04:06:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238026AbiFGIGt (ORCPT ); Tue, 7 Jun 2022 04:06:49 -0400 Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EFCDB82DB; Tue, 7 Jun 2022 01:06:48 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1654589207; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=85yunG6FCrDcuT7+AlR9B02vmXCSLoT2S/bCbJh6Eoo=; b=F2AZVzG1VGSPcnSdb/vki/pj0dFYUI9+21bomBczREqoo/DI5hF14cPAKc7MbYG21n5XSw BPFB26WQVzDq5Twilj0JJ4YMbQcUN3KahEB8/0TVq5PuasA7SRuFpRYcPLT9ZGosiGP+JW seCbtSx6tvG+Zg88r5JDOE/wR4bLLcc= From: Hao Xu To: io-uring@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , linux-fsdevel@vger.kernel.org, Alexander Viro , Andrew Morton , Luis Chamberlain , Kuniyuki Iwashima Subject: [PATCH 4/5] io_uring: support nonblock try for splicing from pipe to pipe Date: Tue, 7 Jun 2022 16:06:18 +0800 Message-Id: <20220607080619.513187-5-hao.xu@linux.dev> In-Reply-To: <20220607080619.513187-1-hao.xu@linux.dev> References: <20220607080619.513187-1-hao.xu@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org From: Hao Xu splice() in io_uring is running in a slow way since it fully depends on io workers. While splicing from pipe to pipe is a simpler case compared with file to pipe and pipe to file. Let's make it support nonblock first. This way we get a fast path for splicing fom pipe to pipe. Signed-off-by: Hao Xu Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot --- io_uring/splice.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/io_uring/splice.c b/io_uring/splice.c index b2cd1044c3ee..650c70e3dde1 100644 --- a/io_uring/splice.c +++ b/io_uring/splice.c @@ -88,6 +88,14 @@ int io_splice_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) return __io_splice_prep(req, sqe); } +bool io_splice_support_nowait(struct file *in, struct file *out) +{ + if (get_pipe_info(in, true) && get_pipe_info(out, true)) + return true; + + return false; +} + int io_splice(struct io_kiocb *req, unsigned int issue_flags) { struct io_splice *sp = io_kiocb_to_cmd(req); @@ -100,9 +108,6 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags) if (unlikely(!sp->len)) goto done; - if (issue_flags & IO_URING_F_NONBLOCK) - return -EAGAIN; - if (sp->flags & SPLICE_F_FD_IN_FIXED) in = io_file_get_fixed(req, sp->splice_fd_in, issue_flags); else @@ -112,6 +117,16 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags) goto done; } + if (issue_flags & IO_URING_F_NONBLOCK) { + if (io_splice_support_nowait(in, out)) { + flags |= SPLICE_F_NONBLOCK; + } else { + if (!(sp->flags & SPLICE_F_FD_IN_FIXED)) + io_put_file(in); + return -EAGAIN; + } + } + poff_in = (sp->off_in == -1) ? NULL : &sp->off_in; poff_out = (sp->off_out == -1) ? NULL : &sp->off_out; @@ -119,6 +134,9 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags) if (!(sp->flags & SPLICE_F_FD_IN_FIXED)) io_put_file(in); + if (ret == -EAGAIN) + return ret; + done: if (ret != sp->len) req_set_fail(req); From patchwork Tue Jun 7 08:06:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Xu X-Patchwork-Id: 12871498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E9DDC43334 for ; Tue, 7 Jun 2022 08:07:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238036AbiFGIHB (ORCPT ); Tue, 7 Jun 2022 04:07:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238042AbiFGIGy (ORCPT ); Tue, 7 Jun 2022 04:06:54 -0400 Received: from out1.migadu.com (out1.migadu.com [IPv6:2001:41d0:2:863f::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3049BCC141; Tue, 7 Jun 2022 01:06:54 -0700 (PDT) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1654589212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O6d18kqa76uQ9DQcaTL4/EHavGPVwU9mmGhKMcUClL4=; b=DSEGDP+E4vMVlhWhQ2oUGgB1EBRPniZCVMalbLMIN857ihdTm4xLTNj4rNUs+VYRdEbAnY OPuwwQL5FFhBBEU0ZriWqnoISi23kKOONof7hfoj3XUeZuFSRmaEEsqOqXcMGra+9qA9vd Pq9JRErpUjNT5rNFAEk8TtQSKBeXsmc= From: Hao Xu To: io-uring@vger.kernel.org Cc: Jens Axboe , Pavel Begunkov , linux-fsdevel@vger.kernel.org, Alexander Viro , Andrew Morton , Luis Chamberlain , Kuniyuki Iwashima Subject: [PATCH 5/5] io_uring: add file_in in io_splice{} to avoid duplicate calculation Date: Tue, 7 Jun 2022 16:06:19 +0800 Message-Id: <20220607080619.513187-6-hao.xu@linux.dev> In-Reply-To: <20220607080619.513187-1-hao.xu@linux.dev> References: <20220607080619.513187-1-hao.xu@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org From: Hao Xu Add a member file_in in io_splice{} to avoid duplicate calculation of input file for splice. This is for the case where we do splice from pipe to pipe and get -EAGAIN for the inline submission. Signed-off-by: Hao Xu --- io_uring/splice.c | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/io_uring/splice.c b/io_uring/splice.c index 650c70e3dde1..c97f2971fe7e 100644 --- a/io_uring/splice.c +++ b/io_uring/splice.c @@ -21,6 +21,7 @@ struct io_splice { loff_t off_in; u64 len; int splice_fd_in; + struct file *file_in; unsigned int flags; }; @@ -35,6 +36,7 @@ static int __io_splice_prep(struct io_kiocb *req, if (unlikely(sp->flags & ~valid_flags)) return -EINVAL; sp->splice_fd_in = READ_ONCE(sqe->splice_fd_in); + sp->file_in = NULL; return 0; } @@ -108,34 +110,37 @@ int io_splice(struct io_kiocb *req, unsigned int issue_flags) if (unlikely(!sp->len)) goto done; - if (sp->flags & SPLICE_F_FD_IN_FIXED) - in = io_file_get_fixed(req, sp->splice_fd_in, issue_flags); - else - in = io_file_get_normal(req, sp->splice_fd_in); - if (!in) { - ret = -EBADF; - goto done; + if (!sp->file_in) { + if (sp->flags & SPLICE_F_FD_IN_FIXED) + in = io_file_get_fixed(req, sp->splice_fd_in, issue_flags); + else + in = io_file_get_normal(req, sp->splice_fd_in); + + if (!in) { + ret = -EBADF; + goto done; + } + sp->file_in = in; + } else { + in = sp->file_in; } if (issue_flags & IO_URING_F_NONBLOCK) { - if (io_splice_support_nowait(in, out)) { + if (io_splice_support_nowait(in, out)) flags |= SPLICE_F_NONBLOCK; - } else { - if (!(sp->flags & SPLICE_F_FD_IN_FIXED)) - io_put_file(in); + else return -EAGAIN; - } } poff_in = (sp->off_in == -1) ? NULL : &sp->off_in; poff_out = (sp->off_out == -1) ? NULL : &sp->off_out; ret = do_splice(in, poff_in, out, poff_out, sp->len, flags); + if (ret == -EAGAIN) + return ret; if (!(sp->flags & SPLICE_F_FD_IN_FIXED)) io_put_file(in); - if (ret == -EAGAIN) - return ret; done: if (ret != sp->len)