From patchwork Tue Feb 14 08:37:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13139672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65303C05027 for ; Tue, 14 Feb 2023 08:37:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 046B66B0073; Tue, 14 Feb 2023 03:37:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F3863280003; Tue, 14 Feb 2023 03:37:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E27EF280002; Tue, 14 Feb 2023 03:37:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D4FB66B0073 for ; Tue, 14 Feb 2023 03:37:30 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7EEFDAA9F7 for ; Tue, 14 Feb 2023 08:37:30 +0000 (UTC) X-FDA: 80465243460.23.EDCAE74 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf10.hostedemail.com (Postfix) with ESMTP id CB56FC0003 for ; Tue, 14 Feb 2023 08:37:27 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SA50Sxu7; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676363847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jiirWQEzjOJm38ClQocxHDg9OYUUCbqX/zh69yaIxOk=; b=VcDbiA6JqdCAMi+jHnXW421c8/gMNDPDdmZjVQOw7kAQgkCkZDckhjGnjy4PqaVbP+Tjxn +N6l89Gm1BEkbRoNsXrr04SX9tUvLE2Kwxgm4cqSSL71PorGznM5DpBysRjUTuSf5PdKPF xMmAkI4+ZyyD75wqofNL5qYiZKCR/9g= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SA50Sxu7; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676363847; a=rsa-sha256; cv=none; b=AYkKP5eXQ15SGXvanTI6P8+9FHCBoHUkR2GnmBCIGTyAb8RO/AtKY+wSRBVW3oXmKnL131 Q1XFrncIdLQVZKPZ39+QrYvcZhV/+Y9xhLHOx0YGgMFp0DuyPwFDCioG0VSPUCr1PSG8d6 vECVTS+bxcQq/ntqk5CLRP42W9IKVzE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676363847; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jiirWQEzjOJm38ClQocxHDg9OYUUCbqX/zh69yaIxOk=; b=SA50Sxu7bxTu2TnVlP5DXQM9J32Ss4PvBYoZ+FY/gzMIUr/jqyCa9d2lwJVzjxGmKgPcEo 5dmGYFbMiJuBRe3eo/Ks+jpM+Gyjw6yS0h7aPAleCuwyjB+kBcmcFovEqI+FUI0KTmvg0e 8j8ZazOqDi6CnELxsA6y7QmSsjZUxoc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-593-FzkSCvMjPFy74fuTaPhQVA-1; Tue, 14 Feb 2023 03:37:23 -0500 X-MC-Unique: FzkSCvMjPFy74fuTaPhQVA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E5B48811E6E; Tue, 14 Feb 2023 08:37:22 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id E2E9EC15BA0; Tue, 14 Feb 2023 08:37:14 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v3 1/5] splice: Rename new splice functions Date: Tue, 14 Feb 2023 08:37:06 +0000 Message-Id: <20230214083710.2547248-2-dhowells@redhat.com> In-Reply-To: <20230214083710.2547248-1-dhowells@redhat.com> References: <20230214083710.2547248-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspamd-Queue-Id: CB56FC0003 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: jzpnd1qw5iqmtxnjqm4m75adongfugqu X-HE-Tag: 1676363847-97124 X-HE-Meta: U2FsdGVkX194RrA5ejthzVfXLfqVoynWTX8rnr54psLr2u0rd/gzJ+CTSdKWsDoEoSi3MOSTicv1I9VstG03zzr2l3HrgzI/zKGMcVafud/2v8qK7BPJfoGwhfeu61bg2iMMJsYlsO5P0RyFy/sbqGTQ7a1b3CArZNTrAu8FjTakaT/A2L/JVImrx2uh+LoHG3uX182GoaVnBeru0Syei5XCyZQIs71NRJ7I+4AZ+vjCrtnU16ubH8fxzZnr5Ln7V1WGcqwmzKpuZuscHN/aJXen7vDTXYbumom95DxLHsnnaqAjyT4ktgsDTjADa3jccrmLfirTTdctbaeNYPKz4jq5iH0vEHz1d+eIdcUE7Xo1RCMgS1ftb0LMf2sR0L84IlD0kPAtOPDdF+0ZOacNKbYjHn2rztNY5RgSoB9xWXbptDRIMrR4nQmSA44EoDL82Rnel5E1nlfsHlhBR6h3IvLBIvxoIiqCHb1YJ71KY84rlAtiG5Z1v+gla4UwExNyb9a4Fglfv6KKZBfE4bvEnGNNWvDqVjhXyCR2INxUAQXOZIpKz0F/A6u/GHEQ16iMa0L28RIOmU0GGZXSraTWOJkVoul0jr77hzkcSt5s8QkZtH4i+zQeaBPHZf7xfuRhutM2WeLK9gc8krp0Pajz+RvdxLQO6cGl8Vm7XY4DoBaV8kTpmHQbIEpHQtRJn3+t1zSATdltX62nVkhsZHnLu4soc6BvEjHW1cIe39wkztaxi0RhYyfKhwGnaEm1BYDPCbnmYg2/oIVQWuOAui7DjG4bhq1/NujUq8HPqKPkpuYGa6uLtbjzQfL3k2dSM+gaJexhRx2XdFTTVUe28ASp18iLH2WFVN1KQcNvnJHonfDu5P5gQNxaMgvTA8GnW5duRdwylAL+4AdKQeleeiuQC71CgsTtqYkFCCaY25P0dcNT113YE3FXiDxReQAxP/I4ILjuKpmyE+yehjRf5ww 4FlaFrv4 WJp+a6O3E2/FTfehcDyBgOmkG3YYTn5Z+j1XI1a0BkYUp2tKJ8Y/PJwAJlv1gkvolC1jrhHbLlqVI/oOZ3D49+0iSrM/nPqMNgRpjjIOrC0CDyKXCl5hkrjTwsFw6sFUC+krcpTiJYn8HRpmtbWrFByZJli23V4SjBtm4RM5YHUBfOcZDVbZnm1rnZ46SAad1HTa+UAXiTeuakARdLYpkK2jKbwkjn8+V8TsjVNA1O6OvVRvearfMhP2vs02rMafS59PeqoatXL1c/ItIUnOigT9BHoL9vJ2VxeEhbJG+TVN2GbzepfAjynl8wip39Nlo2XsKGgWbbgGSViWIlTqdwACxgL8a0LHjKNCUUDpiyqNr/yqKyrxGVMDIr/B1JgUrbn8IUK08GEAGtW7auhtGyfR106lmxED27Ou/wQnkEYSJ0VXuX4BmTP9l+DjY4BHxhn9I X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rename generic_file_buffered_splice_read() to filemap_splice_read(). Rename generic_file_direct_splice_read() to direct_splice_read(). Requested-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- fs/splice.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 2717078949a2..91b9e2cb9e03 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -287,9 +287,9 @@ void splice_shrink_spd(struct splice_pipe_desc *spd) * Splice data from an O_DIRECT file into pages and then add them to the output * pipe. */ -static ssize_t generic_file_direct_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, unsigned int flags) +static ssize_t direct_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) { struct iov_iter to; struct bio_vec *bv; @@ -417,10 +417,9 @@ static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, * Splice folios from the pagecache of a buffered (ie. non-O_DIRECT) file into * a pipe. */ -static ssize_t generic_file_buffered_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, - unsigned int flags) +static ssize_t filemap_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) { struct folio_batch fbatch; size_t total_spliced = 0, used, npages; @@ -529,8 +528,8 @@ ssize_t generic_file_splice_read(struct file *in, loff_t *ppos, if (unlikely(!len)) return 0; if (in->f_flags & O_DIRECT) - return generic_file_direct_splice_read(in, ppos, pipe, len, flags); - return generic_file_buffered_splice_read(in, ppos, pipe, len, flags); + return direct_splice_read(in, ppos, pipe, len, flags); + return filemap_splice_read(in, ppos, pipe, len, flags); } EXPORT_SYMBOL(generic_file_splice_read); From patchwork Tue Feb 14 08:37:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13139673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A223EC61DA4 for ; Tue, 14 Feb 2023 08:37:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 43F786B0074; Tue, 14 Feb 2023 03:37:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 415A2280003; Tue, 14 Feb 2023 03:37:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DD45280002; Tue, 14 Feb 2023 03:37:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1B5B46B0074 for ; Tue, 14 Feb 2023 03:37:32 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 00891140D29 for ; Tue, 14 Feb 2023 08:37:31 +0000 (UTC) X-FDA: 80465243502.13.7F31A74 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 46D9AA000A for ; Tue, 14 Feb 2023 08:37:30 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WPriMMal; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676363850; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+b2VMeGpZTyrheqYPKTaHyxDIQ3yZeSE4YqjD4ZyLaI=; b=FkUUWxy/irK+60p9Ti4cTmXK+ZhJOhO3gGbVmPPB9Ooit1oESvCo7GXz2QJN5JOYqjK8pO qmAtW/pPHePoJDHct7cA5ZQJLf+gnspkpABe7cZmscyvWkr7BmufDuX/xD8FaFvNrSXIiF RWtnKfJBWYbyvXAlSneRqj7GBuIgnns= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=WPriMMal; spf=pass (imf25.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676363850; a=rsa-sha256; cv=none; b=FbhZUw4Ex1Cohk2r1CgboZlVFKYRmp2j9/Nc2GCY9twFkae8poISL7rNXJtn9Wv66mFXY/ OW1yj9gsyurPGcLdSY2li/b/KSJ9PbsR1r2XtvBtq8ei1bMV+B8NhAm6tzvcYVFIbOdz/7 7ruB+S1R1O+qufCOsEEY95p9RSAdLmE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676363849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+b2VMeGpZTyrheqYPKTaHyxDIQ3yZeSE4YqjD4ZyLaI=; b=WPriMMalUclN6U9cLZjF8YbWb8+lnGE3c+zjEgEjfOaJvWA9X2SlBguDnxkP1RC6r10mpZ mofcRCP49CdnNopNnsPzBsmaFDISt9ffzY7SAyze8eAYXwKJAYeb//r8Ymx4AKpvf/4j7X 46R7B7xkY3WbDnhg09VwwM70Akc2If0= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-225-1sNGJShAP3avneZS4IyFGA-1; Tue, 14 Feb 2023 03:37:26 -0500 X-MC-Unique: 1sNGJShAP3avneZS4IyFGA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8859E3811F2C; Tue, 14 Feb 2023 08:37:25 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B3BFC15BA0; Tue, 14 Feb 2023 08:37:23 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v3 2/5] splice: Provide pipe_head_buf() helper Date: Tue, 14 Feb 2023 08:37:07 +0000 Message-Id: <20230214083710.2547248-3-dhowells@redhat.com> In-Reply-To: <20230214083710.2547248-1-dhowells@redhat.com> References: <20230214083710.2547248-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: wxkfj9kp4gk7bfzmuwwz95f3tzdyp59z X-Rspamd-Queue-Id: 46D9AA000A X-HE-Tag: 1676363850-907849 X-HE-Meta: U2FsdGVkX1+dNjXnxtsd/rRqtIPlCOxjC/QEgrB213T6HdzAw3Sla2kMrbNLd/MnSo71+fofU+S9ThBiijjVqJ+nkAMcNVi2ya1KkESlNvaqsLnMg1V6JxqlX/B5c4VMrjwD1HmS4Qe/cwq3HX48sMO/NNBMHGntQPYJg7UmECpd5fdomyKw2m7IGr5pME9C99yqO+JAXXcTXZ5uJkmdxv1ibsbQvYoHxqECe/pPm0tXnLO19C23SGDhx2qaOqpxPFCgBDp8oS5WsgbmKbwcu2FYYiusAsHla/7xHZxIWKlxdlNRO2/bglcnIa0M2aTMSr5X7JivOpslTafC8aEr9M7SYYpBc846G/TPHMj1Mcq2KYSplLw20jiubPR9T+qGl4smZI8z9HTVAc+6Vs/YGOYwQp1ZvncXtYWAEC0AIr5KKXB1ksTxgkQSVYr/D3h/IdNJ24MPxkWl6i9NzmqiWxHzyMw50fHyfUaPLENB2I/QUOYwQ09OhlhSlNPRCiwRT19Lz5Q2dZBVAg9DXNa+6Q1t9cjSjY8Yj6uQYt06lqEdMnZoYutUhYw3l2dsiJzH+scmvLgrJtxbd8w9Dqlns6Dfhd5n8ZKoIZUMVeuLtEAM8v2aZTmYsPeNlimfP3HKZK2leawc2/EsoO/ScwK1eg3ZAz8urGQoq1qzP9dKiLXBqHdMgbl2MsJbgDH3O77UgNfwfUzWfgibPWCdNe9/QouXtj2k41r22u+eyFFyfr3VY1tRe0RI87Kbr9MsBY88JniFXe0wBnhM337SltT9rL9tFd95peh6Nl70e/VhJ6H5r/OgSDf0kX+Yuzwx6+DZ24+YG4bdT1PO16FgP5FY5tWlQkbu0nxFn54/IQ2xeBzAJS5aIIjYEl+0aazD/mHGl8yJyBR+zdkB0DFUc3lsDZXGcHJY1ZFWk9kYbhmitjYohV7pmfO1BKLtyQzGJVhOtRQG9uutlpCLcAWcoC5 CQKj/OaO IMp1bRoI0BGbpgtpqbwDK0feml6SsYzlx/nsJtiKJnRFzgKixsEfV6WndDLGuNaqaldxVFdeOmhSw1N/Ekwd3Hz64fyIHnaDvnirQdLLx2rChJy3wq6a8fsaYKHRbwjK0WjP807MrCTwVLj1VO3C7nqx+USmNuVboUNAaCjU1yJ1Zb4voIL80bB1kvF+1rSqj3W6ElzqDy2PxNDo8zbmPPI93ADXf4fzhNDGN22jyyBflvqwGEr2NEl9RStGpnX/OvdyX11LWAHn7WHkokHY0bnhn6EZzOFyfu6gecx3bbA2hxT9wQyGnOU9dgpuLiBzlEV8BX+gY5M2g3/Ne55++8btP7N9VCm5dWLis+AhIjuSRuCfvxrn1+vkvOzujUTHaXAS/Uo57VKBG0e3xLzzWwMwSmMaoWqPx/GBXTqDSyFbeYfjCVMsgN2rv0O5ZQ61tdjrA/QJsywh+oLBiPzp1XGashwHVrjZjamPL3N79Bz6rMtUCH21DGizQfw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Provide a helper, pipe_head_buf(), to get the current head buffer from a pipe. Implement this as a wrapper around a more general function, pipe_buf(), that gets a specified buffer. Requested-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- fs/splice.c | 9 +++------ include/linux/pipe_fs_i.h | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 91b9e2cb9e03..7c0ff187f87a 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -295,7 +295,6 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, struct bio_vec *bv; struct kiocb kiocb; struct page **pages; - unsigned int head; ssize_t ret; size_t used, npages, chunk, remain, reclaim; int i; @@ -358,9 +357,8 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, } /* Push the remaining pages into the pipe. */ - head = pipe->head; for (i = 0; i < npages; i++) { - struct pipe_buffer *buf = &pipe->bufs[head & (pipe->ring_size - 1)]; + struct pipe_buffer *buf = pipe_head_buf(pipe); chunk = min_t(size_t, remain, PAGE_SIZE); *buf = (struct pipe_buffer) { @@ -369,10 +367,9 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, .offset = 0, .len = chunk, }; - head++; + pipe->head++; remain -= chunk; } - pipe->head = head; kfree(bv); return ret; @@ -394,7 +391,7 @@ static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, while (spliced < size && !pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { - struct pipe_buffer *buf = &pipe->bufs[pipe->head & (pipe->ring_size - 1)]; + struct pipe_buffer *buf = pipe_head_buf(pipe); size_t part = min_t(size_t, PAGE_SIZE - offset, size - spliced); *buf = (struct pipe_buffer) { diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h index 6cb65df3e3ba..d2c3f16cf6b1 100644 --- a/include/linux/pipe_fs_i.h +++ b/include/linux/pipe_fs_i.h @@ -156,6 +156,26 @@ static inline bool pipe_full(unsigned int head, unsigned int tail, return pipe_occupancy(head, tail) >= limit; } +/** + * pipe_buf - Return the pipe buffer for the specified slot in the pipe ring + * @pipe: The pipe to access + * @slot: The slot of interest + */ +static inline struct pipe_buffer *pipe_buf(const struct pipe_inode_info *pipe, + unsigned int slot) +{ + return &pipe->bufs[slot & (pipe->ring_size - 1)]; +} + +/** + * pipe_head_buf - Return the pipe buffer at the head of the pipe ring + * @pipe: The pipe to access + */ +static inline struct pipe_buffer *pipe_head_buf(const struct pipe_inode_info *pipe) +{ + return pipe_buf(pipe, pipe->head); +} + /** * pipe_buf_get - get a reference to a pipe_buffer * @pipe: the pipe that the buffer belongs to From patchwork Tue Feb 14 08:37:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13139674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 667F6C05027 for ; Tue, 14 Feb 2023 08:37:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01EE06B0075; Tue, 14 Feb 2023 03:37:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F1149280003; Tue, 14 Feb 2023 03:37:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD961280002; Tue, 14 Feb 2023 03:37:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CC7046B0075 for ; Tue, 14 Feb 2023 03:37:34 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 901511A0D95 for ; Tue, 14 Feb 2023 08:37:34 +0000 (UTC) X-FDA: 80465243628.24.ADDEB68 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id D0649180002 for ; Tue, 14 Feb 2023 08:37:32 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gIAH3LiN; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676363852; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xJXt4KNNFIdMvSSPmBpCRCAi8g0JFMLHorMyvAgfics=; b=aCrBtErhXgkfzhk5GPnASQAIV1n0aQXt9iFpk7NdwvBs/bTLboHGbqV+BYRnSRQErEU4lB 5XoSVG85zy1KKP7gCv87XM8RZtGnsGJpbUpcY/piXz0MFlNiJL4375nSkJB2vipFPoWh4c 5FCfXP2How205Xu6sFXwxGNXACJcPBM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gIAH3LiN; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676363852; a=rsa-sha256; cv=none; b=R4itzD3Y5w4/Dotur2XdnadKLvHu7tvsfS7h653wkPQBswve8FATFHrZf2Ztjr5xfmwwuT 6qXd65TzjhqeWbrEbyNoUjJfd9FuEvMBdoOexLb+FpDx8xZuI+id/hAG5U7yklw70SObbB VlpCsRGLhuwQT90+AG+lf8X6oL44bq0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676363852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xJXt4KNNFIdMvSSPmBpCRCAi8g0JFMLHorMyvAgfics=; b=gIAH3LiNKLVWbWoHrDW/yTd2ncPan6UF/YqqXlD6xnh+d+Z4sCZ3ZUHOgr+dCwRdi6dRKI clXPvg+2orcE9vI9x11UiC2mWfMkZvlEjJQkht7QRpi1/HfWuDOV/uSIaZJXEGRRotlIBZ DBHV0U1xqjr8PvoyjZwGm3AA59g/dW4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-4m-qRrFTP7OlNbKg3ZIAJA-1; Tue, 14 Feb 2023 03:37:28 -0500 X-MC-Unique: 4m-qRrFTP7OlNbKg3ZIAJA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 14FC72999B22; Tue, 14 Feb 2023 08:37:28 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2717F140EBF6; Tue, 14 Feb 2023 08:37:26 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v3 3/5] splice: Use init_sync_kiocb() in filemap_splice_read() Date: Tue, 14 Feb 2023 08:37:08 +0000 Message-Id: <20230214083710.2547248-4-dhowells@redhat.com> In-Reply-To: <20230214083710.2547248-1-dhowells@redhat.com> References: <20230214083710.2547248-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D0649180002 X-Stat-Signature: 39ok58xymuoof9wxx7npd4z8g76ca715 X-HE-Tag: 1676363852-366553 X-HE-Meta: U2FsdGVkX1+8n6vXZH3fqTvyGVbbUkhqvBiNWfuTVJBSdHVdRg807ShB/QR5zHxN7IK3iIMGa7IsJBaBeRRGXfD/CxSvW+GNFWKDqD8XsPeWOobszhmqcT5NkyWqLcypnAwG+Kst0xQ20efVAfNh5cYHjY1efZy9p6pcaYMg6FCk5662G5a35gDwzKqFZOldeYWoa6yd1EBM2tlhAatXIUX9631tMdugPaWy+Uoo+CNS6mwxUhNg437/5gNmb/1Ho76FRKejcHSqUVnQ3T8iuylhl8sc/F2G8qZgFHnE5my5YnkD7m9K2ji8JHXv1fFuHoc/Vz/lIO/Df2t+dVnbUfJ6SO2wBEIQ0NxM1DbySjRmD5k2CkxNNYx+7CfwZFmzLGQxioSBtcnz4XHON481BxuQHrmscufwlVvKJvSxHhdjnTOJHDEYXqcAcZjRmjn8y+EX8JeJHyKlhrf2/0o3/E2d2P1GIVE9Xdd5NzkxVwGI1RjR2mv91JpiyEs8HuME8ythHzbJ4GYw5tG4tGxUP7lXEEpFF0KNbBUxvOHJv5RlQinuM4ugkPHejKF6YjaMSHLAJZxuc97xYjc1QpxecOG5wFUvQzB8JoDv0wuS8lhq2Y46WM5e8agdk9yLmix75TU3gxlpEZ3rAoZhdTzEzJKitqxt/O+1s0Ok09Kg1TNJYtssE7WU1b3c3WA5o9POJi1UzD1mxGyzTnCzughZB20cy/9fZ32bxLjQrRhdCHnz+4abR2LwId9doRESd8Msbpez9ojbN+BjKjv9p+lkPrzkaL4T+3zIv3IoE2H4anI5Lcmcv+bxQIVIgei2wZxy/GhWkm4ZFDXE7AYcUI3dnqjE8wTOmI5LxacVCWQ0fmyHyPqAXxyaQ/BHqn7sTihsA5/XN3jnyqHPjZ2IpQUzxm96ut+9QAgF9yu62t6NAajCWrnlrYz1HPeatBq0G4V+xD67lgzRJ1WHxRPw1oJ mRBbvxne kXbYDMzVly+otmS2aqcVsIihcGFBvu8H7LOtLYziBesOcU16pbSk/5Mkoee+pZOUCkBmx4X+G4Bl/sQQ3NsnwAepFCLknaqXjLQYY1KXcXywQPPo+xEJzBzZEHbL7u7anjkIScOqc+Wf1L8DRtNrPIzs8tY5nVVg93GnlvGMFHARFl14DNJMGpW9Edi8sNwdXWpf2eLX+CjeQBSUdEYv6UeiNY/xhAEQZMgpNA1/n8siHNK2wGMAzjjVZtSayUo2oB/f8cC/qm2sK2p9rsxJNNu47cjX2QqYIH/risFUCyAsX3qC3XbG3kN1NNPbGybMFMMu2nEjq7MQPCZsl0Q4zI4pK8ZxliU1V6q7vcYhU6P86fcLse7KUv02UoCLDiJLS6c0frKUl628OlBPfCZIhh8qXATwPvU4c0BTWUKWQMOH+UMPFiOiz6LjDFu2koC8bFG7Q+jUxJvnj12rk2t3yde328w2t0KHc5/reFRFbfJ9sg55PKc68EPaJ5g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use init_sync_kiocb() in filemap_splice_read() rather than open coding it. Requested-by: Christoph Hellwig Signed-off-by: David Howells cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- Notes: ver #2) - Don't attempt to filter IOCB_* flags. fs/splice.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 7c0ff187f87a..4ea63d6a9040 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -419,15 +419,14 @@ static ssize_t filemap_splice_read(struct file *in, loff_t *ppos, size_t len, unsigned int flags) { struct folio_batch fbatch; + struct kiocb iocb; size_t total_spliced = 0, used, npages; loff_t isize, end_offset; bool writably_mapped; int i, error = 0; - struct kiocb iocb = { - .ki_filp = in, - .ki_pos = *ppos, - }; + init_sync_kiocb(&iocb, in); + iocb.ki_pos = *ppos; /* Work out how much data we can actually add into the pipe */ used = pipe_occupancy(pipe->head, pipe->tail); From patchwork Tue Feb 14 08:37:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13139675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38F5BC6379F for ; Tue, 14 Feb 2023 08:37:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B1DA46B0078; Tue, 14 Feb 2023 03:37:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ACCEA280003; Tue, 14 Feb 2023 03:37:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96D49280002; Tue, 14 Feb 2023 03:37:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 822076B0078 for ; Tue, 14 Feb 2023 03:37:35 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 60BA2140DB3 for ; Tue, 14 Feb 2023 08:37:35 +0000 (UTC) X-FDA: 80465243670.23.B96C095 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id 8949B120011 for ; Tue, 14 Feb 2023 08:37:33 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UU4u+f6D; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676363853; a=rsa-sha256; cv=none; b=k5UHwNLYPcIA7hu1/eFSNvpGsq2xmqjdFTv4YuMsvUzA9mdkparLr3P4UVN4Ml7ngwG7x4 nUDzGu+ruVMEIAjpYjSeA1CddI8bJiz1SkfZwKZO5yjH3h6Wdn2XU/tqHJJYc+9iR9V4dz VO7Cw5q35RsQf/QHci5cGq5+zRg7tgk= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=UU4u+f6D; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676363853; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=49IJ7O8KaG9bUjSCVtwVGeadJYJCODKyAnH6L7XF9aY=; b=ohf0kN4sZ2OgGUo9qzGKdVwuWVSmbPE5mrP7CtZa3dHMuJt4T72+fES5tpUJi5zZTIjn/E nyiLhik+v7RTbV2U5hIqg+Lrrh9nzX/JquEIn3Vfy3mFm0+/muuOd8sFJMmj5To9gtgIif X+eba5HbIpgVuYED7zgXK4yvw3Ir2QI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676363852; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=49IJ7O8KaG9bUjSCVtwVGeadJYJCODKyAnH6L7XF9aY=; b=UU4u+f6D4ATWa2BexKT//KyLUP/0/VJgt8dfBVUMbMfTM2r+a8q/bZ1FslVhlIo6Dv+1b1 0EQtgIhASXRQKfs86Oc4uZW48/OP16lsSHpFEfsqfAa2fg0teQRwt8fe4FvEVcJQzhtlOy ZZ9AU1G+UKyKr8gnws4nt+aUyD8DTD0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-208-whlcj7bIP-qL1QdHi6m4cg-1; Tue, 14 Feb 2023 03:37:31 -0500 X-MC-Unique: whlcj7bIP-qL1QdHi6m4cg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B63CC811E6E; Tue, 14 Feb 2023 08:37:30 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id C013540C945A; Tue, 14 Feb 2023 08:37:28 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v3 4/5] splice: Move filemap_read_splice() to mm/filemap.c Date: Tue, 14 Feb 2023 08:37:09 +0000 Message-Id: <20230214083710.2547248-5-dhowells@redhat.com> In-Reply-To: <20230214083710.2547248-1-dhowells@redhat.com> References: <20230214083710.2547248-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspam-User: X-Rspamd-Queue-Id: 8949B120011 X-Rspamd-Server: rspam01 X-Stat-Signature: rwwtmtjenmgi6thyajtorc7bqdan7fwi X-HE-Tag: 1676363853-161492 X-HE-Meta: U2FsdGVkX18++xpqEt30LwaC8lDxVOBhLAKW5RC1NIL4DAseLmpXkeF0aUHxwbuThb9hY1dO/OTcLXz+7Yb1SflQIZs+Isbbb9KIMdZVAuRqNaGwemK8CgABrhhLXEFYD1qCc7QncNw3gF7TER0LVQ72wzJH5IVcZwNrfKDlIW75uTOhcxhc6zqE4osQEht4FLLfmKE0rDktklbzmqP+VmQYZrZiT8V5uiTYKiax0yJSXYQyRsMghFsIgTsSVqpyff7NCCbJA1XwT6AgXJ+Kss/hUMzxSYz6dVpPS7J+paiwvR6hoUJqckRTweSjP4gu2gmKZBEkMTLrCo1DeDmvgwMnHle9TvNYd/B4F6Nf08W2wYO2kv4aurlIPnUXGEbSyR9d5TsTAERRy2k8y8+hDjlmzm/YXrjX+jOCqP9o7/e2Uc+aTKWj4bBe42yTVwMR6Uns/jum/v71/4HIOprrRxlTZ5niYH5cl1bp9lWWynhbm/CIxk2GGkGIlzSEvMaizeu+U142Dzhrfh16DAtlW7XnH30gfIoL+r7kX70f47BJFrTZiLhen4D+yMymmQKRwBGQNoBrl5hDl87NXoG7tePHZMlTKfPtN9RirbOxGPHOhic4ZvkEtewY6ivYtUPjIlmLF0gB9/Qq49YvhLaAt4DPkqPY9UkTY4vYigFupnvGa05s5+wfchZnW+wXCPBNauejGTej3DB7yTFvUyWY5sgKEM17EHVneyHdQ3o60av7BYjONMY9dF+fSwPHRdor1k2NHoyo4JPhSgermh9ZGvU0KPCvYt6u9819z586bi4DWRFfKurv1oxwwohbWW4V6Qf7MHdbFTTWL1/SbQg35DLnGUckFTns33nnGfA9eAKVM98h9UciBhNJ+ON2yeGpdPy5MRFfzD6t7CNh4S8+jr9+MZJRPhIl9zrumrY6DfDFbMUlbszhavnJkdGmqWsQI1xJKsEEOlZGex83poS 3zeXVR4J tgvbKuuUoMCm8wvsAsFnyRyCkTFtYXGt9AczDRi/csbFIhxw5oLmhyX2QAJhePWUQWB4elMe82ICW83BY5r9wGEi1P7DrrV2OOyBc6Alxy0bNq/Y2MPMeDWuVBFP0ZGX3yp+zCbj0qT00gLs32syp0wltKCW2YF1zhVy5wFCmDZjW7A76miAgItBHzVE9MpaFtg9YW489jy+EFQfqKsyrj5fVRbkGtUFGRYsrsIkTbGUAVZS25LrY4phNaMrnbmRmlbpG7VVQVKuJz2G7HV9o1ZPlWpKuorhLPdHe7ukRrS3qnWUhtKB/z3vg9yBH70tLmh5Yt4JLYrkyb09uX8tkDFbT88FO0cJDZXBEEU3RZTiMeAzmL+mc8BRBAGj1Ob6LJdHOHfcNttK0IkuSY+xU/W2qRrI7X2mfeDMKtcHjOMrkfQrhUFcDLkwaVoSpSogQ8f5reMKGTc00O1lck3AVTcSsfAu6W5/MnZpnjtA5qxLvXqyCFiysYF5JcA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move filemap_read_splice() to mm/filemap.c and make filemap_get_pages() static again. Requested-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- fs/splice.c | 127 ------------------------------------- include/linux/pagemap.h | 2 - include/linux/splice.h | 4 ++ mm/filemap.c | 137 ++++++++++++++++++++++++++++++++++++++-- 4 files changed, 135 insertions(+), 135 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 4ea63d6a9040..341cd8fb47a8 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -375,133 +375,6 @@ static ssize_t direct_splice_read(struct file *in, loff_t *ppos, return ret; } -/* - * Splice subpages from a folio into a pipe. - */ -static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, - struct folio *folio, - loff_t fpos, size_t size) -{ - struct page *page; - size_t spliced = 0, offset = offset_in_folio(folio, fpos); - - page = folio_page(folio, offset / PAGE_SIZE); - size = min(size, folio_size(folio) - offset); - offset %= PAGE_SIZE; - - while (spliced < size && - !pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { - struct pipe_buffer *buf = pipe_head_buf(pipe); - size_t part = min_t(size_t, PAGE_SIZE - offset, size - spliced); - - *buf = (struct pipe_buffer) { - .ops = &page_cache_pipe_buf_ops, - .page = page, - .offset = offset, - .len = part, - }; - folio_get(folio); - pipe->head++; - page++; - spliced += part; - offset = 0; - } - - return spliced; -} - -/* - * Splice folios from the pagecache of a buffered (ie. non-O_DIRECT) file into - * a pipe. - */ -static ssize_t filemap_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, unsigned int flags) -{ - struct folio_batch fbatch; - struct kiocb iocb; - size_t total_spliced = 0, used, npages; - loff_t isize, end_offset; - bool writably_mapped; - int i, error = 0; - - init_sync_kiocb(&iocb, in); - iocb.ki_pos = *ppos; - - /* Work out how much data we can actually add into the pipe */ - used = pipe_occupancy(pipe->head, pipe->tail); - npages = max_t(ssize_t, pipe->max_usage - used, 0); - len = min_t(size_t, len, npages * PAGE_SIZE); - - folio_batch_init(&fbatch); - - do { - cond_resched(); - - if (*ppos >= i_size_read(file_inode(in))) - break; - - iocb.ki_pos = *ppos; - error = filemap_get_pages(&iocb, len, &fbatch, true); - if (error < 0) - break; - - /* - * i_size must be checked after we know the pages are Uptodate. - * - * Checking i_size after the check allows us to calculate - * the correct value for "nr", which means the zero-filled - * part of the page is not copied back to userspace (unless - * another truncate extends the file - this is desired though). - */ - isize = i_size_read(file_inode(in)); - if (unlikely(*ppos >= isize)) - break; - end_offset = min_t(loff_t, isize, *ppos + len); - - /* - * Once we start copying data, we don't want to be touching any - * cachelines that might be contended: - */ - writably_mapped = mapping_writably_mapped(in->f_mapping); - - for (i = 0; i < folio_batch_count(&fbatch); i++) { - struct folio *folio = fbatch.folios[i]; - size_t n; - - if (folio_pos(folio) >= end_offset) - goto out; - folio_mark_accessed(folio); - - /* - * If users can be writing to this folio using arbitrary - * virtual addresses, take care of potential aliasing - * before reading the folio on the kernel side. - */ - if (writably_mapped) - flush_dcache_folio(folio); - - n = splice_folio_into_pipe(pipe, folio, *ppos, len); - if (!n) - goto out; - len -= n; - total_spliced += n; - *ppos += n; - in->f_ra.prev_pos = *ppos; - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - goto out; - } - - folio_batch_release(&fbatch); - } while (len); - -out: - folio_batch_release(&fbatch); - file_accessed(in); - - return total_spliced ? total_spliced : error; -} - /** * generic_file_splice_read - splice data from file to a pipe * @in: file to splice from diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 3a7bdb35acff..29e1f9e76eb6 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -748,8 +748,6 @@ struct page *read_cache_page(struct address_space *, pgoff_t index, filler_t *filler, struct file *file); extern struct page * read_cache_page_gfp(struct address_space *mapping, pgoff_t index, gfp_t gfp_mask); -int filemap_get_pages(struct kiocb *iocb, size_t count, - struct folio_batch *fbatch, bool need_uptodate); static inline struct page *read_mapping_page(struct address_space *mapping, pgoff_t index, struct file *file) diff --git a/include/linux/splice.h b/include/linux/splice.h index a55179fd60fc..691c44ef5c0b 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -67,6 +67,10 @@ typedef int (splice_actor)(struct pipe_inode_info *, struct pipe_buffer *, typedef int (splice_direct_actor)(struct pipe_inode_info *, struct splice_desc *); +ssize_t filemap_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags); + extern ssize_t splice_from_pipe(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int, splice_actor *); diff --git a/mm/filemap.c b/mm/filemap.c index 6970be64a3e0..e1ee267675d2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -42,6 +42,8 @@ #include #include #include +#include +#include #include #include #include "internal.h" @@ -2576,12 +2578,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, return 0; } -/* - * Extract some folios from the pagecache of a file, reading those pages from - * the backing store if necessary and waiting for them. - */ -int filemap_get_pages(struct kiocb *iocb, size_t count, - struct folio_batch *fbatch, bool need_uptodate) +static int filemap_get_pages(struct kiocb *iocb, size_t count, + struct folio_batch *fbatch, bool need_uptodate) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2845,6 +2843,133 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) } EXPORT_SYMBOL(generic_file_read_iter); +/* + * Splice subpages from a folio into a pipe. + */ +static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, + struct folio *folio, + loff_t fpos, size_t size) +{ + struct page *page; + size_t spliced = 0, offset = offset_in_folio(folio, fpos); + + page = folio_page(folio, offset / PAGE_SIZE); + size = min(size, folio_size(folio) - offset); + offset %= PAGE_SIZE; + + while (spliced < size && + !pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { + struct pipe_buffer *buf = pipe_head_buf(pipe); + size_t part = min_t(size_t, PAGE_SIZE - offset, size - spliced); + + *buf = (struct pipe_buffer) { + .ops = &page_cache_pipe_buf_ops, + .page = page, + .offset = offset, + .len = part, + }; + folio_get(folio); + pipe->head++; + page++; + spliced += part; + offset = 0; + } + + return spliced; +} + +/* + * Splice folios from the pagecache of a buffered (ie. non-O_DIRECT) file into + * a pipe. + */ +ssize_t filemap_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct folio_batch fbatch; + struct kiocb iocb; + size_t total_spliced = 0, used, npages; + loff_t isize, end_offset; + bool writably_mapped; + int i, error = 0; + + init_sync_kiocb(&iocb, in); + iocb.ki_pos = *ppos; + + /* Work out how much data we can actually add into the pipe */ + used = pipe_occupancy(pipe->head, pipe->tail); + npages = max_t(ssize_t, pipe->max_usage - used, 0); + len = min_t(size_t, len, npages * PAGE_SIZE); + + folio_batch_init(&fbatch); + + do { + cond_resched(); + + if (*ppos >= i_size_read(file_inode(in))) + break; + + iocb.ki_pos = *ppos; + error = filemap_get_pages(&iocb, len, &fbatch, true); + if (error < 0) + break; + + /* + * i_size must be checked after we know the pages are Uptodate. + * + * Checking i_size after the check allows us to calculate + * the correct value for "nr", which means the zero-filled + * part of the page is not copied back to userspace (unless + * another truncate extends the file - this is desired though). + */ + isize = i_size_read(file_inode(in)); + if (unlikely(*ppos >= isize)) + break; + end_offset = min_t(loff_t, isize, *ppos + len); + + /* + * Once we start copying data, we don't want to be touching any + * cachelines that might be contended: + */ + writably_mapped = mapping_writably_mapped(in->f_mapping); + + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + size_t n; + + if (folio_pos(folio) >= end_offset) + goto out; + folio_mark_accessed(folio); + + /* + * If users can be writing to this folio using arbitrary + * virtual addresses, take care of potential aliasing + * before reading the folio on the kernel side. + */ + if (writably_mapped) + flush_dcache_folio(folio); + + n = splice_folio_into_pipe(pipe, folio, *ppos, len); + if (!n) + goto out; + len -= n; + total_spliced += n; + *ppos += n; + in->f_ra.prev_pos = *ppos; + if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) + goto out; + } + + folio_batch_release(&fbatch); + } while (len); + +out: + folio_batch_release(&fbatch); + file_accessed(in); + + return total_spliced ? total_spliced : error; +} + static inline loff_t folio_seek_hole_data(struct xa_state *xas, struct address_space *mapping, struct folio *folio, loff_t start, loff_t end, bool seek_data) From patchwork Tue Feb 14 08:37:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13139676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F71EC05027 for ; Tue, 14 Feb 2023 08:37:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CEFDD6B007B; Tue, 14 Feb 2023 03:37:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CA04B280003; Tue, 14 Feb 2023 03:37:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1A94280002; Tue, 14 Feb 2023 03:37:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9F7586B007B for ; Tue, 14 Feb 2023 03:37:42 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 62EAA160E38 for ; Tue, 14 Feb 2023 08:37:42 +0000 (UTC) X-FDA: 80465243964.29.7E8B40C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 9461AC000E for ; Tue, 14 Feb 2023 08:37:40 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=idNAs3u6; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676363860; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Aqt7Ie9m7VBTVoG66ytoaEfOzMGB0BP8eJc00dhMkTk=; b=AVJYremJb2bbk/J3nwx4ewi0ECUp+/yJdqbpG2oupizi0yRKIlnSlQ4WPo1meQEjhxmR5Q +yUViAz7x2KursTUAvK6Bg4qc9OyGc/oc5sIHbxGYMyE0uZ+1x7kjQCYn0VOOlHNMeyhHs xsejKnISLmXHHqu/4XggiXc4Frrv5kc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=idNAs3u6; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676363860; a=rsa-sha256; cv=none; b=DRcwLMwQCHQ34bXnONlG2N36cKnho3WXXg6uzecO+Wh7ts39/dKtyGcoivNa+45ChM+qC4 OfBva3gOEFzs969bzRUKlOu9IwAXkRmB8ADg1TdlI2VdGVLNyDgkBdNVtKOPBMYa2vEnqn q6VAKa1Nux2ZbBCduGDGL+wummifq1U= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1676363859; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Aqt7Ie9m7VBTVoG66ytoaEfOzMGB0BP8eJc00dhMkTk=; b=idNAs3u6Jx9T7p9AoA0T7XiO7pOeb4RoHEZrBKwgvjQLTyevHsi9LfjLKjFK1sOkUiisUc gtSsZNjsBD9dR9mFD8ZYUBZOETUfbkCOLxLVL79g2VNHwm9FB3QGndVlWPjdPn35yx/YRs zx7jLeowbeEpE+xpkJFKJ9gp3bar7pU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-639-REqII07MME2i0C0sg7aRjw-1; Tue, 14 Feb 2023 03:37:36 -0500 X-MC-Unique: REqII07MME2i0C0sg7aRjw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 151B7857A9F; Tue, 14 Feb 2023 08:37:35 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.24]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5598F140EBF6; Tue, 14 Feb 2023 08:37:31 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Daniel Golle , Guenter Roeck , Christoph Hellwig , John Hubbard , Miklos Szeredi , Hugh Dickins , Jan Harkes , Arnd Bergmann , Greg Kroah-Hartman , coda@cs.cmu.edu, codalist@coda.cs.cmu.edu, linux-unionfs@vger.kernel.org Subject: [PATCH v3 5/5] shmem, overlayfs, coda, tty, proc, kernfs, random: Fix splice-read Date: Tue, 14 Feb 2023 08:37:10 +0000 Message-Id: <20230214083710.2547248-6-dhowells@redhat.com> In-Reply-To: <20230214083710.2547248-1-dhowells@redhat.com> References: <20230214083710.2547248-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Stat-Signature: 1sfccbcsipkn18zgwgiq3godeogzfa81 X-Rspam-User: X-Rspamd-Queue-Id: 9461AC000E X-Rspamd-Server: rspam06 X-HE-Tag: 1676363860-705885 X-HE-Meta: U2FsdGVkX1+OqCNqea2ef5L2d39uD6Mdus0nF6osEc1xL8hBewTwUfVJSxuiHsG5ouR3DoWytLMngSZkban8kcwEPZXPN1+42cWM9IE1587HSUVNzEyEmVkLoW4ORkYQp2qB+QnswgYqXKZJgMT9Umz85RQW0/OIKn++Z6gqcQT2yh6bmjouxj9J7HEDoSwbt3TiMBjzJfNlfyRA56z4HpcPv3swYoxE5vGHliKwLugNH1rnRaVaT11DF/esUIcPOft6vI89ok7v8fA3D8Tvgkbet0Y/8kuDaQh7bznHmY6B1V/lICuKTT1wgM/RTBpDwLsLAU8O/TUz98drkrwNETOjGNYnKi/0cDVG8d+PEdUwYjgjRkJ0gnq/F6Q/jsVRSKIbbhUd9HSpt7mNXPEmTQEykYIEY1g5YxBf3ytw73V1x07o0YxGx2PrJJQLTldtKw3FZ9pJsvwBXWX4Y5xKUkKx6WmerN4LGTq7rG2IJ7++dmHr7lhxtqy5LqGsrfl6DDRs6ZcP/+v9YSze0puzBl+PY+fPvqDMqywFR+txn4pOX77qFlpNmGo03BJiq3WaiESKDwXg2LnrSfmLrhEalAlPklnLL58QuF4/67jlry6td+rUSAR7jYL18FaLfWDv1yI/H7tmpK5JsOpFeKt1hFx3pBnUN+yUemM2lNjvzIC9H+GIAsFDHWiA7px0as8reSzemhIItxZ6RO84ekXVRSI2UywBnm2fYoIUbX18TdolygFAxQP68aJ1SzQfAJ0jNH+qw7v92bXh+anFDFzhQwUku+zhiJ6ou2HvZpff8poVangCvb6pQY1H9CTV8nqXz6jeEraOHtzDT+zMJp7SIw5qTS/XCXzMYoFCV7xPM0N2EacdDXFY0eCH6OnV26srANXzulEsZ9mtrUKRZT8TcpPcfp9Afuamr6KC8bbAM6GuQtcnbtTZyGhCgsrim/Pe0GTpwxi3P6ZT7lezZeb SD5yIOGK 6OxkisKRahHyCTZDy+4GaNc0DNXWA0BarPKYHHSTT6iaVmeOugfOh0LDSI6mODza47k4EpoBkAHnoD8e3wlIPY2mb+s7dL9McaluPJy3dZbtn9x8tCWJYilIxMdg+E80FJ4t9Zph3GPJiKYSZoLwFn2t+vZ07bC2EhxoeXhg8Zgy+/rOMD2Qm9G7Yngq9fwHznNKc6IIOGEpvp0fpg35DzOq2pT1RXq4BfFPZNfNs02KUovtmc+TZvJ5vYNPvnsgIxFYMg0hwRvQUNdxroUd722/4kddJEPKA3okUfiUzQqxOKEimW/V6cPhxNfucWfMBuRQoA611DWsa3OimVGAp1H60vJ8SnxVdqyOnue9V5fgSopshVhAHROiHl0wVRZtDVwU4yl6XoG/JJFSNMDECcbRaNrVJFdCEgeEizpAxHOaLpEsLUL/DhvHXfsFkpLnLjL+svJTqE2Q1d+u0RkMmMDYSEGqJhrjbUFfVEm6UGsu1NO2pKTM8o/OfEynaB1rUDxlKlvWO5OVHnV1IVWamRTh4Oippc9R0QX4uvXseGQLJ5LG0JG21xHEFCbBz/NGQsj/v8/r2yMNc1GPSxHw0GXkCTZ/GdHaRHO5rjq9IzRSPyH0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The new filemap_splice_read() has an implicit expectation via filemap_get_pages() that ->read_folio() exists if ->readahead() doesn't fully populate the pagecache of the file it is reading from[1], potentially leading to a jump to NULL if this doesn't exist. A filesystem or driver shouldn't suffer from this if: - It doesn't set ->splice_read() - It implements ->read_folio() - It implements its own ->splice_read() Note that some filesystems set generic_file_splice_read() and generic_file_read_iter() but don't set ->read_folio(). g_f_read_iter() will fall back to filemap_read_iter() which looks like it should suffer from the same issue. Certain drivers, can just use direct_splice_read() rather than generic_file_splice_read() as that creates an output buffer and then just calls their ->read_iter() function: - random & urandom - tty - kernfs - proc - proc_namespace Stacked filesystems just need to pass the operation down a layer: - coda - overlayfs And finally, there's shmem (used in tmpfs, ramfs, rootfs). This needs its own splice-read implementation, based on filemap_splice_read(), but able to paste in zero_page when there's a page missing. Fixes: d9722a475711 ("splice: Do splice read from a buffered file without using ITER_PIPE") Signed-off-by: David Howells cc: Daniel Golle cc: Guenter Roeck cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Miklos Szeredi cc: Hugh Dickins cc: Jan Harkes cc: Arnd Bergmann cc: Greg Kroah-Hartman cc: coda@cs.cmu.edu cc: codalist@coda.cs.cmu.edu cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/Y+pdHFFTk1TTEBsO@makrotopia.org/ [1] Acked-by: Greg Kroah-Hartman Tested-by: Daniel Golle --- drivers/char/random.c | 4 +- drivers/tty/tty_io.c | 4 +- fs/coda/file.c | 36 +++++++++++- fs/kernfs/file.c | 2 +- fs/overlayfs/file.c | 36 +++++++++++- fs/proc/inode.c | 4 +- fs/proc/proc_sysctl.c | 2 +- fs/proc_namespace.c | 6 +- fs/splice.c | 6 +- include/linux/fs.h | 6 ++ include/linux/splice.h | 4 -- mm/filemap.c | 5 +- mm/internal.h | 6 ++ mm/shmem.c | 124 ++++++++++++++++++++++++++++++++++++++++- 14 files changed, 221 insertions(+), 24 deletions(-) diff --git a/drivers/char/random.c b/drivers/char/random.c index ce3ccd172cc8..792713616ba8 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -1546,7 +1546,7 @@ const struct file_operations random_fops = { .compat_ioctl = compat_ptr_ioctl, .fasync = random_fasync, .llseek = noop_llseek, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; @@ -1557,7 +1557,7 @@ const struct file_operations urandom_fops = { .compat_ioctl = compat_ptr_ioctl, .fasync = random_fasync, .llseek = noop_llseek, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c index 3149114bf130..495678e9b95e 100644 --- a/drivers/tty/tty_io.c +++ b/drivers/tty/tty_io.c @@ -466,7 +466,7 @@ static const struct file_operations tty_fops = { .llseek = no_llseek, .read_iter = tty_read, .write_iter = tty_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .poll = tty_poll, .unlocked_ioctl = tty_ioctl, @@ -481,7 +481,7 @@ static const struct file_operations console_fops = { .llseek = no_llseek, .read_iter = tty_read, .write_iter = redirected_tty_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .poll = tty_poll, .unlocked_ioctl = tty_ioctl, diff --git a/fs/coda/file.c b/fs/coda/file.c index 3f3c81e6b1ab..33cd7880d30e 100644 --- a/fs/coda/file.c +++ b/fs/coda/file.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include "coda_psdev.h" @@ -94,6 +95,39 @@ coda_file_write_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } +static ssize_t +coda_file_splice_read(struct file *coda_file, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *coda_inode = file_inode(coda_file); + struct coda_file_info *cfi = coda_ftoc(coda_file); + struct file *in = cfi->cfi_container; + loff_t ki_pos = *ppos; + ssize_t ret; + + if (!in->f_op->splice_read) + return -EINVAL; + + ret = rw_verify_area(READ, in, ppos, len); + if (unlikely(ret < 0)) + return ret; + + ret = venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ); + if (ret) + goto finish_read; + + ret = in->f_op->splice_read(in, ppos, pipe, len, flags); + +finish_read: + venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ_FINISH); + return ret; +} + static void coda_vm_open(struct vm_area_struct *vma) { @@ -302,5 +336,5 @@ const struct file_operations coda_file_operations = { .open = coda_open, .release = coda_release, .fsync = coda_fsync, - .splice_read = generic_file_splice_read, + .splice_read = coda_file_splice_read, }; diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index e4a50e4ff0d2..9d23b8141db7 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -1011,7 +1011,7 @@ const struct file_operations kernfs_file_fops = { .release = kernfs_fop_release, .poll = kernfs_fop_poll, .fsync = noop_fsync, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index c9d0c362c7ef..267b61df6fcd 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -419,6 +419,40 @@ static ssize_t ovl_write_iter(struct kiocb *iocb, struct iov_iter *iter) return ret; } +static ssize_t ovl_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags) +{ + const struct cred *old_cred; + struct fd real; + ssize_t ret; + + ret = ovl_real_fdget(in, &real); + if (ret) + return ret; + + ret = -EINVAL; + if (in->f_flags & O_DIRECT && + !(real.file->f_mode & FMODE_CAN_ODIRECT)) + goto out_fdput; + if (!real.file->f_op->splice_read) + goto out_fdput; + + ret = rw_verify_area(READ, in, ppos, len); + if (unlikely(ret < 0)) + return ret; + + old_cred = ovl_override_creds(file_inode(in)->i_sb); + ret = real.file->f_op->splice_read(real.file, ppos, pipe, len, flags); + + revert_creds(old_cred); + ovl_file_accessed(in); +out_fdput: + fdput(real); + + return ret; +} + /* * Calling iter_file_splice_write() directly from overlay's f_op may deadlock * due to lock order inversion between pipe->mutex in iter_file_splice_write() @@ -695,7 +729,7 @@ const struct file_operations ovl_file_operations = { .fallocate = ovl_fallocate, .fadvise = ovl_fadvise, .flush = ovl_flush, - .splice_read = generic_file_splice_read, + .splice_read = ovl_splice_read, .splice_write = ovl_splice_write, .copy_file_range = ovl_copy_file_range, diff --git a/fs/proc/inode.c b/fs/proc/inode.c index f495fdb39151..711f12706469 100644 --- a/fs/proc/inode.c +++ b/fs/proc/inode.c @@ -591,7 +591,7 @@ static const struct file_operations proc_iter_file_ops = { .llseek = proc_reg_llseek, .read_iter = proc_reg_read_iter, .write = proc_reg_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .poll = proc_reg_poll, .unlocked_ioctl = proc_reg_unlocked_ioctl, .mmap = proc_reg_mmap, @@ -617,7 +617,7 @@ static const struct file_operations proc_reg_file_ops_compat = { static const struct file_operations proc_iter_file_ops_compat = { .llseek = proc_reg_llseek, .read_iter = proc_reg_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .write = proc_reg_write, .poll = proc_reg_poll, .unlocked_ioctl = proc_reg_unlocked_ioctl, diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c index 48f2d60bd78a..92533bd0e67b 100644 --- a/fs/proc/proc_sysctl.c +++ b/fs/proc/proc_sysctl.c @@ -869,7 +869,7 @@ static const struct file_operations proc_sys_file_operations = { .poll = proc_sys_poll, .read_iter = proc_sys_read, .write_iter = proc_sys_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .llseek = default_llseek, }; diff --git a/fs/proc_namespace.c b/fs/proc_namespace.c index 846f9455ae22..492abbbeff5e 100644 --- a/fs/proc_namespace.c +++ b/fs/proc_namespace.c @@ -324,7 +324,7 @@ static int mountstats_open(struct inode *inode, struct file *file) const struct file_operations proc_mounts_operations = { .open = mounts_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, .poll = mounts_poll, @@ -333,7 +333,7 @@ const struct file_operations proc_mounts_operations = { const struct file_operations proc_mountinfo_operations = { .open = mountinfo_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, .poll = mounts_poll, @@ -342,7 +342,7 @@ const struct file_operations proc_mountinfo_operations = { const struct file_operations proc_mountstats_operations = { .open = mountstats_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, }; diff --git a/fs/splice.c b/fs/splice.c index 341cd8fb47a8..0708cf0d12b7 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -287,9 +287,9 @@ void splice_shrink_spd(struct splice_pipe_desc *spd) * Splice data from an O_DIRECT file into pages and then add them to the output * pipe. */ -static ssize_t direct_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, unsigned int flags) +ssize_t direct_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) { struct iov_iter to; struct bio_vec *bv; diff --git a/include/linux/fs.h b/include/linux/fs.h index c1769a2c5d70..551c9403f9b3 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -3163,6 +3163,12 @@ ssize_t vfs_iocb_iter_write(struct file *file, struct kiocb *iocb, struct iov_iter *iter); /* fs/splice.c */ +ssize_t filemap_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags); +ssize_t direct_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags); extern ssize_t generic_file_splice_read(struct file *, loff_t *, struct pipe_inode_info *, size_t, unsigned int); extern ssize_t iter_file_splice_write(struct pipe_inode_info *, diff --git a/include/linux/splice.h b/include/linux/splice.h index 691c44ef5c0b..a55179fd60fc 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -67,10 +67,6 @@ typedef int (splice_actor)(struct pipe_inode_info *, struct pipe_buffer *, typedef int (splice_direct_actor)(struct pipe_inode_info *, struct splice_desc *); -ssize_t filemap_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, - size_t len, unsigned int flags); - extern ssize_t splice_from_pipe(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int, splice_actor *); diff --git a/mm/filemap.c b/mm/filemap.c index e1ee267675d2..c01bbcb9fa92 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2846,9 +2846,8 @@ EXPORT_SYMBOL(generic_file_read_iter); /* * Splice subpages from a folio into a pipe. */ -static size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, - struct folio *folio, - loff_t fpos, size_t size) +size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, + struct folio *folio, loff_t fpos, size_t size) { struct page *page; size_t spliced = 0, offset = offset_in_folio(folio, fpos); diff --git a/mm/internal.h b/mm/internal.h index bcf75a8b032d..6d4ca98f3844 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -794,6 +794,12 @@ struct migration_target_control { gfp_t gfp_mask; }; +/* + * mm/filemap.c + */ +size_t splice_folio_into_pipe(struct pipe_inode_info *pipe, + struct folio *folio, loff_t fpos, size_t size); + /* * mm/vmalloc.c */ diff --git a/mm/shmem.c b/mm/shmem.c index 0005ab2c29af..5a3cc74aba28 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2711,6 +2711,128 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return retval ? retval : error; } +static bool zero_pipe_buf_try_steal(struct pipe_inode_info *pipe, + struct pipe_buffer *buf) +{ + return false; +} + +static const struct pipe_buf_operations zero_pipe_buf_ops = { + .release = generic_pipe_buf_release, + .try_steal = zero_pipe_buf_try_steal, + .get = generic_pipe_buf_get, +}; + +static size_t splice_zeropage_into_pipe(struct pipe_inode_info *pipe, + loff_t fpos, size_t size) +{ + size_t offset = fpos & ~PAGE_MASK; + + size = min(size, PAGE_SIZE - offset); + + if (!pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { + struct pipe_buffer *buf = pipe_head_buf(pipe); + + *buf = (struct pipe_buffer) { + .ops = &zero_pipe_buf_ops, + .page = ZERO_PAGE(0), + .offset = offset, + .len = size, + }; + get_page(buf->page); + pipe->head++; + } + + return size; +} + +static ssize_t shmem_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode = file_inode(in); + struct address_space *mapping = inode->i_mapping; + struct folio *folio = NULL; + size_t total_spliced = 0, used, npages, n, part; + loff_t isize; + int error = 0; + + /* Work out how much data we can actually add into the pipe */ + used = pipe_occupancy(pipe->head, pipe->tail); + npages = max_t(ssize_t, pipe->max_usage - used, 0); + len = min_t(size_t, len, npages * PAGE_SIZE); + + do { + if (*ppos >= i_size_read(inode)) + break; + + error = shmem_get_folio(inode, *ppos / PAGE_SIZE, &folio, SGP_READ); + if (error) { + if (error == -EINVAL) + error = 0; + break; + } + if (folio) { + folio_unlock(folio); + + if (folio_test_hwpoison(folio)) { + error = -EIO; + break; + } + } + + /* + * i_size must be checked after we know the pages are Uptodate. + * + * Checking i_size after the check allows us to calculate + * the correct value for "nr", which means the zero-filled + * part of the page is not copied back to userspace (unless + * another truncate extends the file - this is desired though). + */ + isize = i_size_read(inode); + if (unlikely(*ppos >= isize)) + break; + part = min_t(loff_t, isize - *ppos, len); + + if (folio) { + /* + * If users can be writing to this page using arbitrary + * virtual addresses, take care about potential aliasing + * before reading the page on the kernel side. + */ + if (mapping_writably_mapped(mapping)) + flush_dcache_folio(folio); + folio_mark_accessed(folio); + /* + * Ok, we have the page, and it's up-to-date, so we can + * now splice it into the pipe. + */ + n = splice_folio_into_pipe(pipe, folio, *ppos, part); + folio_put(folio); + folio = NULL; + } else { + n = splice_zeropage_into_pipe(pipe, *ppos, len); + } + + if (!n) + break; + len -= n; + total_spliced += n; + *ppos += n; + in->f_ra.prev_pos = *ppos; + if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) + break; + + cond_resched(); + } while (len); + + if (folio) + folio_put(folio); + + file_accessed(in); + return total_spliced ? total_spliced : error; +} + static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence) { struct address_space *mapping = file->f_mapping; @@ -3929,7 +4051,7 @@ static const struct file_operations shmem_file_operations = { .read_iter = shmem_file_read_iter, .write_iter = generic_file_write_iter, .fsync = noop_fsync, - .splice_read = generic_file_splice_read, + .splice_read = shmem_file_splice_read, .splice_write = iter_file_splice_write, .fallocate = shmem_fallocate, #endif