From patchwork Wed Mar 8 16:52:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BD68C64EC4 for ; Wed, 8 Mar 2023 16:53:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8AF976B007B; Wed, 8 Mar 2023 11:53:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 838A36B007D; Wed, 8 Mar 2023 11:53:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B17D6B007E; Wed, 8 Mar 2023 11:53:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 51EE06B007B for ; Wed, 8 Mar 2023 11:53:10 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E51EC1C6760 for ; Wed, 8 Mar 2023 16:53:09 +0000 (UTC) X-FDA: 80546326098.25.9964823 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id 323F640019 for ; Wed, 8 Mar 2023 16:53:07 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="hTMhpU9/"; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294387; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dL8s47Tp1nK8GIg8PijNkCrDLEgPha2fkCIMRUXm1zk=; b=R+GE1yrrN4aiIr/aRSKZZ77uul/BndOYqgtvNQpMza+vmXmiO+YZ3ha3E6JvSRWzeuxpnj p1MOBrL/IGeMuGV0ALTGM7Tp0rKTBn7+Zycu+v2tzwXs+2slggz52koT0R1VHFJxZo8lbc xp+hCzOUuaEZbTCPtZbXtrq3/PzTvFY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="hTMhpU9/"; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294387; a=rsa-sha256; cv=none; b=RrYZtv9GsoXfggzgcOB2QZSYPAnSzGEbabSSsbzSGi/DGowpAmPQgEIFcUQbxcRBNyb3JF dyDBZ52CR2aWKgBLDXbf7kArq+926OyNY/M2rgnvJu9hPeFl8O0RbePtYKIvQV52mds7sE 4azAcG6B8SjY9nWhnz1QgeZCGxw2vT4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dL8s47Tp1nK8GIg8PijNkCrDLEgPha2fkCIMRUXm1zk=; b=hTMhpU9/JD0HBvjCuHgXAMJK4XcAjZlC0neCx+1s1T5MwxKOIPRPPYSiXLp/1H1PiQcILL l1m9VsdA7zRHIC/0cQP6dtoQiD4WNJgFSnM6JlzHmtbrzNVpL03clSunYlN0Yjez/mWbqm WWWaMMxAI+323FuVCX9yooZ54WMChb4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-609-Q9XiGVkBM5CVEEGBMEseQA-1; Wed, 08 Mar 2023 11:53:02 -0500 X-MC-Unique: Q9XiGVkBM5CVEEGBMEseQA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0ADFE2810C15; Wed, 8 Mar 2023 16:53:01 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 08E1F492B04; Wed, 8 Mar 2023 16:52:58 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v17 01/14] splice: Clean up direct_splice_read() a bit Date: Wed, 8 Mar 2023 16:52:38 +0000 Message-Id: <20230308165251.2078898-2-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Stat-Signature: ukqr9huwygpb81a88cf96mhw7m3hmx6r X-Rspam-User: X-Rspamd-Queue-Id: 323F640019 X-Rspamd-Server: rspam06 X-HE-Tag: 1678294386-812316 X-HE-Meta: U2FsdGVkX184lfWbXcGNSTSw8St9vN2qavXRcpGulf1wKCLKScDCteEe5TA8V7RXASI8hImpcCFlA6BwIvLMLlVdsOw0jpHzCvVtWl/8+/+SgTki8EuhHhAYhmqecnD8mpZ22KbEJFxvE04/0CIhEdC04/odvHUNoOPq2izsR15BtzETPaFtVEg65NNBCXRykAUd3xyIiK2wutv7tklia+pnszvwlwtq4X8jhQWPGWWAzoPjyqLXx0h9vMUJlRBTwDy853e99hr68GM2rszHGsydCviekpApyVDBsgsG8B2ef8uCo9q1hJmXkmlUELNyf6XSdYTGBOBCc9mOeUEn7kYJ8UGAX4X2O9lN9MGjqMW291+kU9ME1pjTjy0ny35aDcKsV/rKzNPaZ0ShQQfgt9Pgl/dfbwjFJk2hcIbEqni+zqYv+yALGPdqieZF8tPHHHEi6p5PLzx+G8E+/c63WAhZ+EiRoyPcyblUCub8ctUdstNIBfIHTpWJ9ULTCj3IDvZ7qne5auvmTCrdBYI0dTxVA6MFt1ZB0zsDYSdmw0B0nhKCnitbbXo6FTNIQpEIYOpGFTmvdDjtb6x2ygnZvvbZWSnSY88IZPJiNKo6vOkLFqu2EPgh79RekYeVaZPngNofBAKCnhzQTYbdrhsKIKlU4zdK4D/elsbZHDWuOn7K1JWj/OCIvMV2kK7LA4fWHI2neiSt5couM9c+yPMg7B7muEdHvhv3xrt47CM+MICqBh5aSrZH/cFkVTALsePjMSmVsq11+AxA5BcnvbF27CpWOLOyGt1OJJ4zbTiwMg3cGExNaG4XVX6zKtWdr5oEdAteb7Fi2HyeW4DVx7SxN2It9nSnSzx+eTttLrgQpxIs9uPiIqj2DA0BL/NP+FOfx06zTmZzCSVJjRyaUNKs8p4IlS6NPGtWUkTgKqP4PsLW6sMHP7d4A01A0+pFM5BIUJVl1uOtrY3YVOvto6A iEI22zIo nsGBtGmKHi06CM+OrrPN8z+0jgbAKhim14yutmGXangJe+xcUNzHR0NKRtu5ak3AXtS13xSoxcfclBG8gd0lKbwuFoa5VFvzrktGe/ZmUPH5sagPGGERfpFam9OSf/ygdBRbs1PzXD+L55iry8lxUAOfxTbHoE8eKG4bmXknTahRsLNRE39GBtxctD9ppEqv7qEsOPnUKtAR8wmEAtpJoc74/bQmSklho8wcHS7c7441/RhoNLezrLjxp4gB8elDXSaCL3tuYeMPTsrkQjfyW+LF7TvypkcCeMnzv8mDzaswdZey/Rp/mtiZayTir+AkCFSNrUY1UtcgXJRlky/GOAv7HV8zjOjQZIMnVjjnQRRIv38n7fqaR3j+y7+qC1CeafjnMYWst18j/TE4xUPE1Sc8MTiJ1BSCskP6h2LQ86MkELM73p7+MaZx1OihHSTSh9Mw6Uj5smPv8PsKVcnYPUe+/pc3iATE/CtJM9b5/ya4CQsbw4du+OHdwVA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do a couple of cleanups to direct_splice_read(): (1) Cast to struct page **, not void *. (2) Simplify the calculation of the number of pages to keep/reclaim in direct_splice_read(). Suggested-by: Christoph Hellwig Signed-off-by: David Howells cc: Jens Axboe cc: Christoph Hellwig cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org Reviewed-by: Christoph Hellwig --- fs/splice.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 2e76dbb81a8f..abd21a455a2b 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -295,7 +295,7 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, struct kiocb kiocb; struct page **pages; ssize_t ret; - size_t used, npages, chunk, remain, reclaim; + size_t used, npages, chunk, remain, keep = 0; int i; /* Work out how much data we can actually add into the pipe */ @@ -309,7 +309,7 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, if (!bv) return -ENOMEM; - pages = (void *)(bv + npages); + pages = (struct page **)(bv + npages); npages = alloc_pages_bulk_array(GFP_USER, npages, pages); if (!npages) { kfree(bv); @@ -332,11 +332,8 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, kiocb.ki_pos = *ppos; ret = call_read_iter(in, &kiocb, &to); - reclaim = npages * PAGE_SIZE; - remain = 0; if (ret > 0) { - reclaim -= ret; - remain = ret; + keep = DIV_ROUND_UP(ret, PAGE_SIZE); *ppos = kiocb.ki_pos; file_accessed(in); } else if (ret < 0) { @@ -349,14 +346,12 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, } /* Free any pages that didn't get touched at all. */ - reclaim /= PAGE_SIZE; - if (reclaim) { - npages -= reclaim; - release_pages(pages + npages, reclaim); - } + if (keep < npages) + release_pages(pages + keep, npages - keep); /* Push the remaining pages into the pipe. */ - for (i = 0; i < npages; i++) { + remain = ret; + for (i = 0; i < keep; i++) { struct pipe_buffer *buf = pipe_head_buf(pipe); chunk = min_t(size_t, remain, PAGE_SIZE); From patchwork Wed Mar 8 16:52:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A212DC678D5 for ; Wed, 8 Mar 2023 16:53:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21FE16B007D; Wed, 8 Mar 2023 11:53:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A9CE6B007E; Wed, 8 Mar 2023 11:53:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EEFD56B0080; Wed, 8 Mar 2023 11:53:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DB2066B007D for ; Wed, 8 Mar 2023 11:53:11 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9EB51140FC1 for ; Wed, 8 Mar 2023 16:53:11 +0000 (UTC) X-FDA: 80546326182.01.C550736 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf07.hostedemail.com (Postfix) with ESMTP id F1BE240019 for ; Wed, 8 Mar 2023 16:53:08 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HqjkFi4Y; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294389; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fokk7Crx10kBwBR9E3z6e4T6rL1k7XhHeFqewFVvsVE=; b=0ujrFeOOe1DdWSX04AkbHWKgOJPcREuCs5LZlLshY3UAI16oOFwBd2+oFqFmt+Fyk+tsks G3zaVKjEYyqMQuLhGYVmnvzfH35R03EfbyAo1n/Vl2SfoTCwNR5TPOfV7QSzIo3rWz214i fza84FJU2ifV3RUmg6u633Ie/a+ChfQ= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HqjkFi4Y; spf=pass (imf07.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294389; a=rsa-sha256; cv=none; b=k5nVpvq9ZBODiim38DDpE4VMtGRtOnwgNgJLqE6bMYFrS+5jQc5A46NgWpRD8A+szfkaGW Yba3Dyf6enb+QA7lZKbA7RMpt1wpyFELgVRx9KGAL+y4SSuvCotsXEwqoa0fmzo2zDvkx+ zbGMk60Un4+9ikx+tN1PQZ0Q2hdXhBU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294388; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fokk7Crx10kBwBR9E3z6e4T6rL1k7XhHeFqewFVvsVE=; b=HqjkFi4YrYJiWI1oUe0ohpgDr9KDjRiL/DUNHD6yAplE2slqHAzGH7DXzP9YhvrQjggHV/ iNfuMaCwB3S37v6+lNQ/VZF6fpLrGN4f/24DsiLqm2yA9erL4JVlxzUxoJGcMqDP0iNqeT I0w87ejOWwwReySJXLXMpUQSaCGrLeg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-194-7CVrQpdMObaxQx7KSWspRQ-1; Wed, 08 Mar 2023 11:53:04 -0500 X-MC-Unique: 7CVrQpdMObaxQx7KSWspRQ-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EB104100F90A; Wed, 8 Mar 2023 16:53:03 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA1FE492C3E; Wed, 8 Mar 2023 16:53:01 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Miklos Szeredi , Christoph Hellwig , John Hubbard , linux-unionfs@vger.kernel.org Subject: [PATCH v17 02/14] splice: Make do_splice_to() generic and export it Date: Wed, 8 Mar 2023 16:52:39 +0000 Message-Id: <20230308165251.2078898-3-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Stat-Signature: i3enkemopup1ffc91k1u73ezosc513mp X-Rspam-User: X-Rspamd-Queue-Id: F1BE240019 X-Rspamd-Server: rspam06 X-HE-Tag: 1678294388-363898 X-HE-Meta: U2FsdGVkX19RUX8jVM73RZDyWf09yVozAaRDgTSCVIOymKQ5YCa0QTrOpoY28vLvrJClwcTiBK65uQ9mcW+LixvWYZg1yHzjDFIPpjKgUHXEMSKTpj1Sw3UBUsUKO2u29qD5s0TCoe+LnSmbthaDf9OTZHjyZ9/8/psjIAKa/oH/pTz3eo+A3TaXOeGGGkT1cS/Tw2S7BVOCoJQWtXbEeDuR3Wk46KCzAB4YL8R11Xf86jUN1eUSLKKt9hggkBMwrBZyqCyX7xIZAAB+cEO86kAmYaMQHQgbBS6JoalPw5sFG2aSAmW7psEZacc1l6HpP3z/YpebVtJzOsCR/0+pC/hUwOspkc40/80WvED/rcm89SSy7M4J0pKe6EKBREbeEykEfB75r/XifFmtliSHdBPI62aqsr53jCtCuZyCTDpu3lHBJNY0ZRCRyT5W1cbzSNBHzK4V4pjb4sNCkaDMxXE5FMgpA4YKX69p++NQXsaAaDvCf4guWQvAZVxy7g9w4J1wVK3Zaf9dc27onM3OSyk778mgjhHP6Pl/KvYq8VjsyknDpi/R3GmEwphfHXowo0DeOC/tvGmTqWOBS1FBVrvuSB2v8jIaCrMDNSbAoTSckBGaHiHtp0tePMZ2WN/0pys6xUm5WMaksmk+NBh4niNGWnSIZl3J3BMjDy4ttSEgUSqMB58r4RqO4NQZbxfxypvbUFqpsBzqRQ5BXr4zvtvACps6+Q69RaG+O1RIjb3bVp+cbySM/zODiyvS0StmYd0X0/m+lgg2dVMeCZw7X1sok+q3lE78hov/DThZLj6RD1vf1KAmt4IZTNMc2UUxXJj24foF2NFTaqk7gSGqjz845Hd0zHTBVOddZNj9uWZje60DZ+PaDih+8W/9iz+Gio/AFkm3pyBJZWE88mRM2lLlwPn0NHGgAH9RhHHm/1qhJGb/Bqt19UYl1f4H4vRxus6kfOm7MkKIdMupR0o AiYxUAnB SMcatSUSi4T9fxbm6ByMZucC0MhUmoy8MH0INelZEO2Tkmd/cQjjlM8FUWLxq3IMPnXlott5vvbiZYBCm3UI5H/0+omTWwDjU2M1/FLVvKeFGYzRqlnb6tlSWR2H73IRKoVfXz/w+TVnocvC57h81iVQoC4VEE9N6zOCk19d3z7z7xxKPoyxcljjIPeFGSqPi6V/E56ayDfmB5Pz19ip2MkRrGFjgfMSgJNLvmeWRFgAQr23ZAvH7wLhCiVNynSmSoOdF4l81B3H0JK9P7fUhROMba7A4oy15CevTlkUMt3TdqwdpyRPyVnQA474j4QGsr0iN8YWylnWQ/+auzOuIir3tvj8xo8aU+1rwxzi/ymM2DmJDBvgoRszPY+EcxQbuLqi9kXhg9ghkS6nUEuGii+akb/GVsd5UjmBOf0DboFNJrvn+flQnkRlJS+snqrjU+Wc7HN+3x1w4BfIktLSz8BZSL9Lni5sOsBc2SANt7JJi5nXDKJUjojKyUeX8FrON8VXoTFGFezx8CpIyzbagtnSpfA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rename do_splice_to() to vfs_splice_read() and export it so that it can be used as a helper when calling down to a lower layer filesystem as it performs all the necessary checks[1]. Signed-off-by: David Howells cc: Miklos Szeredi cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/CAJfpeguGksS3sCigmRi9hJdUec8qtM9f+_9jC1rJhsXT+dV01w@mail.gmail.com/ [1] Reviewed-by: Christoph Hellwig --- fs/splice.c | 27 ++++++++++++++++++++------- include/linux/splice.h | 3 +++ 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index abd21a455a2b..90ccd3666dca 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -851,12 +851,24 @@ static long do_splice_from(struct pipe_inode_info *pipe, struct file *out, return out->f_op->splice_write(pipe, out, ppos, len, flags); } -/* - * Attempt to initiate a splice from a file to a pipe. +/** + * vfs_splice_read - Read data from a file and splice it into a pipe + * @in: File to splice from + * @ppos: Input file offset + * @pipe: Pipe to splice to + * @len: Number of bytes to splice + * @flags: Splice modifier flags (SPLICE_F_*) + * + * Splice the requested amount of data from the input file to the pipe. This + * is synchronous as the caller must hold the pipe lock across the entire + * operation. + * + * If successful, it returns the amount of data spliced, 0 if it hit the EOF or + * a hole and a negative error code otherwise. */ -static long do_splice_to(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags) +long vfs_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags) { unsigned int p_space; int ret; @@ -879,6 +891,7 @@ static long do_splice_to(struct file *in, loff_t *ppos, return warn_unsupported(in, "read"); return in->f_op->splice_read(in, ppos, pipe, len, flags); } +EXPORT_SYMBOL_GPL(vfs_splice_read); /** * splice_direct_to_actor - splices data directly between two non-pipes @@ -949,7 +962,7 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, size_t read_len; loff_t pos = sd->pos, prev_pos = pos; - ret = do_splice_to(in, &pos, pipe, len, flags); + ret = vfs_splice_read(in, &pos, pipe, len, flags); if (unlikely(ret <= 0)) goto out_release; @@ -1097,7 +1110,7 @@ long splice_file_to_pipe(struct file *in, pipe_lock(opipe); ret = wait_for_space(opipe, flags); if (!ret) - ret = do_splice_to(in, offset, opipe, len, flags); + ret = vfs_splice_read(in, offset, opipe, len, flags); pipe_unlock(opipe); if (ret > 0) wakeup_pipe_readers(opipe); diff --git a/include/linux/splice.h b/include/linux/splice.h index a55179fd60fc..8f052c3dae95 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -76,6 +76,9 @@ extern ssize_t splice_to_pipe(struct pipe_inode_info *, struct splice_pipe_desc *); extern ssize_t add_to_pipe(struct pipe_inode_info *, struct pipe_buffer *); +long vfs_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags); extern ssize_t splice_direct_to_actor(struct file *, struct splice_desc *, splice_direct_actor *); extern long do_splice(struct file *in, loff_t *off_in, From patchwork Wed Mar 8 16:52:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166160 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB0D4C678D5 for ; Wed, 8 Mar 2023 16:53:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52E2E6B007E; Wed, 8 Mar 2023 11:53:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B5EC6B0080; Wed, 8 Mar 2023 11:53:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3565D280001; Wed, 8 Mar 2023 11:53:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 21A426B007E for ; Wed, 8 Mar 2023 11:53:15 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 996FD160A25 for ; Wed, 8 Mar 2023 16:53:14 +0000 (UTC) X-FDA: 80546326308.01.D16962A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id DED5AC0003 for ; Wed, 8 Mar 2023 16:53:11 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=REQe703U; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294392; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=n2MkOxcvB8unEejbMWkMEf534o0h2+e07KJxot0+dsE=; b=nCIBWbt1QxJxd5VpztplL2eb+PBVpAIUsB0MYJumguWcR9Nkijg6CP8K1BAnbtQWIUEE2S vpQ/Byn+W1T8ZkhLSboOvwmcRmlFNS1WX6s/CgylH8frM0VliYcfHOQZwcJeJMC383scgy oo2bOy3bUtifLZMxN56p5TWcbwWIm+A= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=REQe703U; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294392; a=rsa-sha256; cv=none; b=FdkrmEz+Lz7naNq/lDPsIRuTzuR6rA9ZV8HW9xMijngMSfbmoTOTdFb2nVT3L0q31AQ01k tgEQC1LKb9ikDf+bsqedqMlqFMQK+edZYIcLFKuCZBZjNt12GbxyCoxvEbFFTNwVR8bdVQ Y6bwBkSyweegYmuTImOa1CEbQIctKxc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294391; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n2MkOxcvB8unEejbMWkMEf534o0h2+e07KJxot0+dsE=; b=REQe703Uk6gd81YHTqgZekG1+bSzV+Z8Mp+d0cNgwfbaKiIMfpt+RCa2esZeUnKT53R/wY 7/+0VwbN2fwzoWTtVzyCtY0M1M9DEVZcM8Vt5aJBsG85WEE5TyNVTSJMEWZDu95sXPbvOJ mw97JaHaCRyNICL1UWH5DfzULuCovPg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-253-uC0SAgkeM7aQ7FeHN1dj7w-1; Wed, 08 Mar 2023 11:53:08 -0500 X-MC-Unique: uC0SAgkeM7aQ7FeHN1dj7w-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EBC6F886C61; Wed, 8 Mar 2023 16:53:06 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E7602166B26; Wed, 8 Mar 2023 16:53:04 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Daniel Golle , Guenter Roeck , Christoph Hellwig , John Hubbard , Hugh Dickins Subject: [PATCH v17 03/14] shmem: Implement splice-read Date: Wed, 8 Mar 2023 16:52:40 +0000 Message-Id: <20230308165251.2078898-4-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: DED5AC0003 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: wcr4jdpcckgbg9b5ugoynbu59onjyn4r X-HE-Tag: 1678294391-359604 X-HE-Meta: U2FsdGVkX1/8UH9sbNngUbiUb6laMoYLqawmVD0uQAsVKK59CGfH5bOZyNbTLCl+z+Xt1SBO5T8OmZLu64TZYj/syl7Qy1AVjaOgHGmoRoaROUhzp+3Rh6xWxXMY6q4pV4xaQgZyyYJimSnuveI62OrPFAEdwSJdtiZZ45OOMYCxvSEkqL2y9O6NWSJECPz+LBsBoGfattVa4So/DW/MxSXlNymNkmB5HY96803cSZsHEn22DW2PN1laKnvnd1rWZKvyfPkP14yO3BB/FBYcK9EieoHnheVhbL1e6BUTqae+BTV6c3L2I8ldzHW/ARbALEREE0V37uP9fXgQ/Un0FuBZ5F9V9RqVSPBKTjEbE6dc+54nABPPIaT1pwlRyH/Md8KdYKIMhMOI+ZUBDLwSj3AYKCYkMRlpmhic6lRc5ZGt6GozOWSpF7rAgDhaepz8cjFST+2j0vIBJQVtrfdpWD1L3Ybh3X/p+OuUmP/Ul0ZZjg0WHzbKsilIfD7HraQ4BMMQBzX8btgo5R8Snn47mSNE2ULKvBf7bXta4UKgZN3MWpReitWkdyjWpGMAjqgjRxGiZUNQWB5Y+h9bwOdte3sPEx0n2nmaAc2Nv3FOtGeAcO4ljSgCLf1JtN29YROwE1HKKT0JY/gsCgd5VgZCXXjymeVc/nehcM+bpU5magjwk4g2dJrhPVqVEpY9UGkTNL883QxEFG5yybgPCJfqafiuDzZgQvG7zA58L1rNTleSbhbyqu9R1xsCe+7ReYUtPuy10ww0HksaDI4HzOgK3jx30GiqUbYtV5k7q2iDe62OBmMoLBpP+V81PS1skrAYjvioXSgFHwedMcZxMbo3pzZuNfbpQGDpI1QSUSpTTD8AnhPigdFG58bnlq+2zZhgTByGCU83qylFvWDW9P4Varp7cI64UuySA6/9uJ9icw92wIPaILtC7AKKZRHiM6yCSjF9AaPWoTb4aJTtpKu 6zu2rD0l pMCWRQKZdS+RB/LRbiL4zbhqOJ9VD+FDQ6a16AjsvDQnygESJN3UuJVdy0/xhzFWYMKwjvcuMi4KWytqaFHZn5qzQf1KdQOSW41QaAtNsSyhO7NU6otN251PqRGuXPw7sB7W899irYMkyJDlYY/Ky5EZsgn0UZ4pEK6WHC1ndVf4BvYwWOEFYSvtH/y1rnHOt9sx4B4IJvkxXxpVQ7Y0KIS6Yt6BYnJUt+ZiTSvtuN9urPfAUeHAFB/TgzmM3MAOLVJ1xrDONsvZO8Gz+Mxra0KN68Dkfauf4tFUrRcHgwjWVwM2iPVqDqW0Kg0e38TRtATWVvbyvpbbDeY8vh33MHICUZep95MF0s0XeePE0Se+p5Uzuud1GDtZ3L6bsA9VXBWs/M7W1XSCFV6WzAwHfNzWK/N90gSAwzElyFYAF1gB+aEJL3yS5Li6pui55ZjEXo5CXkxvTROwIrKfg8iaeSv9GQvt3y6VHnKKb7/uqOQitdxRLSGQ10UvaF53PjII+mYZaWsdfEmhSDYQxSnu0Yupl9hZ6xubYRLGJxxMwfJ2UkS8o0t3KwT8MJA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The new filemap_splice_read() has an implicit expectation via filemap_get_pages() that ->read_folio() exists if ->readahead() doesn't fully populate the pagecache of the file it is reading from[1], potentially leading to a jump to NULL if this doesn't exist. shmem, however, (and by extension, tmpfs, ramfs and rootfs), doesn't have ->read_folio(), Work around this by equipping shmem with its own splice-read implementation, based on filemap_splice_read(), but able to paste in zero_page when there's a page missing. Signed-off-by: David Howells cc: Daniel Golle cc: Guenter Roeck cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Hugh Dickins cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/Y+pdHFFTk1TTEBsO@makrotopia.org/ [1] --- mm/shmem.c | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 123 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 448f393d8ab2..3cbec1d56112 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2719,6 +2719,128 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return retval ? retval : error; } +static bool zero_pipe_buf_try_steal(struct pipe_inode_info *pipe, + struct pipe_buffer *buf) +{ + return false; +} + +static const struct pipe_buf_operations zero_pipe_buf_ops = { + .release = generic_pipe_buf_release, + .try_steal = zero_pipe_buf_try_steal, + .get = generic_pipe_buf_get, +}; + +static size_t splice_zeropage_into_pipe(struct pipe_inode_info *pipe, + loff_t fpos, size_t size) +{ + size_t offset = fpos & ~PAGE_MASK; + + size = min_t(size_t, size, PAGE_SIZE - offset); + + if (!pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { + struct pipe_buffer *buf = pipe_head_buf(pipe); + + *buf = (struct pipe_buffer) { + .ops = &zero_pipe_buf_ops, + .page = ZERO_PAGE(0), + .offset = offset, + .len = size, + }; + get_page(buf->page); + pipe->head++; + } + + return size; +} + +static ssize_t shmem_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode = file_inode(in); + struct address_space *mapping = inode->i_mapping; + struct folio *folio = NULL; + size_t total_spliced = 0, used, npages, n, part; + loff_t isize; + int error = 0; + + /* Work out how much data we can actually add into the pipe */ + used = pipe_occupancy(pipe->head, pipe->tail); + npages = max_t(ssize_t, pipe->max_usage - used, 0); + len = min_t(size_t, len, npages * PAGE_SIZE); + + do { + if (*ppos >= i_size_read(inode)) + break; + + error = shmem_get_folio(inode, *ppos / PAGE_SIZE, &folio, SGP_READ); + if (error) { + if (error == -EINVAL) + error = 0; + break; + } + if (folio) { + folio_unlock(folio); + + if (folio_test_hwpoison(folio)) { + error = -EIO; + break; + } + } + + /* + * i_size must be checked after we know the pages are Uptodate. + * + * Checking i_size after the check allows us to calculate + * the correct value for "nr", which means the zero-filled + * part of the page is not copied back to userspace (unless + * another truncate extends the file - this is desired though). + */ + isize = i_size_read(inode); + if (unlikely(*ppos >= isize)) + break; + part = min_t(loff_t, isize - *ppos, len); + + if (folio) { + /* + * If users can be writing to this page using arbitrary + * virtual addresses, take care about potential aliasing + * before reading the page on the kernel side. + */ + if (mapping_writably_mapped(mapping)) + flush_dcache_folio(folio); + folio_mark_accessed(folio); + /* + * Ok, we have the page, and it's up-to-date, so we can + * now splice it into the pipe. + */ + n = splice_folio_into_pipe(pipe, folio, *ppos, part); + folio_put(folio); + folio = NULL; + } else { + n = splice_zeropage_into_pipe(pipe, *ppos, len); + } + + if (!n) + break; + len -= n; + total_spliced += n; + *ppos += n; + in->f_ra.prev_pos = *ppos; + if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) + break; + + cond_resched(); + } while (len); + + if (folio) + folio_put(folio); + + file_accessed(in); + return total_spliced ? total_spliced : error; +} + static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence) { struct address_space *mapping = file->f_mapping; @@ -3938,7 +4060,7 @@ static const struct file_operations shmem_file_operations = { .read_iter = shmem_file_read_iter, .write_iter = generic_file_write_iter, .fsync = noop_fsync, - .splice_read = generic_file_splice_read, + .splice_read = shmem_file_splice_read, .splice_write = iter_file_splice_write, .fallocate = shmem_fallocate, #endif From patchwork Wed Mar 8 16:52:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08480C64EC4 for ; Wed, 8 Mar 2023 16:53:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C0846B0080; Wed, 8 Mar 2023 11:53:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 922E6280001; Wed, 8 Mar 2023 11:53:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 700656B0082; Wed, 8 Mar 2023 11:53:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 583146B0080 for ; Wed, 8 Mar 2023 11:53:16 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DD3E21C649E for ; Wed, 8 Mar 2023 16:53:15 +0000 (UTC) X-FDA: 80546326350.23.B1A7F43 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 3B82740003 for ; Wed, 8 Mar 2023 16:53:14 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=h9pSBFGD; spf=pass (imf27.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294394; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=74aMisHJvcrsbS5g8R7662ZNEA+1A6hMiypJVvVd5yo=; b=GVIyyxZiFkBmXkQ8RkdZxXQ+zJDKE/fS6dJChDCrCOtBxlIilwKG827/JuWXv9V3cePoTQ fJDS27G+d/iV+XP+9PBhEPs7Mh9nCbIVyuGY4uy47lDqoAeOd6q5Zr+8LJhwl03biXwhnl KJYUs/Yqmgfj8LISv63zWYWSBzpxqOY= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=h9pSBFGD; spf=pass (imf27.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294394; a=rsa-sha256; cv=none; b=5eeA2GqxzTHoGmtO4shE2o+7dQ0RAja6mplAJiI3QKGlToBjpu/I9JXyjER9FbO53IvX28 u1wApvReG1A2B21nUHr8nUATnVJ3XcJOoJIlSI7WiSG3jTq1w0Qf9l0NloEHInDuiyAvV1 To2gU8VzEhCvjqkLrMZ2GGTiC2zBSaE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294393; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74aMisHJvcrsbS5g8R7662ZNEA+1A6hMiypJVvVd5yo=; b=h9pSBFGDozA5WX9rkJrPn5RCfal8Yoo46W1SmPmJ0dEjql3NuuWJo1mw2OiinfUOyCgavo 8r0W19zFEPz8Gg60adYv0Uoe92jAk0k9nFsP8UvYT0guSXV6HvORAchrL3Sfx4yZAedMGJ vfnwQlNrNlR3ForZ9B54B8YRpJM2fBc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-629-CMD8rN7nNP6tp5tN6HG62A-1; Wed, 08 Mar 2023 11:53:11 -0500 X-MC-Unique: CMD8rN7nNP6tp5tN6HG62A-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DFF60100F909; Wed, 8 Mar 2023 16:53:09 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE39D492C3E; Wed, 8 Mar 2023 16:53:07 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard , Miklos Szeredi , linux-unionfs@vger.kernel.org Subject: [PATCH v17 04/14] overlayfs: Implement splice-read Date: Wed, 8 Mar 2023 16:52:41 +0000 Message-Id: <20230308165251.2078898-5-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Stat-Signature: borck6sqryitirijkdedtos6np3icfzq X-Rspam-User: X-Rspamd-Queue-Id: 3B82740003 X-Rspamd-Server: rspam06 X-HE-Tag: 1678294394-379457 X-HE-Meta: U2FsdGVkX1+0jfGnKo2sbLTDYgOK8ghXJO7FQdbpmil1ljLQA72kA3QnmpTU7zW+9/a9g9kG5YE25rcotpWTlJqHlsP/sfx70JyeOizM7RagRQJYCiRHZQqQnmiV181bfsWChQliE5D0gVjJg8OGkO/60ejqVB+iaeeYQcZcKfncAifxI91MtwOQj2wdVA3n8eipqd4vtFXvzCBJoz5vg03OWh1XwCWDCKh3lFqXC/sle4xMBmHFI3eWY2vOP8yWTaqJAtxxX3qW+oi9+xdcb9fn6rlUaqM5WqmNL4nLo/sSuRaeyGnbgWYU8uyeOqDJm03KJXilho625IuXyPo9UTeKAdnG7EEWGvVlpyFEi2hy1mvcUGumN2fGBmTtsz4p059Zf3bnWWKvoDaxfWjQTl5Npmw1IfvS02HeRsVzA1kxVPb1mf9ZyThv7CZe//0qEyCEeHKZUhTEiMxugpU4cm1m6BennLmBCi02Bv9L5qcicyF2a2xGo07RA9GThxcwyy47vbICfGYn7F96Rv9jwJduYyBiFDPX12xpg+IR7juazo52zfTtOynhvdomCB4QhDHboMbj8P/40wPMtWh5d27izbpPqE33eMHRthQZocMQS5Xd2GDAMmgeajVC1+F5pXkbkMnpo9OACfA2B5cYP+K4JPQ2FUJZzrUmVWtZauo60s+yANW/h8/uyEEBTHRzpVeVrFXhA/fFr+7AqEeqmJZXi8GkkTNukFWInqcPhbPNy1/5prV9iJGAG9VcAJChp8E6qXmBUkx70wQ/W6HQUZadI4ey891uP2d9vLN8vSdwF2f5sx3P61uPRmxODd9tQ5sY1QuEf8Nl78oI/NEYaZMyB22G0vxZZW8mUObji0NkE+k6ZmtErhYrg/0exTzeHcrbOTexA57DvLXwxkctO2UHR1YkjsnjP8U1cl1h2pQ0GWva3JtmGVthb8tCNL9Fn/J8+q2navH8SQkCMqn RUimt0UR 9ujOJwmYU8xaELLCicAAgfvLRMOZ7Vfbp0WJqL/n2K20OHiUOZHeJIFj/ecfN3Jui1C9d6ZxRz64DALT/KP0laewpCWjFQ/8eDRJMX/y1gNKQcDNw+M/PA8eTdy2FpNyVwxDNbKE4r94KYq8UVJM5EI117WI8t5UJE8PymQO/akdQJ9hKgeS2Ruh4QvWS4Or4pwiSfK4UTwtZlLYfugwEdweiah9M0OoTz9DKV7Z6JPuHPV8Cm9833qMqWxHS8Kp5Tz7C56La0FLwob4JwTqVK39fkiXb8+t/2+M1Ac9pJCc8OzCMfX8fwQbmoHZGldFpgQHaIWypUFi80Vqr3ZgGquybUsYwlZA7CexYQ0mVDY5i0slVJMqMSrrM/Cri7hffIBSw9kYffoWucQtKfxm8keJWK7+25uocpjUDbtxSsbwX/tt/baKwuNvqIDZcLYnGhstE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Implement splice-read for overlayfs by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by: David Howells cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Miklos Szeredi cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- Notes: ver #17) - Use vfs_splice_read() helper rather than open-coding checks. ver #15) - Remove redundant FMODE_CAN_ODIRECT check on real file. - Do rw_verify_area() on the real file, not the overlay file. - Fix a file leak. fs/overlayfs/file.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 7c04f033aadd..86197882ff35 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -419,6 +419,27 @@ static ssize_t ovl_write_iter(struct kiocb *iocb, struct iov_iter *iter) return ret; } +static ssize_t ovl_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags) +{ + const struct cred *old_cred; + struct fd real; + ssize_t ret; + + ret = ovl_real_fdget(in, &real); + if (ret) + return ret; + + old_cred = ovl_override_creds(file_inode(in)->i_sb); + ret = vfs_splice_read(real.file, ppos, pipe, len, flags); + revert_creds(old_cred); + ovl_file_accessed(in); + + fdput(real); + return ret; +} + /* * Calling iter_file_splice_write() directly from overlay's f_op may deadlock * due to lock order inversion between pipe->mutex in iter_file_splice_write() @@ -695,7 +716,7 @@ const struct file_operations ovl_file_operations = { .fallocate = ovl_fallocate, .fadvise = ovl_fadvise, .flush = ovl_flush, - .splice_read = generic_file_splice_read, + .splice_read = ovl_splice_read, .splice_write = ovl_splice_write, .copy_file_range = ovl_copy_file_range, From patchwork Wed Mar 8 16:52:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 528D3C678D5 for ; Wed, 8 Mar 2023 16:53:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 861A7280001; Wed, 8 Mar 2023 11:53:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7EB176B0083; Wed, 8 Mar 2023 11:53:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6640A280001; Wed, 8 Mar 2023 11:53:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4E7E16B0082 for ; Wed, 8 Mar 2023 11:53:24 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D196BC0750 for ; Wed, 8 Mar 2023 16:53:23 +0000 (UTC) X-FDA: 80546326686.03.52098B8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 05A62180019 for ; Wed, 8 Mar 2023 16:53:20 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hM0nRbrF; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PhbxTzWahXdd/JlmGrRowyeASqSlq5EbDSo0Q9TZAGc=; b=nzii5cEFWmxT6b8hUfKIsbAAtLpUmGG0Mc9xyohpJLvarZTEVpoTrv1ZFknQ8m0GY0W2/9 VtB0Zd2BIcRxoxPGSezDQr5gTiz6ViSHbeHel7uQMbfO4bqZ0Clg7gOEWkQLGcSUdDgJBg IAsyLQKvsin++C+YfF1Y/sFuiXX68pg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hM0nRbrF; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294401; a=rsa-sha256; cv=none; b=givSOfVXcbq1tQ6Vbn+cJ8BYPLpogSjgI1P6zfEodYSnK4+OCPefbi4iQpqM704vFwGo3Z yE1K29Jqspx0zWhPsB3mCpHBykKMDzoN2aWoQASHit7mMjAlnk0CNs/MbLfOXe4lYlx0BK k1yjuz8jarzSw/igmurqj3mbvJCCjSM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PhbxTzWahXdd/JlmGrRowyeASqSlq5EbDSo0Q9TZAGc=; b=hM0nRbrF1MoXBP7RU9/atMQ+2yAGE/JncDHrVqEMsvniy4HISLKJqLedPdFkehZJDswxF6 z87kAhTPw4nEe5oY7VUupjLNUG+lOZ8rXiOwfsnSFAQLQcfjDq0IqMuy4tRJnXlMxIJirg x7n03L5S2ldAK0xcz54MSEVzuwymcYI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-18-Nut58j8COw2eSbYYTRM5UQ-1; Wed, 08 Mar 2023 11:53:14 -0500 X-MC-Unique: Nut58j8COw2eSbYYTRM5UQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1025F1C12983; Wed, 8 Mar 2023 16:53:13 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E0E9492B04; Wed, 8 Mar 2023 16:53:10 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jan Harkes , Christoph Hellwig , John Hubbard , coda@cs.cmu.edu, codalist@coda.cs.cmu.edu, linux-unionfs@vger.kernel.org Subject: [PATCH v17 05/14] coda: Implement splice-read Date: Wed, 8 Mar 2023 16:52:42 +0000 Message-Id: <20230308165251.2078898-6-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Stat-Signature: of8ceb17nj71go3htx7umbi5zcg3cjup X-Rspam-User: X-Rspamd-Queue-Id: 05A62180019 X-Rspamd-Server: rspam06 X-HE-Tag: 1678294400-453143 X-HE-Meta: U2FsdGVkX19xvTHHdsQNzZ32LrgPtrpppoL5BLVvKLq9NSdJoOQRehnVf1DR0+2tMvjJ6HJ4lBqLPgpoJ1Y39VJS3nwg0z/tG6f0+vRvcZD+I9LEYSKMRj4ziRWZ16LXlTyYDDeAYy4dE8j5KQ3t0cwAmEr+Y4Tlf7RrYoFhrmvpeU21fyQesvld1lLHK78/bwP1Gd4enfNxO2Z2wpR6V/E2kVhjzJm3T9z4KNPXDKpE8ChnHBEFVyJx/x6TiDqQZSvPepq917MtsNwQDKB23NMtIqWdtrCLi1Etn1mbMnuysc9iXnJ8bEPPslwGwB0ZL6LrcaU/16YYA1VlFfa3FEOS0WZFbszayFn94FAtGlTQSy51aAFTpdxGJy91iW5mArTQCzJ9yQkK17zsHiIyHFR2cJQ+lKNwfn1w8zmYXoe389+Zg5PWsZuMth18uKciZqwvrrZbHvlFG17Jph5Ni4XWLfGLbKRG20GpAhlmM6GyMIIRmEkTZ6WwEdNNOiL3WS5Jr4U808fufZjBxPNaDZcTgZe75PCyP/QRpGJjxbCAweyPNqRLdyoBsFinrFE54tSMvXwFxEO1XXJ2vk+R19fqUJPaiMjx3lWicLduIxDBOhaaefzCmUNlOUwvb38Z4rdkXy7PXgO/XSQSe0o7pDMAvK0t0RTLEk15V09wyg1MGlFX64ERRJSDlBxKUmuSqauh/ouruhxRTKJXGqM9hM8h+rEkZPlAdDv05efJSup6z3uJCR0vNAqYCd0wepSLtCy9m4hU8bRmkavEp+PtAGtzQlQAyjfS1FxQqHFD8SvoDZmYtF/H4aDOTAJ4GlJMK7t0xd/+Q8PgaNcRSccFNFq3JDyfhgtQXtSHovYDVBDA+YM1iqo5XqFUt2c6/uP05o1RfKnxsldJ4mMFvEMic2wBExTOZ4W/pcf46SUBapHi51yTE0HGjeGBgfoGNprmr3ZUyPAuNzJt/LlTU+N E+PbpRg3 wZKjmNzy8ocH08h0UkpilYSQMmPEXpttDCsKYLm/wOjgv1DvLIYlxgVvxZVE36MFQNopuAVkLyUPRq7Yp6csF2vPy1zWFC8UgCKjmc3QQ6Frc8fA592vtcR+r01xWz71Q34vD0LROZOHWjmOQOdLcfzhCFIQXJrgiRWc2cTme1KgRZ/Z7mJUnEJPUBwdgDTjLkpCw9R2YIhsPbMsLNPaxGrEniAWmMX1bTscPmloQA1sRMJGDcrFKTUtfQ68Le8SkUnQBOjrYwbYgUTaWUCGaJlLEBa6ik2QVSlBrH7lbaNTY6APczzAFM43hnSzX7j+c26eVcITzfTBfGdUkpCTrnaYwzXsum9euaJzJPMB+H/mZm3znUq+mwN7+lRH6y8DLHH7QYkClqUpU7elEjHe1KO1bvzdjTOUhR4jtR+y7MvIs0/yWNQa8PuTXUkjFXfNr7znJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Implement splice-read for coda by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by: David Howells cc: Jan Harkes cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: coda@cs.cmu.edu cc: codalist@coda.cs.cmu.edu cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Acked-by: Jan Harkes --- Notes: ver #17) - Use vfs_splice_read() helper rather than open-coding checks. fs/coda/file.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/fs/coda/file.c b/fs/coda/file.c index 3f3c81e6b1ab..12b26bd13564 100644 --- a/fs/coda/file.c +++ b/fs/coda/file.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include "coda_psdev.h" @@ -94,6 +95,32 @@ coda_file_write_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } +static ssize_t +coda_file_splice_read(struct file *coda_file, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *coda_inode = file_inode(coda_file); + struct coda_file_info *cfi = coda_ftoc(coda_file); + struct file *in = cfi->cfi_container; + loff_t ki_pos = *ppos; + ssize_t ret; + + ret = venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ); + if (ret) + goto finish_read; + + ret = vfs_splice_read(in, ppos, pipe, len, flags); + +finish_read: + venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ_FINISH); + return ret; +} + static void coda_vm_open(struct vm_area_struct *vma) { @@ -302,5 +329,5 @@ const struct file_operations coda_file_operations = { .open = coda_open, .release = coda_release, .fsync = coda_fsync, - .splice_read = generic_file_splice_read, + .splice_read = coda_file_splice_read, }; From patchwork Wed Mar 8 16:52:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166162 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83657C64EC4 for ; Wed, 8 Mar 2023 16:53:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 295276B0081; Wed, 8 Mar 2023 11:53:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 21D416B0082; Wed, 8 Mar 2023 11:53:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 099F16B0083; Wed, 8 Mar 2023 11:53:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EE8696B0081 for ; Wed, 8 Mar 2023 11:53:23 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id ACA81801C0 for ; Wed, 8 Mar 2023 16:53:23 +0000 (UTC) X-FDA: 80546326686.29.4B3F0E6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf01.hostedemail.com (Postfix) with ESMTP id D948A40010 for ; Wed, 8 Mar 2023 16:53:21 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EJweUAC+; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WFhWed43Sc7081L0WA9mMaa7Q22YwjW6QNRxMcW8x9U=; b=QZV7IEO1DmzulRyXYPmLkFzK2MziMuSTDNshV4KVMRZCkH+4/KEejw6c2m43tD4/c4LeTh zZ8P4FJIAg99a47UdzUsv0hazDjdFYDTAXGXMp5XFn+l4MyuYeGwD0j5Bc30m12E+rVYtl Ijtx+qO7CM0LUrERCF/ngzZqajiSvCI= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EJweUAC+; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294401; a=rsa-sha256; cv=none; b=ug7bB2GavGQTcmIYBRqvIJh1AtTII1P8xyqO/G9jGf6NecPhagPg+P4t55uSVOJc74F4lt k4PkFa8n1m79yDVZlaywIVP1VUMKW3oLxH9GCt9lbH3yA2/dGki2LIAjBKAxj+i1r+lS/B KFxA4P7xV3sewsg7BFzouKq7/Y0LpDg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WFhWed43Sc7081L0WA9mMaa7Q22YwjW6QNRxMcW8x9U=; b=EJweUAC+rWeUpi5IhZcmhfA7CJQex9N75cqfY5FYkFISMR72UVRfOgN6rK95H2auNPViqp Lhz8FZZJ9F/upEWghulTxx3A14uK6IxjZ4j9vHFp9jIbFA/qTMAHA+fjWZKeYhKGMI3e7K +tdxBWV3uq5fARhnR3md957xX3Vtqs0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-286-n7_IuBJlMQ-LcFthELANRQ-1; Wed, 08 Mar 2023 11:53:17 -0500 X-MC-Unique: n7_IuBJlMQ-LcFthELANRQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3D8BE8027FD; Wed, 8 Mar 2023 16:53:16 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id C81E414171B6; Wed, 8 Mar 2023 16:53:13 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Greg Kroah-Hartman , Christoph Hellwig , John Hubbard , Miklos Szeredi , Arnd Bergmann Subject: [PATCH v17 06/14] tty, proc, kernfs, random: Use direct_splice_read() Date: Wed, 8 Mar 2023 16:52:43 +0000 Message-Id: <20230308165251.2078898-7-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D948A40010 X-Stat-Signature: nkzbkngrex5furrgqxp34qgzadngbini X-HE-Tag: 1678294401-431535 X-HE-Meta: U2FsdGVkX180byDxWsSwXQazDwKLIGRsudUhs5sDMCzqDxVD6sAVGLmAVL72QQucURBots7SjpLeUTgSSkAN/VK6vi8Lr4EUg6XyDPFhJVg5s+F8kvosnrIQkdOR94SgwODjHEvrYtzGWmzdZ6XN0nglqXA5R+rr98M31M/+oA+LTr/JM5JTiEC0vLM5jhs3f/8ptwj5L35gGNj0RhfHantcSLRQu9LELCVvKoUTxYGzfJ7Pz2kMFxY4fXDhcloTvgRpIjZ7Xqwoe+uWpAL4q+L6/A2SkdixnNWbjeTsxKkIozAX8dN7QU+nuS84+AL29+q2T129VCAjh+WKCkeKQmxmlcwxG2JPW/wSCEQ8XNEH7Rai9v/rbRCI4ui/JowUr5RrezFWfX+6Bo4fxRqcJBKLwU2tvEJYGORwyVa9un/lI7IwYVDI7ilvk7alTq/4GrBNaXPhiwO8BWekZ8uASS2c5uDQCEz9vABA0YZM2scUjztu0+ESH3oMxwI4bqfLr3fo2N8kFV6TmUh+IxI4/s1XQkNAav5geIf/rDHyBquUcZSKvjq5AF+nfgq3JPgQhHucAvoleCPT6GdgofSbygOMzJM8HhEQy0XcUuuj4vAWr/4+kNyzP1htRAxP4slQNXngXB8VHk2cYYlFctA+YJNePEeTfcrWSXsKEO9MxVj0LDofdT5lNlEMXaLs/f+RCq4A25uBoKhnQUVj2LR6iBGcxITGD9GoUMLvfEZV5Pln0VikbgEkTnFHk1ofPEwH8o2v3ItcBAfzheJhucOwpVcfeZDY6QOKKrjiVT8CwTASUubDnyZGMb3rdEms2BtCgb2qQPKDoMlRiDQ+ycCiydPrZYamQqh1y8p6XSRea8zxRrvinsOZcwgkPJLz6kmtWn4HC3nyXk99Uz2CYwFeP8xLAVZWwxvGlOxzrbRLO1l68A0zAoEhHhhCgKHfAY18EbouycFgt5bG5pXadWE vQeUzZUQ A01Jqe6Z0SkbmdrNaDAV+ZXmK7DetwSsTv3fVE0v24bf+tFT0pdCdQw6yb/5hzHuAW3dhMT9GrFpNKTCQsk43RVDv5NpZhPgg/uoSh/3d/TQ/1QsDt2V3OuFFjX8h6rD/yZGRe7uPfxS8E/xGJ4GRy4/7dtCsh/5ivIN2xJ54iqViwblN2joh/N7fRxe7IuYpDlbo7fQq8Aezp3dkgqRC8CUI7oklpMSGs5fjzUlSQT4jLxmo55A/p0ejSnBO165uyIAX0zsyjtapwKmd3eP6zMrYTnNb39k/M36uYTsCq6twgkfz3LhICj4+pyjm1t5uIFI+edGb8Zavc6FljiJXNStT0Prdb5R0Op4FTmLS18EMjfUWdkN/DiO+E3HWr/Y6A3xMwL1mBcHt+ZuELskx7ra4a9/aRlMfKl/k1iEqQuvDKVDbkAqPmRAXKVSbN1PKwHnp1TEth9ljEQa5L4713NIIIu1tueycc+LuKvqsLAcwLf3yxxbQ6KVZleFjl/5OOE+GdbmEG5z/vgZ/1ZJhEWg7M92FX/qdYgtUxU880EVcpsaQw1JIUJQRAQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use direct_splice_read() for tty, procfs, kernfs and random files rather than going through generic_file_splice_read() as they just copy the file into the output buffer and don't splice pages. This avoids the need for them to have a ->read_folio() to satisfy filemap_splice_read(). Signed-off-by: David Howells Acked-by: Greg Kroah-Hartman cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Miklos Szeredi cc: Arnd Bergmann cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- drivers/char/random.c | 4 ++-- drivers/tty/tty_io.c | 4 ++-- fs/kernfs/file.c | 2 +- fs/proc/inode.c | 4 ++-- fs/proc/proc_sysctl.c | 2 +- fs/proc_namespace.c | 6 +++--- 6 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/char/random.c b/drivers/char/random.c index ce3ccd172cc8..792713616ba8 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -1546,7 +1546,7 @@ const struct file_operations random_fops = { .compat_ioctl = compat_ptr_ioctl, .fasync = random_fasync, .llseek = noop_llseek, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; @@ -1557,7 +1557,7 @@ const struct file_operations urandom_fops = { .compat_ioctl = compat_ptr_ioctl, .fasync = random_fasync, .llseek = noop_llseek, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c index 36fb945fdad4..9d117e579dfb 100644 --- a/drivers/tty/tty_io.c +++ b/drivers/tty/tty_io.c @@ -466,7 +466,7 @@ static const struct file_operations tty_fops = { .llseek = no_llseek, .read_iter = tty_read, .write_iter = tty_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .poll = tty_poll, .unlocked_ioctl = tty_ioctl, @@ -481,7 +481,7 @@ static const struct file_operations console_fops = { .llseek = no_llseek, .read_iter = tty_read, .write_iter = redirected_tty_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .poll = tty_poll, .unlocked_ioctl = tty_ioctl, diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index e4a50e4ff0d2..9d23b8141db7 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -1011,7 +1011,7 @@ const struct file_operations kernfs_file_fops = { .release = kernfs_fop_release, .poll = kernfs_fop_poll, .fsync = noop_fsync, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; diff --git a/fs/proc/inode.c b/fs/proc/inode.c index f495fdb39151..711f12706469 100644 --- a/fs/proc/inode.c +++ b/fs/proc/inode.c @@ -591,7 +591,7 @@ static const struct file_operations proc_iter_file_ops = { .llseek = proc_reg_llseek, .read_iter = proc_reg_read_iter, .write = proc_reg_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .poll = proc_reg_poll, .unlocked_ioctl = proc_reg_unlocked_ioctl, .mmap = proc_reg_mmap, @@ -617,7 +617,7 @@ static const struct file_operations proc_reg_file_ops_compat = { static const struct file_operations proc_iter_file_ops_compat = { .llseek = proc_reg_llseek, .read_iter = proc_reg_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .write = proc_reg_write, .poll = proc_reg_poll, .unlocked_ioctl = proc_reg_unlocked_ioctl, diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c index 5851eb5bc726..e49f99657d1c 100644 --- a/fs/proc/proc_sysctl.c +++ b/fs/proc/proc_sysctl.c @@ -869,7 +869,7 @@ static const struct file_operations proc_sys_file_operations = { .poll = proc_sys_poll, .read_iter = proc_sys_read, .write_iter = proc_sys_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .llseek = default_llseek, }; diff --git a/fs/proc_namespace.c b/fs/proc_namespace.c index 846f9455ae22..492abbbeff5e 100644 --- a/fs/proc_namespace.c +++ b/fs/proc_namespace.c @@ -324,7 +324,7 @@ static int mountstats_open(struct inode *inode, struct file *file) const struct file_operations proc_mounts_operations = { .open = mounts_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, .poll = mounts_poll, @@ -333,7 +333,7 @@ const struct file_operations proc_mounts_operations = { const struct file_operations proc_mountinfo_operations = { .open = mountinfo_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, .poll = mounts_poll, @@ -342,7 +342,7 @@ const struct file_operations proc_mountinfo_operations = { const struct file_operations proc_mountstats_operations = { .open = mountstats_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, }; From patchwork Wed Mar 8 16:52:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166164 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBEB8C742A7 for ; Wed, 8 Mar 2023 16:53:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 665AA280002; Wed, 8 Mar 2023 11:53:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5EE3B6B0083; Wed, 8 Mar 2023 11:53:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41AC66B0085; Wed, 8 Mar 2023 11:53:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 243396B0082 for ; Wed, 8 Mar 2023 11:53:26 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E56C4A0368 for ; Wed, 8 Mar 2023 16:53:25 +0000 (UTC) X-FDA: 80546326770.07.7E94FBB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf14.hostedemail.com (Postfix) with ESMTP id 079DB100014 for ; Wed, 8 Mar 2023 16:53:23 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EDqA9xZJ; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294404; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cO3np2De/ugwikklpZQNS+rZPSCRCQBh402/ZSlB0NU=; b=biJUaftstPWuFu6JA64hqmi9L1zI3j6i6y2krOaYttGyd55ATVMdcQNnDtsHLarRrzkUEN yu61wOv5BGNUio6ay1WVosJ2dsTTL5pinKHjhqBv3Hkz/3QJISIQCTedO6KLUumQvHwX2B 7x6n5yWZ+I8mS3Q6B80Dc3beYICIVjw= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EDqA9xZJ; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294404; a=rsa-sha256; cv=none; b=0LT94qTN1brL2xxjWxSgVLHmNDXN11llC2uzBcAqMJJ2NVPdDRu1HWCoLUsRPIgrY6NKqP Jy8VqmqzHlaD2ikUnhHrM2M/CbZP9EnAgBE4o1jRuZsnESwG1X4s/ANgC/rIQ1YGDJwBBa tpIcijjYTNK89UTSuO7Bz63AHp4ox7s= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294403; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cO3np2De/ugwikklpZQNS+rZPSCRCQBh402/ZSlB0NU=; b=EDqA9xZJ2mtVbP68Kf8IaFXwyJM4bw0wYVri0OtKRxuiRITb6iPmvKiMCWbYcqItmQp4pV JC6TVpAktykQDP38AQO9vHCDKPeEtr3GrhTdRgOn/HKdzR///RsLXuZnQ6dt8Rneln2N2y 5cXo4i4msW4tKeFQp7IyuJ9CW9fH1Fs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-582-VQJz-tdFMcWDa0obmcHkjg-1; Wed, 08 Mar 2023 11:53:20 -0500 X-MC-Unique: VQJz-tdFMcWDa0obmcHkjg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 137A33C10C6C; Wed, 8 Mar 2023 16:53:19 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE7BC2026D4B; Wed, 8 Mar 2023 16:53:16 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Steve French , Christoph Hellwig , John Hubbard , linux-cifs@vger.kernel.org Subject: [PATCH v17 07/14] splice: Do splice read from a file without using ITER_PIPE Date: Wed, 8 Mar 2023 16:52:44 +0000 Message-Id: <20230308165251.2078898-8-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: uf1mpcgso69mfoofk5gjm3tkfybgnrko X-Rspamd-Queue-Id: 079DB100014 X-HE-Tag: 1678294403-637815 X-HE-Meta: U2FsdGVkX1+Rbr8mffl1j5p0qWNwqvDhuqUqY8WMj4fLchAXYKJKdcknKiU93U9FSjv5kvGRItGWJYbv9QL4Ebld1hY9YC+s7dY0Hy2uXcTYpHIF6Zk8nmY0ZpFc+agHKKtsN+iuhkadjvAdUVJsXRkcWw/f36sVHFt+nsTidWzYKXlAAGbPTf/S/7yF28CC8WFqqMFjIPRtoPrKG7ml/yiXG1YuAqk6UUwIB72j43egVuBpUNVM7w1aFxcWndLwq7cL+++y+XTclfGWajErpr+PU7lrSvJWJ4ogSmXqEO31veuVwmcp2vgXoP+dTg08APgXVpfH5AHShg0NOc7YvsQlV+taxn41RJrny5zoOc+5Nd2bqvEmIt2iRDhiCIMGg7WakyBi7Y8PYam+PX5q6knfgbrNrZquUzG1Z7SfXfXlj0zWYtnzxbnGPmz5651ncpO6yeGgK3797SZ4guGZTsGOfLAKOvioBzw5eZUwSr468YWtn3Qmg4s66rwly29g55SXDgeB55zyyaLJx4PRGhO8rGRK6EKctjFEGnebsLI+henJ5ZnCGAMiCILBidEV6+l9sN7cp18hO2Ekdx84Xl/oGBRTQ2qf1JUwJXuBlfp+94GumUsvXbEfsl0Zmk5m+A/SCD/eJk03VNKK+Aajk4u7Q5LNOQ/Xrt91UtTab2AgjAFrUHcvvZyz2PQTA1BS8g3mhJe+Szikpj1a0ogHOkGDZYOR9scVdRQzjj4HXMnSDyH7DxfguP9/bUKP9+qszyoN+WLYPjPDVqxc2Qh/pHm1kaa7bkw+zMAY8FybKlzj9We2bBoPLFcR9GEeycvs1pp1VS6Ms1qMqqSdDX/JWLGKoflIn3melf0Bg8zM5FxXX5NfKy34hlMIHEpovWV1kqQy0Qz2RqMXAtrbIfer5npPKnOwuzuf6CVnnPUdUQnzNiqndMxDDU4xKOF2UA/FW/cLN0novpQurA68dGJ tbDgcNCe N3B6B6p+w800xByqCpm48XhOmRPbwAgl8qlAesXrwsd+EggJiQN9UZSNI7SkDRzRYqyIKqM/fA1o2FL8QOuTh/OcEMT/PzNNMy/OdDQM3gQ0Qujr+WrZAi9KgGt7SePw0gIaum/KA0DRXKuvpbGoSoNj9aocz66rdwW1CcS29PTsVEPOmuNZO1Qv6LRT2IcBkZzDIrrIU4pJIryFMNh6c5VM/IvNYL2YvZ6cpA0Rb1xcjYinz1sp3DRQz8V+4zDgDevs3E2yoQB07Jo0u/raClVzDi5L5JWS1f1lsLNcMVvYeI2OyTAj4GewUUC6DgFLIud3h1GW6NlkwPORyCUGVG27v1NCU+2QAXsfPLeiBy/AnfEHVSSK0LdYDdnO6zGlYkRwoIM6xUxO33G2n/zrXndeVV3r1msKHNd3WoNNmFlFzwB0kuyqHa8wo4fO1a3roSt6yxF7bhdKlpF8kJBWn5C2zUMHQlFdcYp4fbxiIKkHYc/weqRh5ph7gzhQVIy8y1jXNaVZnRTqWbvIBVgCDiZJ5Xw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make generic_file_splice_read() use filemap_splice_read() and direct_splice_read() rather than using an ITER_PIPE and call_read_iter(). Make cifs use generic_file_splice_read() rather than doing it for itself. Unexport filemap_splice_read(). With this, ITER_PIPE is no longer used. Signed-off-by: David Howells cc: Jens Axboe cc: Steve French cc: Christoph Hellwig cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-cifs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org Reviewed-by: Christoph Hellwig --- fs/cifs/cifsfs.c | 8 ++++---- fs/cifs/cifsfs.h | 3 --- fs/cifs/file.c | 16 ---------------- fs/splice.c | 30 +++++++----------------------- mm/filemap.c | 1 - 5 files changed, 11 insertions(+), 47 deletions(-) diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c index cbcf210d56e4..ba963a26cb19 100644 --- a/fs/cifs/cifsfs.c +++ b/fs/cifs/cifsfs.c @@ -1359,7 +1359,7 @@ const struct file_operations cifs_file_ops = { .fsync = cifs_fsync, .flush = cifs_flush, .mmap = cifs_file_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, @@ -1379,7 +1379,7 @@ const struct file_operations cifs_file_strict_ops = { .fsync = cifs_strict_fsync, .flush = cifs_flush, .mmap = cifs_file_strict_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, @@ -1417,7 +1417,7 @@ const struct file_operations cifs_file_nobrl_ops = { .fsync = cifs_fsync, .flush = cifs_flush, .mmap = cifs_file_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, @@ -1435,7 +1435,7 @@ const struct file_operations cifs_file_strict_nobrl_ops = { .fsync = cifs_strict_fsync, .flush = cifs_flush, .mmap = cifs_file_strict_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, diff --git a/fs/cifs/cifsfs.h b/fs/cifs/cifsfs.h index 71fe0a0a7992..8b239854e590 100644 --- a/fs/cifs/cifsfs.h +++ b/fs/cifs/cifsfs.h @@ -100,9 +100,6 @@ extern ssize_t cifs_strict_readv(struct kiocb *iocb, struct iov_iter *to); extern ssize_t cifs_user_writev(struct kiocb *iocb, struct iov_iter *from); extern ssize_t cifs_direct_writev(struct kiocb *iocb, struct iov_iter *from); extern ssize_t cifs_strict_writev(struct kiocb *iocb, struct iov_iter *from); -extern ssize_t cifs_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags); extern int cifs_flock(struct file *pfile, int cmd, struct file_lock *plock); extern int cifs_lock(struct file *, int, struct file_lock *); extern int cifs_fsync(struct file *, loff_t, loff_t, int); diff --git a/fs/cifs/file.c b/fs/cifs/file.c index 4d4a2d82636d..321f9b7c84c9 100644 --- a/fs/cifs/file.c +++ b/fs/cifs/file.c @@ -5066,19 +5066,3 @@ const struct address_space_operations cifs_addr_ops_smallbuf = { .launder_folio = cifs_launder_folio, .migrate_folio = filemap_migrate_folio, }; - -/* - * Splice data from a file into a pipe. - */ -ssize_t cifs_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags) -{ - if (unlikely(*ppos >= file_inode(in)->i_sb->s_maxbytes)) - return 0; - if (unlikely(!len)) - return 0; - if (in->f_flags & O_DIRECT) - return direct_splice_read(in, ppos, pipe, len, flags); - return filemap_splice_read(in, ppos, pipe, len, flags); -} diff --git a/fs/splice.c b/fs/splice.c index 90ccd3666dca..f46dd1fb367b 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -387,29 +387,13 @@ ssize_t generic_file_splice_read(struct file *in, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags) { - struct iov_iter to; - struct kiocb kiocb; - int ret; - - iov_iter_pipe(&to, ITER_DEST, pipe, len); - init_sync_kiocb(&kiocb, in); - kiocb.ki_pos = *ppos; - ret = call_read_iter(in, &kiocb, &to); - if (ret > 0) { - *ppos = kiocb.ki_pos; - file_accessed(in); - } else if (ret < 0) { - /* free what was emitted */ - pipe_discard_from(pipe, to.start_head); - /* - * callers of ->splice_read() expect -EAGAIN on - * "can't put anything in there", rather than -EFAULT. - */ - if (ret == -EFAULT) - ret = -EAGAIN; - } - - return ret; + if (unlikely(*ppos >= file_inode(in)->i_sb->s_maxbytes)) + return 0; + if (unlikely(!len)) + return 0; + if (in->f_flags & O_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + return filemap_splice_read(in, ppos, pipe, len, flags); } EXPORT_SYMBOL(generic_file_splice_read); diff --git a/mm/filemap.c b/mm/filemap.c index 2723104cc06a..3a93515ae2ed 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2967,7 +2967,6 @@ ssize_t filemap_splice_read(struct file *in, loff_t *ppos, return total_spliced ? total_spliced : error; } -EXPORT_SYMBOL(filemap_splice_read); static inline loff_t folio_seek_hole_data(struct xa_state *xas, struct address_space *mapping, struct folio *folio, From patchwork Wed Mar 8 16:52:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94A4FC64EC4 for ; Wed, 8 Mar 2023 16:53:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 332F6280003; Wed, 8 Mar 2023 11:53:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BB436B0083; Wed, 8 Mar 2023 11:53:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1344E280003; Wed, 8 Mar 2023 11:53:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F05736B0082 for ; Wed, 8 Mar 2023 11:53:31 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9C677A0F7E for ; Wed, 8 Mar 2023 16:53:31 +0000 (UTC) X-FDA: 80546327022.14.325FC07 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf06.hostedemail.com (Postfix) with ESMTP id B0B2B180013 for ; Wed, 8 Mar 2023 16:53:29 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eLRPOIJ3; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294409; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XsRpjIP88f7UuB0TiBadjal83kAgF2un+t2WHwzlM/w=; b=Bd3bPHho1+t8gFA4ew2FfKwbWOys7ClisXnWU6iEpbeiVkjoVR6epDPQBHBxpReZE/v20t Xo/OlDNu7FjXWVATpomsv//s6g99AIt008EUsvyFXSCVcCf1zJWV82RGa8WGC1i/WuZlN/ wDVLX/d0SwBoFlQIamTLHlXypBUs+Ps= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eLRPOIJ3; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294409; a=rsa-sha256; cv=none; b=kkrco65qa0/Pg46GfdSiCCh1iM7p3l9oDLZ60iyB96JY+oVacVHjr5iOAiHiWF9+fKNYym YPhLq6aqRg9f3fp0MTXlLWdo61+8rRho9MT3Ve7cT5sWYKtMvLrhkciHvq93jXMNw72Nha IXz34IBntySUck+506pCjzi1Xi6cpS4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294408; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XsRpjIP88f7UuB0TiBadjal83kAgF2un+t2WHwzlM/w=; b=eLRPOIJ3I+w0pLbE6IETg8W1speVPurC+sD6dF8L9F7ueqYtQxXIFqn3zNQ6sLRwMMSgOB FpBEmOw82QKOhpHwQwZDUzW4Qs3SXzWL4h4tbV7xtnhbK+bLLmkaJKKyxnz7GfJHFIhoJd l1uEzVdlubL0qVd/3XAu5du73ulITcs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-558-iBdOJoOTPZWMtmN2YA3RcQ-1; Wed, 08 Mar 2023 11:53:22 -0500 X-MC-Unique: iBdOJoOTPZWMtmN2YA3RcQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E3AD8857A8E; Wed, 8 Mar 2023 16:53:21 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id C8EFD492B05; Wed, 8 Mar 2023 16:53:19 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v17 08/14] iov_iter: Kill ITER_PIPE Date: Wed, 8 Mar 2023 16:52:45 +0000 Message-Id: <20230308165251.2078898-9-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: B0B2B180013 X-Stat-Signature: xmy64sogmz9thcewk6w69qeeyzza8su6 X-Rspam-User: X-HE-Tag: 1678294409-279537 X-HE-Meta: U2FsdGVkX18hNr1JK0087C624BzktySW2tYZ7FNW6Yn7mpUdTy0BL+heM5SuJdnWzIUCfltbzHh6QNXmjuGrUiaaRQNw4PlTb9FOaln7CJm/LaaNIrqXkrmF8nSfCUtFNX8QEy7GtHpsgFWoz6655ynD3BI7WM1uWVvXcCX+LH2cvCXRohoq+lro3W6aiD6dVeX3EyZgRC8v5sOETteSzgw4fEzmTtYLyBKEreKgUuTMELgK1O9dCSQ/ymkxdvCqzDCcwp7apP0kxIh+GuwxIvpepA1I/X922NvTTl29mM0U6s0ia7WQOa+ABGtEU/m/L54vLABzSxxzZ7qhS/TqhNiSKp3B+nkZ/58Dx+NzcaLWpmw3CJYEVXHTMf94n/MkSmP3/SIEws7VoVLb3vrpX2/pFjoLlNcFHbgohGP3ZLwjV2alV105oJ2v/wonOqalcCsxlALznb4KWUMYp2Qdui6t2Cqabr+6e8PIhglB8VGSuKgeZWvmX4vWeh7LQ+lhVU2YtNna50VlAs42Ot1uoX0XqRQOF4j/ydaj2QiRquP8wxuHlM66ey4gJcH+UNEz+LRqyyQ8uAxFWbiYlDqiXIZm1/uh9r0NIt+qYgN59fxsYgtCJtUjz6rDEG6CeExYdB7gNE62vsfZHgs/Oxyu7Z1sIav5DouEvMeDRSo683Dttd35qdHyI1aD0KY9N5H2b2pmW69cYBrcmZOE5r/Go1HDrlJLxTrgKX2WlqUdjS51p2PolqtkCYPTWt96U4IogjYc82FiQ2mBA/Q35ZBKXrhPJTMFORr27QDeyL/VUpweaCt28zKeh2Amj0BMYhS+LEREnjtpvq5A7cJhTK+2vvJyAHgZIJNiw6nsY1Lq4PKswFnWzX1/LglQ3C920jStv3DeYkAoKw+kF6NRtSmSQAk7IDGhlYt/A6XOcm/bOmbH1/1SH7OOy/DtctRdgMOi49J+jei/UYg3QW3fxxE y0lVoFJ3 jdlDQ+glsM2unRkoyPyI8o1EAVs7ao3+gqz/KzfkwxrIFqD4GHLol1B1fQ8Ww2AZC0lmAbIsBToKZL1pJZbI/botb+f3arT+MFYQvHraUOaiapReGe8wHNCssPS9jx3mfoeBcsldasdSF0iXQfo/VULD6KtnQDXAXP04fStYNYg4ieE2WtXSlE1qDUiqcYNE+ZHV6UI69BDEv3+llsII5S3DON6GQwq0olXFTnC7DY0xfczGq1K+KwwM3YxPB4UdkG70XACNIajbBrirSLWBcd8sTFg0w/mW5S9iuJ1z53k6jTlTxyCXcTJf630cUyejUypHPV6hsQ50gniIH0zXNgQ8DP782XmEQbCzvQkc/ji92ixQvAlzf26wsE+hPmO1qbiaBbQMLW3gGLW4JwVsQmWezQVYNlykmTy+bImkS1J+291QLdJzkXj4nJy7agPU8KfEm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The ITER_PIPE-type iterator was only used for generic_file_splice_read(), but that has now been switched to either pull pages directly from the pagecache for buffered file splice-reads or to use ITER_BVEC instead for O_DIRECT file splice-reads. This leaves ITER_PIPE unused - so remove it. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- include/linux/uio.h | 14 -- lib/iov_iter.c | 429 +------------------------------------------- mm/filemap.c | 3 +- 3 files changed, 4 insertions(+), 442 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 27e3fd942960..74598426edb4 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -11,7 +11,6 @@ #include struct page; -struct pipe_inode_info; typedef unsigned int __bitwise iov_iter_extraction_t; @@ -25,7 +24,6 @@ enum iter_type { ITER_IOVEC, ITER_KVEC, ITER_BVEC, - ITER_PIPE, ITER_XARRAY, ITER_DISCARD, ITER_UBUF, @@ -55,15 +53,10 @@ struct iov_iter { const struct kvec *kvec; const struct bio_vec *bvec; struct xarray *xarray; - struct pipe_inode_info *pipe; void __user *ubuf; }; union { unsigned long nr_segs; - struct { - unsigned int head; - unsigned int start_head; - }; loff_t xarray_start; }; }; @@ -101,11 +94,6 @@ static inline bool iov_iter_is_bvec(const struct iov_iter *i) return iov_iter_type(i) == ITER_BVEC; } -static inline bool iov_iter_is_pipe(const struct iov_iter *i) -{ - return iov_iter_type(i) == ITER_PIPE; -} - static inline bool iov_iter_is_discard(const struct iov_iter *i) { return iov_iter_type(i) == ITER_DISCARD; @@ -247,8 +235,6 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec unsigned long nr_segs, size_t count); void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec, unsigned long nr_segs, size_t count); -void iov_iter_pipe(struct iov_iter *i, unsigned int direction, struct pipe_inode_info *pipe, - size_t count); void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count); void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray, loff_t start, size_t count); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 274014e4eafe..fad95e4cf372 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -14,8 +14,6 @@ #include #include -#define PIPE_PARANOIA /* for now */ - /* covers ubuf and kbuf alike */ #define iterate_buf(i, n, base, len, off, __p, STEP) { \ size_t __maybe_unused off = 0; \ @@ -186,150 +184,6 @@ static int copyin(void *to, const void __user *from, size_t n) return res; } -#ifdef PIPE_PARANOIA -static bool sanity(const struct iov_iter *i) -{ - struct pipe_inode_info *pipe = i->pipe; - unsigned int p_head = pipe->head; - unsigned int p_tail = pipe->tail; - unsigned int p_occupancy = pipe_occupancy(p_head, p_tail); - unsigned int i_head = i->head; - unsigned int idx; - - if (i->last_offset) { - struct pipe_buffer *p; - if (unlikely(p_occupancy == 0)) - goto Bad; // pipe must be non-empty - if (unlikely(i_head != p_head - 1)) - goto Bad; // must be at the last buffer... - - p = pipe_buf(pipe, i_head); - if (unlikely(p->offset + p->len != abs(i->last_offset))) - goto Bad; // ... at the end of segment - } else { - if (i_head != p_head) - goto Bad; // must be right after the last buffer - } - return true; -Bad: - printk(KERN_ERR "idx = %d, offset = %d\n", i_head, i->last_offset); - printk(KERN_ERR "head = %d, tail = %d, buffers = %d\n", - p_head, p_tail, pipe->ring_size); - for (idx = 0; idx < pipe->ring_size; idx++) - printk(KERN_ERR "[%p %p %d %d]\n", - pipe->bufs[idx].ops, - pipe->bufs[idx].page, - pipe->bufs[idx].offset, - pipe->bufs[idx].len); - WARN_ON(1); - return false; -} -#else -#define sanity(i) true -#endif - -static struct page *push_anon(struct pipe_inode_info *pipe, unsigned size) -{ - struct page *page = alloc_page(GFP_USER); - if (page) { - struct pipe_buffer *buf = pipe_buf(pipe, pipe->head++); - *buf = (struct pipe_buffer) { - .ops = &default_pipe_buf_ops, - .page = page, - .offset = 0, - .len = size - }; - } - return page; -} - -static void push_page(struct pipe_inode_info *pipe, struct page *page, - unsigned int offset, unsigned int size) -{ - struct pipe_buffer *buf = pipe_buf(pipe, pipe->head++); - *buf = (struct pipe_buffer) { - .ops = &page_cache_pipe_buf_ops, - .page = page, - .offset = offset, - .len = size - }; - get_page(page); -} - -static inline int last_offset(const struct pipe_buffer *buf) -{ - if (buf->ops == &default_pipe_buf_ops) - return buf->len; // buf->offset is 0 for those - else - return -(buf->offset + buf->len); -} - -static struct page *append_pipe(struct iov_iter *i, size_t size, - unsigned int *off) -{ - struct pipe_inode_info *pipe = i->pipe; - int offset = i->last_offset; - struct pipe_buffer *buf; - struct page *page; - - if (offset > 0 && offset < PAGE_SIZE) { - // some space in the last buffer; add to it - buf = pipe_buf(pipe, pipe->head - 1); - size = min_t(size_t, size, PAGE_SIZE - offset); - buf->len += size; - i->last_offset += size; - i->count -= size; - *off = offset; - return buf->page; - } - // OK, we need a new buffer - *off = 0; - size = min_t(size_t, size, PAGE_SIZE); - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - return NULL; - page = push_anon(pipe, size); - if (!page) - return NULL; - i->head = pipe->head - 1; - i->last_offset = size; - i->count -= size; - return page; -} - -static size_t copy_page_to_iter_pipe(struct page *page, size_t offset, size_t bytes, - struct iov_iter *i) -{ - struct pipe_inode_info *pipe = i->pipe; - unsigned int head = pipe->head; - - if (unlikely(bytes > i->count)) - bytes = i->count; - - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - if (offset && i->last_offset == -offset) { // could we merge it? - struct pipe_buffer *buf = pipe_buf(pipe, head - 1); - if (buf->page == page) { - buf->len += bytes; - i->last_offset -= bytes; - i->count -= bytes; - return bytes; - } - } - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - return 0; - - push_page(pipe, page, offset, bytes); - i->last_offset = -(offset + bytes); - i->head = head; - i->count -= bytes; - return bytes; -} - /* * fault_in_iov_iter_readable - fault in iov iterator for reading * @i: iterator @@ -433,46 +287,6 @@ void iov_iter_init(struct iov_iter *i, unsigned int direction, } EXPORT_SYMBOL(iov_iter_init); -// returns the offset in partial buffer (if any) -static inline unsigned int pipe_npages(const struct iov_iter *i, int *npages) -{ - struct pipe_inode_info *pipe = i->pipe; - int used = pipe->head - pipe->tail; - int off = i->last_offset; - - *npages = max((int)pipe->max_usage - used, 0); - - if (off > 0 && off < PAGE_SIZE) { // anon and not full - (*npages)++; - return off; - } - return 0; -} - -static size_t copy_pipe_to_iter(const void *addr, size_t bytes, - struct iov_iter *i) -{ - unsigned int off, chunk; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - for (size_t n = bytes; n; n -= chunk) { - struct page *page = append_pipe(i, n, &off); - chunk = min_t(size_t, n, PAGE_SIZE - off); - if (!page) - return bytes - n; - memcpy_to_page(page, off, addr, chunk); - addr += chunk; - } - return bytes; -} - static __wsum csum_and_memcpy(void *to, const void *from, size_t len, __wsum sum, size_t off) { @@ -480,44 +294,10 @@ static __wsum csum_and_memcpy(void *to, const void *from, size_t len, return csum_block_add(sum, next, off); } -static size_t csum_and_copy_to_pipe_iter(const void *addr, size_t bytes, - struct iov_iter *i, __wsum *sump) -{ - __wsum sum = *sump; - size_t off = 0; - unsigned int chunk, r; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - while (bytes) { - struct page *page = append_pipe(i, bytes, &r); - char *p; - - if (!page) - break; - chunk = min_t(size_t, bytes, PAGE_SIZE - r); - p = kmap_local_page(page); - sum = csum_and_memcpy(p + r, addr + off, chunk, sum, off); - kunmap_local(p); - off += chunk; - bytes -= chunk; - } - *sump = sum; - return off; -} - size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) { if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_pipe_to_iter(addr, bytes, i); if (user_backed_iter(i)) might_fault(); iterate_and_advance(i, bytes, base, len, off, @@ -539,42 +319,6 @@ static int copyout_mc(void __user *to, const void *from, size_t n) return n; } -static size_t copy_mc_pipe_to_iter(const void *addr, size_t bytes, - struct iov_iter *i) -{ - size_t xfer = 0; - unsigned int off, chunk; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - while (bytes) { - struct page *page = append_pipe(i, bytes, &off); - unsigned long rem; - char *p; - - if (!page) - break; - chunk = min_t(size_t, bytes, PAGE_SIZE - off); - p = kmap_local_page(page); - rem = copy_mc_to_kernel(p + off, addr + xfer, chunk); - chunk -= rem; - kunmap_local(p); - xfer += chunk; - bytes -= chunk; - if (rem) { - iov_iter_revert(i, rem); - break; - } - } - return xfer; -} - /** * _copy_mc_to_iter - copy to iter with source memory error exception handling * @addr: source kernel address @@ -594,9 +338,8 @@ static size_t copy_mc_pipe_to_iter(const void *addr, size_t bytes, * alignment and poison alignment assumptions to avoid re-triggering * hardware exceptions. * - * * ITER_KVEC, ITER_PIPE, and ITER_BVEC can return short copies. - * Compare to copy_to_iter() where only ITER_IOVEC attempts might return - * a short copy. + * * ITER_KVEC and ITER_BVEC can return short copies. Compare to + * copy_to_iter() where only ITER_IOVEC attempts might return a short copy. * * Return: number of bytes copied (may be %0) */ @@ -604,8 +347,6 @@ size_t _copy_mc_to_iter(const void *addr, size_t bytes, struct iov_iter *i) { if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_mc_pipe_to_iter(addr, bytes, i); if (user_backed_iter(i)) might_fault(); __iterate_and_advance(i, bytes, base, len, off, @@ -711,8 +452,6 @@ size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes, return 0; if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_page_to_iter_pipe(page, offset, bytes, i); page += offset / PAGE_SIZE; // first subpage offset %= PAGE_SIZE; while (1) { @@ -761,36 +500,8 @@ size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes, } EXPORT_SYMBOL(copy_page_from_iter); -static size_t pipe_zero(size_t bytes, struct iov_iter *i) -{ - unsigned int chunk, off; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - for (size_t n = bytes; n; n -= chunk) { - struct page *page = append_pipe(i, n, &off); - char *p; - - if (!page) - return bytes - n; - chunk = min_t(size_t, n, PAGE_SIZE - off); - p = kmap_local_page(page); - memset(p + off, 0, chunk); - kunmap_local(p); - } - return bytes; -} - size_t iov_iter_zero(size_t bytes, struct iov_iter *i) { - if (unlikely(iov_iter_is_pipe(i))) - return pipe_zero(bytes, i); iterate_and_advance(i, bytes, base, len, count, clear_user(base, len), memset(base, 0, len) @@ -821,32 +532,6 @@ size_t copy_page_from_iter_atomic(struct page *page, unsigned offset, size_t byt } EXPORT_SYMBOL(copy_page_from_iter_atomic); -static void pipe_advance(struct iov_iter *i, size_t size) -{ - struct pipe_inode_info *pipe = i->pipe; - int off = i->last_offset; - - if (!off && !size) { - pipe_discard_from(pipe, i->start_head); // discard everything - return; - } - i->count -= size; - while (1) { - struct pipe_buffer *buf = pipe_buf(pipe, i->head); - if (off) /* make it relative to the beginning of buffer */ - size += abs(off) - buf->offset; - if (size <= buf->len) { - buf->len = size; - i->last_offset = last_offset(buf); - break; - } - size -= buf->len; - i->head++; - off = 0; - } - pipe_discard_from(pipe, i->head + 1); // discard everything past this one -} - static void iov_iter_bvec_advance(struct iov_iter *i, size_t size) { const struct bio_vec *bvec, *end; @@ -898,8 +583,6 @@ void iov_iter_advance(struct iov_iter *i, size_t size) iov_iter_iovec_advance(i, size); } else if (iov_iter_is_bvec(i)) { iov_iter_bvec_advance(i, size); - } else if (iov_iter_is_pipe(i)) { - pipe_advance(i, size); } else if (iov_iter_is_discard(i)) { i->count -= size; } @@ -913,26 +596,6 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll) if (WARN_ON(unroll > MAX_RW_COUNT)) return; i->count += unroll; - if (unlikely(iov_iter_is_pipe(i))) { - struct pipe_inode_info *pipe = i->pipe; - unsigned int head = pipe->head; - - while (head > i->start_head) { - struct pipe_buffer *b = pipe_buf(pipe, --head); - if (unroll < b->len) { - b->len -= unroll; - i->last_offset = last_offset(b); - i->head = head; - return; - } - unroll -= b->len; - pipe_buf_release(pipe, b); - pipe->head--; - } - i->last_offset = 0; - i->head = head; - return; - } if (unlikely(iov_iter_is_discard(i))) return; if (unroll <= i->iov_offset) { @@ -1020,24 +683,6 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction, } EXPORT_SYMBOL(iov_iter_bvec); -void iov_iter_pipe(struct iov_iter *i, unsigned int direction, - struct pipe_inode_info *pipe, - size_t count) -{ - BUG_ON(direction != READ); - WARN_ON(pipe_full(pipe->head, pipe->tail, pipe->ring_size)); - *i = (struct iov_iter){ - .iter_type = ITER_PIPE, - .data_source = false, - .pipe = pipe, - .head = pipe->head, - .start_head = pipe->head, - .last_offset = 0, - .count = count - }; -} -EXPORT_SYMBOL(iov_iter_pipe); - /** * iov_iter_xarray - Initialise an I/O iterator to use the pages in an xarray * @i: The iterator to initialise. @@ -1162,19 +807,6 @@ bool iov_iter_is_aligned(const struct iov_iter *i, unsigned addr_mask, if (iov_iter_is_bvec(i)) return iov_iter_aligned_bvec(i, addr_mask, len_mask); - if (iov_iter_is_pipe(i)) { - size_t size = i->count; - - if (size & len_mask) - return false; - if (size && i->last_offset > 0) { - if (i->last_offset & addr_mask) - return false; - } - - return true; - } - if (iov_iter_is_xarray(i)) { if (i->count & len_mask) return false; @@ -1244,14 +876,6 @@ unsigned long iov_iter_alignment(const struct iov_iter *i) if (iov_iter_is_bvec(i)) return iov_iter_alignment_bvec(i); - if (iov_iter_is_pipe(i)) { - size_t size = i->count; - - if (size && i->last_offset > 0) - return size | i->last_offset; - return size; - } - if (iov_iter_is_xarray(i)) return (i->xarray_start + i->iov_offset) | i->count; @@ -1303,36 +927,6 @@ static int want_pages_array(struct page ***res, size_t size, return count; } -static ssize_t pipe_get_pages(struct iov_iter *i, - struct page ***pages, size_t maxsize, unsigned maxpages, - size_t *start) -{ - unsigned int npages, count, off, chunk; - struct page **p; - size_t left; - - if (!sanity(i)) - return -EFAULT; - - *start = off = pipe_npages(i, &npages); - if (!npages) - return -EFAULT; - count = want_pages_array(pages, maxsize, off, min(npages, maxpages)); - if (!count) - return -ENOMEM; - p = *pages; - for (npages = 0, left = maxsize ; npages < count; npages++, left -= chunk) { - struct page *page = append_pipe(i, left, &off); - if (!page) - break; - chunk = min_t(size_t, left, PAGE_SIZE - off); - get_page(*p++ = page); - } - if (!npages) - return -EFAULT; - return maxsize - left; -} - static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarray *xa, pgoff_t index, unsigned int nr_pages) { @@ -1482,8 +1076,6 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i, } return maxsize; } - if (iov_iter_is_pipe(i)) - return pipe_get_pages(i, pages, maxsize, maxpages, start); if (iov_iter_is_xarray(i)) return iter_xarray_get_pages(i, pages, maxsize, maxpages, start); return -EFAULT; @@ -1573,9 +1165,7 @@ size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *_csstate, } sum = csum_shift(csstate->csum, csstate->off); - if (unlikely(iov_iter_is_pipe(i))) - bytes = csum_and_copy_to_pipe_iter(addr, bytes, i, &sum); - else iterate_and_advance(i, bytes, base, len, off, ({ + iterate_and_advance(i, bytes, base, len, off, ({ next = csum_and_copy_to_user(addr + off, base, len); sum = csum_block_add(sum, next, off); next ? 0 : len; @@ -1660,15 +1250,6 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages) return iov_npages(i, maxpages); if (iov_iter_is_bvec(i)) return bvec_npages(i, maxpages); - if (iov_iter_is_pipe(i)) { - int npages; - - if (!sanity(i)) - return 0; - - pipe_npages(i, &npages); - return min(npages, maxpages); - } if (iov_iter_is_xarray(i)) { unsigned offset = (i->xarray_start + i->iov_offset) % PAGE_SIZE; int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE); @@ -1681,10 +1262,6 @@ EXPORT_SYMBOL(iov_iter_npages); const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t flags) { *new = *old; - if (unlikely(iov_iter_is_pipe(new))) { - WARN_ON(1); - return NULL; - } if (iov_iter_is_bvec(new)) return new->bvec = kmemdup(new->bvec, new->nr_segs * sizeof(struct bio_vec), diff --git a/mm/filemap.c b/mm/filemap.c index 3a93515ae2ed..470be06b6096 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2690,8 +2690,7 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, if (unlikely(iocb->ki_pos >= i_size_read(inode))) break; - error = filemap_get_pages(iocb, iter->count, &fbatch, - iov_iter_is_pipe(iter)); + error = filemap_get_pages(iocb, iter->count, &fbatch, false); if (error < 0) break; From patchwork Wed Mar 8 16:52:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166166 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D148DC678D5 for ; Wed, 8 Mar 2023 16:53:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72991280004; Wed, 8 Mar 2023 11:53:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 68D296B0083; Wed, 8 Mar 2023 11:53:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 468D8280004; Wed, 8 Mar 2023 11:53:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2BA496B0082 for ; Wed, 8 Mar 2023 11:53:33 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 02D0A140F5F for ; Wed, 8 Mar 2023 16:53:32 +0000 (UTC) X-FDA: 80546327106.28.0CBCE14 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf01.hostedemail.com (Postfix) with ESMTP id 4965B4000B for ; Wed, 8 Mar 2023 16:53:31 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cLrJwID4; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294411; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T0MgyEE2up5lyn6F4xZiFP7ZLu50ZEhkrR3VW4b8xrI=; b=I6ZrlRsRsDofmQn3lEmfgyWqn9TueQiK3X1MAm/v+IcjUZCcHvWOLaV+IakiqqeRlpyLVU 7XhiR9fDnCP3Tki7qo+oUukdnbX1ttI8rUIhLG8Kiez8yMV7yc/OopE0WbaIGrmuNqmeFJ x8IbYQKlqEoMCsuogpFVicV/eJaAaxA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cLrJwID4; spf=pass (imf01.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294411; a=rsa-sha256; cv=none; b=FqdC3JhRd4oQk3JeJqx7B6p2tlbPewTls1mF1ekS1Mrbf0uts435qW4ejmcVpTnV30pruE pDqZJm00BzY0k+vbZ3g87JfyeMz/lkZg7gep+lSCFU2YQYfclZ1kExc8tS7FmS7Qvi8cm8 RXQTQULktXMys3EtwZBi3U04tdrP1og= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294410; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T0MgyEE2up5lyn6F4xZiFP7ZLu50ZEhkrR3VW4b8xrI=; b=cLrJwID4AaeIrVCox/vxPc6W3AcgZRX8MkJsdRd8McPz/steGxrlZWSik87TnABMyKk4O3 OVIMp9Tybfk20YNYxxxCN/ycgLYCyZ0mlw8cTWnlC+1X1MONX4h0of+qTCDGhlW3veg/3K 2A6nlZc5DwBACUz1gPWXrmdHebRZxFY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-279-GmxZAvM4NvKOIUhelF-1kA-1; Wed, 08 Mar 2023 11:53:25 -0500 X-MC-Unique: GmxZAvM4NvKOIUhelF-1kA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 71C0287B2A3; Wed, 8 Mar 2023 16:53:24 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 858592166B2A; Wed, 8 Mar 2023 16:53:22 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Hubbard Subject: [PATCH v17 09/14] iomap: Don't get an reference on ZERO_PAGE for direct I/O block zeroing Date: Wed, 8 Mar 2023 16:52:46 +0000 Message-Id: <20230308165251.2078898-10-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 4965B4000B X-Stat-Signature: sffdqp6iq9a9w9ycwf11r8jczshfe5rp X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1678294411-270839 X-HE-Meta: U2FsdGVkX19ioT5oQCriZHgiI1VeKk2Dp7CJURKAxIWSSrgrGhRZyFpy7y/cxB8/V+u1oHf+RCE8JDgs+d0NYeTvkun3cBxkjQ2ihaRx8xLBWPj0Zg6i33gSkaIl3xU3v2gCh4IDNrrdmSyGr+E3IhdBcqEToGysSFuiv2yoNVii5D4VW4thoqTiSSyzynL48WwTORK2pu06RK9PoAJ+mY6dHtO+WEBZcnd9EhE5nJHMyiVYrN7WjOTVam93hF+IH7o8fL86bhc9JV2ir9mfG+ObG8k5rOA2szj4EBbfFcf3wUslqLlJlMy32t4hhiluLtaNh4H9yIQ8cSH6U2y6PhdFViCKaO/67RB7+k1XSPQ4flbs19mU/uyD+UkkjY1y250sf7ewiBV9DOtgbwml4I4gPrCEsK//eDdlg1MOY0sdqq8NZop5D4dHjQhLHS4eCjJ+4Dvh10lnYqEzfL2gM8vqRl5V/oieiUEksTZPNv5Ss1A+qjBDinXk9kqm/91DdzFbaPgp+0CD0zPduDsNfSzLo3iFodBZbfAGaBAkEy56jCNQ53iBJlxu+H+d0BEUUx/9jfy32HDR6jfO5EAocyAirAOVtZNZG75H3lEjtd4k0lT5YfigNQkBk2QLlXeb4rf1aN46whTdBrc0GhgVf+9CLFrHihu7AwQe/BjuVruqJlzoLsUG+tueEhheYBwB1bfDEb752sVy4MwDJMIQQ1nbWC4hl5iFgdIkvidjwIM1EMoKsQYvJ7owbSdxCTwZurFDlhRIgIJnY+AuX3GOB9y3sEsgwRjBrOIO/r8FM26fNCiDnJK3A9ulLoAwdZRwHQYrsGgvsn9hDr1fSc7o8CEmK7K2FpmCgQxV7eRAVc3amOLHx4FgCGghAfZ18K9K9kPJB2UCMi8pf73oAffsFslSXLtnQyxtlnNhmF+ARQAYkoCrOpFGYmwcVUqDPX1n7tRY/8a+Nqy+WXseA2d 4vjvEwKE 9GOPKyn5wYjLT1f1u0VglocW4WodIlb2bnm2vHbTWFbxHFird1c7eVohTX+IXeK7kmX3u1UQArbxK/BHHf9ip5SgQ5qb7UzXdbG3KALi6ewR6mlayEJDt/SIHF9knlaqaPVJk9bMqufIWgLxhJ7pQbH2YyhPTqbbtlV5hv9frYueIffzLDB7NA/mtJgYpTm6JcQsHVxHIm/h/CHi16geLWdm82YOdRHPbpxvCYXCd990Ma9O3yHZUe/icgH4s8e73j/fsRRKQwkbocUxUzSXZvn4sefp6yK30jhMCY77qGqXYJqWdi8XyAuvvHfCCeZh085kjdQxyTmxDN1Iw5KM7uM3YWysHb/hTs0rCX8YpMlyDHeycMObeENiOND7ercT9mxya X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ZERO_PAGE can't go away, no need to hold an extra reference. Signed-off-by: David Howells Reviewed-by: David Hildenbrand Reviewed-by: John Hubbard cc: Al Viro cc: David Hildenbrand cc: linux-fsdevel@vger.kernel.org Reviewed-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/iomap/direct-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index f771001574d0..850fb9870c2f 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -202,7 +202,7 @@ static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; - get_page(page); + bio_set_flag(bio, BIO_NO_PAGE_REF); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } From patchwork Wed Mar 8 16:52:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 706D0C6FD1E for ; Wed, 8 Mar 2023 16:53:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 443EA6B0080; Wed, 8 Mar 2023 11:53:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CCC6280005; Wed, 8 Mar 2023 11:53:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 21F286B0083; Wed, 8 Mar 2023 11:53:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0A9AF6B0080 for ; Wed, 8 Mar 2023 11:53:34 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id DD81FA30B5 for ; Wed, 8 Mar 2023 16:53:33 +0000 (UTC) X-FDA: 80546327106.22.2CFA19E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf16.hostedemail.com (Postfix) with ESMTP id 2B31418001F for ; Wed, 8 Mar 2023 16:53:31 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cgaoeWDA; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294412; a=rsa-sha256; cv=none; b=K/lsPWAPlGK826EVrJ1waXp1InyylpQZy9KTrs3M/4UDN6LU3XTI+y8QekNKgySrqGdasV xmmEAd8Nf+JWqz5u1hYDT4Ehq0jQWZTZa9i4bx2wRbnKLLJJZbaVlwFj87PDmVFRlG0VLg ovrznds9evoIUoLB9r88b6fpeHQbkFc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cgaoeWDA; spf=pass (imf16.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294412; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0+QymJ+GCZW28tJ8JK9HTXv9GxubpscA9882LeMgv9w=; b=2fpYjV8Xr3/p1031cpdS16N/n4XeRBbfSuYshdTLjdaut8eMVD50qIB0vvlVPkzlAzHyrQ re6f6XQQPeZiOaYUPuQtC1jiki2G0HiLRF72ELghVw2V8SO3rY9GtbHg26m34JkL1RIWGE WdyQpJ3DJYN1swpzSMfGDPuX+IUyNAE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294411; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0+QymJ+GCZW28tJ8JK9HTXv9GxubpscA9882LeMgv9w=; b=cgaoeWDA4GLtYnWKdXmE89cDwJGZvbmzo2xBJLx+bLIoqJvxpSum8duznH7Io5cyC5FGZO MEGDEGdkvuA6NFqFiY9wZyyEgoRNrhJjs5RRV15o5Gb5qfI7ypTXn5lEN3Gaq5k9t1o2gc 9zkiri3K1tzue68Koxg0Lyns0RmioMU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-581-t3_NrMduOtmhP4jO257PkQ-1; Wed, 08 Mar 2023 11:53:28 -0500 X-MC-Unique: t3_NrMduOtmhP4jO257PkQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 32CB7811E9C; Wed, 8 Mar 2023 16:53:27 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B99618EC7; Wed, 8 Mar 2023 16:53:25 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v17 10/14] block: Fix bio_flagged() so that gcc can better optimise it Date: Wed, 8 Mar 2023 16:52:47 +0000 Message-Id: <20230308165251.2078898-11-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspam-User: X-Rspamd-Queue-Id: 2B31418001F X-Rspamd-Server: rspam01 X-Stat-Signature: y5cnfismzwbkaucji6zz3dbi4ya6zp13 X-HE-Tag: 1678294411-674171 X-HE-Meta: U2FsdGVkX18WSiW+tN0MX6BcMzm1vx93RfvAG8FsUgvum1lMX8ePvrbWrbyoLS/FDfdVLtYp6atECQDfwcfsvjDVc7eTzhgFEf2D7T0QwpDiAgL+gi8EPGynyJZYBrqQOyzUKxpuVYiV1J5zqWkvaqFqSNLY84sCXJeiY6fBBv3DfxtOF69mFzqAx0q+WXj22wnopMIWZX/KI+YGVJ42fdSkYoUwZuYgxcbngLt412a/AgQnaUzEdXa04HLB2LnSVdbOFqUz5p5RAb8sYKxy+hPiPsUSKjHMjPlPIhYt1TlfRO5IhBjdsC8VxGWGGkfjsIxsSOn0KQN7Z2XNHzjOW988qD2JERT77zjfJXU4FTU0aHXgqHia/VZdLOf8pwzzHj87nuPuPMnfIe+1Yz1OowsXOyRUlKPtLCy7YfdUbYC2JRguNFuvch/BdKJe55Z5h6WQpA8wkbiHrVvbzH/M2/2sYTDtauwMXgGXaZ6VRov4eOt3cfSzf4dWXRbgpcD6RLN0Jdj6GXXORisGbJbZhGTxqf9mYGESq2PXwYOSLssn8X27vjaC++XC69FrU7+w6cNKsyoigsl60gFA5WelFRCwneAciRo//t71jAgsbkWacvAMccxtL0TFre2+ghsbkJyCyo6SuofFvsQjAMEdwZoRQVws5GsM1UwD2HKt6ZBGmegelBS8N0MFfJIKJEWj81WWKolCV42UDsygo5hBWBjOZJ75MCLvHzFNtjXJIvQCgXYpaxZVb1dY8ndMhyrKLLR5Q4h0Z5nSmGtwyrmX1VFB4ulwYsGMyENLQHZvkOwKTOwm1KETU6y2tVp9Sg7Ck2UlJoiP0ZCbATVGW5pKFUzZzbFLF6k74znHDRrCwgcgbZ1VldlKqSin/seKKrlLbpJd+mpZCKrruDq4PYuunUPR6a2nzofvAYr3RzmZwcxUAmytmWkFV1ke3PSjt2P7hGVP8laSQwzuU1eNPUd zZyeXhFY opMPiL++K0uPVxjzWtnmakFukBxlWSS3uAxRLfowdM8rKqMNFNbYZc8Iuwest3aspTyfoqOTPHALgS73y9AYVLXW5gTbCwAPz2triT1K4udEUH2J4kpumUj85H4EkEmSuEzpoGBEomAYPWvSiwKT5+SgySurdxEufWCko5gOfpLuYcRD45PcUeNof79ooYqV4XoZoe2bRCh1pBxLWYTGd6RZxx/0P/P+IrP9PYCqozeLlwuWT7uyCRL5gc/kHla0PVOhpfFj7+1qlxs/l7qPbXDM9aD6MIjkoSVIsRNt3XNOFV7/DGvjMUEvLn177fWFIsHK+bA4rVACSXCxMGRLljcjlLx9w5+6TuUWvVZO8a8m2koZ23sEGAcVFqu7lH6Dp6VvL/6FZbnGEumG+jZAhe327JCIas7eEDRwEbAOcm1b29y+hMOpaQktrea394NiNNyCJ5VEwjK8nwQuJvjy1BNPc2Xo8ZedAhor9yG6CK4YNrM8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fix bio_flagged() so that multiple instances of it, such as: if (bio_flagged(bio, BIO_PAGE_REFFED) || bio_flagged(bio, BIO_PAGE_PINNED)) can be combined by the gcc optimiser into a single test in assembly (arguably, this is a compiler optimisation issue[1]). The missed optimisation stems from bio_flagged() comparing the result of the bitwise-AND to zero. This results in an out-of-line bio_release_page() being compiled to something like: <+0>: mov 0x14(%rdi),%eax <+3>: test $0x1,%al <+5>: jne 0xffffffff816dac53 <+7>: test $0x2,%al <+9>: je 0xffffffff816dac5c <+11>: movzbl %sil,%esi <+15>: jmp 0xffffffff816daba1 <__bio_release_pages> <+20>: jmp 0xffffffff81d0b800 <__x86_return_thunk> However, the test is superfluous as the return type is bool. Removing it results in: <+0>: testb $0x3,0x14(%rdi) <+4>: je 0xffffffff816e4af4 <+6>: movzbl %sil,%esi <+10>: jmp 0xffffffff816dab7c <__bio_release_pages> <+15>: jmp 0xffffffff81d0b7c0 <__x86_return_thunk> instead. Also, the MOVZBL instruction looks unnecessary[2] - I think it's just 're-booling' the mark_dirty parameter. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Jens Axboe cc: linux-block@vger.kernel.org Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108370 [1] Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108371 [2] Link: https://lore.kernel.org/r/167391056756.2311931.356007731815807265.stgit@warthog.procyon.org.uk/ # v6 --- include/linux/bio.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index d766be7152e1..d9d6df62ea57 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -229,7 +229,7 @@ static inline void bio_cnt_set(struct bio *bio, unsigned int count) static inline bool bio_flagged(struct bio *bio, unsigned int bit) { - return (bio->bi_flags & (1U << bit)) != 0; + return bio->bi_flags & (1U << bit); } static inline void bio_set_flag(struct bio *bio, unsigned int bit) From patchwork Wed Mar 8 16:52:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166168 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47395C678D5 for ; Wed, 8 Mar 2023 16:53:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD0F8280005; Wed, 8 Mar 2023 11:53:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C09A66B0083; Wed, 8 Mar 2023 11:53:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A36A4280005; Wed, 8 Mar 2023 11:53:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8424E6B0082 for ; Wed, 8 Mar 2023 11:53:36 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5FD218047C for ; Wed, 8 Mar 2023 16:53:36 +0000 (UTC) X-FDA: 80546327232.02.909565F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 90CB51C000C for ; Wed, 8 Mar 2023 16:53:34 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IH4Xd0fh; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294414; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cRZ6zwkATo5WH/jIVyGNX8/GaSvHJQOFy9f4tGJFuL8=; b=xXljYfczNvqgTswxFZVrJPQL4n26rGv+S0kVkrpjpYWpfxpHozgk4t567Fmxemeq/zYdrM SDWIwdWGmPDT9Y2oHgU7ew7UqyczP5BFUKLYzPYNxd3vmJOg9ssTG+27iPF3uMZIlmJLnu p6c8sbtRFcfuLe1mAKGZ2yXZRzcEKbQ= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IH4Xd0fh; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294414; a=rsa-sha256; cv=none; b=zEuR8xqTvZtg5Pvqz+3JdHfw6hQA9m3GDIjOfJnQjM5r2KvU4s15BDU/NljS0hF8ZXEZXk lq63gmkvRDO2AisBIGRyZIT5MZJiLVKBichRmZ6lpCyL7N3m2Y72BBNxlVa0IGCsdNDzGu 67MA28nA6zyh6T54x2lGRWaA0ijjRJg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cRZ6zwkATo5WH/jIVyGNX8/GaSvHJQOFy9f4tGJFuL8=; b=IH4Xd0fhGmlQaqcIZS5y99v6Sio0SAgXQlyczdYMadyLhx0n3FghUX07gYnNBGwkhsXsoD MibTgM4pZozlHP7MaPFZEsPxM0nsbjMtBtNBhyqk18tBFIlgwKn4UluuUQd5x10dKBXZDF MLgGfV9uIZ3jH9HOld/b4E1aZVFs224= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-481-sN0k0v4zMtK8j61b-GBk5g-1; Wed, 08 Mar 2023 11:53:30 -0500 X-MC-Unique: sN0k0v4zMtK8j61b-GBk5g-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C3113800B23; Wed, 8 Mar 2023 16:53:29 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2335492B07; Wed, 8 Mar 2023 16:53:27 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v17 11/14] block: Replace BIO_NO_PAGE_REF with BIO_PAGE_REFFED with inverted logic Date: Wed, 8 Mar 2023 16:52:48 +0000 Message-Id: <20230308165251.2078898-12-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Stat-Signature: pat43yfjemh1z8n7jaud54eib89y1zc4 X-Rspam-User: X-Rspamd-Queue-Id: 90CB51C000C X-Rspamd-Server: rspam06 X-HE-Tag: 1678294414-73978 X-HE-Meta: U2FsdGVkX18nBdmSyR5mTtWpAU3nzTT/IZbvqa/DenJvPw8ENMefMd941THYCJh2DYzYgWoy6krnpMePwDNI+d1pAdfPUGIateduonfdauuTZ8NBUEAko9SBs3BNm36Cg8EdGX3erjCzXHYdQ74rpVQo9AnVSTUZcG5vValRXlhUBp19IaXAWvYA1tAWVkKsrmQ+UVuEqfk6vHi3WP8jXxBJVb6JqMyA61IIhqRjE0fjqAvdS2ClDRBkKIjeJ8icTbfHiRAUKz30dW66lck6FfLz4hrW44/1two1V+9/V8z49c3f+PxQH10LelHoAi6Ot2BAD0HYs+vWhcuh4O2klHFgnIdyq1BWCmzMtbmZHBiMUnop/FZruILsRzqKHGb1a4PKzsdhwUj/SaD7q5YHgDPAgdgSwQG0kCO0TQAPeXtagUAzshubaHZN0J74P7D/qX4KxRAAspqMnHij3kfHtBKNuS4kUcX2r+wHqLY+/U1zhOkmiNtLuot1bW3Ubyzml9dXiHwYPaHuLFr082ehXCIGFy4UV3nwg7ELie4L988Gjw+wnvwWcP5BWVuAg7LP1usM8WW589YK580T1keioMIqDT31JVkjHAYu0GfCG1LWuehQA2uvbUUQC14J8TmelM57bRy/3CVqZWlLd39+CVuMfyJpAfYjyG5uU26q+TG2zh7bogr1GVEtiCOD+WN75kU867ZiI0CUacotSXNUmJdUyVMBL4F4DWYbKpkvOSJL6KXNPCWj0STep9BPjo/d/PpVsGPPiwNTrKgWSHSl+VYCpP8z0aPkGjW1P9rDpVtpjML5Syf5A5YJyT0h6BiDUDSdLXKQqJ63TZanYd5qGaSu1s9MrQUVr92vWRGvyqULTe0CpCZX2MdvyIdGr7Pc4orZ1j8n5BiWSAtsfOBqB0Unx9gn2twCYxHLPaqIZ8uBKhgp2pOUlwMS6uGF0sUQnL6WMlAhZNlutFHUqeL rm7Mq/ef yFaV3uej2ED2g0oMx2KgRQ+0YKvGBncPRKS6u6TpGvQ7VIbGQg7i85LvVBiEldrhUNw+uXgCiCmYCQj8GfGMb0cO42xgUrirG0BYXLObWXIXZBWwDRKxw734YYOl+CHwr4OPaFveb4l5Brsv0rVxZH7prslmI+Xx73DKH+cjYSouhiSa7PwPuk9P2Oz771TfRiMFAj92vEOTiFjOxHl1zZ50DYZVIY+PMoB4lpvhRMHqWhEMlnxTamg8dDnPQb9UcULhV+1E/VaR7QD2fUhzRDxi/flKAElV3ruoM+Sc8rhmx0DLba6Ykk1OeVdF1SH5M+OSG+DbQvFPgABZ6tCApJoTFQTOXTk/AWmgbLS87HccYgXXurJAYutwqVOtI/JDJ+PK6QHV/NLp2YYu0mlK1TGIJmu2iqP9BGtqxAkCOsdXwZsyAwpCRRarNXnLXZFSmnkzCnWe8wLTKfLo0aU4aYW1kGpe4a/h2fHsNGye07RGyaXAnb7w4aKmUVw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Replace BIO_NO_PAGE_REF with a BIO_PAGE_REFFED flag that has the inverted meaning is only set when a page reference has been acquired that needs to be released by bio_release_pages(). Signed-off-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #8) - Split out from another patch [hch]. - Don't default to BIO_PAGE_REFFED [hch]. ver #5) - Split from patch that uses iov_iter_extract_pages(). block/bio.c | 2 +- block/blk-map.c | 1 + fs/direct-io.c | 2 ++ fs/iomap/direct-io.c | 1 - include/linux/bio.h | 2 +- include/linux/blk_types.h | 2 +- 6 files changed, 6 insertions(+), 4 deletions(-) diff --git a/block/bio.c b/block/bio.c index fd11614bba4d..4ff96a0e4091 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1190,7 +1190,6 @@ void bio_iov_bvec_set(struct bio *bio, struct iov_iter *iter) bio->bi_io_vec = (struct bio_vec *)iter->bvec; bio->bi_iter.bi_bvec_done = iter->iov_offset; bio->bi_iter.bi_size = size; - bio_set_flag(bio, BIO_NO_PAGE_REF); bio_set_flag(bio, BIO_CLONED); } @@ -1335,6 +1334,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) return 0; } + bio_set_flag(bio, BIO_PAGE_REFFED); do { ret = __bio_iov_iter_get_pages(bio, iter); } while (!ret && iov_iter_count(iter) && !bio_full(bio, 0)); diff --git a/block/blk-map.c b/block/blk-map.c index 9137d16cecdc..c77fdb1fbda7 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -281,6 +281,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (blk_queue_pci_p2pdma(rq->q)) extraction_flags |= ITER_ALLOW_P2PDMA; + bio_set_flag(bio, BIO_PAGE_REFFED); while (iov_iter_count(iter)) { struct page **pages, *stack_pages[UIO_FASTIOV]; ssize_t bytes; diff --git a/fs/direct-io.c b/fs/direct-io.c index ab0d7ea89813..47b90c68b369 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -403,6 +403,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio, bio->bi_end_io = dio_bio_end_aio; else bio->bi_end_io = dio_bio_end_io; + /* for now require references for all pages */ + bio_set_flag(bio, BIO_PAGE_REFFED); sdio->bio = bio; sdio->logical_offset_in_bio = sdio->cur_page_fs_offset; } diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 850fb9870c2f..ceeb0a183cea 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -202,7 +202,6 @@ static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; - bio_set_flag(bio, BIO_NO_PAGE_REF); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } diff --git a/include/linux/bio.h b/include/linux/bio.h index d9d6df62ea57..b537d03377f0 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -488,7 +488,7 @@ void zero_fill_bio(struct bio *bio); static inline void bio_release_pages(struct bio *bio, bool mark_dirty) { - if (!bio_flagged(bio, BIO_NO_PAGE_REF)) + if (bio_flagged(bio, BIO_PAGE_REFFED)) __bio_release_pages(bio, mark_dirty); } diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 99be590f952f..7daa261f4f98 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -318,7 +318,7 @@ struct bio { * bio flags */ enum { - BIO_NO_PAGE_REF, /* don't put release vec pages */ + BIO_PAGE_REFFED, /* put pages in bio_release_pages() */ BIO_CLONED, /* doesn't own data */ BIO_BOUNCED, /* bio is a bounce bio */ BIO_QUIET, /* Make BIO Quiet */ From patchwork Wed Mar 8 16:52:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A918C6FD1E for ; Wed, 8 Mar 2023 16:53:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C783280008; Wed, 8 Mar 2023 11:53:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 618936B0082; Wed, 8 Mar 2023 11:53:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CCC1280008; Wed, 8 Mar 2023 11:53:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 278F56B0081 for ; Wed, 8 Mar 2023 11:53:49 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 594C01A024B for ; Wed, 8 Mar 2023 16:53:48 +0000 (UTC) X-FDA: 80546327736.12.6FDF456 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 9A510180025 for ; Wed, 8 Mar 2023 16:53:46 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ZOBLq2Fz; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294426; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HEPrb4kIqMqW7eebyDDmobfSBttcdx5+yTOr0krGf5g=; b=PiS69IPr/qwnQSAvxWiuRDEyLzFSgrr/aI6LyYri3H322tKAznVkElw7rrPjsQKgwjx5V2 QAJfQjpAnVAa0Vf+JSzKl1b0SZ3W5G97UNw8MazY23q2LxE2B5IPIZVN+e6NjYQZmkcO+Z 666txmiWWN01/XXf+Zb+9KL/ji8b75A= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ZOBLq2Fz; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294426; a=rsa-sha256; cv=none; b=R1J0mPVtF832DujeGY3+nmDuYghtAW046auYg5YJ6mgc9ydj4TzDHtqqPdZBeyf4xe8ytV VLrwAJInvYUnZ8q9sNSCtc6/ldDZpwylSPatjoC3xxEgASMSaiNb4qeWSp3hWr9DlDECNn 0USY6116vs84V2QgpdPZJE9idyzde+8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HEPrb4kIqMqW7eebyDDmobfSBttcdx5+yTOr0krGf5g=; b=ZOBLq2FzJP55pDzVYqU4jwCmk+0lAPMZh7qjadxj34/rYUrypTMZ5A5jifzpHJwFqlJnjs f++1St9VZXKupgqIDITXamRYbIatfJaNHK1omg2YWGWGfgjao5f4RKLtheN3Ow8O5oiwOS sLcAa/3yRxeRAIkyGETtylf7YIwvLC0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-297-lMpGSRQOMG6okmNVI3BTqA-1; Wed, 08 Mar 2023 11:53:41 -0500 X-MC-Unique: lMpGSRQOMG6okmNVI3BTqA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 83C6418A650A; Wed, 8 Mar 2023 16:53:32 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 81ADA492C3E; Wed, 8 Mar 2023 16:53:30 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v17 12/14] block: Add BIO_PAGE_PINNED and associated infrastructure Date: Wed, 8 Mar 2023 16:52:49 +0000 Message-Id: <20230308165251.2078898-13-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9A510180025 X-Stat-Signature: zrqehowi45nk5r5b8cb95p4gsofb9ztc X-HE-Tag: 1678294426-199152 X-HE-Meta: U2FsdGVkX1+VdWQgloaavxeD9CV+3SOUosk2+6j6Ko2xiuuDutavQxuY9AfM+D11csWRznDLJGBfmPrN3j1TeSq6ouFFyDyec5fLWP6FUM/bI4/hQuFYOL2q09bcud31GmYPh23ji68Z/S57DvzNpOv8cmesQnnPeHwVuhJvnucjAMSqb288HZAMStwHfAyP9aKNVyC/woh7OSZYtPCQy37aXKcP/r7MP7HUMgS/G/8MtxEXNRUNxqkyFvcBNoh7LvmZFYOTZPu7ZGGJg6pTgevY41YWz/mq/gBgwY0LH+i6jBxcQnST4huYxjHCnybIIfmACgtSDqKTXApbd2v/AuRywMfyMWkVPGV/MfnZaLTJ6VJ6eobelz69pRQYjjvjJ/U0l4FxYvFsrBdlj6CLQhWpIiditee3CRkaGx3se+I4kIdz3cjq8Nk2YRWYnKiDebZomcZ9/9+GN3GQLNZ17apkvfxR/yrJi1onjGR+QnA+rtsxafhe0wSvoP4z1c7nP9BSGtkzVzxEPRvFv393KVBPSYutD6H0wPWykQymcELVUqcNvo+8nOgGC1rwr7snhlLmWN3qR186EnVGdyTkzg9vqZUzXJ/xfi7CABxTQljoQWGzzBoIVUY/eU2idVqaYmyheRBf7vdQJzK9RNTQz3ojxeme6GTNriiN3R11hl6K8RroyYXmpTNkGW3NA34YwSUveu4QHijuDhrmJ6RINY6VQTg54C8m15P/HmvJ901lr26BMTugKFOS+hX7P73oWOr21J0raeKXVJUwjFr3X93xGAuu43gd8HAP4X0Af/tExtyoqVByxsSi9OBTPV7ZT/iEng4KlXLb+JQ0lgOf7qfxA9yMZPhrPSJZjtdB743awcLMPnuyowcNK8OWHWori/2/w6FFs0NhO2x99ghveUo4EKugY8wLfdAAhPZ3NIGMoQ/QogqMW8qX3GoAL2FkaWHnzM9NZptOvdXbRxc d2rZitt+ ggh1nmKzn7hmupFY49ALvD0mJIcRrcQBvE8KDooh85ysHpFIUViTngSCFsS6uiL/Ykb5whW+gYiLEItf9USTXTFcKHMcE4CNmBkITJDDMx3nUm228V153Nkhcb7ohQgywcS6WZuH3o/7NhYl6t6EPlOPiUDhn1Sxq1k55G2yUFZCSGgNNZ+QkLQLxvMu+gtQhEZvalRJlPFJhaeDEFGMhh8aX3PmggMgQjm4Jgrb658U+WrVVxQ4en1wwPjviQ2irziGI7OfH5TpUnql10Fdgs0pzrYvSq/gyCABOCZ1Ek0TBRCaZ6QHJaseArhYEeMsrLKmk9PkbTQ3/x7WhqON2Y0+1cVy7T+0dxEP3LUQ1N0YAVhVJ4xy9sNRob57i9djlfOi7beeggAZVDa/jenlLDZUi6oDoaUvo+nROtigRVZHPM72mpoY3ExaWk02xTB/vifaPr5RyDcIlbjYI7ekj62E9665wX//pUlOMJ8qulOK3M3IGRkb0fDGTDQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add BIO_PAGE_PINNED to indicate that the pages in a bio are pinned (FOLL_PIN) and that the pin will need removing. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. ver #9) - Only consider pinning in bio_set_cleanup_mode(). Ref'ing pages in struct bio is going away. - page_put_unpin() is removed; call unpin_user_page() and put_page() directly. - Use bio_release_page() in __bio_release_pages(). - BIO_PAGE_PINNED and BIO_PAGE_REFFED can't both be set, so use if-else when testing both of them. ver #8) - Move the infrastructure to clean up pinned pages to this patch [hch]. - Put BIO_PAGE_PINNED before BIO_PAGE_REFFED as the latter should probably be removed at some point. FOLL_PIN can then be renumbered first. block/bio.c | 6 +++--- block/blk.h | 12 ++++++++++++ include/linux/bio.h | 3 ++- include/linux/blk_types.h | 1 + 4 files changed, 18 insertions(+), 4 deletions(-) diff --git a/block/bio.c b/block/bio.c index 4ff96a0e4091..51ae957cc4b6 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1168,7 +1168,7 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty) bio_for_each_segment_all(bvec, bio, iter_all) { if (mark_dirty && !PageCompound(bvec->bv_page)) set_page_dirty_lock(bvec->bv_page); - put_page(bvec->bv_page); + bio_release_page(bio, bvec->bv_page); } } EXPORT_SYMBOL_GPL(__bio_release_pages); @@ -1488,8 +1488,8 @@ void bio_set_pages_dirty(struct bio *bio) * the BIO and re-dirty the pages in process context. * * It is expected that bio_check_pages_dirty() will wholly own the BIO from - * here on. It will run one put_page() against each page and will run one - * bio_put() against the BIO. + * here on. It will unpin each page and will run one bio_put() against the + * BIO. */ static void bio_dirty_fn(struct work_struct *work); diff --git a/block/blk.h b/block/blk.h index cc4e8873dfde..d65d96994a94 100644 --- a/block/blk.h +++ b/block/blk.h @@ -432,6 +432,18 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, struct page *page, unsigned int len, unsigned int offset, unsigned int max_sectors, bool *same_page); +/* + * Clean up a page appropriately, where the page may be pinned, may have a + * ref taken on it or neither. + */ +static inline void bio_release_page(struct bio *bio, struct page *page) +{ + if (bio_flagged(bio, BIO_PAGE_PINNED)) + unpin_user_page(page); + else if (bio_flagged(bio, BIO_PAGE_REFFED)) + put_page(page); +} + struct request_queue *blk_alloc_queue(int node_id); int disk_scan_partitions(struct gendisk *disk, fmode_t mode); diff --git a/include/linux/bio.h b/include/linux/bio.h index b537d03377f0..d8c30c791a9a 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -488,7 +488,8 @@ void zero_fill_bio(struct bio *bio); static inline void bio_release_pages(struct bio *bio, bool mark_dirty) { - if (bio_flagged(bio, BIO_PAGE_REFFED)) + if (bio_flagged(bio, BIO_PAGE_REFFED) || + bio_flagged(bio, BIO_PAGE_PINNED)) __bio_release_pages(bio, mark_dirty); } diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 7daa261f4f98..a0e339ff3d09 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -318,6 +318,7 @@ struct bio { * bio flags */ enum { + BIO_PAGE_PINNED, /* Unpin pages in bio_release_pages() */ BIO_PAGE_REFFED, /* put pages in bio_release_pages() */ BIO_CLONED, /* doesn't own data */ BIO_BOUNCED, /* bio is a bounce bio */ From patchwork Wed Mar 8 16:52:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166169 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36F09C64EC4 for ; Wed, 8 Mar 2023 16:53:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3265280006; Wed, 8 Mar 2023 11:53:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB9FA6B0082; Wed, 8 Mar 2023 11:53:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0CF8280006; Wed, 8 Mar 2023 11:53:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8AD066B0081 for ; Wed, 8 Mar 2023 11:53:46 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 698291C620D for ; Wed, 8 Mar 2023 16:53:46 +0000 (UTC) X-FDA: 80546327652.02.08D5644 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf18.hostedemail.com (Postfix) with ESMTP id BE8FD1C0011 for ; Wed, 8 Mar 2023 16:53:44 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FzhWk6eQ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294424; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7b6tezcmsCRtrHxO2uDV+B0gtFM5s5ObrJQQe5H2QZI=; b=n+DxwoCkouwtjxPtEvKl5qVpiotnrJa+BGzvLvI7h6pWSfGxixKqZsnsMNp9lIMFmCjyF9 e7rbV3K+y4gzu6UFRlcJks98WXqyYVxbhQfA9bYnsVhC5FBsWTzsjGLReo6Mukoo0aRX9K LZssMKpr5BN/zZqcBA2Ez/4Q7HKEMbA= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FzhWk6eQ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294424; a=rsa-sha256; cv=none; b=Wkt1OsSaWHUoNhmWceyGSfRNsQrC3dQosrJ9bOi9/rIwQQxq0i6rmSk7C6FftwESXhjtXo /uKTcm/pE2SRl/Xm49UIiRn6PbnUtd902b4QwbHg2IaczKqPHK88dd/+9EpTfqkV3O1GQr eOFpcbUrPW3P22q/kGNPQKGioOaBWDE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294424; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7b6tezcmsCRtrHxO2uDV+B0gtFM5s5ObrJQQe5H2QZI=; b=FzhWk6eQ0tFtt/XL5Uoh3CrVojjHuxQUjAkRyeuX1jJ2+sh313CaGkzbYdZhGwKYu5DlPd ZkuZcS0c7b0wRJCriZr5OH9alrD63rCO5sNX17Nx/8o4eNcZBDajnBVYMa3tOD+TahsXub HYF+QalEaLiY+H6g/h/7wZDHo2/L/RQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-457-phLgeZkLN8-e2tx3s-Pb0w-1; Wed, 08 Mar 2023 11:53:39 -0500 X-MC-Unique: phLgeZkLN8-e2tx3s-Pb0w-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2E66E88B7DE; Wed, 8 Mar 2023 16:53:35 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20FAF492B09; Wed, 8 Mar 2023 16:53:33 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v17 13/14] block: Convert bio_iov_iter_get_pages to use iov_iter_extract_pages Date: Wed, 8 Mar 2023 16:52:50 +0000 Message-Id: <20230308165251.2078898-14-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Queue-Id: BE8FD1C0011 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: qhpnoee6o1fo67wtkpjo3epm3y6jon1f X-HE-Tag: 1678294424-64977 X-HE-Meta: U2FsdGVkX1/b8W+cT0wCK2PL//kd6RUtP/9SmvBaUGSbsOoYSueU1/1IX6QuG84fHyFFeXe56wu9vVYKKICcCVfCGD8GWCNDWSG8Cf/9dslWq/yA9DjDF2SpQeUkE9AlXX7ySrU8SsWYepbfD19I1ZOAqLJOf8x9L6SkbRxjVX/O9vI+I9I4kprqHqzKLvurbsEyBo177RAuK+mUpf50tZtq5qi3jeXAMghYDk895ZJcBpg6OX/9Mfj+orLj2rSpdG+drMwhby/7LFqJnWxtXnydw0FseYJgvfCt8IZALqX1TzH7O2dwt0wDAAgsOJ8H8oLdepvJAhku8fVvqed7e/spMhXjZhP31GtIMgd99qTVrXb1sU2dY2T/NDA8uhXqug4RqyqImFWwGxLTyW5wKulbfN7SonyTwDSe1kzcAAumWVif09PZsTmgh4mKRPsRxEJWXd4syj/G7HZL1vcFmjrzNPutqEmyk+qamfByoSUP+1EB2a8kzpivdOYdlTyxrDhk8KPvkDEZ9hHjhJos2dEMI/E0naMKsUypD034CqIH7EN9kbBv31Lw4mzQ3wBwO1te3f4ehnL5qONI7jp8p78ASIBTN/L0Q6C7ZxlzFDHsBtyyTjx/nFig09HvMLJ09K36OYaz2/rHKgggRvhTiYi+rvgE/HyEWa7q7mrlas9fIdmf+J8SamPiLbhLRrlmjixCnSDVi67RXxMzH3ZZl4aa8Ipug4tC10W61BlkjbTXRAhQTCKYG82vaYAlzAhN5mHjOd9lqKiR/ZwVHu1wjyMt5c0jOU9M5cIJp3OXHFKPMVnoexjeG7k/e/pkHa9AK6z65GM+Lzg97rnIVS2mUDch4iqFmDie79iMZHkrzsCx85BvU7M+Swkl7cHsLRxFTpvS20JT5xlKaYcG+yZ+IFqE2xQXOIweOivqf4lmEX1aSYMjeHKQ/T6XfTg2Qma4goaQdHRLzM2W3y0HnT7 9IIAPacR 5Or+LJWH+CDs7LFu3ahFiymco+tTgrvWy9qswgk/9fANV06UqImrOrcJ5XpXkyOKeK67RhzHbCc2s6N02IdyfC2oXNbPRK7IFdUjklYSQI/1OXqGABKJgA8tw7qDUQ9hWafltnUTm2LMlozQdxHUq2K23Tcz2pD77sBcLargd/nGTDYulSjb7/jtU/bg7iV/Zx+i8Dbzk62NV4pzBGEIVBEvxDMLn7dWsljB4cHNSUl/lLz5CszbRQ6eNQi+Lm4UdZc9MZosr25ErOIpqagvurrmTxo5pcghW/wrM92SdPYiAtoGvxnAzdxIEl9za3kJtmNhSy/4HSWE49PfAsB6GDk+5GJoM4mv5Bq6WSRNy3Flwz3F3pliIG3SJXqc+Ih6OYZZin8VO39tV+k8Pa3juz4AV3Hf2u0e7GQNOXYGg3RymK4IoqvqJLLbGxfON4PFZBbMSzPNFjDdPA7S5ax0siGDJv0lV/Oiv5fn03fYpBXQcSEc8J3vH0H13fQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This will pin pages or leave them unaltered rather than getting a ref on them as appropriate to the iterator. The pages need to be pinned for DIO rather than having refs taken on them to prevent VM copy-on-write from malfunctioning during a concurrent fork() (the result of the I/O could otherwise end up being affected by/visible to the child process). Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. ver #8) - Split the patch up a bit [hch]. - We should only be using pinned/non-pinned pages and not ref'd pages, so adjust the comments appropriately. ver #7) - Don't treat BIO_PAGE_REFFED/PINNED as being the same as FOLL_GET/PIN. ver #5) - Transcribe the FOLL_* flags returned by iov_iter_extract_pages() to BIO_* flags and got rid of bi_cleanup_mode. - Replaced BIO_NO_PAGE_REF to BIO_PAGE_REFFED in the preceding patch. block/bio.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/block/bio.c b/block/bio.c index 51ae957cc4b6..fc98c1c723ca 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1204,7 +1204,7 @@ static int bio_iov_add_page(struct bio *bio, struct page *page, } if (same_page) - put_page(page); + bio_release_page(bio, page); return 0; } @@ -1218,7 +1218,7 @@ static int bio_iov_add_zone_append_page(struct bio *bio, struct page *page, queue_max_zone_append_sectors(q), &same_page) != len) return -EINVAL; if (same_page) - put_page(page); + bio_release_page(bio, page); return 0; } @@ -1229,10 +1229,10 @@ static int bio_iov_add_zone_append_page(struct bio *bio, struct page *page, * @bio: bio to add pages to * @iter: iov iterator describing the region to be mapped * - * Pins pages from *iter and appends them to @bio's bvec array. The - * pages will have to be released using put_page() when done. - * For multi-segment *iter, this function only adds pages from the - * next non-empty segment of the iov iterator. + * Extracts pages from *iter and appends them to @bio's bvec array. The pages + * will have to be cleaned up in the way indicated by the BIO_PAGE_PINNED flag. + * For a multi-segment *iter, this function only adds pages from the next + * non-empty segment of the iov iterator. */ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) { @@ -1264,9 +1264,9 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) * result to ensure the bio's total size is correct. The remainder of * the iov data will be picked up in the next bio iteration. */ - size = iov_iter_get_pages(iter, pages, - UINT_MAX - bio->bi_iter.bi_size, - nr_pages, &offset, extraction_flags); + size = iov_iter_extract_pages(iter, &pages, + UINT_MAX - bio->bi_iter.bi_size, + nr_pages, extraction_flags, &offset); if (unlikely(size <= 0)) return size ? size : -EFAULT; @@ -1299,7 +1299,7 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) iov_iter_revert(iter, left); out: while (i < nr_pages) - put_page(pages[i++]); + bio_release_page(bio, pages[i++]); return ret; } @@ -1334,7 +1334,8 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) return 0; } - bio_set_flag(bio, BIO_PAGE_REFFED); + if (iov_iter_extract_will_pin(iter)) + bio_set_flag(bio, BIO_PAGE_PINNED); do { ret = __bio_iov_iter_get_pages(bio, iter); } while (!ret && iov_iter_count(iter) && !bio_full(bio, 0)); From patchwork Wed Mar 8 16:52:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13166170 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B215C678D5 for ; Wed, 8 Mar 2023 16:53:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CA6A9280007; Wed, 8 Mar 2023 11:53:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C31066B0082; Wed, 8 Mar 2023 11:53:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AAA386B0083; Wed, 8 Mar 2023 11:53:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 95A646B0081 for ; Wed, 8 Mar 2023 11:53:48 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 032FA1A06B3 for ; Wed, 8 Mar 2023 16:53:47 +0000 (UTC) X-FDA: 80546327736.19.8AA8CDF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf10.hostedemail.com (Postfix) with ESMTP id 09729C0021 for ; Wed, 8 Mar 2023 16:53:45 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JxKwvN2s; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678294426; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gIKMSgtKI0+T2jjDihR3uNZ9dKHoZzHE/RnqYef6EBU=; b=wZhYo+I8SSSGArHG3AwmNJ7aK5FQif+SW1fxiMOeCMYLyfLlBdozsuOOmBjWKY4zB3fALp eh7Ab8q+DFt+0LKFPWwXI84vVwv4xeTS4K9Fxs5tNfSXs54rMdoJQMS4OY+BpadlcNpVih k57lVINw44hNiT0mxXgR3WBBrStmQco= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JxKwvN2s; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678294426; a=rsa-sha256; cv=none; b=RT/MObXGEf7oEOt9JGbuCCoP//BVKNjYzFveoB88oMJkc9s52tSzqlYbniSNXVxreH6Nr+ zvyTV+OOD1yNkkoeUFtG5EW93bbmJacZdJma6a8tzijlvVqdGyqlezVAXDw5qp+AOQcgaw w1v+y/mQTwH5/zfFCTDMXwsg4XJVDCM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678294425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gIKMSgtKI0+T2jjDihR3uNZ9dKHoZzHE/RnqYef6EBU=; b=JxKwvN2s9N7I+fG6nrIs1jEtX7w4AvWrrk2xiZDCVOwuEj5TGBqXpKrcyB6jq0G13NP70S SR4meaACkYlJyD/X1ohZYJC+TMgg4suqr7lwQZpO+eFE+YB65palEDAD2OyUpzlq70WUJY 6MvnJY60qDiT3WFhf0jWzULpt0p7FH0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-653-kBmAFUeENu2Sp64iGCfrUw-1; Wed, 08 Mar 2023 11:53:40 -0500 X-MC-Unique: kBmAFUeENu2Sp64iGCfrUw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E02FE882823; Wed, 8 Mar 2023 16:53:38 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id DCEEA492B05; Wed, 8 Mar 2023 16:53:35 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v17 14/14] block: convert bio_map_user_iov to use iov_iter_extract_pages Date: Wed, 8 Mar 2023 16:52:51 +0000 Message-Id: <20230308165251.2078898-15-dhowells@redhat.com> In-Reply-To: <20230308165251.2078898-1-dhowells@redhat.com> References: <20230308165251.2078898-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Queue-Id: 09729C0021 X-Stat-Signature: s6hb7uzmqpa78mpnpsxab7np54y9ujwg X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1678294425-899934 X-HE-Meta: U2FsdGVkX18ezBxNLYSPvXXAzkrnUry89iHy//gsQ4SbdBX+O8/0OnF+12ci2FLPZC0oZBdOPPMCgs86psn1uqU+CXTM2H7krBQ98Wb+rI3ti0cjfaG9pQ6cwDLSMlPSasr0niMbDt0FV052usgZZxgOGswimZOZD4pnO67TTrS/wjjhE5fsOYZc9UxGHo3iX9MrJvXWsm6dZd21Qwu626VxgKqrnqi8TXENkPxe8uvUJUPMonoPVpyFvkj6uVR/4d8PAG9sYwFKj2h56MpNLd7xG/vPtW0pc6D8YzFu0NGx0gRrk448UxLunnLfBnAxT3aXN3jDxZpu2zXIdXijPILL2lv8ap5DOh0f2rQ1OIDs9tNFq4HlUSKhIv/IOFPc0bM9s+3YlWeCUJ/zku+iLM9UMxPC9BzMLDzaMVS60I6HDibfu708BYRw2Ge7R4uD52XeLB2I9J76UYfZYDEdmnh4edD8e9E9mqugif58MYjaZVQ0YIDHfjF2LKX75r3THyooJ5tGesnHY6544iDpLW3ziwR/u3Waro223e058NS7r+LTMCNPr8jUA0NE7fvbpWNtg/sLNJuMOAR3YZ+28g8mbBMskoZ4P7PTFyF/4ThdSyzKYY0OKgCaylEN86wjcQ1IpH2VuZFfY795yov09YH8aih3pk9H75OXj6juyMVeF5mc2WAjcdIypE0cpbdNMPg/rNQO1mhmrjsDxoVZW2llv5tNpcNf3a75sDDLzAA3XBQ+CW7uwykhnlY6UEiV8j8cdXvhCTxOERTUJMS3/jKUKzJQXDrAdT1dsH9p/Cied0XH7AbnsclfzxH052yoKc+ISb47r8NvZdShttXa1ERM+iSvO49Bt3bmlAgQjK75vw4ZGFlOGQAqAEbqfEktJlYNzXX9A3oJpc+FJ1evumBAM4blzYiqTgu4JMC9MT1NpoLyltlMlN9tNCTWndl4F8UKdfgFfr9rS6deqVI fbs8idzP nCAfQL+imGeJbnuTXUHrTDqI3WscRpns0npBU4ZGKfH9QdGc/2L53tRaJg77fRiRcj+U/50qn/yHvliSP3O0uOnsPOvKd4y1YZw52yQ+bNEaWhkNn1rsPO0qvu2qpTBDlEyy1jRztXapZlL0jYs8QfBVGTClgDoHp4UMmnLp3JPfg0S3+XJy8Dd5fKZOP6B8XVzsMou9cVkpkH1U1iQg1hrSw931UxUGbnEI4d3Qipl4oZtZj3G4iCHcDy1KExQIAzHXcAEmN6gAwYop2BQ1zSStTquKAQc2isx2oQmTsTIqnzluATMmqSHRaAjJK/ZhWuwVC1HtDDeI30mLjID5au0Fwqc8yaMWtVyEKhS9nujsBWQSm6wV2ZYx+hUqkCXOMfPvVhWIHy5IqfZdn2nuek+1NII3QMFXUUQdTLGVJJtdqDOnxgLAEcysUynRF8WN4pAUAylGO7vBGXW79xF8ALrN7JfFw8Zj3a8Pxmh7VNeJCjw+zbpIMy/1ziQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This will pin pages or leave them unaltered rather than getting a ref on them as appropriate to the iterator. The pages need to be pinned for DIO rather than having refs taken on them to prevent VM copy-on-write from malfunctioning during a concurrent fork() (the result of the I/O could otherwise end up being visible to/affected by the child process). Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. ver #8) - Split the patch up a bit [hch]. - We should only be using pinned/non-pinned pages and not ref'd pages, so adjust the comments appropriately. ver #7) - Don't treat BIO_PAGE_REFFED/PINNED as being the same as FOLL_GET/PIN. ver #5) - Transcribe the FOLL_* flags returned by iov_iter_extract_pages() to BIO_* flags and got rid of bi_cleanup_mode. - Replaced BIO_NO_PAGE_REF to BIO_PAGE_REFFED in the preceding patch. block/blk-map.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/block/blk-map.c b/block/blk-map.c index c77fdb1fbda7..7b12f4bb4d4c 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -280,22 +280,21 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (blk_queue_pci_p2pdma(rq->q)) extraction_flags |= ITER_ALLOW_P2PDMA; + if (iov_iter_extract_will_pin(iter)) + bio_set_flag(bio, BIO_PAGE_PINNED); - bio_set_flag(bio, BIO_PAGE_REFFED); while (iov_iter_count(iter)) { - struct page **pages, *stack_pages[UIO_FASTIOV]; + struct page *stack_pages[UIO_FASTIOV]; + struct page **pages = stack_pages; ssize_t bytes; size_t offs; int npages; - if (nr_vecs <= ARRAY_SIZE(stack_pages)) { - pages = stack_pages; - bytes = iov_iter_get_pages(iter, pages, LONG_MAX, - nr_vecs, &offs, extraction_flags); - } else { - bytes = iov_iter_get_pages_alloc(iter, &pages, - LONG_MAX, &offs, extraction_flags); - } + if (nr_vecs > ARRAY_SIZE(stack_pages)) + pages = NULL; + + bytes = iov_iter_extract_pages(iter, &pages, LONG_MAX, + nr_vecs, extraction_flags, &offs); if (unlikely(bytes <= 0)) { ret = bytes ? bytes : -EFAULT; goto out_unmap; @@ -317,7 +316,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (!bio_add_hw_page(rq->q, bio, page, n, offs, max_sectors, &same_page)) { if (same_page) - put_page(page); + bio_release_page(bio, page); break; } @@ -329,7 +328,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, * release the pages we didn't map into the bio, if any */ while (j < npages) - put_page(pages[j++]); + bio_release_page(bio, pages[j++]); if (pages != stack_pages) kvfree(pages); /* couldn't stuff something into bio? */