From patchwork Wed Jun 5 09:24:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kundan Kumar X-Patchwork-Id: 13686526 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3DA534963B for ; Wed, 5 Jun 2024 09:47:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717580849; cv=none; b=SijugeGezkd0AkKeRnWYsAEi3TQwdHqpO2JXBNz4LuauF31EA3JoVr7XH0u/ck4Tc9mDHHNyW/+u3VLk4mspssiKJPcpaSya02hmDJIyKDCAAwMoYsAWE8nwSD+uxi4Esx1ms4+DFNilCTmow01Ha2NjoEL+FM33VDYBJMxeSb0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717580849; c=relaxed/simple; bh=z9bv2Ist6mwPl7hHSCtjwyEw0bOmwOXU2jRMieukxYk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:MIME-Version: Content-Type:References; b=bUqnptphF1+IYxelOJyh4Zx7e6KgGMaP/9wQIEnC0C/eVaxWfejcOvIpm6C20cjQH+mtNH8rWgegvPpzMSdfgkh8kI0XhcslgJUr7IgsxH38jq4CSrNrPcTW2kvHM/Wzp7czG3fo4dBcc8zEQ1no9TKWvTakQPcRFrRz0AoaqUw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=JTuqvpPB; arc=none smtp.client-ip=203.254.224.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="JTuqvpPB" Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20240605094725epoutp04526668b32919273b3715048b5917c1bf~WEq283Rgv0987809878epoutp04g for ; Wed, 5 Jun 2024 09:47:25 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20240605094725epoutp04526668b32919273b3715048b5917c1bf~WEq283Rgv0987809878epoutp04g DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1717580845; bh=+/FY4qp65VT+KCyw1+tjmOORXZF0ADAnQINAlnR9E8I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JTuqvpPBSZ/Ayd3KBxv3j3isJtZ1dZ1loQy5bsdmNmS08w6TcAVjkAr+W6sb85J5N g6JRpmD6CoTrNfuxb206IoJ6Xrga3nHhGwMg09vTL6/qVgiqxdqLVeuLH4TghU/BCl ULm4Mpy3dZOlbsZAMrgpNfdVjqR+5FzOmlqmTRzg= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20240605094724epcas5p3bbe87c1aa8618c22f65ba5d229fa624f~WEq2gSsIb0300203002epcas5p37; Wed, 5 Jun 2024 09:47:24 +0000 (GMT) Received: from epsmgec5p1new.samsung.com (unknown [182.195.38.176]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4VvN0q1PVxz4x9Q5; Wed, 5 Jun 2024 09:47:23 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmgec5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 6D.15.08853.B2430666; Wed, 5 Jun 2024 18:47:23 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20240605093225epcas5p335664b9185c99a8fe1d661227d3f4f1a~WEdw8ZGy70617006170epcas5p3u; Wed, 5 Jun 2024 09:32:25 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240605093225epsmtrp1f382af9fc9a3f78596ddb3bc9ddc840d~WEdw7gyQ_2905729057epsmtrp1V; Wed, 5 Jun 2024 09:32:25 +0000 (GMT) X-AuditID: b6c32a44-fc3fa70000002295-25-6660342bd4f8 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id D6.F1.18846.9A030666; Wed, 5 Jun 2024 18:32:25 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240605093223epsmtip23e3a23965b97bc361abd85b192d77956~WEdvLezuB2783227832epsmtip2b; Wed, 5 Jun 2024 09:32:23 +0000 (GMT) From: Kundan Kumar To: axboe@kernel.dk, hch@lst.de, willy@infradead.org, kbusch@kernel.org Cc: linux-block@vger.kernel.org, joshi.k@samsung.com, mcgrof@kernel.org, anuj20.g@samsung.com, nj.shetty@samsung.com, c.gameti@samsung.com, gost.dev@samsung.com, Kundan Kumar Subject: [PATCH v4 1/2] block: add folio awareness instead of looping through pages Date: Wed, 5 Jun 2024 14:54:54 +0530 Message-Id: <20240605092455.20435-2-kundan.kumar@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240605092455.20435-1-kundan.kumar@samsung.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmuq62SUKawe0rLBZNE/4yW6y+289m 8X17H4vFzQM7mSxWrj7KZHH0/1s2i0mHrjFabP3yldVi7y1tixsTnjJabPs9n9ni9485bA48 HptXaHlcPlvqsWlVJ5vH7psNbB59W1YxenzeJBfAFpVtk5GamJJapJCal5yfkpmXbqvkHRzv HG9qZmCoa2hpYa6kkJeYm2qr5OIToOuWmQN0oZJCWWJOKVAoILG4WEnfzqYov7QkVSEjv7jE Vim1ICWnwKRArzgxt7g0L10vL7XEytDAwMgUqDAhO6N1+wXWgkXKFV/f9jI1MB6Q7mLk4JAQ MJE4NSu9i5GLQ0hgN6PE7/U/2CGcT4wS3x9PYoFz/h9/zdzFyAnWcfdRLytEYiejxLbTH6Cc z4wSP3v3MoHMZRPQlfjRFArSICLgLjH15SNGkBpmgbOMEiemPmIBSQgLhEpMnPSHFcRmEVCV mPNjGxuIzStgK7Gq/RATxDZ5iZmXvrOD2JwCdhLfW3tYIWoEJU7OfAI2hxmopnnrbGaQBRIC vRwSu/ueQ53qIrFjyw82CFtY4tXxLewQtpTEy/42KDtb4lDjBqhlJRI7jzRAxe0lWk/1M4M8 wyygKbF+lz5EWFZi6ql1TBB7+SR6fz+BauWV2DEPxlaTmPNuKguELSOx8NIMJkhYe0j0XOCE hNVERomr+7tZJzAqzELyziwk78xC2LyAkXkVo2RqQXFuemqyaYFhXmo5PJKT83M3MYITrZbL DsYb8//pHWJk4mA8xCjBwawkwutXHJ8mxJuSWFmVWpQfX1Sak1p8iNEUGN4TmaVEk/OBqT6v JN7QxNLAxMzMzMTS2MxQSZz3devcFCGB9MSS1OzU1ILUIpg+Jg5OqQam/iBNDecen7n/0q/t Z+z6KOw0e8qTDZFH/+/4v3PT/U956le5eJWOf9COzmhODzhxd5WFenr/knMBlR9EHghlzqix O5tSrvPKan6/1hbVK0lHLR47FUzd/XwyQ/HiB8e6Jr3g/vRxqvQq+do5kw/urpy+1+muvk31 CZuTNx4cWvCx0vSWwqFj5X+0NGbued+4cKH97Oof3m89Vy2tcjQ+fuPruo02Tie4J3scPd25 ZAXrv5BjPW3zoqUbfz3bGpd9uzd65sebEzP++d7vTPd09Q6NP1EvHH/tvok+x6lb9cwx85af YOn526TUFp3DruV3KPK7e4V8yfmwD/Y5pd+FhM6es9dUav7ikDUpKvzRQSWW4oxEQy3mouJE AFPxxtc9BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrLLMWRmVeSWpSXmKPExsWy7bCSvO5Kg4Q0g2vnxCyaJvxltlh9t5/N 4vv2PhaLmwd2MlmsXH2UyeLo/7dsFpMOXWO02PrlK6vF3lvaFjcmPGW02PZ7PrPF7x9z2Bx4 PDav0PK4fLbUY9OqTjaP3Tcb2Dz6tqxi9Pi8SS6ALYrLJiU1J7MstUjfLoEro3X7BdaCRcoV X9/2MjUwHpDuYuTkkBAwkbj7qJe1i5GLQ0hgO6PE/LVXGSESMhK77+5khbCFJVb+e84OYgsJ fGSU+P8jvouRg4NNQFfiR1MoSFhEwFdiwYbnjCBzmAWuM0rcmL6VGSQhLBAs8WtTNxOIzSKg KjHnxzY2EJtXwFZiVfshJoj58hIzL30Hm88pYCfxvbWHFWKXrcSuHVtZIOoFJU7OfAJmMwPV N2+dzTyBUWAWktQsJKkFjEyrGEVTC4pz03OTCwz1ihNzi0vz0vWS83M3MYJjQCtoB+Oy9X/1 DjEycTAeYpTgYFYS4fUrjk8T4k1JrKxKLcqPLyrNSS0+xCjNwaIkzquc05kiJJCeWJKanZpa kFoEk2Xi4JRqYHLOclXyfObptuFEGdPNYrMqKf1Z3UK885lueb31WRj3aMmsz3Jmlu4/M4xr 55x3tfxq+Otu+PQf8w4Erqn/ZftZM0pgSlvL7KzL147uVcg1aCiya1GKvjVHW6s1+PrEtQp1 nCc1V/s7fu+KuvNj7q6G3ba6r88bPH4aoPl8s910s885t3UehDzlUVBSELXoVS9Wl/1nMDFX ymqTsZpVyn8DlSVrNy4QCdUR9t9+JW7m4ygxb489J1/EnlDfUf36XHCz4qGf1Vy99kEnJi9w SRUr7Fll/1Nby+bCgrQMhZoX8/8nO53jTQ5doHNYcnVvzy3hF1nfty6ZUuKzcs7M0ocVvs4L RPlW5XocYX22TYmlOCPRUIu5qDgRAG5j387wAgAA X-CMS-MailID: 20240605093225epcas5p335664b9185c99a8fe1d661227d3f4f1a X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240605093225epcas5p335664b9185c99a8fe1d661227d3f4f1a References: <20240605092455.20435-1-kundan.kumar@samsung.com> Add a bigger size from folio to bio and skip merge processing for pages. Fetch the offset of page within a folio. Depending on the size of folio and folio_offset, fetch a larger length. This length may consist of multiple contiguous pages if folio is multiorder. Using the length calculate number of pages which will be added to bio and increment the loop counter to skip those pages. There is also a check to see if pages are contiguous and belong to same folio, this is done as a COW may happen and change contiguous mapping of pages of folio. This technique helps to avoid overhead of merging pages which belong to same large order folio. Also folio-lize the functions bio_iov_add_page() and bio_iov_add_zone_append_page() Signed-off-by: Kundan Kumar --- block/bio.c | 62 +++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 48 insertions(+), 14 deletions(-) diff --git a/block/bio.c b/block/bio.c index e9e809a63c59..7857b9ca5957 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1204,7 +1204,7 @@ void bio_iov_bvec_set(struct bio *bio, struct iov_iter *iter) bio_set_flag(bio, BIO_CLONED); } -static int bio_iov_add_page(struct bio *bio, struct page *page, +static int bio_iov_add_folio(struct bio *bio, struct folio *folio, unsigned int len, unsigned int offset) { bool same_page = false; @@ -1214,27 +1214,27 @@ static int bio_iov_add_page(struct bio *bio, struct page *page, if (bio->bi_vcnt > 0 && bvec_try_merge_page(&bio->bi_io_vec[bio->bi_vcnt - 1], - page, len, offset, &same_page)) { + &folio->page, len, offset, &same_page)) { bio->bi_iter.bi_size += len; if (same_page) - bio_release_page(bio, page); + bio_release_page(bio, &folio->page); return 0; } - __bio_add_page(bio, page, len, offset); + bio_add_folio_nofail(bio, folio, len, offset); return 0; } -static int bio_iov_add_zone_append_page(struct bio *bio, struct page *page, +static int bio_iov_add_zone_append_folio(struct bio *bio, struct folio *folio, unsigned int len, unsigned int offset) { struct request_queue *q = bdev_get_queue(bio->bi_bdev); bool same_page = false; - if (bio_add_hw_page(q, bio, page, len, offset, + if (bio_add_hw_page(q, bio, &folio->page, len, offset, queue_max_zone_append_sectors(q), &same_page) != len) return -EINVAL; if (same_page) - bio_release_page(bio, page); + bio_release_page(bio, &folio->page); return 0; } @@ -1258,9 +1258,9 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) struct bio_vec *bv = bio->bi_io_vec + bio->bi_vcnt; struct page **pages = (struct page **)bv; ssize_t size, left; - unsigned len, i = 0; - size_t offset; - int ret = 0; + unsigned int len, i = 0, j; + size_t offset, folio_offset; + int ret = 0, num_pages; /* * Move page array up in the allocated memory for the bio vecs as far as @@ -1301,15 +1301,49 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) for (left = size, i = 0; left > 0; left -= len, i++) { struct page *page = pages[i]; + struct folio *folio = page_folio(page); + + /* Calculate the offset of page in folio */ + folio_offset = (folio_page_idx(folio, page) << PAGE_SHIFT) + + offset; + + len = min_t(size_t, (folio_size(folio) - folio_offset), left); + + num_pages = DIV_ROUND_UP(offset + len, PAGE_SIZE); + + if (num_pages > 1) { + ssize_t bytes = left; + size_t contig_sz = min_t(size_t, PAGE_SIZE - offset, + bytes); + + /* + * Check if pages are contiguous and belong to the + * same folio. + */ + for (j = i + 1; j < i + num_pages; j++) { + size_t next = min_t(size_t, PAGE_SIZE, bytes); + + if (page_folio(pages[j]) != folio || + pages[j] != pages[j - 1] + 1) { + break; + } + contig_sz += next; + bytes -= next; + } + num_pages = j - i; + len = contig_sz; + } - len = min_t(size_t, PAGE_SIZE - offset, left); if (bio_op(bio) == REQ_OP_ZONE_APPEND) { - ret = bio_iov_add_zone_append_page(bio, page, len, - offset); + ret = bio_iov_add_zone_append_folio(bio, folio, len, + folio_offset); if (ret) break; } else - bio_iov_add_page(bio, page, len, offset); + bio_iov_add_folio(bio, folio, len, folio_offset); + + /* Skip the pages which got added */ + i = i + (num_pages - 1); offset = 0; } From patchwork Wed Jun 5 09:24:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kundan Kumar X-Patchwork-Id: 13686527 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9708718FDAD for ; Wed, 5 Jun 2024 09:47:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.33 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717580854; cv=none; b=Ox+mKEG6lopT9xfR9MIqV4hSl/v6PsLTWn4ulinG0P1uT0DbQIgqxwkJAuqh2RBe5zzM6a2ePWfxf4uVfr7EnFvdXpFxFTdj3kaMEfQGbjy0OhsvjzQYt4YpJGd8XktQw/3HHG8+XOYveD59inavZSryVqyk1YtDXSThU7m+92s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717580854; c=relaxed/simple; bh=twWVhE0qODsH13l8jNlnYU6LMTSQAMjcBl9X8fIvn10=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:MIME-Version: Content-Type:References; b=VpRDrPBZNfkWfH6Kx/ViSbAXiguyXAD0PrSY5XaAuIBgtzacaN7WxSSEC3hVF/rU7yqY56Y1sPyfMo3lYT7XRy1qO8qs7R9DkAerRHzdMPQb2P0dEZ0ERrONycG9odVuwG1H5MUFUJ1GP4KEgYt3F59PGljTsEVwPy7Wre8RvNE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=AoiKFC96; arc=none smtp.client-ip=203.254.224.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="AoiKFC96" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20240605094730epoutp03ca7ac9488d436ddee03440bf29df098a~WEq7-McvU1454214542epoutp030 for ; Wed, 5 Jun 2024 09:47:30 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20240605094730epoutp03ca7ac9488d436ddee03440bf29df098a~WEq7-McvU1454214542epoutp030 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1717580850; bh=vEiOcRtnbcv+SVB6C6T2s2ws1aVXj9VFiwsT7ZEL6UQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AoiKFC96jgTnImtRVM8PQA9OddAAdvNtlNKlnhJoLZp8hI90c4pcIep/7fWsE2/oD rMxOKMe5TrTG6+3s6Z7G9cHaMxPEijV3Hvqqj6QK0xjN9rEiI05qwG2OE6C78ynwmS dGLbwJUt7m07T8s9joH3TpasBjt72cYSlqrxMDa4= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20240605094730epcas5p1fa8ffcb50c36f69bdab51e7e6d383b1a~WEq7oqPOm0305703057epcas5p1E; Wed, 5 Jun 2024 09:47:30 +0000 (GMT) Received: from epsmgec5p1new.samsung.com (unknown [182.195.38.178]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4VvN0w3bw2z4x9Q2; Wed, 5 Jun 2024 09:47:28 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmgec5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 20.25.08853.03430666; Wed, 5 Jun 2024 18:47:28 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20240605093234epcas5p151b3b39b2f2e1567c6e4ceae6130c6b9~WEd49sY1W0833508335epcas5p1w; Wed, 5 Jun 2024 09:32:34 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240605093234epsmtrp2d559341e5b8690ae545f91b7432b48ec~WEd48gCJH0105501055epsmtrp25; Wed, 5 Jun 2024 09:32:34 +0000 (GMT) X-AuditID: b6c32a44-d67ff70000002295-32-66603430f1bf Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 44.23.08336.1B030666; Wed, 5 Jun 2024 18:32:33 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240605093232epsmtip2258d7d272d666f567578917b46da1492~WEd3ODD-G3189331893epsmtip2s; Wed, 5 Jun 2024 09:32:32 +0000 (GMT) From: Kundan Kumar To: axboe@kernel.dk, hch@lst.de, willy@infradead.org, kbusch@kernel.org Cc: linux-block@vger.kernel.org, joshi.k@samsung.com, mcgrof@kernel.org, anuj20.g@samsung.com, nj.shetty@samsung.com, c.gameti@samsung.com, gost.dev@samsung.com, Kundan Kumar Subject: [PATCH v4 2/2] block: unpin user pages belonging to a folio Date: Wed, 5 Jun 2024 14:54:55 +0530 Message-Id: <20240605092455.20435-3-kundan.kumar@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240605092455.20435-1-kundan.kumar@samsung.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrJJsWRmVeSWpSXmKPExsWy7bCmhq6BSUKawZ014hZNE/4yW6y+289m 8X17H4vFzQM7mSxWrj7KZHH0/1s2i0mHrjFabP3yldVi7y1tixsTnjJabPs9n9ni9485bA48 HptXaHlcPlvqsWlVJ5vH7psNbB59W1YxenzeJBfAFpVtk5GamJJapJCal5yfkpmXbqvkHRzv HG9qZmCoa2hpYa6kkJeYm2qr5OIToOuWmQN0oZJCWWJOKVAoILG4WEnfzqYov7QkVSEjv7jE Vim1ICWnwKRArzgxt7g0L10vL7XEytDAwMgUqDAhO2NL42KWgsXcFZdOPGNqYPzP0cXIySEh YCIx+0Q3G4gtJLCbUaLpT3EXIxeQ/YlRonPyMVYI5xujxOZjH5m7GDnAOqbeqoFo2Mso0ftX AaLmM6PEjNvLWUBq2AR0JX40hYLUiAi4S0x9+YgRpIZZ4CyjxImpj1hAEsICLhI/f/xlArFZ BFQl7mw+xQhi8wrYSix8cJEF4jp5iZmXvrOD2JwCdhLfW3tYIWoEJU7OfAJWwwxU07x1NjPI AgmBqRwSTTOeMEE0u0gc2NDCDmELS7w6vgXKlpL4/G4vG4SdLXGocQNUfYnEziMNUDX2Eq2n +sEeZhbQlFi/Sx8iLCsx9dQ6Joi9fBK9v2FW8UrsmAdjq0nMeTcV6n4ZiYWXZkDFPSTOfDwO DdCJjBIvPq9inMCoMAvJP7OQ/DMLYfUCRuZVjJKpBcW56anJpgWGeanl8DhOzs/dxAhOs1ou OxhvzP+nd4iRiYPxEKMEB7OSCK9fcXyaEG9KYmVValF+fFFpTmrxIUZTYIBPZJYSTc4HJvq8 knhDE0sDEzMzMxNLYzNDJXHe161zU4QE0hNLUrNTUwtSi2D6mDg4pRqY6sQdJT4YJ1f9epYy 9+L5hISfAdV19VL1gjPYNoTrNgiL/PwZkX/xwKv68szp97j3vL28WHHZwtDFqzPn3i+pCeq9 9rp03iS+O91Lft+beP9Z3ln7m9u2Gk+YWLRz0kKOPxaf5JYfbWcw47ly17nEmI0vdnrnEcE5 HTNUlyyxPtoaHrK4vCKIxalHKm/F8v/ft7c6MyZ+tZpVa/h7+5wfd01ilnz8rDm1Y3ls8Ksz qeekF5wPPNTx6vCS90ePnIlde1PlqhnndvFwLQfBtNxr4R8EIngFJsks1bsRsPWI8eRPPWtS j8z89cz8+Kxg9a69N35fvDMl8+va5mCbpX4z3e5Nz/l2Rvp6ufWP/mXPT+5TYinOSDTUYi4q TgQASGWUmzwEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrALMWRmVeSWpSXmKPExsWy7bCSvO4mg4Q0gyvOFk0T/jJbrL7bz2bx fXsfi8XNAzuZLFauPspkcfT/WzaLSYeuMVps/fKV1WLvLW2LGxOeMlps+z2f2eL3jzlsDjwe m1doeVw+W+qxaVUnm8fumw1sHn1bVjF6fN4kF8AWxWWTkpqTWZZapG+XwJWxpXExS8Fi7opL J54xNTD+5+hi5OCQEDCRmHqrpouRi0NIYDejxN2XnaxdjJxAcRmJ3Xd3QtnCEiv/PWeHKPrI KHHu4DZmkGY2AV2JH02hIDUiAr4SCzY8ZwSpYRa4zihxY/pWZpCEsICLxM8ff5lAbBYBVYk7 m08xgti8ArYSCx9cZIFYIC8x89J3dhCbU8BO4ntrD9hiIaCaXTu2skDUC0qcnPkEzGYGqm/e Opt5AqPALCSpWUhSCxiZVjFKphYU56bnFhsWGOallusVJ+YWl+al6yXn525iBEeCluYOxu2r PugdYmTiYDzEKMHBrCTC61ccnybEm5JYWZValB9fVJqTWnyIUZqDRUmcV/xFb4qQQHpiSWp2 ampBahFMlomDU6qBKTf2ys15+4vUdD9PuZddNfPQ9S37ExgCq7vTmCUqZTmXT963ascVowK7 5jSHhzZa5s/4rp3Zfvbb0RXn716dyNnQ9b6ssM6kevn1dyuyohZ0+xye0Jpkf5VPT8LpcQmz yxyNLBerXA7nirUru23uHpl/qOCKVe8yC9s9y3tuvDXNv8wvcviKw6bsb3yXd6RtYp4irPu8 u7zUyvbIu7lMnpv6s3+fzLwbc9SvV/Ki9aQ390KyzHKEIgtu/s3nu1AkOn3xBu7leWcjdViK /t+dqPP3kcC1nje7ZywyPDLTqD1hysSrN2bd1Xv/QPdhvE2Sy13jK6mRdgqh85YkaD688HbL pqaYpHv7v4qXnZy6sFCJpTgj0VCLuag4EQBap/ns8wIAAA== X-CMS-MailID: 20240605093234epcas5p151b3b39b2f2e1567c6e4ceae6130c6b9 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240605093234epcas5p151b3b39b2f2e1567c6e4ceae6130c6b9 References: <20240605092455.20435-1-kundan.kumar@samsung.com> Unpin pages which belong to same folio. This enables us to release folios on I/O completion rather than looping through pages. Suggested-by: Keith Busch Signed-off-by: Kundan Kumar --- block/bio.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/block/bio.c b/block/bio.c index 7857b9ca5957..28418170a14a 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1166,20 +1166,12 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty) struct folio_iter fi; bio_for_each_folio_all(fi, bio) { - struct page *page; - size_t nr_pages; - if (mark_dirty) { folio_lock(fi.folio); folio_mark_dirty(fi.folio); folio_unlock(fi.folio); } - page = folio_page(fi.folio, fi.offset / PAGE_SIZE); - nr_pages = (fi.offset + fi.length - 1) / PAGE_SIZE - - fi.offset / PAGE_SIZE + 1; - do { - bio_release_page(bio, page++); - } while (--nr_pages != 0); + bio_release_page(bio, &fi.folio->page); } } EXPORT_SYMBOL_GPL(__bio_release_pages); @@ -1342,6 +1334,9 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) } else bio_iov_add_folio(bio, folio, len, folio_offset); + if (bio_flagged(bio, BIO_PAGE_PINNED) && num_pages > 1) + unpin_user_pages(pages + i, num_pages - 1); + /* Skip the pages which got added */ i = i + (num_pages - 1);