From patchwork Tue Jun 25 10:18:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kara X-Patchwork-Id: 13710842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 159C9C2BBCA for ; Tue, 25 Jun 2024 10:19:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B80CB6B01F3; Tue, 25 Jun 2024 06:19:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB9806B01FE; Tue, 25 Jun 2024 06:19:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49DE06B01F3; Tue, 25 Jun 2024 06:19:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EA1B06B02F6 for ; Tue, 25 Jun 2024 06:19:14 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A53FC1A1939 for ; Tue, 25 Jun 2024 10:19:14 +0000 (UTC) X-FDA: 82269013428.14.F7BD43E Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf05.hostedemail.com (Postfix) with ESMTP id 4B4FF100017 for ; Tue, 25 Jun 2024 10:19:12 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=g6XL5W2b; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=vdQLeaCI; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=g6XL5W2b; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=vdQLeaCI; spf=pass (imf05.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719310738; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+QN4nEN9WKE2t9Mr+WNlqEXF1yaaR/IE18O8y0vBXQ8=; b=uQonRdIf/W4WYCmcoXDQP0XHLeaAv+8Cv93eIDAJtk5SI+84/ajOku2T4dgG5Op1NpOCRt g7poMMM0JqGEQcwiPB47gX8XNYRQR9l6/2wEzg4lG8E1mP3JHF2FHW/W7GPM2MyFfy6Q9a cdTR+7GD5mpRmzzR0jp1OEXtjoQOurA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719310738; a=rsa-sha256; cv=none; b=WV9O9BTCL3YjrYCcf8kR0O5zwpxotpUducXjaWtAh6QEzSV7ngTLiQRMfkFfGnkW/M23Ez LTDdkfLoqKlgNXfReW9wBGjaCcT3QlfdtnaL2XEu+WdjdSwwIiP5r4XJku64CNDhelE5gc WCVBXabkBvhOxNGdVc6aBRBvQ7/RrvU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=g6XL5W2b; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=vdQLeaCI; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=g6XL5W2b; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=vdQLeaCI; spf=pass (imf05.hostedemail.com: domain of jack@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AD8471F851; Tue, 25 Jun 2024 10:19:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1719310750; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+QN4nEN9WKE2t9Mr+WNlqEXF1yaaR/IE18O8y0vBXQ8=; b=g6XL5W2bx1lBimk/YVoIuBtM965nhwDCu2xk9Onv41jmFcbHqn9KuvihvDSSaLuHEowOJ2 /UN5aJGYMF4LA2+46i33DojSXDUa6jeXRssH/ht/bvel2vGoHwJD0U5xKGqdCY+Flcwreh S7zRmJNxULCDmjwdOhB7PKc68UFbZlQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1719310750; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+QN4nEN9WKE2t9Mr+WNlqEXF1yaaR/IE18O8y0vBXQ8=; b=vdQLeaCImos+Rr80rKGiS+1VVpK2XFb71QhEaeaFB/E7YvP4OYXu0EjLI7e+0o5rsVUNQh s0hkst8/4/1bojBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1719310750; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+QN4nEN9WKE2t9Mr+WNlqEXF1yaaR/IE18O8y0vBXQ8=; b=g6XL5W2bx1lBimk/YVoIuBtM965nhwDCu2xk9Onv41jmFcbHqn9KuvihvDSSaLuHEowOJ2 /UN5aJGYMF4LA2+46i33DojSXDUa6jeXRssH/ht/bvel2vGoHwJD0U5xKGqdCY+Flcwreh S7zRmJNxULCDmjwdOhB7PKc68UFbZlQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1719310750; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+QN4nEN9WKE2t9Mr+WNlqEXF1yaaR/IE18O8y0vBXQ8=; b=vdQLeaCImos+Rr80rKGiS+1VVpK2XFb71QhEaeaFB/E7YvP4OYXu0EjLI7e+0o5rsVUNQh s0hkst8/4/1bojBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8BCAA13AD9; Tue, 25 Jun 2024 10:19:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id zTIcIp6ZemaDWQAAD6G6ig (envelope-from ); Tue, 25 Jun 2024 10:19:10 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 0AA73A0958; Tue, 25 Jun 2024 12:19:10 +0200 (CEST) From: Jan Kara To: Cc: Andrew Morton , Matthew Wilcox , , Jan Kara Subject: [PATCH 08/10] readahead: Disentangle async and sync readahead Date: Tue, 25 Jun 2024 12:18:58 +0200 Message-Id: <20240625101909.12234-8-jack@suse.cz> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20240625100859.15507-1-jack@suse.cz> References: <20240625100859.15507-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7472; i=jack@suse.cz; h=from:subject; bh=kzZAggEGB8nxww3IMybdnW/wrXAOWD0Xm8Cb8kJ8pv4=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBmepmSvtCyAHd7WRo5r/OLGFdaWl9D2kCzIBgqLdiE FKlKbk2JATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCZnqZkgAKCRCcnaoHP2RA2eLXB/ 0dO6bkv2zDY47lCTrKzJepiIL+qGydhiLaAQjXHcOGbE4kM8EnU8XQLHG5D+SfUNxo/nBCcqGAnyHC Jyi3YPBdfzpZuxcK7Yp8LckjSCPEe++CiPP5kJ8RSkAy1sf7srpMB7bezdySY1DTPVcuAKPW3UZzek gHUq0IyvUhMercFXtAd1A8VQdDi8z43l06WseCBgtK7Pw/kMFHFPHGdgedUfgsCwYaP7uzkAw/yBs2 cURzHIg5Q7/6xzNEAPoH6t7/E/3VAdnQZrJwaI9/VJzcby2Ujo6GCNyAslXUyeAZXyqlyXXdzUgVtI UT04cbN4zUManqKe7GMOhZQ5vimZDQ X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C X-Rspamd-Queue-Id: 4B4FF100017 X-Stat-Signature: xqp1g3dsrzd7yggyp6p97tsdek7ncj6q X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1719310752-910358 X-HE-Meta: U2FsdGVkX18S2R3Oy7OofOIMKdqHY8mg/7EYRanRkelvVYkm5WKCht5IbE6rRjafxhoh4I3bJrb513e/laxBEpmMfMnWnFt2PaiUqQHzfyt8UZTrdp0ZFLHVG/1+0SzhhbSE4pFfaBQYwmrhbYiYGkBa35AwOqt96b/t//EXgCkCw9yzztd5GhSMv0yQnexp7Ikfg/36L93uFhkx+HxQv9M4EYUmXRQ8Pqfxy8gZJrDJeHyPA1f1MsT1+04/JfCuNKYanxUc50DUCkjXoYQ4BlNYVIFxptiPG3fLSxDH+Rzqg+XVZsB510NoYqKFC7900BjBNTdSdn6sf2u71tFdP8hws31tAlVDiSzp2Z8bg0ifyl28KVNh7QIo9VseoIQTbvR5wh6p+iKbFeTZdI0mnL0m4YONr2GHXlA3T6JEHAHyRDT8Grkhdbzw2oHcOJFtzJkA18tKc2VQMLzc9hiLxdTyNtOhFF3rMuZqa3eEgOCskSumj6HqRoxbxOYAOOjm5Y8IgrC22je6GnKhdhaW88sixlhk3m3OmIgXtaPdJmOnaAp+HOA0seMDAHsL+9K05jANiT2C7O6471lTPBYm8QkvqmiywilBPqURD1bC/LBVa5yqmc5YFwqdWQFvhZ+o77uY8vpBB1aUTTWPQD9FrtggEzA4PLGddOkqNtXc9dLOHMLi3Ga4pz80K3zPhdqSaWbJAY/gKLkUYkCRpCEpEnbOubsbTGcSoqWCtZj2lHga/VtZ3Jz3MG2l0Hv1QpfHkr1NrqO9ndpUw/I5yUpxRH5zyt4xppIzaKQLK3MjHOINISF7KWzTugBthTUgRA564TSBXr3e+VYDzDp6YxPNdxU0gG1xnS940qgqONSwHe59jJsyse38VPmrbTMU0DnD1Xin4htBSGw0pn40axbbHORXhii9NtpwIEdUeFz3bwzgBa3ibNrB68vynFWgpKkbo5XxGW5pAILJIRIpdEq H+U+O+xf tPNPJWFEbHu20VXpO6QnUnJI0CfBHXfxCjWaOAGMq6O/anYLGjyw0tb5aS6lSU/ClSvIw/E9xneT8I/st+BQX9wBfEAueSi3KsBNg1pphj/709WNFSK5O5IW8OAazH3zu46c6eOvKw8kCzqI3xhXxQE9Q0UFDyYzmTt4TvgmDAdtKxlnkHXXDtYfOHMyU2isEpF24mf8QKfS7s3Zfmtb+8aNtJQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both async and sync readahead are handled by ondemand_readahead() function. However there isn't actually much in common. Just move async related parts into page_cache_ra_async() and sync related parts to page_cache_ra_sync(). No functional changes. Signed-off-by: Jan Kara --- mm/readahead.c | 162 +++++++++++++++++++++++-------------------------- 1 file changed, 77 insertions(+), 85 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index d92a5e8d89c4..a44daa12ebd2 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -540,18 +540,11 @@ void page_cache_ra_order(struct readahead_control *ractl, do_page_cache_ra(ractl, ra->size - (index - start), ra->async_size); } -/* - * A minimal readahead algorithm for trivial sequential/random reads. - */ -static void ondemand_readahead(struct readahead_control *ractl, - struct folio *folio, unsigned long req_size) +static unsigned long ractl_max_pages(struct readahead_control *ractl, + unsigned long req_size) { struct backing_dev_info *bdi = inode_to_bdi(ractl->mapping->host); - struct file_ra_state *ra = ractl->ra; - unsigned long max_pages = ra->ra_pages; - pgoff_t index = readahead_index(ractl); - pgoff_t expected, prev_index; - unsigned int order = folio ? folio_order(folio) : 0; + unsigned long max_pages = ractl->ra->ra_pages; /* * If the request exceeds the readahead window, allow the read to @@ -559,55 +552,42 @@ static void ondemand_readahead(struct readahead_control *ractl, */ if (req_size > max_pages && bdi->io_pages > max_pages) max_pages = min(req_size, bdi->io_pages); + return max_pages; +} - /* - * start of file - */ - if (!index) - goto initial_readahead; - - /* - * It's the expected callback index, assume sequential access. - * Ramp up sizes, and push forward the readahead window. - */ - expected = round_down(ra->start + ra->size - ra->async_size, - 1UL << order); - if (folio && index == expected) { - ra->start += ra->size; - ra->size = get_next_ra_size(ra, max_pages); - ra->async_size = ra->size; - goto readit; - } +void page_cache_sync_ra(struct readahead_control *ractl, + unsigned long req_count) +{ + pgoff_t index = readahead_index(ractl); + bool do_forced_ra = ractl->file && (ractl->file->f_mode & FMODE_RANDOM); + struct file_ra_state *ra = ractl->ra; + unsigned long max_pages; + pgoff_t prev_index; /* - * Hit a marked folio without valid readahead state. - * E.g. interleaved reads. - * Query the pagecache for async_size, which normally equals to - * readahead size. Ramp it up and use it as the new readahead size. + * Even if readahead is disabled, issue this request as readahead + * as we'll need it to satisfy the requested range. The forced + * readahead will do the right thing and limit the read to just the + * requested range, which we'll set to 1 page for this case. */ - if (folio) { - pgoff_t start; - - rcu_read_lock(); - start = page_cache_next_miss(ractl->mapping, index + 1, - max_pages); - rcu_read_unlock(); - - if (!start || start - index > max_pages) + if (!ra->ra_pages || blk_cgroup_congested()) { + if (!ractl->file) return; + req_count = 1; + do_forced_ra = true; + } - ra->start = start; - ra->size = start - index; /* old async_size */ - ra->size += req_size; - ra->size = get_next_ra_size(ra, max_pages); - ra->async_size = ra->size; - goto readit; + /* be dumb */ + if (do_forced_ra) { + force_page_cache_ra(ractl, req_count); + return; } + max_pages = ractl_max_pages(ractl, req_count); /* - * oversize read + * start of file or oversized read */ - if (req_size > max_pages) + if (!index || req_count > max_pages) goto initial_readahead; /* @@ -623,7 +603,7 @@ static void ondemand_readahead(struct readahead_control *ractl, * Query the page cache and look for the traces(cached history pages) * that a sequential stream would leave behind. */ - if (try_context_readahead(ractl->mapping, ra, index, req_size, + if (try_context_readahead(ractl->mapping, ra, index, req_count, max_pages)) goto readit; @@ -631,53 +611,31 @@ static void ondemand_readahead(struct readahead_control *ractl, * standalone, small random read * Read as is, and do not pollute the readahead state. */ - do_page_cache_ra(ractl, req_size, 0); + do_page_cache_ra(ractl, req_count, 0); return; initial_readahead: ra->start = index; - ra->size = get_init_ra_size(req_size, max_pages); - ra->async_size = ra->size > req_size ? ra->size - req_size : - ra->size >> 1; - + ra->size = get_init_ra_size(req_count, max_pages); + ra->async_size = ra->size > req_count ? ra->size - req_count : + ra->size >> 1; readit: ractl->_index = ra->start; - page_cache_ra_order(ractl, ra, order); -} - -void page_cache_sync_ra(struct readahead_control *ractl, - unsigned long req_count) -{ - bool do_forced_ra = ractl->file && (ractl->file->f_mode & FMODE_RANDOM); - - /* - * Even if readahead is disabled, issue this request as readahead - * as we'll need it to satisfy the requested range. The forced - * readahead will do the right thing and limit the read to just the - * requested range, which we'll set to 1 page for this case. - */ - if (!ractl->ra->ra_pages || blk_cgroup_congested()) { - if (!ractl->file) - return; - req_count = 1; - do_forced_ra = true; - } - - /* be dumb */ - if (do_forced_ra) { - force_page_cache_ra(ractl, req_count); - return; - } - - ondemand_readahead(ractl, NULL, req_count); + page_cache_ra_order(ractl, ra, 0); } EXPORT_SYMBOL_GPL(page_cache_sync_ra); void page_cache_async_ra(struct readahead_control *ractl, struct folio *folio, unsigned long req_count) { + unsigned long max_pages; + struct file_ra_state *ra = ractl->ra; + pgoff_t index = readahead_index(ractl); + pgoff_t expected, start; + unsigned int order = folio_order(folio); + /* no readahead */ - if (!ractl->ra->ra_pages) + if (!ra->ra_pages) return; /* @@ -691,7 +649,41 @@ void page_cache_async_ra(struct readahead_control *ractl, if (blk_cgroup_congested()) return; - ondemand_readahead(ractl, folio, req_count); + max_pages = ractl_max_pages(ractl, req_count); + /* + * It's the expected callback index, assume sequential access. + * Ramp up sizes, and push forward the readahead window. + */ + expected = round_down(ra->start + ra->size - ra->async_size, + 1UL << order); + if (index == expected) { + ra->start += ra->size; + ra->size = get_next_ra_size(ra, max_pages); + ra->async_size = ra->size; + goto readit; + } + + /* + * Hit a marked folio without valid readahead state. + * E.g. interleaved reads. + * Query the pagecache for async_size, which normally equals to + * readahead size. Ramp it up and use it as the new readahead size. + */ + rcu_read_lock(); + start = page_cache_next_miss(ractl->mapping, index + 1, max_pages); + rcu_read_unlock(); + + if (!start || start - index > max_pages) + return; + + ra->start = start; + ra->size = start - index; /* old async_size */ + ra->size += req_count; + ra->size = get_next_ra_size(ra, max_pages); + ra->async_size = ra->size; +readit: + ractl->_index = ra->start; + page_cache_ra_order(ractl, ra, order); } EXPORT_SYMBOL_GPL(page_cache_async_ra);