From patchwork Mon Nov 11 23:37:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7432D3ABFA for ; Mon, 11 Nov 2024 23:49:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75D336B00DF; Mon, 11 Nov 2024 18:49:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 70CBA6B00E0; Mon, 11 Nov 2024 18:49:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 586196B00E1; Mon, 11 Nov 2024 18:49:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 381CA6B00DF for ; Mon, 11 Nov 2024 18:49:05 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B29D51201E4 for ; Mon, 11 Nov 2024 23:49:04 +0000 (UTC) X-FDA: 82775454552.17.3997E4F Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf24.hostedemail.com (Postfix) with ESMTP id 73D4B180012 for ; Mon, 11 Nov 2024 23:48:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=JGuNvISr; spf=pass (imf24.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.169 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368710; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=yj/XblxlQdaXaZ0jZg4tRkPxXWjojAX8j8kMc07EMmqrNlf+OEiih8yg8gm8lN9NOPnlQq GwDp8+ZHJvz7pSTQNXJOe46fslqWwcDS08mAw0bNRLzPq2fCUo0bFOESXqy8DfLiLeW/qA uqKWpzaEaEXjjyoo0f+q7iB+yL8Dvus= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=JGuNvISr; spf=pass (imf24.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.169 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368710; a=rsa-sha256; cv=none; b=VKF+hmWOxtj8eTBCLqIm5dkkn99Nvt7vfHAp7wB/gTCuebtv2C4FmqBYRpWxveg837OmSQ pn/2SfERMwUKW0IspD2xmDCozUuvwKK9HlHNuR3tw4u5Vc0kPvnf7UmAOkZVa/bEi8263Y HGESiTuE3yNOoB60gMxwYObhob4RLiA= Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-71e4244fdc6so4274080b3a.0 for ; Mon, 11 Nov 2024 15:49:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368941; x=1731973741; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=JGuNvISrMLZ+twvZFUtJi896hJhSMnNRR/9/0ZE6niNUAqfQsD7w6JdW1u+4kRCDQ8 61acoUnnp9kNaMT+vGeVraNn+JbN4nazHaBqJotsgIm/h5gI1XmFo1qENxgm1Sxh1xiT KawbYJfdy8BS3vuMJRXEEnrWX4Di+mL926TMLyBrXbj+vP+rkovBiWYZiDvDpqmdAkRQ fimq/Vy2v6Q9VkNvlcLMTVzwJgfamfATPjpl74KKaBef+W+AJfcniCWz9TLA+jNsH8Pm 0QueHRR3Vg3dixLQi1l/ID8XO98TtJNXDmqyqgG1qZ/50MFycmB+Jeo3BEAkDnLYgv29 v8mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368941; x=1731973741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=hM50ECGcamv2ij9iR2ADxsYEQitSo9q0dsmdf+Ct98FNVVz23XKT+QfBY7jQF5Qcz9 tnjXqedBgUSdEFQ6HEAzkfxeyWxxyvSjr4e3fRJkSPmeg+i9tynzbyJBrtmeeoXFotxQ awtuPpmJebfCF9oK9UXEy5AaK5kyx7FiGcvYmgfMwc8cDJ2ypU05vp5dOeUIFC3umQs9 LgilfJCGDvAyTTtVnDufxbqlrWC6a0TByiEGnDYbnI+ZAWZFUikTKVtD+KkvHLOBHviI IH4ITXMNXNaN/2GYzkHqzGExLu0Ayy0i1vB54I6+CXc4cQp4KtBt+vMXlrvr8agVn7TU MyFw== X-Gm-Message-State: AOJu0Yw6USS3y10BbtAjorDCAQuC3dq6zlsoRrmdpRMsjJIuZ1u5vM4J zJrrHS2mXF369DwiDKX4CI9I+1Hsi/XOq1LikkKhenASM7hRCC/M4z1eeO7OAF422KdNpzbX7ev bSEU= X-Google-Smtp-Source: AGHT+IHVAuYVOMfDSGlPxbnuC87wShBLb7OXW58ajqvAinndw3wlptYvFjbRCeInYbnH7Qo2aVFY4A== X-Received: by 2002:a05:6a21:9101:b0:1db:e3c7:9974 with SMTP id adf61e73a8af0-1dc228ef1damr21176002637.12.1731368941302; Mon, 11 Nov 2024 15:49:01 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:00 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 09/16] mm/filemap: drop uncached pages when writeback completes Date: Mon, 11 Nov 2024 16:37:36 -0700 Message-ID: <20241111234842.2024180-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 73D4B180012 X-Stat-Signature: dsg8i5z19h8cc4czge4s96chhpebabf7 X-Rspam-User: X-HE-Tag: 1731368937-444204 X-HE-Meta: U2FsdGVkX18PTAkTt3IBCvz72OmfSL3uS1pF7QqCnODjTXNiTKw+i7hOy1kx6HUyQO+yVxJq9JFyOP+hHa5UF3AXiu7FP3viy7WbidkqMDInbGQFxfX0H2D43qFEbrZJ5xWYVnsfyA4U84RadzIUOixl9Dtbhx+jNS7e9CvSkp5yxDEGivoWmhg83QF8uvriCAKvqXuBrkvHnQvfySAqndfGjneR7cS7ohGGhDydAxi/J++kMOZXF4HOMk90st+LwCrek8qcZBsth7ZCLHbSX6MTzmWwx+UhLXwRVhVWpu6QF1FlpBKANvtGPo+iNgiAmkhummnEZnxEoKD0C55nDIUarbNRLlPuDbJjdceZsGlWL49Yi+5O+P604iU+g0yH323LfmFjC8mGlpIwA4cJmajFMG1xAxvzaHffuW8+Z6tIJvHu8IfbJzsUBKPPZFkKexcvr1nbhf6qvbQMVbdk+EWZtLfUknv6SXlw+SK31WhhEXr+Np8GlxePKVnj6CrmVAJe2PRwtm8UAdRs9wR9HZdfgZQfDRbheqKoB3zeR3KSzti9M+oyqt23hBLpYOYhR7ZwBevrRlZbrxp5Okhuo/UpKLIoW1ksC0eElQqqSeEzoXy0uid4txE1ZcnmTXZIY1L7e8ezE6thXtvIVVGA4RlA5yLsz+mxdMrt8AS8Ug2EL02fEBvhTj6kTZCkUXb4mylm4qc5BP0Js0lEDnIKMauPMry9dS7ymJ7T9fDi8eUNRhLwANypLMh+TbsU2vgyTskkN+8ufbaewgc/FXjAEHewqlgilbs2atvWIyWQlhNYfbtweCoHLH7nZOdQ+PfZ0+UElZwnwL97M846FdcUkVTrFE1pvWw7h7ibhjGbWJpEcY996upR0FUFu6sfPn/OS/zIjtgHbF9UzoD1O7NjaHkh3V7LMakLPQ7vInRJ1JTNXAVOjvAjOP5XTE+7C0tELG8V0Li8aFSF9jgRrnr S4/na0uA f7ebPitL16E9a1YYyaYt1vTGLbOsQZOdV928LVUAgzxILUdny8fPC4wx6qombPTS7t9LK/rzawoPP+OaWdSGnWD4UQKj0LGsTHpDNKk+Mg/UFz0j6q4wpj/6igo32KrmJjYBkep6kXjyyVj/Y9XzRd9fTewwc114kMnUxa/pzsaX25fziKkw7Wf5G4EIlzA9rcaGlsiRjnFDswQ2bFYxUPeDL9/3yw8CbJUMzHSZo5qyGST/XkekhQ8RJdZHvMB4wAvk+k5t219DbZzjwiXVGljPkoI9css4Di2qAylR3bxl2YPHdemCQkeMRkg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the folio is marked as uncached, drop pages when writeback completes. Intended to be used with RWF_UNCACHED, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 3d0614ea5f59..40debe742abe 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1600,6 +1600,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as uncached, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset uncached for that case and latter completions should invalidate. + */ +static void folio_end_uncached(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_UNCACHED writeback, but + * can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1610,6 +1631,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_uncached = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1631,9 +1654,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (folio_test_uncached(folio) && folio_test_clear_uncached(folio)) + folio_uncached = true; if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_uncached) + folio_end_uncached(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback);