From patchwork Fri Dec 20 15:47:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916940 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2BB5E77188 for ; Fri, 20 Dec 2024 15:49:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D1166B009B; Fri, 20 Dec 2024 10:48:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 00AA26B009C; Fri, 20 Dec 2024 10:48:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA0B36B009D; Fri, 20 Dec 2024 10:48:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BA0D96B009C for ; Fri, 20 Dec 2024 10:48:50 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4BE9E44B91 for ; Fri, 20 Dec 2024 15:48:50 +0000 (UTC) X-FDA: 82915770000.28.1B1E17E Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) by imf30.hostedemail.com (Postfix) with ESMTP id 8F3D680017 for ; Fri, 20 Dec 2024 15:47:42 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=gD5SDLdE; spf=pass (imf30.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.182 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734709689; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CWUKIERhmBif0xELkyRs9dlqXC97dRJQR9fjviiscOo=; b=LNXgW8YhXQF9NKR1Vq9IXb4U8reoioKr4D/Xui6xJOK3Izk2gKXlGug8y6mWTBpQfms5Ke oWgnQ+Lni0mHO5avfvQwFJmvcQfUgLWzIeuedMNL5ZW6PMEMdLcfpu3OWzirst6CUq75zc 4xXSA+0hQcxY9gH+jhi9CG53ALHi054= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734709689; a=rsa-sha256; cv=none; b=e6BWzRZmYNZdzMAQ7+wvZfyo4HqMXt3zS4t+CMWqAbHUv3t+GxEV7uWDekZ+Hp0N5NIL3i DJmxhkusuT9nLpGtKRnj1Q2jn2lnIFINd1zrPEfa+/t9qmG7EGd+aMpmR/BnwlIRHBAamU fhnlCWxhnw2/wp+8rE5slT1TxXbYZmI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=gD5SDLdE; spf=pass (imf30.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.182 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-il1-f182.google.com with SMTP id e9e14a558f8ab-3a8160382d4so5926695ab.0 for ; Fri, 20 Dec 2024 07:48:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709727; x=1735314527; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CWUKIERhmBif0xELkyRs9dlqXC97dRJQR9fjviiscOo=; b=gD5SDLdEGC75hacCs4Bwt4h/Wug+AxvmDHirqYObVOoVNGZ36pFmntmXpF+HhzqqiB AEQV7Qj5tJXlGIljaBdTS8+RV6sv8M3JVe73hbgS1lkTCVAKLk1F1JJtHM0qqHQqKd7P 8kk/3CRnYYWDapYAfNqohd/A5OXoVjWbjjUNNn6Mz9oB9j5xJe6SzA/nn19NB3ljz0GQ Ov0NxaT1m6cCWzhn23A1lrysXSIRwQANqtRJcKTWzdYq3sRpJ80UDx3vF0daqCW3yeJw KCfNcnWSIeFPb1X7/drKMRhqlk9zX0S8KsXi4nZgVxrKAEynCkUGAMy1HfjjMF7mtQbu jvYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709727; x=1735314527; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CWUKIERhmBif0xELkyRs9dlqXC97dRJQR9fjviiscOo=; b=ALuEB4O6j5U1o4Eg9UnIsBHXxxkdiZIavHI55OE4Y2uYqZB2wSG0vL5nQxAIh5Ne2v Uhgpc0LGhzgvb0cM2yXfhelEDtA9ddTq2vHRxM29DsiSVUqQeBLLcxuUpRr2dR9ticoy g/lOAz6DEoHl+x0hPE2tj0tAvSP/xeWQinYuiFlLxO7PEVHfD1VAu2HCi/CX0IyrLaYY fVtNVVbIrfrlvvt2455RTGEaRfAANjmTTFRn5iSGqWyoBMObKUGufYbZ/JJw3gByRssH 7AerwJIVthnXCukqaAH7K7VwQGNTnUQGn3vNTcmw7jIxXf8Lo1mkx6yGNKCO2ghugOPx Bd5w== X-Gm-Message-State: AOJu0YxFZJ73KPc2vDcyfvcI8/lNGb90i9bXBmsAZ5EAi23hTbSq/yOB XbjkHzUgNaFaTN9Zh68aM1YFkTGNecH03rMaz6LKyzYfIhbmCgi4eJkySKVW44BELut3ocw9JAZ f X-Gm-Gg: ASbGncsZnQyFgISkG9BWBiRsU/dpEUK6VGSLN0PnoOjkVRqP2b2JnDrcFooLfUt1pph rakuHeuaju7wCeDJBGjv0uH+EV/QJ6eY7BObJV6yNdGklgPFie2kwND/P51Y4JZchT4FLW+iHVb T65r6Npb7toggnpDfeM1EAGLCxkpJ6QsrM5FBCJlzyDpHTHS3CMR1ur4V9rOJ/xjDHMUwDAW2+q Tyjqxcq0Y1abHK2qI8dLX9O2pd2Q1LhPEU4n8pjyIvkRcPDqGkGB7Di7mms X-Google-Smtp-Source: AGHT+IGjD+wBfW2QpC5lojTx62UJZa1o0QwQiIlqK6Hx/4wukTE9KhfBEOmSehD69MzvwKGp6r7FTw== X-Received: by 2002:a05:6e02:522:b0:3a7:e0a5:aa98 with SMTP id e9e14a558f8ab-3c2d2d509b6mr25164805ab.13.1734709727201; Fri, 20 Dec 2024 07:48:47 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:46 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 09/12] mm/filemap: drop streaming/uncached pages when writeback completes Date: Fri, 20 Dec 2024 08:47:47 -0700 Message-ID: <20241220154831.1086649-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8F3D680017 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: ennfz7q1z5qn8ox3qbfqthf4a6dwtsrw X-HE-Tag: 1734709662-522190 X-HE-Meta: U2FsdGVkX18b2kcXsodhHtBmEYk8TQwZBAzUvUhfW2D8appr2HdwoPopdnDagC/08IDPt/xaJLhxDzkaz1+PHcWTYE8Ty78y3+6dotAhUPRt+lfTvcWZncVWj4yCVKxoucMY4088WoYHU+R6iq8HvHI5gl/NZwisQtgsqy7hnLliUvQVPUZvtI9iBhzN4Af3RYSGKRExe1Mt3Ub7BoZx5rP/ukq8ozX2B6i6a8lxznIzEHP4sBOT2vwoSpe55ZIzidwRWrRt61+0MT3cXf2Zc4/l1YN+k5vO9tbBZs9vcYNKmCnlRlddVAqnTSe+7ko4g8AYWz93Iuk/zTzRZzHn3YVhZ+/fW6GgUWx5999HzS52QqMPMtbO5bZ9q4/LXaz4FjJn9Gu9yQf6+MFp33h3+1Hnfv/qDQ3LambGnh7psNWPkqPYzzzwdA+Al+1Ix3eJMcS5VzjQ1RXlUJgb4DO8ygaSfUeyjfD8a5hmwuY0WVG4HoCqxNqmn+k83DIu+ORhmcuhV2UI7fZVajkvngButULOucqQIQUdcLvbI4+mgAlVoDfcfCmVGeN4V4+JszZV95qrUhqSng8M60/+QkcOH3FLzoS+HpxXTVkiHypd0wu1oxRAAUibGtZ+RbLr0YU9KmLj7nlPGH+JYTfBbJr590t9j6BnxUrvSUE84X0uTusIOXcwZ0cJIJHIjYscg7AMgjtVNDb+i0rYflB6hjobBLnYKzUdWl/W50zmeDpcCvTFl8b5uG8DSqkRl7JFBX6RHo9D2eJQflFjLVu4fceHr22uJxSF6nKTfE+iT/nmxZ7Z1QspJdtIob7QYbd6GpF7z/566iuD/1qQENRqrYx4EeWxJoXq6xGK6ITRB9bUYB3RBOJPiS0vegZ4jqlTj2TORGJFIVK9hSRuG03MXyFFJPirUUexi937n2KnwjF41nNTnPsPYEvcXM6JVcdT5AQpYhnzMfhWsn9drfIR7dw 5e9ZpRE4 XAM/dJAC60vI10ADz1gV2obqHj/dHQ6D9idXWo/EasYlmE3XRhaxtKk5rr+DRbDw2WuDBGvLvqzTxCCo8cBPh0MSTDunGsrxd27WQZ6c4uTs5KrB/IN+HCLf2ZOykHFa2RQX0O9Mh21fGiN/SH9ahThpXEWgtbdsqmXvYeLcpj73WVWZFlBUelhS2PNJ9jrhsIC8hK57ZRKowqE8HpULmsPRHkcWBlNizChPYUufPyVL2CdGKoNntjcTlKqNMIHxtjYu4Zq+7gCNhkk/sZ2+aOKp2uZenJj1Jta7Omm7tTqbmHWw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000474, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the folio is marked as streaming, drop pages when writeback completes. Intended to be used with RWF_DONTCACHE, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index dd563208d09d..aa0b3af6533d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1599,6 +1599,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as dropbehind, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset dropbehind for that case and latter completions should invalidate. + */ +static void folio_end_dropbehind_write(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_DONTCACHE writeback, + * but can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1609,6 +1630,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_dropbehind = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1630,9 +1653,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (!folio_test_dirty(folio)) + folio_dropbehind = folio_test_clear_dropbehind(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_dropbehind) + folio_end_dropbehind_write(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback);