From patchwork Thu Nov 14 15:25:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F313D68B32 for ; Thu, 14 Nov 2024 15:28:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E6B9D6B008A; Thu, 14 Nov 2024 10:28:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E1CE46B008C; Thu, 14 Nov 2024 10:28:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CBCE26B0092; Thu, 14 Nov 2024 10:28:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A62786B008A for ; Thu, 14 Nov 2024 10:28:06 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 385F3811F6 for ; Thu, 14 Nov 2024 15:28:06 +0000 (UTC) X-FDA: 82785078264.17.218C444 Received: from mail-oo1-f49.google.com (mail-oo1-f49.google.com [209.85.161.49]) by imf23.hostedemail.com (Postfix) with ESMTP id 59A4014001E for ; Thu, 14 Nov 2024 15:27:35 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=EtR1GyuQ; spf=pass (imf23.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.49 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=MNyvQBBsanXNWq2jnXZhZb6e6a0grCJm+CP8nc+uUPpNHJocd0v9ZhVtfKxasaEYUg9Kmb 1Cr51qYQ7McKqhSmMdGkIKJft3Lkkqoye7RToBimhGo0jT3L8h1/BF+EDVNW4xTECr5o93 xAuHjrp121+C6rSw/7LMbQmYVY/3pu4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597995; a=rsa-sha256; cv=none; b=q927qs+bV5xy3c4nWR0vurJwWo7sl85zt/PQSFsQRTXxLlgAtKffLF4SSC1qKShJ2PUwUs +nTFuDHqwOxl64MERAoNYj5eaHQ4OvWvgv1ZfHFv/JOKX2g0b5k1lRktgmrzEkSn30xhH+ JHLIBIOJ46O1ferVOkuGzjKRlYwzz1g= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=EtR1GyuQ; spf=pass (imf23.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.49 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-oo1-f49.google.com with SMTP id 006d021491bc7-5ebc349204cso382957eaf.3 for ; Thu, 14 Nov 2024 07:28:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598083; x=1732202883; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=EtR1GyuQhhP02s0NUgTCQ9PyIS3lD6HsgVePfpY4sIQFp9BLX1EYR3hNTvv7dK6WYh Vp2KNW9QKgEuJWapfoh9adu0x5SaaMvNGR4VwaNHCSRkD4YuSZN8/rrkm/Z1MS0XcIYk Tps3Qe7dL8d2a8Kmp78UHMr4Op1ZD/JEzrUT3ne7dwG0jw8JWfU2G7VnO6nH/MHhTT/W 2lPzoBHJonjGuSEF/6yMSrGwY1RtwhdycY2ktNUMRFw3oZbRuGtYzA4mh65nay4VY6yC GhFqZHZrgMFZWZLt5Iw0znsGhPifcyGie6USWL6q3eAEXa1zqgy7tkQVlmlpsMdpLXcQ mCkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598083; x=1732202883; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=ZqCTHI6ZG5MlCM3XsF4ZmuCb5JoZwKX4CXBb0wQiUc8lsDUcVGtWvUvG61l996wKoi fwgmpjq9JzkHfn36JjY/bqD9msiUjhdfx68wgKr4jVFesNUASaFAq6bPxJl9/yZYx3l2 b0JmjZGOf/yBVywAekao9lD3IiANFaK3pKTTtbu3e+kQub4OIBNC26FZt4X/vzqbQts/ 9DjIyROvpYK6Gig7C6H0v649IIpN6cHlXfgmvg0xniwgc2jibposkzASHMM/01K5tKNG f5RpY0lbRPauo2y1Ci5RBZyiF5/bkkSGGZZqqkDC1idEvjWpsxjQqzTF+ATr3TVjDZlp 2ocw== X-Gm-Message-State: AOJu0YzGzDF5nqKxyqJWW2ty/ziz89uft2+d2hlKQw2QV7hJmwpZT+5n 9tT468tjCu8ce/XHCu3RNdh6qBDPMreSo3Y2tFEiQ6mEWr0tTMhadEu/HeUa5DFoi7xdtRVVjtc CBSM= X-Google-Smtp-Source: AGHT+IFRnE0ztQRJK93idhhgrf7Bc5KUMDNOKjdutZVsJeSEeh/0aaiCxUu41HRdHQkrlLeWeSTtMA== X-Received: by 2002:a05:6820:8cc:b0:5eb:6a67:6255 with SMTP id 006d021491bc7-5ee9ec3bae1mr2680975eaf.1.1731598082894; Thu, 14 Nov 2024 07:28:02 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:02 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 01/17] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Thu, 14 Nov 2024 08:25:05 -0700 Message-ID: <20241114152743.2381672-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Stat-Signature: 1h13yggsuusxnsyfpqp31c6u66fqcm5i X-Rspamd-Queue-Id: 59A4014001E X-Rspam-User: X-HE-Tag: 1731598055-934488 X-HE-Meta: U2FsdGVkX1+ALKFSCqHIFadfXI7aidHFpPmrFD5O5ZVixzc70d41i+jYRuMi63RWRvt4Gv4m/JIPoqbU0hPr1GxHmEN56TB3QV0jT0hM/pEc/Cn0I+vCwJYnjcM8p9ry2ry0kVRikzeL14lOEvQfTFCb2aDboAZDkLEDzQJ9jZElC1OIinlukMGwDlti2iStvResIPpfNl88ifQYDigSMHqhQgWrkeGKokOkzam4ooRormpY7wBSx4Q6jQjwD5AEw42f/K1Yxz1GXy4la+37iCQVpZt5oWTRs1AyQawTk8vsRpDW+34Jw4NVMIiPB1N8byipucCTiqYOwHjOaHISOla/XDEdqzyGydMtNcwUVBlJYaLNrWTW0GzpRCMhVbjPqPz+x5hVzYWqUgCoG4GToE5j+N6TeD+LvCvdavnqdcnPcP0jpLX7pN9BNHa4C47ZMvOny/7rJOlOn+FZWDZpsuP9f5wVZH0lYAfM/2nIZJhkqBpNBEzr4rv4UvcgSHkcervgfYdSrNmtNBcXxbo9otCBu0GFcyiSONWnhdouU+T4llnqkCi5csujfPNjPIfQZqODgOGTv0fDIwbYZTNtYGMnqVpsKhbtMF4iB1AFYQMkU6FAJ0/x/ACJ1u4r7nWAah6FDcBlBu+lAnnLGxcLMhqJ3aEnvOr33hOVZv9h7xuzNuOLqniR6glGgxFKmPuavTIapztCs1T8EMPa88C/5CYi6+/zLcRyNIBF1GUoFg6oZjeLKnrxpMlY0mkXZLfhiwziYYC4BdytgkbB3p5NpRYQjGPZzOqgCujv/5SkigcYDTuXAqW6pGahUcsXWFcs6nkDo0ejiWgLF2fu/0ryfJv6oJzTj0Pu1PpvN4qbDdh09d3ZedSZWgfGRf6oPrd6aX1PRH6Y1MIaOEPSlK71cP9LAZ1r9DDJBGws/2GOqYBFjhT4zJYNBEv/w/bOKSgHnX6wfzZ8rb8VoCn7GF5 6weDgG/A XzV0TxpSOqnFIFLMH9C/r9Lzf2y7q5rak19SbUCsg6Ri8KyhPcxqFXVMyf9VytV7YhDJ4d22LvR5H4TyRdxDfW65GwS1n2fmXU74Ggc46rr7D6QWQzsQNQenDxkqr4biU6KygjUyH67f2WuaV0ePY2cbwdG4z/Q9ny9mevOmQbVRu2/82FQ7uJ97+Ed8GAzrUfpgb2E6eWciLjsqOL/V6Fwxqk7QugTxHrBo7irQmkfAaBMwGZT2xDJdRCKvYc4Mo+C+mt5L5Q0dpBTjHnmT+WbQ0+URgQi3Ja1CV8I90AF3RzLn0b3F8RZCyPX8mf1pmmx4qukTu+SOn6st6apqBLRTKGIW0DVBP/BffHIDwmS+3QaH+kHKpItn3PEOKrQasMPyIhyrOuFy47myRPxydznII93U1RYxFvEfKr5lgS4o9o0wuG+L7h/pMqsPMBK1fDow6tu4D/IMW7qKHWppTm6xIFA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. While doing so, move the ki_flags checking into filemap_create_folio() as well. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 56fa431c52af..91974308e9bf 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2460,15 +2460,17 @@ static int filemap_update_page(struct kiocb *iocb, return error; } -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, + struct address_space *mapping, struct folio_batch *fbatch) { struct folio *folio; int error; unsigned int min_order = mapping_min_folio_order(mapping); pgoff_t index; + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; @@ -2487,7 +2489,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index = (pos >> (PAGE_SHIFT + min_order)) << min_order; + index = (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error = filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error == -EEXIST) @@ -2495,7 +2497,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; - error = filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error = filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; @@ -2551,9 +2554,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); } if (!folio_batch_count(fbatch)) { - if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) - return -EAGAIN; - err = filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err = filemap_create_folio(iocb, mapping, fbatch); if (err == AOP_TRUNCATED_PAGE) goto retry; return err; From patchwork Thu Nov 14 15:25:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C568D68B31 for ; Thu, 14 Nov 2024 15:28:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 25EA76B008C; Thu, 14 Nov 2024 10:28:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E8846B0092; Thu, 14 Nov 2024 10:28:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EDEFE6B0093; Thu, 14 Nov 2024 10:28:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D54946B008C for ; Thu, 14 Nov 2024 10:28:07 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 70BFF121319 for ; Thu, 14 Nov 2024 15:28:07 +0000 (UTC) X-FDA: 82785080532.16.454D61E Received: from mail-oa1-f48.google.com (mail-oa1-f48.google.com [209.85.160.48]) by imf19.hostedemail.com (Postfix) with ESMTP id 0986C1A0012 for ; Thu, 14 Nov 2024 15:27:09 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=xLgOGZqq; dmarc=none; spf=pass (imf19.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.48 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597955; a=rsa-sha256; cv=none; b=HgodA5mbVd3VAzK8gQ2puGyAMmJJs0ZAz+XdCXEc2cij1jZa8uwQkypW8xnSeCZ8ClkX/w 589e9Tt1+fDePhSXO4PNyPxaFIaQ2PPC2kIhogj8e3KsNDblY7MQ4CCqXyNsHpYxy6IqJu x+toXhCT6w/PNijU11twvw72gnw9usA= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=xLgOGZqq; dmarc=none; spf=pass (imf19.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.48 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597955; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=IqVLl4WluVvxMuI5Ce1aZVxAo0AhXtJIRGOlXh9/VSIvw4ZW/tkI8HjbHX2BvAl7OHhBIm cDiIKZLFsFRvjHdqhb2GSuUfsUSMVjyPFVJEInP5vH5whjVezOXxEV4Fv+0elClmDo3jt1 ibnj8Q5dnvQG4Ag7hILmkmgl0+6XMuQ= Received: by mail-oa1-f48.google.com with SMTP id 586e51a60fabf-28c7f207806so322602fac.3 for ; Thu, 14 Nov 2024 07:28:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598084; x=1732202884; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=xLgOGZqqQa4IzxNpGKjWC9oZ5n9hhWEtTL+sanUUdVqKlIxXP1QT1da0/gnwC1lcpp Ir7+Njrwyz3oh5Sbj1aa75IjI8yNk9u4NlOQ7kPBhKYgWA1EiYRp5Vp8Ubh3w3PEP+MI ekditAE+l3UxpHg9/QF7m6zLsgDknot6CO4tLhOK98GgkdjJMZvoAhrt94i+Bm3hTxIL 4H5G/kDugapP02b5Ie6jwVPX2xLWxo0WvJ3DovKTgfSViXnN+F5YHyyaHOglL2W++knL 74AY9/V4RWxSglii0A3C256K72LSIPkmEHH2n+AeY3K8xn9Bed5dy+gCUHeZvc1WuCOs bjug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598084; x=1732202884; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=c80IYsEwXnr8n7uDnKKOTDBxHhqqr3esGoi43sFpMeQ22MhxjYh4CfYOSt1ZCxCH/W ONQfxaFex2o71yghw39VdJ3vbAH9gSEDi3GXVxzo7OoNd8mvkPuD9jdEiN6Ev1K2j796 mOWNIHw5jIlqf1TjLoQC7tLvwF5s0eKRbTuOWgdK7pAn3yc2gkmmS+sapVSm9VH2AZMw bG6HfPPpOyy5BqJ4uxWvfB/epGC/SuweZN0DborufEQzINfLrvU64YUWcsOiaAxVrAyj kiKqeYW2XgC2fNA1m55JebWGB9qq+kJnd1huxRJJkD/YJfki3pVquUaW7Nn9sf13Oyhl za7A== X-Gm-Message-State: AOJu0YxUKJd+39VAf5YD3GUtM8Ab+hgyDqjZZW1gJRZjWCmql2Gxs/iq 2ZX6S66368WruMdh6chw3fxJ8J1KS77QN7ULfFwsMZK680m5fLU+Yodp1oJv/3AZwkcJYcLJ9zH ioj4= X-Google-Smtp-Source: AGHT+IH6odaDwIdA82zwSIWff/OaVLT9OGNonkILORb6hdBpcBvYTtZBZw+TFEiCAtvLZtie7sKouQ== X-Received: by 2002:a05:6870:5d8b:b0:277:e512:f27a with SMTP id 586e51a60fabf-295ccfd9242mr11462005fac.16.1731598084309; Thu, 14 Nov 2024 07:28:04 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:03 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 02/17] mm/readahead: add folio allocation helper Date: Thu, 14 Nov 2024 08:25:06 -0700 Message-ID: <20241114152743.2381672-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0986C1A0012 X-Stat-Signature: 7ra3rc51pwiw6sprthp5aewbmad6jt1u X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1731598029-9528 X-HE-Meta: U2FsdGVkX1/81vQkJosVEAhzzSh4yXzOXkc0Gc1JtylnezPKSutfcAoBdS0z+HQDs2F6ayMJQEupmtUaITYbRvIOIwTPzwBK+imkwZYkA9Cs8e7tFBoC+U5pdxrQALAPxCza+FQTt/bZU666ityOqQ6UPIwv31jsA/b0aeU/8e+uWb88t+98dshgMoj/4Fs0uh+n2HSIxllsWV4crGLOppsIV2kSjQvHSZcjdIAZDjKe+KXaDstzWmhCZQyUnLURRKoRR7kvv7k/+Out2B1B68t67g3s/DxRutAsFvAX6MIEXGxza3KM+7g3XIW3ocCU+Rp8/9S7g/M387/malfCsIuqI+kBJjIOIschDcUmsEJvz2LehhkjNNr8bUB8yb9LL00ECag+WPfWtpTTPgLj0idqIc3H3YDg91qI9itlx79YFkDQ8Hqv4mqo5Fxf3h2dVSkKbAqDBVH7ruvzTExbm6ILVtjgPNjSgxGVwVCMUl9YC2DMJCmXNlMsItV+qWmUn7wVUBmoMJLvVCSSE1TJgay8veKyVofQCyX4PM8WtBcs4Dce6DHgqD/VBpRws1DACROHvW8LRFJfn60v1bl/7+KP/Fl67W/H5Yg07fKkSU4RYZ1QZF6yF5aX3y+pBI8Ohyvm+vZb3iZ85a4D++HrKaCaSUqx97nrPf06+nIpc37LOcAht/gfnpBo/wctxMtyn1VhKTwVm2uTruLV6thZkotxEtIiAGcWOKFOrQKabk33k3Lgqzdit441zR5Z+bG7wHnlOgKxj0KGAosv5gHwmeFIhEGwZQcYPgeKEYWMXncThVtQDzMT9kLoyUUhJJTmr8IaUd0pXotXCqRBXPR4Z7F9Cp4G6UGAZMlvFFv5iAsjwpuxWvcFKy5tp/6+hNtSayn5FRIG9bjbkYuNHWdfIqBAXcaSpDd6F/d/eQ+mn7crTnLRCX4q4lYi2EBkXPflet9JlJdjB5CmaoEMYfZ QbEvQ293 bxdAh/8onfRBXtecJbaECjN41ekGeVRNgTak48ShnwX+hezG7oa1lceN9li0hzY7NboHOk4dNu/Bbwl5i5nq6Q0JG1YuuFLmAXLg1BLv1QfCoUAJr07Ld7N2OVoGnmybMUZYtoN94iVVfViSWSduYALBysVapN+A/rH65cVDyOtQLOk7UIHvcV5g6r4+RpEt9OZAeF10D4KtxhAu7wkNWePNUecZVTTcBSth+vPZuTvaeCqdHIIo0XB0jvZIaWaK2QKiJp82h/38t/xz5WZJyORdbg6tkWFHE0R7w04UDXMlUctorFI4DhX3zubKAYS0mCBkP07sBbt0HXZkBuudOuqPbwBd5gVfHrAUACczfVDFTwC9ULSjiOcfmPvgC+CBDfJVJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index 3dc6c7a128dd..003cfe79880d 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -260,8 +266,8 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, continue; } - folio = filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio = ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; @@ -431,7 +437,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio = filemap_alloc_folio(gfp, order); + struct folio *folio = ractl_alloc_folio(ractl, gfp, order); if (!folio) return -ENOMEM; @@ -753,7 +759,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; @@ -782,7 +788,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; From patchwork Thu Nov 14 15:25:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4CA7D68B30 for ; Thu, 14 Nov 2024 15:28:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B2926B0093; Thu, 14 Nov 2024 10:28:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 963B46B0095; Thu, 14 Nov 2024 10:28:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F1856B0096; Thu, 14 Nov 2024 10:28:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 480136B0093 for ; Thu, 14 Nov 2024 10:28:09 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id F214641238 for ; Thu, 14 Nov 2024 15:28:08 +0000 (UTC) X-FDA: 82785080196.22.4C6962B Received: from mail-oo1-f45.google.com (mail-oo1-f45.google.com [209.85.161.45]) by imf04.hostedemail.com (Postfix) with ESMTP id 0679740007 for ; Thu, 14 Nov 2024 15:27:10 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=MLhkA0Wn; dmarc=none; spf=pass (imf04.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731598023; a=rsa-sha256; cv=none; b=iNNc8kldDQZ2mO53WDQg6nu+Pypk5yhBTUx5vSzqrVkN+qJYFuTVCiiQJR/piJqgTOEwyv 1JUT/zmcmlo3v6BF3RRTJqPA5HnrRXDGszDm7xWE5MRDEqbDbbjPPzH0dO/s1UW9A+QAVw KKY7armOGZftgjcncq6PGzGOeMM4saA= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=MLhkA0Wn; dmarc=none; spf=pass (imf04.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731598023; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=Xdzh+zMRYt6AQgcMB0NmsKmS0R3Ox/sXkmSkBSL/4iGuS1AA+w6njcFPlDtkLrhBGVgBhM GDitgMxuHnZa05hcXCu342N40SO+rFYoiZndMcGG6vPC7irffAzjGFNUA6FlUhhPpr8cQc zpAJURlGuTmT5MrW7pPd72dInNmey0A= Received: by mail-oo1-f45.google.com with SMTP id 006d021491bc7-5ee35ef95daso321520eaf.0 for ; Thu, 14 Nov 2024 07:28:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598086; x=1732202886; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=MLhkA0WnyH0Eaat9fmG0KsoJmzo28cNWc3rXkjVuS8pkLcRWjGL8PywyEWjfREESxC 1Q0uK6pQ+HHUBdBoSEl5dAIIZq30XlRQ4ZDvP/y62DCcU8L0GFvpAaPkYZbe9bx7uhoX hE85loSPKI+7p5swhSbzaVzm8ZxMBu5nuZMzskqmSKMeL0zKJs58GeMYRzCnM7hn4rzg 6xEL+7iAaGp2XmYTDXaLRGwR+LUscDxBGopG/5LMJLBZhtABmuhf9suLvqWjzZ3TTN9s 7bc109YLn4gjy1eakHitrF+ofTtb/WKef7N3k7KDkeb5jwHwvcXskpktx2qJf+CW/LhW mrwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598086; x=1732202886; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=rZax5+p6jtu7tJ06jy0vcmpKm0Dyf64vMHHGWlH67kI8tArj181dcboansoP2Cw39M VSMIatiS2GEy+COKARADWyheaAKq5g3G2E16RBVdzaUk55rHFtdatPLrFMfj6DxcThKD t7JrBOAmZCrE2M/j8NHRMPvKQcckm5DU8ylN9OaJsFer7TNtRq6v69EoGQDM4mvXMFHr afBBpQbw/PIabXM3ZnvPdpr4NeUpXOxx3p3Ar9+agBbOkMNkgIGdbR2DTcbHXQrG19QW xvHriLsisJiQSEH9dKUCpEAaN4S/MrNAC06Q3+562k0NcGFcGp424z/o12RtVjmz8zL9 h8QA== X-Gm-Message-State: AOJu0YyyyCpV8eGGfjJRGkqqdga6IFwruyijfpWKtj72Qza28vdGy7j6 oI6jIfGE0fsxcM2rb2BlsQfYHuv7ErpvRgpbHLIijwNiIDydv3YyjfKDSwbrhNZrsBZxCdE8+Xx RkSM= X-Google-Smtp-Source: AGHT+IHYfzNU9jsYZVqq6BubIUmPINzkrBJpOzUKZ5nbA/vQYGCWpQqOD4e90GxHDczr5rmeIPTulQ== X-Received: by 2002:a05:6820:3088:b0:5e5:7086:ebe8 with SMTP id 006d021491bc7-5ee91feb02cmr6995384eaf.0.1731598085665; Thu, 14 Nov 2024 07:28:05 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:05 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 03/17] mm: add PG_uncached page flag Date: Thu, 14 Nov 2024 08:25:07 -0700 Message-ID: <20241114152743.2381672-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 0679740007 X-Rspamd-Server: rspam11 X-Stat-Signature: ya6mwko4xmpzzg98ekb78i34s1zyu54y X-HE-Tag: 1731598030-739630 X-HE-Meta: U2FsdGVkX19qKG/c5+MukvV2Pli0q24p4ncG6lPIVVsATtAO2ilq25UZQiI0vWwyP8S7ltckFQ+T+T+9x1adS1woSpT50iadP8xitbx5Bo3MPFw3aPaM+t43spPNCVyf8ARbsQZPHCgfsxUTQreDVefcW5IphCWy9D4aRcOHWa0fSyq/syoYS/1n3JwJUOCVdG6ST2AVsBKDQD08dD1aogiH9lVom9bK189QVx3+XBMwwGiItPd6507oWb2WHCNE5IiwyWzdBqYIO8b2cebdQTarva/QerG0Ey3TQM0YOB4Qy4+nG34lqxluz2xoSTm9xyDF4ltpistIkdqokAZ6j8pM3FyuEe7CiMlZZx7/lxCzZwSEAesLpg+BsugU2wzo7oKyGtxtHBS4pTcDfrj71YLHV67rylkJT+ea3315P10E/Q4Ewa4VvTN/MODqkP3XkpiScvAMWOOJipAFRdWSx3hem/03TKZANctrpmQ9eUCWEDDY6sqkamuSZ5YN65vK7nN1yISyrzyHesgH7AwHrPbGB/Q2jeVRZH/ty3Ex7JuIeMPSH7WNSyt6iy/TEacdN/Ro+6qZEkN35Q7dZsGFhAYzfZZS61ppeBgEzex49+aDYRJRi8EoVkv0Wjd6EqYtom7ntTT12Jz9ty1gTjNsiaglGx55TYid1akRNwq0BJeiSdWYUPOf+J8vMB/go2l7b2Lwhr2cm3LxbimYJq2DywoHU2e1VbeAh2F0Js7ZjTG7N8TYD8dD0wZaG44RFWla+ZXfVOcHxLByaHiIjiHV6kbgk+YjemNLLa0whJORwT1/Q4DmuEm4I+i3jKM5nwScDHjLLZUadDHQoyLnvLTd0EPq74BVk3IvlspKfbFy+QQAbFc/PppAIuLjOPEohVSg9hoYo590RndirsT7cOkKHUJHbntlqOlGlkqGM523rmMJGikAx7UPBjLqr3ojHfFME0A0LGCfyYLW7p7VS0V T23++5NK mXpnhI26AoTnrYtzQECHziFDek/9NoWf7xV9k7UpSMx8i4Au8hd4gxGLSHPVXtTUUSeeJnKeveWXu3FZWZp0dxZg7YqTmy3uCpKzUr46vG8CXv5tKWv/4rjPqcUBW65nihYK9C6WtZ6i1+B2k8b063RyzOqPvB/T2AlQh8W2Cmr67mTN1VFO5DNSt8y6MgsbSTl5khVOdAav1c2w2PNmkqLmiC/AEGF2a5w6YCnpKAnlFGSCaKbRJlm1Xr0Ie+YOmgu9JwoPJtISHda/xfQq4O88wYGt99y2GKkkcth1qYWHz8uqAmVN2g05x33VhIqC3HsC3uMB6GG2fKK8J3UA6Bkfy0jUUNklarZ5ax0aq6PYq2sXSIQOYGbegCltvG1zsJhmhV+6ppEftVKh+ilCroewPgQ482NTNaFN/syXZGrAoiUI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a page flag that file IO can use to indicate that the IO being done is uncached, as in it should not persist in the page cache after the IO has been completed. Signed-off-by: Jens Axboe --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cc839e4365c1..3c4003495929 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_uncached, /* uncached read/write IO */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) +FOLIO_FLAG(uncached, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(uncached, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(uncached, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is not diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..b60057284102 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(uncached) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ From patchwork Thu Nov 14 15:25:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3B0ED68B30 for ; Thu, 14 Nov 2024 15:28:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1283C6B0096; Thu, 14 Nov 2024 10:28:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0FDF06B0098; Thu, 14 Nov 2024 10:28:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6B0F6B0099; Thu, 14 Nov 2024 10:28:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C46256B0096 for ; Thu, 14 Nov 2024 10:28:10 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 72782ACDC2 for ; Thu, 14 Nov 2024 15:28:10 +0000 (UTC) X-FDA: 82785080700.04.0965E2D Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) by imf19.hostedemail.com (Postfix) with ESMTP id 17A2F1A0011 for ; Thu, 14 Nov 2024 15:27:12 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=v3XLxXS2; spf=pass (imf19.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.50 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597852; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=2tL/CR5yd0Wp1P0YI4R9P+K7DutFonCf3irZneYyWgqXOi865+xW9PId9zBzr8ioQMIBAG 9wbwrnl9NZvWRInbHXAkC3ISILOCAXAmIfFpWIE08eCNkON5ieJyb8ivESBXen/6Fz3chD LsUJAsDGPFe0ls/WFVFVtWmQrOtWbY4= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=v3XLxXS2; spf=pass (imf19.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.50 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597852; a=rsa-sha256; cv=none; b=yVUcwGpeCsfvSuywO4Yp3wTt4cTo1OkobrHh8i05JD0c/gjcJ/gldRbi1Qm3Lvr+l85Tlt o5TkRdgRm8rGfx3H1rdKYzJIWESlm6BrPUsdAxBdD6B/lF0K9189Iw2Iwprnsoiaslr1U3 SuvAxq+55j/1mzu/jgOk3I5udns6DfM= Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5ebc0e13d25so280195eaf.1 for ; Thu, 14 Nov 2024 07:28:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598087; x=1732202887; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=v3XLxXS2AreOR+TbtneqfI4QdQs+0Dmi5JdMvEJzP63mdqXyWvfmO4dYWxKWQrU21Z 28ZHtZTRG6cGvk2rttVUqE4WgcblVkAAX05Xu6FmOgrjXdMQqMz4TrFqtUF6iV0tAs3I wjyfbW0hKdqw0Ubq78J8PGBGqHDrXAVVe97pVjIMYxIyFztWza6gJRSYYL1+jY5lobl7 SS5R/q+njDHzO83nU5/X1hbQ4GnzruzVtycDVCjJ91tt9B+AnGSFKCi1KwINFbE96CSD v7fzex6rK4SnjpMO7Kl4+LiBi4hokvu5nQmxJUVcdFjmOGeBiUqYMtLehbk6o4WXUoVw dCwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598087; x=1732202887; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=hjz7YKJWGEAtcHJb8RU2QCVTN3MNt4DU62On5jyEg76scqY1jvMj/7bOdEkAC85OZz xZ731KcX72gFxJh2gbUe0icUbZlQkCsHcKuIgpNAEagX2Ht1sN7I6tPgyq56YiYRmZR5 W0sV0qegs6XCSIlXAqIl8rh5z7vi0snh3j3mQ4mOR5WjnwnsyZQGt7ECffAZzg1hSdVD ObKL4BDUvv4GbJY7sBALt41Z2payGKMkOPEpUMELaegsp5BvKFS/6ksPwZpvhctuZpQm w6l4HKOvFCVWk23dOcUyb6Ya4QiyvfrKLlJQbfYYPvjM9IkSe8M/tS4uMLAg192W6ebk xbyA== X-Gm-Message-State: AOJu0Yws1k3Bipd1kTsphYttJp9ItEkB3arJm0HKu4qOQU3Ss4BQv/RA +ZHCjoXC4ujwRCCKAFn9me6a4mbhhvEY6mSaVc0T2GPdS/2S7TbxhNT276AfC2gjJvBIFrA5MOk QESA= X-Google-Smtp-Source: AGHT+IH7eepGaxoKX7XFVQSCKcZ5tiiQfUtPjwDWCbNdm4JNlvZQy9y+41dOTUM8EVcPpq30RA+88A== X-Received: by 2002:a05:6820:278c:b0:5ee:710:83fa with SMTP id 006d021491bc7-5ee922a81aemr6235784eaf.8.1731598087197; Thu, 14 Nov 2024 07:28:07 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:06 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 04/17] mm/readahead: add readahead_control->uncached member Date: Thu, 14 Nov 2024 08:25:08 -0700 Message-ID: <20241114152743.2381672-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 17A2F1A0011 X-Stat-Signature: 76psnsc4a78gf1raxjpfxgxto341ghat X-Rspam-User: X-HE-Tag: 1731598032-993269 X-HE-Meta: U2FsdGVkX19rbghYfoWiOMtqTU5b8PHbNG14G6hiOCHpRKe5u9kPuuRgA6ZvUQja2Ddlt0XoDFKvdnwxX0uSfNm3+md+OG5k0zqYC93101wfw+TNDySSrCMwNA9jtGM/q1AH1qgib9R7PbzJkhDl6VMlFlLWj4fxlknko3+lDcYm1AycSXQARfD1X/fq27CynDdvnausAnzjdNVGikK1EOXxEA2QvFCzD6fskJaehnmNybEqUe/unh3k6qvaQXCKMwZ0KsLAsGnsEmTt9q9IMakJh5bmA2EdX7nhiSO6qNlyrz3D8THDJvcBRsgu0UXguQJ9pkc1nondDW/TsXB279UDFwL87WnXUGnAMn6B32++2xADK7mqfcoxH28tiwxN10mViLDD99iuR7mjUv9Tpi61oWSUWFBFRmG3hFerWwLRPbz7ZFQ2NiIqQ47DyYwD7DnQcwxdUHwFbuXeqWVYUT6JrM5IW4t6fpaD/obbW6sgDWk7Dbw9oiCCflDfYNtfmyzBHX1e12ctNPfJnXY+WMBx0ulKKszqXO0yOuaUpT9fZ9RY3PX87+CCpWeAcYxdycAPvf0Oq8Or7x1GSg5PZWRZeExj2/2EqmewzddU83CuyHjPa4/5gn6jSDbP89sbMhPUw6WM+8z2FY0Bivb5Izwn8LZ4O9qFrux3AFvP1S5Srawvf2akzKsdyyqXDik5urwmet+eEiM8+JqC/pEYpjCkqJpqaKzG+tVqGY30tKtQ6WzB3KfFl0s8GZuav/h4G6MnQukLs3NIXANLy2qQfmhCCSJcwsgTdxEh8c99fxhXX1evZKX2iqvASlZjeF0lhBcDAesOY4814LpcRfnDd0+QP5wxwNuss+j0nNy4BFsxisaTMfSd4xj4mv6rHhj2dLm0I6YaT1DXoB2509hfSS/pMAl0N7cZDYzdSGLblBMFerbawKP/001R5lfvteWYdfnaqr0/LHBI3fLrur1 c0zgPeHQ WboDXLiIjfiWTEclg6LbneDNpeI6ZBV6F4HHp4dRa6rg7SWgEMeN+F2iPmgei1kK21WQDn+pS9FaWJtPoenGyB42zc91P6OYbQk4xzfsd3vOuSZxPT13EN/ExZ9DMnzse1/cqQR/XmVrsS4X6EFBW4Ebd5ZMdKuHMnjkD2IYub6p8oNLIsi9yJa27OLE1VIblBtksTkGIVeNFSRunWzuZ+06N3Xc0lcHWkNlPad30MX9uCbMEbdvpBKrkNVMmSBbSoDBDkcyBvFiIlrxMWQl/npDHvlsXc1BuS+A3HY85YoMBUC9xJYx1goUOeBtQUY6zfPbHsG0JD543PpFKcRv8WQ3G2vx149aDQTCXYhWjd5H/brmM9nNPl8YBlUw85cxDZ8TuUbx329YFDJNVifnJk1evkseGYh6TojhaN8yRxZmFjUQT685qKzxgyUlAZJ9Z4syxylJ9DZ/AMCqKk1m5rZM+MA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If ractl->uncached is set to true, then folios created are marked as uncached as well. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 68a5f1ff3301..8afacb7520d4 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1350,6 +1350,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool uncached; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 003cfe79880d..8dbeab9bc1f0 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio = filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->uncached) + __folio_set_uncached(folio); + + return folio; } /** From patchwork Thu Nov 14 15:25:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2D0FD68B30 for ; Thu, 14 Nov 2024 15:28:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A63E6B0098; Thu, 14 Nov 2024 10:28:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 655E76B0099; Thu, 14 Nov 2024 10:28:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 459616B009A; Thu, 14 Nov 2024 10:28:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 25C276B0098 for ; Thu, 14 Nov 2024 10:28:13 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BCF2FC0BB8 for ; Thu, 14 Nov 2024 15:28:12 +0000 (UTC) X-FDA: 82785080280.26.A601836 Received: from mail-oa1-f41.google.com (mail-oa1-f41.google.com [209.85.160.41]) by imf30.hostedemail.com (Postfix) with ESMTP id 6AA7580010 for ; Thu, 14 Nov 2024 15:26:46 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b="UHisqi3/"; spf=pass (imf30.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.41 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731598001; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=41RY9a9Karu9Ll8QTAvyVcKPbD+peFqfPmbw9uYa2vROvgYOCGisuq8U3AVINnshDLdDUG +TASqz3Pw5b5de02POduuTrsOHmLK2RQb6MzCd/fawfsnbFs0Aa8AkqfJLzbCEmFGwNX4X CSCeCR0d6ozbsScRokDuw9ThGPws504= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731598001; a=rsa-sha256; cv=none; b=oeyhVJDWJet0u/A09fQS1ppMD7rfZ75plIyOAz3P0rp5/kNVzFXKHjm/nzSK4HAqX5BZIH lzoxzvll4MwnHt5tTWu39Gs4canVZYHtIdQZarfFx0K7YLraioCMB+K3psUP1nRzo7/6kI cXlcJpsEUGpDRDJqffimpqYcg7Rty0Q= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b="UHisqi3/"; spf=pass (imf30.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.41 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-oa1-f41.google.com with SMTP id 586e51a60fabf-290c69be014so365765fac.3 for ; Thu, 14 Nov 2024 07:28:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598088; x=1732202888; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=UHisqi3/86P0kWuZ775aqVECMg/0lyhI41BTRVzzmfLxnZthIdrnbvgZPPC5v0GZdU v+0lu2RuQjUGb+dOjP3CvmykByyUT8UO94QwJEE8TVnYzzDbWw9XgNlP55TIeyiP6DE1 UAHLcRgi6wPxaQy5Ym2kTUJb6AavuwiQBZNQbQRaJ0llm6mvOkNOyfTT7+gIHK+U4kPO OctN9SNr816y9Gdi4wUMUmelpeoNcerkjePK6f1KikVumk/bGpZoyEpygQ7mGzT+N5UA +uGIxMcITjCcZToMi9xKrZb6UOgguOv/Kc5JuRIIJy/rVCtNjci8lS8K9ZldoNpj5ssb sH2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598088; x=1732202888; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=hHOGLYlTDuiCW3HySVRNIxGQOtBNKRc4LOXQo3r9s4W+rJK1puSXCASCf7fX+eRqSW MyajwlRK2C8OPn1U8DNNSRC5xn6IUsiVV/LpW+7jYoiMO1szDg+SroFn8WyhI1/3Tmcj emMqhvaS7bwEXBli7+VDZXv/Lkryz19i/AlWKl5uIM9ipD6CDCV9QwdpXhLJVhmwpFJ9 N5oFtPu9ODxhfiDh4mqGp8ebpHIHqRD5kHGmwtyTHjYtrZ7Jw3JAOx1y65nm5OFnZdQP R0CwxxwobSOwY88iZKuhDEyFxMiyHTuYKfKs9hDgUkNALqWT0GGeSw3or5rtbg9boKaH bMTg== X-Gm-Message-State: AOJu0YxPpgxtVI8PdYNnGHJMxkA4nrhrBGpmTKUJMdd1U7JCpEiSml0x yvL4joJYQYbJM7rPUzH2yledA/V4trh4BIAphA6SxhCI9YjR1NjiywMbDsYyixbprDEZDkLp8td PEYU= X-Google-Smtp-Source: AGHT+IGRJhbG3Iz8YyLsc6/0RWtRhZhYbcSZVTwigVtxhi/CGcrJEwq+4Kef8lO3NDoSjpzlANfJNQ== X-Received: by 2002:a05:6870:5312:b0:286:f24f:c232 with SMTP id 586e51a60fabf-29610696786mr2665053fac.42.1731598088623; Thu, 14 Nov 2024 07:28:08 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:08 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 05/17] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Thu, 14 Nov 2024 08:25:09 -0700 Message-ID: <20241114152743.2381672-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Stat-Signature: iuznfdqhcf8h9qbk1gz4aaeam8th8te9 X-Rspamd-Queue-Id: 6AA7580010 X-Rspam-User: X-HE-Tag: 1731598006-673982 X-HE-Meta: U2FsdGVkX1/ZwXvJEbN32GeqiQkwiFYeU0yWdwgkvULAP6u9v1JB6rYKL6ghyHztATNtoYgB3Jqridp5bb42fJnyLkwkUCPVgO1jTANMdBuM02WUPrEaQ7KvjkrwFAUQZf4HiAESNJEFjsISgPBg/ICQKSIKdKPOYvxjQodrpwnN4c0t6kMmRLZahj+39HtX1toEhJOANtJwbpSakGIzXQPC6F243LWtYOErkYbyTS4hb9gIzTq3ShtcT1G6PqbokDTDeMXzTLt9cRYnlp2WwxpWOD7KmW0jamLxIOAmusgXS9Z2NiXrAdCk9FcvX2BDSUDNzCCqJ5swwcA2NfMJR85Ru8TFfNiG924nn0j81jm3Xwj+ANRlL0ZxHqI4PtbMwWWCh54kjo1b0SR2ryR3d5Ds+qWftwLCxHax/VXwhisFHWlTys1IDsefWldEvKh9PK4K7CfC56Gm+yI24jlaJaMaGz4CcQWTtLhm5KJJWvX6epW4spensyyIZlMYUa+FQ/IMF2lfhphqzh9x6goX4OsJWl48FYDGOLw0U/Ru3QmJYAEyNIXS3nNRTIQ4ktM4Gd88Ceh5mjTA7xCM2S1qVTXcHy5dzOPjcTET2aHEYB8Ozbp1rFf58+zMj/BgcOn/ooV3mPQ3r3/9eT5A/3oNCF1Mmt77dmbm0QYPPLtvNBU3j5Mzi+mFzfyEbXYZ7pUqm+jd7TX6aQ5eA5ZjylKowx6M/Sj4PqNbeu9X60XBw2FbIVwYZ9yAd/vqPNmiCaQc8pBARHfNR1T7MzoEGNV5be7eIrZhHC896LnqKsRek+qhnh/0qwu1+koyDe6zNGGl6iCdx2Q+O0emhkEOT3aIAiqA/keiBcd9vKCthZFIcu1ykpgdmN6uVwFzimY/caSyaAt3p/pVowGECs+QqK9yzQADGk55fZMkf2Zzujn70JtgG1xbNHc8VXvSv5l96p4v1mO3znkgOzbhGUx1ckO v3R8oYfi QXaOIG6e7eriNsdzWeNFcGZ6I7R+EtHdfXoyMX0h3SqZPXoqmIVpKGHYQMFEQnn/N/aS3i1PvGVww1ZKttw1xqrUkFyY3nLJigQ3uiOxli5A4mnHX7k1yQNjvCBzj23+pY+CB+cnFm+7jaVP1nNK5QNdm42Pn4vLKjE0qh3lzk1lFS2xnDSjx3IijezwyFfX8jE5B8/PcGspRAEaiYb5dyfDt+5bP4LRJ1oc+HutpzENZ5RVcGmTQHEXB9WfvDou0GOHJmBLfjQuekF1TJ+iv9IV2jt9Nh3JE1QGDOrYbz1Z1EmbqigG4sDsCxFo6P6DnOzlxro2o9wub1pYVBaf+JYK2gN/rWet6lvQ5P+ysanWsnPuCsUraD4TInSJFaWC2gZOp21QjTqyO+9G/ZfZ5mYuHK8y5vT0o3qTwAm+WixpjeycaxoiYUmFgVrCFWggTf9cW+o002YBqedt3CacxXtSxHQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 91974308e9bf..02d9cb585195 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2528,7 +2528,6 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; - struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2543,12 +2542,13 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); From patchwork Thu Nov 14 15:25:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA224D68B31 for ; Thu, 14 Nov 2024 15:28:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE3F36B009A; Thu, 14 Nov 2024 10:28:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E6A986B009B; Thu, 14 Nov 2024 10:28:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CBC1E6B009C; Thu, 14 Nov 2024 10:28:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AB6046B009A for ; Thu, 14 Nov 2024 10:28:13 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4E028C0BB8 for ; Thu, 14 Nov 2024 15:28:13 +0000 (UTC) X-FDA: 82785080784.04.B19DFD9 Received: from mail-oa1-f41.google.com (mail-oa1-f41.google.com [209.85.160.41]) by imf26.hostedemail.com (Postfix) with ESMTP id DDEC6140021 for ; Thu, 14 Nov 2024 15:27:37 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=sM7OhIVC; spf=pass (imf26.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.41 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597915; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zZIb5wvnGXtO89gOBSt/Tk4W0KJQpsBnaIBuXsS72Rg=; b=7Qk05XueoGfQxpSKfJOSVdzW6oRqAjAYsF3JtiEb0Dr8vS1n1EdogkmPHzTOpHulo4ghHe sOiQM6IJIYYgr0Cv58RG61cKqmW7L8iytKQZUfOzgmQ82z1acixfb/8AXbjKUIOo9War8p NNb1sQmux4UDUc7C1OkY5HWYJKKkXlw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=sM7OhIVC; spf=pass (imf26.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.41 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597915; a=rsa-sha256; cv=none; b=LJUKdVGbK5I+iO+P7wHIM/YHpti6L4lF9kQeVmtKl2fcEiILs2nEe2ItSSRfDhaa5fiepG s4yyNRpErd9VjeWwc/y9lDcRbs04mjtMyRO1gXMecWSNG6PpXciypC6ZzWTqTYBKW0pTXd /y2/WMkBL5dIIoZiqckzuK4LdeEVXlY= Received: by mail-oa1-f41.google.com with SMTP id 586e51a60fabf-2951f3af3ceso447830fac.1 for ; Thu, 14 Nov 2024 07:28:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598090; x=1732202890; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zZIb5wvnGXtO89gOBSt/Tk4W0KJQpsBnaIBuXsS72Rg=; b=sM7OhIVC526DIs4la+8Gd2O4X8z3MplH75exncxqvQBBNkequ2/itF0bsingGtl5yD f2oa45xGfnudKeQdMgtsWSf5VC+AR6TJiNXONXs4Wn0tgH5NO5rBmc80cqvy1ohFMDP/ mrWD0Yd94KAXjvArg0curgULaW5T7LlM8ogEpjwkwkzEtAO+Uf7WhQPfab/u4pPLsBsK isBpNG3gjq1QOYKe1EVxrNzoPFdHAhUpNOWC6ymcuRonIV9u88Voog/umnsXfrmJ2quT xVLQPOYCWMEE8r68gu7JRK7WvIzxOqgU2sOtSdtub6JMBpviDhu21U/f5PIrYxFImDaY PAzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598090; x=1732202890; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zZIb5wvnGXtO89gOBSt/Tk4W0KJQpsBnaIBuXsS72Rg=; b=WZeTVlw3T33Yai83S7sHBg+pKzNN2jNK2MhXNrIktuB8CAvRqSCGnJvHSRgr5qY9pg 0ldHKM4cTaMYh0F3mQLG/HGfI3To0XQXC9Z+Qp8IfoIqPDADPhahIioIoLVdcZgEofEO o7akkDJSjrrmjD1Wr5TQ70KqHnzuXxy5EuZgspKbazQSc65vuc/dhaWolOBo8aeuC08a 0RkUaeHrmJSdtw0Rv1DnRjnBpf3mkjSVx8ztVHGXgqjcoDnyJT/cHKYnSX8P8+i0xfVf JRf+TjiGsvthZ8AZtnwEJllby3inEQ68GtGGLTNUA764244gqcZxAJVGLch407027dZF qL2A== X-Gm-Message-State: AOJu0Yyv7RhM1zTuZoZwX6XgMS93jlG6zZoc6tDFymQceTYCileFyNDG mUw/WuDLcJ+vsAMozOSyZcclb97PwR+1hTNtuTh+7cx/z16wsH87RJfK1JAxk4i5ecYVINU5hgc rc+A= X-Google-Smtp-Source: AGHT+IF0BwKZuS0hD2QtJaajxUzrtQ2SwlZch/PHXd4jMsGrqLjoDD1qWu/LCzOrVtrsjrS2cDAn1Q== X-Received: by 2002:a05:6870:910d:b0:277:c027:1960 with SMTP id 586e51a60fabf-29610376b59mr2544315fac.25.1731598090064; Thu, 14 Nov 2024 07:28:10 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:09 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 06/17] mm/truncate: add folio_unmap_invalidate() helper Date: Thu, 14 Nov 2024 08:25:10 -0700 Message-ID: <20241114152743.2381672-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: DDEC6140021 X-Stat-Signature: q1bmf99bqog813mr8q5ay16ocjp116ds X-Rspam-User: X-HE-Tag: 1731598057-496187 X-HE-Meta: U2FsdGVkX18bCI0F5cl2GpNJhPro18czc9V1HGFzlmBgaprBnDhNSiIaHAwSAFzNre7iPNwrZQgdXecr8o9WQSYK8/2Sxh31UiAtBkrajMZPTkDpW1E9riWAqC6z55MqTeQDYwnMOg5OtE+aEU3Umm8CLUmND4vAn+8VN6mQyNWysvk2d8nHM/0k/MDehHWeAoc74YjamdMwx025kQiZCAc5c1uxi1aZw2VjQ4q+ayrqxuwtxk9VfYg8qJeX8gzYNDlyR9wGdyhfhRbazmsMcQ2PdTaPTDihTi1F8JRbsCq2MC9wQO9mdnx0ob9feJXJNaQWL81x0ok2LhGpP0E5hKKeqbt6KEbFuNs/pd13q+AvhIhczcf1JoMaexKKXla8h0E3toGu1xRhQCo6VoEZImKav6Tbi9wRS1MJ66egGET78AgOUI3rVQNrnq/oxvxObmCoCMnbBFZ3MEtuloBeNaYCFmldpJER6KsDV4vYyzQlqsqLgpbQ4fLgSAAfeQzjvVfx2uve00I22BOl0oVXyyB9VGQnbZznpg+riThEMp3/BmO0SMUa+E/DGXknGdw6ChUwIHi8Cmqchx3S5gcLdqewNVnHwy1uSMaSofkeYR482SJCaXmUvJYtqGzL/DlrPA0N5ovyg4sQFTn5M6Nt4UjiRhxEUOpBNW9mah4BNYWa29W2Vj+bkTi3G0NVN6+W++byUTxi/yVFFblJXTRRHCj5R/BnWGnPR3SrAVkKP0uAUd01VRZUKHeRh/EXaEjCONttRWslVS/8dlOd9ERHW83CDiShZUjuv3WgiXToW1p1tV2OjEc/owkiY87z1TBaPhU4YlN5xGZtQsL23hcigqpMyqm8nELdqaN/pgrVo94I3U0vqhsm10Z2wgh5bSvZusIdDTnG1WTxb7l/a54C6QL8TgeaJkwInXdSA74auu9QOMMfWNLmYyQ42szIAcH44v5kjGMrZ6bXMntlkPq SPwRpsqb ZW2b8cY80lSRLtE6aNOGhlT2N53PMETTxyOnAOh6uE+XW/8eoHPWqLugNq4yUxwOG/bnRJvqtczXz8QL/3EOy2GiyynCvJvqe6VdwjnvUSrzPxHC+aU2/ml29Bgti9MF8XWffTKfZFtNYa48UcPZ1uQj8QNUS1nNQsdFk6HyN9qYsx7l2trFIqGM3+BIQOgI8PhOR3Jcpahcnh3ZWJLNh2AcNoeGUwD8P/5Vem1QlvLn+hUb93Q+tGd/JgD97azfctfnwsQQh8BN3K+4HRneauS/O+WXegzQjImRaWWgLYUip2h8Oy8wy9QTnpR7ot95jVLaINC/6k/kyng50KHhD8NjsFqNjLd9fUD2q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a folio_unmap_invalidate() helper, which unmaps and invalidates a given folio. The caller must already have locked the folio. Use this new helper in invalidate_inode_pages2_range(), rather than duplicate the code there. In preparation for using this elsewhere as well, have it take a gfp_t mask rather than assume GFP_KERNEL is the right choice. This bubbles back to invalidate_complete_folio2() as well. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/truncate.c | 35 ++++++++++++++++++++++------------- 2 files changed, 24 insertions(+), 13 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 8afacb7520d4..d55bf995bd9e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -34,6 +34,8 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); int filemap_invalidate_pages(struct address_space *mapping, loff_t pos, loff_t end, bool nowait); +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/truncate.c b/mm/truncate.c index 0668cd340a46..6ea16c537534 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -547,12 +547,12 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * sitting in the folio_add_lru() caches. */ static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) + struct folio *folio, gfp_t gfp_mask) { if (folio->mapping != mapping) return 0; - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, gfp_mask)) return 0; spin_lock(&mapping->host->i_lock); @@ -584,6 +584,25 @@ static int folio_launder(struct address_space *mapping, struct folio *folio) return mapping->a_ops->launder_folio(folio); } +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp) +{ + int ret; + + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + + if (folio_test_dirty(folio)) + return 0; + if (folio_mapped(folio)) + unmap_mapping_folio(folio); + BUG_ON(folio_mapped(folio)); + + ret = folio_launder(mapping, folio); + if (!ret && !invalidate_complete_folio2(mapping, folio, gfp)) + return -EBUSY; + return ret; +} + /** * invalidate_inode_pages2_range - remove range of pages from an address_space * @mapping: the address_space @@ -641,18 +660,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, folio_unlock(folio); continue; } - VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); - - if (folio_mapped(folio)) - unmap_mapping_folio(folio); - BUG_ON(folio_mapped(folio)); - - ret2 = folio_launder(mapping, folio); - if (ret2 == 0) { - if (!invalidate_complete_folio2(mapping, folio)) - ret2 = -EBUSY; - } + ret2 = folio_unmap_invalidate(mapping, folio, GFP_KERNEL); if (ret2 < 0) ret = ret2; folio_unlock(folio); From patchwork Thu Nov 14 15:25:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DE3FD68B30 for ; Thu, 14 Nov 2024 15:28:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F6286B009B; Thu, 14 Nov 2024 10:28:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A6076B009C; Thu, 14 Nov 2024 10:28:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36D556B009D; Thu, 14 Nov 2024 10:28:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 168856B009B for ; Thu, 14 Nov 2024 10:28:15 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C8C68121230 for ; Thu, 14 Nov 2024 15:28:14 +0000 (UTC) X-FDA: 82785080406.24.7C459DD Received: from mail-oo1-f42.google.com (mail-oo1-f42.google.com [209.85.161.42]) by imf14.hostedemail.com (Postfix) with ESMTP id BF84C100006 for ; Thu, 14 Nov 2024 15:27:23 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=qqgMJeSl; spf=pass (imf14.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.42 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731598004; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LyRvRhMBG1NsG1pjAu7RsorwHLATahyjt2W9R5OMw+k=; b=gzSogBsS6Z+QWyRX1hDahG04qonJ565KuuhEoLe1ojZ+RaJizf7Y9VgDC+a+fswJvYrKpl s4qf3E1XioWMGKToOMm549pGj/GtnshKCd3O1susuGhcOV7GzXbJyY7Sqoz4SMeLdnNgLb oC29PFyT4Om9rY6g59e80dPlgpIyHCU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731598004; a=rsa-sha256; cv=none; b=Cuf2ACFL8unu6amHvRuyzbaCfWWdLqLkB5pUFXw3M0Xmw4O3s7y3TTEDGMFfpnDkDQCewX nL9cHNRSBSSAwJLhepyah8Dt4OArpgyVdbJKi8tdDp5RCLLnJgDPACNfl0ajJHSN6kVyY9 qrSCqfnJTV1C4qX7oJSt0tproGRA37k= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=qqgMJeSl; spf=pass (imf14.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.42 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-oo1-f42.google.com with SMTP id 006d021491bc7-5ebc1af9137so297638eaf.2 for ; Thu, 14 Nov 2024 07:28:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598091; x=1732202891; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LyRvRhMBG1NsG1pjAu7RsorwHLATahyjt2W9R5OMw+k=; b=qqgMJeSleBTzV7yoPOt574T0qTMvDRcio0ANw2tmkUDACEtojwxJ2zTKvqcUv4oZ+/ xoBPwUheCzEdanKzJQs0nQfdn3CKrKRUksT+HHcIgEIZk5QTfLZwxxgzceJooKBUBZcb aQMRkV6p4tBv48V9iKekbqFYR5BdnRUH4IQuqtgoK9Zk0m49/KTP1ydLcx3wm5ap2LXZ v0FDee36eq4z3XAE8oieSSFZeUoOENAc3PMqa+9cb5e5gV1A2pg/RvEWCCWl88ogktrT g7eKEj8mM9LrGnvkvKpmL6TzRxMwOQa4tIUEgldj/wCvSBRT/ByxSI+oU00Se+7JjbH6 cWUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598091; x=1732202891; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LyRvRhMBG1NsG1pjAu7RsorwHLATahyjt2W9R5OMw+k=; b=u15JovT45oe8o1Kjoz5Dtj/Nvo021Hv2Y8bk58d/rViy79ckFomdwZJOfBsetK2sjT ceoK1QD3jOl4Vrx6ocOHmIMTSwHVxJnjDhWPJPyQIAIIJsdI6riBwxUv0Rq6CsyULsCo bi0uototNK3iFT0B82jAhOc4V15/uZCHEkUvOqE1Db9P1wt/eadLbUQJVvxD4vKolDG+ 5vxYtHt9dyFKQ/S75h6kqL9Ix8aaTpnzKqeMqkMZ2PeNA6OzsG4Jj3fjSxLz2iND8QAj 99tDtrnr/c40S9hRChT8N+zEOSi1HLe1KhtNh55exNp5epGJ9ZOWjuHYprhuim7SrYKx 9l5Q== X-Gm-Message-State: AOJu0Yw6839VfT/8A056BxMBoLEDbxZCWHv24pPvOVeyl6Oo2Df+vClh BqEujX2eHAweSDm8MTGIjzix03ZZP8aT4M628jAt3pqM0tab+BuSiewcuyMcw7lgQrwUQVP5YH7 Kujc= X-Google-Smtp-Source: AGHT+IHQ/GenKedH6rwQd+su0Ph/MRWIuzytWarVEOsqT+imV9hIASvSe13ZVKnr0H0iVbl20zcXwQ== X-Received: by 2002:a05:6820:4d06:b0:5ba:ec8b:44b5 with SMTP id 006d021491bc7-5ee57c4397bmr19027877eaf.3.1731598091501; Thu, 14 Nov 2024 07:28:11 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:10 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 07/17] fs: add RWF_UNCACHED iocb and FOP_UNCACHED file_operations flag Date: Thu, 14 Nov 2024 08:25:11 -0700 Message-ID: <20241114152743.2381672-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Stat-Signature: pjomoq9nn77uo5i4puounbw68yct9gys X-Rspamd-Queue-Id: BF84C100006 X-Rspam-User: X-HE-Tag: 1731598043-585705 X-HE-Meta: U2FsdGVkX1+UXrtUFrTfEkvH3eYmd7x3CeCXXEyukujCOc4lfkxKrwe/Yk+e1uBwTBH3jWluBXej9qwlu7mfIFMj628MTRub5naeHXEspSPD1Ik/OSwGWtpR52AOJ0qAKvudODj+Qd8CN1uZnhJ+G+pCOsbIDaV21Q9DuCBnkhZf7WL38UAvbvrsfXPbETy+bBhFVjjxEa9R9ILc57bm3VXmzaENp5+NjFhOUvs2vZhBRZvh37N95xJlDielCEH8pyhtqCgdbdF2naZvKVk11DuPypRvIwJcKgzs5YFuKUuYEFImydb1BvcUCDc6rPcCyGV4gnGiWRGV1CBlCNVg/V74NYIQ9jwXcI2lQYcZJorPrhLNZ0z6/1gQeRNg4aH6ucuFgS8s4qohOn9aLEUG/qpJQ5MiR3YYF0h53uv2DRGoTs2q92pl2X7DgC3VcPvCPhBlhxMtm9pSMTn3l7ZC95UCx8RJVlhcBbal2wXB17i2DFv7Y1afOea9soCO3yZioyciYLwpiPx9MyHeu7yuJdsgENnJ8eF5xpN8yN0lx+Bez5UlUTxhBW3h1e2wBLEPu+s7OzlfirlEMnsQC2eN+riqX+V4KDExy8ZnE7jHhe5DT/GPllkoTPM8C0J5pYy7NCuDcMXclaipCEBcAocQaxWGDQW8c9Oq76k2r/QWdgwTiWzpYY6k/04x+8Upi+IBiC2E13ivXtC8IMvqqpOf+zRjBwA2KnLxp0S0ASNIHK4PwP0BNtQkTBqJANqonhFuACa3qEbmeLzZwCpTX4k2f4K3RgF7Ix0LLyXm2FUcZEpSoaVDefKVRKypCYvnhlvmcNX80hpyh08lw8MooUjirwRA3j8ZO2l48ZhqhQhyRvtqFlIsw1AwoPajZIWNjmQuezsiN1tsPnhsAd2lSudA4uT56sc7pC/qP3OUS87MiG/MZv8V3UrB7AF6xTiUbJlzlv9HL/zZKU34fEK3jXl jLRlGAn5 f5qaL6LjkXV1Rp7gvT+LlGrXkfs+cgMcsgJJSwr995BGOLt/IUZJ3fjePvcWxMl6DHEE7wdcn2VgqzKe0YW8O1x2O2DZiIy3mQIkpsTpWc48ppGKRz6IE72wzedneM/Pw0x6/nq5Q5kP5O29IwpbnMZovKzY9Bj8qh9I0D28IG2QvGXoAoMz4da3KO8Ph4olbJRo8ZGBNAr9ALp2D2XvDLmyrBFdsjyYpKWua9uzdvAeqWhDx5cgGmvSBJ2hCRiFeJdUVIeo5r89qQQn3oryCdi6NgC9MuUcULEPGfnj2YEkLXMXq+yvUHHlnv1sjSqAv69V+VTQxOhTLRQ0i6MuMkTGunf1NQZWN7uDy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If a file system supports uncached buffered IO, it may set FOP_UNCACHED and enable RWF_UNCACHED. If RWF_UNCACHED is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 14 +++++++++++++- include/uapi/linux/fs.h | 6 +++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 3559446279c1..45510d0b8de0 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -320,6 +320,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_UNCACHED (__force int) RWF_UNCACHED /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -354,7 +355,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_UNCACHED, "UNCACHED" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ @@ -2116,6 +2118,8 @@ struct file_operations { #define FOP_HUGE_PAGES ((__force fop_flags_t)(1 << 4)) /* Treat loff_t as unsigned (e.g., /dev/mem) */ #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) +/* File system supports uncached read/write buffered IO */ +#define FOP_UNCACHED ((__force fop_flags_t)(1 << 6)) /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3532,6 +3536,14 @@ static inline int kiocb_set_rw_flags(struct kiocb *ki, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_UNCACHED) { + /* file system must support it */ + if (!(ki->ki_filp->f_op->fop_flags & FOP_UNCACHED)) + return -EOPNOTSUPP; + /* DAX mappings not supported */ + if (IS_DAX(ki->ki_filp->f_mapping->host)) + return -EOPNOTSUPP; + } kiocb_flags |= (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |= IOCB_DSYNC; diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..dc77cd8ae1a3 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_UNCACHED ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_UNCACHED) #define PROCFS_IOCTL_MAGIC 'f' From patchwork Thu Nov 14 15:25:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63992D68B30 for ; Thu, 14 Nov 2024 15:28:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BFB776B009F; Thu, 14 Nov 2024 10:28:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BAD796B009D; Thu, 14 Nov 2024 10:28:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B16A6B009F; Thu, 14 Nov 2024 10:28:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 739826B009C for ; Thu, 14 Nov 2024 10:28:16 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 369761A10AD for ; Thu, 14 Nov 2024 15:28:16 +0000 (UTC) X-FDA: 82785080448.26.A80E6D1 Received: from mail-oo1-f45.google.com (mail-oo1-f45.google.com [209.85.161.45]) by imf15.hostedemail.com (Postfix) with ESMTP id 21CB7A0004 for ; Thu, 14 Nov 2024 15:27:28 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=O21dH7z5; spf=pass (imf15.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597918; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7749KjgMy6eatM2VSy2jowDIab8tvoATfJQmYOPl1pc=; b=TJqjXlxQwhVZ5kypUWlwu7IoI151sx3IJCawAKZN1EgaXVUbvZO+necDNS9xsfGnXJNI5I 51cpMGlQ8sqK6ALK3UiXxKpARCrPq8FE908m/+sCvIn0EAvZ+Qf6YWfqPiKK1lPRV8OIlX jSjAtkZV13e3vBsSQWLJY1kY20tOPXs= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=O21dH7z5; spf=pass (imf15.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597918; a=rsa-sha256; cv=none; b=19S9DG6c8x38zoIhFCDhIcy19FahEQZd5afRWJRfSzEWF+kfgSLO7NV3Cy16GgVApn5rhL LOxxm89JiaFwJcKN9jy2Oh/ypOaAOnWwT4jvWCAXG01PcUOpgDnwnaQvr1rUb4QCV3jUX9 Mbth4Xq+4sFEefzgBOVuheQRIowtp4E= Received: by mail-oo1-f45.google.com with SMTP id 006d021491bc7-5ebc9bda8c8so358733eaf.0 for ; Thu, 14 Nov 2024 07:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598093; x=1732202893; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7749KjgMy6eatM2VSy2jowDIab8tvoATfJQmYOPl1pc=; b=O21dH7z5HMdG+Te68rp47FoXmAlTwO4YHlGdSi4JJioNLXcQQCF2VFx5tuOH6URr8M UqpC6c0mzbID7qhykskG2RzOvGo+RIv/CQKh58bft4AVZp4a2mHI8BY0ETkJDJp1g9VA XBPkFAbNp5sMfO5VTCI5qa6DsM8x+c3A4nrVhCDNWAJnXDvIlvIYaR77T4KYRa/12nrN 1VX4FhVa/zhJJyUcPTCQBxTFtJcmQa2ua1kbwUtJmVs3kbF6B0LfFj57wVTFf0eQyFkQ SOjPnDLvFvbu3BTYX03EyOyGQANyuIUbFApXvbl4up/uIZ649m5v7SbndA7Bqe99gDU+ A8vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598093; x=1732202893; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7749KjgMy6eatM2VSy2jowDIab8tvoATfJQmYOPl1pc=; b=DWWz1s3d1cWyC4G4BnKgBznZ7UBUGcyQvBZAsaC9gLucacR8WfPnlJajSlECN/XxmP 8atwZ1koExPX7SLQfzvUFZoN3JZs9OE5BChl4SG17GYean6XgemWkTcWHPlMYblPzZmh iAc9jy3q3oztbrD0ck3rliwwc38fUZS3to+9yyZaOCu2uJGIV6FTXkGxX8HkPI64jFA1 BYerS5GrCq9FOtL4aWaMOD7OQ9kaU4hnCqZ2a0/fnHFws4kWdvuryWO10mxWRSFa0U+u QyJ2rlxAgbbWdn3VOFxzkR9qGKV84lAF2qVKfR1dMnk5wrg9kgkwrLz15ku+6hxejJZq X4Hg== X-Gm-Message-State: AOJu0Yy8lgeq26DnjrS5R6qzv/wg3qu2O1DqEitd+kEuOOL9jdMU7+AX f4qJ3/k47WQbLcjJZxDuxI8oE+S/CZFFfQK+yw7xZSIorl5zBaLVC+ZY4ESTmcl0HQFm3weC7Vs /zGI= X-Google-Smtp-Source: AGHT+IHV4oSrYqhQg8SNV67NfRcdIlRE7mzCITp5FLKpYfj2Lpw4MvYi01EsizY0Kl5ORLV8Nbza+Q== X-Received: by 2002:a05:6820:8cc:b0:5eb:6a67:6255 with SMTP id 006d021491bc7-5ee9ec3bae1mr2681521eaf.1.1731598092948; Thu, 14 Nov 2024 07:28:12 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:12 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 08/17] mm/filemap: add read support for RWF_UNCACHED Date: Thu, 14 Nov 2024 08:25:12 -0700 Message-ID: <20241114152743.2381672-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 21CB7A0004 X-Stat-Signature: j6o7gw5h7phu5t85t6uj1jj3fobccagk X-Rspam-User: X-HE-Tag: 1731598048-615612 X-HE-Meta: U2FsdGVkX1/IVte4kWLIOogDdprZauiprl/HtSevVb76kPaW0sa7SATJ4CAtnQh+x3IAHEa6fF9SEjy0o4Fc21XbE86FdRus9Plk2Wj98wP9G5P+dU67/fgim/wBJXnq9fLJ9stqcG7LT61WQqZ27oIrBMVe7KbgMbn8ikt8NYGO14kQZcbcL5x51XQQdnn7Egx+m4IVMy5X30BgNWl+mkG5hOGq0/gKZD2wdUcl/uXt2VfIAWetjsFxYpHNi/GQeHoSXf9arKNryBZF+8Xo/+2tOkrh223LPvXNoeXAJqTdFe+JnuRgrtgoZQcFTJYS7PvNiry+elX2Araynmf2vgJFY42oLrAh7XrCvhUQN/DqMApFOztlP6Ca4T4fzjV7MjgGUHbIictOwbfPLLmt5Lz4LcI4KNH/aDkuhEelukgMmHFHSwh/XEyvLIS8NuQfbkPfh/l+CEdZelEJfSTkadcJ+mkTdNGg1rmEQuCJyxY/B3wx9j11f1mnlfdwJp5syjmZwoC6NIDbdRdDLSaFIsrBSf1YdCTGla45IBTymMjva9xL6DlvqlUcQ0eOD4f4TKeBfU4t7mr3gHkZMoFM0/9BacPx04Lf8BwfNjgYBkefJKx8yXQqZapnj/Sqy8WslESmHTx3VQnblWGa7fNOIwe41H/YDRj8hEB7DeXk7g3tNrM/0+/zS+D90KoPqHuWqp+d2wxKQ+e/WePhMz+6VgkaRTuxXSjqAbL0bu8aMfSUAyEAt1cfymsz5ivllYjIrzYW8DmK/MFJWvFo9IF3TN71GuCRvHuKjGFL6rWOOXl6E1d4xXKEFDerzQLNp1MsQxNhYAWvVJd3AO3d6PpsrLRX1sA0hzmZL6wHCJN3GB0UH1qQ6Sg8Lz8UrSk2QDEs7rQyOA8swQ2tcmlISu3vSfgZELpVQOwrZduJ9jVaubUVPYNe38A6w9im8Vu0fLWHxNR2iX4dQUP7fb7ZN60 tcYpfjzg ee0fjv8FweKAB2Kr+XdFKEC85PGzn4zSmwZKnUvyn2IF/hjJGIKsOTqSG7a1cvR5qUXc8CyNZfSBSAqLQ+RtYUsOK0aZ/glwX1SdZK1dkWsdCSyr9BZ+Bj8APCgRIPTbwDzYjSMGkzseSBoBoAj7zCYnxoYmurVI0F6V9a0JK0zx27QITU/8xgpriOv5TEnOW9Fa7LCXPwNpd/vAiuPbHR50G5cM68okr+GMAT5xVk+MuvH1t7ZPhYfEymQDEqzfOXRv5+guP0QsgqFsG85PT1di3kueYpwB6selHnngK/r8jQdir/CtV2okXIpG6eZcWSTcXE9GPp+zEIzENyzm7ahNbyORkmeVxFR04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add RWF_UNCACHED as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousin of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_UNCACHED set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++-- mm/swap.c | 2 ++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 02d9cb585195..3d0614ea5f59 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2474,6 +2474,8 @@ static int filemap_create_folio(struct kiocb *iocb, folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_UNCACHED) + __folio_set_uncached(folio); /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2519,6 +2521,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2548,6 +2552,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2595,6 +2601,20 @@ static inline bool pos_same_folio(loff_t pos1, loff_t pos2, struct folio *folio) return (pos1 >> shift == pos2 >> shift); } +static void filemap_uncached_read(struct address_space *mapping, + struct folio *folio) +{ + if (!folio_test_uncached(folio)) + return; + if (folio_test_writeback(folio)) + return; + if (folio_test_clear_uncached(folio)) { + folio_lock(folio); + folio_unmap_invalidate(mapping, folio, 0); + folio_unlock(folio); + } +} + /** * filemap_read - Read data from the page cache. * @iocb: The iocb to read. @@ -2706,8 +2726,12 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, } } put_folios: - for (i = 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + + filemap_uncached_read(mapping, folio); + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); diff --git a/mm/swap.c b/mm/swap.c index b8e3259ea2c4..542f298d3dcd 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -472,6 +472,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_uncached(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; From patchwork Thu Nov 14 15:25:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73CD5D68B32 for ; Thu, 14 Nov 2024 15:28:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C4316B009C; Thu, 14 Nov 2024 10:28:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0FE3C6B009D; Thu, 14 Nov 2024 10:28:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E43B56B00A0; Thu, 14 Nov 2024 10:28:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C13D66B009C for ; Thu, 14 Nov 2024 10:28:17 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7CF16C11C4 for ; Thu, 14 Nov 2024 15:28:17 +0000 (UTC) X-FDA: 82785080448.10.1706255 Received: from mail-oo1-f51.google.com (mail-oo1-f51.google.com [209.85.161.51]) by imf14.hostedemail.com (Postfix) with ESMTP id 6E800100006 for ; Thu, 14 Nov 2024 15:27:26 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=S0lOFfBB; spf=pass (imf14.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.51 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597859; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vSo08QFzm+JiqHz4hWmAqopvA0C/7UNivlhAphA9PAA=; b=qLs1atll3n4vZhJkHCWTz6BqUUPMbw+ZHxkxNagxDpKbG06hQMB81tp0sw/mP6meTWImoP xmOw4vQfuiBQo/SmjugSGGuj09fhypxl9UnXKpWkWcs7N32uM2NkR6szEANMUGUGdxY0jq tLYx4OMhoMEFBYfrh4yrQKz+Ix0/8MI= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=S0lOFfBB; spf=pass (imf14.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.51 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597859; a=rsa-sha256; cv=none; b=DhGrku9jCBWZVZPkkOzXpJldGcAyZNI1Pqyk5fDqYP5PYOPzJWQH/tShRsNxEnxucsY3tJ 4laLUF5bsw3zfWa2ObK4l2MWnCv6BVOV5d+ZGiEfBsSgvaavyv8LBQXUI43G40luY8zcSV NVa4SdfDmUlQgkh4RBolWTrBZLRLfE4= Received: by mail-oo1-f51.google.com with SMTP id 006d021491bc7-5ebc0dbc566so315420eaf.3 for ; Thu, 14 Nov 2024 07:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598094; x=1732202894; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vSo08QFzm+JiqHz4hWmAqopvA0C/7UNivlhAphA9PAA=; b=S0lOFfBBxzXoXUSkYRVHISzZ5gptp5mcdbE1/h8nF17l12STP4fRmdLbXSaMs45BWl RemkpndS3k2NdI0WsETqsGE6Ix7yqnViwGbe8Lj2WMnM/u9spZ5trRkPkddxY7mNzxpO WuvEU8HIGn10R5/XTaeEJzgYxBFAzdPVdlIhvORGhngP87J8VwPNq6EVC+/qU0hFx51p ViLfYJwVO43mdzlDL+Xa055qwPmpOWwbmwcm8ZZyK+3tmSCcpCeunOdUPtHnu936fVjn ejvt9ODFbdtiEkg1kQLmnDvTy3HMIBPvTu1qxonF+2n+j5Sc8FmDBpqv9uyQZtUCGX6f xcbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598094; x=1732202894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vSo08QFzm+JiqHz4hWmAqopvA0C/7UNivlhAphA9PAA=; b=rGVNLwZVHvPKON6rNzFkw2xDo92i6ePcT2OhC7SEMY3flpgnzDba+QeMJq2XDXIgp3 vhRRPUE3wSejbyR45yZqFFEOhFRAdoeJSuL4ukfZtbeTz1WVS291WjZOOsyElfccQmum +V4d9jHrfAxVkP/NyhYr26Kg5A0YLmAJkRkSJev4TDKfKz2xWA5nLXKOcOqVDb7++Ls2 xuizfIF0pkgKMTBydekjflrzDwIhjjVRfi2PLY5fSxXGMZWYH+ghWlZLJlfyKus0qfVP W4K5JGtcSlO2bK/fuHM9AZKepa6pzdPE6ZDTSaqI9CO0Qc1BnJ0BcAhSEmESDONLnOKf byWw== X-Gm-Message-State: AOJu0YyHJ9gYrWxz/tA6fnG5XSPa/K5N0E7Eeh/5PeiJHPdNkXSrXE4p 3Zv6wRnwTSzIOWyUyfBqCdbCpINFZxLUBkB/g3q8khW241GhkbEInrSdVtQFrtJC8hBnF6dboQO yuec= X-Google-Smtp-Source: AGHT+IF/wU8hdLitBYUw1Xm5/VEAvnGHF/Lmj1TdlA4Dk/xHse2V/nkks/Gl33gMzxUMuX6RfmQ7hg== X-Received: by 2002:a05:6820:983:b0:5ee:bb2:bdd4 with SMTP id 006d021491bc7-5ee57b96aadmr18195949eaf.1.1731598094397; Thu, 14 Nov 2024 07:28:14 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:13 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 09/17] mm/filemap: drop uncached pages when writeback completes Date: Thu, 14 Nov 2024 08:25:13 -0700 Message-ID: <20241114152743.2381672-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6E800100006 X-Stat-Signature: a487xsezazcxrbfa9mdts8ntxr9xocx5 X-Rspam-User: X-HE-Tag: 1731598046-891501 X-HE-Meta: U2FsdGVkX19XXtggrG+qK5X6ZIGYAxYB9iNoSuEHX02EExh4aTj6/tkxWKMF1Bi9sN3mD5PFo6QUIRmbARwYm+r6gMhziSEq75x522P9HUac9Tl9qFKeOYofYlJRw7j/yzCXQiP47CH7yD850FnE6FTAni/kQaC69RqtydS1gb/1TCiTHlAdvkjA6ucmttVuPn5eWp2Rxhzx8GMxO2I9py+Ad4lwriCKr3ePq25OflZcR/m5/IcoMzWb+l54OAJMFXARdkMNZq41GPzSOcpHEL5hGLFYrPCHXmSWVWB4FED7YuL5aZOgITLC99ehbPinPtXRPvvIpAF63cV4/MIelzkY/6sGKgUS3UR1AwdezQ+TKit88TPgMFJu/oGeXOJx1GaMTQM2m1pArpHRzhzV7igWMMngY23PtXHInEe6n/TRzmscvwvdIjCxCkXdNa/SWkc3d4hafsfBxmUjb2f2Lxovktvn0fcSeegfJMDzYGMHT9All/CP2UqHt8q3v9oUksAEk7vGOIslXCiyp1Al51bX6qQzgULPqqIJ2wjSMKLo9idEFfdD5RrvZTIVWmudFBv+ga8yuH3DgT0z1eLW0JPv5JRVwd1F1m2uvq6/V4xXvQq9GqwvWOUgjO3bq4NwMbJOclr3GNxhE/oICM4HX9m/6FbkGcwmJNNeCqtI0LC+AlBEHrK5giszQdrVAY1rZhVFDMPqM/TTHdiSzaUjgaM11qmHvM/eJ9qNDfaS6PPqvIc7q29SFb4CR8ucWrM+vgM81exZPEt4etdHyxOVpxaOoyiQf2nTkiYoVVBRJRpGSXhC+8tbOqXDRtVmvCJnx7gsfEiy+6FfVe0RM67LZtqee0EPqhEarazD4+PaitNiq0l7IBIpOJtzEUc1H55nFwIA9ftO6dysBNP1ifjufX9mwn+TD2YMhXNX+i+O4d/UknYzG6GhLJpQq39hivQ9L8JEM6HO9/6BOdVY70B Hg3EeKVS MZjzlxwQemB4ZW68tT8Gz4kuo7Um/ZxJdkdwsCvXF4gFcIcJtj0yg9twdkUqDleMzEieGq1BEwyfI2/tasJcRVB4qJA4GATQbxkVqtUeCc654CmU/PgM4RONpEs+ud+LmdIY74ATpmKPDB4ks1F5g8qS98ZEGHIGj8YP7UtIY+EtsNuf4SmisdIaC7EG9rZbmyhecVvvfS+sCoZM4wKIMr7JUPibl6B91xkUi8i72ntMgiY+656ANwqRHzsDxECyLdw0jFWVcXaPEfEgru9mW3ddGkmdPQKRyCQv3L7SwNdq/Rzgip9YSyB1z0g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the folio is marked as uncached, drop pages when writeback completes. Intended to be used with RWF_UNCACHED, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 3d0614ea5f59..13815194ed8a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1600,6 +1600,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as uncached, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset uncached for that case and latter completions should invalidate. + */ +static void folio_end_uncached(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_UNCACHED writeback, but + * can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1610,6 +1631,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_uncached = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1631,9 +1654,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (!folio_test_dirty(folio)) + folio_uncached = folio_test_clear_uncached(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_uncached) + folio_end_uncached(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); From patchwork Thu Nov 14 15:25:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D0A6D68B31 for ; Thu, 14 Nov 2024 15:28:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C065F6B00A0; Thu, 14 Nov 2024 10:28:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B83976B00A1; Thu, 14 Nov 2024 10:28:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A01176B00A2; Thu, 14 Nov 2024 10:28:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7A16A6B00A0 for ; Thu, 14 Nov 2024 10:28:19 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id F12FE121314 for ; Thu, 14 Nov 2024 15:28:18 +0000 (UTC) X-FDA: 82785080532.23.9654A3B Received: from mail-oo1-f46.google.com (mail-oo1-f46.google.com [209.85.161.46]) by imf16.hostedemail.com (Postfix) with ESMTP id 1FA0F180015 for ; Thu, 14 Nov 2024 15:27:32 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=SzevJp7D; spf=pass (imf16.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.46 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597952; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xBThbwgpFnDYguERwRJ3Nd3OfeHrc6kNH6KxzsoIqNs=; b=zNoNHX/O25Hw+WPrFzjTZUyWth87hc8sN6zQqRpBeN7C9Sf+bqcHCrPFmyCP734bDUZjja iaKN4NaCISfet7RnyLuM1VlqAIy93tKYcihJWHUUrdyBp2en3Q8RkWzqH4dGy+XS5AuSG9 wZmrVvZlg/AlepBCOKrhOGngL5xZ/eo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597952; a=rsa-sha256; cv=none; b=1hzen1qGQnNJzylEvQrJfb7uRrGWWZ9DtHqY0zHbKI20jY7PgEL+Qi6OEMHskNbMjw+hh+ 9wgQDWd39kAWbeA9N1RDXr5ag0nYaQ81CxvQr9NpBmb6q/A9yk1r1eHTZ7I1TEacJYqlqX lsGTfgHsib+uaPcDkaKMQ9YBU6h3jak= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=SzevJp7D; spf=pass (imf16.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.46 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-oo1-f46.google.com with SMTP id 006d021491bc7-5eb70a779baso359155eaf.1 for ; Thu, 14 Nov 2024 07:28:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598096; x=1732202896; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xBThbwgpFnDYguERwRJ3Nd3OfeHrc6kNH6KxzsoIqNs=; b=SzevJp7DGBDUdWfKncudqYOWXlOL9pjl8BUKgF0C8YE1LhijIiX6JywxCKsBiIuFam buE//0deKgrl2Ph5vUq8lbX96RQHpU4AigHsvb3rT78yYLxMaioLPTj8t5tUmW72dUAn khi/VS94lHilu8zRUv3g+qpSTrpMPgdX+8ysqE+tpZ58EU6MfBftS9QUjxABMTApNxHo 69smrTTMmeh0ZxbS5T36gT91yN+RwYu7vULbUiIxUsXvW/nGn9HIvnXZFKHDfHkXKbJl YIn2IQWmtKi1Sq24v+aKvWKChxcZiJ2fnrxYGPIyy1djOaHfxw0oz1qH9hD+AtGh35nV Q3qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598096; x=1732202896; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xBThbwgpFnDYguERwRJ3Nd3OfeHrc6kNH6KxzsoIqNs=; b=LL1Tzyx9p/VDqUKFZtZyTyE7ATLiPX6s1VqlkBSShY6s0BzDZiFdAhnyxvTgbZIIis TwMSjeTQLxzOC/GQkHO3hq/wH7of4nyHuvj7M+fm0X8m25NOECx0K3OnZGxMqZXPXYsW 4RwdRdf735nSm3QyK56NPdhwOlNS26VVtpKUUtuQ/aMTW9uM4caXns9DBqmjNV341JDX DjRW04KlEwN9AwzNKnYpBO/IPMG0e6DZFOcqYCMXdyAaeuKIn6RciipgDKvw0H3E5cy3 QdLTLglkcg8edPn1Zbys68jBuwYu6hnWxm4iRhfr3bJgXGr1Esy0Hu0h8+XQyRTZLFIw 5lug== X-Gm-Message-State: AOJu0YzHaiUeEfbRs1/Y41RW/uBztgkCIDzDsGJxD737KQ3FI6YYbClV 7iceGTn6u5/S8733LetC/ke2ff7BRAv9Ok0CD1iJNuUCxHgnpZDti191/xXoAUVFBIMX/eYbH6k bQoE= X-Google-Smtp-Source: AGHT+IHjZdeOSBooCNtlJS5s8uHCzSyESjo+TDYveTg1n/VGl8pzguc4+c1Edpma/7ckXWFt8H0gGQ== X-Received: by 2002:a4a:ec4c:0:b0:5e1:cd24:c19c with SMTP id 006d021491bc7-5ee868b0bf0mr9239441eaf.0.1731598095749; Thu, 14 Nov 2024 07:28:15 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:15 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 10/17] mm/filemap: make buffered writes work with RWF_UNCACHED Date: Thu, 14 Nov 2024 08:25:14 -0700 Message-ID: <20241114152743.2381672-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 1FA0F180015 X-Stat-Signature: c56iriicug7sbh6nr6zt3e738oy5hmsh X-HE-Tag: 1731598052-874731 X-HE-Meta: U2FsdGVkX1/CCEITTiJRlbo60Xo6e25K/j9GeIkYEyh2BoIlDVVpuIlBhd52v7ZqViYJyngmKbWLlR7pZ8WKRMXgsR6DNi3RcL62jlqxsMQgtRV9vZOZC9g7LjPtb457YBFRRovD5TkWTvOhV6yElo7FeE0tRenbtgYK5MTeOGeKgcpkMQbI3epDEoRW0X2QK0CF2yCpKa5Rh3P1m0kgIJuSwSbaHwJKRjiXNKUFyvOfUIEzF8dKzQHALDNibnk8emkVx0ERgukBSd3sMylQG+x+PqtshhD1J9iPtSanByYGl5ri8YhD2NPYznCPhKvegdy+j3bZ37Al1Es3u8sZk7xqwrVA77v9TaS+8xPEPyfmcY+JQN1v9zor/WUZgWo06nhWdbhjgOFN53lTtQ7lt3u7aSj22+DwETUopJiRt7gPHpB8CRTeevAxVI5yVSTDfmQYd/bsZEMz3GwI2vSwjK005q5mHFyiwJDH85J+olPIYoej3ttjYufv+jDEGoQMAkaCo9HaEcp7WgLUSN7FM/u7jrzyb6SvAuAi00d3Mfai7VWQylrqgJ7y5nEiehIIlyVfSGGqOoOT1+5zhGUAgNeQyRA7oNT9isQ3b4uApRrctM4cgU+WP6jZ2Ug52I8/2XH17UdMIKL5sUbgq36pdVp4aB95NKI66Z99E6jBVV63V1rpTJ+b6N0Xwnegm2nspKAQYYSL0PfvphAbNKLK369gFRBjRaYBMfQmpg08EVLlAOwbFND6r4kxv4FRAXuQUr+/OBwjtTJuYX9+JDAGmL7SJhYNhKVOBcZt1ZhLjDtnru1MDFS4gRZaY9BDddd2sedJoJn9z8yTTrW0YAYNvzCuW6TfbWa9NZPnknvFP3ROU5c1QL5TS4q3ct7ftaiAExGijMT9WxrHfRggKr32eh0LYSgWiNJXu6KDePITyGOtlXbc6TxnYagvnGOp4DqQON+sWpWdLlI1Ja87uo3 YS2o1zkh tb6PH0sG+ZDe4PwTZZpa20sl5u48F5mZCjP6dwzVw/v4hIxEQg8VL9XkW9DFRo4ciPTqDeM8QSKH5I72A+Qi9SF5ScCwZnC7/P4zJk4rIXhmcu12XRvWlslrlN5zPNx2GJlELMqQP+w62ZuyxzbNUQqSmyDlFa9edFobA1PjlnWO0vuGrUo1ZiV71Bf1LVxmdSAI0+XI8ZeIVi9qrFVGQGGTVWVhIp2xFacpk2VjMvIv4TrLpCJox/QvoHxs2cZ2ppMggTD+37TIfoEU410D1M7cHSY9DjDaDLWu6COkrWrRA8z2TV6yT8IVz/ZDM1UV7zRj+r5vlxfZ+kwawF1M4phWCS9PfOiO70bsv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If RWF_UNCACHED is set for a write, mark new folios being written with uncached. This is done by passing in the fact that it's an uncached write through the folio pointer. We can only get there when IOCB_UNCACHED was allowed, which can only happen if the file system opts in. Opting in means they need to check for the LSB in the folio pointer to know if it's an uncached write or not. If it is, then FGP_UNCACHED should be used if creating new folios is necessary. Uncached writes will drop any folios they create upon writeback completion, but leave folios that may exist in that range alone. Since ->write_begin() doesn't currently take any flags, and to avoid needing to change the callback kernel wide, use the foliop being passed in to ->write_begin() to signal if this is an uncached write or not. File systems can then use that to mark newly created folios as uncached. This provides similar benefits to using RWF_UNCACHED with reads. Testing buffered writes on 32 files: writing bs 65536, uncached 0 1s: 196035MB/sec 2s: 132308MB/sec 3s: 132438MB/sec 4s: 116528MB/sec 5s: 103898MB/sec 6s: 108893MB/sec 7s: 99678MB/sec 8s: 106545MB/sec 9s: 106826MB/sec 10s: 101544MB/sec 11s: 111044MB/sec 12s: 124257MB/sec 13s: 116031MB/sec 14s: 114540MB/sec 15s: 115011MB/sec 16s: 115260MB/sec 17s: 116068MB/sec 18s: 116096MB/sec where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages. Running the same test with uncached buffered writes: writing bs 65536, uncached 1 1s: 198974MB/sec 2s: 189618MB/sec 3s: 193601MB/sec 4s: 188582MB/sec 5s: 193487MB/sec 6s: 188341MB/sec 7s: 194325MB/sec 8s: 188114MB/sec 9s: 192740MB/sec 10s: 189206MB/sec 11s: 193442MB/sec 12s: 189659MB/sec 13s: 191732MB/sec 14s: 190701MB/sec 15s: 191789MB/sec 16s: 191259MB/sec 17s: 190613MB/sec 18s: 191951MB/sec and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write. Signed-off-by: Jens Axboe --- include/linux/fs.h | 5 +++++ include/linux/pagemap.h | 9 +++++++++ mm/filemap.c | 12 +++++++++++- 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 45510d0b8de0..122ae821989f 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2877,6 +2877,11 @@ static inline ssize_t generic_write_sync(struct kiocb *iocb, ssize_t count) (iocb->ki_flags & IOCB_SYNC) ? 0 : 1); if (ret) return ret; + } else if (iocb->ki_flags & IOCB_UNCACHED) { + struct address_space *mapping = iocb->ki_filp->f_mapping; + + filemap_fdatawrite_range_kick(mapping, iocb->ki_pos, + iocb->ki_pos + count); } return count; diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d55bf995bd9e..cc02518d338d 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -14,6 +14,7 @@ #include #include #include /* for in_interrupt() */ +#include #include struct folio_batch; @@ -70,6 +71,14 @@ static inline int filemap_write_and_wait(struct address_space *mapping) return filemap_write_and_wait_range(mapping, 0, LLONG_MAX); } +/* + * Value passed in to ->write_begin() if IOCB_UNCACHED is set for the write, + * and the ->write_begin() handler on a file system supporting FOP_UNCACHED + * must check for this and pass FGP_UNCACHED for folio creation. + */ +#define foliop_uncached ((struct folio *) 0xfee1c001) +#define foliop_is_uncached(foliop) (*(foliop) == foliop_uncached) + /** * filemap_set_wb_err - set a writeback error on an address_space * @mapping: mapping in which to set writeback error diff --git a/mm/filemap.c b/mm/filemap.c index 13815194ed8a..297cb53332ff 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -4076,7 +4076,7 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) ssize_t written = 0; do { - struct folio *folio; + struct folio *folio = NULL; size_t offset; /* Offset into folio */ size_t bytes; /* Bytes to write to folio */ size_t copied; /* Bytes copied from user */ @@ -4104,6 +4104,16 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) break; } + /* + * If IOCB_UNCACHED is set here, we now the file system + * supports it. And hence it'll know to check folip for being + * set to this magic value. If so, it's an uncached write. + * Whenever ->write_begin() changes prototypes again, this + * can go away and just pass iocb or iocb flags. + */ + if (iocb->ki_flags & IOCB_UNCACHED) + folio = foliop_uncached; + status = a_ops->write_begin(file, mapping, pos, bytes, &folio, &fsdata); if (unlikely(status < 0)) From patchwork Thu Nov 14 15:25:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3DBBD68B31 for ; Thu, 14 Nov 2024 15:28:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D85346B00A1; Thu, 14 Nov 2024 10:28:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D0FEF6B00A2; Thu, 14 Nov 2024 10:28:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B86096B00A3; Thu, 14 Nov 2024 10:28:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 965B86B00A1 for ; Thu, 14 Nov 2024 10:28:20 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 52A1D141153 for ; Thu, 14 Nov 2024 15:28:20 +0000 (UTC) X-FDA: 82785081078.16.87092EA Received: from mail-oo1-f54.google.com (mail-oo1-f54.google.com [209.85.161.54]) by imf27.hostedemail.com (Postfix) with ESMTP id 344D440016 for ; Thu, 14 Nov 2024 15:27:33 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=D7jUcfcN; dmarc=none; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.54 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597920; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wvrv44z8Apd8qqgxqZBS6Do0FSmkBjZMAMWmq/c36cU=; b=EMTbmCo6U072C9beREeDHrmvLHb4pFp2wnuTRpE3pxUvlDpzDPM8pO0NqmhiSfWT/OpZvy zOOqWCjhey52t1YQ4gEWy98d0BZ1zhmh1UoZlSA9XsaN/BW/DyTiZ2aXVWBz/R60Z9Bx1h qgIVS2w2oabmyRvmb1Cu6KCOLoSnDLo= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=D7jUcfcN; dmarc=none; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.54 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597920; a=rsa-sha256; cv=none; b=basBuljPgur5nS1XVko4ccT0Beetf2OviNf3atjM6rq/TKDKmsNkG6Ubk2XlDo1/E/VLfV dKpm29CIbiqYSbhymXl7gM+TkFj/iNnlHmfd5IkKD018dFKOi3qkzxJ3T6nSigvV1LqNyr fT2zlLlvHM9DJAtJ8iT33YyXOloSOQU= Received: by mail-oo1-f54.google.com with SMTP id 006d021491bc7-5eb67d926c4so322169eaf.0 for ; Thu, 14 Nov 2024 07:28:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598097; x=1732202897; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wvrv44z8Apd8qqgxqZBS6Do0FSmkBjZMAMWmq/c36cU=; b=D7jUcfcNQnlSayxpbyns3fxNzokeaCDukiZSz/tC0sQZlBnkGAf9pFoO+oSkO58+Ko 3qeiGsKSRFmku4cCMDQVRUZDXNoQCi7+yP/Jm7xmGrumKsdWmy6rXs/oVYtbbDUFHVeq T/uruzoIsJmKFBsc48eX6wOup7YkTh1EdrYdUzDqC6dlCIQwR46qq3Vrvq+kS6XJMqz7 vUgJSTZlnGqqAQplqYgAnaN+J3Fi4CuuLFdtKVUrYzs6qA1ddJp1y50aRoSvUvz3fr2o PRMQJGTwknmE8Hqqtu2bpFLmCC3BUhEQYWEX9ebkYo/eqA0BJTzDQV7J+cxDXYIYZoZl Vm6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598097; x=1732202897; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wvrv44z8Apd8qqgxqZBS6Do0FSmkBjZMAMWmq/c36cU=; b=Srutm/P6+Qu4XRzgk1czNGrF6dxrgcUQ8MQJoR5kLDy54MyUjZCg9JxXVIhH2QlN87 Zi7WTRDa5i0ZvZpZUzq5wk4ybBsSBLsL2s4ChxyB9d5SBEmHAHedhd1eP/QmQ9+aHQ/1 GBERkA3GewGgdliBRFE8tnuggIUOhxY6ziizy/In5MiWZ5rFKwNeP8PwnBRKJvyzaNIh w21MVKtlX7HVLFtex7v35a9e5veY8Rp/UMIwRHEM+S1MlBP4mp6Ra6GTJ51WJxD3JMFd E25GGGtForDyXeiD1AXTqvLuPJTHFF3WvCpXRO5KyALuX2FSc7WrZPZjKApc6wPQxcsM es7A== X-Gm-Message-State: AOJu0YyzHh1y+4eS6h9LxtbUY4CX6Z/AnLHyD+koMCwah4E6OB6BGWwU G2YcD3SwXQaJa6Novs/FCCR89zxDCNY2A6mPuqiQL8cwrgS7D7pOHWYmm6rhSU1Fx08JljfrLke 6D8o= X-Google-Smtp-Source: AGHT+IEaTjdnOMAkEmoEcJnRXLnfxDNshZw+Ax439RAi/pKdYn0q0me9qe1oslEKsbvc7WBkYDa4cA== X-Received: by 2002:a4a:e906:0:b0:5eb:6b3e:ce7f with SMTP id 006d021491bc7-5ee9c6c80d1mr2431138eaf.0.1731598097161; Thu, 14 Nov 2024 07:28:17 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:16 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 11/17] mm/filemap: add filemap_fdatawrite_range_kick() helper Date: Thu, 14 Nov 2024 08:25:15 -0700 Message-ID: <20241114152743.2381672-13-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 344D440016 X-Stat-Signature: k4b16414obsaxt4ob7ffw99z8z869bpe X-Rspam-User: X-HE-Tag: 1731598053-803393 X-HE-Meta: U2FsdGVkX1+cvCY54b+cniLMPnyU987zJVarRm929nUp9MC7CJpODmHgas3Orkfut0rXgHBWxmh3b/WoBdgV8+RhKx4wSaQ2MHt/sjeh8jFKtXdnASAl/dY1FP5BmGil90wtCOIPg8vYZ+x+l8jGNnVNcHaOeYcJ6PBhr37ECMgnFewj0gMfveMV+obHMJKgg9ZXeaxh5uwngwJAuyOzb7eAN/FscyTCyfedxE06w1THR9F5Y3o/ZILkKnlLic0eOIFVrAXc6Mkoo1JdkTPlXNpjcI7lWKDEJ566FFWOWv2dc7Smx504h/ElS5vraOheKBEZiVB1X/2JE1dTPICRVner9YiNGqy0kjp/auCdvA+NMU1p/r2/L2mA6KqZ+S9DFkD9SO/IhN1cE1bMcjNSjp0Kt6Y8mzyeFTzoLIzQnaA5edsh1GVWNL/oWOYMZ12YVKladaHWVETtpufkjk1p/h9gTAbfZuMCw8D0nUOCb8TYK+gAoe6LnmXkuw7WvNTdoPmr8VbkKdHDcZrXwr0VCDGPbkJTs8W0EiCy9EkvOwPabGk6eRS0Q/FG+nG5LylELvsj6Ra3s/OOyGKdCrLc410yjORp5+bCEEm01YD0v7PzyrTvQxh4aknQIin0A+oyNLRh9zu3aH24vhrWnvWyh+njg+WEHPdb7MPa4RphUFG3l6VIurRhcIt+abNrs5UGkXOnjnam834VuKZBFroWiHbrNmsRljNiUghoUjqH9uvWp4+v2AwyWPcG+ZZJeeEIBLc2R1ihKg8hGcZdUndCZg4LgJ8rltn6CKvY8ysI7cDgID2FRpC86v8uF8T38MQwmd3UOxiMOvL/6XdDRkfQGAqytj2keVj+DpKscvVarQaxfxNQFKY3G2orZSCE7vUj50/uCuG8NKcs9Idmm9ON24gURe91vxFSt3z0RyGsFxjvIIiWcBnCr484K3tkVzMdB/9BIP8KS7R+uxY7afZ zW4IgC1+ hGc09C8vHOQlX7lZTH8CzexgwFd6EgSFuW/pb8/142H8cPXEid4ledKy64Mzb7srV/u1xJU5Lk74wNoqHWZYxLhd2Md3ozAROQex777AiY5Ao/bnBWhG7HC13UCPilrYbwJ2Lf1DPa6Xhqtl/lpmgQvin+nv1DnyvlSMoVa7gLuD328JDCjvkYFWhmHMxOYvE68f7tTBtBiMAKAaCa+JLIpAy/+CEUKK7ILPEpXDPuBkendPUggpAJ3qqORkF0byX7GahmNTrEYJhTSFHBX7uukxk8py8rbdzkpjz2Lv7s6V0gXhW624jdhroDYV+xq2lhxXvdhr2TKOKdPcDTedmms97+X7iR+x7WUPM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Works like filemap_fdatawrite_range(), except it's a non-integrity data writeback and hence only starts writeback on the specified range. Will help facilitate generically starting uncached writeback from generic_write_sync(), as header dependencies preclude doing this inline from fs.h. Signed-off-by: Jens Axboe --- include/linux/fs.h | 2 ++ mm/filemap.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 122ae821989f..560d3ee1bb8a 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2845,6 +2845,8 @@ extern int __must_check file_fdatawait_range(struct file *file, loff_t lstart, extern int __must_check file_check_and_advance_wb_err(struct file *file); extern int __must_check file_write_and_wait_range(struct file *file, loff_t start, loff_t end); +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t start, + loff_t end); static inline int file_write_and_wait(struct file *file) { diff --git a/mm/filemap.c b/mm/filemap.c index 297cb53332ff..a8a9fb986d2d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -450,6 +450,24 @@ int filemap_fdatawrite_range(struct address_space *mapping, loff_t start, } EXPORT_SYMBOL(filemap_fdatawrite_range); +/** + * filemap_fdatawrite_range_kick - start writeback on a range + * @mapping: target address_space + * @start: index to start writeback on + * @end: last (non-inclusive) index for writeback + * + * This is a non-integrity writeback helper, to start writing back folios + * for the indicated range. + * + * Return: %0 on success, negative error code otherwise. + */ +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t start, + loff_t end) +{ + return __filemap_fdatawrite_range(mapping, start, end, WB_SYNC_NONE); +} +EXPORT_SYMBOL_GPL(filemap_fdatawrite_range_kick); + /** * filemap_flush - mostly a non-blocking flush * @mapping: target address_space From patchwork Thu Nov 14 15:25:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3C1FD68B31 for ; Thu, 14 Nov 2024 15:28:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 549A86B00A2; Thu, 14 Nov 2024 10:28:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F65A6B00A4; Thu, 14 Nov 2024 10:28:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 321136B00A5; Thu, 14 Nov 2024 10:28:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 07F356B00A2 for ; Thu, 14 Nov 2024 10:28:22 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BA7941C6E9A for ; Thu, 14 Nov 2024 15:28:21 +0000 (UTC) X-FDA: 82785081540.02.1DBAE90 Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) by imf05.hostedemail.com (Postfix) with ESMTP id 3F164100016 for ; Thu, 14 Nov 2024 15:26:56 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=ZWtWqjMo; dmarc=none; spf=pass (imf05.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.50 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597969; a=rsa-sha256; cv=none; b=j1PUCh+//BX4bBOxXrGHpmOGdqHc1+MV22FyCXDspdll5hT13azr0vBSvzfcYPeUew5BkI KPCaLtZvGQ83jrsyn8jnTyILgyDCGX7CdRJ+3Ea4Ed12K8cpCDAZIzR8Ooh6tXY2S9by5G UbkAQPri8+bW3GloZhrE03CXP9Jq7TA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=ZWtWqjMo; dmarc=none; spf=pass (imf05.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.50 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597969; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bh4vHLcSKYCCRkbDjlhjn1EJt3wp9QgKXbTvV2Vr6u0=; b=vVkNKYRMfKhQZE4DAv6TdiflWonmPq/1w9xPa8iIGgfbXU3bNRm3MeD1aRT94/mb03SpZJ dcsrYd9YFBYxfZRquz46m1ajCYOBENtLP5LnOoutslftSUxKXZxE0Rn7+OxX6ymH4AnnVH 11T1n19DY9qgLUBHVHR6R2Ykb8Y957w= Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5ebc0c05e25so344713eaf.3 for ; Thu, 14 Nov 2024 07:28:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598098; x=1732202898; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bh4vHLcSKYCCRkbDjlhjn1EJt3wp9QgKXbTvV2Vr6u0=; b=ZWtWqjMo0ErKdOUlp2TZrCQAP+kQ+4NbyFK7dy1VbaDOIGdIU6V9Wu9hXO4Q95NLXv xP49tL9wTqZsxPiyAqHFIRm5zrkroT3Ww6sxp7Si+r6mGbNq9cDFQSFbueJAkLVSULw9 0hopTvs3A3JrEwSXHLjuo5zZm2zQ80g5dzi5cnGu1tdUIKFRse7AZlWHMjc4m4K3XeQb hbw8NlrlmTScS8r51KNJhd0FqWY//xyVNAtZaVGj1R2XPzfdRFsxAVsPV/n88z9QXBA8 OYzrgZaNQ/bnI9THx7nJy29gErNttQp6/syG0E6uDwX1R8FRDgp0KDF4xIk1brY94r/9 S3NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598098; x=1732202898; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bh4vHLcSKYCCRkbDjlhjn1EJt3wp9QgKXbTvV2Vr6u0=; b=Occi9A4FId37jV3hLwb12G/4I4JGa5FNZgK+2sJpWhlzZgfiasQusqf9TGIA1skAr9 m6kfsz1hg3AVq4CiAoxse9YITpCS/dV0ptRQsF6Sb+Wa/bW9Zxm/07OSybo7+Ass8IQ2 OLKg/9QSp81uCHNzKO8yXkPhdlf/2tvoXMlowruLntj2jIL+YZY/8vijT7vaWR7MTUpo EXUPI+kz6Eu/ewcNMfk6YFbYHCPdo2mUt4KZy6skS2Tab1rhUjQqMYNI8NDGAblXDomC tNkLjgAZyV2GqurRY5VGx+F6ckEUqB9jFJccrcyLBB5WYhK7wbneOhdTTuCQwjnqiKbW 65Ng== X-Gm-Message-State: AOJu0YyXC0V7dcu+7hodRZzqrLQ5+lZ4ebDWl4RwhfvsuCKXi4D7f7hv XW3o467vCdXqO84l0o/8eLXmr3zqY2Y9Rwg0oRh4Wk97bmlBtMfdnuSSj0Y3IjqHtlmUV11cBqc 5y98= X-Google-Smtp-Source: AGHT+IECtJqHxvsDZk559Jqz444iSDToPVSz/2QlEeAwby3Gwu7O3Iv2UCv4dhdd6n28ENyvjODyaw== X-Received: by 2002:a05:6820:1e0c:b0:5e3:b7a6:834 with SMTP id 006d021491bc7-5ee57b9c8e0mr20035625eaf.1.1731598098551; Thu, 14 Nov 2024 07:28:18 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:17 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 12/17] mm: add FGP_UNCACHED folio creation flag Date: Thu, 14 Nov 2024 08:25:16 -0700 Message-ID: <20241114152743.2381672-14-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3F164100016 X-Stat-Signature: h8mu4gx36a6ks8ju1h8zfkgz45mon8w3 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1731598016-165024 X-HE-Meta: U2FsdGVkX1+CmsUddXByCblPDDtI/weFCeqez22V3Td/ucB5fWhc9LfyEEPjmhosPsaCN3lGjMoEzMvmEsz5ET3IVVcSbEPGdE0rzhusG88EcNyi0WbFWzRGiE2GCuEchNIN8Db6zA2oTREEvF+x1m8olwzLGyQRqIcyRJDcNADyz/60vMH0LCLliXNOunfWzGL4CKZ6Q4YRlnKvcHH9u8M6bH0SNhwPptipSZRNhu7LBzGowjLp9eGVEP77/AXHBERxsksnN3WmqJUAQpm5zSlfZFENK4OY4XTTmv9Zg5hRmrhKeOncnsXiTHXEFokBWnOXk7WptmNfuMaMPLseo3VepsSM7z2nkTBcBTsl74bKCHx4VG/Gf+PMpzFqOSvn6suEGQwnLUFMc9NNB51hru3Q3HCH3tyI+oL7X+I04un6/07sFEWYAOr5LDOue2DVdEyZdBiJsW5FTE8dOX85bTuDPKlUn7NTRIlRLYMHo0AbmoUXioTadddRR2h4EfannYHj65xwc1dBUoGWIJlC4NkYa8Lw/CZ7AfrMZsPnqR7t6+DjwxcdqTKWoNOX9mZG/dzYOI15u7D5tUdbr6XX79pCwQzWfFl8w1FRZ/M1+blfU+fMJMcaAx4VVNDn8sfvZ6YlvbfWanQ4xVIBsYaPxM1X+hBQWP1sMaxoSu2Tt17GOkgD9kAma18Fg6CEgATFV0ObecTwHPaQUV+plTxsWttbz0JW5oOp5CdJjOTcEHt6f8EEqCVMvpudDEn6aIK68ND3Zu7HALn+YMxtiHuiPRX8NiGtZCe4isSSgCoG/TOHo1TftuO20YB4etHDrw8SqSnSHngBNhe6+8T3nOj9qhlicPFRZrwoDwOqQ85QvT89ZBgkRmfeFI0Q1BD6QnceW1JtyppsvFDhmSzQN1nAFff3we4736fpUOHNU4zamlshI2e0KdbeOJ3i5u5UZRWCbpqWniL4WgTJFhcYRrS QbJWM6g4 +SLfpf7snTWdWMlTBcWnv7qhtW4fIeVM+75nbI7yknxXljz/QvWDzqltDn+uSEPl2AHn85ZhYW9XaexkVvMiEtliPNupEm3SdyWdsW3U96GZgVCoL+W2ynphDXJNRfvw3ACurifE4ZpNbnHevH62whIOFr1NFnDWqEyOxAhH/aCO9XCyWJ9dTjctKvcUulLicSH8v8xsihYQPKlSfmNqq+DODMdMFlz7y3LyIgsacxB5Bxha3Q2FQ915nKFyBdlwKnkDEaiNs3CRxaE/ZV9T527djJCWBd98lYeMBjEzdEUxFnTywzHkJ7VN6R6fjL7vcMV1ixOZ3fA5SIljXH3VDhPZz3GPnIEV4zRDkbm2hWRyj8cQrabt29mO+HznppukNRzPMUZO49jAAhjVt882QtljbcZ/v6I+Ip2gtxDmOAAZsQX/Ho0J+1vIJhvqsacAmXkhXG5lpYdVI0bt5Bj+93dA/3Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Callers can pass this in for uncached folio creation, in which case if a folio is newly created it gets marked as uncached. If a folio exists for this index and lookup succeeds, then it will not get marked as uncached. If an !uncached lookup finds a cached folio, clear the flag. For that case, there are competeting uncached and cached users of the folio, and it should not get pruned. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index cc02518d338d..860807e34b8c 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -721,6 +721,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapping, * * %FGP_NOFS - __GFP_FS will get cleared in gfp. * * %FGP_NOWAIT - Don't block on the folio lock. * * %FGP_STABLE - Wait for the folio to be stable (finished writeback) + * * %FGP_UNCACHED - Uncached buffered IO * * %FGP_WRITEBEGIN - The flags to use in a filesystem write_begin() * implementation. */ @@ -734,6 +735,7 @@ typedef unsigned int __bitwise fgf_t; #define FGP_NOWAIT ((__force fgf_t)0x00000020) #define FGP_FOR_MMAP ((__force fgf_t)0x00000040) #define FGP_STABLE ((__force fgf_t)0x00000080) +#define FGP_UNCACHED ((__force fgf_t)0x00000100) #define FGF_GET_ORDER(fgf) (((__force unsigned)fgf) >> 26) /* top 6 bits */ #define FGP_WRITEBEGIN (FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE) diff --git a/mm/filemap.c b/mm/filemap.c index a8a9fb986d2d..dbc3fa975ad1 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2002,6 +2002,8 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, /* Init accessed so avoid atomic mark_page_accessed later */ if (fgp_flags & FGP_ACCESSED) __folio_set_referenced(folio); + if (fgp_flags & FGP_UNCACHED) + __folio_set_uncached(folio); err = filemap_add_folio(mapping, folio, index, gfp); if (!err) @@ -2024,6 +2026,9 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, if (!folio) return ERR_PTR(-ENOENT); + /* not an uncached lookup, clear uncached if set */ + if (folio_test_uncached(folio) && !(fgp_flags & FGP_UNCACHED)) + folio_clear_uncached(folio); return folio; } EXPORT_SYMBOL(__filemap_get_folio); From patchwork Thu Nov 14 15:25:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9BF3D68B30 for ; Thu, 14 Nov 2024 15:28:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DCF8C6B00A4; Thu, 14 Nov 2024 10:28:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D7FC76B00A5; Thu, 14 Nov 2024 10:28:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BACB06B00A6; Thu, 14 Nov 2024 10:28:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9B44A6B00A4 for ; Thu, 14 Nov 2024 10:28:23 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6298FA1293 for ; Thu, 14 Nov 2024 15:28:23 +0000 (UTC) X-FDA: 82785079818.07.E26280E Received: from mail-oa1-f48.google.com (mail-oa1-f48.google.com [209.85.160.48]) by imf26.hostedemail.com (Postfix) with ESMTP id DFE13140019 for ; Thu, 14 Nov 2024 15:27:47 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=Ms8EYliF; spf=pass (imf26.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.48 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731598045; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rF419cnvCI1sSo/lAda6/Cm28hf6CisfTzw5UCKZsOU=; b=NvsMdHnCL0EuvaN05xvrlmhuAaL42d8tQzxBMClHIwRS3mXVrVtiXOjU+TDHrrJmqPtpUO hj84jOf93NoPCt+UMuH6Lc8AAE3UUSwtTkG72gduUMRqqOGV2nfspkWf9uBLwyia8m8vqU XUUkgUel836c6Jc/CFU4Dcz2w9TrbWA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731598045; a=rsa-sha256; cv=none; b=iwUB7gOJFqY4nTxOLbcXhGQalLsnfqTKoonL3McFnVYJqeGkHysAooMXzYNwpK7K5ooOMi fFM5RyPRI4wnmWeTjktJ8bLY8B3kAK9KLbJxhqQE+LcPO45HnH6kHZcGtfvSBhpbiWz7jq sQgwdAmSVGrnudBadkwmOTwBqABqvlY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=Ms8EYliF; spf=pass (imf26.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.48 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-oa1-f48.google.com with SMTP id 586e51a60fabf-288d70788d6so301301fac.1 for ; Thu, 14 Nov 2024 07:28:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598100; x=1732202900; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rF419cnvCI1sSo/lAda6/Cm28hf6CisfTzw5UCKZsOU=; b=Ms8EYliFLER4ec5aigixeWXUMue87neawzLxNHsNd/VG/gQF4HexL2/zRqaslaaoSK GCglUoXfJaZY4evb68FU7aEE+edInd8cqQ07ZWxm1Ibm5N1jHftj9c6gSlmm0/IcmOeB JTasPca0qlSTkOGYN9x4IBHf3uibj3RpuoD363+9au42j0IO8cTtfx2M/MKXNUK+63nc BHwk/b4coLzbioM8AUzF/YvoHkoRc4vMITDRmu2D5fqt7pk/RzlTOFYED2809PI9HkZA cGYw+CBIB6f4lzCQeosIHpwuuX9JDKT1xr1eHvtyyQGazX4J1sWjF1tEOu+pxZQERMlI /yig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598100; x=1732202900; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rF419cnvCI1sSo/lAda6/Cm28hf6CisfTzw5UCKZsOU=; b=pPWkVr9lc2v5ibi6vcYg7Zpj3CDQ1WhAQ1DSG0WJVp+h43eFt2/kLmXeHn78lQo+wI 5SzTKtzs0ljdkGVPleNJYCN4W2/F/SToKdkDNLxm4ejZ05xjXtnoOP4kQm0pY/qaXAy0 ADY+v0CDP2qJS7CsU1G6puvlUS9T9++L0SjNTXn6LL7F/aSvrqD+nt1ONJwjgtD4FDhC FLOmEaRZzQ6I3w3hQT1H8vzkkn9kHXfzKKERkP2Uqu+WekVELK5BB0OQ5+iLkCqXON1Y Gl/CyA6lXYUIcGh5HalZ3Bal10EudMjOZLgeBQddEW/Zx1IaPWUEmm+cBuTZfPEcTeWh W1Ag== X-Gm-Message-State: AOJu0YxRtm+orMBoh81D4c7PZ3q5hfX21c0PNnX/sKH+ChSZupCUsaD0 19PV7Hy2SFqyF3kFUeF33dno8eWv83lFzEIDQuRXLbNd16ZISHkmb/234MjUTnt4ykBF9nmEAY4 x7UE= X-Google-Smtp-Source: AGHT+IFrxjJdmj36oQy6UzPIhY6HkDZabl7PIZsT4CTZLD+yls1jl9nPP1SVyU+w5pEmbGgfnynQrw== X-Received: by 2002:a05:6870:e0c6:b0:261:b48:3c99 with SMTP id 586e51a60fabf-29560143149mr21908350fac.23.1731598099964; Thu, 14 Nov 2024 07:28:19 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:19 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 13/17] ext4: add RWF_UNCACHED write support Date: Thu, 14 Nov 2024 08:25:17 -0700 Message-ID: <20241114152743.2381672-15-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Stat-Signature: hctwnhh84nxehia39tgseuw5kg966ysf X-Rspam-User: X-Rspamd-Queue-Id: DFE13140019 X-Rspamd-Server: rspam02 X-HE-Tag: 1731598067-15004 X-HE-Meta: U2FsdGVkX18eR9dcA2m2PdmNL9QIqLDU7wbCHlYQxDAC2bKcEManr2agslV6Dv6tbb1Rk3Aiz9KUFVCOJKCId+jTD4mbGKz7TMNJ/1VpEVDeQSBioH5wfV38/6gxRCrznYs88KQwRCIJ/mvbE3gTmxhntqhZY6tmkPWfB9rGFbvAwRC1V6aXjZerBiAIEWjmY8HeKJXS2u57iBroG27pkVM4eCMc9LoBWQNVIRGAT5ooDWuVOm7oLtus+bj9hnC57G5vOtfGWNo6KtSmhheMJlofbR3nUMGdmFoGnu+qzGxVWNyXvUiSv+uwbdM8tNIKnceVc/UOo2FWLITWQaMdw1Pnewj+keR/Teqvd3aKFzhH9zKi/PETb+xorv+rznH430I9Fa/w70TK3SCTICmFDWbYdk8cWiFAFXEaAhL+u+FsfCLBfTucKVy6oYnaiQ4Gc1pE3+ITPvIhD4qf8Nelg6xSNZwOoJDfgyDiSv8qLJ1ocVw6HRQoWaTmFoH0Q59qRyjOfei0t1fXuAtbgl9+uiYLrYA6rnECZktbfczpPrAwsl+J2LOz6+9/e0mgyOIHvYg1ZZWWukolqNZVzkryU7xX04pl3x1SRb0CybHhHnvwgOG8UseZUg3ipHCQH3zzsnD0Yut2To7OSFwiIJWcyDO9iw9eCCGRPXGV1JncaTNYtPuZJK90Lz06Qf5nnkmPe54MTpRybZIF8L8BLnOAogPXDvK7F3VUoDGJzPd5bZPisnAZQsqtvm4OKPhp1d2sVX6KDrMdb7KjO5looNP0+TZVbi1IQi/Y6ZY8Mzk6PA0YTcDpnQyigJ6YqTbY7pFhmzl+ZXeUdYdFD9ZW2MQNy9S4g2V8UzFkB4S1x4ddenAxKdk08pJk7IpdW1PhTMaAp6TgEsWqRkN5Flf2r/YBdvoGCbpPFOu8hDiYDu7KXlDw4f20ahk1nUIj7YkdNyRniK4N6YyCz14zOtb1TtK 1uJd1YK7 ssUJGcEbJn4W1XMRRNX5p6infLFFX9bjbAfKeC0uTCxE4XZQq0kt9aZNwWxTUm1hLVBBzmoUhi2EsFGtikeOHyGpuv9NgMyNmvYTPJ/4zHv84/luJtuO061x0Z60qy42mNffMRgjBzCedqhtmhmF8t2KjyVV8m/miP17zuOhpAAENORyZD/5Al0VIg+LWljhBF6g1LJIbqerJkciBz3+Spvsm5xImHbz36vfxzYc/bTq2Lm4YA5gKrB/LNAfziJQSzuvivg/lqFneBmBovUh9o8V6A6JMG3KiVZpkC8sVwCeARH3rzunrZMfweBfd95IZoR9fNmsB7M4sFiOLE3SjB7Dsz6+vzt3eNC7H X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: IOCB_UNCACHED IO needs to prune writeback regions on IO completion, and hence need the worker punt that ext4 also does for unwritten extents. Add an io_end flag to manage that. If foliop is set to foliop_uncached in ext4_write_begin(), then set FGP_UNCACHED so that __filemap_get_folio() will mark newly created folios as uncached. That in turn will make writeback completion drop these ranges from the page cache. Now that ext4 supports both uncached reads and writes, add the fop_flag FOP_UNCACHED to enable it. Signed-off-by: Jens Axboe --- fs/ext4/ext4.h | 1 + fs/ext4/file.c | 2 +- fs/ext4/inline.c | 7 ++++++- fs/ext4/inode.c | 18 ++++++++++++++++-- fs/ext4/page-io.c | 28 ++++++++++++++++------------ 5 files changed, 40 insertions(+), 16 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 44b0d418143c..60dc9ffae076 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -279,6 +279,7 @@ struct ext4_system_blocks { * Flags for ext4_io_end->flags */ #define EXT4_IO_END_UNWRITTEN 0x0001 +#define EXT4_IO_UNCACHED 0x0002 struct ext4_io_end_vec { struct list_head list; /* list of io_end_vec */ diff --git a/fs/ext4/file.c b/fs/ext4/file.c index f14aed14b9cf..0ef39d738598 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -944,7 +944,7 @@ const struct file_operations ext4_file_operations = { .splice_write = iter_file_splice_write, .fallocate = ext4_fallocate, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_DIO_PARALLEL_WRITE, + FOP_DIO_PARALLEL_WRITE | FOP_UNCACHED, }; const struct inode_operations ext4_file_inode_operations = { diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c index 3536ca7e4fcc..500bfb6d4860 100644 --- a/fs/ext4/inline.c +++ b/fs/ext4/inline.c @@ -667,6 +667,7 @@ int ext4_try_to_write_inline_data(struct address_space *mapping, handle_t *handle; struct folio *folio; struct ext4_iloc iloc; + fgf_t fgp_flags; if (pos + len > ext4_get_max_inline_size(inode)) goto convert; @@ -702,7 +703,11 @@ int ext4_try_to_write_inline_data(struct address_space *mapping, if (ret) goto out; - folio = __filemap_get_folio(mapping, 0, FGP_WRITEBEGIN | FGP_NOFS, + fgp_flags = FGP_WRITEBEGIN | FGP_NOFS; + if (foliop_is_uncached(foliop)) + fgp_flags |= FGP_UNCACHED; + + folio = __filemap_get_folio(mapping, 0, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) { ret = PTR_ERR(folio); diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 54bdd4884fe6..9b815137fb2c 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -1138,6 +1138,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, int ret, needed_blocks; handle_t *handle; int retries = 0; + fgf_t fgp_flags; struct folio *folio; pgoff_t index; unsigned from, to; @@ -1164,6 +1165,15 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, return 0; } + /* + * Set FGP_WRITEBEGIN, and FGP_UNCACHED if foliop is marked as + * uncached. That's how generic_perform_write() informs us that this + * is an uncached write. + */ + fgp_flags = FGP_WRITEBEGIN; + if (foliop_is_uncached(foliop)) + fgp_flags |= FGP_UNCACHED; + /* * __filemap_get_folio() can take a long time if the * system is thrashing due to memory pressure, or if the folio @@ -1172,7 +1182,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, * the folio (if needed) without using GFP_NOFS. */ retry_grab: - folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN, + folio = __filemap_get_folio(mapping, index, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) return PTR_ERR(folio); @@ -2903,6 +2913,7 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, struct folio *folio; pgoff_t index; struct inode *inode = mapping->host; + fgf_t fgp_flags; if (unlikely(ext4_forced_shutdown(inode->i_sb))) return -EIO; @@ -2926,8 +2937,11 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, return 0; } + fgp_flags = FGP_WRITEBEGIN; + if (foliop_is_uncached(foliop)) + fgp_flags |= FGP_UNCACHED; retry: - folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN, + folio = __filemap_get_folio(mapping, index, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) return PTR_ERR(folio); diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index ad5543866d21..10447c3c4ff1 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -226,8 +226,6 @@ static void ext4_add_complete_io(ext4_io_end_t *io_end) unsigned long flags; /* Only reserved conversions from writeback should enter here */ - WARN_ON(!(io_end->flag & EXT4_IO_END_UNWRITTEN)); - WARN_ON(!io_end->handle && sbi->s_journal); spin_lock_irqsave(&ei->i_completed_io_lock, flags); wq = sbi->rsv_conversion_wq; if (list_empty(&ei->i_rsv_conversion_list)) @@ -252,7 +250,7 @@ static int ext4_do_flush_completed_IO(struct inode *inode, while (!list_empty(&unwritten)) { io_end = list_entry(unwritten.next, ext4_io_end_t, list); - BUG_ON(!(io_end->flag & EXT4_IO_END_UNWRITTEN)); + BUG_ON(!(io_end->flag & (EXT4_IO_END_UNWRITTEN|EXT4_IO_UNCACHED))); list_del_init(&io_end->list); err = ext4_end_io_end(io_end); @@ -287,14 +285,15 @@ ext4_io_end_t *ext4_init_io_end(struct inode *inode, gfp_t flags) void ext4_put_io_end_defer(ext4_io_end_t *io_end) { - if (refcount_dec_and_test(&io_end->count)) { - if (!(io_end->flag & EXT4_IO_END_UNWRITTEN) || - list_empty(&io_end->list_vec)) { - ext4_release_io_end(io_end); - return; - } - ext4_add_complete_io(io_end); + if (!refcount_dec_and_test(&io_end->count)) + return; + if ((!(io_end->flag & EXT4_IO_END_UNWRITTEN) || + list_empty(&io_end->list_vec)) && + !(io_end->flag & EXT4_IO_UNCACHED)) { + ext4_release_io_end(io_end); + return; } + ext4_add_complete_io(io_end); } int ext4_put_io_end(ext4_io_end_t *io_end) @@ -348,7 +347,7 @@ static void ext4_end_bio(struct bio *bio) blk_status_to_errno(bio->bi_status)); } - if (io_end->flag & EXT4_IO_END_UNWRITTEN) { + if (io_end->flag & (EXT4_IO_END_UNWRITTEN|EXT4_IO_UNCACHED)) { /* * Link bio into list hanging from io_end. We have to do it * atomically as bio completions can be racing against each @@ -417,8 +416,13 @@ static void io_submit_add_bh(struct ext4_io_submit *io, submit_and_retry: ext4_io_submit(io); } - if (io->io_bio == NULL) + if (io->io_bio == NULL) { io_submit_init_bio(io, bh); + if (folio_test_uncached(folio)) { + ext4_io_end_t *io_end = io->io_bio->bi_private; + io_end->flag |= EXT4_IO_UNCACHED; + } + } if (!bio_add_folio(io->io_bio, io_folio, bh->b_size, bh_offset(bh))) goto submit_and_retry; wbc_account_cgroup_owner(io->io_wbc, &folio->page, bh->b_size); From patchwork Thu Nov 14 15:25:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26E7CD68B31 for ; Thu, 14 Nov 2024 15:28:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 915926B00A6; Thu, 14 Nov 2024 10:28:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C6986B00A7; Thu, 14 Nov 2024 10:28:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CB926B00A8; Thu, 14 Nov 2024 10:28:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 44F3A6B00A6 for ; Thu, 14 Nov 2024 10:28:25 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 027901C68C3 for ; Thu, 14 Nov 2024 15:28:24 +0000 (UTC) X-FDA: 82785081372.11.4B65F4E Received: from mail-oa1-f47.google.com (mail-oa1-f47.google.com [209.85.160.47]) by imf12.hostedemail.com (Postfix) with ESMTP id 4086C40006 for ; Thu, 14 Nov 2024 15:28:02 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=SRUGlO5G; dmarc=none; spf=pass (imf12.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.47 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731598039; a=rsa-sha256; cv=none; b=DvnfmlnDCpI6nd3phzjycIEd309kY5mmDc8ftRe/7xFukRtfHcVpY3YTcN0St4INMHCSfH nbPDUpE1mjy9/2KDPDW2JTGcofLDCLGHmzazhWeuAn3r+oCbZGjk54lYg7EJpN9FcFGdQU +syDelhsST73wJD1E3l7oqjKL+S7yQQ= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=SRUGlO5G; dmarc=none; spf=pass (imf12.hostedemail.com: domain of axboe@kernel.dk designates 209.85.160.47 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731598039; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JnydZ5ahnoTDPrjopxRhuD9AlpgfRxt1ZJ34FZKhxI0=; b=V0qg5HHqnHgcaNFfbb8MDoKfrEAjvqx7cQSzxbkObIL7Vr25gnjYIn2hp9j5P1MFYLJil1 OETPqSdmfbyxJXoe0MsGeb6oP7lq0V+sjzaHTXl+NWoZOHMxrEFIG9KiG7I3+xQEpl2QhQ qw8vkrMmIbKD61B5apCDPABFzKgpg6E= Received: by mail-oa1-f47.google.com with SMTP id 586e51a60fabf-2958ddf99a7so605806fac.2 for ; Thu, 14 Nov 2024 07:28:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598101; x=1732202901; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JnydZ5ahnoTDPrjopxRhuD9AlpgfRxt1ZJ34FZKhxI0=; b=SRUGlO5GPyB076ZzCS8l6a1S0rOg95OD678qNAw7bAFkCPmrTLMeFhEUzPbdcGUMEL YKarmzNZq9OtqlPW+ojEhEnuI2RCAXs0N0HmAOx9xxK42C8nYKcdt7SOXHqrRGeHVlkq tSMDID4G1WcJUsfxI1785vx7z7aDHkyr5GKc+DGe3sMOpBYmPzviSQ39gOXudNKud8Dx DkSDR3nKA6QWP0zfiBQ3ZyM6etMxFaeVLcFTiRrEn/bb3+AtEUBJ2lAgBBpNmsH2Wws6 fgINiwZSZdYjSMyx2gtnm+7k7fBA7tmpNU2zHvXfkThEuJzlNEg/v7EMzKboEFYjvKfg IOtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598101; x=1732202901; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JnydZ5ahnoTDPrjopxRhuD9AlpgfRxt1ZJ34FZKhxI0=; b=qJELg8rZ7Z+5CkmBfIp/TPf8ikeM+eOwpEp5Dpi/BL3QFLKn5BkMojcTI5VqqrASJD H2JA9vLf4T8WB4M47lgdLVoEAOPfoWrojrJfhoo9wAGkDvgYDBZDmZXGXnA013xsf4Bg OFVymnCIoawmR39E8OuyAJy7rQBXMmEQQpWNEkH0hACghsFnpK13sy8lWqFTbwSyopuc PL18KSKzzGQXXb76PSMY9Cl1wKQ9nasSVDWWavU7x6pJJKoVDXE2ho6tgoGYD0Pw8LfN nHy0uZc87OPIWZHa7WIK1V3b2tNNXsnlkD3jmI80jGtqIbzG9JoF+enYAiyw1/VklBQt eO2A== X-Gm-Message-State: AOJu0YyVYQFhMvRSqcIePY/OIseDoXB9eDu3Gz0ZMcjY8eEk1G2oVDTQ eTs1P1lf6YqlGyNAFRpXlJ3QXdsoIVsKbw3T1i92Z0u2LkdlsmX8hokfbe7FfUzM52gZRtuxate pbaA= X-Google-Smtp-Source: AGHT+IGNabSsCKonQoe8b+3leqigUS1MxYLou4wkGKabmMMQ9sy0zqR8b1138Alaw/M4bLorq8cnFg== X-Received: by 2002:a05:6870:e2d5:b0:288:2fb6:a385 with SMTP id 586e51a60fabf-295600dc985mr21855599fac.20.1731598101550; Thu, 14 Nov 2024 07:28:21 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:20 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 14/17] iomap: make buffered writes work with RWF_UNCACHED Date: Thu, 14 Nov 2024 08:25:18 -0700 Message-ID: <20241114152743.2381672-16-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 4086C40006 X-Rspamd-Server: rspam01 X-Stat-Signature: wx7zxudiuix9ansi9ewkfohuyywguhmh X-HE-Tag: 1731598082-585309 X-HE-Meta: U2FsdGVkX1+hb9UfdWymb8ocx6/Vcau+HAcVN4ai0v86XPGlhg02jAFfW3oWFe1LxELHF3LFuwjQSTZcAAYVhcmSHkJGm1Maa0AqkEnPeJtU6C4Y4CVWmZScCsr6KoD7GFhj1weF57kT/5bmlW4yfOhc1U56DEo97FfELCDljPkq9yycvBNbl8W9dqrQqpJE3+9B81lleARUgnWLmJdVrrsCBfgN2dihm5xTAKYpvfdWzKP1paZwROfBf6WB/K5OV5KbVVfwVssdT6uLKxspK/6suVWos0b72oGsewdhgvq8kmzTHTr+Ezeu3u37EnnLsh9tZuvXTGmOUkNTPBL2Xi0Zb+ozz1aOsfXKCClnl+IYRLqDrxAsVNxBX008GF5ynZsWiZ8zk+XAX9xief8jozvzcZwayLp0isEby6Ci70I5WS9bQH9XCNBrNoWWGRRT9h5to7zvpWiOpd39PVQJ6ZlLha8n8hHFq27Z6qC2jIb8jkBEeYORfIIXWcNPD4DdajBD/ycclHdjTU//cpdclywbE65sJpd4AvyDpXMir1HzDHJFhBW6ZgATfd/AFAtZpWe0GbkPm2nDWKvDWtv5FLUPHWoQYE2EO8iwTtSdc1dN+DMHYgs+y/mdeths3o0TQaDeUBtjzWdzTwhcKhpxR3H1evV5gHkJz4dEr7f8ViZ7JTWC5vaqNI6GIkPPWKmXWIodwoN5bmHTFxaZV8w86i4/mrViRjAkqo28sM92r5Ip/X4hPUmACfp1VToJfWu8UtUdIQvC5Q9scuxeYVLn8lKp7TE5TH+26XtMFpRnPuqY2opF0kryRUSQk6YSgueWbiTF2nTcKXzvpWatGT20ig0MI3+KthhjYWQPiA0A9Xn3wzUKOxJYI50miC9D25uLzM0cyEfV1dTD2rPaoa3B0HHpwJJ+rkEKdjo0K5zQCgLD1R1WLuXClWfJ6NDMbt9ziSgS0F2To2Yw6ug6V0/ RneqFD2s INiJJqsmyRpR4PjGf3h574Zd3mNms+PMsgWgriSjb958vmD9UtY9h4b3Le7X6gPZlk+QHywIY28Is6bJ1BKwb50PmLx12mKnRzrhl331Lus0Un+CBp7B4x8a6ZPYKzYO2CxYnnUEDEXV153ib+t42gwFx5Zm7Jr0IFQCX2PCxPiEpkHoZhHqbVbv3U4At+nq29jGuGzHqCwmCsfSFNnsr5U2EzI5GkLJjRs6R1zATbi3etcv5UglsOT6lW1cOSvvRMzBn8sv7y3sUIKz1sFq1Wg3p6+ZCGCSWWwcqRwaFaek5sqRfHtOMajvFXztHu6k0DD/ASfQrfcsBduaVG68x/B54iUkrMLs8p86k X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add iomap buffered write support for RWF_UNCACHED. If RWF_UNCACHED is set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest. This still needs the user of the iomap buffered write helpers to call iocb_uncached_write() upon successful issue of the writes. Signed-off-by: Jens Axboe --- fs/iomap/buffered-io.c | 15 +++++++++++++-- include/linux/iomap.h | 8 +++++++- 2 files changed, 20 insertions(+), 3 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index ef0b68bccbb6..2f2a5db04a68 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len) if (iter->flags & IOMAP_NOWAIT) fgp |= FGP_NOWAIT; + if (iter->flags & IOMAP_UNCACHED) + fgp |= FGP_UNCACHED; fgp |= fgf_set_order(len); return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT, @@ -1023,8 +1025,9 @@ ssize_t iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, const struct iomap_ops *ops, void *private) { + struct address_space *mapping = iocb->ki_filp->f_mapping; struct iomap_iter iter = { - .inode = iocb->ki_filp->f_mapping->host, + .inode = mapping->host, .pos = iocb->ki_pos, .len = iov_iter_count(i), .flags = IOMAP_WRITE, @@ -1034,9 +1037,14 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, if (iocb->ki_flags & IOCB_NOWAIT) iter.flags |= IOMAP_NOWAIT; + if (iocb->ki_flags & IOCB_UNCACHED) + iter.flags |= IOMAP_UNCACHED; - while ((ret = iomap_iter(&iter, ops)) > 0) + while ((ret = iomap_iter(&iter, ops)) > 0) { + if (iocb->ki_flags & IOCB_UNCACHED) + iter.iomap.flags |= IOMAP_F_UNCACHED; iter.processed = iomap_write_iter(&iter, i); + } if (unlikely(iter.pos == iocb->ki_pos)) return ret; @@ -1770,6 +1778,9 @@ static int iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, size_t poff = offset_in_folio(folio, pos); int error; + if (folio_test_uncached(folio)) + wpc->iomap.flags |= IOMAP_F_UNCACHED; + if (!wpc->ioend || !iomap_can_add_to_ioend(wpc, pos)) { new_ioend: error = iomap_submit_ioend(wpc, 0); diff --git a/include/linux/iomap.h b/include/linux/iomap.h index f61407e3b121..0a88043676f2 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -74,9 +74,14 @@ struct vm_fault; * IOMAP_F_STALE indicates that the iomap is not valid any longer and the file * range it covers needs to be remapped by the high level before the operation * can proceed. + * + * IOMAP_F_UNCACHED is set to indicate that writes to the page cache (and + * hence writeback) will result in folios being evicted as soon as the + * updated bytes are written back to the storage. */ #define IOMAP_F_SIZE_CHANGED (1U << 8) #define IOMAP_F_STALE (1U << 9) +#define IOMAP_F_UNCACHED (1U << 10) /* * Flags from 0x1000 up are for file system specific usage: @@ -173,8 +178,9 @@ struct iomap_folio_ops { #define IOMAP_NOWAIT (1 << 5) /* do not block */ #define IOMAP_OVERWRITE_ONLY (1 << 6) /* only pure overwrites allowed */ #define IOMAP_UNSHARE (1 << 7) /* unshare_file_range */ +#define IOMAP_UNCACHED (1 << 8) /* uncached IO */ #ifdef CONFIG_FS_DAX -#define IOMAP_DAX (1 << 8) /* DAX mapping */ +#define IOMAP_DAX (1 << 9) /* DAX mapping */ #else #define IOMAP_DAX 0 #endif /* CONFIG_FS_DAX */ From patchwork Thu Nov 14 15:25:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 501B6D68B31 for ; Thu, 14 Nov 2024 15:28:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C72B16B00A7; Thu, 14 Nov 2024 10:28:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BF9CF6B00A8; Thu, 14 Nov 2024 10:28:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE8796B00A9; Thu, 14 Nov 2024 10:28:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8D0D86B00A7 for ; Thu, 14 Nov 2024 10:28:26 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 47ECF1611E2 for ; Thu, 14 Nov 2024 15:28:26 +0000 (UTC) X-FDA: 82785080910.22.D1C4835 Received: from mail-oo1-f45.google.com (mail-oo1-f45.google.com [209.85.161.45]) by imf08.hostedemail.com (Postfix) with ESMTP id 05E53160006 for ; Thu, 14 Nov 2024 15:27:54 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=pFFE1afw; spf=pass (imf08.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597909; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=1FWporYNvAYxielTeukCiRB/LN6O4fKMIbsPY5c9Uq+djhDMKpj6aCUOSY1adEvsbzDoHI M3US3i7FGrRNBkbzPMRa9as8blGz8nTtTLXFORppa7L5up4WcStvQFxg1nmPe3mkam4PP4 VkZENicvRxyUk8L80SNCxnDS9uxb2Fc= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=pFFE1afw; spf=pass (imf08.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.45 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597909; a=rsa-sha256; cv=none; b=wO8+e8bgdU2wHFyKymBDuMZW1LhqIl9zYKyb2b4ThTIaUt87WbP+AKBjRMiRYqlGYlPD9p G6ewI2jaZaxwkWkQ7PWydkWQSPIsd3Nu9S7GJzMLbkEgJ0gjTXmFY2uKGyX6DQzbmL+eyB n0HYztixvdV5n9R9Tqn0Xg/9Ff2XD10= Received: by mail-oo1-f45.google.com with SMTP id 006d021491bc7-5ee3c2e79aaso340646eaf.3 for ; Thu, 14 Nov 2024 07:28:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598103; x=1732202903; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=pFFE1afwk4DJM1A0hIM05D4GFTYdCkjvJXT5Bjbf8EZhrbBgN5h0XlJeE1RG17NwJm RHINmuJuIuIWlpXG1rL6590WF56bygGU/A70eZeExARIKpjzZOPs0zrnU8p5/6DJcqHn 3ynl9MFgU/QcsZ+EVtC0n+CMBAhxFc1YohZBNK0x5/JXjJwI248yDmouDa4H0hr4t5Ih 9oVv9R1aeDc0u9gDMuYH/FYve0uOxv+sKGZcGUTVqrzwpwuVwXoKrsX0kvuIgMoVxMAl D8GbS75ZDgYcRVUSgQqbY/hndqqg6luT4mccsf0bOe5fACuE/S6c1uHnySWmsd2TFzm+ sqeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598103; x=1732202903; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=LbxeGC7BSm1kjxSEh9GMNuZyD7gpCaUxiVo/hjcn2gSR7Wy8ShBqQ162nZ/oveWdmN ZRACAnS9HVv5T4aVia+5a9bb+4Tum/6cqW+HWdmd+mXLKK9sEbzLvDANOSMsSS0xO64B +b9P+YWBv3P6E7EhRSIfbcJLKK/1psqtBcaTwhmPDPSMH1FDpXI2M2izfXWbxx7tqTLk u2oA2dBbhTw26T8kb3rdjlrkvts1Inl93bjoY5aQAwVQV8Tr+iJgjxWZxEyoINUOmRim ja45vMborg6DxcScWg6dRT4CGF2BPKE2wwKekU9mbzRP+MGEWyk7DT1Ms0q3nG0RnNvb P/4Q== X-Gm-Message-State: AOJu0Yz5KpBZ5CKLJplmlXc6xQBEveKpeZc1AfHGUdpitEDQ1hPl8mt1 2K/XCz/84A1iffyTMeHIJ1u5wPRroy2yQyRZPgUxUnyPWkZ4ILw5F1MS+UYNkKor2EP6R7T8Vyo THUc= X-Google-Smtp-Source: AGHT+IGxBOidvnNn2woru7d3dY6RKyVHYMtr7QGUJGpSzdzZfL3L6KYNggyKAJszR8sQZicyWvWugQ== X-Received: by 2002:a4a:e90e:0:b0:5eb:88a5:20e8 with SMTP id 006d021491bc7-5ee9ecc2c38mr2597677eaf.1.1731598103136; Thu, 14 Nov 2024 07:28:23 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:22 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 15/17] xfs: punt uncached write completions to the completion wq Date: Thu, 14 Nov 2024 08:25:19 -0700 Message-ID: <20241114152743.2381672-17-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 05E53160006 X-Stat-Signature: 4kd7aftqnizfr91rhmn7q5j71aezua3p X-Rspam-User: X-HE-Tag: 1731598074-118871 X-HE-Meta: U2FsdGVkX1+lxW6os1JAIVDT6g4n60HNiXu58zijtix+wTa2Ke4jHPAQU+4nyGD317cIIKoRSmoua9DPJp//3ajOOTvn1EJuh3kHykHJ3Qr3jXTiUHWIiEYQmGhSBCRPYikmCxiBgzyo/jccbhFUSHPak7tItC1BEt7wiCYuj3mZAv7cFOtRzT+ZCk+kWcC2DDIHpQTTisQaTq2vXmRfr6HcYDIblp2bK9rMoSR0pRAxwMeq6dH1PIYer1N3nYifvTPF3q3jaHtZrtB46e17pU+cZVxphdaGtX1N7voAv6cBiQCSq2yYTv2XkltWmjFnVyKT4hJihn2RYlgwg+NqR3AtqQYxtYenqDoh5vuIh5vqYx9XLqLW6IKVKDewnP5Bv77lBQk6C6sp4BZNBBnDChPcxmWjywYKKl/dX9ZyfyOVgEurt1VKVjdsRkw7rQlYt/w7HfxZQwLj7TRBBKL6jibLsx3bAD0NP0MCxpj0iaNiGm5MzAz2IyAkZfS0wqUbHRjhhpLb5mPbENJtKHzO53JCiNnqRBC1mU7xrcrD/fgAc/E1zvd13uEkZt8FR3btWL1IQQ/l2B++EBLgYwy0L9PaTLjPGJjonqSWvxFtfqg+IHTMWjNxieO3ge4cH1v6X/0adsqMBQWgt+rU2WXUZiyj6Hnxhg8aiV6XdfethQAcxOMsGVhaKEkd9ayjNIXi5k2vmObOZCglHMja75zx1cxX0xx12fPtYQC+9E+SRnNGBxpU29kMB7rdWqmrem0GOndH7qn2C1RCeq/4aN8odc8i2VrH2KYAJ8JcB+5QtAkuwrG/1o4ecnwGhc6ZsY991h/USQYMyfyigtT3tRqGfg23DyXmB8mnM+ngde7gfMsNkeIxCEP/h0+mP1aaZhFvOU/R/Wewj7FSIOR1wlg/WrniLTamRgvCQqJEItZFyh1HrIU2+FB/veEHLfrK3W0X8fuXAcLgvY9m6QRt7SX bs6nTmTt XDGTqi7CibnbFu7rP4XJOD0FoIWCjh+7uS7L6sfV6A5XJfUTEa936E20zmL9Tay0hvEyqmNQjAw9Y/ZmenhcLqN41mlpxJb2I7vE0vma3RLTyw7RRxM5wRxlVBxxkxvbjBXOHRckJ8cdfjnhbdevbUZFpMzSwY034pvHe5Mm2IxMx12rU843LIDfhzX5X1rnCUk9C8dvcKcBio/yV52GbD7U7qz2k9iU+RUs3MWJW8rlpOUazbap7wm30sqpA4Fo2AQl7DK7/D7zi8bixjt0yAXIYXluZCsxUR6skDwzJ6yaHKK6rW9bx9aipHyPQc6n/140WU/0xrGNyK1mZAuHbWEMLgeXiy/YkHC/orsSQb5ag+lTPztZ5qCYkxJsOf1qjCdxXIxdWHoVGcSwRduq41fOF5OQQQItwa+mjXD3p6Dm+f26z3CuYDmaJSquN3Zfp2ZWx X-Bogosity: Ham, tests=bogofilter, spamicity=0.015269, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: They need non-irq context guaranteed, to be able to prune ranges from the page cache. Treat them like unwritten extents and punt them to the completion workqueue. Signed-off-by: Jens Axboe --- fs/xfs/xfs_aops.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 559a3a577097..c86fc2b8f344 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -416,9 +416,12 @@ xfs_prepare_ioend( memalloc_nofs_restore(nofs_flag); - /* send ioends that might require a transaction to the completion wq */ + /* + * Send ioends that might require a transaction or need blocking + * context to the completion wq + */ if (xfs_ioend_is_append(ioend) || ioend->io_type == IOMAP_UNWRITTEN || - (ioend->io_flags & IOMAP_F_SHARED)) + (ioend->io_flags & (IOMAP_F_SHARED|IOMAP_F_UNCACHED))) ioend->io_bio.bi_end_io = xfs_end_bio; return status; } From patchwork Thu Nov 14 15:25:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A41BD68B31 for ; Thu, 14 Nov 2024 15:28:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CC226B00A9; Thu, 14 Nov 2024 10:28:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 655C56B00AA; Thu, 14 Nov 2024 10:28:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F6196B00AB; Thu, 14 Nov 2024 10:28:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 25ADA6B00A9 for ; Thu, 14 Nov 2024 10:28:28 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D4313AA588 for ; Thu, 14 Nov 2024 15:28:27 +0000 (UTC) X-FDA: 82785080952.30.2749F87 Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) by imf03.hostedemail.com (Postfix) with ESMTP id A98DC20028 for ; Thu, 14 Nov 2024 15:28:05 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b="oCCce/1W"; spf=pass (imf03.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.52 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597869; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NyBHBKICHAilzWbXxQX4w3ArnCN9Xkn7LoXdkHxZ+NE=; b=4P0xiwGT7WRKyI3n9X1Xxm4pc2calCVbbXy4LJG1CT1YT9tRdo6V5WCpoknVeD5LccC6cq yLMYDQ9yNpL4LXHkGaChNIpWqMQIZ0IZW3dn7MbvNJ+i1DrexY9WjuIjCF6CsVej2yydId pDv5/ZxQ/DBh22HpQkFw9XL9V47F36o= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b="oCCce/1W"; spf=pass (imf03.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.52 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597869; a=rsa-sha256; cv=none; b=t7N/5SaIZe22c6W8l6KsIWApso7ZTIi9WLXYMFCZ8fBnHQDOi1y7Cl4LDAfw0heRKw58FW 8NQLNpj5Rlda4o3nYQ/NeDvD4VpgAeakqv/KU0FGoaW9DnszygHpiK/TCjWNLdtVBnJ7Q2 0Q1f8xeuCxqXaAXF59eRnyFxbkgX0gU= Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-7180a400238so372602a34.0 for ; Thu, 14 Nov 2024 07:28:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598104; x=1732202904; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NyBHBKICHAilzWbXxQX4w3ArnCN9Xkn7LoXdkHxZ+NE=; b=oCCce/1WQZJOnTSJUIn/US2Hf1qGhVkhB+1qny/z/Lg5MEgqBoRxk+nbR6bISNCw07 lydwhqTmMYXYtz8VlTVWBA951Esk0pG0+CfX85LPutlRYMfF5ACC8qMzZ10YNkYFTbFw yNvdI0Vb/dLQ2T8tJywTFREY7d6+ebEzx6uuhcqjpGdJElIx/LNX3hOoqke76HOidgRg blWeFgM5R8IPgti9Ftz2cH51XsOwxRMdJTGx9bCXT7ICIPQS23CqdIIp/FGsINmuRVru NC3V56qUp5XR+EMeHwAhcL73vY7mOgVO1iXs1mePefAMaxxAtFF2Idx7nltVujtzske0 Nyjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598104; x=1732202904; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NyBHBKICHAilzWbXxQX4w3ArnCN9Xkn7LoXdkHxZ+NE=; b=Et1ARHL3TdduZLzhZGY2R5NC9P4o21dsO2QYOl3x0WweLjUzhkqA/RY8YBBQo6YBun 6j8+5oQnRpFz9Zu/XGIVAZE/BOQCK4uKPGusHoz74SmojGhs1oHl8y8RJoPnLZIwCgcI yEPUHD+tfFZ4rr6HZjPSXVI7H9W7NXHjnze1rXBwCyTOOP29cwfwqxFu+PEmu+BcH3od l3pV9snJVeLQMDUt6tIMEBnHDwjk1f9Gy2Lu3v3A3WByUAc3tlbgayvUPDZF6wbftubp 9qGMm2gH5XcxqhmlCVLniLAPOK//zAJuePQrAlQHQFsDdbDTP2WUHLjsFGX8bkW9Znzt +HCg== X-Gm-Message-State: AOJu0YxV0V9avANF2+hL7pUcEH3vj+xC1EoJflmnU44AzOEol17KOE6p E6IQ2ftc1TAznYfgYFjQHa/hbEv9nm/fndYr1uFjDfipFWY50ka0042ak0Xf1yIpu3ONY0LIfqJ DKng= X-Google-Smtp-Source: AGHT+IGTccgjkWiSk1+iFGbOjrOkQSSEn+5BvJmTbBLfatp4glFbzMa7kfcy1vSIijdABdXFBUl4NQ== X-Received: by 2002:a05:6830:7101:b0:718:8ce4:6912 with SMTP id 46e09a7af769-71a6af419f8mr3092006a34.14.1731598104601; Thu, 14 Nov 2024 07:28:24 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:23 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 16/17] xfs: flag as supporting FOP_UNCACHED Date: Thu, 14 Nov 2024 08:25:20 -0700 Message-ID: <20241114152743.2381672-18-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A98DC20028 X-Stat-Signature: jrdicmj6zh56hpy84spsjg6nitaakp9a X-Rspam-User: X-HE-Tag: 1731598085-141058 X-HE-Meta: U2FsdGVkX1+ETh0pJ88WQuZnsh/PVXTQfInBsqVzfYWdratfzWvf53NXMAnhxQSPrbEpycUiyl97QI0ZhDxpKWHdqed+1La4H3zDPGdRyDMLJklRscO/MEN+15lvgiKmF2q/aNCzU8sZM3oCaW6o2zcBAACjY8VdhPYInTU5lYpwEYOfdAjnJLAMko3MCSF8nh/KPMD/QS+33kmnSic85xTFFmi8+LDXjfUQ82p2ZwM1tGWdmsMHPPHBUPGl9EkvagTOTVoFUQEVYZfhHfwP13sPU097OrsLN+t6tnz/A5Q8fRNf/kJbFi/xjmEhMM1Gr0vrgt1BYbhN5tdG7E/5Q6SjO3zpAPOI84GHzauPknoAVMjzackWx+c2FVkF9YY0mg4XDrz8JsWImwJ0HEm8dNdZn9kHoH+5XSej7RHe/CriqdDGOoFEffPEzEjhYwl6SpfhcrCqWdCcG8uXasGbqyL46FZExFRKQYPglC2CxXZYJF3DWKrdIGu7j4rWPV6eXvz8s4z7o9rkP6Uorw5STsfSA01ukpoA8IIpGuiOrMhDDemygkFLiicLIfxoiq0Tw97B1UlXkBpRa6TSWeUGVEZ05mY/BoOn9B/7kB5nbOjZiWxHRU4QJgtw0eYHbwGn6yMZOnpDvl3kucn/N8LcIyXJ55Nd/IguTb3FES9nYvQN26DiVPFhQjD9EX1onsCpuHGhbQBn+T6HxaOuBFh915fnlqx3gIOkC+rj1PTYByBo5vnM7EHxonLw/oXXS+fOuXCOb+/TRf+ANtZobvi3Sb8UlzEHryp+7Uf7/xGn9zAnoglD0QV4vfXhIkEMN7k7tBYYMddiHSExRkmAgVTXbTsI/UvO+Gj9gTrSm8+7esdOjUwRbUiZQLJar4AJHnOx6z4XkWpvBIU1gJHRNNFVy5TLjgvaAl5qzHt6sTIPu8BgEPiAhvF4cdorGgzYigdfBTDKpufwiWKNoXiLtpb WswOALEY YiH8BNgT8yt+Dl+jlkyhZZVpnNwry4lRAHdHQRYaH34NJUmNSU+/CmydHrsaIXbUgRfL5SLphItK1pCwZd+eDf1Z0iWX0Mf+qak6cwYO/53fZEybug1YaV/XZHH/NXbGowHQxtR0R+/sGmT2xJm4GNyP8QrvilXkMQqpTTYEefuCqKdQoqo6gloIw6eYll8eg+I7ird4y237zrzhtxzt929NhWluCT9rubV6rfMPnxPaCdarATInBaCj3wNLQF31sUA0irmkiuZ5RKXVTr9N2Z6Xrm9VyTahQRU7NgCAH0BnpOb+q9Y7RCtd2grV3q/xZDZ1Y9ueE8CEXOpyFm6gYgCUe4xmE1XifkKmR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Read side was already fully supported, and with the write side appropriately punted to the worker queue, all that's needed now is setting FOP_UNCACHED in the file_operations structure to enable full support for read and write uncached IO. Signed-off-by: Jens Axboe --- fs/xfs/xfs_file.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index b19916b11fd5..4fe593896bc5 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -1595,7 +1595,8 @@ const struct file_operations xfs_file_operations = { .fadvise = xfs_file_fadvise, .remap_file_range = xfs_file_remap_range, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE, + FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE | + FOP_UNCACHED, }; const struct file_operations xfs_dir_file_operations = { From patchwork Thu Nov 14 15:25:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13875263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 988B9D68B30 for ; Thu, 14 Nov 2024 15:29:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78FF36B00AA; Thu, 14 Nov 2024 10:28:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 740F16B00AB; Thu, 14 Nov 2024 10:28:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56D026B00AC; Thu, 14 Nov 2024 10:28:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2F1CA6B00AA for ; Thu, 14 Nov 2024 10:28:30 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E350C120B06 for ; Thu, 14 Nov 2024 15:28:29 +0000 (UTC) X-FDA: 82785081456.16.B2A7470 Received: from mail-oo1-f49.google.com (mail-oo1-f49.google.com [209.85.161.49]) by imf20.hostedemail.com (Postfix) with ESMTP id E5B2A1C0011 for ; Thu, 14 Nov 2024 15:27:32 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=d3YygrcX; dmarc=none; spf=pass (imf20.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.49 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731597929; a=rsa-sha256; cv=none; b=8aR9HMHHALnAFNjLD8wddYXR4PBYCBuczy3qpT26m16UJ7d1Y5Dg5uWehb2AijhX+wtIyv MmHzc6JvoUxW1ePyVXNpPa7Z7H6GF0vByjT4j6Jm11SIrr5nOlKWXnm5OZb2Kejn7trvq1 eQEPHI0tprv9VcMvO870zx+PIZ9V6sk= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=d3YygrcX; dmarc=none; spf=pass (imf20.hostedemail.com: domain of axboe@kernel.dk designates 209.85.161.49 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731597929; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jsUz6pEpY+ox9az2F5q2xnny+koVz5fK77xTMpol5Tk=; b=l1jwco9T1x8mtkakDiGe8440pxCldWHVX2zL8Ynq4AjSxi+qHRu9xS1YkRPABntW/Xrb0w r2KgLTIvf3qbsMG3cFRJKDDQPfIgwGWT3QRLpLh6UaifBJFOCOCe3QJxWE8cFzAEiTTMhP bvCe4hrLF3VKQI3w8VTc8f0VmJxxQuU= Received: by mail-oo1-f49.google.com with SMTP id 006d021491bc7-5ee763f9779so379380eaf.1 for ; Thu, 14 Nov 2024 07:28:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731598107; x=1732202907; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jsUz6pEpY+ox9az2F5q2xnny+koVz5fK77xTMpol5Tk=; b=d3YygrcXMUO0IJs4Sd5yHlivpGI0l8R1y4UwIO12XA4m3Rb3G0opRJOCAerrQFvEsp ZSkbLgLTFJQzG28S9MP9EIPcVsqIQE7XU9AqWtzQ9YKxYRdwE0vXUo0zmkP8OE7pbK+L QTUMqhMFhZcPwf4h5rJ3hW8d50i6P3wPgB4+mjLM1gDH16YbY51LFTQwdH7yfl46qtB5 3lBBOF82c4yTuIpWRi4odtXGLVQcAyjvmDlDgjUGiVjOlPqrxmu8DBLpaAWEZvqLQMU+ 5DJbqrKPhFiBqZnsgxrcPHa3m8lvJF5alksORyYFgWNwXnSg294iUYhGAsPj5ubhWAYC vvGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731598107; x=1732202907; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jsUz6pEpY+ox9az2F5q2xnny+koVz5fK77xTMpol5Tk=; b=UjFqTDonSWXianeDK0T/V9/Pl2/jaXhzo7QMemOu1swKgCuymqUqwK3LjCssD00IP1 JdpT42wvpwjUmH2XXT0JoJk5rNiF+3mS0rM1VUb5xvf739So6DcBrmQszdji9ao37jFt uRUKni46iLydbcmtKR6lwetzcLzG06CAJlnLmqgVywZOqULtDA0rL+kGAWD9q5OF+Kg3 Ocixn4v9rQVcJ6WJtb997wFpsDKOtPTpH038cVIhn3Oa5q6pIlIz58IkCAuxaKnbcyFI J1xg0SmhrPq1kwNjCkGq4yLMIfZbnSt07i1E2hAYJSM3UfoO58EAL6bPgTLxzSec5DR+ 75Qg== X-Gm-Message-State: AOJu0YyaIkenR2iYPB3Zmn0LmF+Bqo1Z/FwjKk02/yG+HwQluX3WMCk4 C19qN+7f3qZgrgAJRZzyjWX3sDPDi3xMo57z+pcFYvmng96EoWN49jK74ZBHFWAYVLaytUkLkJq FJO8= X-Google-Smtp-Source: AGHT+IFc1nzkbPZXQe4aKtw+LtP2VNQBWJFtVjU1qLyGyhTsHecVf/ilzi9Wv0164LJAdz5N+SGBoQ== X-Received: by 2002:a05:6820:1b08:b0:5ee:a5b:d172 with SMTP id 006d021491bc7-5ee57c530cdmr17735794eaf.5.1731598106027; Thu, 14 Nov 2024 07:28:26 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5eea026eb41sm368250eaf.39.2024.11.14.07.28.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 07:28:25 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, bfoster@redhat.com, Jens Axboe Subject: [PATCH 17/17] btrfs: add support for uncached writes Date: Thu, 14 Nov 2024 08:25:21 -0700 Message-ID: <20241114152743.2381672-19-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241114152743.2381672-2-axboe@kernel.dk> References: <20241114152743.2381672-2-axboe@kernel.dk> MIME-Version: 1.0 X-Stat-Signature: 36nqi4p7wm7ixsntig63xp3sdgs7fame X-Rspamd-Queue-Id: E5B2A1C0011 X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1731598052-542000 X-HE-Meta: U2FsdGVkX1+RB4tWXqCyC7HiNR+V0BevlrcACDCskmXlaxpYZfEbXUdSqseIT7t95/Wzet4+25nVx7mjhWU+BdB4j2WGdXbVthMkk/fXf363+jEYb2Ep45uylWPKs2WfDooEP8CKyha/TylkACdkc8rdtqZPBj0Ru18fv//x7NC6GDiEuhwmsLQSj/0l1N0NWlyl2Xp7NWoWox4OephTNbEgE6S7GG/uxheTtXXf+g/nRpoUIZSef5eEpxr1BjQ+0I4sa70cjBijUuAphIFVduPM00qpWs8ONV/vp3m3h+ORrL5eijfoF44Tkc5DLzwMzWr+Kjm3Y2Q90yjt+6vgox4hnduacQj8YcABYapdwisFOQkevJtmN7rf8lC2KOcsCzgu6TcKiOAGlbMXsWcMFuAC0CP6lk8X7QKDULxona/HyNCk0ii1CKMIlHyH9UWxD/zOeS9qFu7nQ65MwnTQz3U1pOW+VGSldPtAzz1rL7MKlmqomwYDc2Z4P5ox0UWK7e7vkrI6VT53hF45I6ttuagDc871y8GCfy2sdKryUDwmhxb5RjEmQNFo0oyzWqMz3qep029V7Ra/ORS1ir4T0RHLtGUxj6F7l+M/Kd4AYttFJeQGQBbzq2ZR/dz8kpjHieROkqFFHD6xJS1jTmJ2koKrkuPcm2i+eLraIfekTZae6n0IK0bwjqc7rYJWrMma7zsElXa+XoBUNy5Pyasnna3EOO5gwm7AZ/7hpIGJmm056mkMcm2ED4D/XEQVjznrM254bXVhHBArBDgOGeJK2uwkp8kNNrQW/kax7Vmu8HO1RzqiP9t2TIRZ2zntoj8wXUQaGIgWM803+jlv/zUgYcnz3NI7fZhVMnQrX+v1E6Lv94WN2zOGhZRveYjB2sNbQnuogS/ma4Hrz4no9ZvyP9duLYZOA44120PzeBHli5qh9eIurTKP7lWhG07uOrQLZwLOQTPiBiH1QjZeC5o +gzP5sOu V/YHWmoYZIqvqnegKyKOhkccgGy8KW55HFQTgV7QQyGX0RGmjvvW3GxR9v1ovs6q8c8t5kGrTzOtNspvO4ekCBAljqBOvMF8Pu/6TXzRA2lWQPPPOLO4T28o8s8gF4BwYNeacrwi5uIR+1BHiUhWvn/1RlKUV+hW+9S41OAds4ikNvJzQHzdgWVfcuMhWomIS6HvrVk2St6F+MUS2Qw2kPSVaJf+vtFkuls9xRFhqSlj/niPe2UpdleJ6j/GOlqxI9yj0cSqT3mjL2O7OrlyJKK9k6ECIsbmnSe0sogc7oK/Q17gwTfO4oSydIA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The read side is already covered as btrfs uses the generic filemap helpers. For writes, just pass in FGP_UNCACHED if uncached IO is being done, then the folios created should be marked appropriately. For IO completion, ensure that writing back folios that are uncached gets punted to one of the btrfs workers, as task context is needed for that. Add an 'uncached_io' member to struct btrfs_bio to manage that. With that, add FOP_UNCACHED to the btrfs file_operations fop_flags structure, enabling use of RWF_UNCACHED. Signed-off-by: Jens Axboe --- fs/btrfs/bio.c | 4 ++-- fs/btrfs/bio.h | 2 ++ fs/btrfs/extent_io.c | 8 +++++++- fs/btrfs/file.c | 9 ++++++--- 4 files changed, 17 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/bio.c b/fs/btrfs/bio.c index 7e0f9600b80c..253e1a656934 100644 --- a/fs/btrfs/bio.c +++ b/fs/btrfs/bio.c @@ -334,7 +334,7 @@ static void btrfs_end_bio_work(struct work_struct *work) struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, end_io_work); /* Metadata reads are checked and repaired by the submitter. */ - if (is_data_bbio(bbio)) + if (bio_op(&bbio->bio) == REQ_OP_READ && is_data_bbio(bbio)) btrfs_check_read_bio(bbio, bbio->bio.bi_private); else btrfs_bio_end_io(bbio, bbio->bio.bi_status); @@ -351,7 +351,7 @@ static void btrfs_simple_end_io(struct bio *bio) if (bio->bi_status) btrfs_log_dev_io_error(bio, dev); - if (bio_op(bio) == REQ_OP_READ) { + if (bio_op(bio) == REQ_OP_READ || bbio->uncached_io) { INIT_WORK(&bbio->end_io_work, btrfs_end_bio_work); queue_work(btrfs_end_io_wq(fs_info, bio), &bbio->end_io_work); } else { diff --git a/fs/btrfs/bio.h b/fs/btrfs/bio.h index e2fe16074ad6..39b98326c98f 100644 --- a/fs/btrfs/bio.h +++ b/fs/btrfs/bio.h @@ -82,6 +82,8 @@ struct btrfs_bio { /* Save the first error status of split bio. */ blk_status_t status; + bool uncached_io; + /* * This member must come last, bio_alloc_bioset will allocate enough * bytes for entire btrfs_bio but relies on bio being last. diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 872cca54cc6c..b97b21178ed7 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -760,8 +760,11 @@ static void submit_extent_folio(struct btrfs_bio_ctrl *bio_ctrl, ASSERT(bio_ctrl->end_io_func); if (bio_ctrl->bbio && - !btrfs_bio_is_contig(bio_ctrl, folio, disk_bytenr, pg_offset)) + !btrfs_bio_is_contig(bio_ctrl, folio, disk_bytenr, pg_offset)) { + if (folio_test_uncached(folio)) + bio_ctrl->bbio->uncached_io = true; submit_one_bio(bio_ctrl); + } do { u32 len = size; @@ -779,6 +782,9 @@ static void submit_extent_folio(struct btrfs_bio_ctrl *bio_ctrl, len = bio_ctrl->len_to_oe_boundary; } + if (folio_test_uncached(folio)) + bio_ctrl->bbio->uncached_io = true; + if (!bio_add_folio(&bio_ctrl->bbio->bio, folio, len, pg_offset)) { /* bio full: move on to a new one */ submit_one_bio(bio_ctrl); diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 4fb521d91b06..cfee783f4c4d 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -919,7 +919,7 @@ static gfp_t get_prepare_gfp_flags(struct inode *inode, bool nowait) static noinline int prepare_pages(struct inode *inode, struct page **pages, size_t num_pages, loff_t pos, size_t write_bytes, bool force_uptodate, - bool nowait) + bool nowait, bool uncached) { int i; unsigned long index = pos >> PAGE_SHIFT; @@ -928,6 +928,8 @@ static noinline int prepare_pages(struct inode *inode, struct page **pages, int ret = 0; int faili; + if (uncached) + fgp_flags |= FGP_UNCACHED; for (i = 0; i < num_pages; i++) { again: pages[i] = pagecache_get_page(inode->i_mapping, index + i, @@ -1323,7 +1325,8 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) * contents of pages from loop to loop */ ret = prepare_pages(inode, pages, num_pages, - pos, write_bytes, force_page_uptodate, false); + pos, write_bytes, force_page_uptodate, + false, iocb->ki_flags & IOCB_UNCACHED); if (ret) { btrfs_delalloc_release_extents(BTRFS_I(inode), reserve_bytes); @@ -3802,7 +3805,7 @@ const struct file_operations btrfs_file_operations = { .compat_ioctl = btrfs_compat_ioctl, #endif .remap_file_range = btrfs_remap_file_range, - .fop_flags = FOP_BUFFER_RASYNC | FOP_BUFFER_WASYNC, + .fop_flags = FOP_BUFFER_RASYNC | FOP_BUFFER_WASYNC | FOP_UNCACHED, }; int btrfs_fdatawrite_range(struct btrfs_inode *inode, loff_t start, loff_t end)