From patchwork Mon Nov 11 23:37:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 145F1D3ABFD for ; Mon, 11 Nov 2024 23:48:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8BE626B00D4; Mon, 11 Nov 2024 18:48:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7F6C06B00D6; Mon, 11 Nov 2024 18:48:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 670436B00D8; Mon, 11 Nov 2024 18:48:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4727C6B00D4 for ; Mon, 11 Nov 2024 18:48:52 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DC1A1161AE0 for ; Mon, 11 Nov 2024 23:48:51 +0000 (UTC) X-FDA: 82775454762.07.ADF78E1 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf07.hostedemail.com (Postfix) with ESMTP id 5BD2B40005 for ; Mon, 11 Nov 2024 23:47:52 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=nZw8RX7S; dmarc=none; spf=pass (imf07.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.171 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368801; a=rsa-sha256; cv=none; b=k/Aj1N9xVcbexfKDj6PILuAS5Optqu/CpAN3pNeshjFGo+1TRafAK4ZkjMimVpLiRmIo/c qaP8erNmYmYIZwj7eqyGiMSYiE9oNs+rCnZzO6Gc4Dt9KSnrD9i25Wp4lmGlR17rIBnYtp QMvPpDeMEYPi0rWaVPbCJwILZY6ZAwM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=nZw8RX7S; dmarc=none; spf=pass (imf07.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.171 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=InlV1VYvfgPrbw00kx0NXAhK4Dtz16qLkohbyhMyFJ5v9OEyVhRj+Vpup1ecpGXTd6jrEv 2g7Nk9y9NXbh4uPleMenDGih2QzPWIg1dFRmsQbM21dKTLS+Srwob2tULwbzUdj5NCv/EU cchyRmID4A8qXIRGgtQp52TPBCAMIVQ= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-720e94d36c8so5156763b3a.1 for ; Mon, 11 Nov 2024 15:48:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368928; x=1731973728; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=nZw8RX7Sg2JvCXPZ/OWee6WZBtXpxJPVjcxk/O9020MOmgfxrFWnhwvw9SWMCsk0sN G6O/Bs7whHxTBknkcUehB1/fOMvgYm7mlTCcYOQK5KZSI6vQNSUfkS+BQTU/nJM8ed1g OHd72BLa6V2S73sP800z6yKSxpTJhvcnxsG9NF3J5xNsqqL0aXJlKQCL4OISOYnXJ0EV Ze5lOfVBmsymr2g08DRysHR/CipD+4Kd2te4aXLrKg42iRGekDJ5Q/QS4Sa7gR9TzXaf EfZ0fG4ZkvpzHYX4cPUYq+4+99KLR0ZY2+PtnbyQUsW88Ex5ljpWlfW9JP80zvFm0zlR BuDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368928; x=1731973728; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=YFVOqScrm6iRQvDFBRDD1HIKB15jKQ5Bmkz0fxBRLD9yZIdbX0f4m7XhT8EYeJLP+i 0pgXXpTUUF3MOb9C4mNBHe04+jpIHWftHxoYNszN5WeVxsNqx/ztbxSN0SkYGAdDUevw 4cvsvvNaOG6ksTCBHz8kWY7TWx5TY/K0Qvw5LDjBflAu+bkZgZMwAkpwBPb6Tw0pDZIy TpIZ8q2K4G3R0qZDiMCwSmiNGj8uBy8BMl9dLNEN0usWiBusf2T2M8oe4xCmyD9uSFjE qyqWlAwguDVzR9zYddx7bXszhoF6zGVjY8e+2kky4pRc9tViu1vr736ZQSw7Srpbkhg1 cL2g== X-Gm-Message-State: AOJu0YwkSmYbJFeU7195bnJ2ahAunXzhXlXWXpK9/s1aGpAUGOQG5w1E beiFz5oje2rr1X7N/eSs2hWxeIOc+QbZtskXQj2zHJLifMnQYBuhAIV2EN1G7PbqOh+wrqUSM30 69jM= X-Google-Smtp-Source: AGHT+IFPjohwvXk7XeJJhxpW4zvvN4HHe41fbVtZtFFrzITrw1kyHBpDFfhvHRsSHEkN13Wz8BpfRA== X-Received: by 2002:a05:6a20:12ce:b0:1d9:15b2:83e with SMTP id adf61e73a8af0-1dc23322093mr23031517637.7.1731368928327; Mon, 11 Nov 2024 15:48:48 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:47 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 01/16] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Mon, 11 Nov 2024 16:37:28 -0700 Message-ID: <20241111234842.2024180-2-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5BD2B40005 X-Stat-Signature: dbft8asanhs1ee8qrmsqm5pfq7zau6is X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1731368872-462871 X-HE-Meta: U2FsdGVkX18KtmaI2pa/3RG0Sn6iwYJKd1EhhKFjcPjkfHh0qz7m7zmdApdPcMoccdG9OpWLFtfFAV6wzJZ5GFmq/GqM5OwKsMOWsVloBNvpXiE7WUKDblWeoOKhS+/m+wxE8KyOOVd6rLNm6c2vDBQNh4fs9FOE8wZ4rAnV8UY2gHfJx4rMkRdG+hIa+dB0eTgD0HBsXGuHje766YAghNIoNdfIShFDrYE7NYa2ihjhYpVt0S33G4KqttKb2vuVPXuqWtY5spfC6nloYUT1gGxNkQ4oYbAgtkWHhxYRBDLfKC43mDJ2mnG3sm2bilVQFASQCv30y5vWPcaMFJ4VVIpDBhWujLKtxX0BVNlK0HIMHHL9tRGDy7Pbrpmr39ETrE+T6hESckJG/DwRJzm/fQ3juiqr4LFYbRKTrk+W9fVY3kbt9QhLAMmL3+RRxgwVqMh4VyEt1mU+3y/jAo6tds1ysDcsFs4CVqcnbqyBErFsvLD9YXiqjE56OUgtj/xdZRQN1iVu/GZs9d5BH7Y3ntNEz2hKa1NX/7O8vn+fz3L8Usr9KdaccPoMo5gR89M7vP/ipkPqNp72KGH9X1dzUzs1nJGxU3viAAFXpKXjq8vdMecjU6p/ihTMnVuFHrDn+9o19kzjR7/ueSmG22fbi/KlQRChmw2KAPG7vHjd2wT8lZ/dW9FLd0OedEwzlZqoR8JERF4f+lNGPTweCOoxb+Qog33MIFUogHmC1T6uzi7l+Ip6l1ibUzzpyttFdf/vAleI5KUNQ4mVE9dkzLVsKiH8FmMhzIgnHN2aABU35qzHRt94pS2STcCU8ShUS5XPD9yjxg8saNS2dqxfgc/oJn+prb31vnmCvS6t8oaEdwYEXo7QRwGIUqIm1TVg2zVJ8PtSpug1PjXtti1Q0FUBQPUDqabig41OAcKafEoIynS3ZRVs1sUHI/Bu+MooEeuxh3N2UodSWMaPJa4rbjJ mal7sVTZ SK1fWTXah+WnyPPCFleo8yLDdiQT9iRUH9C7dxnq06MMgSiNPU/A5Ky8OyyzxcEJajq76/Mdc9q3Wen4uVqmg/a6xGKNmmHiJveGJbqA1hKtvKlIAl9F8ZuFd1x1wEjp51mqR8tjVZGe5oRlmeepXXeFJl92m8dUDgy/pj9EJuP8wZzyEUvOvf0QDxI1qpLSKs1IkMXsASFr830r9ONmEVY9lLqUd8buYCuMDee+N+R7QkVsle6A9PoOZPh/w0dGNGZ5gxndecKk7o6g/0esFa/ZswWQD9Um4aZx3XFy1Ih6khKatf2r2YbqhvmC7+hrY5ceLvRXCHPvO3vjlu/k47BemVRx0wqyIaTBn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. While doing so, move the ki_flags checking into filemap_create_folio() as well. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 56fa431c52af..91974308e9bf 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2460,15 +2460,17 @@ static int filemap_update_page(struct kiocb *iocb, return error; } -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, + struct address_space *mapping, struct folio_batch *fbatch) { struct folio *folio; int error; unsigned int min_order = mapping_min_folio_order(mapping); pgoff_t index; + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; @@ -2487,7 +2489,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index = (pos >> (PAGE_SHIFT + min_order)) << min_order; + index = (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error = filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error == -EEXIST) @@ -2495,7 +2497,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; - error = filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error = filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; @@ -2551,9 +2554,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); } if (!folio_batch_count(fbatch)) { - if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) - return -EAGAIN; - err = filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err = filemap_create_folio(iocb, mapping, fbatch); if (err == AOP_TRUNCATED_PAGE) goto retry; return err; From patchwork Mon Nov 11 23:37:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A118DD3ABFA for ; Mon, 11 Nov 2024 23:48:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31C9E6B00D6; Mon, 11 Nov 2024 18:48:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C7A96B00D8; Mon, 11 Nov 2024 18:48:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07BAA6B00D9; Mon, 11 Nov 2024 18:48:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D24EC6B00D6 for ; Mon, 11 Nov 2024 18:48:53 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5C76F1A1C2D for ; Mon, 11 Nov 2024 23:48:53 +0000 (UTC) X-FDA: 82775455770.12.1E16292 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf07.hostedemail.com (Postfix) with ESMTP id D2DC240002 for ; Mon, 11 Nov 2024 23:47:53 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=HKXq4SsM; spf=pass (imf07.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.173 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368759; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=U6r/JEXN4p6sDL2Jl85tuSl4cdRaIwaG6I/xjxiFUjwPAFeJVXE6cfu4JfzqcZAKOE0bSn OydcnJJJysvUIxGSCPU1lKTIQwqYawwITGb2lV4RJ+cD2DJ5uydqqZJ7jEl0rB1grOuAx3 Gnnreclnj8kFKytYqNywEmG0YzpBhKM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=HKXq4SsM; spf=pass (imf07.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.173 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368759; a=rsa-sha256; cv=none; b=K+brqu111E69B5OY62lfUSYvHJRA+MJUoqsdZY82jGfaZV/z9W2Rs322cL/3dBwzNrbBQi pNadAAPMCYdnWr8F0FUggZmpfHZIxzrsjWjIYD7UtA7acU89hylmzXQ9UErqDMLYzYHjU/ fdTEN+1U5Cj/4ThrxkMBG0EubFO765U= Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-71e681bc315so3584258b3a.0 for ; Mon, 11 Nov 2024 15:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368930; x=1731973730; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=HKXq4SsMROa9tKq1a8On5DGnQdsy93t++oM1FHY2SeStcVTJ7xiAoQhuP8ndj1ZJow eFt3A8IRM5c7c2r4KedZugGn44Ovp3jjogLu1Pyewfpq6RGrRiMM1mhqwumKwW7rYHYR XL7SPDSdhRJ2E5wivI1wVy2kPnEHTHISXaP8Qeb3TdqjAFz/uVRDzMyhVOaZ/uoWr40O FM0EhHy9NSDKIDCq3e7oL62ISR71Zp7sW9z5HnaKOHSdXi2cIEI3ddHz/ioaTv52wmD3 jrXaehYtJy84+EjqSrOHvD81FIK0tyMliP1UzS3rSqqP8XM2XflYisRV+Viwq/xhHKcz iOmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368930; x=1731973730; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=RAFZnaddl5ikcBDhKCcRk00ztxE1MyUCfgoi74T+eKVtAv+LmaGxCj/vVnaWO9LemI PvpxFuYxVQ0vZojzR+Fmn8CMVFoBRd9+tzbSroVwZ5aG3SVTXeEOxDFFDAQO1IleQMVp jNX9BwPhcYkATPesuhlild+UkfAb5a68ZZZ+8TwX+xImZKqGp8F1xxxuEkhxxH2pPLLU pRk0asFF+2gRUD8SQl4lO9ySFi62/+6DQ/y+f0NTRhd73fBqOaYWZJpmfof4Ilj9F6FW 8H5VMFvijt7cwIo/EkPt+qOCICywBQVteExME1x6GHW/oDz4j6Pv3EHIhmgQpXD+4L8j Z8EQ== X-Gm-Message-State: AOJu0YyNw8hCwYd24N1uOZQbkFtO/0j0QOeUTFTksJ4LwMexmGMEgyDk 223Xl4c3ae8cg3y0CwNFkndg6skARISSE6KF56xRsigbkn6gSa3OwoCBBpSJBkhZEn17CUnjalD 2tPM= X-Google-Smtp-Source: AGHT+IFIT751yc6PWfScZ6t4wwtYiUFG2/DBhEvJ8IDhssjBxrF1+5xRFKTmZX93Ms0bG2KlEc/8GA== X-Received: by 2002:a05:6a00:1801:b0:71e:427e:e679 with SMTP id d2e1a72fcca58-7241223447dmr25540831b3a.4.1731368929963; Mon, 11 Nov 2024 15:48:49 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:49 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 02/16] mm/readahead: add folio allocation helper Date: Mon, 11 Nov 2024 16:37:29 -0700 Message-ID: <20241111234842.2024180-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D2DC240002 X-Stat-Signature: i6b5ajf4nkng5xwazyjkahineqm1i6uq X-Rspam-User: X-HE-Tag: 1731368873-589269 X-HE-Meta: U2FsdGVkX1/8uZVRSrW3yYQVjPbwC867EIYQTyXA3RXiQA/A7HkB5Kqdjx2yVy1N/y3VSY8UvaIKUCrClQH2QzIM5eJkcJ6mqNmnpyhyGptveojy2CLzagS6UUMAgRiaN6k+SSwmQ/KfWRrQq5g/L97heXBHjfACfLbElXbe+AO/hHLzGACtn+zw79KHznRX7qvHSDOI2EbCqYtHGfaNNPTrR2M4DI53rOBL9LjB3Fjv868KYwMFX5oPtIvQM1onSwbvur5x83J24hd1tpN+q3m3+86MXpJP6IPyCg2ILGigbY0LDp0vhUu9mRZvBYp3tY+JLjJR6CGx064Se4VIG3xohFIgel7fmGlI2gGN7Pf6qEYNVoIUb22gGiVT1hyWO9nCDe+w7DB7rZXVtqYO3n9xUyD9Pz9kkewCarlVst0SXFU+G1PHBBpbS03H86U59oNr9lgqv4xlq90ixGEVoR2TldBEcWZz29CxtvXlR6x1j/Wh/ylJ9GQayezwoHBMyi8RPzyzW+O/7lqe2eh2oTeudIDXP3KIPukMlLISK2ng34g3r/2ok7Y1jnhJr4YEOsHTVsi/ocEM3V9syWftYYjWboByn43fW7a2klWf4lEvrxRsqqCy7lz6BVO1qv5ORTse+gBIwkTZuFRU86fiHUqXFgmbOVdtLBCgrvVYNcAA+0razGmJ7WgRHQkNaFSbnZU3pzy0R0P6j9hV9CYhTygfMroFGHtfh4GVKDvHeHCewrCkIgFzHL5KxRi6y91DLejvVnLxFn+Fb9XYLJtJi5e9lrD+X0G9S2DDm2sy8j3MopT6xzK7O812W5PFml+s4XBsxKfffJYhuWUcxI7YXyp0lkgnoTXrtGatf7g5MlFbSBEfgMLIEpEkUUY+tsTGE5iLI3TjpH4e3M+I/jP1IaqbuXrlgYSLRjwjjfyxZDcOG9BmU8O8G60dC4SfWKusrgxWKyY3w4Vg5uWrXIb urveEDQF 5KGIziwiwvpyPrDIKWIUC3zzCLCAQb6R+gGtV5kcjm/XNs1n/PR0dItK5kSz9paPdfX1zfAO7elj/ZhGSEUuUhYbhNCgxfk9bEuskrW8blWYilLBlT8DBMF045IRhpFIHVvW6A9Lw6+HxamQW6/XqantXFDN/dM5AfIKxx3PPvXMyEBYOOt3d2d5raaS45MGpMSVnL9v/Nw+m1bhqIt2U2wanFBa1uK2e1yCK7pWSFWqCxSKqeeuxHVmfomJ2X84/xtGXGpaAz2vm6kHYGnLz105t2Xdsr9AKr3CeMMsgHfYTjBkvj/NxNVx5dJusrBmHPiW+fZqpaz+Ax/obZq4/1rDxPEUTSW78DwEgwcHQB/XH0MAjxc6FYfbooswKGJmL9+Xb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index 3dc6c7a128dd..003cfe79880d 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -260,8 +266,8 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, continue; } - folio = filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio = ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; @@ -431,7 +437,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio = filemap_alloc_folio(gfp, order); + struct folio *folio = ractl_alloc_folio(ractl, gfp, order); if (!folio) return -ENOMEM; @@ -753,7 +759,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; @@ -782,7 +788,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; From patchwork Mon Nov 11 23:37:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25CACD3ABFA for ; Mon, 11 Nov 2024 23:48:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A94DC6B00D8; Mon, 11 Nov 2024 18:48:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A3F666B00D9; Mon, 11 Nov 2024 18:48:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8942E6B00DA; Mon, 11 Nov 2024 18:48:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 523E66B00D8 for ; Mon, 11 Nov 2024 18:48:55 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1388DADA84 for ; Mon, 11 Nov 2024 23:48:55 +0000 (UTC) X-FDA: 82775455728.10.B332522 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf18.hostedemail.com (Postfix) with ESMTP id 170F61C0022 for ; Mon, 11 Nov 2024 23:48:34 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=eP5y1qmR; dmarc=none; spf=pass (imf18.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.178 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368871; a=rsa-sha256; cv=none; b=GVClNtgyHVqbxuCp022cI1LEcyuIOsrYuxC9yiduwyHh8pwifb37zWFQumeBdL3gbS0BQu 39i6dR/jZTNSxxqDXChOb4bq5Ym4aq4v92gaIWI85ERVuBSJCrXyZ1woHJai6eOC+qYMhn ZeiYTKiHHZykMyhfKNV9Rn/X3PnJcyY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=eP5y1qmR; dmarc=none; spf=pass (imf18.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.178 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368871; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=2S5dzN+CHZy92Gak4wZiSFku562bpHqoYP1rU9jQdNP17215THwGr6QQDeoWh33b2AbL7D DDWMWCgWklJYZUf2tiwJQXAueQQWzD09U83vP3jinnLTLC3kU8L1qRIl9+7URSNtJ8oONV S1OJppK7IjmMUuT1RNUyVRRNZkngLEo= Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-723f37dd76cso5168963b3a.0 for ; Mon, 11 Nov 2024 15:48:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368932; x=1731973732; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=eP5y1qmRuKv6evfWg17wzEOhk0Zc2269oT/R9mhViVmI6iOnfUV6dvcFI4mvqxJUl/ wPKZtqCP8tC3iKBudzYOpRH2EInMhKiF4aYBGazgomfmfZj2tsTZkebCmHeGXNWtuLTF ESL2msQT3VwoAcjbFRnxAUYaICLarRNqtO7wXjnkuJTbDI/gw5taBnra76bDnYOyIkrS 0tdxVqqBCt4W9Im/aJvd0AmzullVb1VdBMHAx9SkXlV27uucmde0Ovm5o6IeNAWgE0lS 3w5bnKvZ7OGyfs9RpfPkeVjEFcJ9DV1CcaFDU8MSaIL6ndFeYnuCX0Y+dp1zDqSMXPhu JDRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368932; x=1731973732; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=tvEaoF6XWH+DDXpVfiiyn5bG/Kk2DKR99nJb2NYJDMGNQyBGrtkxNzTCO3EhE0770/ K/IJuZBqcVHMTG1/IST41XSzDgf4GaNQmytCzaUuNN80t6Po+npFnlnVvnoezlRxriIY D+W4Qqb79gMV27iO1AhOfmnGk5giPbggXlkv13sNZTpObDMTf46YC4JYN/o2sY98aCnu j2FTdJ7slhxk2V9lZtsbgzpWevsYs4KnlC5mcvOW1Nw0B5wyRyeiOPKc6wX/gy5O7iea ALVxVP2L+sGiXfMuxTSYoBS2727CNHUPU2lX6ndbKj1veBX3thCV+HxA7lr/oUyDS3R6 ZNCA== X-Gm-Message-State: AOJu0YyjUioeE3qq1litv1Mnij2J+5UGc3EOXT9kPDJmsQXTxR29KGK1 0gfSsGp3HlXAKFnfIkRrxbjU8LS0OpF4+pVyNrkbZZrN/1lZx7vygKAqRq3DEGDIJOTlBT43JSW +Mh4= X-Google-Smtp-Source: AGHT+IH9/1Olbxn3KTdIUuB7+y6iPc85Xc8pyUcg5cMVAU1yzHvt71d7cxX7HqTkOKWntgPA3yDsaw== X-Received: by 2002:a05:6a00:2e99:b0:71e:4ee1:6d79 with SMTP id d2e1a72fcca58-724132749f9mr20118106b3a.2.1731368931568; Mon, 11 Nov 2024 15:48:51 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:50 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 03/16] mm: add PG_uncached page flag Date: Mon, 11 Nov 2024 16:37:30 -0700 Message-ID: <20241111234842.2024180-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 170F61C0022 X-Rspamd-Server: rspam11 X-Stat-Signature: aj9dwpdj4j3x9aouwwrzo3iz5n6kpppc X-HE-Tag: 1731368914-774953 X-HE-Meta: U2FsdGVkX1/e4RHOlyeHhlszPdeMhKKywYyGwJ87BpWvx6OtWSG2VkwjKKKa/m98yZxb+JjoL5w868aw3FDzENSwNN14CKcvDiP7i4vZfJ0SBf7zCDOSST+g3GaPWjyoU/r7GOsAfzufixPplkiSDEEfVV0+K5NWpo+K1bZLnhUREQlI2GeYt3lXojcC2DlZYUp8bZjBt1cAok4zQlvEOfVveNC0i5Foq4xRrMt+KWfVHn7CBK9LX9ilN24/27T2wlEgEvnz33652irmnQ2IoS6N50T84M08DUzyLSJpkJRlcLzMFYYWChX81RKspzqcKmScHiaq6GbkMZxIwOgrlTK7fOt+UQVIrusitLvS+4+njAcqAENnQ224QIsFJ+pQN5B6uZNeIhlwR8d4WR2sYH5fjaX6idNK4pY5hRduS0KhLESkkLC7r8KafmFbUsRbWDse+bLuCo6hS1RONvmTFacKMdiaLxRFw+bI0sXu20+LAU+ESA1Ap++kHG7+bSEOXEvZtivac4+qeHz8+FMFzkGuFndWvZw8Qe17IF5EiDAgqJyP3AYnZe+VBLUF+kT2jft+FtaIiXiRyywdJiU63o1a3eMZinLDNzoSPzGDS9LqWW+ArnIezbyXOTPcKc6r+W/6t+aYGP7BUOuG4mWhEMXIbdb7uNA3eIefxI1/77w+dUP00RuOsEkm1gXcAkgK1JS1Vz5gDgbDuPqGP8B9RZOnDTiOg/zxY/KKwmwJu9x0Ck3BYn4BwvxRfl9SBtWWwEWWqebg1khMWGlDv0/lNPgEGVmUFDt6BXmOTLHBz3qIlxHpCqBXscV83UsHbPjB1QPpH0nGbkNhmLTTwdytIuh6cw3eq2XgHn0lqfK2Nm8iaHq0VSBvmdpER3pzGNwNaWYZoC1foXay4jMLyDFdzLCvzZpdHgzu0g/JwbNdoQ770eGLfHcAVJkwAqJK66UrSICWL2oIyVo0K1aRm3S /VyC8caQ sCNLBdEydJwdjuof3vrj43sDSQLRRpV+rqFNRLmRxLcHk/qahyoVuPvex0+1zaK6j3Tj3MhvaK2p19Z1yr7P3W13G/wmEO5tSWBdk+H6+v/+QtbP9JUg0WxbwRP8KL8wl41AXtz7Mllaq2Uqn80eHstZPAxuM45s4M2QN9q17fp420Kj8f3QJGwSHVlmrqGrSp7YFpWoAJ5PcaIPnvpEwzAq+7BIBDDKlI/jvPqfieQ9mxLLxB7ceORxMqBF7UqTGFkMEDu8dFZj5OnAc8STJqbgxInu7Yr4iwC0RuVxj9QvEom6Xl/1CmDjbGeTRMD4cMYuDAyVpE/BL79wWmZ1k8kxtcpdVdff9RfjMWoBlJ2nCJn1wPdsGVv1FVv1hp1Km4CeE6AK6/Fw4Uy7VK2Ql/BNPDZlxOpyNaLUpBasVE4m2VKA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a page flag that file IO can use to indicate that the IO being done is uncached, as in it should not persist in the page cache after the IO has been completed. Signed-off-by: Jens Axboe --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cc839e4365c1..3c4003495929 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_uncached, /* uncached read/write IO */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) +FOLIO_FLAG(uncached, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(uncached, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(uncached, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is not diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..b60057284102 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(uncached) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ From patchwork Mon Nov 11 23:37:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFBF2D3ABF4 for ; Mon, 11 Nov 2024 23:49:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4794B6B00D9; Mon, 11 Nov 2024 18:48:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 426E16B00DA; Mon, 11 Nov 2024 18:48:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 278EE6B00DB; Mon, 11 Nov 2024 18:48:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 061F96B00D9 for ; Mon, 11 Nov 2024 18:48:57 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BD289C191A for ; Mon, 11 Nov 2024 23:48:56 +0000 (UTC) X-FDA: 82775455224.08.5FEDD81 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf04.hostedemail.com (Postfix) with ESMTP id 03C6240018 for ; Mon, 11 Nov 2024 23:48:02 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=AR46BxLR; spf=pass (imf04.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.175 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=Vb3rose1J0RcXxDusxclKNOwQ5xNejHgI9BaOGKzG96LS+/AAjaWSg8l09ifKayXvzEdTj qa3HWrtJvXEF4QM6galssYuF/po7/JhIlEt9O+rUOK+SxFyMGJZS4HmtzimtFvhaSPvLe4 x+aqxjzCSimeOpOQ0LvmSi2/pFeVHMs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368847; a=rsa-sha256; cv=none; b=F3QLwb3NY10lN1yXID9h1vUMOm0efOGkuhf0LhBDga4mzXzCpha9c+pSYrdTtflkqRRoNs WJmfDkUP2/8MA2rkqwIWQdViSdr42CnUhVb/quxs0/ittS+dBgxfBSam6jvQQk+TuFkbqd fCb+te94wcKFjJxov8bnWBkqv63UoZQ= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=AR46BxLR; spf=pass (imf04.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.175 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-72061bfec2dso4622402b3a.2 for ; Mon, 11 Nov 2024 15:48:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368933; x=1731973733; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=AR46BxLRUfngD5cKVZNuXbcjpV334SGgzLv9H5GFwyGIXjpGGjVI+dRGC5mXrtlvXF koe3/FT4Tf7N2fZexefYWa7nJ/1BI1KO8Zpmg3mXsecsrg880WditHLPzBNIsp0IQxnj RA/K4327c/ziqdzRZo5ml9AriBClyI2SkAkuOIXLohqCQlTt/4KDaPGDp1sLaqokWg8a 7CjAbkpP9YpJGLrr4oy/p2itUSjxvvJfukzqxWtaP6xU3rX34eaP8LZSYXmmse6eArfJ PDiry89jOoiZfXYTFLpXW+fCHKffhYfTUJrrWzPoF3HlcOtN8+70c9BJtCtW64w2KJ6p gcvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368933; x=1731973733; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=YJVfhGQkJTvGp+VuuzCBQHBbkv+mjinTYcStLw2eHmtYfrigYL+3HD2qIrixT9cOsv XvobdTUTltp7aQ/oMDb/w8SAO/T5bTuT/sCL5zHYDyU/nvpRlv7De+L/A7iX0xHS6QSG j1+AP6RnN9ygYRzCG/whl6/H/HGKDzCL/Z4jgUAE6v3DlcnG1O0jWjfM13kILxzHLfb0 15+KAzm3jEG2p3qo9FwYAG3aUa2hOciB3rUAXSA+YsOw58V+5m+Z8iMzZlh5pn6dKdnU 4AzjmvwykreN9iAmbmKU10AIADBG/o48Hh0hpYvtZ031CjH2BsYD4Nyzy7XONECjSGEx 5N4Q== X-Gm-Message-State: AOJu0Yw04SegaykEQjPyI/umoa8zC0g952lvcSdupESzTHSaGcHNneN8 KXIs6rLJ9JKL8ncCaAO0SlVCbqOtOrY+JkRIwzLH1cEIw+xa8gEFyX5RXuWIgL/rSrPl1AVXGV4 vwHQ= X-Google-Smtp-Source: AGHT+IHeHxas0hJ7ji/qpv6NspxjwSxpVOtrX7CmDHrNgbtKnM+jsbZo98ZnZg8WufLJazefZ+DkyQ== X-Received: by 2002:a05:6a00:138b:b0:710:9d5d:f532 with SMTP id d2e1a72fcca58-72413380f6cmr21319143b3a.19.1731368933147; Mon, 11 Nov 2024 15:48:53 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:52 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 04/16] mm/readahead: add readahead_control->uncached member Date: Mon, 11 Nov 2024 16:37:31 -0700 Message-ID: <20241111234842.2024180-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Stat-Signature: m36958dzc1c3rewpnyd6gig77nkqk164 X-Rspamd-Queue-Id: 03C6240018 X-Rspam-User: X-HE-Tag: 1731368882-635720 X-HE-Meta: U2FsdGVkX1/8wgVCGirhpFt6qjp4CYssDlO/QDORqxKexCDqFbUZM46FRwPVl1iFqV5ovVQ2zmriBY8BSKlIkWgU41AaQak79jGttuz8kbGdfrlkm2uJPKrqK7BiZDqS0aItOPZmllZKTL8LRe0c+T+Po2pBnHpeMV5azA6s0J76ckSOB6slBI+mAoMjH95m5F9fUUJRujBnrJ/Qa5FysUstlOInAHmjXZbiDGdR0x35opZ2hjFe4xlhFIhaRx3eGQf3HXEAqGwI/Vtv0lKIVrPpm0LnthT8wPy3++4ai05NLJ3GZ9sMApmuBm4WKioXwM8ZyOLDx5Qbol7pd8GMeG3i12rt/Z2kxu6kFdFkUi93SWvVZ428FRnCZ4tZKk2XycwUS6YAu1kcm2Yx5k75NihrbqD5oRUz8+4nBas2yFO/3Zm4i74R7RRjYXUXaPIbw9B5pbzKTt6cwPvy+pZ218VLMGiWWGtTGV85s4GoI3MvMTGK6vWuFKooQlprxwjN8z/4nS9WHRnuw+/dvcRqEfbbCf1aVb/49q0qU4CCCDdPd5Axlf6VDjINfD2leWvyrcmpYk1oO/ebt9oreTn4gi3Q78TqUXVLJLbqMV6zo8SGgJg3kkwQlijUGaMLpyLJv/LcFVxzKJOGbI5hLz6SuAQ20LOj3V1EOUXeCPHNRBieTd8CcvkR/k7TQVF36DF82l/wf/93ToWoDiB8bdaQ7zFyZ6Wc/gHmzhUDYnXipIVr4v/tlB76ESV5c4cfsDVMXxPz4PJZxghqN7XChYZvIpiQOqaUfQpssj4mQh/zMBv9OsZ/Zoc7jd0s+B2w8Ol/i54tXvkca5OLnXALAvlHWxWKYGXXaOJ81/rS6g+YzdCWX3jv7D3OJk04AeMLpM7RjXL+/1A81RNeDASzN8QDpzoZOoyA4MHNVieQuEeRH+nuW5B6K0T31ETz9sJMEyg/h9cXY3XVEJ+LijyrJsO YfxUCP46 JHQmoH4VjIgCW7OJjdzIx1jxJ0012V0nKRznZ52+rnFnDNjNIhmZGBl6cGxJojVvuK9X+FzyciZslrG4pqtHSqhkg53+EGiDj5x1Ho9M50Tcs5eIsGbeb7gIXkt0+IpOkfeQ+QuA1AmmNv8VbuAlw80KV/QqAt3lfpER5lqsuTasY+V93sWTJYj93a7vHTY6HurACx2IL+1HabXdDmRvgMkpRH756L6rEegFJIsxci0Z/UFRqsvvu8zx5Ig6Vc5UWTT3C5T+BgnVz2N64D4IkL074Y/WVmE38RvJZL+0msI3gWF5JE026HzUUSR0A/IsSWdUTzTaTGi2pfQKNWegtV3KVcM4A2s62cLna4uKTx/wGGZbSVLOTRq9Nx5C4NeEUD7e4hO/8Gow7pdcI5ojPWyTqLLUJJABK7mgq0gZeMgILgDk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If ractl->uncached is set to true, then folios created are marked as uncached as well. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 68a5f1ff3301..8afacb7520d4 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1350,6 +1350,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool uncached; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 003cfe79880d..8dbeab9bc1f0 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio = filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->uncached) + __folio_set_uncached(folio); + + return folio; } /** From patchwork Mon Nov 11 23:37:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C19F2D3ABFA for ; Mon, 11 Nov 2024 23:49:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE1296B00DB; Mon, 11 Nov 2024 18:48:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A670E6B00DC; Mon, 11 Nov 2024 18:48:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B9BE6B00DD; Mon, 11 Nov 2024 18:48:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6A3F36B00DB for ; Mon, 11 Nov 2024 18:48:58 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2691E161B31 for ; Mon, 11 Nov 2024 23:48:58 +0000 (UTC) X-FDA: 82775456736.28.A6E9B1E Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf13.hostedemail.com (Postfix) with ESMTP id 1058B2001D for ; Mon, 11 Nov 2024 23:48:13 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=rQetA09N; spf=pass (imf13.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.181 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368704; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=qtnwF19OWeF902E8lM3CPHdHHJ3pC8TjWYP1IyjlnoTT1xHhkiDJpmA5sHCgbqeQtq5ZN4 vWEdCc1lELqk6J/vzKoyvA3tKLb3du8qV87L0RS0wPNx0SDUb1VqipXjCaEJNbIeDwAqjI WoLU0ScAF/v0Nt8L1CEcKTGxwGfYC8E= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=rQetA09N; spf=pass (imf13.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.181 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368704; a=rsa-sha256; cv=none; b=LhJTKjqux5euJeJBmD6C5aZrv9FzOC+M+IHUbiHuHiRUJ92oLcnSc5Ch25k3K8bKpjvma2 9SPkeUBh4xzxDIe7HmKGb2IwP1inuDNTvps3oOsAZV0cQfRvScgDZ7o4LuewMKtvK95nqJ SPzFMvFRzM+ov4QwCwus5xmKmhm/+sk= Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-720aa3dbda5so3585110b3a.1 for ; Mon, 11 Nov 2024 15:48:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368935; x=1731973735; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=rQetA09Nx5h0a0iBQCt6HPt+BXA1ILYxse88Qeksz04OLtL5eVN26/xsfRPKRh3iIi mLPvYwqdU/xul/KQWsO93rWTyV+9iZTwj/NsqHrXLrNCz0MGG2vvvrSm2qTP+AxmpJt4 qZT4zN2rS2KwTE654wsDeCMVOyjr3/meHD/Xp/qMhztMVuNk3SuDXJf7C4XQ0QM1IZHq O/4nQZZj4k+JGbQ9zKbRCyfUa18WQ6lHOvODCPbXVNIZsLOU/ekJnP5q8ixrBWctfss7 hXQqPdGNpt10uOyt+D3OVmr6CtPFbbo0XLL2sXXsaWBSznev37AJimoyBzakdxy9HVkW zOTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368935; x=1731973735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=BcwJcdCDBLPDncBL/GXuvch78AswiSP/BZwhcmZZbyCxm41QvMopQejfPb89hpfIcF /4/JR0d3+GXgOA7s6O2Th6+oUc6fiz3CbsdIvgbTyB7BB23Hg7JG+h1ZiwYakfBddGMA eB6ghXLBEtdVsqxTi5m9+xLrxIICJrgLtf6Yp2Y9NzU7RAmb3FltYSNGWp3jrOvMUv3/ aLbJ2NkF+LX/LF3aVPwBGj5A+00vgMu9U1JuEaXiEyoFUsKCuixSwx9Q1K3FJb8jJfH2 HhtKFNqW4oI8NjGrfY0JFPdggpg1bSlnpq8Sic0+ClugEQHSSbAmAvgtYsG0c60KhIaX C9oA== X-Gm-Message-State: AOJu0YyMg1jNxVYO4DXxI5DDVVOLlCjvWZvd+2rHWMj0RmvyATabFGxQ K4VkIluKyW4CypvScnP5dafSbvHbZqUfDGVdlN7N8+uQGI5Q2OiPbgP08kdL118ystWCg9H5Q9h irbs= X-Google-Smtp-Source: AGHT+IFFv9yEg9ETjAil4BLRBKswDm5bw1d1rwvQrtFRdIgpywcgln1Dsm8EO2zN3a/ZuXT+tu0A8A== X-Received: by 2002:a05:6a00:460e:b0:71e:75c0:2545 with SMTP id d2e1a72fcca58-7244a5d480fmr1035562b3a.25.1731368934821; Mon, 11 Nov 2024 15:48:54 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:54 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 05/16] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Mon, 11 Nov 2024 16:37:32 -0700 Message-ID: <20241111234842.2024180-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 1058B2001D X-Stat-Signature: i5zfa7kzqisyx9eena58uqeqn8nyfg4a X-Rspam-User: X-HE-Tag: 1731368893-268526 X-HE-Meta: U2FsdGVkX19roNJ0kh4zAsou9DghE41fQKcfAGKYjMTzfdzr1CzfaWQOAMEy6JNKFlXxWSE3H3Tz0y9DxP5ayViKIc+2BQvugIPhm2X8c2M+6vHiC7O1bxFcdbUs6942E8ndpYJf6wzN1F99PibC19I5pfbMSWRG7h/C1/c1fbR6KAMakz7Ap2EHwe8YcXT1FQdxbR+bvmn8NdCp92dd6fPkc7LOkex8j1d1BBmOi1PypBGXPWM0gOMPimYh1LOzelHERtZdfrv11R5HZjmoIlx3lpGZWmkhCJOZaUZJLEGFFu2cPrg3HXwodUdjOG9QhQe4H18I2h71JLEDCDKxlduB5nS8WSiMzH1BHMozOIpk0Vj91XS3kc5Nti4Yu7hd6eAUwSYe5jMmb3DemWWeeVs37VlrOVI9ATRZCmRSk+wiEYihtKo1DUJNhKCXeFhaO9y2m3e62WsKNDab860lTUDQohZagbPjRvaGVfA29WHmrqsUQpWLJdVTnBNrTbGKmPflm9RbZRMwdgDhacqxfkeruP+NpQCRs9xgONEy3vkcqUN0HtAtx4xQGojoTcQSfrRJ1yxRLO2m972s1YkkLSuZfsApgDqymWX+mwXR13nwL+Qhv7NY17etUNwcF+hn1zn4aef5STAivbXwSAEa5+sq2sxVPPFhk4W5sM4xphh2VOuXSeEKFLPDOirQsh/y7NGjHgfobXN7/YTCMEXrN0TevlYiQmAlX4qBC1NijqqV8yW52TuxPk4TDVW3sKWLQigVq8sBpJV0PqKW2l2//uArHJJqTOM2XByhTOkC2sy3So5L1X09xcLtd7cUC38iXSe5GUStParPaRR1XTDZy+DRIIoURSv8KoGCByHFXQfUh1fc6arHm+HKXlVbgXK6Xvvt7Uv6EA1HxkqOsnJXzxaXoHAhG0ywsIuyGs1PAwqEQaAVQrdNof0Gl5nEsgJMjJMx9B/Fh/y1ogYbJLs cbOuxjAr UaVgmgGI1tqjCRoLfZBMM2KURH1W8uP9ovKaxg3KLkSS2WChKk+UfC2j4gsQ3j4l06vdorR6LTZHyXVVra4r/aIG+yt7VWeRHRj76T0F1OvXgqbCztd+7yW8D3HNS8L0b2bdqGG/pdWJ5Iwhz4D2c0iopUGxSqv7NabBDYi4PTOT6JgMZrVYTK+3bhMpa2C/Sv+8x/ckixhrZcdwV8ilEuELEbKlMbL58gZR47ww532lF6qP9B93NDtOJmLV0D63m1NRDFWUsox/LRjfr/ROcWRQuKUyF+k/I6x/ozwyib2P9SBhV/VFmdzzgCTU0eY8VpUVdWa7R1adOCr197U53cAqUTEdaZYqfZfpKgG9nFyQO9L4Bxv9/XptSl+xWD0DKWhJPwzl/4aMuPE98ljpiTrJhLyxkb14KVy2mkTU2c/G97FA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 91974308e9bf..02d9cb585195 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2528,7 +2528,6 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; - struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2543,12 +2542,13 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); From patchwork Mon Nov 11 23:37:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B49F6D3ABFA for ; Mon, 11 Nov 2024 23:49:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 710476B00DC; Mon, 11 Nov 2024 18:49:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 670856B00DD; Mon, 11 Nov 2024 18:49:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49A676B00DE; Mon, 11 Nov 2024 18:49:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2AEC06B00DC for ; Mon, 11 Nov 2024 18:49:00 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CD576A1B57 for ; Mon, 11 Nov 2024 23:48:59 +0000 (UTC) X-FDA: 82775456274.05.AD384B9 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf03.hostedemail.com (Postfix) with ESMTP id 24D102001D for ; Mon, 11 Nov 2024 23:48:38 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=OmPBwuEX; dmarc=none; spf=pass (imf03.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.171 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368763; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CL+fLnGK0BaMmMKm57rghb0D3nDN7TOwfaAxP5ktg80=; b=DB6YNE7ogRXbm/JY02FDjA5UfpXk5hWW0Qs3bdO45NHOZ0PzAo3k1/zfreeELbZBY4tHdB /ex8MQzn1ElPkRpVzH2LnTTGNkusiuvKORi7LG03yM1QNYWLm6TkvS6emuvTt1f4CQAhWF vQsBQa3lcgR1C1NahFhBe6EO9z4e1sw= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=OmPBwuEX; dmarc=none; spf=pass (imf03.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.171 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368763; a=rsa-sha256; cv=none; b=2BP+NAjujBAEOOHJqFHBBBcKMbjf/803Tohs1n2qPcC00dQRhk/fRI1deO42T34h8cwvTe 5aRH5D+5zwGI4ItPd9WZePO8AOLKUdJSeOKgUMqCHsRWpfnmcuj29QCXDdSXfdqqcioLYD OJULPYJYMr2zOksarWcRavyoTksY3SU= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-720d5ada03cso5128578b3a.1 for ; Mon, 11 Nov 2024 15:48:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368936; x=1731973736; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CL+fLnGK0BaMmMKm57rghb0D3nDN7TOwfaAxP5ktg80=; b=OmPBwuEXz6spXEaQ7hjpWtiLHDwQrXdiP7cwLhZivKC5fXYsK69gGVWFCty9POQygW 3LZVnxhGJDGTM0SxAQ6CtvPcOMrPGzqxZ9ZrwJ8f/d0qmEkte5kGK3tu7D6ETgcbZN3s 1O2lyt2hoeC8HS4U2gQaeIHVHEWvab8dCh6wEQF+Kto92U2syIl6sz3uZNvjEyz4SOrp ldErnlvMuxVV12X6ev59GUN3LnykUObe5rq7a6M4palAULeIdObEpVTBawettLB2u3vZ HWX+7bDnLUSXpx++LE9CvlgW/Cuu89lhPtIi+YiYlC2y6lo9yX2Tzg8w7jCKsVUaZ41f c6Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368936; x=1731973736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CL+fLnGK0BaMmMKm57rghb0D3nDN7TOwfaAxP5ktg80=; b=QDmeZsxg0gILQtVChBx89rJphNdZCIPgWkQnQW3nAAbvnxfz9pNst1uwtJhr5XQXjC lfGF3mvIMMGsUsd/Zgq6QTAfHZt2LcYWYAkpqX59i2npKE254Awglgn4JCwpn3H+C5pa +25okjoOk+WAdbkEZZ6D/2tEuOF/v1+ghL5sDvpSIRJ9Hp25eh1oNgRVzhG1rUPaylLk IZAHleeeDmsywWMXTL1s171ocT2yrlXYM7IfJ/FWbTsQmGf7tKf+isg2y9hd4c27lN/u /W38YTV864Yh983H9OsKWlvaU4qmsMdELQB1nG9yzKNTXuqdL5zy4P8YMrM8NdVNkdMs 0zBg== X-Gm-Message-State: AOJu0YzguIIU7dAZfI764LM8u+xdrVWOfpsKVbuPnfLqnCKV83q3CZcG XfhujG7QfUltiyqzh41MtposTlFGKRVX7S8I2weT87gXw6cbkj0X9A7C+gvX6rHxWoRE9yjwNan QK2U= X-Google-Smtp-Source: AGHT+IGkT/vleDHQ8M3Hw8Ha2+EPGBpXcHS4Q/PdXfOEbEN6KEGRSFgIDbBBEJ7VgYq4tyZ3FLhGBg== X-Received: by 2002:a05:6a00:3cd1:b0:71e:64fe:965f with SMTP id d2e1a72fcca58-72413350e9cmr19827356b3a.20.1731368936369; Mon, 11 Nov 2024 15:48:56 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:55 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 06/16] mm/truncate: add folio_unmap_invalidate() helper Date: Mon, 11 Nov 2024 16:37:33 -0700 Message-ID: <20241111234842.2024180-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 24D102001D X-Stat-Signature: obcjdes88ye1m7ynfakbpxmugmu8k87q X-Rspam-User: X-HE-Tag: 1731368918-335832 X-HE-Meta: U2FsdGVkX1923YhhfpxwQvKu3VbXVCfFJPZOxI1HGJAz847t1nuZ4jGB298Lzpf0DXaPy4feR+Ws4dn0Qzd35RbfQaDMPzrGxT7ZCsealv33DqHjsiNveSwmjP+L+wmfLsJVXaBmohxsZS1Zb8U+dkGfJA5Dqotd4g0ciPF0j8FdyNeXN7aAG/lxlAzFNG+gTtw8dHNpoxbDJVihY0dU+JqJoG5LV5eL1mMNPNn4chDnEYwSesQ8V0ZBA8RLkIDfGvgsokw7sryGAVdYaVjcFDT2JB342ww6sBBVnQrA85CeHgjFd4tEy65+jXI7Pc9kTmTGQtdzGUpAoT3lvj8TT+5L3Rqy5FlZaIMZ2dkMs9CZqB+p+WIYnv91/xFFgnYu+bVrccbYqM6R9gW/PgSKDbcFQykhFGU8AGwNnSqLe9OLoZsYZuHhfGc6V635Bvcy0uIk39I8uOr7qC+VNqFtUtTw5I/ft4m4IdsemIiV4/7Se4/pjibzBsCW5fJAQ27HTmBA10qN6tl5q3cFxBropyK9T/TnXQ1XrSA0VBeohqHBoVBSfkviCSp4zMnXKvsjTxlrB6L9OKjoNEa1OIu/9gDWMAR80GaGZVTNbWx71kZloxsdlBq5RReEye6GJ+//rdgRh7f3cRSCegl+F/01hIQwYWKQos9FavWlesP0m/DbTBdpkEF3PBnd24H//YquoGpWx4XFtHGGq77AdH5qpbzxfWmApxlkAZ3ahsVOb2ENvS8xfpciThEjBMwSHwV1VqyECv1QYgfu4UlMkMfoU7zZjY6MGefbb2eqcLrwK23IpJXKUBbVirqwJFyu4zmDUSKJ7So3UT4Gq+PsIOZ6CZ46d0h98HyUcH12X2CwiQN5DMzKVU9d4XQm5fRJKAV/r+wQLuhxAMkM6qqmbrMZ0O/lhvqpzb0BLoTmUpF0W3eAtABIHvlqEvjh7PJ17kSVhHk7CyVafxxXHrp5XGA 7VlzxsXR 9a875sbjv6J2FOLNGLAE1hIiHJcCCJNDK/UCKuCCsdFeIxJqCezqA/DsBssPvwJ0sdX4Rp725RY+66d+2jgIegNor46lNoTWyrvzR3ZFrZVtOtRsThYbC20gNvkaWhuwbp6dVxkdwae51WEKOvNKvsGPsW+YypMUNP4+IvGSp14ySxzckutPdO61IG3Lq6PRS2VkCVwvCwvhAQCmHE4GHoLfJYnyTfxYyhqN/WnDbrmfW6nSGaT8gSqmDf+RTiV2gYj0U7ofT71sa4b4axJ4N452RHSoKKQZjp4wzjyOOGpFWqwNkkJO6gWmWZkoBIX6DpF49OlruRluWCSaV+QkfieXoFc21fvNGPu4B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a folio_unmap_invalidate() helper, which unmaps and invalidates a given folio. The caller must already have locked the folio. Use this new helper in invalidate_inode_pages2_range(), rather than duplicate the code there. In preparation for using this elsewhere as well, have it take a gfp_t mask rather than assume GFP_KERNEL is the right choice. This bubbles back to invalidate_complete_folio2() as well. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/truncate.c | 33 ++++++++++++++++++++------------- 2 files changed, 22 insertions(+), 13 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 8afacb7520d4..d55bf995bd9e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -34,6 +34,8 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); int filemap_invalidate_pages(struct address_space *mapping, loff_t pos, loff_t end, bool nowait); +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/truncate.c b/mm/truncate.c index 0668cd340a46..5663c3f1d548 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -547,12 +547,12 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * sitting in the folio_add_lru() caches. */ static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) + struct folio *folio, gfp_t gfp_mask) { if (folio->mapping != mapping) return 0; - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, gfp_mask)) return 0; spin_lock(&mapping->host->i_lock); @@ -584,6 +584,23 @@ static int folio_launder(struct address_space *mapping, struct folio *folio) return mapping->a_ops->launder_folio(folio); } +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp) +{ + int ret; + + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + + if (folio_mapped(folio)) + unmap_mapping_folio(folio); + BUG_ON(folio_mapped(folio)); + + ret = folio_launder(mapping, folio); + if (!ret && !invalidate_complete_folio2(mapping, folio, gfp)) + return -EBUSY; + return ret; +} + /** * invalidate_inode_pages2_range - remove range of pages from an address_space * @mapping: the address_space @@ -641,18 +658,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, folio_unlock(folio); continue; } - VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); - - if (folio_mapped(folio)) - unmap_mapping_folio(folio); - BUG_ON(folio_mapped(folio)); - - ret2 = folio_launder(mapping, folio); - if (ret2 == 0) { - if (!invalidate_complete_folio2(mapping, folio)) - ret2 = -EBUSY; - } + ret2 = folio_unmap_invalidate(mapping, folio, GFP_KERNEL); if (ret2 < 0) ret = ret2; folio_unlock(folio); From patchwork Mon Nov 11 23:37:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E913ED3ABFA for ; Mon, 11 Nov 2024 23:49:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A6A4E6B00DD; Mon, 11 Nov 2024 18:49:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CB806B00DE; Mon, 11 Nov 2024 18:49:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 845F46B00DF; Mon, 11 Nov 2024 18:49:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 633556B00DD for ; Mon, 11 Nov 2024 18:49:02 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 25D03141B16 for ; Mon, 11 Nov 2024 23:49:02 +0000 (UTC) X-FDA: 82775454468.17.C717F36 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf23.hostedemail.com (Postfix) with ESMTP id EDD9A14001D for ; Mon, 11 Nov 2024 23:48:32 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=uMhKMlfR; spf=pass (imf23.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.180 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368886; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BpaeGG8sCxZf4JRh/p7NGjZl2gbHvBRPjf0TUvSvRrE=; b=toOS87bljX8vLheymARaYmeCIVvV8JLgGBnlgwBZvn2RBDJXfO+rJ/U9ccgHTlyofu/KOs 1IJ0GBaL4Fl3LY4l2OMbCdiUkEJloBnNPGPgeJWQrn3y8ejSfGTPKoxftaxBFCagT/Lk+4 1QTBhXC+5lFxCfTV1D7WtfijD5IpGR8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368886; a=rsa-sha256; cv=none; b=49FZaJTBofBOuga5GL7uCLEOydU3i6EB09fTPQkiVWA2AKnIxKvMd7KeRnW4WOE2fq9IFi FKRYlP+N02rtS33bwKH8b/POqpYY30YNKBzX14p+K4xcDUqHr/1RI/yuHigZw2hRngO9u8 HST9g8IASfOX8BQcRHaAMNnH3HQFi1I= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=uMhKMlfR; spf=pass (imf23.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.180 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-71e61b47c6cso4610836b3a.2 for ; Mon, 11 Nov 2024 15:48:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368938; x=1731973738; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BpaeGG8sCxZf4JRh/p7NGjZl2gbHvBRPjf0TUvSvRrE=; b=uMhKMlfRjFv6IrcoY7XmipOmpT5ovpaZ5vbWZByyJSYlxOhWO4KpFXQ4ZuG8P+CjJm 80fscvJFfITmaEen8/DIHPeuBDOcy/bnsNYjPO+4W1e1aTk8o/L0ivR3wpP5f8cgt1NG wVB6L6wcBmqT+zunqiOyvU7TDSQqx+LqkS7J8YyJjwSlEn9Cj2tuuJL7A/6dOEevt1BG itWVji5BnEYqHa7MZo/za7GvHjtIjXRc2pt0V2qzDoSCYm/uRf3tOUlLF0QrEc8qzg3t 9R1n/xfK+6726UCwj6HulbKHtIENgbuIFBIsis8phuP4AJ8/k+WDTgcmUPpjxJwYW78c emcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368938; x=1731973738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BpaeGG8sCxZf4JRh/p7NGjZl2gbHvBRPjf0TUvSvRrE=; b=Zuzou/NAEie/swegocvsDlMu8TIaivdsf7xtuL1Q4rHhfKiOo3NKlVxoaA+yT3sFid aNOLdY0fhjF5SaBfDLkvyEymc7wYtWLmIfVJ0qyQJMCvQL6Yzut6H1dr5M8Wr4GkEma2 Sv7a67Ip9SxR9vgehPQH4MkBDQi1HqLaQC3YN/VPunBaH4/M6nZ0nGAN28bYkHA9J+tJ +3LU2xkf8xf6m+gfcgY7iW3XGyntrLetEBGYTEwgD2BnnVPH9zdAI/oFWzTX9LLBUt2E 3XxJOSy10cVyRDn4Be6D/R8G3fP6VrzZVAOj8TOCQcsfkoOF3KNZZ0XFIKpLCELqiBl1 7C6g== X-Gm-Message-State: AOJu0YxwUpRujm4tN1AEr9YxAUwzelKq8omlpe50zLdIULNfW7/Pf4lS yM+wwfA9737j4RPBj5VbaSL2GcZor+8oB1LZ9e0JlpsBoThYPkYIuMZ8VOPPVmEwJIVv4Ty5Tgo iaAY= X-Google-Smtp-Source: AGHT+IGZeOeX13AG9u0SljALCGlqZd1HzXcprLCQtKHVA2LyzBnUVzw9lDFogBpMHW34whNWJjJhLg== X-Received: by 2002:a05:6a00:3a28:b0:71e:7846:8463 with SMTP id d2e1a72fcca58-7241334a3e2mr19790618b3a.19.1731368938117; Mon, 11 Nov 2024 15:48:58 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:57 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 07/16] fs: add RWF_UNCACHED iocb and FOP_UNCACHED file_operations flag Date: Mon, 11 Nov 2024 16:37:34 -0700 Message-ID: <20241111234842.2024180-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Stat-Signature: 9qaa7b9rm16gcqcwk6f66to7kuy9sxdn X-Rspam-User: X-Rspamd-Queue-Id: EDD9A14001D X-Rspamd-Server: rspam02 X-HE-Tag: 1731368912-768588 X-HE-Meta: U2FsdGVkX18dvnbHQlfT4WjKJsD/8c9cXZlPuy2XKsQ/0NdUYgm1Y33B0CMTGALLZBxnRiZrhK/jEnuWp8htxpfe2Nz9ae6FUn5De6OQMYy4Pz0q+Tq7f0QDXpmlRQypus9Jy/Qs3X+4qc6no1P75F2xfA1Vqq+cE2+L3Ig43JTe+LODL34pUadcm0I5RJKFzxE6AllOXSn0UVY6n4/SGZOoyOFzIffexjdZW1KuBwdgfE8pM8t5UA39m2mhclfHZeAZdtaTeHSXh0WKdMdlWHBoURhoogvBLfW2nVm3Zou1f6jY4Gz2o5uBwW0plOvaXZKubIlz0RHAK3g0g394ncZeQIuEt/PlUfeIU8tQXgM2oFSw+KdPE+C1qdyXTd5dDCPyhtAYW413G+dkusZj7u073jfXGezzFeqzu7WJqSQX0q6NL8b6rIK8NG3DnKZTTlw+E28kJPekTxjfbFdJzuGB7052i5bSDXxL5XJq/XutNyiGQdXbGgBnbqVsfXIQy+On+8cVVm4V/SgG3NClyuETyCTeTv1uBxNzOfkPb1Ql6Xg60xqwiS8bpVzPDPO3Ph03O4YJXqLtx7gIlax3esPSa7tj4orKkfXluZ/IE9E7V2z8iA/6jZ83BtM15lU5EwBT1G3Q/zZKTKSU+YRqZB76zxKqTGS/y7upSaK3E5SvfGoBsDjxjCc/ndhXB95KJE2IbRo/0iV6yN5HcQDSw62pZsW3FrRbR1lGursrTSBfywJfSn++JHf33WcHLEEVzsK/ZuZXF9tVDQMc1UwoFiEoN9rjiSaM3AA6tBA0Iuo+ETwcyK6/DWvZC2sPSOaNOPXqJRoFO1V4DzB6813FtoOJeip6/wyjbz8B6cJpssrlyOTh6gMufcy09qLN0K7knEdgL8Uz6sDbyuOicJJ3HwTUPtjuxpnV8ZFw1ZBkPKFJXDjPBV4vay2PwyzmlOBvATNGC7lufvUk3hiswGx PetqzaB5 WxD23yvNSx2Ta5iyrA1NMgA2kEitAq7HsP0wtLRs9F3W7loseo+I+BGPYyWYldsGxxbh09Fv3h1Nt/S4/mlHGkAOkROSwod6OfPWFFGYcESFAEJZ8Wnu4Xvwegupej6UVK0ILP3e+kPDP1NWajM4e8vjR5AVPu0a8crD1dCC/x69twqCUV32VeS3qlauHF1lIpg25axp05sMPrI1KHQ+Gd5VNNZQ/RWN9rkCRPReAeT20Nvn0JlccvDXWnTNNDPFcd7z/iO17Uuysn/51nt8OrLsj+YqAHRReGGAOgmwo3fEMOcEHrGRvdGqobhRczhg3JIFdbYattrOg7maMmRN6ZBdzEo0siGqTylRB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If a file system supports uncached buffered IO, it may set FOP_UNCACHED and enable RWF_UNCACHED. If RWF_UNCACHED is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 10 +++++++++- include/uapi/linux/fs.h | 6 +++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 3559446279c1..5abc53991cd0 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -320,6 +320,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_UNCACHED (__force int) RWF_UNCACHED /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -354,7 +355,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_UNCACHED, "UNCACHED" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ @@ -2116,6 +2118,8 @@ struct file_operations { #define FOP_HUGE_PAGES ((__force fop_flags_t)(1 << 4)) /* Treat loff_t as unsigned (e.g., /dev/mem) */ #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) +/* File system supports uncached read/write buffered IO */ +#define FOP_UNCACHED ((__force fop_flags_t)(1 << 6)) /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3532,6 +3536,10 @@ static inline int kiocb_set_rw_flags(struct kiocb *ki, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_UNCACHED) { + if (!(ki->ki_filp->f_op->fop_flags & FOP_UNCACHED)) + return -EOPNOTSUPP; + } kiocb_flags |= (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |= IOCB_DSYNC; diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..dc77cd8ae1a3 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_UNCACHED ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_UNCACHED) #define PROCFS_IOCTL_MAGIC 'f' From patchwork Mon Nov 11 23:37:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42783D3ABF4 for ; Mon, 11 Nov 2024 23:49:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2AF96B00DE; Mon, 11 Nov 2024 18:49:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CDB316B00DF; Mon, 11 Nov 2024 18:49:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B2C7A6B00E0; Mon, 11 Nov 2024 18:49:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8D8516B00DE for ; Mon, 11 Nov 2024 18:49:03 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 454EA41BE5 for ; Mon, 11 Nov 2024 23:49:03 +0000 (UTC) X-FDA: 82775456106.23.0FA3F81 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf29.hostedemail.com (Postfix) with ESMTP id 076FE120021 for ; Mon, 11 Nov 2024 23:48:05 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=dF9vSObW; dmarc=none; spf=pass (imf29.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.180 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368879; a=rsa-sha256; cv=none; b=ewc0WwgVkyi93fiD0d+cI1FFOF8H/QY1lL/zA4APJdOlr+c+Y1A7PtiDIFt8B1dZUL5xU1 FPcm5NA3UVETvCDFSZLkOZrU7TTu/oV4E1FfUc6crrzXH3jgydGBcsOFWsbJx1MH0SuZjG Uyz9tw2Z/5M6b4/FxErcdIk6yvDvXM4= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=dF9vSObW; dmarc=none; spf=pass (imf29.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.180 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368879; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FNT2xlrRbXw1IZWpIgVp1lVCBQhpFw800CH7QKwlGio=; b=v53Rj4ecynRD1zQk74RmOEMcei+/KLiwvL6WpM9M71JGSg1MH9YBQTO0fa2jPswNChPQWQ Q2MK9J5d/CkWDMefQ1fXgfcETHPzS7iAUkIbNCfk7Nku8GASfJ7PcjoElKzGYaITADeKxO m4d/Rd+HvlU45upRq2VUxLU+VBrcA08= Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-71e5130832aso3885408b3a.0 for ; Mon, 11 Nov 2024 15:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368940; x=1731973740; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FNT2xlrRbXw1IZWpIgVp1lVCBQhpFw800CH7QKwlGio=; b=dF9vSObW640m0PKACQs8xBsPcWznrTQZbtKz1GUZn0Yst3rK91C3Em2g8tKG1x/IN1 bZ6GXFgbHatQmU7Ud25GhC78Z1G8f3JnezJWxumbhinxmwb1Vc0hOfQpTf37ut0TP2GC DVB/SWLwJb6OeTVME5XmlMDT9AS29owCbMKbptMKxKxi3oJrTOEr1cPSva1FuVSfxBvx LkKY5yW4Rwb869k3vCNuPiebMeT+gK/qE3u99+5JMvCOmJHWLU1wnxDdDAwyEwZJlkA1 ZU0bCrk11O8E/snefblcbwycnv2ihwvztHP65BW6mqkBvh6UORAxR47yUEYIY3mV7FTJ KRzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368940; x=1731973740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FNT2xlrRbXw1IZWpIgVp1lVCBQhpFw800CH7QKwlGio=; b=crAqAFSbEoRVj6sTKVAHB+8+D+cnytG0VPeE2j+JG3tT9/5VrfvpXkIIcmFn7oavmY qOWzLATGz0a6FYOKrAVa+9tUk6z27ZiayCNwZcp/FLoSmAjzhHBQsMzTIyDxF+GbNNnT ORgCqS0yFifxUlXjPRwtjy8J0I8oGZNIx+3VFlBCPEjq7lNdSerjc4mZEionsV8n6UkS ++dwtfBnc15ANeYaQI2HOcpf0ikVWnqoxHk55U2BscbRPZkL/AKGvu59/PMBuDLwcAdV DoS3C+0wH7zLlbBk9LZAQNUtT3ZIJEidP9iMiOnDhWdKJcrFz0nQXwXSdyTX7FEuboGF Cnbw== X-Gm-Message-State: AOJu0Yxm8Q3A6Nh9jVfpq3uqTRLSiiELK5cop+Gs0BNy513YOBtXC8cj jjHTILOMLeBgYESRZiWs5ed581w08AUqrfCfHpOOJBMVvM7YrnbZjETlHYyRJf2SPKsssh15OF3 zZmo= X-Google-Smtp-Source: AGHT+IErnoZ29qXFIn5gwmYcaGsN1FZzl0JZhBUfjT+VcEVWLi3eokc1bOzqdTtbaNvb8n8ixmr53A== X-Received: by 2002:aa7:888c:0:b0:71e:cf8:d6f1 with SMTP id d2e1a72fcca58-724132c4d71mr19840990b3a.14.1731368939764; Mon, 11 Nov 2024 15:48:59 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:58 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 08/16] mm/filemap: add read support for RWF_UNCACHED Date: Mon, 11 Nov 2024 16:37:35 -0700 Message-ID: <20241111234842.2024180-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 076FE120021 X-Rspamd-Server: rspam01 X-Stat-Signature: gr9ayobmcrswwxb15yz3d4mtmfihb69s X-HE-Tag: 1731368885-784977 X-HE-Meta: U2FsdGVkX19pBl/3BqrJUoWUKmhPIlTQtxPFF3EHlm5HPUO6D1DavO/pmCaxMYq2XAwOG6cuxcdzcSLo9Cu/mCq4dS2Dvoj6P2ykGsGD0h0DYnQ+6tb247Js2NLoGbNI3oaih6HIT1DdKf3FIJsV/YMmw5HYHWm6jCg/L5v7SlGI6fPlySNnI4dHGmEFH2bdmnPc2PCcM3QvpBUjGBWBpWe0UVHvzqdCkMC1JqVrzldml0/WIdgqQc+TwhabEbnnAWXbW7ltRoa9bj58dISj04rHO0AuzmVc0Z92H0hkoEuJTv8+0GSsFeuWhF59qhU5s5vZagFEutc0njixrh8/DnCnENVukT9leL/YYXESML9aO83OJBwXgr1NVPeZTfGWlwL75x0R55XvEV1vGQWkbkGY7gaGqYEQitB3GbT7TC7H/xQCBNbBkcKsJt5ntFN8lai2VgBQC5YvaehfCbXdKPnjZVuDOCs4XrzsjSKj+joExOXtuMjvJprJ3cpS+JtKcoL9aFTFOQKmOojpEAz65aqziZ7x1nWv//94DcrSS3wKuZFdhNLxt6H2mM+OpcRHRw2Mb6BLV9/uF5I6vQLZJneSBnnr166ha2BbddhqSx/AJe6FajGUQ5XGq7rpTcdNXq8tY/qqWU28u2s5m3oyQlYhuzrEqLxwn8Elo9ul+u6/B9aMk9EBfyxLXeu7lZk34+W+86gjZjIL7utQPxzgK2n6oX2PpreUYTgEto7tTbBFO4gOSg1BuYUq8sGt2wuMNujWlUrf5NMmgQR168fF+7xMo9ygOksG2I8gqEgXnrK8a/2NaFMfPZsQz3Aru22IGL1cwzSFZRb4MC7p8ctjWY0BwkURhC/U5QQ0QaPFfvIopn9ws8ZylYMrgEZBjzn5xXgOoEMaJ0TxPMh5hKuPUJTAQgoooaBm+Zep+CmNkG+ve8wKdAzEd0reQpY79hAYs6U9070tkP4meocfgRh DZAj0bqJ Oimv77DU7cBctLYq6Q8qWYQ8UlHY62eYycDETQuNtZiyxMknifIbfFPWai/r4Ae4VlpTIBj8A/hh6ydYIVKtIZs5SFQn7tIpVpCpB1TFETN+5YObGRHSESa1q2NBjmssTJ0DmRDsmaZZBNxraaaswQOVJ5Rq+s32kbYFvHOeou0uq6l77qpWTmELWqXB3bBLqV3rzuAqo1AHzUf7oixbRhx/K7XTvwK9Uj7w16+WIElSNLYIj185v4uY6R4jOyl10ojsLuUY6InXrfDroPTqpMrWXP1DrQDgPc4ThFEaZlJjflOj78H1n36noEUtDuxfppTrjo2aAqyoWcyRGErrN3/GMQD3HgI0+Q74Y X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add RWF_UNCACHED as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousing of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_UNCACHED set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++-- mm/swap.c | 2 ++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 02d9cb585195..3d0614ea5f59 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2474,6 +2474,8 @@ static int filemap_create_folio(struct kiocb *iocb, folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_UNCACHED) + __folio_set_uncached(folio); /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2519,6 +2521,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2548,6 +2552,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2595,6 +2601,20 @@ static inline bool pos_same_folio(loff_t pos1, loff_t pos2, struct folio *folio) return (pos1 >> shift == pos2 >> shift); } +static void filemap_uncached_read(struct address_space *mapping, + struct folio *folio) +{ + if (!folio_test_uncached(folio)) + return; + if (folio_test_writeback(folio)) + return; + if (folio_test_clear_uncached(folio)) { + folio_lock(folio); + folio_unmap_invalidate(mapping, folio, 0); + folio_unlock(folio); + } +} + /** * filemap_read - Read data from the page cache. * @iocb: The iocb to read. @@ -2706,8 +2726,12 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, } } put_folios: - for (i = 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + + filemap_uncached_read(mapping, folio); + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); diff --git a/mm/swap.c b/mm/swap.c index b8e3259ea2c4..542f298d3dcd 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -472,6 +472,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_uncached(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; From patchwork Mon Nov 11 23:37:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7432D3ABFA for ; Mon, 11 Nov 2024 23:49:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75D336B00DF; Mon, 11 Nov 2024 18:49:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 70CBA6B00E0; Mon, 11 Nov 2024 18:49:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 586196B00E1; Mon, 11 Nov 2024 18:49:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 381CA6B00DF for ; Mon, 11 Nov 2024 18:49:05 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B29D51201E4 for ; Mon, 11 Nov 2024 23:49:04 +0000 (UTC) X-FDA: 82775454552.17.3997E4F Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf24.hostedemail.com (Postfix) with ESMTP id 73D4B180012 for ; Mon, 11 Nov 2024 23:48:57 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=JGuNvISr; spf=pass (imf24.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.169 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368710; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=yj/XblxlQdaXaZ0jZg4tRkPxXWjojAX8j8kMc07EMmqrNlf+OEiih8yg8gm8lN9NOPnlQq GwDp8+ZHJvz7pSTQNXJOe46fslqWwcDS08mAw0bNRLzPq2fCUo0bFOESXqy8DfLiLeW/qA uqKWpzaEaEXjjyoo0f+q7iB+yL8Dvus= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=JGuNvISr; spf=pass (imf24.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.169 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368710; a=rsa-sha256; cv=none; b=VKF+hmWOxtj8eTBCLqIm5dkkn99Nvt7vfHAp7wB/gTCuebtv2C4FmqBYRpWxveg837OmSQ pn/2SfERMwUKW0IspD2xmDCozUuvwKK9HlHNuR3tw4u5Vc0kPvnf7UmAOkZVa/bEi8263Y HGESiTuE3yNOoB60gMxwYObhob4RLiA= Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-71e4244fdc6so4274080b3a.0 for ; Mon, 11 Nov 2024 15:49:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368941; x=1731973741; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=JGuNvISrMLZ+twvZFUtJi896hJhSMnNRR/9/0ZE6niNUAqfQsD7w6JdW1u+4kRCDQ8 61acoUnnp9kNaMT+vGeVraNn+JbN4nazHaBqJotsgIm/h5gI1XmFo1qENxgm1Sxh1xiT KawbYJfdy8BS3vuMJRXEEnrWX4Di+mL926TMLyBrXbj+vP+rkovBiWYZiDvDpqmdAkRQ fimq/Vy2v6Q9VkNvlcLMTVzwJgfamfATPjpl74KKaBef+W+AJfcniCWz9TLA+jNsH8Pm 0QueHRR3Vg3dixLQi1l/ID8XO98TtJNXDmqyqgG1qZ/50MFycmB+Jeo3BEAkDnLYgv29 v8mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368941; x=1731973741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=hM50ECGcamv2ij9iR2ADxsYEQitSo9q0dsmdf+Ct98FNVVz23XKT+QfBY7jQF5Qcz9 tnjXqedBgUSdEFQ6HEAzkfxeyWxxyvSjr4e3fRJkSPmeg+i9tynzbyJBrtmeeoXFotxQ awtuPpmJebfCF9oK9UXEy5AaK5kyx7FiGcvYmgfMwc8cDJ2ypU05vp5dOeUIFC3umQs9 LgilfJCGDvAyTTtVnDufxbqlrWC6a0TByiEGnDYbnI+ZAWZFUikTKVtD+KkvHLOBHviI IH4ITXMNXNaN/2GYzkHqzGExLu0Ayy0i1vB54I6+CXc4cQp4KtBt+vMXlrvr8agVn7TU MyFw== X-Gm-Message-State: AOJu0Yw6USS3y10BbtAjorDCAQuC3dq6zlsoRrmdpRMsjJIuZ1u5vM4J zJrrHS2mXF369DwiDKX4CI9I+1Hsi/XOq1LikkKhenASM7hRCC/M4z1eeO7OAF422KdNpzbX7ev bSEU= X-Google-Smtp-Source: AGHT+IHVAuYVOMfDSGlPxbnuC87wShBLb7OXW58ajqvAinndw3wlptYvFjbRCeInYbnH7Qo2aVFY4A== X-Received: by 2002:a05:6a21:9101:b0:1db:e3c7:9974 with SMTP id adf61e73a8af0-1dc228ef1damr21176002637.12.1731368941302; Mon, 11 Nov 2024 15:49:01 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:00 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 09/16] mm/filemap: drop uncached pages when writeback completes Date: Mon, 11 Nov 2024 16:37:36 -0700 Message-ID: <20241111234842.2024180-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 73D4B180012 X-Stat-Signature: dsg8i5z19h8cc4czge4s96chhpebabf7 X-Rspam-User: X-HE-Tag: 1731368937-444204 X-HE-Meta: U2FsdGVkX18PTAkTt3IBCvz72OmfSL3uS1pF7QqCnODjTXNiTKw+i7hOy1kx6HUyQO+yVxJq9JFyOP+hHa5UF3AXiu7FP3viy7WbidkqMDInbGQFxfX0H2D43qFEbrZJ5xWYVnsfyA4U84RadzIUOixl9Dtbhx+jNS7e9CvSkp5yxDEGivoWmhg83QF8uvriCAKvqXuBrkvHnQvfySAqndfGjneR7cS7ohGGhDydAxi/J++kMOZXF4HOMk90st+LwCrek8qcZBsth7ZCLHbSX6MTzmWwx+UhLXwRVhVWpu6QF1FlpBKANvtGPo+iNgiAmkhummnEZnxEoKD0C55nDIUarbNRLlPuDbJjdceZsGlWL49Yi+5O+P604iU+g0yH323LfmFjC8mGlpIwA4cJmajFMG1xAxvzaHffuW8+Z6tIJvHu8IfbJzsUBKPPZFkKexcvr1nbhf6qvbQMVbdk+EWZtLfUknv6SXlw+SK31WhhEXr+Np8GlxePKVnj6CrmVAJe2PRwtm8UAdRs9wR9HZdfgZQfDRbheqKoB3zeR3KSzti9M+oyqt23hBLpYOYhR7ZwBevrRlZbrxp5Okhuo/UpKLIoW1ksC0eElQqqSeEzoXy0uid4txE1ZcnmTXZIY1L7e8ezE6thXtvIVVGA4RlA5yLsz+mxdMrt8AS8Ug2EL02fEBvhTj6kTZCkUXb4mylm4qc5BP0Js0lEDnIKMauPMry9dS7ymJ7T9fDi8eUNRhLwANypLMh+TbsU2vgyTskkN+8ufbaewgc/FXjAEHewqlgilbs2atvWIyWQlhNYfbtweCoHLH7nZOdQ+PfZ0+UElZwnwL97M846FdcUkVTrFE1pvWw7h7ibhjGbWJpEcY996upR0FUFu6sfPn/OS/zIjtgHbF9UzoD1O7NjaHkh3V7LMakLPQ7vInRJ1JTNXAVOjvAjOP5XTE+7C0tELG8V0Li8aFSF9jgRrnr S4/na0uA f7ebPitL16E9a1YYyaYt1vTGLbOsQZOdV928LVUAgzxILUdny8fPC4wx6qombPTS7t9LK/rzawoPP+OaWdSGnWD4UQKj0LGsTHpDNKk+Mg/UFz0j6q4wpj/6igo32KrmJjYBkep6kXjyyVj/Y9XzRd9fTewwc114kMnUxa/pzsaX25fziKkw7Wf5G4EIlzA9rcaGlsiRjnFDswQ2bFYxUPeDL9/3yw8CbJUMzHSZo5qyGST/XkekhQ8RJdZHvMB4wAvk+k5t219DbZzjwiXVGljPkoI9css4Di2qAylR3bxl2YPHdemCQkeMRkg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the folio is marked as uncached, drop pages when writeback completes. Intended to be used with RWF_UNCACHED, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 3d0614ea5f59..40debe742abe 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1600,6 +1600,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as uncached, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset uncached for that case and latter completions should invalidate. + */ +static void folio_end_uncached(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_UNCACHED writeback, but + * can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1610,6 +1631,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_uncached = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1631,9 +1654,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (folio_test_uncached(folio) && folio_test_clear_uncached(folio)) + folio_uncached = true; if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_uncached) + folio_end_uncached(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); From patchwork Mon Nov 11 23:37:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BF6ED3ABF4 for ; Mon, 11 Nov 2024 23:49:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12B976B00E0; Mon, 11 Nov 2024 18:49:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C9F36B00E2; Mon, 11 Nov 2024 18:49:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E10946B00E3; Mon, 11 Nov 2024 18:49:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C03486B00E0 for ; Mon, 11 Nov 2024 18:49:06 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7ED65C191A for ; Mon, 11 Nov 2024 23:49:06 +0000 (UTC) X-FDA: 82775456526.15.CE0A1CC Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf17.hostedemail.com (Postfix) with ESMTP id 1086040023 for ; Mon, 11 Nov 2024 23:48:32 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=YL4pkVzs; spf=pass (imf17.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.176 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368752; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=34CEBvu7gjOMz8fnHdZ3R+2IdL/8ao7S7M5vWTfYjmM=; b=LfHqzHvgl7OLv5QIzkKo2Gj5jVc+CrILnJvQQnxowQbxTkDRleKWKVqEZwsl7c2Zdtsb+K inYOgHE1VQqRjDV9H5K3LTegfBh4Pux0iuf9HjwDIg5491zTBZNagfzxb2djSj41+5ZOnN Yx+r21oX6W0So79yK2gaBI4AGofynAQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=YL4pkVzs; spf=pass (imf17.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.176 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368752; a=rsa-sha256; cv=none; b=rHBrNTzASc53l9ew0Zb1dL8ne9OCwKTdLtrLvcDRp/R2YZ7KStkleVwvxDB9z62pxQ3eaN Uu1V0t+Q0PZZRtyrp3BiCQtWUGd/jk4C62+/EpFy6MtR3ZetjqlRNj7xCJcHTiwaPF+EHZ mLtFyw2q5O5gTz8cB7sjiwfkbu8jP8c= Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-7240fa50694so3694794b3a.1 for ; Mon, 11 Nov 2024 15:49:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368943; x=1731973743; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=34CEBvu7gjOMz8fnHdZ3R+2IdL/8ao7S7M5vWTfYjmM=; b=YL4pkVzsvJYHcWLNznVUZCQkD7oBWdECXl7pDfQaq77rmTjGMQaBfBP8pH0R8xeJDi x0yMWaznhRS+hm4bagclFvWl3mOxyWAxk7vS6sAfwIZRKisWZtwQm5Th++hontKeyWNV IUrnNRDjuok47EDHGIDQEqefsrSMsqsx2ozBPk2unCfAjnX1V+vBgg/9qD+J/6GIKF0B 8JrH09KCRf9y2xEv3ODd06WZyGc/Pz+sq5t1KPvZ0tgvKRyyzjJRP8nMrL0jAFDb0DUU l4x0Iw5H3KW2Sm129MW9saVVH1okL77kh8DvD+1sJhThnx/fj6ZOpohLfMTjB1H4SnIr v9qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368943; x=1731973743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=34CEBvu7gjOMz8fnHdZ3R+2IdL/8ao7S7M5vWTfYjmM=; b=mUHmd9ztbOJBxYe/kJVnBXdAMEcBDzc7mq1Zke2Ttbrgr7x6NMqW4Ol1GChIewnitn +a1cr3bSH4Tlcj04IOW+JxgzD+UFulUmppLPVv6AmCWisPbX2eoYZTCcLZcxthpj06pK FjurYKkNFq2wJLkjlSPw/ZM7NoUQ6CYK7NAIOb8B7tsZ36JbCZ7yWCIOZbLBEXBeFcfo TJUU2BF3TJSJLR7U4GR78O1gT3L0DNAlCbdZnMH4RAZYpvRhSb70BJ+Cu1veQbjKwPA8 aH1DMpbCMLs9Td8udXwM/oAFv/lLC2Eo56puHNGq/dD0S07HWMg273mtoMw4/JNjVO5H DaoA== X-Gm-Message-State: AOJu0YwU5dhxoq9iEdxyC5WUdbwtWC4hnJgpD5Vdjq7jyHXyas8p6bgO OCWrZChO2Xi82dMLhAdGp/1mELNB6bE1IM+umaV0NXXOMg18Il+Zn/PDBjTU2y5Gbi+u5kkuRG9 H4VQ= X-Google-Smtp-Source: AGHT+IH/Zn0FG6bgzME+vU3LW7llTjZ8LDSnQYNIFX21AVqsAPSSvTFEg9AexjHSBcXEhmlzou7m5w== X-Received: by 2002:a05:6a20:7292:b0:1db:e82f:2a63 with SMTP id adf61e73a8af0-1dc228c6973mr20790006637.3.1731368942803; Mon, 11 Nov 2024 15:49:02 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:02 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 10/16] mm/filemap: make buffered writes work with RWF_UNCACHED Date: Mon, 11 Nov 2024 16:37:37 -0700 Message-ID: <20241111234842.2024180-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1086040023 X-Stat-Signature: ikiwxzruoenyt8m1sh45m8ixgf43yciz X-Rspam-User: X-HE-Tag: 1731368912-130061 X-HE-Meta: U2FsdGVkX19QPZv+UIVkcsavj+E/8KnVhPmb9ct2XPs4/o2kuGvlidrRPDyc8g9B9SfuPH2vBRqzy/IOMAAnhkeB3hdvOXqXzo3sQYB+uBxaFXB5capWk7WzYNBhx6CV941YgPqyeTSTeOWVjUxgomkM4oRoQrzJUcC+gkdLyNEW9/7922jclDamsJUvf5wqjw4KhO2qODoAJ819VZEPIGCngajt4JO36IbbHvBEBlJkeuWXuGUVlMlmAaWqndwgsdJbPx0MvHOymAJfIoeEtdvXogyW6Re0lVIh0MDoP4dFqSDApEeMFCQxyVu0Uc00XOGHqX/LCnRVgmLpPKFeBa8uj31Rqu1Yc6WdzocpUwyG3p74/IbBkBLxmGwGY+GuEpc94998WzYyvXQkBkIJfQiPcA5d7XfN4KTVIUDSvlcrKi4vcli/kEiSxhlt9lizuL3h7pCeA2j2+FwFq2ve3ZusEOEAFZMPgAj+dYRUcr8nxZYqyQbjHDxh8WAaDqz44u+wJP0W0mrdEcFnobiW+0+zyHC6nbDTNgbswcEs8EOS2OPoBJTGUPZIW2dOqdkuAp5Y+WDxmllCec9gZkbC1GDOmqvYPKcn+zOtWI7/PJZnfsJayfwrRROTKjRgBo0GdvS3JvgHS40x0LuKQDJyFEkHncdCc6FWXEvkzz0tPnBWg3MMjVh8J0N3FYKpEwid7qR1dXXCsrLjZ69dUvmoTfdfdoXhfljLVW9+Tj88PbuD0TjucCXBCSDTYUSfeh+FSSsoPttJqG4gV11hYVJyknpt8teFj1oroZi9KBzfqXUdTjdXr6DdNtYKfSJFut+mMrGVIFNlk550w2IZNVSeEtDMLQEMLkuIkOsVURhNyenlyl90Pu3jgQn4gpd4IxOsLg/LOQLoZGOnSkKuiFhhhmROKNg5rPE3bJjfjo5ZnTXdOrQ5jEU+ImFtcuN3MQWPfO9T6w/N0/iYJRQ5Tsw iusM/mcP MteGexN1tWpg5emK49Tltjj99Eq7fexqdoFmIeoOLtWyh8hWieuf6/7dFjkcErLEj1jEiYPgfsBi5vqF9G1t8d2FpaROCkonApnuX9RAf15hgE+lSN0P8G/7PVPCfwjOX3pK5Lg1KPCCK/cilfobQeLvhaPlBbZs6TXvAWJ1O9Kegf1TmwiAXzug8i5UsJLyK74lRb75XO+1/CFAEtvAX3pKHugMjnkbOOFbEkDywUDc1eKaf6RfeMobN/xv+OYIWvYvZ4p9pobY6cas6XWzL30yPGvB6/rr+D1XAXoM4oS08tolVKm0AkK7zlILmjiRxk6K3m2caa47dKXN/p3GVZdGm6jZUY4htxYTM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If RWF_UNCACHED is set for a write, mark new folios being written with uncached. This is done by passing in the fact that it's an uncached write through the folio pointer. We can only get there when IOCB_UNCACHED was allowed, which can only happen if the file system opts in. Opting in means they need to check for the LSB in the folio pointer to know if it's an uncached write or not. If it is, then FGP_UNCACHED should be used if creating new folios is necessary. Uncached writes will drop any folios they create upon writeback completion, but leave folios that may exist in that range alone. Since ->write_begin() doesn't currently take any flags, and to avoid needing to change the callback kernel wide, use the foliop being passed in to ->write_begin() to signal if this is an uncached write or not. File systems can then use that to mark newly created folios as uncached. Add a helper, generic_uncached_write(), that generic_file_write_iter() calls upon successful completion of an uncached write. This provides similar benefits to using RWF_UNCACHED with reads. Testing buffered writes on 32 files: writing bs 65536, uncached 0 1s: 196035MB/sec 2s: 132308MB/sec 3s: 132438MB/sec 4s: 116528MB/sec 5s: 103898MB/sec 6s: 108893MB/sec 7s: 99678MB/sec 8s: 106545MB/sec 9s: 106826MB/sec 10s: 101544MB/sec 11s: 111044MB/sec 12s: 124257MB/sec 13s: 116031MB/sec 14s: 114540MB/sec 15s: 115011MB/sec 16s: 115260MB/sec 17s: 116068MB/sec 18s: 116096MB/sec where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages. Running the same test with uncached buffered writes: writing bs 65536, uncached 1 1s: 198974MB/sec 2s: 189618MB/sec 3s: 193601MB/sec 4s: 188582MB/sec 5s: 193487MB/sec 6s: 188341MB/sec 7s: 194325MB/sec 8s: 188114MB/sec 9s: 192740MB/sec 10s: 189206MB/sec 11s: 193442MB/sec 12s: 189659MB/sec 13s: 191732MB/sec 14s: 190701MB/sec 15s: 191789MB/sec 16s: 191259MB/sec 17s: 190613MB/sec 18s: 191951MB/sec and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 29 +++++++++++++++++++++++++++++ mm/filemap.c | 17 +++++++++++++++-- 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d55bf995bd9e..d35280744aa1 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -14,6 +14,7 @@ #include #include #include /* for in_interrupt() */ +#include #include struct folio_batch; @@ -70,6 +71,34 @@ static inline int filemap_write_and_wait(struct address_space *mapping) return filemap_write_and_wait_range(mapping, 0, LLONG_MAX); } +/* + * generic_uncached_write - start uncached writeback + * @iocb: the iocb that was written + * @written: the amount of bytes written + * + * When writeback has been handled by write_iter, this helper should be called + * if the file system supports uncached writes. If %IOCB_UNCACHED is set, it + * will kick off writeback for the specified range. + */ +static inline void generic_uncached_write(struct kiocb *iocb, ssize_t written) +{ + if (iocb->ki_flags & IOCB_UNCACHED) { + struct address_space *mapping = iocb->ki_filp->f_mapping; + + /* kick off uncached writeback */ + __filemap_fdatawrite_range(mapping, iocb->ki_pos, + iocb->ki_pos + written, WB_SYNC_NONE); + } +} + +/* + * Value passed in to ->write_begin() if IOCB_UNCACHED is set for the write, + * and the ->write_begin() handler on a file system supporting FOP_UNCACHED + * must check for this and pass FGP_UNCACHED for folio creation. + */ +#define foliop_uncached ((struct folio *) 0xfee1c001) +#define foliop_is_uncached(foliop) (*(foliop) == foliop_uncached) + /** * filemap_set_wb_err - set a writeback error on an address_space * @mapping: mapping in which to set writeback error diff --git a/mm/filemap.c b/mm/filemap.c index 40debe742abe..0d312de4e20c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -430,6 +430,7 @@ int __filemap_fdatawrite_range(struct address_space *mapping, loff_t start, return filemap_fdatawrite_wbc(mapping, &wbc); } +EXPORT_SYMBOL_GPL(__filemap_fdatawrite_range); static inline int __filemap_fdatawrite(struct address_space *mapping, int sync_mode) @@ -4076,7 +4077,7 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) ssize_t written = 0; do { - struct folio *folio; + struct folio *folio = NULL; size_t offset; /* Offset into folio */ size_t bytes; /* Bytes to write to folio */ size_t copied; /* Bytes copied from user */ @@ -4104,6 +4105,16 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) break; } + /* + * If IOCB_UNCACHED is set here, we now the file system + * supports it. And hence it'll know to check folip for being + * set to this magic value. If so, it's an uncached write. + * Whenever ->write_begin() changes prototypes again, this + * can go away and just pass iocb or iocb flags. + */ + if (iocb->ki_flags & IOCB_UNCACHED) + folio = foliop_uncached; + status = a_ops->write_begin(file, mapping, pos, bytes, &folio, &fsdata); if (unlikely(status < 0)) @@ -4234,8 +4245,10 @@ ssize_t generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) ret = __generic_file_write_iter(iocb, from); inode_unlock(inode); - if (ret > 0) + if (ret > 0) { + generic_uncached_write(iocb, ret); ret = generic_write_sync(iocb, ret); + } return ret; } EXPORT_SYMBOL(generic_file_write_iter); From patchwork Mon Nov 11 23:37:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871494 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BAFBD3ABF4 for ; Mon, 11 Nov 2024 23:49:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75D7C6B00E2; Mon, 11 Nov 2024 18:49:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E6B86B00E3; Mon, 11 Nov 2024 18:49:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C3876B00E4; Mon, 11 Nov 2024 18:49:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 24A396B00E2 for ; Mon, 11 Nov 2024 18:49:09 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2E142141B16 for ; Mon, 11 Nov 2024 23:49:08 +0000 (UTC) X-FDA: 82775456316.23.67E786D Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf27.hostedemail.com (Postfix) with ESMTP id 701D64000B for ; Mon, 11 Nov 2024 23:48:24 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=2PJDInq4; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.170 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368714; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MuWHSESvZxScxt1gtQJTtWY+NoQXodLd0kknM0wzZSI=; b=SwjWBUtrkfRW/nVxvHKYADOkhPW0vWnE6QnDjkYCWNSkIp9XRazNEg0xqOL0SBQCKn/I4X 6jNbpKu0fw1B8/AhWsYPZqB8TwVYH3KAg/7uf9JT8cQTM5InJZHNPfTaR0IV1/gAOJUf5h NETUDVgXqO519rN5d1rDLEx2g+cOqB4= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=2PJDInq4; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.170 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368714; a=rsa-sha256; cv=none; b=Ic7Dc9MmrTHP+z4k7sCSqSNo0BP7gFsohfpdM4s/uGxrwjtaRcxW/TLJe8h0RVeVF8Ulfa QzdbQ0cjX5HNxz8FB6et6/W2zUe0yMjS8aPh3m8AwrmII4uF0csd4DRYoJojHB9f3m5ouM zkreiSuTkLG/dGfD8suHewriVl7WwXI= Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-71ec997ad06so4154542b3a.3 for ; Mon, 11 Nov 2024 15:49:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368945; x=1731973745; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MuWHSESvZxScxt1gtQJTtWY+NoQXodLd0kknM0wzZSI=; b=2PJDInq43te7Q2p4LBx95oG6F7FfIq2lqCU4T2+ftJkPsIgnamhhvb8QJyyC+yfSgl LB9576RMDb3Ex1xWiCrs7Z5AYlMALxQmPpphFHEWQ3AF6fouaYcptgDU5QEfZSVERzVM xfrFTfVeNLv5aNb17Crw8VwDFLlYyKe9WadJHawp75iEzDeb2KNtiSIQg16irbXdSs5/ cTzGStOqALLf5khtkROetZEnDJXBuO2DLa7avOW9DFolJIYsioUJUV+lAtxebzgJG3A5 XdDtaUrtUcHW+JohNE2hkY5lyNzmn0HHZ3zDO2DMEDzrnGZE3oRUP2z8KmfJjd23lqpy Kvvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368945; x=1731973745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MuWHSESvZxScxt1gtQJTtWY+NoQXodLd0kknM0wzZSI=; b=tnXgAIfwVdtjTNUDZntvDB11M8G9Atf1rBfN0tF+EqEALjV9rqChgPV/FucnI9FI2T HCzoPDxiNzIXTl+J5HANyzteamOjTNSx3gYN4HhG8nS9QGxhOrF9GLc2GTAywkgq5y6k 845G+UacRmlop7Kj1UYEEvrCwbioL8X/SWaNTvLgF2de/Jkc/XQKnnvfA1orCm95GSiW RU33GqKGQTe0nYtdC10HAIkUcLFXMAqDL8yIt+nxQFbFODUzRS7Nc9aKaNGs+2TNQlmH 9u0NjddECFxp7nJ+NbqDyaxwrsD9hdo+tvJLp0TwzL1mv5iAdD5tFB/nTHFqBbowRT+q VtLQ== X-Gm-Message-State: AOJu0YyqtcnuJ9qrWI+0tX6OyD1SFp/1eMX676Pus0eokqpLq0RVLdMv qnhogSnYn7E7MVseb6sZxsBCtH+hncgEdeYe1umOY6Tzojnr9xVDqOi7MvUAAGwMjRzjS7C/S04 vZqc= X-Google-Smtp-Source: AGHT+IGY8U2EPa5TOjOCgJt8zYCC4on7T5T+FZCdrixkNsVQpkNRkQsZMbpHQUFVB4Pf9fNlDncpCQ== X-Received: by 2002:a05:6a00:22d2:b0:71e:755c:6dad with SMTP id d2e1a72fcca58-7244a4fdea0mr1056066b3a.5.1731368944673; Mon, 11 Nov 2024 15:49:04 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:04 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 11/16] mm: add FGP_UNCACHED folio creation flag Date: Mon, 11 Nov 2024 16:37:38 -0700 Message-ID: <20241111234842.2024180-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 701D64000B X-Stat-Signature: e3zpqmqz4cfb7hhriug86obxb8br64cf X-Rspam-User: X-HE-Tag: 1731368904-535872 X-HE-Meta: U2FsdGVkX1/StCpz95sYWLpUGxyoBmJELRPe5RAA8tY/chJ5J8gfNmyBRulDSzaHPC3GUvqB/924oU33jD1zR4eBF2Dz1miLnXowWqZc6WyvlwOHjA6eOZMrrD4KGbKIeUCMmerDmxU34BRRy2eJDIlvYywLqLS/M+gqisUIgnz01MQSHAjHSPUMkW8rLvL6+s3yonp9pOSxsHjp4Tlw7DfBUT3zEKlNtynbhY7f+ShEYADwaGkIocNRw2tIC/mnOQKYHFCkgjsZVWGlStpA4ScPku97iDLnxx0zAF+gFs3DBQ0lgAAM6sTEeptLk9mYIpiHcYuFkjQwQWqkHYqRXaSN+q89yXAv1xQx3Bj4L9sxTpqGJavq4yOwQX9MGpezvjUHrH0u8uq6BT7HXNjPldugSpfLT+lMnOp0rfs6Alf6eBzwKpKgmUopN1KDlvj+S19FW62G1kMJ8LSlLzE43WKDyrZfsh/lD6u9uc4NY6f0LnpgKOpLNntM7DADuwCf/moCLxhOGfQBiIwDwwV1Fax+znwn5OnMfvUHJkSySK2RLno6sNnQyB9E5SvQx7z1o4S0xzGNrhAI/CPOu7KcfrY8SODGQqhMPeo3pRPupIiCuiZ6Xrncr7W34P1VYaJoxiHB2fLPq6enwOF2Vume2ZgcE4sjNtsOSN4zyaxkw1Ww1Zo2CjLAPloKrOMInl9gsox23C/YSl8GOKK0YfwT7Nsoa/4IVWid3FRV+BbDiAupZVIkFYqbG38Z4xIKSTFT6fit6z0S9oWtxcmefTcGBnmFepbdgdECzRPg2k4qB/yteD3W1lv4eqmJO5Jje6xB2CuH2x2WtcJ+bfOfAqJPtX+6x/q5ECZxM1h6QqRpkAgQ88PLu4dWYNG7iMyN85guR0nyiVP1jFjD1XGiedpODH90E7BZscSdpAMZu6fvHJt2Kfwq/SwbudMQi+XL+wvMvEjBGETr2HUjtKD9BFy 9Qa3SBPS lmr2bAlpuu+CmTk5gRjxtqHfByluwwxDIU2Vhvc53kxZHFD19YN3JUIO+DLh4y/SXBf0qLx/eROHiU0O0fytYTlwYCvfhIAJbg26UKcHB6JXQzsTMECYbgaqfNLQJRrLDY5aEprN4EoThjyynMCI1qV2IxTM+poFmex5mLL0FjcXuXKRYYf1RdbbcKOaoL6IwYVcK66EuogrAf87mzbgQmPOcenUUX/4EvJR6Zg8Zg1HE2uUxETAbWE63/4QGB0JNaxURdUqeJa43R0RMtBpLki7GSeOEnUqDTke0zOzH2cJp+CnbiJL34nMtFXovk2yBFf2I/OxTw4jgSvUUHZt2Ez4dgje5on4xGcC3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Callers can pass this in for uncached folio creation, in which case if a folio is newly created it gets marked as uncached. If a folio exists for this index and lookup succeeds, then it will not get marked as uncached. If an !uncached lookup finds a cached folio, clear the flag. For that case, there are competeting uncached and cached users of the folio, and it should not get pruned. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d35280744aa1..0b298e81fcae 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -741,6 +741,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapping, * * %FGP_NOFS - __GFP_FS will get cleared in gfp. * * %FGP_NOWAIT - Don't block on the folio lock. * * %FGP_STABLE - Wait for the folio to be stable (finished writeback) + * * %FGP_UNCACHED - Uncached buffered IO * * %FGP_WRITEBEGIN - The flags to use in a filesystem write_begin() * implementation. */ @@ -754,6 +755,7 @@ typedef unsigned int __bitwise fgf_t; #define FGP_NOWAIT ((__force fgf_t)0x00000020) #define FGP_FOR_MMAP ((__force fgf_t)0x00000040) #define FGP_STABLE ((__force fgf_t)0x00000080) +#define FGP_UNCACHED ((__force fgf_t)0x00000100) #define FGF_GET_ORDER(fgf) (((__force unsigned)fgf) >> 26) /* top 6 bits */ #define FGP_WRITEBEGIN (FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE) diff --git a/mm/filemap.c b/mm/filemap.c index 0d312de4e20c..0949f0f340f5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1985,6 +1985,8 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, /* Init accessed so avoid atomic mark_page_accessed later */ if (fgp_flags & FGP_ACCESSED) __folio_set_referenced(folio); + if (fgp_flags & FGP_UNCACHED) + __folio_set_uncached(folio); err = filemap_add_folio(mapping, folio, index, gfp); if (!err) @@ -2007,6 +2009,9 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, if (!folio) return ERR_PTR(-ENOENT); + /* not an uncached lookup, clear uncached if set */ + if (folio_test_uncached(folio) && !(fgp_flags & FGP_UNCACHED)) + folio_clear_uncached(folio); return folio; } EXPORT_SYMBOL(__filemap_get_folio); From patchwork Mon Nov 11 23:37:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871496 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FDB8D3ABF4 for ; Mon, 11 Nov 2024 23:49:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5881B6B0092; Mon, 11 Nov 2024 18:49:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 537C16B00E3; Mon, 11 Nov 2024 18:49:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B15E6B00E4; Mon, 11 Nov 2024 18:49:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1772B6B0092 for ; Mon, 11 Nov 2024 18:49:10 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CCBF612131C for ; Mon, 11 Nov 2024 23:49:09 +0000 (UTC) X-FDA: 82775455896.27.87586AC Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 89B9C120021 for ; Mon, 11 Nov 2024 23:48:12 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=YLdE+fI9; spf=pass (imf29.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.172 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368894; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TkS4Adcluut9E9TBSf0Q5Dr6BdVnPYdslAPkTGEhwiY=; b=Y/q1sfEXzw7nPX1Cd7J2d64FN8cN00KBZAKab8n4goatVfownnO41qudc9ymS30hsaiNue 5a2pMMCd8zxiKPSfh1fG3gmeTRAHZAX6FBGzkeum7QgjoiSr2zUN9/eMfaZPqSzSPSSea8 aFiuGGhQhK7fefhfCSVOb0C5xt4dS60= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368894; a=rsa-sha256; cv=none; b=6lnKtkYtZWTpflDzZsdSeh7AVAUB9jHQXOQLb3FVoV9X4uTbcdL1aup6YpVwMjUusPNbTI RgsKfS2HHwe+mQ3yGpHROZ4ptTxBGU2ZDSXtVvEUr1Qy2cvMAQDKv4e9OE3jLURXyDPf6+ /2AbPjzD/aAfFaoHSNqxgOtTHuDvFDc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=YLdE+fI9; spf=pass (imf29.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.172 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-71e681bc315so3584369b3a.0 for ; Mon, 11 Nov 2024 15:49:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368946; x=1731973746; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TkS4Adcluut9E9TBSf0Q5Dr6BdVnPYdslAPkTGEhwiY=; b=YLdE+fI9R9wQWL+kmGTjJV3CN1+eVnF+gOugIn/dpPsA6jJdHdfa6J8lCDEjIcI4lS v8UuYZwuT8cN2bvYqDwk1dZ8lY2WBwu7XSYzUfwq4YGtNffhPkGSv6BuobfMbDaiBmSN Pok/XXBKz0+1t/2ZPK2sjlaf/cEEtTgwRwt8wZA6XsErf0F/Oc5c2D1iUrL9WaW9pRYD dUdUU+GJDSGFFkF2NQHYMWFUoaxgCqNnN4bwF2aIQphyBmtE35b/wsDrNGIfBXACxlZG ULbKhFeaEa0OchVNPXqGx62KjLgu4F6qVaILaWJHaXMgSAvLto6ET7CvdkH0jcLEcqmV vcjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368946; x=1731973746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TkS4Adcluut9E9TBSf0Q5Dr6BdVnPYdslAPkTGEhwiY=; b=UXNSNJgPy383sOupaYjz0XfB4YM6LsCyCbKK3YuIAo5IUzMd6cDquYwLt9E+6kuHLV vH8H/2/4Y5pZ860Q+Iyw4dQ4tUN4eo9GxYjcG9hN/pBbANgZvgElA78gGk8QvcPjaU1e XcdFkNeITgaMVgIs0Q17llJBjcrfs6UmrEMLcykYBcc9Fhbyr7j0+ud1yFI4m1qUMYz1 uefOV5ASrInzt3di7TTGtKG5PlwjO8Q6d7Io9vLGYxPcic93IVxTDiAP4+/4QsZFYWn8 A91PTeCpduyKloKOhhmecJ1zLYBxpTulL/BF6oZ6TRc/U1qKQHwqtAiAjeTTBqa3Tvrw YlfQ== X-Gm-Message-State: AOJu0YyNQYeXyghoJ8D5WehEFlVQKXQmHaKjwhJs7TR70DrPhdZhkyxd jr1HRGsEPsRJ2d/0wOa1xioTtky98HjCQJROYWTWDh3cXJIwxSW2uRurNzUUk8SlBXOiGdjVJk3 Lbtk= X-Google-Smtp-Source: AGHT+IH28qnmC+d/sJZ3EdxK+NK8za6V+wXT31dNckiLvxFVwcmETmxIGu/BBIMvr9bfEpwgK4E2eA== X-Received: by 2002:a05:6a00:a1a:b0:71e:71ba:9056 with SMTP id d2e1a72fcca58-7241407b632mr20781928b3a.10.1731368946274; Mon, 11 Nov 2024 15:49:06 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:05 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 12/16] ext4: add RWF_UNCACHED write support Date: Mon, 11 Nov 2024 16:37:39 -0700 Message-ID: <20241111234842.2024180-13-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Stat-Signature: weqb3a7hr91n8jzg51mbq13b47uf33hm X-Rspam-User: X-Rspamd-Queue-Id: 89B9C120021 X-Rspamd-Server: rspam02 X-HE-Tag: 1731368892-996565 X-HE-Meta: U2FsdGVkX18BneiZenvwuiOGUkMzEJ8IrXqbYZKEDe+xLmArUmY3pkFtuDWt9crF60Sv9uOYXSIYheKOtD9iHZIDKPWWpFeM+XUuDCmWblmlzCxiSzBiWraZQwWAW5dvBXK9/pxxjeoRujEm/TIeb0to3H3HuR8CvJvu3Gv7SkjHYKTXnNstcPu75LQaVPYZLdbfRgq9SLwB/oTduKa0WyDZuA25GkbSmgtB6CEgq0zVZPuCEgsu49spekrVhoOcnhifBB2xbySFJOe5rH6AoAegRQ2sK4cRUSumDYn1tJTo/6CtsCyTeyleo1qk02ZAXS70gyhRTUIE2v62T4Zxr5DsTMxJhQekCH1tb1eSiwas1sI0yPwllP30EJYFm1Mk0iZaFI/E64OPzYtExsrNdAdTzP8b2EQkM58eu8cwqslsTeYeTi+vq3srOp4qDbVIcXMuZnJJ2unt9cOz+QVf0HMU6pdSRjhgLkG4TJhEmIAS74l6cG0cqqa3QL5kJ3+lGZ12wUV7cSc8+78ntVvo7OGank0tFUOt8whimyTO3+AeBr9ztICyFoVVxFLIv20WO9IgdTRQUMqg/rGvNT7/FlZg6RGp/sLlQgl2vpBtd8CtaJ84YoZ5BYAXjcaLLfMr4gJMvlmN6jgqelnG4iQwUj00ZTeeITVs8K0JVbFR6sQDKKw7hsTOOnAZuSPqZIrdr8KOkqJuV6Fq3iTUrPi7DbRO8VCxtHopbnSAiPjrWSpRvrDMvU0LofyDyVArd/W3thaXUpEVX9tqZtwrMtEUH0KaoUVilOJ1odVNuYkBLkv9PvbQ5X9qoF5wfvbUAgUc1WS6r8c0IVhnib4pWSiWEZM5Xd+spedAuCPTpP1DD9hm6iYYR4kavabqycWN7OmNyFP3v4AYZlMf4Q9MR/4TU9aIuwM9EYkTUII/PVyevH2WG+Gn2D46SjkdfygPAKdCwbUocjcyaDz3Pik4J45 u4Xsg7NT 9xvdgD8tlmCIfBaU+tF+tuzmessAd4Oj55O+xGojaOPmYBd00aDM81c8gkUX8U+f6igebOsyhaPoMulloFDbfGK5los3DwTmzB2bc4u/Ek3HI/TC9vDieoUp5aUfI4rVWcM+ptnPl0myCVbnjX+TfDArYt03DJ0G20nXqKQe8tQBrlFRloaBnblMMp1CvUi5LuAXK6jw7e6lJusEBiDlOVfiYCscRvDMYdJ/xMj5mWraJCURCf0nhyn553F1xMrZqWuBZsqVTvmZ4CkNE+HnXi8q+W9qmUD1xRT1Qm5iGOn/PMy/LaFWJtlGAaJKUq+0Z9WCHWk/m0LYbFDHDwFuDJKj1jAX9/LAj9tDs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: IOCB_UNCACHED IO needs to prune writeback regions on IO completion, and hence need the worker punt that ext4 also does for unwritten extents. Add an io_end flag to manage that. If foliop is set to foliop_uncached in ext4_write_begin(), then set FGP_UNCACHED so that __filemap_get_folio() will mark newly created folios as uncached. That in turn will make writeback completion drop these ranges from the page cache. Now that ext4 supports both uncached reads and writes, add the fop_flag FOP_UNCACHED to enable it. Signed-off-by: Jens Axboe --- fs/ext4/ext4.h | 1 + fs/ext4/file.c | 2 +- fs/ext4/inline.c | 7 ++++++- fs/ext4/inode.c | 18 ++++++++++++++++-- fs/ext4/page-io.c | 28 ++++++++++++++++------------ 5 files changed, 40 insertions(+), 16 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 44b0d418143c..60dc9ffae076 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -279,6 +279,7 @@ struct ext4_system_blocks { * Flags for ext4_io_end->flags */ #define EXT4_IO_END_UNWRITTEN 0x0001 +#define EXT4_IO_UNCACHED 0x0002 struct ext4_io_end_vec { struct list_head list; /* list of io_end_vec */ diff --git a/fs/ext4/file.c b/fs/ext4/file.c index f14aed14b9cf..0ef39d738598 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -944,7 +944,7 @@ const struct file_operations ext4_file_operations = { .splice_write = iter_file_splice_write, .fallocate = ext4_fallocate, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_DIO_PARALLEL_WRITE, + FOP_DIO_PARALLEL_WRITE | FOP_UNCACHED, }; const struct inode_operations ext4_file_inode_operations = { diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c index 3536ca7e4fcc..4089d0744164 100644 --- a/fs/ext4/inline.c +++ b/fs/ext4/inline.c @@ -667,6 +667,7 @@ int ext4_try_to_write_inline_data(struct address_space *mapping, handle_t *handle; struct folio *folio; struct ext4_iloc iloc; + fgf_t fgp_flags; if (pos + len > ext4_get_max_inline_size(inode)) goto convert; @@ -702,7 +703,11 @@ int ext4_try_to_write_inline_data(struct address_space *mapping, if (ret) goto out; - folio = __filemap_get_folio(mapping, 0, FGP_WRITEBEGIN | FGP_NOFS, + fgp_flags = FGP_WRITEBEGIN | FGP_NOFS; + if (*foliop == foliop_uncached) + fgp_flags |= FGP_UNCACHED; + + folio = __filemap_get_folio(mapping, 0, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) { ret = PTR_ERR(folio); diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 54bdd4884fe6..afae3ab64c9e 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -1138,6 +1138,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, int ret, needed_blocks; handle_t *handle; int retries = 0; + fgf_t fgp_flags; struct folio *folio; pgoff_t index; unsigned from, to; @@ -1164,6 +1165,15 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, return 0; } + /* + * Set FGP_WRITEBEGIN, and FGP_UNCACHED if foliop contains + * foliop_uncached. That's how generic_perform_write() informs us + * that this is an uncached write. + */ + fgp_flags = FGP_WRITEBEGIN; + if (*foliop == foliop_uncached) + fgp_flags |= FGP_UNCACHED; + /* * __filemap_get_folio() can take a long time if the * system is thrashing due to memory pressure, or if the folio @@ -1172,7 +1182,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, * the folio (if needed) without using GFP_NOFS. */ retry_grab: - folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN, + folio = __filemap_get_folio(mapping, index, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) return PTR_ERR(folio); @@ -2903,6 +2913,7 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, struct folio *folio; pgoff_t index; struct inode *inode = mapping->host; + fgf_t fgp_flags; if (unlikely(ext4_forced_shutdown(inode->i_sb))) return -EIO; @@ -2926,8 +2937,11 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, return 0; } + fgp_flags = FGP_WRITEBEGIN; + if (*foliop == foliop_uncached) + fgp_flags |= FGP_UNCACHED; retry: - folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN, + folio = __filemap_get_folio(mapping, index, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) return PTR_ERR(folio); diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index ad5543866d21..10447c3c4ff1 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -226,8 +226,6 @@ static void ext4_add_complete_io(ext4_io_end_t *io_end) unsigned long flags; /* Only reserved conversions from writeback should enter here */ - WARN_ON(!(io_end->flag & EXT4_IO_END_UNWRITTEN)); - WARN_ON(!io_end->handle && sbi->s_journal); spin_lock_irqsave(&ei->i_completed_io_lock, flags); wq = sbi->rsv_conversion_wq; if (list_empty(&ei->i_rsv_conversion_list)) @@ -252,7 +250,7 @@ static int ext4_do_flush_completed_IO(struct inode *inode, while (!list_empty(&unwritten)) { io_end = list_entry(unwritten.next, ext4_io_end_t, list); - BUG_ON(!(io_end->flag & EXT4_IO_END_UNWRITTEN)); + BUG_ON(!(io_end->flag & (EXT4_IO_END_UNWRITTEN|EXT4_IO_UNCACHED))); list_del_init(&io_end->list); err = ext4_end_io_end(io_end); @@ -287,14 +285,15 @@ ext4_io_end_t *ext4_init_io_end(struct inode *inode, gfp_t flags) void ext4_put_io_end_defer(ext4_io_end_t *io_end) { - if (refcount_dec_and_test(&io_end->count)) { - if (!(io_end->flag & EXT4_IO_END_UNWRITTEN) || - list_empty(&io_end->list_vec)) { - ext4_release_io_end(io_end); - return; - } - ext4_add_complete_io(io_end); + if (!refcount_dec_and_test(&io_end->count)) + return; + if ((!(io_end->flag & EXT4_IO_END_UNWRITTEN) || + list_empty(&io_end->list_vec)) && + !(io_end->flag & EXT4_IO_UNCACHED)) { + ext4_release_io_end(io_end); + return; } + ext4_add_complete_io(io_end); } int ext4_put_io_end(ext4_io_end_t *io_end) @@ -348,7 +347,7 @@ static void ext4_end_bio(struct bio *bio) blk_status_to_errno(bio->bi_status)); } - if (io_end->flag & EXT4_IO_END_UNWRITTEN) { + if (io_end->flag & (EXT4_IO_END_UNWRITTEN|EXT4_IO_UNCACHED)) { /* * Link bio into list hanging from io_end. We have to do it * atomically as bio completions can be racing against each @@ -417,8 +416,13 @@ static void io_submit_add_bh(struct ext4_io_submit *io, submit_and_retry: ext4_io_submit(io); } - if (io->io_bio == NULL) + if (io->io_bio == NULL) { io_submit_init_bio(io, bh); + if (folio_test_uncached(folio)) { + ext4_io_end_t *io_end = io->io_bio->bi_private; + io_end->flag |= EXT4_IO_UNCACHED; + } + } if (!bio_add_folio(io->io_bio, io_folio, bh->b_size, bh_offset(bh))) goto submit_and_retry; wbc_account_cgroup_owner(io->io_wbc, &folio->page, bh->b_size); From patchwork Mon Nov 11 23:37:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 083FBD3ABFD for ; Mon, 11 Nov 2024 23:49:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D43D6B00E4; Mon, 11 Nov 2024 18:49:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6EA3F6B00E5; Mon, 11 Nov 2024 18:49:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 512566B00E6; Mon, 11 Nov 2024 18:49:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 276E46B00E4 for ; Mon, 11 Nov 2024 18:49:12 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A1C36C1A21 for ; Mon, 11 Nov 2024 23:49:11 +0000 (UTC) X-FDA: 82775456862.18.45B9729 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf09.hostedemail.com (Postfix) with ESMTP id 6488D140006 for ; Mon, 11 Nov 2024 23:48:41 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=OS6IDF2T; dmarc=none; spf=pass (imf09.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.170 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368887; a=rsa-sha256; cv=none; b=2EqyqvQ+USECkfVCiVREhxgVwRByv29J2XDCc4JOCb4llEK+PhobiWC9Nnm+KsGhrInF79 B1wG+3BjqCpix4rvwZ5KmRV9CbmL1Ld8ibHHxFKA5SMo/zb4P7jGBCG/11TseHj0Ue7WKq n4tJVf0LVZxwx6xHvFj3lGxiRJqucLs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=OS6IDF2T; dmarc=none; spf=pass (imf09.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.170 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368887; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QHHnA1NsQjr1van9cnJ+PQRWRXhY+a6ETIPA3kXFfjk=; b=vPbpy7utgf5O8nMsL0Y5VpdqCxa+1ty1v9Sn5QB1UtjpmWlcukx2a32AsLapve9guyzX22 nmCyyg1oxvqnlH6BpMfFMlu89gr9RlqvfrFCoaGnZe3g4tZbo4DGf01lloqNELk6Ah2RkN L6fTjH6I4KBeLgMAJgIGE9OkaiWYggY= Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-7240fa50694so3694862b3a.1 for ; Mon, 11 Nov 2024 15:49:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368948; x=1731973748; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QHHnA1NsQjr1van9cnJ+PQRWRXhY+a6ETIPA3kXFfjk=; b=OS6IDF2T2YecJ3Ua2eCHf5/FzcQMorYmvaxLMmDlIevNirs5I3DXFwAClyeEfZzxMt +AiB9WLEaxaXEadTCsb7L+LcH2YFiAIOMkXYh/Ze8uQwp8nrzNbmkzm7cfHPS7itOMUb o5UpYWiRUKQxPJUeInRArEu+wQV+cRiotDWiAyCLfa0hvogLqTk4Gk4BeXdSJt+Apd+Q l25eNZ2Jk8ZDc93gUWHMxfb4USYAHOSYSzsr0iQVKo4ofQUy3QljQLyFv6EAgSAOq1/2 5OEEK4gPGwsPhcwIeTShmWlXD6fBI2qVmQuDAUTQ/5p4bg5Pip0Xz68gM4p0kOKIs69P PBQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368948; x=1731973748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QHHnA1NsQjr1van9cnJ+PQRWRXhY+a6ETIPA3kXFfjk=; b=NB31gYvYbN68nFyZcQAYDaLDCoQk8e8kdcsHvopWrtI7p8A5ujC/t116UqiIBNfQt1 DI/DKBsdzkJ5twKuviJU/45nRVYfUVSWGqaAht1zBCVTGZzqjf4HD7Tc50NnEHZEQg24 UV3t8mN6LqjrlJR++HVegAWVtzYJ9M28uEHHU9isBFh7bkhru1PcqHRTCOuvLtknHV4m 4+mhy7q5pFFkxyKnTend/k1GY3rhbUrLPFHeeELRBalg37PJkzZcKeVINDaPnhYMNtk6 YGt2WkO9q1l+CKEY+a2dLAiZecrobQHLhK81V4oTu/CVkvBkMzr/gjH2h1hL7pfjBlCO yoTA== X-Gm-Message-State: AOJu0YwPQWSynT8ePbuvsZ/RbZ5VteIxC1vMOEEwIIS9DXzFLR7hGGCg vfyHGmgoZ7P5w7+25n9UgrhoPIpbWcUkqTGiFGHQ8i6oVhE58ocKhjmO3sj2JWiKNioS4fC4FTj rp7Q= X-Google-Smtp-Source: AGHT+IE+DUrjoGziw6EjyU9hF9MosU8fZ1F2/8qqkRJb8RqY6XTyhSKojrzpZHSNP9ce3wMHzfWwwA== X-Received: by 2002:a05:6a00:181d:b0:720:aa27:2e45 with SMTP id d2e1a72fcca58-724132c535bmr19020120b3a.14.1731368948118; Mon, 11 Nov 2024 15:49:08 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:07 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 13/16] iomap: make buffered writes work with RWF_UNCACHED Date: Mon, 11 Nov 2024 16:37:40 -0700 Message-ID: <20241111234842.2024180-14-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 6488D140006 X-Rspamd-Server: rspam11 X-Stat-Signature: upa11tow3iu6qc4wkchpkjnw5gepbp1k X-HE-Tag: 1731368921-460131 X-HE-Meta: U2FsdGVkX18cVOgnc3eG1+v10G/vHq8aFH4Ha+3N79AbnJhvEWIVu3CmgFrVWQdRacgfSGs4oXcMS5X3Ki9+vFNC5u6hMkpaF0cN/ump7ArI/g44d8zXWIqOjmFgYNpES/xlvkpBG9fykWiDeMUHG+Hqa7UXPWZhAHjJ7QWqwA0gtFIowCXKxJcjevFcxzzd4BTw6bvkCgMSGcvrjTGzK4Yau7V2FPkbo177Qu+tCv+t/qt8YYxlLkuxMQP9rfppAq18LwO/ZLmgmqmFoWGyEx8ZMqbBFwjA/zbOq4QWmFc13ko0ig1Q7uNB79RQ0B5qUlzCTN+8gKzQCAwpFHlEdwAK/wQiywFDQ/RmOPq+q3c+XkT3xU3qQQOtoj5wpNefW4zIiIKgxdLopcIS+Hg7915fUoJf5y4XbW8hPEU58Arvl8TPmg4w5QNwBCw6wwVmcLqBv68CGe76+9/HyZzolXJJAS0uJeXkUHgYdVnZgeOllayVSaln43jggYNoSRKkbLNgKucx7cvXbJuebPVWlIMyS3FteDc60rPne0++tmSGIKJsqe09XxdXjdzhxlvtCKQWOz4X5ZnRV54vros2l9EUkcvtBB9V6xENQY+b4mV26rK0jEKS4JaOilZBojDG0kMMRWf8OSdX3LWmpdslvE4jjXWoeye8L3hiX5p3chBBmFbuPEoUEkDCgacX9YYrtZJi3zTvshwSWhgOl8kc4ZjPTDxoi00/7z/LLOSIzVX6XlERStLX2OBdRumJPdTaV4IG7QGPAimnBquibnc8Ao+4zY8n1TIxJfj7VSgB2Is9x27RIEF8VBJ3OuqsVbgYMDqKx3/XU/GT4sfPI5wBfB4CrH+o6mI1wmkwXPjwmNVAf64xYDH5Eb0OahMTfe8eWcC+tryeWrN9HXytD35BEyW0oL9aqvrwA2GqP3qpamnB5Zz0ueZ+aAGZHQQsbS8CubH3lGiaVno57r3OETq Pfk7XhRX HWV1pZgHW5HIyDIpwIy5sQl04dMmJ8jvt7MEh74aZClKfomvieXijcXhhY0fWj8pUH+P93oa5EfklvpYZjIEdDH8xj28/7YDRQhfSpTgHqDacZOfVfVDto8wHixnDXU36EpI4Mh6AJzAUXBaMP5PSzysEQ2HkmcyFMfYds0Sw2bMW4lTg2uhMWjjqLVKubswtA8oG/GntFQJjLzrW6Ua7sHn91VloJl2GssoVxNjFK9DPWQyC7Xb+BTz3UoacRrqW1r4tUmj8ltlUlWhq9mvvdavkGfI4Lz/nqKeOSqtm3sgenbmKruttaugfvDCPMTaA73ZwKEvlvM1tZV0sIJ2c72SWtKGpV+NvOCp5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add iomap buffered write support for RWF_UNCACHED. If RWF_UNCACHED is set for a write, mark the folios being written with drop_writeback. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest. This still needs the user of the iomap buffered write helpers to call iocb_uncached_write() upon successful issue of the writes. Signed-off-by: Jens Axboe --- fs/iomap/buffered-io.c | 15 +++++++++++++-- include/linux/iomap.h | 4 +++- 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index ef0b68bccbb6..2f2a5db04a68 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len) if (iter->flags & IOMAP_NOWAIT) fgp |= FGP_NOWAIT; + if (iter->flags & IOMAP_UNCACHED) + fgp |= FGP_UNCACHED; fgp |= fgf_set_order(len); return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT, @@ -1023,8 +1025,9 @@ ssize_t iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, const struct iomap_ops *ops, void *private) { + struct address_space *mapping = iocb->ki_filp->f_mapping; struct iomap_iter iter = { - .inode = iocb->ki_filp->f_mapping->host, + .inode = mapping->host, .pos = iocb->ki_pos, .len = iov_iter_count(i), .flags = IOMAP_WRITE, @@ -1034,9 +1037,14 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, if (iocb->ki_flags & IOCB_NOWAIT) iter.flags |= IOMAP_NOWAIT; + if (iocb->ki_flags & IOCB_UNCACHED) + iter.flags |= IOMAP_UNCACHED; - while ((ret = iomap_iter(&iter, ops)) > 0) + while ((ret = iomap_iter(&iter, ops)) > 0) { + if (iocb->ki_flags & IOCB_UNCACHED) + iter.iomap.flags |= IOMAP_F_UNCACHED; iter.processed = iomap_write_iter(&iter, i); + } if (unlikely(iter.pos == iocb->ki_pos)) return ret; @@ -1770,6 +1778,9 @@ static int iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, size_t poff = offset_in_folio(folio, pos); int error; + if (folio_test_uncached(folio)) + wpc->iomap.flags |= IOMAP_F_UNCACHED; + if (!wpc->ioend || !iomap_can_add_to_ioend(wpc, pos)) { new_ioend: error = iomap_submit_ioend(wpc, 0); diff --git a/include/linux/iomap.h b/include/linux/iomap.h index f61407e3b121..2efc72df19a2 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -64,6 +64,7 @@ struct vm_fault; #define IOMAP_F_BUFFER_HEAD 0 #endif /* CONFIG_BUFFER_HEAD */ #define IOMAP_F_XATTR (1U << 5) +#define IOMAP_F_UNCACHED (1U << 6) /* * Flags set by the core iomap code during operations: @@ -173,8 +174,9 @@ struct iomap_folio_ops { #define IOMAP_NOWAIT (1 << 5) /* do not block */ #define IOMAP_OVERWRITE_ONLY (1 << 6) /* only pure overwrites allowed */ #define IOMAP_UNSHARE (1 << 7) /* unshare_file_range */ +#define IOMAP_UNCACHED (1 << 8) /* uncached IO */ #ifdef CONFIG_FS_DAX -#define IOMAP_DAX (1 << 8) /* DAX mapping */ +#define IOMAP_DAX (1 << 9) /* DAX mapping */ #else #define IOMAP_DAX 0 #endif /* CONFIG_FS_DAX */ From patchwork Mon Nov 11 23:37:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57EFAD3ABFA for ; Mon, 11 Nov 2024 23:49:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E1576B00E5; Mon, 11 Nov 2024 18:49:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 119DF6B00E6; Mon, 11 Nov 2024 18:49:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED5966B00E7; Mon, 11 Nov 2024 18:49:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CBABF6B00E5 for ; Mon, 11 Nov 2024 18:49:13 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 882D8C191A for ; Mon, 11 Nov 2024 23:49:13 +0000 (UTC) X-FDA: 82775457366.28.2C2EFF5 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf03.hostedemail.com (Postfix) with ESMTP id C03AB2000A for ; Mon, 11 Nov 2024 23:48:52 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=NBsRF9fh; spf=pass (imf03.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.180 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368779; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=tyLfkpokH2JpSbNxGhKv0ng1zTTzleCRtNRZe5/XXzuj45txIQ1mgp0RCSqsnWUjxuDhm+ iK8WmEGe6DksD2idGHFpH6dnkAoNgMIE+so5YcBt223WfY0e3C7sJ9G9uiT2t7LPdO179c M0GfL5TtL0N6qzzRUNR9UlbUZI7PN2M= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=NBsRF9fh; spf=pass (imf03.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.180 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368779; a=rsa-sha256; cv=none; b=S7n950sZdqlbz76s0FhnEDrapkauEDL3iACkEdImyZyzz3j6GKfu0+rctDZTzT0f4JdFkD NnV3UN375kBU1N/7vvmmMGvN5q36Mx803gaZKNcTFebDwsnFOx/zDYmve3vW0Evdjwq2Tv VomWdPJcWqVnLL799Fe9RLE6B3tFpb8= Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-720d14c8dbfso4843549b3a.0 for ; Mon, 11 Nov 2024 15:49:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368950; x=1731973750; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=NBsRF9fh4U0120nIZzIZYUqq1lgIsaJIymUqHiIrMtmimrHum7iYKjCmw+L9QrMLt2 NZs6/WtdUwvPTC4ZbUOLkWOGzBjLVe8LoGKI2Gs8aN1nq4xBgN6ytNN11VcE5NR1o8IG Nq+me4/cecyYhmzt2/6Xk7tNPhk7STjZRlh0xMVoKaVio8dypv9TTlGXm8IuWTbQcJ3H 92sYt0vd0yqNp0K2EG3jGdCvqnXk3DrMaGoWrmH8eIZe5kWcXD+xLWRVuce8dX75mP/z rq/NGzT0xdSHOpp+cqR9jHjGjfuKdy+0h+Iw/e5D1G2b2cOtwduij8uP6YfOjSmFBKu2 JwoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368950; x=1731973750; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=g8j2weBMCxcIkyKiYZUC0/ytRrXNg9uyh6YCd6eRW2fPF+arm91FPCNE0pxQnfgSzS 5bWqpkxCnU2Gvqfm5kG2gfrRCVZ7yfJNl8RVI11GPm2E9witXUoJqo5QTshm4byXSBhr DDe/OJcSUyuj8NaTiHP+IBs3yEowY9fYwTfVT2HelLJDJ/ZlTvNVLwRIqHfF/rklOEza Z5ZDRdOjUWDFb4RXBxHJtAxB0x+IbpYgVud3S8uGmpQ71zwnu9cXCM7nps1mbDyUcKis bIqjA2caO8/WvmGzEAKnkkS6MNhPW6bmiZTxBXShnr6zQ8tY4LTAEt5X5BQVm7jMBzlF a9Lw== X-Gm-Message-State: AOJu0YxQEQ4hxgYncAI9SchsgHHuAvb2JwWcoTmsWCowA0PRiewr2I4F cLk+gAzYIIOe1/pTlPNNHWpmyXK1qT6Nns8Oo2DtYKZ2Yt3hJGHi5VLrfjY5i1eQaRiy/rclxoO +cjc= X-Google-Smtp-Source: AGHT+IGxdsMu1dfGOYWXi1R28aWFhjsN5Q3I34Dwpy3CjKRXcDtvBr7LZlUVTVK3RyQbj+sGtC8u7g== X-Received: by 2002:a05:6a00:3a29:b0:71e:680d:5e94 with SMTP id d2e1a72fcca58-724133510d7mr19932677b3a.19.1731368949731; Mon, 11 Nov 2024 15:49:09 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:09 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 14/16] xfs: punt uncached write completions to the completion wq Date: Mon, 11 Nov 2024 16:37:41 -0700 Message-ID: <20241111234842.2024180-15-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C03AB2000A X-Stat-Signature: nk3iw7jqi7a1a4ykzno1way1aixfsizu X-Rspam-User: X-HE-Tag: 1731368932-347957 X-HE-Meta: U2FsdGVkX19Ndp/yfVkBvs1aYGXq/g5NasTBVD+1nrpOCenxLJSHwA94jT75etKRTxMZUx+IixbHolRDDDZAHhQuiphQUoAxdeJR16vqRTggsKTsflnIkoTK+3EexjE2YtxTh7/lKCXY/L/OjlxJWmcg3f7qSRsfx/7NeSZ0B6MNZ8tx5o118C+r+mk2J8WL6AR5gImp8NOtFWLIzpsPwFO8SCQ7Sz8vQQp7QsfmghytX9H2wQLU++RWvkFzh8SMA146mqF8bgYVEVVxK6UYYHzPBUQj8lCfwpy5sZnwFaZHtvcj12n7HP6t19VLaXWhJi/EUdrrVloUPoOZ69igFMPxAcjPPNgyzVfW7gn6rFaKlExiEfLt4icpoQS56q9Jin1vcVleDsQIdXBUiPqPTeK+WYUV9+82alDGKiBapP7fwP4UaL8qEUabV+ioL4ebWG9fKzjCZ2CqYQsWsMB5IFk7EuLOoQp6kXkojkbGEaHtH+1G9pDsx6SJPx7XtpdsVWimvsgHFiiyQ9qw34jkjQ1U4B7ZDkl6P8yu8KiN7YD9Igqnfdf3ALJLDN99+czggHpgDPnTRmyUFJigtBhnfVbYPcqXCb/p5krn8fiXhXZ7iL84FZONW27EJRssj0yWAFL02jjrNQZv3SY2ugt/bCUAOFUFa4+aSXjFxul2t7oxkTFmopWgXiinPlOOTqjeT0wU1Cv9BIvFJFiQoDL6KEfhypR0mi2DCQnEJC1jj/AN0sm44k7niTmrjb6nkS6bB+achHimL7mWN9eYCN0ZjiZuDWQSYdOs+iZONxS7/F6MQVP4iwgvlGBi0KN5IXxIw7Xwslo6pRH588vkQ0Yiyh+U0IyEtXo18p0t1ObUyngk9Ng1EXNTwFm1tayCRO7VGSUtkrRUzzELZJfxVUnHoSIAOT0hzs/fcH1Gy0/+KBxCuEi2JDTxc9pv63cLMPYie9tX0V9iS+AtGjLmjud cICP96ih 68JceMK48HZx+bJ9yrx3yteUTTH/f5J2cb+VYBgUxiim9IR0I6ZhjAQABeG3n6JQIE4eangPTsU9gUpKdjM0GZt9l/IUqpqsIhpyDNbodxE6sGBnRM6AvN2AMijQsaXnxGv4SSr8fvQrhde/7bMTDpLeByXKMjuXU1IUpg4Ylmu2eZaZivfqYJUQTZ2VERsN0UqDup2Xr0t2iGoFzDitkoHcF2CDiDpDBAE4Ze8LBUHc0qauROJO2KzJlL2RagBRXqRm05tX3P5mQtHH23eJgnqpa8wwsN/k2P/XwMvyuTmUI6LiIAoyQ8AlNt4SvPcjK5B0GkkZkJ2SDGFQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.034356, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: They need non-irq context guaranteed, to be able to prune ranges from the page cache. Treat them like unwritten extents and punt them to the completion workqueue. Signed-off-by: Jens Axboe --- fs/xfs/xfs_aops.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 559a3a577097..c86fc2b8f344 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -416,9 +416,12 @@ xfs_prepare_ioend( memalloc_nofs_restore(nofs_flag); - /* send ioends that might require a transaction to the completion wq */ + /* + * Send ioends that might require a transaction or need blocking + * context to the completion wq + */ if (xfs_ioend_is_append(ioend) || ioend->io_type == IOMAP_UNWRITTEN || - (ioend->io_flags & IOMAP_F_SHARED)) + (ioend->io_flags & (IOMAP_F_SHARED|IOMAP_F_UNCACHED))) ioend->io_bio.bi_end_io = xfs_end_bio; return status; } From patchwork Mon Nov 11 23:37:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB6FFD3ABFD for ; Mon, 11 Nov 2024 23:49:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97EE96B00E6; Mon, 11 Nov 2024 18:49:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8E03C6B00E7; Mon, 11 Nov 2024 18:49:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A9256B00E8; Mon, 11 Nov 2024 18:49:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 582B06B00E6 for ; Mon, 11 Nov 2024 18:49:15 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 111B01A1C72 for ; Mon, 11 Nov 2024 23:49:15 +0000 (UTC) X-FDA: 82775456904.05.896195B Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf27.hostedemail.com (Postfix) with ESMTP id 5AB3E4000A for ; Mon, 11 Nov 2024 23:48:31 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=yXH8CiHi; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.176 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368899; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ieWCO7ZvnQOESu0rOvKqnHS1oC/GZ0fSczrbrfzlBys=; b=Y7DTia7Q/kNIwaBZccHH8n+mSZRa/sIiay5mduhpO+zSGZXfDXgvRbvTcf83VyldlrKgIv rVusYAcn6r0Q8FmtkWJ2PyKE6B+Wk1RmIUdxW3Xl2w7Rip9S0fU45Wi2DDNqO5b1yxUc5D Aro4XavDFMhf9f3D2ZHJR55UQ2cD+Y4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368899; a=rsa-sha256; cv=none; b=52qr0ZL4Z4vujCx9wyM+wUdLxvH9Z12YYIRRA7RmWchni8ks5Y5Joppw0INYyaMRacVJhK JGepQUOknvCVDC9VxmLxtVbQSNbe8iNDbPNgaHziAUWDO+Z4nQ19HjB+ouw3q/qp52h0Kg XkVT8mGlqLmxcKdfw629OvG6rutXkLA= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=yXH8CiHi; spf=pass (imf27.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.176 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-720d01caa66so4744625b3a.2 for ; Mon, 11 Nov 2024 15:49:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368952; x=1731973752; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ieWCO7ZvnQOESu0rOvKqnHS1oC/GZ0fSczrbrfzlBys=; b=yXH8CiHioyODYDz/FYO2otWNugOEw0Hq+XZ//K1tWdjsJaViJU4KH0J6+BHur7xmcT 4nJPS11bf/cCJC8BpsufWXp2Xzhl1POIuUSsqZ78h8nhnhC7xi4SdpPcNcaOPAHGychN xmfraCFByyfo5nzim3Qamb0sDl7MSHYeJykyQtcH8aR9LrFepa7y9oUlZjw/X3hX/fmI P+P1I45MvrXXe8wbELfDRuPOLeUVqpDp+JyupQWDPLC48wqWaUJcJQLzQwJPOzKA3PGE e/f1aIgCd4u6T81VojX+ZRbltwhmcgxINHfRAYX3QqC0St5+qGPVMLLTazb0oLmzBRE0 inoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368952; x=1731973752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ieWCO7ZvnQOESu0rOvKqnHS1oC/GZ0fSczrbrfzlBys=; b=U4WKTBkQTTM+W/gSqENhh71hCBHQoKcsQFZ0nlA9k12UIVimm+JXcLp77UIn+XeiVA NUl5XuArzCTVr9AKqrhZqk/3fsKIY7pCbeEeupgoNH4VdNmQgViEN4+PBY5vnFROJGkO oDem3PdonrF0jePHvavrMEg+P1h4KPeIihuVKHnI1wIZEllwyi49rCrhimcxoQp7c9E7 brKFNM8euVMHSkAgvqe1/VH0fVCGb3P2XfEguZWi1RlGK5euZLp7wgkPFwwQTq39mQXv N3pBNtEb76hRdfM+19obqjVqrjUKAfLVYY9KomqStf8h+jAMruzzrVtVGogyYDkkZZ21 ft6w== X-Gm-Message-State: AOJu0YxDm0ruRlQXSVQLr/LmRJ0J0UJDiEmhpWrNUmrOUrBH7OQ78pwl RLIfwm12ESbsNu/dkvrcrzb650P4ZA2QodZvEI59uHXQ1VYxGe9dw3UtAHMy9oH/7CP+1EDbojM oHQg= X-Google-Smtp-Source: AGHT+IEuKUU8LmiDGdVSE8HEQD2DSqpmQWqCoCseRpXFHWRMzDS4n8tdQLpPQA0yFxWxU+7dmSvSgA== X-Received: by 2002:a05:6a00:a1d:b0:71e:693c:107c with SMTP id d2e1a72fcca58-724132c15a3mr18685493b3a.11.1731368951711; Mon, 11 Nov 2024 15:49:11 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:10 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 15/16] xfs: flag as supporting FOP_UNCACHED Date: Mon, 11 Nov 2024 16:37:42 -0700 Message-ID: <20241111234842.2024180-16-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Stat-Signature: 6huu5oe76u5fnbcx3p5ige9dybao5ou5 X-Rspam-User: X-Rspamd-Queue-Id: 5AB3E4000A X-Rspamd-Server: rspam02 X-HE-Tag: 1731368911-302506 X-HE-Meta: U2FsdGVkX19VfCmxcRnzVsp/arxqgqrwT6R4Sk99v8PsoU18gZCj6UrJBkqLzL2aXRY54j+ZKYYF9QQAHKPrfYyL2918VVDBDlda+YzF3SqiMnNU51QqucDw3CUjMpzlxmdAPFqghRdI62B4TJh6Bxc+NMap1yt34nTSKOO6RcGaqzGMmrlUHoDKuFZb6jT495X9lWn0XTv9TO8h8s/0s+WBvoW7FoVQagrrahFbxw6y0NNKCRsX9OyNnDswqbvrrNLVuJ/qoH2BpUY6Z60PX6rMSTarPS30BVi046lnotKWkSCShWdoz+vWYpVQzUEPIAPTCNjHM9YbYZiUgXFZthOP5zRBrcx0ku0Lc1GIQyKFDK0R0nIMXdrbsk8YlY4tN+9arBH7rofk8KkkG5ofeLTfGiKgQKVxVkDxl9Qb09POP70dytGfEILpsJRuzjb9R+1elSSAzAkXbsjgCqE+7135Y3JFbsxvk/Sg/OPhu824B8v7cqTsBJsuYCmBz4OrfftgrxIut8lEnmiUgEcRjJkSe996TZM5eED51n0QAnDIVzXiBTzl+egzDNOQ0Lpw1YBykgUt/kSG1a3ojYDe7pzuePhLGJYnPIQLVV6IOQtaWtGCVOngf12l54xPIypOi8c5GincJeMUCBReTGeNBpB0K0yPPAyvhEXFBcEC716QMYyroBJmXNuZ0gAXwFzXeSW8VrUjKMMDzkU+ReZmI34BN7K72UzOq/eCqUjwqgHNSF86PR08qYSVwGjwQWR4MAqNOJJxPg5FrnpMgB6/z+g1QdKUgBTu4xhgUKSXgbU3eL6n26dX0Q48pa0YmAz9XOjPNdbHwosu48guPe7xQmZ9gk0L2WNGnKkjCYgd654oECyKicAafaMntAtVNNp2bbxyRDWX8JwjLEsOQg6jxCmOeF/LJ9+MpJrIIXf3/AhLNdxk+kNs9PpbLoDhifKySOXAFOaas1bVAzlVnmE jaZhBLHu L3y9ib5DiM954aFrKRqIPARPNLkm6SmodN2IjNGeQ3/X4PKQwUZ1jgbACptCvZHpy67vjnoqhtG8oVj5TXt7Vp0EGp4HE6XYuFhlMMGwm2wJrxcuNHNWvAc9HmAHtQ45ovBtxvEWsLPo1G8HBWIRBrVd2oBeQq347JilpjOC9l855Zs/OL3aX7u+yeARUVSawS+WEOZ7RfCoD5tWWrUANQQ6DsOxGg4Bqiy5P8+9sSbbadtCeWmthrb6KuFTshMRDfZmnld+A1+V9ACE5bIyfn2+3X/vsSmXoR+/tI2aBtRBTN7KPNp75QmGy1wt/TNUERJeLimT4kVjh9cT5YtoRO0clAY4nXqlKmiiM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Read side was already fully supported, for the write side all that's needed now is calling generic_uncached_write() when uncached writes have been submitted. With that, enable the use of RWF_UNCACHED with XFS by flagging support with FOP_UNCACHED. Signed-off-by: Jens Axboe --- fs/xfs/xfs_file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index b19916b11fd5..1a7f46e13464 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -825,6 +825,7 @@ xfs_file_buffered_write( if (ret > 0) { XFS_STATS_ADD(ip->i_mount, xs_write_bytes, ret); + generic_uncached_write(iocb, ret); /* Handle various SYNC-type writes */ ret = generic_write_sync(iocb, ret); } @@ -1595,7 +1596,8 @@ const struct file_operations xfs_file_operations = { .fadvise = xfs_file_fadvise, .remap_file_range = xfs_file_remap_range, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE, + FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE | + FOP_UNCACHED, }; const struct file_operations xfs_dir_file_operations = { From patchwork Mon Nov 11 23:37:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13871499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A330D3ABF4 for ; Mon, 11 Nov 2024 23:49:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F7EA6B0093; Mon, 11 Nov 2024 18:49:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9562B6B0096; Mon, 11 Nov 2024 18:49:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CF916B00E8; Mon, 11 Nov 2024 18:49:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 596376B0093 for ; Mon, 11 Nov 2024 18:49:17 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 201701201E8 for ; Mon, 11 Nov 2024 23:49:17 +0000 (UTC) X-FDA: 82775456358.06.E0A1E15 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf28.hostedemail.com (Postfix) with ESMTP id 8A947C0006 for ; Mon, 11 Nov 2024 23:48:33 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=mzNil4tJ; dmarc=none; spf=pass (imf28.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.177 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731368780; a=rsa-sha256; cv=none; b=XpCJpbmifGINpaop4T7d/cvDZqR8uakIpapIXEjasA2tnvSr89wM2mWwBvMezU2dbouGrJ +8OhTIuum4Tvl8DYUvtVI9kaU3ynhHLdY/QLcxSqRRWPk0Tq+g+4m2LyfUJHNj7nK7YRiS PinJ6l9C0jSYgfD/Rc9MOM3KHZB3odk= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=mzNil4tJ; dmarc=none; spf=pass (imf28.hostedemail.com: domain of axboe@kernel.dk designates 209.85.210.177 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731368780; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F8VuOfX6p9OA5iVa4jUFPNHPqu8A2RaR+lAgXUtkcVc=; b=zO2rBKvIp5xkV0jZAsCVrTBDtl80v9JANNLWFylbsqiAhe7DAb8U7pzm+0p5E6Mx0wEt5c /e5m7sT/H6AgufdcOm4mMyNfaHSUTxI7e/dvmLCfZG1FYyy7NW4WBSzW7WYl7wgFV4Crdw e3CrK94WlbFZbZYsFaeYDWjzF0JeNag= Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-7240d93fffdso4130017b3a.2 for ; Mon, 11 Nov 2024 15:49:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368954; x=1731973754; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F8VuOfX6p9OA5iVa4jUFPNHPqu8A2RaR+lAgXUtkcVc=; b=mzNil4tJSBhsYVhagvk16RKrIPRcZfqIc6TP4ORForefCWd8WcLzoQ1rbF/6gOfO+4 Ul/UIiRk493Az19xg96nXAkEGXFrD1rguDtwGCGr9W5uLM3jzZE4JBOevogKf1vN/pNh 0lBm+lwMu5JWToBmIseH0xNPgK+H0coLd393I3qrlpoNFoL8G8InI5x9oAjcz1WokYlY euRLm+2C1whCDiMnUV02kMJjcq/7IIP4LxvPauS1cUaij92c08CVWUcKPE8CY/0AWTFa UzTJKdVOmndopp57el6zjvicK4TixCRmUaaWJ/3VNIy3EHDvWEXHYYJj4vW1R9gqVS+1 bhdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368954; x=1731973754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F8VuOfX6p9OA5iVa4jUFPNHPqu8A2RaR+lAgXUtkcVc=; b=g6TZW8dk0h1vb74RIm20jNJtQLDBlFYftNM8tWsE1SOWLDuaDcd1mPxIDLS5EShpBA JQiUXL/1k4+K74D7H1T/vno1gj0B9kPZDGwkxrxjDxUFhzlbMwphBwKca3iQ24COJ5xW 0EMMuDG7SawtOEdb0XJzO87rurj5PHHbkblwXYYAoh0qma300uWqiViQ2SCvLP4co/6b nV6gErpMaQOXINWl/OaR74FWfC0Fsw3jEX+jsUcj0oDItgvSoBcMHZxtu6s4DIMtJkX/ AeGc+ofphbAlgTJGW6zRwA3BMEHMAu65hFcm7YWshmBLK21ys+sQ4PQ31+d/QFMG74LB 4bog== X-Gm-Message-State: AOJu0YwaiImJ8/iViakbj9eH9hFIRDACh8dGWf0azwhlNBAHq+xbLn/V 46uarybnzrOv0ZLr1+gEUBax+b/ao9D6EssRcBI6ImFEb+PeF5A2oxsxc5IZ/Xa5iNdVFyNKqo9 mYu0= X-Google-Smtp-Source: AGHT+IF2+wW3TLckV7VSNmFRA6TqTX3hVCOrji7S4MicSJHEM7Y626gshLEQCdd+Y+7SrufulOQgng== X-Received: by 2002:a05:6a00:2354:b0:71e:60fc:ad11 with SMTP id d2e1a72fcca58-72413354230mr19707325b3a.16.1731368953536; Mon, 11 Nov 2024 15:49:13 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:12 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 16/16] btrfs: add support for uncached writes Date: Mon, 11 Nov 2024 16:37:43 -0700 Message-ID: <20241111234842.2024180-17-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> MIME-Version: 1.0 X-Stat-Signature: ajcubymspb18aqbie83mfte7ampyuaeq X-Rspamd-Queue-Id: 8A947C0006 X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1731368913-448601 X-HE-Meta: U2FsdGVkX18zw8MOwBwP+TY9u4HTGPqxEPQRG1uBolDlhuPnfyy2HCyzuPsC9vccB4gRgfgkZgOOrBuaxPCzslmZQ/ouzYbwMB2cGl6GdB/KvxGmnp1eKEDr2P7udQ6ZFbcSveC1A9iN/lYndD0OCY7QRKqlJCTIGJqaEMFnYvmsNALs9XrbuHsNA3b8FpR4Ecio6TmPOZn/S+EtWFR7P4/HPgX/FLc7AFDuiEc+7FqLViDR8XHaMyZMNV68euiQ6dYEZCJ2ajfOJ7re5KF/gwAooT4DZZxK2qyjf6zLwCIRuVsIxJ9fmTK/ZoMR0B/uhBgqmNKu3y/+dySutiyZnn+BXvUkvzXZ+86rxnZy3NyHG2VEJAJkUGeHqFG2Zur5/ub+vAOhrhg6q9AsdP+eI3iCLmC85RTVkitSZl+Y/hmJs81LwqXSwXq16rNdA0MLkNdPdHc/atl8EmJ3KTN+ksnidtTQG+th6L2hSGUkGKkErW/TPIaao6Iptn28q1aAnZmvMshu5g5Rh0iWrlSQzOb9Dc7NcRlHJ0w9LUI8VNQUGJVBbA9R8gTiiPrOtvYCiHIVjzvg/n7t6Nu0nnDzVvnh5vKlwPaEHi0yNIQe/EXROUZC5eHShregd5T7y1BBOhNOPZvCJXzAvtbb315tm2wLeK/VPZevNJCdTBSbQl8VP5249sL2TupghaAMD8HTC7+FN7mHeG2VC9k3MprpFqOQm5EeMx/t3LQ307WXpBC1LM4Evit7r+6q26T8dp/gsYnG61M0AnGmARRuubnF26dkTtfFDs7FNE2Rzj21HfWnQ0YeQIQaIPBxUfDLLE6RX7mdQm9nniRVOe8q38DQkWKYcDqy0+dsfqm50K2inQKRb6KPUGJ7nY33ou2f049aRtGUzbHK0mRLg/vS/vD2zPueVHZ/uaCy2JHM9oLXUN/oympAHhAJh+v7eFtH7IdFEL6gx09k7vBFn8MSWjw Iy5Snryj bMH1v7MMmyJelXJJMsMg3wTVyEFcOAwAcedWlxa/YLSLdO+BbJoolwtXX/HrV7SZWSvmPllZotK1t79FUV8SZMDySBePEOmwjvbCt6HBEs2Svz0WETxht/ebDgd7+TJAMaS+d+oZojy2Bd9H+6GkJBaRl0DjWFR5stYcaWeQUZ4tJ0/vNwi3W7TCGf302GY9C3UitrPyYycEILXOVUs9JNnQPtSg1GGl0nvJ8FoYTxgJc/4zIHLukc1OVSSG626mhfUJJ5GI1PPqmanw3WZMhRDHwO0WqJH+d904hD5nH6oGW6YpfLGzpAmbMZg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The read side is already covered as btrfs uses the generic filemap helpers. For writes, just pass in FGP_UNCACHED if uncached IO is being done, then the folios created should be marked appropriately. For IO completion, ensure that writing back folios that are uncached gets punted to one of the btrfs workers, as task context is needed for that. Add an 'uncached_io' member to struct btrfs_bio to manage that. Outside of that, call generic_uncached_write() upon successful completion of a buffered write. With that, add FOP_UNCACHED to the btrfs file_operations fop_flags structure, enabling use of RWF_UNCACHED. Signed-off-by: Jens Axboe --- fs/btrfs/bio.c | 4 ++-- fs/btrfs/bio.h | 2 ++ fs/btrfs/extent_io.c | 8 +++++++- fs/btrfs/file.c | 10 +++++++--- 4 files changed, 18 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/bio.c b/fs/btrfs/bio.c index 7e0f9600b80c..253e1a656934 100644 --- a/fs/btrfs/bio.c +++ b/fs/btrfs/bio.c @@ -334,7 +334,7 @@ static void btrfs_end_bio_work(struct work_struct *work) struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, end_io_work); /* Metadata reads are checked and repaired by the submitter. */ - if (is_data_bbio(bbio)) + if (bio_op(&bbio->bio) == REQ_OP_READ && is_data_bbio(bbio)) btrfs_check_read_bio(bbio, bbio->bio.bi_private); else btrfs_bio_end_io(bbio, bbio->bio.bi_status); @@ -351,7 +351,7 @@ static void btrfs_simple_end_io(struct bio *bio) if (bio->bi_status) btrfs_log_dev_io_error(bio, dev); - if (bio_op(bio) == REQ_OP_READ) { + if (bio_op(bio) == REQ_OP_READ || bbio->uncached_io) { INIT_WORK(&bbio->end_io_work, btrfs_end_bio_work); queue_work(btrfs_end_io_wq(fs_info, bio), &bbio->end_io_work); } else { diff --git a/fs/btrfs/bio.h b/fs/btrfs/bio.h index e2fe16074ad6..39b98326c98f 100644 --- a/fs/btrfs/bio.h +++ b/fs/btrfs/bio.h @@ -82,6 +82,8 @@ struct btrfs_bio { /* Save the first error status of split bio. */ blk_status_t status; + bool uncached_io; + /* * This member must come last, bio_alloc_bioset will allocate enough * bytes for entire btrfs_bio but relies on bio being last. diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 872cca54cc6c..b97b21178ed7 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -760,8 +760,11 @@ static void submit_extent_folio(struct btrfs_bio_ctrl *bio_ctrl, ASSERT(bio_ctrl->end_io_func); if (bio_ctrl->bbio && - !btrfs_bio_is_contig(bio_ctrl, folio, disk_bytenr, pg_offset)) + !btrfs_bio_is_contig(bio_ctrl, folio, disk_bytenr, pg_offset)) { + if (folio_test_uncached(folio)) + bio_ctrl->bbio->uncached_io = true; submit_one_bio(bio_ctrl); + } do { u32 len = size; @@ -779,6 +782,9 @@ static void submit_extent_folio(struct btrfs_bio_ctrl *bio_ctrl, len = bio_ctrl->len_to_oe_boundary; } + if (folio_test_uncached(folio)) + bio_ctrl->bbio->uncached_io = true; + if (!bio_add_folio(&bio_ctrl->bbio->bio, folio, len, pg_offset)) { /* bio full: move on to a new one */ submit_one_bio(bio_ctrl); diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 4fb521d91b06..a27d194a28e0 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -919,7 +919,7 @@ static gfp_t get_prepare_gfp_flags(struct inode *inode, bool nowait) static noinline int prepare_pages(struct inode *inode, struct page **pages, size_t num_pages, loff_t pos, size_t write_bytes, bool force_uptodate, - bool nowait) + bool nowait, bool uncached) { int i; unsigned long index = pos >> PAGE_SHIFT; @@ -928,6 +928,8 @@ static noinline int prepare_pages(struct inode *inode, struct page **pages, int ret = 0; int faili; + if (uncached) + fgp_flags |= FGP_UNCACHED; for (i = 0; i < num_pages; i++) { again: pages[i] = pagecache_get_page(inode->i_mapping, index + i, @@ -1323,7 +1325,8 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) * contents of pages from loop to loop */ ret = prepare_pages(inode, pages, num_pages, - pos, write_bytes, force_page_uptodate, false); + pos, write_bytes, force_page_uptodate, + false, iocb->ki_flags & IOCB_UNCACHED); if (ret) { btrfs_delalloc_release_extents(BTRFS_I(inode), reserve_bytes); @@ -1512,6 +1515,7 @@ ssize_t btrfs_do_write_iter(struct kiocb *iocb, struct iov_iter *from, btrfs_set_inode_last_sub_trans(inode); if (num_sync > 0) { + generic_uncached_write(iocb, num_sync); num_sync = generic_write_sync(iocb, num_sync); if (num_sync < 0) num_written = num_sync; @@ -3802,7 +3806,7 @@ const struct file_operations btrfs_file_operations = { .compat_ioctl = btrfs_compat_ioctl, #endif .remap_file_range = btrfs_remap_file_range, - .fop_flags = FOP_BUFFER_RASYNC | FOP_BUFFER_WASYNC, + .fop_flags = FOP_BUFFER_RASYNC | FOP_BUFFER_WASYNC | FOP_UNCACHED, }; int btrfs_fdatawrite_range(struct btrfs_inode *inode, loff_t start, loff_t end)