From patchwork Fri Dec 20 15:47:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916919 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 442FC2C190 for ; Fri, 20 Dec 2024 15:48:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709720; cv=none; b=sjdpwYFuxrfyi7arkWl000SfQbRwK05Pb+CnvlqG1IdYbI5WOLKFxJFNdcbo9e2YZMwiiYQj8bossNgS9JVwz1ETykOKjOelzIOM4O4bXcVKYaxKkA99028ljzFTx2kGgt/9zQdkx5xBR5qHW6ign1bO6BoTOHGBoa2QX2R2JOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709720; c=relaxed/simple; bh=b/y5rN0twqq/ec0xNXkKmKx5Y1U8KwWxvQTbJi27mNM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ncZpZLrlqKu4PkQPoIYvOPHQtjh8mnl4yAPwMe3wbB0KS4Cta89Aaba5OL/5b88tSRA4R/CBXuO1Ua+lSMy2eNiMBxTRKtsRHcl3XOWV22A+dKLbNVNNW9yFoK/MnZ4YjC4OVRu2Ex7yvSbi+vyvd9MJLKc/DICFWpQc7BDq5oA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=jkL83Moh; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="jkL83Moh" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-3a7d7db4d89so6533095ab.1 for ; Fri, 20 Dec 2024 07:48:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709716; x=1735314516; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1VCPzs6l+RtFDMTSDAiMfSmCwqht+PTO7Juc8Thzg3k=; b=jkL83MohDYhQY4bjlZQSU/5NzrH55KTXCTdEjCvqC5zm0NEGSuy+axb4CHoLYS0nw3 JXqDzqCPAezo31DUrtnls2EAQvQfLgiq30e+/5VUFDmd5imZQrpDb0z+tCq6llMaYR65 +/0SyHyRerEpNYWGu/H8L2gyTbX6eFxapOUk4VsJ7yNuxKj6qm293RAD+e+rYul672Xw HkgT1ta3CnX/hqtIrm0FAPNmRuts9AEg3nS5RzgGiJOpfcROP/zcauIbQAdRg7wu9pzs cGGpS2BBJ9DX8SehPlElyHtvi92P7y/lwDLGrLjoeyt3Gx5UkJXjCeGo7gNLEC9hcfJp RTdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709716; x=1735314516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1VCPzs6l+RtFDMTSDAiMfSmCwqht+PTO7Juc8Thzg3k=; b=AmHgUNr0UE0xFWs9eO64NR0+ZccQvfRwYHimUh8Oe8OnlPYDpsX+fryzXGLWjINQ2r k8FXUg3JnTHuPdhDIyFfs5EbMwbmy5Q7h6Uifoofi1xCgas91ctWKGLHWoHxs6zB0D8L btFYjDOjzgI7hSHHBN0urFH7Ub4Y/s87eogsrD0J8mb/VA7EcBgdKfqbvz/1mEQyYM7F fGAbSH/c56xBQ7AxGjZRODlIblMeD3mQZTuzSh4Xsg5xSFV0dVq3jHcNIbxyOM3DYg3W vou05LaB1LOZ93E4q/TkCQqonxHsfiME4D7Ozp5SMnlTWwq7aE6xqv+Sku+FMuctrxg2 oHrA== X-Forwarded-Encrypted: i=1; AJvYcCV2jxyQGZmgCDd0CyrmfkYji3qk9ehMu45R1X0uBEoebI6PljrDCdpXEZVNQsg7AuXtbKhoqqYBYdK4f40U@vger.kernel.org X-Gm-Message-State: AOJu0YymY5hGzJyRg4grQ9JsG88CS0D7CwOBWTSVYqeCy1JXg597/jfQ fvsFOJj5eiwzOI2JipoH2hfXWsuX/Htfmodcx4Z6lXiKdkS2ZU5KzxZdsuCru3vJkWzJWS23zLu s X-Gm-Gg: ASbGncsJlCccjnt02smKh6PSLE5fuqahim0gqFs6mVhGo1HVCI7w4xLCygu0o3v1e1V 5xgPog00NuAO4kGdazcYBJpZrkxJejcSS0cyAcovmyaHLulqcxp7O7+bG5kMA6+y+ddmkqbXhF0 O0uEOPYYA2JC88nhyToJkwxIBBcRB3/+nC7Zed0BhIVrix5LvBkS5y+2WSjvyasN4ytmAb+z8TX DSAqhQERtEYun67wJKXp9u+/fmNgrDi6RtMBxMVZvpANcTztxnILR3eAy7c X-Google-Smtp-Source: AGHT+IHcbw9zIKJRxdjYyzhRRbFlhvov/fILoOcF1iNDb9M6UqXSGlIvkLvTLfoaxOcUHsgp350Srg== X-Received: by 2002:a05:6e02:1e08:b0:3a7:8720:9deb with SMTP id e9e14a558f8ab-3c2d2d50993mr23190015ab.11.1734709716455; Fri, 20 Dec 2024 07:48:36 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:35 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 01/12] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Fri, 20 Dec 2024 08:47:39 -0700 Message-ID: <20241220154831.1086649-2-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. With the kiocb being passed in, skip passing in the address_space separately as well. While doing so, move the ki_flags checking into filemap_create_folio() as well. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe Reviewed-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index f61cf51c2238..8b29323b15d7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2459,15 +2459,17 @@ static int filemap_update_page(struct kiocb *iocb, return error; } -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, struct folio_batch *fbatch) { + struct address_space *mapping = iocb->ki_filp->f_mapping; struct folio *folio; int error; unsigned int min_order = mapping_min_folio_order(mapping); pgoff_t index; + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; @@ -2486,7 +2488,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index = (pos >> (PAGE_SHIFT + min_order)) << min_order; + index = (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error = filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error == -EEXIST) @@ -2494,7 +2496,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; - error = filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error = filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; @@ -2550,9 +2553,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); } if (!folio_batch_count(fbatch)) { - if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) - return -EAGAIN; - err = filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err = filemap_create_folio(iocb, fbatch); if (err == AOP_TRUNCATED_PAGE) goto retry; return err; From patchwork Fri Dec 20 15:47:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916920 Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB990218EAB for ; Fri, 20 Dec 2024 15:48:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709720; cv=none; b=b77iE2+92ooUqsC5Kbfx8dpT2xiOEk+KR3hJHXjTlCJGCLRy9xHxPmBvLP3P7sTqocKTqWLoAuJmyNd2fqFuiPbMU4ZzT/QcFyXE+ZrrAYnSBSuA2txBAWZXxa1lhgUNWpP4D8GL1m5jI4Wiy5lPDAgm3rH1ZOcqkuy06YbsGqA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709720; c=relaxed/simple; bh=nP2D5ty4r2yjDcWsJ4kVfAz8c3OFcpVU3g/7tFCHj+w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Kc89i5PoV6T/ZPq+/xCFC3OPy9yERvfaRz2mfHp5Jk3Hx+MbvWCepDBlisYoojcav59lk3xcWD/x+iVpipuhcc4Zgyopr3q4fOPNiLWHiv3jL9D3VRS900JUvIf7cL2ydBODthxbVjdzeDf5Ed4j7DhPc4NB92MbqP34Pxv6e6s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=u0xMnDTF; arc=none smtp.client-ip=209.85.166.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="u0xMnDTF" Received: by mail-io1-f46.google.com with SMTP id ca18e2360f4ac-844ee166150so68485039f.2 for ; Fri, 20 Dec 2024 07:48:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709718; x=1735314518; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ymwG1n6nWD/uJHK94RoKI8EVCXuU9v7PVyju+T4v7w0=; b=u0xMnDTFqisiHf/UFpkVAqfaKJnXuM0CefNU15ODr/AWgfu4NslH7Njz41dtEJPEnz kZTbY6s6kfjRRBMWiN0eiYqEXuijdGCIQgrAQo7lyt/FAQC/HCwTzvaZ2NDBrTNY++vt i9AJha4TZTp769kHtXOEBdv/IIJMh7BrSeYAZpdvLLaoBgHsi1MPVa7H4Ji8GsGIevAC xlj5NOonwj0K4pAiALWP4iw8dkD//dGWZ904OdmJ84HXYPRgMFlUqZg+SgTvev21Yuhm 7DQu5IIRGXRCw9eTRuN2Iim7CaynNJ+ImZlrHkkD2BbgfwBpX9cuUy3X90vaOaQn3IZN wxyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709718; x=1735314518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ymwG1n6nWD/uJHK94RoKI8EVCXuU9v7PVyju+T4v7w0=; b=CT34TX0IaFezpNtt/o43eb/guLc8jVFS4fUQIr1SlbSUDLCmwciLL8Iu3A6AbDO7FY X1GSBeo3qo7RP6qvDrU5Hyj4MkjngWZ4WQpT9sYAilDO1swq9ZQbdLhEWPkkb1XW40Ok TMYS6nIQ2Cxi9caAWaAe6IspVAk/JTnAlvMZiCYlhmMOk/tpoWPxI5/WtJK6LS3fkf68 s+mXHXqn1yGeCjvSM0JeTswaSEoOSO11B5/ffvwdsYi9cam5AcPQlYj2X8nxDK3497TV J2KqisfShhtRUhiE2ar0oJaw8CNepVzCCTMmB4CZqS/axkTCdVB/OKW/1+aXysb3U8ae x35Q== X-Forwarded-Encrypted: i=1; AJvYcCWRsmEwQ2zYyxEnLaloQiZN585kRd3ou77HB1ytEKvWX82tGxueX6oddWHPixq/T6b2eapm3+h8kf/aRX7D@vger.kernel.org X-Gm-Message-State: AOJu0Yz7W70QQ4f0NXm4Kih4OSUTewbERcAX2Lq1bokMwUN+t9v5b2E8 425tun14qcT0g9gwnJGcAsjYg/onLNeA2S6GyEjm31xhlz3t3Bp/z1LU86JfZjU= X-Gm-Gg: ASbGnctt6srt0rJK+HJVN4z2HTH+TpgQJGn7XTxE4g6mJfQNskOLVPkV0bt0XYA7QC0 S2U/0nBjcCp2KDC511jQwTjJ6zxXJs2mLDadCfGJSc8dt8MgthKXvT+WuoI3Ppb4wminXQRXpKk A1rfzUvJYHzZ/h9yFbOO9Zcq1CwWxU17OR1vItVO/e5iECdCBS4l7DV3jv2/qqP/TvuNzKsGVvM 54hTZh+hZoWDDUlwzczG7M7oq71fgi0Eo8jWMto5ScnyGNmYH6je3NzJUEQ X-Google-Smtp-Source: AGHT+IG3fN9OsN4GGTbFHoS9tD5hs657TGN6y3Qqvn5pakOnrgdzlOR3L4/c6Fhv1UMBXZPo8R74Hg== X-Received: by 2002:a05:6e02:156d:b0:3ab:1b7a:593e with SMTP id e9e14a558f8ab-3c2d514f9ebmr38935955ab.19.1734709717719; Fri, 20 Dec 2024 07:48:37 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:36 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" , Christoph Hellwig Subject: [PATCH 02/12] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Fri, 20 Dec 2024 08:47:40 -0700 Message-ID: <20241220154831.1086649-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Reviewed-by: Kirill A. Shutemov Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe Reviewed-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 8b29323b15d7..220dc7c6e12f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2527,7 +2527,6 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; - struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2542,12 +2541,13 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); From patchwork Fri Dec 20 15:47:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916921 Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0ED5D219A77 for ; Fri, 20 Dec 2024 15:48:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709722; cv=none; b=B2zSuThE9QnnYRcMTCJgq9aOYqA5ftsAz9R7MrMVBP7/xhgJA8vZNJc7YWBZ0oCpaSae15EehxMx9vymT9Jj0lFEU4kuwsvAfOX3Ysz5n5coHCrhDsniCuNCd2tj+37t+26T867F7nyqJ6EQjClJsu83uXrLy7bBtyD3HgKKa0Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709722; c=relaxed/simple; bh=gjkhKuIh/F81dbqkaKljfYc7jS3icLdyh4zFI+rhjmA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ASudJ0V5XA/MYqCU2KPvamDWJ2FZOEVYv6HYLXmMdAkcfdp+HZtHpph6YZoh/dd99oFMAU+L+gC/XDYOhh0iYIyavhXV4Bim45l8PDGSXPG8DHOme7FhJYmLe9q9q+vfr++Dvdgf37ErKc3rxnbRF3bz5kMB1pPK3VEuZ8P5OFY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=nQV3wjDc; arc=none smtp.client-ip=209.85.166.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="nQV3wjDc" Received: by mail-il1-f181.google.com with SMTP id e9e14a558f8ab-3a9cee9d741so13093275ab.3 for ; Fri, 20 Dec 2024 07:48:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709719; x=1735314519; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U1i1r3ArZkX2vgdPDoEq+WK9gDY1/MkPULwwiAP9Trg=; b=nQV3wjDcNTgJ4GchyVBCil7RWw3kEBcqovSCDdq407h+Fp4OUxJAHtkRzl7caMbOCW QUQRa9it6AvF+DPyJKC/Ng6Z4LxF7LYXb+fH1HP/8nLspS7vc3ZXjwC5ajUz6lVsIiCb waUtyWQBb+MkdhbP45bVZvLRmdt7yUnioSorXOs7kFcjTfDlw/5Z3RAkJRxGSPf0dXRx 4nL3ElUqrA7V22XHs9Ce1OtmnqPRTrETi53u+6od7ScPx3Yq+9UlkO+rMKBMzBN5N/ee GShvNHmHpXUyQfioJXe2VWYi+wLENkEgj/k3X2561CHf2sI0cTpR8qcX9UIxsHNGWxG9 JEYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709719; x=1735314519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U1i1r3ArZkX2vgdPDoEq+WK9gDY1/MkPULwwiAP9Trg=; b=MzGirhVMc8vJj+9ALlNRkWNeYfiolnV1tB5g521i6hp5thSJUBinRLSOjMDy+OTRCN HbJD9hvHRpNU5ZBBDAIGJbrUFAHYOs6ioApuoqPk1j55cuYMqNrqcsMCwWnVUDOg+W3J /UnxnJaoyRq1JxF1uzzO97d9YCc8GAAvY7GZHP+4msQmgM5Yb0TtloT3Vmx54KufIKQi JxzfD9JAzdHvvzc0qLStZZ4/g8FoTDRT6yGZthFEzFX4AKJWzn1MnB3n/E9bjuViKWZ4 CTDiAaD6e6NBG0FC0x0BLKB0a3q1m88VarEfDjTvGCbSQ5QNvW1WCKTe1nj+xJn0oh56 H6yg== X-Forwarded-Encrypted: i=1; AJvYcCUNZff0HTDz5908KeYzIIJM0LNJqgihB3Yf3FQgc2d9keTfwfiNNBFqLA4vsS/jpxKlValcjanlRPp30N88@vger.kernel.org X-Gm-Message-State: AOJu0Yx3z5Gws/MChkU9TJy2DGASs2DUAtAanMoKODqGMN0tY84Z5kns IPjB/a/D8fWnKKJPa8usNuLrZW7xHVCT/RfWjXbBJYQE8+cusBRmdk6mVwhdt/o= X-Gm-Gg: ASbGncuIaO7e5KfLL5jCJ5IoOUbSPry4k8DjQaT8CUhFdrKPVUatpo3U0ZbopxfSTTD Z+IPPzjgZH2VFEbMH7gH/ysyzRJ4P5VOxUYeVSIfzXkwGXRvGZsvMc3QQuIINqrP0TiFQIWEKLs T6vyvhlMWl+4IREtZT2nrglS0r996yN0AxQwSnA9DeROyFcpsWj5jPMOC9lOiN/no8hVBQAwQvG WGQfY+mITJQOJwcFviRVpw1Mjws8UcEUDuyueHYZHITYj3v1KzA/+2+L2GL X-Google-Smtp-Source: AGHT+IEQ+rQahAFsMNf0wBgiaHc3ybkwynnm3ue1l0Cc7XmizswXF3l+0I9aVhCYoLyroOrTRrUFJg== X-Received: by 2002:a05:6e02:1ca5:b0:3a7:88f2:cfa9 with SMTP id e9e14a558f8ab-3c2d2d50b1emr32075375ab.11.1734709719216; Fri, 20 Dec 2024 07:48:39 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:38 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 03/12] mm/readahead: add folio allocation helper Date: Fri, 20 Dec 2024 08:47:41 -0700 Message-ID: <20241220154831.1086649-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe Reviewed-by: Matthew Wilcox (Oracle) --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index ea650b8b02fb..8a62ad4106ff 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -265,8 +271,8 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, continue; } - folio = filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio = ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; @@ -436,7 +442,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio = filemap_alloc_folio(gfp, order); + struct folio *folio = ractl_alloc_folio(ractl, gfp, order); if (!folio) return -ENOMEM; @@ -750,7 +756,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; @@ -779,7 +785,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; From patchwork Fri Dec 20 15:47:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916922 Received: from mail-io1-f42.google.com (mail-io1-f42.google.com [209.85.166.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5ECB021A423 for ; Fri, 20 Dec 2024 15:48:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709722; cv=none; b=UMiTGtsXQdNqJ5Z67GUf5qKTWc7n/06c5FfaHIoZHP2LUN20lC+dfrUjdYIhYovxsLKbAJ8M8kn3EWMpGUdg+xE4/3YHD+f36Cqb8o2tGynXMNmdJ6Yhv13t2i5935vkxcwAcj06Ghw6uJMkQ9atqcf6YnnHFbQDoPRi1iGi2lo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709722; c=relaxed/simple; bh=JO6tsxSlP852WA3y5S797XWTlY9uJCLdRkQsjOqf/SY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IsbigaSKBTXajXOGcDnYLS1MkgZapn/Arok/om6IrEnX1zuKcaQ1WHsqX1V6KNUGfP3ar9zyeidTr+OEoEUX+PObKanVRXWM8WgBXep+u0XKSgSj+44iCxeV8jgGcC+0fTaQeA9MhtY4LwZ6MA95l5ysRVh1CLE5vQ3Ae+OfaNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=Oc8efsLy; arc=none smtp.client-ip=209.85.166.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="Oc8efsLy" Received: by mail-io1-f42.google.com with SMTP id ca18e2360f4ac-844e1020253so72383839f.3 for ; Fri, 20 Dec 2024 07:48:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709720; x=1735314520; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TZWY6vJPfqFGr1DBynm1KpCMDB4zdovTdnWkeolaIOo=; b=Oc8efsLyd512POrstMUgXa+FvteRc1oMKe9Ma51mEWEnjkPmq8lmhCK02N/JPMCUs5 fFoKBE9OyTb6NPJr46dyQ3rprc68njEKgQqczl1/azIDQpmO1tLR8mE0N5aaDPkJ6jtk Dz6fi0Z+6HAOco6MZtXj8mmPgNOzU8dGSTXFKqhdIX1DCxfprvoES1r6czX0z4OuH1Gw rqbPn8gvjVcq55EFFL/GpuZ1wMUuq7Ma1qkoCi2gh+IqV+vLh8PDYSivfeP25WQnE/Po SYelfNI1xK8fcErXUkdstKmaALvH/us9Lt9J1WETzro/u/inGdlUmTWf8pU1PvH2nGl/ EMCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709720; x=1735314520; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TZWY6vJPfqFGr1DBynm1KpCMDB4zdovTdnWkeolaIOo=; b=weWOVpf4mNuPrev3jnS5TiFYQbMXFLC/gCKiFuDQibKGDiqNDRcYSKIRFY/ktfqmEt GnM64+JfFhrljYeboMNzLORXjV8cYozDreXvf1dZADSBO+ytqLgQMLcBuUvNLC/MZMRK gc/tOvLKsa3u1vcN2J0M66m195YWj+MOsZMygPwP+IhpnkJUa6L3jnyPuCqCuAscdezo tmCRWZ5A9rZIops47m5Lk3hvvkB49vVLALNbFhS7e36CpdBucAC0eTzXtWcKgpckLLYm AGTU3hxgzMD8VKvMA1HiTwBvJ6vIPJ73YwfQWFr3CaZ8ABqw0mY2xZaW4qhawvc1Yuc9 Ym+Q== X-Forwarded-Encrypted: i=1; AJvYcCWSJEOg6xwhRa1FwZkNwWTXypn94Crgix+LLoCwhncTQdEmjp+7IqBxYg7xMM9hhloYJnHj4tXTt0Ubaq9C@vger.kernel.org X-Gm-Message-State: AOJu0YwPLooWDrGbKQKoi4svk1XTkKC2Y527V3JtrTs4Jm2JEXxBDCO9 kqDYNLPGyfy5NVTkOLFjGbBnXMfRvgBEA0blGXjXBuR2Si+XKHLYEwomKhIjSVo= X-Gm-Gg: ASbGnctvQ7AqQuxI28n/U25O1VpdYWl4cu9Z90Ve9yhZqHwQFIvP2Dnk/sLJjwJclat CjbJpeNhHzMHChYIS1uSI4OGRO6thYf/0G9Bca1DMaUygGtvlyTzRdaEwOQ3lcHxwwQt0ms0T0f zP9HIeOYqFONhHPoVwz269/h/949edUOxsRY3HIJSouqRkDtoRG/MJZETSMuzYVUq3Wc57EAWYm VHLJZNeus84vl2Fe2VKjhIddmWANFkKbt0jUxUEmigkApMnHd3l4tSCh9CJ X-Google-Smtp-Source: AGHT+IFvp8gxmeuG0px26ehs16GxVvbXtCVpdcyc+9Br4GzsJzwQZ198tT+fp2EHMgvEVKANcydJew== X-Received: by 2002:a92:cda6:0:b0:3a7:a738:d9c8 with SMTP id e9e14a558f8ab-3c2d14d1839mr28931745ab.2.1734709720617; Fri, 20 Dec 2024 07:48:40 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:39 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 04/12] mm: add PG_dropbehind folio flag Date: Fri, 20 Dec 2024 08:47:42 -0700 Message-ID: <20241220154831.1086649-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a folio flag that file IO can use to indicate that the cached IO being done should be dropped from the page cache upon completion. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cf46ac720802..16607f02abd0 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_dropbehind, /* drop pages on IO completion */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) +FOLIO_FLAG(dropbehind, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(dropbehind, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(dropbehind, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is not diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..3bc8656c8359 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(dropbehind) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ From patchwork Fri Dec 20 15:47:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916923 Received: from mail-io1-f48.google.com (mail-io1-f48.google.com [209.85.166.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A15D321A45C for ; Fri, 20 Dec 2024 15:48:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709724; cv=none; b=FKObvWwDGzga+bMF36Ui1OM0W/rHOaXOkcSKxCI3wRjqJX5TW+OO6d8CeOwPAYd70AtOUcQ4uSSsYv8FVlAvB7EK5VUg1jpezY8/11827UKOKdd/RPYyzd1IZU1v0mdwW8/uDdwENqX8vMvX0y6nKLEY2GwMYOp7gv7OxApTUAw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709724; c=relaxed/simple; bh=cl+oP5X8N7YKHDElO28Q8MeUv/ZKeS8cyPvyEaOZdN4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U7Kj4/nxP/YmV+tFEOpRO3SXOzKB0oa4kJCNqMcpTYSuFuR8amrSs+mWnTUs8m+dXGxdECFzupdxCkVDpLKLWFT22XBBh5rNG5AApBvDN5Ixg9zbNQQa+9+6jUH893Yee4wzEHu8NX1FbkFVgUkrjejrxxGWESep/AO+gTEItxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=nqqzrpEc; arc=none smtp.client-ip=209.85.166.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="nqqzrpEc" Received: by mail-io1-f48.google.com with SMTP id ca18e2360f4ac-844c9993c56so168921539f.2 for ; Fri, 20 Dec 2024 07:48:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709722; x=1735314522; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qy66q6foyU2ZBCe4X6zLGxi2eTMkd0mxpiy36jLMGYw=; b=nqqzrpEccq1eo7LKwO1s4rU8vfiuRVkl3K3iumPJlThFajAbySOyap4/hIP9+X2wjV Bf8qUD3GSHB6JuvtBTOGC/sDN6Co3SXndULxWUdDGKZfa0STbdMXIe/5G4auM4+yGkbx 5In5SD7acOXTeVOTrWdkv/v450oWVB4lkWlyCqfcGwC3KZmTU7dQTu8PmsEK7YIPtxi6 nL0JVMzTa6kaaZ2g60OYPPlRSb3TAsOyzqd2e3MKHHDMoAj1La3uzdf+o1KA9CIdP0HV fHozHWEe5rNw+z5oTAnZaE42w6tnXZUbUK3ST6YX51RUT9/2gMMD2to0DdBgNZSe2me+ xHDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709722; x=1735314522; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qy66q6foyU2ZBCe4X6zLGxi2eTMkd0mxpiy36jLMGYw=; b=MiJ57SlwP0gn5URGa4rxQwNT4R08NkxxYG5MWfBpke9eZ60KiJmVjt9CH0eWrvLvV2 rH1vsfNnOC54I6F23EchMjrI7VysgryIGLN3UnPvmskMI+5x6oX5eohqOJwy3xjBiOwZ 1znjFkmS18qOECQkQzGpAoropcjsrcNhOHFfcmRlJ1e4tsexMTfX96nBILdQHtjLgBo1 d3MzhDqKM9JyFanu+TJ0jvs6ZbcqmdbnfmmJgUmp+622GSvLeNrXpgl2Vh1l4bO0/0gV FjPi0TxNBHfqy9/JTxKRbRIE7XBRfxncs2SLUrDcqHaqu5vlfVVXaqurdNBCrDtYzxmJ Wzbw== X-Forwarded-Encrypted: i=1; AJvYcCV+SYpo1Y0j6wPvPrCk+yGh5f5hvGXo9fJ5bEkDLMYcpDxoQlsK9dNWtm80Dtfg5TdCJ/GDO8VhY5CTsP4q@vger.kernel.org X-Gm-Message-State: AOJu0YxpWd7RBWcq6D3i9ZvlsZwwgsm7TsW/Qtwd0IGLOG/GD/aNFevQ RfajCJIat/F/0dMb2uCViiyzainzHL7LSMU87mSobGKNr7f/rsnLXWSJPioAL6E= X-Gm-Gg: ASbGncvUagzHp56/hzFLeCtMytO7xBO144TbFw4wvkBX3DKIWTgu5TqU5HoaGxlAdEN iLIPf5GMuszUvpZUj7TOOAkAwa1dcHz3C0reW6lvV3mKlhCW6y9Fa2YjtrZtIaIgJyILNV0DFvH ycN73aXMhFo6fjiG9vrqdziFC/Yf0+ZUG3k947SyYThByVFNZ99UNxabjb1jrUdMN+L86GiOJO9 FlT7FbawveFCAeHv6PLbg2lG/hY9NEuivpYNmRMO9dBysmXh6usmikHkdh/ X-Google-Smtp-Source: AGHT+IF2sIPl1WVDVbn7fgvbpII69l3j4cwL7QghFjhSCJ3SFTcAXuckA2Q7OzRuPKU0QQKKTwe1pQ== X-Received: by 2002:a05:6602:26c6:b0:841:81ef:70b9 with SMTP id ca18e2360f4ac-8499e605a2bmr323110339f.9.1734709721832; Fri, 20 Dec 2024 07:48:41 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:41 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , "Kirill A . Shutemov" Subject: [PATCH 05/12] mm/readahead: add readahead_control->dropbehind member Date: Fri, 20 Dec 2024 08:47:43 -0700 Message-ID: <20241220154831.1086649-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If ractl->dropbehind is set to true, then folios created are marked as dropbehind as well. Reviewed-by: Kirill A. Shutemov Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index bcf0865a38ae..5da4b6d42fae 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1353,6 +1353,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool dropbehind; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 8a62ad4106ff..c0a6dc5d5686 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio = filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->dropbehind) + __folio_set_dropbehind(folio); + + return folio; } /** From patchwork Fri Dec 20 15:47:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916924 Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F19EA21A423 for ; Fri, 20 Dec 2024 15:48:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709725; cv=none; b=g2hLoXBzuDJinjgVsdCP+XWfwtGlPbs8JvDxCeG0j/S63WLOBCokSQRcYf9M01NzoI1nfEhlyGxX+PnJasHkYe4437Q4gqRVhq3D6ueMLAJ7CZiYCLUPtYuDX+sU4G9o9Z5kzJgwiJhTrx6NYHgkjQkZr+5Q5G1PuEdg0FoF4rc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709725; c=relaxed/simple; bh=hlD654noMspdeqksOMGzv5hrDtGKzL2oiFa3eBWzT6w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hJXLSj5fxCtSgXUYrQRngtaEv7ATTrYDWrTp4E9SsUpui0PqtF/wxV9nXHEdxzetYMMe3oqRuGUHh2Pjyc7z0qKVsHsv5+3UBWHEt8br5ZNtPnl214l6GaS9IaosCgnzLKfM9bXTtMUDqLcS5czzmnGoMpG78kYYIIo4SN/1asc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=GFIoSUsA; arc=none smtp.client-ip=209.85.166.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="GFIoSUsA" Received: by mail-io1-f54.google.com with SMTP id ca18e2360f4ac-844ef6275c5so70171839f.0 for ; Fri, 20 Dec 2024 07:48:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709723; x=1735314523; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hj114LsQAeZlZx1JIhp9yNhhSBSN5VfiZ1DE7mn9M1E=; b=GFIoSUsAw7RekpRQilDffDad1KbwSZM1hR0LU4Qn3qd5kH4bD6IlPJu6ZYVTgRnnJO OD36Oyg0QQoAICyGD8yqeDiGamkRD5sejSyB3N7ghYOf6U3+xdFTWsQsaaLSbvLy81ZP voY5m/+wX5ztbehFd1nKvmfI1KZmWRRldniEXZ7VMqJNi/naRHFQ/atyibjfZ28I0dNC 6fr0u4rVA/B1zBrnszuq4a1EIo+FyC/L/tS/p7QGhi/1IUPqlaAhdBQKZG55disGFjS7 Pxsia5sj1pcqLkPWF39CBq1ZGip4hwod6++xWDn/ca8awvF9pIIW/AfTgtnGpVdu65Cz SOkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709723; x=1735314523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hj114LsQAeZlZx1JIhp9yNhhSBSN5VfiZ1DE7mn9M1E=; b=PEsOFHY9r+hljGR1vj1zcqFXfVfK20tyvHBCOmmIMD1+TDWvepL9Xj+dcRY9btTojD LbRQA04Gv3VYolNePqavrVczPDH+cKwkDbz8qik5Lv0DjSQG88CjXNa1KAcy0F5drAFP MOCxQjtRmRXyTrFquWyw0sJrKfo4K+QDGgDXqKW9ezpTU0vfQueTfk/2SqrraUH91dIG Qz7sQA3fSyMqHagXj1jO3Dg2YZcIUEscHmbNVNGsKLuID3j8ckR158GdhqSq2Vw8Nr+9 ES8vmCPGuQvzesUNczq/jJEV5hZe31pkLAsAr4VJ4AJVXdpFaxhOwCd5w42YsxuDG987 LOig== X-Forwarded-Encrypted: i=1; AJvYcCWTmpWUiqKo6AnDZFUfvTgoTFaSc7L+nBrq32qck3xsvntlqqUONiin6M1GHYl17dpH5eMwHwKnAsRAq9SY@vger.kernel.org X-Gm-Message-State: AOJu0YzcoVE6wH/39rCIT+xxIHQ/MArZOwMNEbszqe8rIYcFOeHElJFU 5JvP+ZUknBwYLS3qcxkelPZSLKemxRZxHkGZqWOWfU/yhu4l1Ge+rn7cGLn0Ir8= X-Gm-Gg: ASbGncuHGwRbnjNOnG0s11ivBJ0NKemt0HnA3HLk7be1gPcm/OSH0Uf1oDjylORLS0y oTP0DTgFKimbO3+o1nSFCVN/lhYqthsG+DHSNiH2p7zIIKboSiTvye2hiyi0Bz2WNydkYUYnze+ ciU60FvsH8aA0XgMjfLYYRnlT5LbMe3LyJnNaUv9HQj8u8aciNYhLKm4eoCVVPDKqa2y3I5UQU2 FIT/eyiq3I01tuIPChAgtalrs7ONHhceYJMrgbYIlVWNww2izfcXkfAtB3F X-Google-Smtp-Source: AGHT+IECOzR43hLv/D5wLt0YmDzKJdUpeT0f1YGhsBnnrmOZn/oym4rW0bRMY26iQGlNbtfBjUSBgg== X-Received: by 2002:a05:6e02:3683:b0:3a7:8720:9de5 with SMTP id e9e14a558f8ab-3c2d1aa27e8mr34276355ab.1.1734709723099; Fri, 20 Dec 2024 07:48:43 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:42 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 06/12] mm/truncate: add folio_unmap_invalidate() helper Date: Fri, 20 Dec 2024 08:47:44 -0700 Message-ID: <20241220154831.1086649-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a folio_unmap_invalidate() helper, which unmaps and invalidates a given folio. The caller must already have locked the folio. Embed the old invalidate_complete_folio2() helper in there as well, as nobody else calls it. Use this new helper in invalidate_inode_pages2_range(), rather than duplicate the code there. In preparation for using this elsewhere as well, have it take a gfp_t mask rather than assume GFP_KERNEL is the right choice. This bubbles back to invalidate_complete_folio2() as well. Signed-off-by: Jens Axboe --- mm/internal.h | 2 ++ mm/truncate.c | 53 +++++++++++++++++++++++++++------------------------ 2 files changed, 30 insertions(+), 25 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index cb8d8e8e3ffa..ed3c3690eb03 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -392,6 +392,8 @@ void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long addr, unsigned long end, struct zap_details *details); +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp); void page_cache_ra_order(struct readahead_control *, struct file_ra_state *, unsigned int order); diff --git a/mm/truncate.c b/mm/truncate.c index 7c304d2f0052..e2e115adfbc5 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -525,6 +525,15 @@ unsigned long invalidate_mapping_pages(struct address_space *mapping, } EXPORT_SYMBOL(invalidate_mapping_pages); +static int folio_launder(struct address_space *mapping, struct folio *folio) +{ + if (!folio_test_dirty(folio)) + return 0; + if (folio->mapping != mapping || mapping->a_ops->launder_folio == NULL) + return 0; + return mapping->a_ops->launder_folio(folio); +} + /* * This is like mapping_evict_folio(), except it ignores the folio's * refcount. We do this because invalidate_inode_pages2() needs stronger @@ -532,14 +541,26 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * shrink_folio_list() has a temp ref on them, or because they're transiently * sitting in the folio_add_lru() caches. */ -static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp) { - if (folio->mapping != mapping) - return 0; + int ret; + + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (folio_test_dirty(folio)) return 0; + if (folio_mapped(folio)) + unmap_mapping_folio(folio); + BUG_ON(folio_mapped(folio)); + + ret = folio_launder(mapping, folio); + if (ret) + return ret; + if (folio->mapping != mapping) + return -EBUSY; + if (!filemap_release_folio(folio, gfp)) + return -EBUSY; spin_lock(&mapping->host->i_lock); xa_lock_irq(&mapping->i_pages); @@ -558,16 +579,7 @@ static int invalidate_complete_folio2(struct address_space *mapping, failed: xa_unlock_irq(&mapping->i_pages); spin_unlock(&mapping->host->i_lock); - return 0; -} - -static int folio_launder(struct address_space *mapping, struct folio *folio) -{ - if (!folio_test_dirty(folio)) - return 0; - if (folio->mapping != mapping || mapping->a_ops->launder_folio == NULL) - return 0; - return mapping->a_ops->launder_folio(folio); + return -EBUSY; } /** @@ -631,16 +643,7 @@ int invalidate_inode_pages2_range(struct address_space *mapping, } VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); - - if (folio_mapped(folio)) - unmap_mapping_folio(folio); - BUG_ON(folio_mapped(folio)); - - ret2 = folio_launder(mapping, folio); - if (ret2 == 0) { - if (!invalidate_complete_folio2(mapping, folio)) - ret2 = -EBUSY; - } + ret2 = folio_unmap_invalidate(mapping, folio, GFP_KERNEL); if (ret2 < 0) ret = ret2; folio_unlock(folio); From patchwork Fri Dec 20 15:47:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916925 Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E52121C180 for ; Fri, 20 Dec 2024 15:48:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709726; cv=none; b=a0N+1azZIbmJqnMXmvfR8C7vL0cXioL/gabLFAlfz4oSeomxGJP4q0YmoaP2vzahyPFjUTQu8jP4zSqR7itv0dS61IPf3ArF4mTUTVGKs2w09DhBanTVqJSwf5qkzffAiweIbNa0TIwa8bw0mLUNkMBY84R4piPe+MG+wekrTAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709726; c=relaxed/simple; bh=2SEcDojzAqUnPgfjf8E9nRmr9dou/inG3/DhaNUqq8I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=U2hwHU2yFiEpj/bJP0S/WW2TwXNnNPZU0xCa5nqFx3lZ3ffC7SfXmObrj6mZZoKqW9ZHeWvFWc7S1RrioiVEui29oqxaVwkfrZvaeLd7iP+ShVDwx2iOvNjMocjUOv1gh6dDEez0azSMXKNrwFQzSeuC0Adtzr2iGa6oUK1AnRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=pc4Oh3rK; arc=none smtp.client-ip=209.85.166.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="pc4Oh3rK" Received: by mail-io1-f44.google.com with SMTP id ca18e2360f4ac-844e61f3902so168710439f.0 for ; Fri, 20 Dec 2024 07:48:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709724; x=1735314524; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=pc4Oh3rKs651mSGbJFsxIq7XbGM9DQE81MtSvSAEJpzIAX+GD09TqGFN4OhkXpEjCJ de8Afb3cdqyyMI9lAQtd6+O4c154QWEfsLMfmxdymGWhPMzRC2DkWsurRm5H0Mk2SnrP lKRyvz6Xj7dl/c4vLRMrRY9iACZVEzmyCQqOUJI7naeRq6IX92GHebPZV9Y6Exis68uW VtQdiGFo0NnEi8Fp0XQRrbBPgOuwUpzSlqkfycmGH8DNDkQL1WmoWvOTZsCUX7+t8TBx pF+VaWV8VsNbFr2Gb1X2SDUoXc8h2T4jUh1RYHODrzzak4DnlWqjlnuCjPLpXf4kT1z0 M8AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709724; x=1735314524; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=nV2iihMw2JZKj6CuGafqBFXjlu4SXBKWwCRRnvIgIWZyF+/Cph25aZEDzqhuVGpCrV x6MJrJQGvEBY2E44br58o3CIS/QmJq0QxkC1Hh2hufbp4v2WE+sae4qVBR3cQC4RmRvI a2Ghe0/6vhnjLcK4tzoUhAnMgut3Txfqrmyfbr6SFtgpTd8kWM/OmuDpvH83Zm1EsMi6 VAge4Lhcu1iWkCfPjhFgaDyiRI6xIOGotJhDeKr9hQwyVa2Vj3vZ1MQPluGPEtkPC3i9 DYgDaTno8h9C3iTQTCW5xjhLQi0gzzUFpOWLiUj9gY66xGKK2O/TrrfJ1p9Xrk9fJ7yH xfHw== X-Forwarded-Encrypted: i=1; AJvYcCX6FtZsKdu6cXa/5DtjMHmJXZdhI3gzffQT+60wKK4DBbanMc0yRpVJefMIdRQ7Kro8OqKyOfXEUMPlIHUI@vger.kernel.org X-Gm-Message-State: AOJu0YxCaFWBIbvS1v5sKxEjoBYn4RTSEXQHzca2CEsIKRAY5UQK/YYP hPJb0FyOpqMKNT0FsLvNLMmz08tBWZ3iNwyptas8SYW4J8Ng/10Ln8pAfmeXG90= X-Gm-Gg: ASbGncvD9dlmCxNI1zlMdOoog5u7FqnNrcc3x3c95aIbjUE6X7wN2OuPOLy2ApFNA2e oseOKHNJf7GZLIAhyZZ3wLbl29aTZ9mTmeM8VkteWolq97NHPzxhCc8HDnJdopwqSZ52fQi7NqZ J4XxZaJrdKmOpkLawq/wCPXQ+3grVQYr3NI83GWFVL3HHEp9FxCV2IUcjPYvLy6taiFJ0vXfZjb n1eSDNskWwumwCzHv+x1+ufUDQFImUktWWURZTFkFJSbq/xg/jBoiUnqOKI X-Google-Smtp-Source: AGHT+IERkjVXfRiBHnc8FlfnVZJTgamzH1zCvDow9NijC4/j/GJhywdFRh297ssQYYJsqQo3rjiWFg== X-Received: by 2002:a05:6602:60c6:b0:83a:a746:68a6 with SMTP id ca18e2360f4ac-8499e4d888bmr383870539f.5.1734709724489; Fri, 20 Dec 2024 07:48:44 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:43 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 07/12] fs: add RWF_DONTCACHE iocb and FOP_DONTCACHE file_operations flag Date: Fri, 20 Dec 2024 08:47:45 -0700 Message-ID: <20241220154831.1086649-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If a file system supports uncached buffered IO, it may set FOP_DONTCACHE and enable support for RWF_DONTCACHE. If RWF_DONTCACHE is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 14 +++++++++++++- include/uapi/linux/fs.h | 6 +++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 7e29433c5ecc..6a838b5479a6 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -322,6 +322,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_DONTCACHE (__force int) RWF_DONTCACHE /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -356,7 +357,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_DONTCACHE, "DONTCACHE" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ @@ -2127,6 +2129,8 @@ struct file_operations { #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) /* Supports asynchronous lock callbacks */ #define FOP_ASYNC_LOCK ((__force fop_flags_t)(1 << 6)) +/* File system supports uncached read/write buffered IO */ +#define FOP_DONTCACHE ((__force fop_flags_t)(1 << 7)) /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3614,6 +3618,14 @@ static inline int kiocb_set_rw_flags(struct kiocb *ki, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_DONTCACHE) { + /* file system must support it */ + if (!(ki->ki_filp->f_op->fop_flags & FOP_DONTCACHE)) + return -EOPNOTSUPP; + /* DAX mappings not supported */ + if (IS_DAX(ki->ki_filp->f_mapping->host)) + return -EOPNOTSUPP; + } kiocb_flags |= (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |= IOCB_DSYNC; diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..56a4f93a08f4 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_DONTCACHE ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_DONTCACHE) #define PROCFS_IOCTL_MAGIC 'f' From patchwork Fri Dec 20 15:47:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916926 Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B75EA218EA8 for ; Fri, 20 Dec 2024 15:48:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709729; cv=none; b=lCnEWkpkvKzRfE71x+ZMMTlFwkFY0PknDXelQuVLgviuC/wqP7IjJUpbp9JgkKxiUCaBMdGi+Nc+18MfP+ZI6j6xHz1nXX1E1hzrQspaclBaa/ucLu6I20y9gJT5B9U36+xyXebDNxxT39kHCSAqbGs4tyCAbJsTc4Vi+Z3ocTU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709729; c=relaxed/simple; bh=sLgD25Nhzbud83QHSsVSAK9UDdaoG1oUsOEMQEZOwUg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RUslldes83cq3IF/HO565J++R2E+oecXWvzL1e3vB0LGyupNm7Z5eL67dOGEd6W/fjB5/6MrgRE5y+a80Gf5VjvXdwS1di9kRSfWgufXlBvEDQnrD5CMK8ps30Wzhgh7ldGT/e5LAxrT5HuII6EDkXim+jjY77G8yIzvwbliCB0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=pTwLuQKR; arc=none smtp.client-ip=209.85.166.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="pTwLuQKR" Received: by mail-il1-f182.google.com with SMTP id e9e14a558f8ab-3a7dd54af4bso7540895ab.2 for ; Fri, 20 Dec 2024 07:48:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709726; x=1735314526; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ks2mp2gCJ77yF3ZCGIs6WN32l2VaUGiu+2pglDT6eiI=; b=pTwLuQKRV9/Q3bq6gpmLR//R6ym34L5m6oW8S1GwoSYILJ9wTgck8/oITVcsgXUSCD AOe+Yn9S9UTOb61p7PakQV49SOm5f5Zh0torgzxBycAnzBFSPkUQ3RyhwmIzR484EQ1l HXsJMt9DQeyDf2YN0a6iDqkWCNiyK1CshRFATM84Nc/bzVIdX1QUcuqMpjLZ+aIfVfpJ xJCFHfxNzb7PrHh/G4gxmKippkxR4AeAvi9FydrVqS8XMbLrBhgjvXBu4CjgKuZGVqIH FssnBsxyRamd41o2MX9tkArMzab4R8c3DI1zWxfyxpTwVVD2PXqSrEuyVCeZR6sst1iA 8/bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709726; x=1735314526; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ks2mp2gCJ77yF3ZCGIs6WN32l2VaUGiu+2pglDT6eiI=; b=RJPVgWURj7eVIh3hAjzsw97nToiSnGfUi42KH6kG+pT/3+c6ZMezzBwWFkocDysGmy mQXkf7T33OsneLmTu8mGYtIOss8FzRsSOIYMs/2kY6sHxlUtHUuCK4FF5mcnsJch6MAd w9LCJj4wXQBTrcRn+xcgqRKJvRz4YABTyhfsqQzNwnRCpPzAN1oK5xq8p4kxt0swMi2K YVosFkgdq6wOhKumti2NmmZ6FFMY3V6jt0/jCLSgJKjoZQLJDol+tTk9uryrhJaWURwt kK8Iuu0kaG5IVuFCucd6yTAuLwJG+EYNKz9SR56xB3cuw6oxfrTvxzaxcUYEXDdKvT5N rNlg== X-Forwarded-Encrypted: i=1; AJvYcCUIGp+l7CJ8wy6W+a3l/IT31IilkgMiRt8DejnvlFwYk5za3RTsG2HKStd0yElVq+VHMLnjBfuxsRAWU01r@vger.kernel.org X-Gm-Message-State: AOJu0Yymerwbn2DgKn/yNMCBnpC6QCtCKbpa29k9YTuhX3LWSoCkLnea BMSFMomyq2bK97RNLb5llARLZ60UDuQIZ6ZDBJdtPi7kBlbv42t92sIFwDxJu3w= X-Gm-Gg: ASbGncvzqQqAgX7UA7n2Au6ML7C5+kbF6B2yaDPvxz2dSNQXbl9/qv2qj3RkED+W6HS kXkd6p3kPgKITMQzuV7teXl6jGuQa1J4+IQHl2z5qr0X6FptHQhwbMjqc4aCy2PXnZ4TXmHAY0B Hrl7dHZdqxehbAQfG5w0gZiykbLWHZNws2XUmCLBik5/888IcoEksAEXljnOAgoDpnorbe+L2Kk V3qE64tP7dtqdyq/3UZgZpXG+oj6BislPxN4Cy9m5sCzvbAIdM5Ij9YCxF9 X-Google-Smtp-Source: AGHT+IG1Q5MpgKTbxJc/Yphg2SqFQXSKzJLSB4ruy07feq+/t9lmMQDAi0QHm0P3EjCYdC27xoRmeQ== X-Received: by 2002:a05:6e02:3201:b0:3a7:be5e:e22d with SMTP id e9e14a558f8ab-3c2d1aa3e86mr30848135ab.2.1734709725732; Fri, 20 Dec 2024 07:48:45 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:44 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 08/12] mm/filemap: add read support for RWF_DONTCACHE Date: Fri, 20 Dec 2024 08:47:46 -0700 Message-ID: <20241220154831.1086649-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add RWF_DONTCACHE as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousin of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_DONTCACHE set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++-- mm/swap.c | 2 ++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 220dc7c6e12f..dd563208d09d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2473,6 +2473,8 @@ static int filemap_create_folio(struct kiocb *iocb, struct folio_batch *fbatch) folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_DONTCACHE) + __folio_set_dropbehind(folio); /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2518,6 +2520,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind = 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2547,6 +2551,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind = 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2594,6 +2600,20 @@ static inline bool pos_same_folio(loff_t pos1, loff_t pos2, struct folio *folio) return (pos1 >> shift == pos2 >> shift); } +static void filemap_end_dropbehind_read(struct address_space *mapping, + struct folio *folio) +{ + if (!folio_test_dropbehind(folio)) + return; + if (folio_test_writeback(folio) || folio_test_dirty(folio)) + return; + if (folio_trylock(folio)) { + if (folio_test_clear_dropbehind(folio)) + folio_unmap_invalidate(mapping, folio, 0); + folio_unlock(folio); + } +} + /** * filemap_read - Read data from the page cache. * @iocb: The iocb to read. @@ -2707,8 +2727,12 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, } } put_folios: - for (i = 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + + filemap_end_dropbehind_read(mapping, folio); + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); diff --git a/mm/swap.c b/mm/swap.c index 10decd9dffa1..ba02bd5ba145 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -427,6 +427,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_dropbehind(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; From patchwork Fri Dec 20 15:47:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916927 Received: from mail-il1-f174.google.com (mail-il1-f174.google.com [209.85.166.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0AC321CA06 for ; Fri, 20 Dec 2024 15:48:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709729; cv=none; b=CrNjSgqf83YlUU+Z8J8JwLAlwahe+V9CiwkNMoV0jGClG5xHM0r6uWR+O1edRL+hIgVZuoNlSdlNm5yv6rCG17rPz+ajWhJ6s17ih5UJ/rw5+0kzZrVTde8Z2i57DzcyRhiwR+RThR5/a72WMcXnYrskb5xyzEfALtD5tT4xtus= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709729; c=relaxed/simple; bh=0z+Su/j7sFm3LNscSJlbortbXvVAgYiyFzcEVovFWT0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z3Y7Xgd+0em+PwmXAPZlouYtUli0jnHcVpRWfxTXn/Af6HaDlOBy/mi0YBW0WZOkmNZ7zHToNe3iuqAXRIbK09MSCrWW8lWsoIpcn8lnipUHFwFT0D9uDMOYtmG3WcZ/R+5H5quM+AXd448xhYpn4dnfEL3LUsSjh0qPN2QqXlU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=FPp6vWfm; arc=none smtp.client-ip=209.85.166.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="FPp6vWfm" Received: by mail-il1-f174.google.com with SMTP id e9e14a558f8ab-3a78b39034dso6240995ab.3 for ; Fri, 20 Dec 2024 07:48:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709727; x=1735314527; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CWUKIERhmBif0xELkyRs9dlqXC97dRJQR9fjviiscOo=; b=FPp6vWfmmwe9NAWIBJmCxLtVD/hNP3BfSRpL38o0SUhrItLpx26f4Yg2aryhVcVvoM oPpp3NOtj7OKWZxGCnfwcG9g9XNK06KVogtbQHCDg8aY2+KQWK+Gml3annJycylGeLWM VQ1kjdOYDuMMIT7W2aVyIxb1yTomtke8VSfksgOXvvkFxe9p1Wv/YGuGwkhvJgu9kDY3 kxOwEwqwnebKAvLbCMZrqSUwIskcxGhWX71jrH+qY6XTE8r7WH7lFQhuo7xqJStur45R FdMuoeSXbu+PxhnUHTXiYz73dqknNbkjP1X3wZUw36JxSQNonJKRdhryqdAgT7BlbPd4 hl6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709727; x=1735314527; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CWUKIERhmBif0xELkyRs9dlqXC97dRJQR9fjviiscOo=; b=bGfdQ/n283u5KUME6WoG56xtvFVl3yQu0FKYIt6cFL0Y77jLqiSoGWHsj2yL8lcFVC O72x5Rcng6VieSMMI5tRqHnmpy/sd9lg2FHYoSMG9+5PHcoPvz447fHwi+rd/+68Z5f7 64NmBYHNTFPbhVyu16kXyTfKO8ihoM2eqWv7OwSfUv6R9uhKr4xcZ+n8wdLm682f4KLi vQ4P6cJKzO8v11WVER2Ms/gvWFyXfWzjZglPBI2wZQKWvj0VFPpe7cPXq/NEaPN9WmW2 WpJVdn+Ex2Blo2XoNOUZQAFW/8qY2dmC9iPp3zK8nxA81tMVH7S3V09d4+M7/+OuQ5kh lbNA== X-Forwarded-Encrypted: i=1; AJvYcCXH7G46iPqo2hYYE3Ujj1qTXSDnwn4gw2Bfj+4ZqrVO9smLZZYNj+jttI37PB7yEN+Vl2oZ3EaXqhv4zwOd@vger.kernel.org X-Gm-Message-State: AOJu0YywsDp08351DOXyKpfd+OVkpB+7h2BbCFOxps1bC7yG4sxbvFp8 vdH5M6k+e9aiAoeX5AEMD1ciKLY+R/VLwP4fxPs9D9TlMViWpxnDKsvI7WPntH/HciKF3zrAnEq S X-Gm-Gg: ASbGnctoJLm8kPkeo7Td90NNg28L5s9NA+jPJvNDKk2n6OBXc4Lf9G6FG7MJTq3JWEs /FyFDWxdn8XlJMxR9MNNP+2R5nsfnybehS9qKnEL8hlq9yuTveY9jrs5EeW/nMFhZZKYzhE64jj W+oi2b5zop/QEfKu5sRjLD7r/2IOBGaaAAVUUDvSy25KLPGiWZVOWh38OxeRF3nrf6bmHOK9oU6 3Aqt3YGjhWHueauy5p1sdhGkFxKOx3REgM5DTXQPEojLJVJkKRvEpunaglu X-Google-Smtp-Source: AGHT+IGjD+wBfW2QpC5lojTx62UJZa1o0QwQiIlqK6Hx/4wukTE9KhfBEOmSehD69MzvwKGp6r7FTw== X-Received: by 2002:a05:6e02:522:b0:3a7:e0a5:aa98 with SMTP id e9e14a558f8ab-3c2d2d509b6mr25164805ab.13.1734709727201; Fri, 20 Dec 2024 07:48:47 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:46 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 09/12] mm/filemap: drop streaming/uncached pages when writeback completes Date: Fri, 20 Dec 2024 08:47:47 -0700 Message-ID: <20241220154831.1086649-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If the folio is marked as streaming, drop pages when writeback completes. Intended to be used with RWF_DONTCACHE, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index dd563208d09d..aa0b3af6533d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1599,6 +1599,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as dropbehind, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset dropbehind for that case and latter completions should invalidate. + */ +static void folio_end_dropbehind_write(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_DONTCACHE writeback, + * but can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1609,6 +1630,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_dropbehind = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1630,9 +1653,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (!folio_test_dirty(folio)) + folio_dropbehind = folio_test_clear_dropbehind(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_dropbehind) + folio_end_dropbehind_write(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); From patchwork Fri Dec 20 15:47:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916928 Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C31D21CA1C for ; Fri, 20 Dec 2024 15:48:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709730; cv=none; b=WoAY0E6TAOmA7ADe/IlceJnvOoCrDsF6Qg6iK+H34lPZ+wwLyq2aZsTFodK1litODuYmiccCzvekEGXpYV54MugX8EgeDwb9W58nsVWVYkkFTYGtKEjeSPdcisOSKiCwPTvlnAyQ/+c+4qYE6NxwzrsLArr19JRm1R6DymK08fw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709730; c=relaxed/simple; bh=GaN0/oBzXsAiWqwBFn1n2Fc++lywThntHtcNeWBfcz4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EzuSwI0UoeS5resPVOC6S71/q6kCcLvI0UGHIjc0AcI+w9WKjnlKH8/peg8Z9f5equ9NS5/Y8IReFFAtJ6PGM4nkhGTl6ZQrvAMsbKqPlmZ+4dYxXV9/bJhh9AnqvfmGLktP3yoVRdXyyon/1BGEHWNejfqqQsUZ18GtzID1s8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=ncr/MD9e; arc=none smtp.client-ip=209.85.166.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="ncr/MD9e" Received: by mail-io1-f51.google.com with SMTP id ca18e2360f4ac-844ee15d6f4so158518739f.1 for ; Fri, 20 Dec 2024 07:48:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709728; x=1735314528; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z4qQwDJE6vdlPLyJQIpmMrltr90cAGUkJZOMxstE6os=; b=ncr/MD9eAO4S+4YZqT2PvSRUEUfRH0XRRnl8I516z8jRSkZ31sND98yfS6V+rRRspf 5Kf/Ih7ECLvuydILFZhyqjSWG5a0bFHOQuclk7Aa5IkNAXA9StMT23TpMMzuiaE8Reop s+/SZtvp/g63rRqlCEC4yu/eGk5P5iygxFE7q1pQGkGw2K1J91g8DrjgTMMq4WDIRrSD D0Fv68GrKAqWRL0VI0jnvmWPRajAKViBpycTXrWNDBoIKkqs0jJq5fWbWXhb1zrIdMDv OaUM3NEj+wa0kS9lobiBPDHTorWqD9w2cbpGIhaDA4lYoZVx4sZFwChBBXDGWiJfmQyw Z1zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709728; x=1735314528; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z4qQwDJE6vdlPLyJQIpmMrltr90cAGUkJZOMxstE6os=; b=tvYNE71TwtBsdxzILtbDbizj/otezOWyGjWr53Wut92Dw/1IU3oNfHmk/jw0c3E2F+ 0A2/4Lh0HyzjrPXd6GQYVSr8FfQ4yiLMtUZcNgeaVPcV9aevIo+dnAUxu771nhlVfS2+ YxdSITp/xeTuj/W2RlfaIuRl12NpajZcRJ2hHA3ZsBYIBssp3dSTeB8rXVbVBFZcqJri myxlWmYjDHnfaqW9aET8wPE/qtK9qk/lBm+3/UkWYgFp6dutcUJALBjc+J4wudmip3Nl R8G6SOXMLMtKs5GFk4yRfV7YQ4zQ5H3V63rexTDXvaBEBsKI/fV9L6v6npbKWeIlI8XI s5GA== X-Forwarded-Encrypted: i=1; AJvYcCWow4vuk0umC0OnAfkTIiNkpe0o2us+J9XHf0rpS4SlSG3ZSxi8OufbYk5phSHJPdhghEtMIOGLJF6S9GbS@vger.kernel.org X-Gm-Message-State: AOJu0YxT3oCPmXbySibIMTq3MXOJ1AiS3X/Uk50BmIqDdCEDTVjj++va zBYfQKKnZnYf7HjsHErX80CE3RgwFXM/dR2cbVnCYq3l7LoN26ArpWpVQyXrmcPqdDYirAqGqYv 9 X-Gm-Gg: ASbGncudR2PW7cLoKuDHUfSIjP89JAk3rQZFtjhDrslQqBh7ZunboLDmV2s4+Aclm7v waOWVCGPgzn4IRUFweTCwuL9n3+7aPtt48UxnfMnHPgiN6nAoTPtTiU/t6NU6kN0iZnWCjWast/ MOWe5/nRVXDUzfRq/H1dmTL0gDDWLX3suNM86hJU7u8qlM3H+6KDFuUvEI7fERA8LNihMoV8Om1 l45K46gqCbm1oXbqKD8TaLg1YwtBb/QFOahBdNzkOBbR8v81LR3SNocAY2f X-Google-Smtp-Source: AGHT+IFGv05g9yMPyknBeMlKwprAW5pwl3vrHXnYcOynrc2BQ0O5hQEIENaNx6lgdvRHQ9LzhuK/ng== X-Received: by 2002:a05:6602:158e:b0:844:e06e:53c5 with SMTP id ca18e2360f4ac-8499e4fafe3mr298729939f.8.1734709728444; Fri, 20 Dec 2024 07:48:48 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:47 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 10/12] mm/filemap: add filemap_fdatawrite_range_kick() helper Date: Fri, 20 Dec 2024 08:47:48 -0700 Message-ID: <20241220154831.1086649-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Works like filemap_fdatawrite_range(), except it's a non-integrity data writeback and hence only starts writeback on the specified range. Will help facilitate generically starting uncached writeback from generic_write_sync(), as header dependencies preclude doing this inline from fs.h. Signed-off-by: Jens Axboe --- include/linux/fs.h | 2 ++ mm/filemap.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 6a838b5479a6..653b5efa3d3f 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2878,6 +2878,8 @@ extern int __must_check file_fdatawait_range(struct file *file, loff_t lstart, extern int __must_check file_check_and_advance_wb_err(struct file *file); extern int __must_check file_write_and_wait_range(struct file *file, loff_t start, loff_t end); +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t start, + loff_t end); static inline int file_write_and_wait(struct file *file) { diff --git a/mm/filemap.c b/mm/filemap.c index aa0b3af6533d..9842258ba343 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -449,6 +449,24 @@ int filemap_fdatawrite_range(struct address_space *mapping, loff_t start, } EXPORT_SYMBOL(filemap_fdatawrite_range); +/** + * filemap_fdatawrite_range_kick - start writeback on a range + * @mapping: target address_space + * @start: index to start writeback on + * @end: last (non-inclusive) index for writeback + * + * This is a non-integrity writeback helper, to start writing back folios + * for the indicated range. + * + * Return: %0 on success, negative error code otherwise. + */ +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t start, + loff_t end) +{ + return __filemap_fdatawrite_range(mapping, start, end, WB_SYNC_NONE); +} +EXPORT_SYMBOL_GPL(filemap_fdatawrite_range_kick); + /** * filemap_flush - mostly a non-blocking flush * @mapping: target address_space From patchwork Fri Dec 20 15:47:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916929 Received: from mail-il1-f179.google.com (mail-il1-f179.google.com [209.85.166.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B72F321B199 for ; Fri, 20 Dec 2024 15:48:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709732; cv=none; b=HAGn/ou5IsrgH0EySg/PfpgNc83H98k09RI5cSsK1KvsktUkUvooSj4IGI+oKF2RltbkEo3fBMKiu72fK4E8PDVY30aI10LK+Gbm3oB7t5dqJCYN9cVPdGP8R9ZuUzrzOdb5ezjWx6NYLMtB8WZ6J15JDbaKo2dUJyZAXSHkHao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709732; c=relaxed/simple; bh=zE0xjm5rhzd2cUhrSxtogiQia/migrwaLvBNVMhxtxQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZyY0YIZgq04nRsDSrkK2XWbAJ+Hskb18qNWLulJw01bxUPgZ99Wb0KjAHbc0w70gm62itnLZIEEUO3oNqOLGfy/+hmrcgJwTRpC+/HblUaPleorS/uUqVJBQedhbubLxr6Yrq6+MnGUgiWoS4alfgoJ43Hp9pQ3F2u4fMNTIN3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=XpNPsYgT; arc=none smtp.client-ip=209.85.166.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="XpNPsYgT" Received: by mail-il1-f179.google.com with SMTP id e9e14a558f8ab-3a9cb80dbfdso14303875ab.0 for ; Fri, 20 Dec 2024 07:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709730; x=1735314530; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=XpNPsYgTpipkSpVs7LrbeF1UL0my+iixYP3Cr4OQkBjATc8tdVJhLIT476q7wihOZV BkB5ehPy16t6rAFUTluHuYjes5Gaf51+jjOZiXYKMf6qtE5GjeUIijoOA/ODjm8NdrLn 0XZ8WETcvLRc+t5zb707wNiMIptKwv6c0v2+0NfazgNqK9co/x4RAnB4QT+wpvvb6GmZ 5Xmm3bZIPMrEsJLXaN8u6rDnXrVegjcLy1lcYFDlE0OsZzbhB7rYdv49n1kpQ6M7o3D6 I9fNEVK3Mz1L9I6osQSY4N8BMInx4CeTCYQySqVBUI3Ntk5Ho718230kzdmgVMGjbNjS JvhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709730; x=1735314530; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=Ch3mweB8GIShpZUN/l2FDy0OHdTyaS3zZn08/oyuHTXpkbdHrX68FzMcX9zuEssYOu q4QRKYbh7CnOyCZECfaHuMVTXubHKdSny2A5TIR37ys1CCCwrwQsQe0koeTigyGRtLuR zXjSNgsa1KTaInu8DzpUTcZIVwgAVHDMuUZ4b96zvaMCLTk7jIjrSiSdejmtuFOJMC8p AmogsTPBJ5H3ZUhAY2EE6chiGvRwFMOIlF5TXnCBX44gfnnqmDgvoBLWadMneohF+gVo aJ3I7hqFiILAY4qWC6LOJ8tJ0sYGvBcC85CBUBQNwHUiGOw0Zm4ns7zM/XzULWQgpSx2 DGCw== X-Forwarded-Encrypted: i=1; AJvYcCXmquuT5c+MYz8PNSYfOOkZ8wirX22NGWzdPiHRzi8hTuNBGXk2NSE3XcGfNZ+UF2LzyiFXPGeuD9CTJcVX@vger.kernel.org X-Gm-Message-State: AOJu0YyJAYhFHajTGFj8OHJ2StO5AEVE8UyYc8+7BDYdvj4Jo+XiOlOo DZB5PFfvd+T/J9+EyFx5iIP7ct+m9XgbWrQ4cP7tZX60Sb2x55E3ux1kqJQNQPo= X-Gm-Gg: ASbGncsCkEclB9b6kgLnKApaB3t3A+KCWHeP1xaHyYfftgNeHbFC6bFJ3otoiDMwOTy TSH1+8v86xO1iGh+gZNmNl8QSy1iW+zgUthPDWIT0hEkU0H7orvSrcd+TwezSbZYsDsOx4dhL0L nyx3FukykiM66luKJbC9l2K/WQA38WZ+FJVC8E6kPckUo1yAtXsa2tm03NNE9EyZdR2Q+6VJNRU M4cBe0uIXg+I8OvfRminOwikJ4ZcBGsj/aBIeVx8zvnpubSon4P7oal9fAU X-Google-Smtp-Source: AGHT+IHBkJtXw0hIyGWIXWPfQ49T4FfZrn/htFVCpySmbejAxdhZ6x6C0BhRXfs8Jh6Ol55mQ+pxBw== X-Received: by 2002:a05:6e02:17cf:b0:3a7:70a4:6872 with SMTP id e9e14a558f8ab-3c2d257934dmr36428035ab.9.1734709729627; Fri, 20 Dec 2024 07:48:49 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:48 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 11/12] mm: call filemap_fdatawrite_range_kick() after IOCB_DONTCACHE issue Date: Fri, 20 Dec 2024 08:47:49 -0700 Message-ID: <20241220154831.1086649-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When a buffered write submitted with IOCB_DONTCACHE has been successfully submitted, call filemap_fdatawrite_range_kick() to kick off the IO. File systems call generic_write_sync() for any successful buffered write submission, hence add the logic here rather than needing to modify the file system. Signed-off-by: Jens Axboe --- include/linux/fs.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 653b5efa3d3f..58a618853574 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2912,6 +2912,11 @@ static inline ssize_t generic_write_sync(struct kiocb *iocb, ssize_t count) (iocb->ki_flags & IOCB_SYNC) ? 0 : 1); if (ret) return ret; + } else if (iocb->ki_flags & IOCB_DONTCACHE) { + struct address_space *mapping = iocb->ki_filp->f_mapping; + + filemap_fdatawrite_range_kick(mapping, iocb->ki_pos, + iocb->ki_pos + count); } return count; From patchwork Fri Dec 20 15:47:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13916930 Received: from mail-il1-f178.google.com (mail-il1-f178.google.com [209.85.166.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14E8321D5BB for ; Fri, 20 Dec 2024 15:48:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709733; cv=none; b=QKjqPpeaq/2OJ74Yhe/o8IfUSoh8b+e+LlU6utmp+IjMDo1wnqch0Q73nFWQxWlXHAUGn8yHCb7TJt1UYKlRoua6pdI9Js1KZAPGW5HAGxCYzfPmxO9P9v8w07Q7ciy7sTn9vZnGAunWph4Q0NKlt604k1RDccFbE4Xq2/dCj3g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734709733; c=relaxed/simple; bh=nMl1WIMQHt2bPQz6aG6MCeSjhSpxjtnQ2LzuXFwS3EE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FBgqOj7GID5cfWzYMwKs3spJ8MJIFWA9/j1fg8aGTOfs2N7wHW7smSZAe2fjfMGwxzgybewh6jYq3s7tjTORCQzqMETqf8DB5JkIflrDg54xhoim1YajYBctce5pGmocQ7fEeusu8t+18/J0437a4rcGUDQ+70CYKTp8ojtuJYE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=rQoXGEdg; arc=none smtp.client-ip=209.85.166.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="rQoXGEdg" Received: by mail-il1-f178.google.com with SMTP id e9e14a558f8ab-3a8c7b02d68so14944085ab.3 for ; Fri, 20 Dec 2024 07:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734709731; x=1735314531; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=86GdD84SMshyNZnB2LgwxYI5vb/3z8g/QfHnuRLr9vU=; b=rQoXGEdgBAzash2UkPI/h6AWqTly2sPjVW9b7hA4HKvUEDfPRY9hME8xK+5OHk9dPB imdsvJiZzoYLSP1vgailaI0VzLusxo0vLieDcqeeQsKtokxL9Y8ZP3QaILODAfuWpe8R 1StjFxmLHfw26Thl9noyWzN8NYsirgyQkUFyhe2nQAs4waoVe01LDIQsJOzfrAd1+4b4 reZMrtNARzwTYhjFHd+s3dWJ7Wjg9zBWH1SZ//2Lk3JMTczFKAFrBRdlDnf8vYolCv7N kynphxojBktG2JkHYrhoDacuNHMOPY9LODRFxN4kdxcuG4IeO6nP5L9fpm6gAs/Jm4sz 3vZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734709731; x=1735314531; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=86GdD84SMshyNZnB2LgwxYI5vb/3z8g/QfHnuRLr9vU=; b=SWtgNKkDnJ+CxVUV1ZWZAeFSWK3HLURlBhNYZnjEj4AB8glM7vGCLwBjZ8usBriNCa NBUHVJWbbwh90t0iRQL30grFYXPJvO6EcG/EhwQKyW2ac75nw9NyotCh+zNyur+iH9o/ 9kQ1+uVO1LGbQuS82cGora/KvjJ8hZsEaoHHEwbNzUArofi3paaukxOCH1QuUn/r19u6 cscQXrHaJsX8xa6UW+8R62j2whQeIC2PtAXYPcDkEAamEMcwx/Bs+KDs1hwFnemz7t1i pJvKvjw5AJpLd8z6qLAdJVj8k2wJbCvhdqCnoNT8nfrKNCubM39fttT6FKCVUHxS53J6 FV9w== X-Forwarded-Encrypted: i=1; AJvYcCU8zAmJnPhBRCG8RGQLlT465JKPtfwDKf62XSPUb5jbH6gPDudk+VaCj0966rP/3n+Z7gtL+DZnaRCbvgC9@vger.kernel.org X-Gm-Message-State: AOJu0Yx050LSyP8dLhUcZdVS+eNLojT86+uitwoYk7jALcPnzV/RzM1Y 0ANyoo/kIRezoBBCT4nJL61FYtz7CegTIMOAmpIDPoPcspmoCxkhAvRYNaa4ub1vXV0DhISqdBW D X-Gm-Gg: ASbGncswUyYu//iXZbkAHi+FSttQKEjTjXxtIa0qbAIDJongA25JuKlm1/hTvSjU30m dYpW0wxNXoSfxgMd8Fb3LW/NJDTl+7h47IxKx07ii/bqJ0vknaz4O43oeZghtZ2XTjjmDanZbx2 3o3uhKvuJDfNRvAJ6gMTK4slIYUzMLDloVrTE9EM8whoFsHuJZHnj46I9NG4EM+xYHdxwbJ4vB+ N9WgQ3P8GYk/ZlxhjjQIIIfJe7v4Ku45qy1OcecEVRMu/9ftJLszD1ItWRH X-Google-Smtp-Source: AGHT+IENwEpB78XkZV/FFvO7zp/8wAKDMGCn+bUr8XfGQ9+IvUs//gy2KTAuIDXDGGYGOcgB+kegLA== X-Received: by 2002:a05:6e02:20c1:b0:3a7:e3e3:bd57 with SMTP id e9e14a558f8ab-3c2d533e943mr33235325ab.15.1734709731154; Fri, 20 Dec 2024 07:48:51 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4e68bf66ed9sm837821173.45.2024.12.20.07.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:48:50 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 12/12] mm: add FGP_DONTCACHE folio creation flag Date: Fri, 20 Dec 2024 08:47:50 -0700 Message-ID: <20241220154831.1086649-13-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241220154831.1086649-1-axboe@kernel.dk> References: <20241220154831.1086649-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Callers can pass this in for uncached folio creation, in which case if a folio is newly created it gets marked as uncached. If a folio exists for this index and lookup succeeds, then it will not get marked as uncached. If an !uncached lookup finds a cached folio, clear the flag. For that case, there are competeting uncached and cached users of the folio, and it should not get pruned. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 5da4b6d42fae..64c6dada837e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -710,6 +710,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapping, * * %FGP_NOFS - __GFP_FS will get cleared in gfp. * * %FGP_NOWAIT - Don't block on the folio lock. * * %FGP_STABLE - Wait for the folio to be stable (finished writeback) + * * %FGP_DONTCACHE - Uncached buffered IO * * %FGP_WRITEBEGIN - The flags to use in a filesystem write_begin() * implementation. */ @@ -723,6 +724,7 @@ typedef unsigned int __bitwise fgf_t; #define FGP_NOWAIT ((__force fgf_t)0x00000020) #define FGP_FOR_MMAP ((__force fgf_t)0x00000040) #define FGP_STABLE ((__force fgf_t)0x00000080) +#define FGP_DONTCACHE ((__force fgf_t)0x00000100) #define FGF_GET_ORDER(fgf) (((__force unsigned)fgf) >> 26) /* top 6 bits */ #define FGP_WRITEBEGIN (FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE) diff --git a/mm/filemap.c b/mm/filemap.c index 9842258ba343..68bdfff4117e 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2001,6 +2001,8 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, /* Init accessed so avoid atomic mark_page_accessed later */ if (fgp_flags & FGP_ACCESSED) __folio_set_referenced(folio); + if (fgp_flags & FGP_DONTCACHE) + __folio_set_dropbehind(folio); err = filemap_add_folio(mapping, folio, index, gfp); if (!err) @@ -2023,6 +2025,9 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, if (!folio) return ERR_PTR(-ENOENT); + /* not an uncached lookup, clear uncached if set */ + if (folio_test_dropbehind(folio) && !(fgp_flags & FGP_DONTCACHE)) + folio_clear_dropbehind(folio); return folio; } EXPORT_SYMBOL(__filemap_get_folio);