From patchwork Fri Dec 13 15:55:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907386 Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D581C1E32B9 for ; Fri, 13 Dec 2024 15:56:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105368; cv=none; b=nATTqOTbiMrzDhzCNVun1wefJcfgNfSmfCC0Bh6OL1TQOOY+qiCHsZBJyDpuSKC/iZDKEGlej33HyszcGX69eF/ZHhAIEFiNd/GWcRs+lth9tlhKykY4btTQju61NYf2Q8MGgXHI4lbj6ZT5dTDZzyuEWBHY0yf6kbKzcHz9P1A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105368; c=relaxed/simple; bh=CQHljKL0HvF5DjNsL/7Q8SjdRe1DhHG7gJBg0F4F8y0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rfe9OrZdU6j68OImPwA0V5jCw9TNhoJz9zZED2rZE4hkjyoQnsC3+v67m8zY90IvFB/BA9xRNXQGl/I6HLrOZE9lnLsE0rd1nAU+Bfc41Ys5gMioyktnkZHCAD7MD3MqqQQ5hX0mM2kZdOiI6TXNAUeQkomxZEihaWfYybU7VSU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=rA2TyAiT; arc=none smtp.client-ip=209.85.166.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="rA2TyAiT" Received: by mail-il1-f182.google.com with SMTP id e9e14a558f8ab-3a815ed5acfso6552375ab.0 for ; Fri, 13 Dec 2024 07:56:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105365; x=1734710165; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aJTsU8FOCztsZChrgynI/se4Hg526H6WFOVyatkHjPA=; b=rA2TyAiTEnZDuJUEzMLUg7pzz4q8wsMtXMQ2nso5c9/p5yCYWDwsrcSh98ibhdDugo QYV5xGrQopJndH/3afNqdFjHohTuwzsGMkT761zJSNSavteA9lV4LtZy3bVeWeEALQC7 y2U5OgpUZOM/ilJDz/2vlhIHaKxNy7YNbaHa0Y6V1CvSFxFSqL0Zed5VpY7KA2Mbm13b RIPhV06nHYQpDmJvWZLtjsFn9pkXVnRcIJ/wY0tu4icjeX7eaBnpKohIA8TsfsH9ZdF4 g8AAiof2EMeyErrAX30lLtyacgcuzUBw4bCRybt59rq1NmeuU/eLUcwz7Tcm0Zh4oVim BvWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105365; x=1734710165; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aJTsU8FOCztsZChrgynI/se4Hg526H6WFOVyatkHjPA=; b=ks0U1PAU4Y7+JIWIlt/OKfy0EW6jb4oDaDzdMZ21100Cq/JU/L9T66Qnq4iN9akIeC iB3/1anDzGClvQ670Qh3YETr3FFBCobVxQ++/dVT3rvhQ1TDmeIG6fShf+Bl42Zj6L2m L3qWs3+QsPnm0QDtA2cdVkAfBDIVpQBEqwlRmOQG+0QeF7MqR5e4g4eI/3fDyCPoduAp hBWb5hauex0QNZ30kUwYqW5Ypt359241JdxAHNJrDj0MqXKBvR22n+Ag2JuaI5ltk9tA QOTgtAeiaXoEluY/EL6aT0Cr2L2EqKRptcydFSxH+GSjXjyrsQ9i/NaX7tkqZodbDsSn e8Ug== X-Forwarded-Encrypted: i=1; AJvYcCUJasr3D/GaIUoocE6mHUXrBo2eoP3ObKr7Vwa9vbMu1XasMJnsvB4hS310YEqDkITeAaUxT6IZRxLmxvGO@vger.kernel.org X-Gm-Message-State: AOJu0Yy6myom9X3/HDebwfYswpDpdN/3Bh2IWZrW29vmVOqqgRsi2h0t K2EnKoFh9OajZdPeAjx3+Qhr8rb6UVR9LtFPEUhmQGh7aM1Gx8HKY4BtjGHKslA= X-Gm-Gg: ASbGncs4eTJaaW02jwp5gJkTwxnSrPzmidsJQCOZk3Plv5+Pf3HUgtxTXQHdbD7zNk0 d2vGCEZBE7Jvgb3mv+ebY5UDF8cftY+hDdk5cGoIKyRTuw0vp8lWhPPdldEcKY5k0FPeusvc85g VQkRIOaxg5Nu65qaoZQp5+wfhzyVq+O7aojq8weLlqZKcJXVTJrKFLid3ZJg2kp/5yzRC4JWO6C 2qupT50gV/QDa1QiWmvYIQpenUshTJC20mooR7AfydWeHZ8QqYr6Ug40WAv X-Google-Smtp-Source: AGHT+IHzvgK4Rw98UmNxw7TWkyvAABSCjKuo4ks1FFrTwghvSS0Ll84bbCaOcol8J/RzXzpmHkduIA== X-Received: by 2002:a05:6e02:1a8b:b0:3a7:7124:bd2c with SMTP id e9e14a558f8ab-3affab62206mr35203095ab.19.1734105365052; Fri, 13 Dec 2024 07:56:05 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:03 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 01/11] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Fri, 13 Dec 2024 08:55:15 -0700 Message-ID: <20241213155557.105419-2-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. With the kiocb being passed in, skip passing in the address_space separately as well. While doing so, move the ki_flags checking into filemap_create_folio() as well. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- mm/filemap.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index f61cf51c2238..8b29323b15d7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2459,15 +2459,17 @@ static int filemap_update_page(struct kiocb *iocb, return error; } -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, struct folio_batch *fbatch) { + struct address_space *mapping = iocb->ki_filp->f_mapping; struct folio *folio; int error; unsigned int min_order = mapping_min_folio_order(mapping); pgoff_t index; + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; @@ -2486,7 +2488,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index = (pos >> (PAGE_SHIFT + min_order)) << min_order; + index = (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error = filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error == -EEXIST) @@ -2494,7 +2496,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; - error = filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error = filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; @@ -2550,9 +2553,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); } if (!folio_batch_count(fbatch)) { - if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) - return -EAGAIN; - err = filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err = filemap_create_folio(iocb, fbatch); if (err == AOP_TRUNCATED_PAGE) goto retry; return err; From patchwork Fri Dec 13 15:55:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907387 Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E9DD1E3769 for ; Fri, 13 Dec 2024 15:56:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105369; cv=none; b=RTPiG/yICS01Z8H+0sHzcRdOXCnDTwnpmQsNtxU59P6crgeE83ofYv75CV3Zbqxa5ot+/Zk0Mdw6KmV/gt74RWW9UL4uwO3SiyB3pZPO6KugGuys2HvWHPjcs+Cshs+IkWA4Q1+/gL15DcSUIrD9Rhg6waNkjNEdrhKjPJF97dY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105369; c=relaxed/simple; bh=QAWYGVCQ4XEJ3pXTfPeihz8wMx0MW0Etgn545GbAEGI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=knLKEhdGtjF45znDRosGCuI0geUfCMPUh2/ErrkXYraF+8d5e3oenlyTN69CMDdqr1/CSKBArrrNWh1NCOqN4fbGk6yfQ4rh7N787pl+gQU4hndHzlpBvEvnRUnj78L94vsLD1P6kcyCYiY3qdnbmMQzT53B7nMImnj/cx4mPFk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=t+ltFlte; arc=none smtp.client-ip=209.85.166.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="t+ltFlte" Received: by mail-io1-f54.google.com with SMTP id ca18e2360f4ac-844e12f702dso56956439f.3 for ; Fri, 13 Dec 2024 07:56:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105366; x=1734710166; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hANs0KTrn4AbTh3ILmowq5bBaAQVBAkU7MnXkJd6Hic=; b=t+ltFlteBGcA0UWWo45gjFMi5F6YPSar7cOq/L5NtqarY+9zPmveE/IW4vEek6yAUb QlstFsbxAIu9kEt0zwXjNGD88a9t3fWpCAZpiZUmrG49S4IaVM1WmlpK85VlP3rg5XV/ kMwNchTRlDJ12UPl0JzyglzdszaKdN9d81I9v7cFM02VYogMGZAtiTOvqbgnIkb1iN1e AGmN29i9WDOTh0IGhZqjmiUgwP/YGL7zkZdN0tf0mN8GKf9/vgeDbKYExIw4tkFb15EY AdPfmvt+7GErjc501iosLF4jFlpzJPiyobSzkfAB68bBhxVlpp0WLJLXaOrVlhxZAwuh 3wpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105366; x=1734710166; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hANs0KTrn4AbTh3ILmowq5bBaAQVBAkU7MnXkJd6Hic=; b=DOpXXLb3dsgK27/Azemnrh/EfcpbBAZ+dOW5lE+uqzd2Dt/f2KSr2LrEBdlH5iFZUS F8RyydGIqqFz0+z29yLFBRj9DoKC+OAN0kBK7Ajg+dPWvzf626oknacf7oCGur4BHgcn S+k7T+dQK6A1z/szKU55MdZ5vkk5GTMGwFd0l9Ted3BT+LYu0Lr2iNwnMV0KE4SMTjyO 3Gz5onGTzDM4Qs6kIkg4XSQFtDUz3ZxHgI0Z4BTtS3ms+80eLTkoYjCZhoRl3vIkgLEe gjMCjA5FgpXD/3VMk514JgA5DyhTlm+FVh+U+bumOB2V/sY7DgUexHilJf4uOmt2GgJz RACA== X-Forwarded-Encrypted: i=1; AJvYcCWZB8bpwruwSV1479tTSZGhztYxMVjRVZCHisgp3LsClme4j6wlKNnpZ2bKo6SDHLWUZX+EPsIf4e7z4qnK@vger.kernel.org X-Gm-Message-State: AOJu0YxK/z3d/eP1TQXkXXJiUqeaOrE20S8gSVZv0QitgOrTc0BLzgg3 NMpufP2StBDPU8PDfVG/U7wStOdSk+EPvQb5sz0HDg/fUgp/KRstfuzbOlmJnDuOiuRgxA3qley 3 X-Gm-Gg: ASbGncvyEWHRhfGe40Vr5f1yMUS8wV718tupdi7/W+sKDWSuQTvfHPK8TwWwsnaoDFS 4r40mubQIvBrW07XWh1nUnkmXfuL66EXVBz5vchLSWtPS8Iw52OCgEpmGoZKd9ubbqfsUI05+FS plrjpDh0FTN6BNIzXkTMq9pgaIEu5nPnNTh0Evzw0swCWF2S4Yt0XSCkHBdQDz/20yiooibPUn6 fjB1uCm1ua/L4NT+wdcGME24zDgSYkbt/jWpz4BU5sYffQJpCtuWFykwdub X-Google-Smtp-Source: AGHT+IFVLcWwM0MIr7oz/Yvc9CHKROeMGY95g0ljJ88B+/rBynMSTGl1ysoTz/aE+AP7fUiTK21pEQ== X-Received: by 2002:a05:6e02:1a07:b0:3a7:8040:595b with SMTP id e9e14a558f8ab-3aff50b64a0mr26251175ab.9.1734105366472; Fri, 13 Dec 2024 07:56:06 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:05 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe , Christoph Hellwig Subject: [PATCH 02/11] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Fri, 13 Dec 2024 08:55:16 -0700 Message-ID: <20241213155557.105419-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Reviewed-by: Christoph Hellwig Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 8b29323b15d7..220dc7c6e12f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2527,7 +2527,6 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; - struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2542,12 +2541,13 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); From patchwork Fri Dec 13 15:55:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907388 Received: from mail-il1-f179.google.com (mail-il1-f179.google.com [209.85.166.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85F091E378C for ; Fri, 13 Dec 2024 15:56:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105370; cv=none; b=lD8A1dzmIiz8xdPdJlDMxm5M5UWVLMZXv+LWS+7Hzy6X+mc8TTwuMq+fdYW/klD5mp84oDy+OqdYV6mOKojAzrHMkUgelkZ07nbS4vtMKh2xGEhIrbVEkyARaf0GZBoPU10YZbpykAHVUqakL5uioBhZPyP2eSEtNYcpJ4E0sZY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105370; c=relaxed/simple; bh=VkQhoZOrHyZykna0hJTs0oX+MEpAmUB5ySWSsyus/GI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TU6BI/OYE8Gf2K12zci94HMSqrwcx1KN0eCFAlzywynPe8sgx5P26TCHWKBHTMgHSdNLYKN4JQE9ngZITs1kYMc/Rr/wHOWnNQJ76+fyDxqlILnmm10S0cP3HjW4QdNVRFLM9BKg4C1j/CfuCHNN4J+LMsG3kqNY8iN1f+zN0OE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=gn/n6u5x; arc=none smtp.client-ip=209.85.166.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="gn/n6u5x" Received: by mail-il1-f179.google.com with SMTP id e9e14a558f8ab-3a7d7c1b190so6594985ab.3 for ; Fri, 13 Dec 2024 07:56:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105367; x=1734710167; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BpcNvbMq6OW5fN6dnGRgZf80nNArt3mFYuYvhVaSlK4=; b=gn/n6u5x+8hDMuHR9n/5QfPXMLgtc2zKUHy8C+LfAqaUiOzmGwTZOzXpER2DEDaeCp aCAv0YyN8aZKftlj6NdlPXwUmC1l+aa8oYNf+33hv3rMpHWk9kwA503Yt9fZc5iX3jXj dQLybLF63c9Dn8jA2HcluDYFbK6zH4FA/7484s0VSlYmd3YMq9pYKVyuYzFMK00haI+4 y8QHfpiUia/RXDAakqyqO2974gWkFeBOMDHuObf5MJuKoa9/gXHUxOFeVAFCqeySQR8y u5rgZrAp3Y9cIkD0chUiE628vi1g/Kqo1BzRRoI3EBGmWvBPOOL6zOQfq4b4RdzMU44S 6uqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105367; x=1734710167; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BpcNvbMq6OW5fN6dnGRgZf80nNArt3mFYuYvhVaSlK4=; b=m6z2tyBJWFMOJUgVo6jJRfh4eCE3GQ8N/RN9fHoMLFX/A4Ll6KAivQ0yRJXxMNG4IL wQK+cSRJFt8KNXPd5WeNEwKKBy8+H4IVNL2SuyJ7vToFjs4cXYsZAOqG9xUmyQd82TAr OtEXQvE5NFofxQWatcw+4xS0F7D2YM1/4Nfjt8TcBDn1I09BZ6jq5HnxrT+PSSC/xMHk blMD+tmJhN4LMbSO1q76XRtvHI3CtSbcDV+5hYFm4VWEVvufGdnoPGY033M/SvOJVEB3 pE/Cb0k4CCqaBGokj1lzM9sImI3KbDk+zfRbHT0Vc+1hLpXQzB990ewI0C/muDsuclZv JXcQ== X-Forwarded-Encrypted: i=1; AJvYcCUTyQD9T50iiUsqyvcuLEP/qYhqZr34TC5ZJHj4s1tWKxIHJAK01AcWG/ZhEK9J3G8W2J5kVkyY4Cfd8js5@vger.kernel.org X-Gm-Message-State: AOJu0YxWskG3S/UpAodiUjGuTXsdEnMW1lrEYHOPFCDbmXCjaZYctESl UdWEsc4b8rLeXABhMI8ETf76p+ye7LBdRC8zyNFhZpJtiItw1eZt1RGRb9XjLmI= X-Gm-Gg: ASbGncvT9/I3QmHjGYXD3lnnxhCB4Jncx7rgO3tl2zzih0rD8clVSV2DcADSMuXx0Ez jdzuf9DO4YQSNCH3hCHPWasep2qLCvx+ks5GMh9MMxAhRMpZqEacxx/RTg/WM+7/ecxjLpmwo/5 GmweIOprbDnAJmmX7iUkzAiJCupK72tx4b1ixn4WEY+vFp1XdrOMkcalKBty9ox1C91MP0GfhGH 2fgl49xxgwZobaZ20QxAgVJAlIecOh0Hh9XYzI+b+kFr2KtMlvBTFRMlgMM X-Google-Smtp-Source: AGHT+IFWkhf2olmYWP9+u8WH9KgVfslThGYwEFiU2/KIyOObE8orPa/AadsJR4yt5f2xYXvlvdRoMg== X-Received: by 2002:a05:6e02:1a07:b0:3a7:e4c7:ad18 with SMTP id e9e14a558f8ab-3aff80107a2mr31899055ab.18.1734105367708; Fri, 13 Dec 2024 07:56:07 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:07 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 03/11] mm/readahead: add folio allocation helper Date: Fri, 13 Dec 2024 08:55:17 -0700 Message-ID: <20241213155557.105419-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index ea650b8b02fb..8a62ad4106ff 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -265,8 +271,8 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, continue; } - folio = filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio = ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; @@ -436,7 +442,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio = filemap_alloc_folio(gfp, order); + struct folio *folio = ractl_alloc_folio(ractl, gfp, order); if (!folio) return -ENOMEM; @@ -750,7 +756,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; @@ -779,7 +785,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; From patchwork Fri Dec 13 15:55:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907389 Received: from mail-il1-f181.google.com (mail-il1-f181.google.com [209.85.166.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC9791E3DF4 for ; Fri, 13 Dec 2024 15:56:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105371; cv=none; b=G+JULgHIm1nEcGKsO7rjfXLIXS3V7CNiel0LsOrnMTB3fGcn0rsKoYQy1Ma/lc9p03dEKfU7+bj9kz3IuY9GQeeJL6/9INE9Um4pvRHHwl26dyBqfwcHCiX/GbP8BGLIQBjFlGfGqGu6YhQeE1Gt4oofk8K7XKRugBwAkSVmdJQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105371; c=relaxed/simple; bh=sPnR3qBnsWLGEr22pRK2TCMcNSO2RW9SlspmNOIT9ew=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ek+LzIUrAiT1jXOTns5m3HO0X1f4tAZ+srJGDLdEWwY86mW9JN4F2e0cun6tkLNefX2Ts/SjF+07RNTEfSe6786o+OHYAVWQ/xUCEF3VvZk8k8Ag8AUsVyn+xCyLM8lIzvjvPJRCaCemjnCDUxjgTYGm0/Zy2AjwCb75Sr321z0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=obyOaWIl; arc=none smtp.client-ip=209.85.166.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="obyOaWIl" Received: by mail-il1-f181.google.com with SMTP id e9e14a558f8ab-3a77980fe3aso6265535ab.0 for ; Fri, 13 Dec 2024 07:56:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105369; x=1734710169; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HOQaUrSMrbMaSJ8tKHsfpEHyiDws1YYfXq+2bH3/e/Y=; b=obyOaWIlLgEsnjXV7gMx/G3rChEhT3ViesKVO0mDuP1ASnoUTdHT/MOdQLx6tFeFCS 68sfgPX2jZBlKMS6C1TnsGBTyAlkwhdeL3L0rpJqF8gdynDHZGWrxPkH3Z0w+n2gyjuG AYUcKJFnFw4qGqcXUGxleGgzUSU9eaAtaubpt4iTHYuofBvbY9MU7xOPAkeV4kBcACny WpuiSxbjIxGn89SGOWcguKS7+2MN2HjdezPSoIzCp/88od/7d3eN0ImQtejzPHwvlb7B RHO9uDxi72CCUPgseMsdr+YW6Sqjrk22kbaCmHlG3X3OZBs91D1Bjd2tkQqWVvNqv/Py kpjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105369; x=1734710169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HOQaUrSMrbMaSJ8tKHsfpEHyiDws1YYfXq+2bH3/e/Y=; b=a7JaUuCLpOOJNEK9f9MuS9Fw6urcMu1qZ2RPaKaxQ8J7T5nEgfp8SYfP7NYw29eEty 2wLbZqf41AKPM0pyktlNoAGM+OSfpEWpFBOTwshDW9qynnj9uKQ6DoKcynreNa8tBTLi YJctRQ47HVLiFCiKpBSdnu661DzQnNSQ58NNcIIHEUBpbWqoS6sQp3K/6UwEIjzTyZwD P/flvzl8ptTgeqTpX78/QjWg064LkvHhAe+4ylaMiUhvmSSOdEs/HPQGjPWLG2jwVq+H yxqu30D2pplo0UA35NWpEe8fFGPnPyURsF0uao3v1P/Bj+15geuQyj0XYozFn2K6Hd0r MJvA== X-Forwarded-Encrypted: i=1; AJvYcCV5Kiif5rZdQ9tCZ8Rdi3O5XXviM0CrDj0AohAvLS7OBffSL1XqfNm7t0rxApkTbyyDhzA28RxmNtdlQx0o@vger.kernel.org X-Gm-Message-State: AOJu0Yz6kg9jNtsM6CsV29B4URyJkQV/XPuFo4kICcotLZK+plKorvnX oYkS73GUfDZawrcK0mCVf7tcvEh55GNcmFEgbXmIi5WAhATkjGag4ristLPJ4Lo= X-Gm-Gg: ASbGncuO+BTKdvts0I5PrlJTrmoOoDPAn22Rd+e5g0XMikMY9j9Wk7D8ppiN0nNW+Z2 +OOJeFonHqDvNfrdqb7V+5xkXX3rS/PLo0P3oe00u/rgtZRAVdz9lDRu6TDyifeS9i8JIJFbasH 0SQHB/chYEuEQvU63ol+2ZOwMMqim3C9jxW/+BSN1AV1uHvBP20SfET3+KCYJsuxVo+mtx+7Ovs 38nDBTKsTXBkfwHAfutrD2Nxxea0BsoT9n1AjBCoNtzlDCQ+H0gWzkxs5CY X-Google-Smtp-Source: AGHT+IFBVlz5yZz/k+skWWprsCIycLK8jgnRkqf8RE4HqF0RJQv2gkIr0yeXfbClfRWoooTGgqlEcg== X-Received: by 2002:a05:6e02:156f:b0:3a7:e8df:3fcb with SMTP id e9e14a558f8ab-3b02d788a63mr20845625ab.7.1734105369001; Fri, 13 Dec 2024 07:56:09 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:08 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 04/11] mm: add PG_dropbehind folio flag Date: Fri, 13 Dec 2024 08:55:18 -0700 Message-ID: <20241213155557.105419-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a folio flag that file IO can use to indicate that the cached IO being done should be dropped from the page cache upon completion. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cf46ac720802..16607f02abd0 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_dropbehind, /* drop pages on IO completion */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) +FOLIO_FLAG(dropbehind, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(dropbehind, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(dropbehind, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is not diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..3bc8656c8359 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(dropbehind) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ From patchwork Fri Dec 13 15:55:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907390 Received: from mail-io1-f44.google.com (mail-io1-f44.google.com [209.85.166.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DF7B1E47D9 for ; Fri, 13 Dec 2024 15:56:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105372; cv=none; b=N3YYNEmJGBou60++uw+vb+JIp38+hXIMsK7Ksl+GGtGHG4IuF3Yi0ZmmoY6cnle5WJDPcghctr8Z9WJmCKS/hsfo2OXOSm5hOzYpCWexrzSXA0M9uHuzBv2Ya7s5H28fJTanP/QJ0U/QoY0XDdEgo8AsIB9naZll+ijod9kkDGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105372; c=relaxed/simple; bh=SKl6hxJdy3uyJbhZpOOdAiYhB+0FB2yQzq5XqnnCoF8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KBEOr0Lo3w4tqdw8NjL9VL6O6K7TQQrHb4r5skCYyDZwGiZVj9g19r23iqIhhvud56wa8gVuUrmTKGKnboQILDcL2X2/OwaJFBNBVfpZB6C86aiZ1AR1TZxaW82ToWfy5o1IM+FNG+ER9h1tYeU3f5AoVyDicAb9Pq1RKulNVFo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=AAeHJeZd; arc=none smtp.client-ip=209.85.166.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="AAeHJeZd" Received: by mail-io1-f44.google.com with SMTP id ca18e2360f4ac-84435dbda4bso104648539f.0 for ; Fri, 13 Dec 2024 07:56:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105370; x=1734710170; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kz0Pdg9jqnEJMYLqH2tprv5bkemx7uetlhB6shaJiPY=; b=AAeHJeZdtfP/2H1iy+ldIoEDGS54o7//Dw0LyEhQdwB2FK9yFFGFcrabdhflnp0wSE r4/OU71VRA2b2GHywltcJnJlEL1ybczlTCu7MiRyPZL4gHKbl9uq8e/I+jYAtiAR0p8b /HSiDCDdIwWN2KJ1ZA7CVvimVvrQK8YBnpEeKzchSpnBKlVDSH4xXiwlMNSs+rj7CWQn /ZPx9n6tpXNmdHn+5S2uFX+cZJDFowx0rXuu50y3DH2XqOyofqO4v32vjRTUT1OuleQG PEMbCUTiTt15tdYbNozMlbAvfmBcw+52Y+IVA1bOp+woVNHYK3HQ5w9p/BDSOI1UnKxN QM/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105370; x=1734710170; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kz0Pdg9jqnEJMYLqH2tprv5bkemx7uetlhB6shaJiPY=; b=ZanIzqQ11t3Rk7HcpOKrA36zoTVlI72XFfzrhN9YStaTThta6pL647SauhwxOid8fY uiz2LBfS2AtruDoTTVIan+Ly1gmyJmCXjyjRJ3S3GrkHNzGueqklH2Jw9NsVAPqR2o73 O5q+XFCYRHrWdE/BI64BptDwcrAkgg15D6pkkdYU52PBvFJnx8OjRiDTvdp7Gbw0aM8k SWQVyUB5eZV7iRoRjZi7TsqPZQEZr5JevxhZSHv91oi97BAG3JLLb86gK+aoQJH1NUX3 +5j7daDjmZ12V3J2nbo20p30iDBQ/ZdFgn1vGVyXCcKXhRACGpvHGFa7CxqzTJE2QMKP eeUA== X-Forwarded-Encrypted: i=1; AJvYcCVLAcd269dBn3NwWVMR+KkotEp6IlgYWB+hG+5p3Zi1FgJ1F6mW/lh6hg3lkspH0NiCLcvW6sVKdnD9Bu7L@vger.kernel.org X-Gm-Message-State: AOJu0Yy+FqwSYfQPNKt6YYVdO/OS/3B3URYv35SLV4UmnqZimuXZxyA6 Ikj7mCDu9yLLta8ts09yO2B921kyLGEqNtr8s94xOVQCHpVT9gJrEBdaO2aq6Os= X-Gm-Gg: ASbGncvafu1Tispm5/bNgZt+fGvefMdGSHmhtXL99cPKweRDj9iDEoSSLUt8cy/nmOR 04QYIvpAT8ucNR62pwlzpn/nsXCAAVGbDzKlDdiEnokw0FXyYfeTEz+QiDirBtQRegX4O8o0c1a HK44ZRrTNwq53JMYyVVgavc4XcHepArtVplznQaKl7wCaRIKk+KB2QoljKK+z0xFXgfAc2D/Lbx sRzcDsTU3xU+kT4DW02fEg0hCWXkncAr3R0OzrwXvrxfgF0YI3tQ501zaHZ X-Google-Smtp-Source: AGHT+IHcUplw3Q6ECt5DnQxOZIMw6sSCxN/kZ+qm1VNkzK8N15/r675oe1NRLkEoEeFprb07mxKCyg== X-Received: by 2002:a92:d10c:0:b0:3a8:13d5:bd2c with SMTP id e9e14a558f8ab-3ae61f23bb3mr46583235ab.2.1734105370422; Fri, 13 Dec 2024 07:56:10 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:09 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 05/11] mm/readahead: add readahead_control->dropbehind member Date: Fri, 13 Dec 2024 08:55:19 -0700 Message-ID: <20241213155557.105419-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If ractl->dropbehind is set to true, then folios created are marked as dropbehind as well. Signed-off-by: Jens Axboe Reviewed-by: Kirill A. Shutemov --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index bcf0865a38ae..5da4b6d42fae 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1353,6 +1353,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool dropbehind; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 8a62ad4106ff..c0a6dc5d5686 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio = filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->dropbehind) + __folio_set_dropbehind(folio); + + return folio; } /** From patchwork Fri Dec 13 15:55:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907391 Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A836E1E6DDD for ; Fri, 13 Dec 2024 15:56:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105374; cv=none; b=kWyl87eTLkXu9SiqQRUPEXdVjHa4wa5UZxyfK2vFXi0E6gmaCBhwz25eG6T0PADuty7/z6v51MEwSOd2wXRJXZJMXuD8ELZiN6E+ihrQuVoiySlZcWTVteDhsVFtA6181EiRw5yl2T85lB+Qv/gznqNKn10FQyLEiUG7qBh5ukw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105374; c=relaxed/simple; bh=EZq/FArZwXgCBs4pZJDUXjfhlPafSJVT3vDdAzxjcuc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nk/OWhURQCZJncaQZiZy5n8XwQoNWGBOZNdzorLiq5YCmtHInOcKpW/JpwhBe+EAORiFWHxrN9lxW25nZVaWeufF89Y1ExnpDFtXvEIzhNJYzQenrn76Ib7lMbRFzaAR0ef/SYP78N6xl62WjU8IpY3qI0S8R1ZMbAF+BVuUvdw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=weU06r9Y; arc=none smtp.client-ip=209.85.166.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="weU06r9Y" Received: by mail-io1-f45.google.com with SMTP id ca18e2360f4ac-844d555491eso65543339f.0 for ; Fri, 13 Dec 2024 07:56:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105372; x=1734710172; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S6x6BO6+6Zb8b9h/4S7enLTX5qkkGgQROhVyQCWWxzU=; b=weU06r9YxvdS1WDcTqD0yw/mJvNIBBFPFHoPtJ/ANqgU/1+YICPLCz/p5ZaoKtruH/ wcQYTGUmqBhkZM0QqifHRu75JqEB8tRvlMYkS5D5yBzooFTvJSPUF27t4MaPau+QkGg4 /qyTert9WI/G7MGDy+CYNgtFiGFtopXWxGhOXvuohTQPNjrdIF+Nhw5pCmxo/tnf8ggT i4TNbQRfY1laF49TLG486wRYWdhfAHccLPai/Nql99tzfax0WISR2/70Kx29+FRQwHbr jS8ALKfGQ8HEsTvfuSIryOM2yYQhCpdLJYDGYNTv0NlG0ONSwxfkg3wYewuVRQpwFVis C37g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105372; x=1734710172; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S6x6BO6+6Zb8b9h/4S7enLTX5qkkGgQROhVyQCWWxzU=; b=dCfP7qGqQc0S7HbV+A6OXqUQ3wVWMdhQp1IGomjdL2iQiVzKZt4tAeiQwKbmExcXfI jHwnzLmB5Q8iDza2KY+81Em1+r31ELk9hRkTkVR13FEzJLEa+t2ffSKX4G2qqF8IX1lB BRI3CB+5DE0KZv9OhUqdGRgddGPp0yz/VDlDyxtDgepRzBfxIN4YSU9xPEUbVf8bGEz9 IEEVzNO6y7yDjokt/Vpu17ZOGVdUI7B6HScnBC/JqCsdKsUGwDSf6C/rCA/FfpnlK5ln yquLFH9oi3J2BjpuQj7e6vMK8wiiB6bLoJBy95TlkBvC85OYiWj5tQZaymHG+P0d2v0Q u/4A== X-Forwarded-Encrypted: i=1; AJvYcCVa+I8rukFXfEwwA+49wAN2T2/oFBRog/r+QFPxDI1U1mhAN+NL8E/W9/utY1VrFhpB/Xj7loFyXt8o6GeI@vger.kernel.org X-Gm-Message-State: AOJu0Ywj4UjSzCCdaA1GiDWNW20mHzPw53csHiHePI+qSM8+hPViD7iX rYtjuizj8JasuHSydIiSwQawlyMq1ItQjOyIRSFxpPs16zZfMrmZ33//Hc6MCwg= X-Gm-Gg: ASbGnctAJv/NJ6lfuNveGkT4uGlPns953BBc3Xn91HDBqqazedFfXVanKxluM9SXTu6 65ZUuaHqmlG3q6LQLAmJyt8M7IXY0Pd4eJdotiambEdZXrFv2hLWWSRKvDi0CasCu0iUrkcGDza eqo1lP30EFPK7QECnDTNb42PAd1UdNEhqFhEFDjP1dREqxjcaV0u9cfOaKKIWrYOvqdvnX/qfdJ GgGKEhX0O5BqSm4/5y6BzG1oA/Hyl++fPmF4KK4w79B+AHGaqUNl+ixJ3L5 X-Google-Smtp-Source: AGHT+IEynku8b05ugapn+xW5MO4tP+/urn6SKqialSRpaetMXPe/mw9YxvkBEr2Hl0/nNihyBVcVWg== X-Received: by 2002:a05:6e02:164e:b0:3a7:7ded:521b with SMTP id e9e14a558f8ab-3aff800f100mr28298385ab.17.1734105371849; Fri, 13 Dec 2024 07:56:11 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:11 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 06/11] mm/truncate: add folio_unmap_invalidate() helper Date: Fri, 13 Dec 2024 08:55:20 -0700 Message-ID: <20241213155557.105419-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a folio_unmap_invalidate() helper, which unmaps and invalidates a given folio. The caller must already have locked the folio. Embed the old invalidate_complete_folio2() helper in there as well, as nobody else calls it. Use this new helper in invalidate_inode_pages2_range(), rather than duplicate the code there. In preparation for using this elsewhere as well, have it take a gfp_t mask rather than assume GFP_KERNEL is the right choice. This bubbles back to invalidate_complete_folio2() as well. Signed-off-by: Jens Axboe --- mm/internal.h | 2 ++ mm/truncate.c | 54 ++++++++++++++++++++++++++------------------------- 2 files changed, 30 insertions(+), 26 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index cb8d8e8e3ffa..ed3c3690eb03 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -392,6 +392,8 @@ void unmap_page_range(struct mmu_gather *tlb, struct vm_area_struct *vma, unsigned long addr, unsigned long end, struct zap_details *details); +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp); void page_cache_ra_order(struct readahead_control *, struct file_ra_state *, unsigned int order); diff --git a/mm/truncate.c b/mm/truncate.c index 7c304d2f0052..7b6f4399319f 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -525,6 +525,15 @@ unsigned long invalidate_mapping_pages(struct address_space *mapping, } EXPORT_SYMBOL(invalidate_mapping_pages); +static int folio_launder(struct address_space *mapping, struct folio *folio) +{ + if (!folio_test_dirty(folio)) + return 0; + if (folio->mapping != mapping || mapping->a_ops->launder_folio == NULL) + return 0; + return mapping->a_ops->launder_folio(folio); +} + /* * This is like mapping_evict_folio(), except it ignores the folio's * refcount. We do this because invalidate_inode_pages2() needs stronger @@ -532,14 +541,26 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * shrink_folio_list() has a temp ref on them, or because they're transiently * sitting in the folio_add_lru() caches. */ -static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp) { - if (folio->mapping != mapping) - return 0; + int ret; + + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (folio_test_dirty(folio)) return 0; + if (folio_mapped(folio)) + unmap_mapping_folio(folio); + BUG_ON(folio_mapped(folio)); + + ret = folio_launder(mapping, folio); + if (ret) + return ret; + if (folio->mapping != mapping) + return -EBUSY; + if (!filemap_release_folio(folio, gfp)) + return -EBUSY; spin_lock(&mapping->host->i_lock); xa_lock_irq(&mapping->i_pages); @@ -558,16 +579,7 @@ static int invalidate_complete_folio2(struct address_space *mapping, failed: xa_unlock_irq(&mapping->i_pages); spin_unlock(&mapping->host->i_lock); - return 0; -} - -static int folio_launder(struct address_space *mapping, struct folio *folio) -{ - if (!folio_test_dirty(folio)) - return 0; - if (folio->mapping != mapping || mapping->a_ops->launder_folio == NULL) - return 0; - return mapping->a_ops->launder_folio(folio); + return -EBUSY; } /** @@ -629,18 +641,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, folio_unlock(folio); continue; } - VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); - - if (folio_mapped(folio)) - unmap_mapping_folio(folio); - BUG_ON(folio_mapped(folio)); - - ret2 = folio_launder(mapping, folio); - if (ret2 == 0) { - if (!invalidate_complete_folio2(mapping, folio)) - ret2 = -EBUSY; - } + ret2 = folio_unmap_invalidate(mapping, folio, GFP_KERNEL); if (ret2 < 0) ret = ret2; folio_unlock(folio); From patchwork Fri Dec 13 15:55:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907392 Received: from mail-io1-f51.google.com (mail-io1-f51.google.com [209.85.166.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 074641EBA08 for ; Fri, 13 Dec 2024 15:56:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105376; cv=none; b=mtjc6Js06OFajuQnBL+piNRjfjky7oLVSNLrtT6HnbxPiDGjsZlW7ybCZB/6bWa4JKWycx0f7cTVxN/y96Fg2rwymvumMWlwYZpVBltBNtMsl98WXziJjWhwhQBMk5D+/QM9dScR5gJkOEZE+bw6+XGY38gD+UlnZSpC0x06FyE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105376; c=relaxed/simple; bh=2SEcDojzAqUnPgfjf8E9nRmr9dou/inG3/DhaNUqq8I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AyYaLxNdaxPOzo+o+tMQCj+e2soYjfKEBqnNWuDl+Yj14bIeCxmDA04gKrYbk9PkDyW/It/cu6+VbpWIoyfdyR0owgF/inHUXzGPEcrRazrqoO9FE1Oe1JRYty2YdMa4kvx8+m5yUoJ5FYc//4+aUF0nyai8WvuA41gij/728k8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=01Hs9enO; arc=none smtp.client-ip=209.85.166.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="01Hs9enO" Received: by mail-io1-f51.google.com with SMTP id ca18e2360f4ac-844e9b83aaaso65283739f.3 for ; Fri, 13 Dec 2024 07:56:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105373; x=1734710173; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=01Hs9enOnkwKRmuqv8OPi5cm5KXxcKFhJlkctvjFgT6DFRRQIS/rAYbdV/lvyTW3Lw sbiSSuGpUO1nmFNR1hLh0BtKs7xfKfvEiwQpiVm7nVqSGJ/hT/5+Plb7PGC/Bt2vI4WY QAJWdCaYyEo26sZrPOlYErQq+dlX5WObrlqnCgBy06zFkj/9Bc+Df/BCuPbFkMe2/Yj/ tF5zSPUbf59oWTKv+YkkQaZ4RSoBu4MFQv2/IPc8VTUa46726BfPG2gwdtYP+YDDWYi4 DPcJQlGxRyu15Gt2gSwceQ80X6hY6iH355vg8dJhHAmEwxsA7CAsVybF5RLXrMEzEiAj DV7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105373; x=1734710173; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4w74VD3kuQguRINknUSjUP4VNHVLoGClDPYL5LdL0oM=; b=LTBzLNSksJ3ECU5lUJdR0uZp7OTOgT3+2WYWfspfWl5jmn+z9Zq8ekjWMNSbbIYYrw 9LnDeccGskuyI1CFW3IE6IX6gA9FjJiFQFzBJdjnmM/Dq0vinZBBnW0L6/YnMda/W43V hhBl/aoKPmxrYaSxiRWm5TUkTU1kXCbg4ouDadMFg5eV0Hshj8KEN7t64UEw0s6RXmQC uMmIA8IervVhb8fMAcSO6EMRXzOZnCaIUo9KjMk7XTLxiDxad1F0GmTMlTt+t4N1Ew2q h8uOSNTcduK9/TPirxEOiswnt1G76qsZBLIX7o0hnBEFIIinj9Nsx1ogjxl+pHtJ8ceK rQhg== X-Forwarded-Encrypted: i=1; AJvYcCUzlWr1M5kZitp04TMQNjvXg+pNy/X8hncCiVBIFa1cq1IP8ATuf1QvLPt+CL1RuREkgar77QKEZnXdAiIm@vger.kernel.org X-Gm-Message-State: AOJu0YxEEGxAz+P9EhSz3PUT3cozLrczGKdxoOWwJMdXpdOjubCEGewU mo0tdAHyj/qgIQ1/fCDInwdIvof8GsSwb3zR3CiTfFFg4bejpfyTC0LEsmHQjFM= X-Gm-Gg: ASbGnctmAL1J+li/ODYQehH2TGSrIua4eRuuVJJB7xAig1rsgF/c7xYTpckwUHEsuvv 4Zz+rNqhWVxRrjpbK60ZGv6zFE5HWv0N+fwd5nrXruL88HXQcNA1WtEhN0E26q233n+BLdU6/Za dhVd0x3H5lFcWZtJEIm4/n7FbSFx5m5WXgRT8VeTmNJIUDfyKe6BgiK2opgz1fWE0WEqzluGERo TN+C66+VtSKUtL3QCIPM0wmAJiXnPRUVBPF7Okdy6IaEefPuNvFwbP5cizg X-Google-Smtp-Source: AGHT+IHLruDe6Zg9uYRMCFFj0re5LQoOZLn3MM3DvSbmN1NLcJuI0/2En87svfvn51ITwB5/NqHQ+A== X-Received: by 2002:a05:6e02:3042:b0:3a7:8720:9e9f with SMTP id e9e14a558f8ab-3aff4619d10mr35119105ab.2.1734105373195; Fri, 13 Dec 2024 07:56:13 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:12 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 07/11] fs: add RWF_DONTCACHE iocb and FOP_DONTCACHE file_operations flag Date: Fri, 13 Dec 2024 08:55:21 -0700 Message-ID: <20241213155557.105419-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If a file system supports uncached buffered IO, it may set FOP_DONTCACHE and enable support for RWF_DONTCACHE. If RWF_DONTCACHE is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 14 +++++++++++++- include/uapi/linux/fs.h | 6 +++++- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 7e29433c5ecc..6a838b5479a6 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -322,6 +322,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_DONTCACHE (__force int) RWF_DONTCACHE /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -356,7 +357,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_DONTCACHE, "DONTCACHE" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ @@ -2127,6 +2129,8 @@ struct file_operations { #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) /* Supports asynchronous lock callbacks */ #define FOP_ASYNC_LOCK ((__force fop_flags_t)(1 << 6)) +/* File system supports uncached read/write buffered IO */ +#define FOP_DONTCACHE ((__force fop_flags_t)(1 << 7)) /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3614,6 +3618,14 @@ static inline int kiocb_set_rw_flags(struct kiocb *ki, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_DONTCACHE) { + /* file system must support it */ + if (!(ki->ki_filp->f_op->fop_flags & FOP_DONTCACHE)) + return -EOPNOTSUPP; + /* DAX mappings not supported */ + if (IS_DAX(ki->ki_filp->f_mapping->host)) + return -EOPNOTSUPP; + } kiocb_flags |= (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |= IOCB_DSYNC; diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..56a4f93a08f4 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_DONTCACHE ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_DONTCACHE) #define PROCFS_IOCTL_MAGIC 'f' From patchwork Fri Dec 13 15:55:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907393 Received: from mail-il1-f179.google.com (mail-il1-f179.google.com [209.85.166.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57C711E3DFE for ; Fri, 13 Dec 2024 15:56:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105377; cv=none; b=R52eCfEcdmNOAZJgUhpjLSrTWn3b7Q0Q2tEaPZJgMB1LtLacSVGz8tLaMGt4CiqaxcUNtmcH/KWiuh7XahLB+9ebhfKPFEpIgMOq3KGH2+nVwSMm+lJl5fX8f73XHtBpf30X1ohe3FfvXgQzctpBuqjJYhiOD4XJMu4e/1xqCS4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105377; c=relaxed/simple; bh=vDmNnPIlxbQFCnastmzOm8F+RELQsnyCtSC4KgXE2I0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bHYlyLLo4eL/DxMztcmLH/VEFrV9mjFMKy+gCxttgBE/8rGjmk/2Q1ccUHGfFs6lTQAMz/HXO4fJAPUUUvUsx7r3WONCNYTiXvSlf73wMM0bbN/fgfmn0oYvhtnrivOpc7+9MGSiLpcurJvAs+NQyj7UidgBclEJlxZOWEs4AIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=w4XdXwHr; arc=none smtp.client-ip=209.85.166.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="w4XdXwHr" Received: by mail-il1-f179.google.com with SMTP id e9e14a558f8ab-3a7d7db4d89so5835235ab.1 for ; Fri, 13 Dec 2024 07:56:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105374; x=1734710174; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IP/eZVPa/Uul8epuK9rEl1xWlFc7aGT1EOVSfKN8LMY=; b=w4XdXwHrYVwFTvJ/MqoLN+c2dm61xKQFuUZKIwd9uV037G2HBS2LFfxCmr03/Ijc/j Nk7Pg8xdZyUXbbdukjEFXskzc70Vt6AyrAa2OICUAUeoNLIc38lBfL7dcl0M6bdLRTB5 vgHreqf2XZ/BjsBZ7VEi6YH3OohKGNo4rUEp4ommh6ZZwRLTkwhspRKxquwO4vqbZot1 ZtAeAxZTV2ChkbXCzL/LZ/iFGR/BtBFpPrBstn0Z618+xcvsV4I0d2L3mOFswtZO6Vib omrWpihLU5nNR74wLtz6b7r9AqP8zNMqgvgP7UERlJtQv1582/d5JTge3yGOmWUHfFoT iX4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105374; x=1734710174; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IP/eZVPa/Uul8epuK9rEl1xWlFc7aGT1EOVSfKN8LMY=; b=wSiDiOW9kQ0zlxaYVooi2ucH/R+PVVqCwIKOC2j+ge4TxuRn3+ClRbJk2FkgXlsN8U xnRPMxyfVvI/QfucXgzXPXm6ZhuKzLJXm8BHEcGtuY33WT8PpMek33RJz7dOixE8zf4A 6BnMOWClB7keNaZDtslOejfsQUisWoPjHsE8nTUG93OaNXpXatDTtbbmHW2B5+j7HWvx w1QYOeS8fV5FMHC63Jj1iMiuAQMzzgndVb/IHC1LcQ/a8ImVqyfSiSfgkJjI5qSqmmUP yUfAn+ff8qqs9BuYf0iUAmMD3O3peewkEw53J37D6YR1YvCX9q2o/Y+D5NXzvcactXZH gPbg== X-Forwarded-Encrypted: i=1; AJvYcCV0X0WVkH14knJWborjg2ZE8L6g0h8NHxOw/mLOOV0/BDrWfBwjT/9m0VNJwSivmVzDZI9qJIEHKXna9rzu@vger.kernel.org X-Gm-Message-State: AOJu0YyvZndbvzD7sXBn0A4CZH4nVJIJ1WUo7qJqMSzJgiGHlap1Wuxq oBqL5T2MRqCSU1+d0CNIkzzFZxSPwyu0UL3KRHHt6wRrWquey+DpeZvi1cpen3A= X-Gm-Gg: ASbGncvYL8fbWp2eDrzqmyU5RkAzJ4SuKyAg2gfmUwx68FA361/LXOLdj+cDXQDdCcW AvetMNj6SE0jjbAps8WnEHTYis+MG2MmW/8ZevYJQi8fA0m434WkESauqJBi0ewj4k7gBj0efp7 S80tUxjAGt7XWJr6B1CcfVPDv1RBCh/Mo+gmYi9NKnhnyWEnSh8gGcpMzP9MataB9NKMmZNMhS6 XB6AsgV7mlAZbRoWSeVQFJMsIYBtIoA19dB3urP0KqeJ2EEt+00BV0zLigH X-Google-Smtp-Source: AGHT+IH9sg/APOeVcvmxNi3lc/CaFUy261G7dVBoU1ZPhJ36GVox2hlXSW2u68WXUBo94t1FWfRBpA== X-Received: by 2002:a05:6e02:168d:b0:3a7:c2ea:1095 with SMTP id e9e14a558f8ab-3aff4615779mr38827345ab.1.1734105374532; Fri, 13 Dec 2024 07:56:14 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:13 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 08/11] mm/filemap: add read support for RWF_DONTCACHE Date: Fri, 13 Dec 2024 08:55:22 -0700 Message-ID: <20241213155557.105419-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add RWF_DONTCACHE as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousin of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_DONTCACHE set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++-- mm/swap.c | 2 ++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 220dc7c6e12f..77290ac205dc 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2473,6 +2473,8 @@ static int filemap_create_folio(struct kiocb *iocb, struct folio_batch *fbatch) folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_DONTCACHE) + __folio_set_dropbehind(folio); /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2518,6 +2520,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind = 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2547,6 +2551,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); + if (iocb->ki_flags & IOCB_DONTCACHE) + ractl.dropbehind = 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2594,6 +2600,20 @@ static inline bool pos_same_folio(loff_t pos1, loff_t pos2, struct folio *folio) return (pos1 >> shift == pos2 >> shift); } +static void filemap_uncached_read(struct address_space *mapping, + struct folio *folio) +{ + if (!folio_test_dropbehind(folio)) + return; + if (folio_test_writeback(folio) || folio_test_dirty(folio)) + return; + if (folio_trylock(folio)) { + if (folio_test_clear_dropbehind(folio)) + folio_unmap_invalidate(mapping, folio, 0); + folio_unlock(folio); + } +} + /** * filemap_read - Read data from the page cache. * @iocb: The iocb to read. @@ -2707,8 +2727,12 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, } } put_folios: - for (i = 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + + filemap_uncached_read(mapping, folio); + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); diff --git a/mm/swap.c b/mm/swap.c index 10decd9dffa1..ba02bd5ba145 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -427,6 +427,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_dropbehind(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; From patchwork Fri Dec 13 15:55:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907394 Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91A081EE00F for ; Fri, 13 Dec 2024 15:56:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105378; cv=none; b=g+zmbhz/gOqi7EHhl4EulK6VpSyAYlQX9yEiZ2DMepgQLUF4bNNLKfdvBLR7zD/tFLj2yWu5rgA98N/4WM4UdGlghl+5g7E0Zx5pq3hqh4xJDH2QlgvY8RM/Uj1JmSYJRcDkpnPJWCR8RGtkl0Ru6E2rbWMYJbQ3wjnNVWaeiTo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105378; c=relaxed/simple; bh=ZeMsDsphln50o1XFtLmHIP6lHmPNtmzQzlZFWxh1ZaQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cbS5wY7hBKZQ5/RCFnkJ+xjAs2BEmGOpGdu7zMehtCDoQRzP8L6/tIRowc/KwLgMcuUwfQ7ip58Bs0ezpTtfMj4iXZLBSCVWRREI3CBX/HN1aMtVOlqWg3sL2s+NF1FSGBu7u5mnbBG5tgYKXJYA8C7Lbdf8bBe1OraOOVg1i4s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=VY3watQ2; arc=none smtp.client-ip=209.85.166.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="VY3watQ2" Received: by mail-il1-f170.google.com with SMTP id e9e14a558f8ab-3a9cb8460f7so14039715ab.2 for ; Fri, 13 Dec 2024 07:56:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105376; x=1734710176; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eL7TH1+psm5CeVquxo+D9Ecvv3+RlpadAVoxegN1mFk=; b=VY3watQ2NR6jGnPZ6XQYSClGFKgxvsCoMHMZGD/dDEHNkXkum2+pHmho6aFmqJhf6h ehfSPV74UjBpmV85H8x0IbLd8VJCG2oLkYJctyquH7XYpePbvIZT5Vcyf6Rkm8Uw4LAH i25IJI0J1mFgRzMLo8fVstBkRzjnfxZCzu2PYl0pyTsNIBYlhMYqRwk1djlTCgWzZ6rk sFyMW/rBmnfw0yeawjS3wohXuRO/BKMJrRQqNaO+I0/ksVNiK9MKc/E4ofzGy5oAiuwO tHaCZKxU0TbTHUwDSt8rceIU2ttCHKamjds4J9/M3DRYMIlOljSUYiaUADvvqBbJMUZy sySA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105376; x=1734710176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eL7TH1+psm5CeVquxo+D9Ecvv3+RlpadAVoxegN1mFk=; b=kDm3KkeCqeLTEv3NUSvq5to9fNiykKEdcl1qWEolJxIAv6TWkmFNYGDv3zKjTcKKNc nwH0MSoRu46d5DMbz5WBVw8HXz2xpYeMgQsC4lfTVGO2BqmPO75KFYmsWh1l+4MUP3BC 5+GN053mYyMBYb+HqhxhJ94AcLO49m+31tHYebhsQuE7YAVkhsKazg81RBIYAFCREYbI bEnfXFobgajqA4uBjtHA+RoeTIg4lhaS5FBIujRYyjAPRdH1ot7I/e3I0VipbECfpJf4 h/X6yRh3bly1hseLACBPaNjl1FsjthN+iX46u6FdcZ9XImwxbCAO4AwcxGHv0p3K9RA1 X/Xg== X-Forwarded-Encrypted: i=1; AJvYcCUASonfxQVyQIFcAo0alOsw5yPDNTK6YweBVQscsB1v7sQHlWJqrYS5ry/IuAAR7ezlxc0EpYuJ5idlLvlE@vger.kernel.org X-Gm-Message-State: AOJu0Yxo5tz0jhT9i98VYlZDG7Q2qEwNlsl+63kbGauIlhz1Msg7eQ+K digvLv5WCQM+qfloJvIgZmuP3oX9zUrKFPqByopJK+iyQgjiST4DDkDT2XnatJU= X-Gm-Gg: ASbGnctkOOqfrkv+uzJYTdfuQj7tdEXnq352CZUJExelu0J4WxUR8Y27JXUHAEUzsFU mM+pBytZibGxRhJPXqwaamt6Og0F1ZnITSnF+S1BcjosNzXgM0w1yBTeX88z5bHbGGz48ZIX1UW Tn6jVh9hXGmooiC7LoMXwxygIOL8p+ZAJuSL6uoLxnYTSN2TX8/heiGEdm6kgRpKpJjawk14es3 TXcxeXkOyc0YagJlgZtAC4JHuI1bWNI4v0tAPprt7Iiwt2U/TJsCa037Tl6 X-Google-Smtp-Source: AGHT+IG5PcGGwL+OAF8BWXN0MfzrquLg0U95m3qvCvU2aqYiecyxUUHKeEBe+q/HNr+0PzMX/oeVbw== X-Received: by 2002:a05:6e02:12e2:b0:3a6:c98d:86bc with SMTP id e9e14a558f8ab-3aff4619e11mr40735545ab.1.1734105375832; Fri, 13 Dec 2024 07:56:15 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:15 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 09/11] mm/filemap: drop streaming/uncached pages when writeback completes Date: Fri, 13 Dec 2024 08:55:23 -0700 Message-ID: <20241213155557.105419-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If the folio is marked as streaming, drop pages when writeback completes. Intended to be used with RWF_DONTCACHE, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 77290ac205dc..ec087bab1c97 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1599,6 +1599,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as dropbehind, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset dropbehind for that case and latter completions should invalidate. + */ +static void folio_end_dropbehind(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_DONTCACHE writeback, + * but can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1609,6 +1630,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_dropbehind = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1630,9 +1653,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (!folio_test_dirty(folio)) + folio_dropbehind = folio_test_clear_dropbehind(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_dropbehind) + folio_end_dropbehind(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); From patchwork Fri Dec 13 15:55:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907395 Received: from mail-io1-f48.google.com (mail-io1-f48.google.com [209.85.166.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20E341EF081 for ; Fri, 13 Dec 2024 15:56:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105379; cv=none; b=BY1ajBpbS/Ab63aSSstDH2hNtTLKnkamW0S9QbPpcYF/WY5rHYOuO58uOc7yP/GkgtAAAwZrC1mgIorE8zdCCdPBvi22AJ9g2xD4AJZ2Vv341S+vWbYpvzIRSghO2MsVH9gCZJalaCuU040dEPLkHLGPQ/dljG8ONuHdXD2DUF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105379; c=relaxed/simple; bh=5nnCX/ugaux253bLZRpzYYnFsErlHtBtfq4h5ryDOj8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H0hp9ak9WnBlWM5O/JSmem7UvXw43xEiKeS4+xeaycXckjVwGb5DIAmd7B9cvIbrGQKlowxAO40ROMFT6pKiT61EUdVLBuQft5HKAOn4wPIJdj3pYXpOXA61Xxgr7+3ObAHuWor12HJAS5w38YP0dXV/oMvQr6k0UZJ3DiX5UpY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=xBsqQG4W; arc=none smtp.client-ip=209.85.166.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="xBsqQG4W" Received: by mail-io1-f48.google.com with SMTP id ca18e2360f4ac-844d555491eso65546039f.0 for ; Fri, 13 Dec 2024 07:56:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105377; x=1734710177; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CJvpEUV5aCutO0n1vzbjd3IdvpDLlQuRWwd0hw2DTJ8=; b=xBsqQG4WWzJ40QYjSZYrM1RDXg+bzMi0GiUO4zqLy4cZRfW+o08/20ZGDrQY2xx9tf 2fAeWjLv/apNFx53ZsgOSVYfiGEZLt5NFsXDnQVtMBkp7FdI15rVO/VBgloMjPVb9Y8e F9aulmRj6oBbflzIRYO0Xo+jwlNYxQx0IoMqkyCRXZs+d7g4EkS7Ag/x7ZCSHTSHp6aL qHHHHQgJX0+3U21vc7pIVSiochjtnyM4woNvbvqAQasC52GPpCW4bPnXN9/GiwULZNpJ p7uaYm3Is+Kx952PvSZDTwAHZR/+q9OASOfMBkVMItN4iJJBOkhYumDdsl+m72lJ0XDD Rakw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105377; x=1734710177; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CJvpEUV5aCutO0n1vzbjd3IdvpDLlQuRWwd0hw2DTJ8=; b=mg2Ul2toWZG4botpGN8Lhvcp0oSOokmHrQ3ZzUVejCxIs47cHaqSV9m80d9bXG2ExD oFfBjKeIkHRJLy2zppfhllX+ix8XVQKnNrvmAM8SjuTFj5bNvloNFYdGI1YWspjCKkJB 3pIgnazRBh49MwJv2Am0Fz/ruzhjgNo52oJbKm7OmJLEpn3S5icbjCOEQ+tqF54pIUfs fnCdpy6/1i7hKt7t977p1OS+WRtE3aLqNP7e60bXoqOFK7dEu5MZn16/tEJ7ZQ5QX21f f7LBVxwEczKSZ41EhuBdn30VJYj2EUiJU0skExeNYuG7c3SH7QtWOMjIExFeMz/KOBd0 mZpQ== X-Forwarded-Encrypted: i=1; AJvYcCWSAJdOknqcQpaCqwQSzCvrBoHV/T7RUyrKZ7csOk2M2sX7yEbkySmmUAr4Ii6A+KMPBiGVvztaTxbDuhjS@vger.kernel.org X-Gm-Message-State: AOJu0YxTX4jKGsvrwHOHpVWIty3+z8SPJpm4kK9Lv/UmoWs0g0c4iqms 4gNLKpki660LcWDvrZ3H1rOIvLepfgNjFCF7tfSsLq7KD3dfIcc/4c/aRk0mMGg= X-Gm-Gg: ASbGncs9BzPDrfaXsr3P9J/PHchWd16hBFiONpv4r7xg3DLtaKtt8/NKpGOUTbzDuqv 55hn33fdpm+QU715/nXLEmj/HT7gr70VNCRQyqy/YQZaUw0LNDQ6hUrKIiybktwzmGOYMfOGHmA gsKgnCR6TSGS6bvUGFLVyTC/JFbu058hqiLspGuclMbA5e3MNCUs5EoGc4NXe4OOQcOnFYqbR3K avhYLE50FzBpJ1T6S0/xEpauK1gOF/VOr5uphVQYFYGcpU1cxrfEl2BzAbF X-Google-Smtp-Source: AGHT+IGauJ9uKAF5WxLAKMTF0QyH5oGnff/+E24GczIOzswP0NPnXSGxgq/rQ7IZv36KxS5lv0tqHw== X-Received: by 2002:a05:6e02:1564:b0:3a2:6cd7:3250 with SMTP id e9e14a558f8ab-3aff50b64c1mr29557815ab.10.1734105377176; Fri, 13 Dec 2024 07:56:17 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:16 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 10/11] mm/filemap: add filemap_fdatawrite_range_kick() helper Date: Fri, 13 Dec 2024 08:55:24 -0700 Message-ID: <20241213155557.105419-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Works like filemap_fdatawrite_range(), except it's a non-integrity data writeback and hence only starts writeback on the specified range. Will help facilitate generically starting uncached writeback from generic_write_sync(), as header dependencies preclude doing this inline from fs.h. Signed-off-by: Jens Axboe --- include/linux/fs.h | 2 ++ mm/filemap.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 6a838b5479a6..653b5efa3d3f 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2878,6 +2878,8 @@ extern int __must_check file_fdatawait_range(struct file *file, loff_t lstart, extern int __must_check file_check_and_advance_wb_err(struct file *file); extern int __must_check file_write_and_wait_range(struct file *file, loff_t start, loff_t end); +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t start, + loff_t end); static inline int file_write_and_wait(struct file *file) { diff --git a/mm/filemap.c b/mm/filemap.c index ec087bab1c97..7db995cb5179 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -449,6 +449,24 @@ int filemap_fdatawrite_range(struct address_space *mapping, loff_t start, } EXPORT_SYMBOL(filemap_fdatawrite_range); +/** + * filemap_fdatawrite_range_kick - start writeback on a range + * @mapping: target address_space + * @start: index to start writeback on + * @end: last (non-inclusive) index for writeback + * + * This is a non-integrity writeback helper, to start writing back folios + * for the indicated range. + * + * Return: %0 on success, negative error code otherwise. + */ +int filemap_fdatawrite_range_kick(struct address_space *mapping, loff_t start, + loff_t end) +{ + return __filemap_fdatawrite_range(mapping, start, end, WB_SYNC_NONE); +} +EXPORT_SYMBOL_GPL(filemap_fdatawrite_range_kick); + /** * filemap_flush - mostly a non-blocking flush * @mapping: target address_space From patchwork Fri Dec 13 15:55:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13907396 Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 476811EBA02 for ; Fri, 13 Dec 2024 15:56:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105380; cv=none; b=OFXLkQaEpDcSOMEObM9XnwQEmtGpb+0H7w60DZSc3zPJt2sTjKEKOzTU6FmB+1tpypbynPp/0EXybhlXkVZtajWpf4ABUhtT3DXQTv/4URLMLQQzLHscFLHaUjxguB1DIi852ppL2t+iaFvgl5L8PUEbRX7McLeeVPmRFktWR5w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734105380; c=relaxed/simple; bh=zE0xjm5rhzd2cUhrSxtogiQia/migrwaLvBNVMhxtxQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T0kAsGm8N5wE4MeLDpZrujtQdcr+hf3sUgJU9jJhvVbpZ6H3P4vd9LcsqX+bb4CNoCAaNJuMf7SLsNEuewxzLlS6avwc7cbt2Pjd6XM0JRzrb5lxnMNFPpcYE3GXAHWqV4PQIWvGitidHr0H4gnTPVZGAZpk2d7USvXTVcnu/9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=HA1gc6ir; arc=none smtp.client-ip=209.85.166.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="HA1gc6ir" Received: by mail-il1-f182.google.com with SMTP id e9e14a558f8ab-3a8ed4c8647so12588525ab.1 for ; Fri, 13 Dec 2024 07:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1734105378; x=1734710178; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=HA1gc6irLN/yCOG18U3Q+66W/+z4/bTyZiSmJMqj1/w+cvYgQM7rHONzuluiRSQkBb cweE2VTc/HK/3DIwpjt1c2Nf+Gxv7SVYHiJHDaKlL6Ue4358GqyJqk4lFnNX4KOo53ku VmFbHFJypEDtZ/AoDel7A6O8JplP2uox9ONHtdpGANY26YTB9qjBVeCUpnTIPrYq0ADq LGEw+WA50yc2FslUOcedqLOjA5g2WjxE2ecuUhJGb5e4XK+bOgfINf86n4UFnP7qlARk Nc+mPBS0Zq36JNexBhw4nymkuB+o+0BvszRwuF2bqFwfAyaUj0YG3/g+Q75RwMt318lO 3Sog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734105378; x=1734710178; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ol/7y93UZDywrWGNYi5AP2kj1AIRWUpJ2p+2aggyqzo=; b=kFCqg8e2Am25pxA+O427jHBqf0rTIMpQztKs04DKqZpQc9chDRmkVf3XdbgGb868LR gNjcQaNr0/en2SDb/A0cr8Xh/jayCJRNMBULn5sromZ0nN3nTQ+2qQRnO7QuSf5YIXsy bYGcsWOLsMyq3Ee3lQHwCQW6nTZYQoeLOTShgfaVI2DvOW+d/lnYWP/4zIICnQIsZMNz 41DFzX+Fe/kyNbCzLpSEbO247RfHU6hSTrd8uGkEp22dCfigsXEHKWJKr+sIkf1VS0ZH aUDJ4xguOYi3MeGEpOsAE0Fm2wX+8+aY7LjBXvXlrQuYY0IRplT7KLREUgCJJDZdADLX HA8Q== X-Forwarded-Encrypted: i=1; AJvYcCWdhe5dWElebikbfTPCVGcdjDzV+5h4cfNZrCYsZCwQmW/dgiCcDazUxGhwLciUuyXn3f5ryFWG/QZ7X5m3@vger.kernel.org X-Gm-Message-State: AOJu0YxiHqJ0hlQN/4W1L0e9X/l17uv1+IFeiQzt9Zq6Qx9aMy497wIt ihXNvj9JmuuMVJOxhScuCWIJ/t2c2VLAp3H+V4KSRvSOERacOlWPQAhR9iUJWpE= X-Gm-Gg: ASbGncvj9vFQSU9opJxUl/nDgz19UbktZrrI/9uBTvY+2D/VXuBOST5rylvcI6mxM89 tiUBdaXOqn9tQL6IkIsEdAzbsnrrk9YOkT5/e6Fe+8/CzhsqHdwgXM/S0etg83crGL1s6BaAKyN sGCXXnCz+vttEypcSaYdHn/2seGyTSqL829S/LyxYis0UhkqH2MIcf3bsMq3RooIzVyWAtsRyrY xf3hBudTtK3UzjzkweoR9gcSX1GFr9NgkHP0+C1bc0kLsDaw9rusxnrJh0X X-Google-Smtp-Source: AGHT+IFA+GuHLn3+MK1ZeRZZP3C9VMVBGmb7rBcbA1fV4I0OGtHKvL+W7OoGXWUKvvS+Tr+1pcK1JQ== X-Received: by 2002:a05:6e02:2169:b0:3a7:7d26:4ce4 with SMTP id e9e14a558f8ab-3afee2d0367mr35186015ab.9.1734105378594; Fri, 13 Dec 2024 07:56:18 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id e9e14a558f8ab-3a9ca03ae11sm35258405ab.41.2024.12.13.07.56.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 07:56:17 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 11/11] mm: call filemap_fdatawrite_range_kick() after IOCB_DONTCACHE issue Date: Fri, 13 Dec 2024 08:55:25 -0700 Message-ID: <20241213155557.105419-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241213155557.105419-1-axboe@kernel.dk> References: <20241213155557.105419-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When a buffered write submitted with IOCB_DONTCACHE has been successfully submitted, call filemap_fdatawrite_range_kick() to kick off the IO. File systems call generic_write_sync() for any successful buffered write submission, hence add the logic here rather than needing to modify the file system. Signed-off-by: Jens Axboe --- include/linux/fs.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 653b5efa3d3f..58a618853574 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2912,6 +2912,11 @@ static inline ssize_t generic_write_sync(struct kiocb *iocb, ssize_t count) (iocb->ki_flags & IOCB_SYNC) ? 0 : 1); if (ret) return ret; + } else if (iocb->ki_flags & IOCB_DONTCACHE) { + struct address_space *mapping = iocb->ki_filp->f_mapping; + + filemap_fdatawrite_range_kick(mapping, iocb->ki_pos, + iocb->ki_pos + count); } return count;