From patchwork Fri Nov 8 17:43:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868781 Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 130141EBA1A for ; Fri, 8 Nov 2024 17:45:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087918; cv=none; b=nIAhA/vaKbb6ABkWJc+894EyzrmdX1HSkJ3iaM9udsoTH+d4NNUZdok+rSumxKhUGCp4bkF9tcLotx/2X/s2Y74Eed0g5Cugbgh4Bpq08jymlfPDvlin3082yh/FHxxNcrgdpLdzVnkKsTwdEOcIW9PfA8kVD6cfWaZcQAvVdG8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087918; c=relaxed/simple; bh=RlKjacUZys9CzUs88jCGaGTWGkptV948qQSd9Cz5KPM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ftq5lWKKsqBl3HjqT8YNJghNRGCqeXnIOu+8sUELJ8iEK+5I6J9Z5F32cX1bujf9DsHlC2+jM0r00hXE9A3b0M+qc7koUgo89UVrAExyUyjegJ0AJoG8u1Em8PyB7jLxcaeoS96kDVw2WsmeNmAAkKM1FNcXiwRD7A1h3wgoCFU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=OWe0BsWf; arc=none smtp.client-ip=209.85.167.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="OWe0BsWf" Received: by mail-oi1-f174.google.com with SMTP id 5614622812f47-3e6010a3bbfso1610379b6e.2 for ; Fri, 08 Nov 2024 09:45:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087915; x=1731692715; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fZxELLVFDjwUYsFIGQEeVxILEzmt3BirwJxOvJB40G4=; b=OWe0BsWf3HiSed1r6ptVYceXWgt7yTZ9toOJCy82z1ZpRduRdKVEOQmEFSZfDD1sWT J3EaxVOZdewoQsKNPhshWULaWv1Q19Du5aGl5gqi6ts9sKgYYTTNXUIUELB2ZE5cwvXL AULq4hZoO0z4yWPVStnWMQnqDeOaFUah66oycklJ724rq6WA/urwuOgaMqmXckjfGxSw TtHYfTkDWMMhIv1dU+t7fUdoxQ3Bdx48PtMvluQR9rqR7btk+7Cr6syT0VU3qE0KnQ21 FT6eS3QuSgZKjaW2QEkeQPur+fC4ssRyK3K9lJ68meL9LobEWIPEdn2a6HvZHttWeQjd jmBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087915; x=1731692715; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fZxELLVFDjwUYsFIGQEeVxILEzmt3BirwJxOvJB40G4=; b=ABJX/AyFaFehTwUS14naEfvsaPYv94JRw9KZmYTr60wq0QHlpPMg1iZEdHkM8ftmor hseIe87a4OmgGkhgNTP+V6iG2JMMb+Ice0JVdUs+4DKQl1wTp7ZTsHMzi6qfH/CeUFoO LJ8XX6po3edoLsFI1NBNooZxNDE/9qN16B5Cxq/gx667PwG244Nv3CLaA1CNoWgvVV80 LP6PI0fZ62DLVPzoeJojF2rdZsax75JkeFBC9nmSjlHiCBqQUEfhdrdv0K+FE/4fqHAr Gaktzekw6iH29rJ1KlG8NDmLYBaPKtgIqgmurVEjNyV7o+0cb5JoYG6CNqbqbBHlSVU8 K8nw== X-Forwarded-Encrypted: i=1; AJvYcCVjo4MtyEPI191WWGsj8sIJUXjUvuLIdyc7FSt1Wn8PYvHhrDNCJSQoE2mMy6SEDDsHCboEVZJa4Dwd0CMk@vger.kernel.org X-Gm-Message-State: AOJu0Yw3HRmBsTeOO/ohdC11OkqpTPtAN/XuTH1iU1QxudWG3YTytbqz Vg/FZ5d8ZowVMzS82TcK4FCxw7AHxqTY/fYpjV9Vl8BHdIEC894pyN4t7suBxOsvkBHg3x/pLvA kQq0= X-Google-Smtp-Source: AGHT+IES+vxslMkWIJ9Qt53dufEtarFT3S5DaOvldgjpcVbvhQDjZUhL8AdkSUoXq58a5ka/shhJOA== X-Received: by 2002:a05:6808:1524:b0:3e0:7005:3f86 with SMTP id 5614622812f47-3e7946c80d6mr4180469b6e.28.1731087914910; Fri, 08 Nov 2024 09:45:14 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:14 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 01/13] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Fri, 8 Nov 2024 10:43:24 -0700 Message-ID: <20241108174505.1214230-2-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 36d22968be9a..2ae26a0f961b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2460,9 +2460,8 @@ static int filemap_update_page(struct kiocb *iocb, return error; } -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, + struct address_space *mapping, struct folio_batch *fbatch) { struct folio *folio; int error; @@ -2487,7 +2486,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index = (pos >> (PAGE_SHIFT + min_order)) << min_order; + index = (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error = filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error == -EEXIST) @@ -2495,7 +2494,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; - error = filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error = filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; @@ -2553,7 +2553,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, if (!folio_batch_count(fbatch)) { if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) return -EAGAIN; - err = filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err = filemap_create_folio(iocb, mapping, fbatch); if (err == AOP_TRUNCATED_PAGE) goto retry; return err; From patchwork Fri Nov 8 17:43:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868782 Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19FF41F26C4 for ; Fri, 8 Nov 2024 17:45:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087918; cv=none; b=o+AV3ta3UhQyQTPRdXxtY4D8akVZcZxWpBCCt6v8pihR+gZkMGvdZPbSO7L0I1BKqGiJ5rgrZDLSdZq7FSC7jitgAR28Ea+CE4Ls1AlspXd1SaKPE0zTJpuNiEFDIYmDX8Zl287YiPETx7f3bCIZbLJTyr5D0Uc3TxAkqSq8O0o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087918; c=relaxed/simple; bh=wdE7j3SXsvf8mZ3WLeO9bDbqT1xBl9uoI5Ta1MifPnA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tvqRiR0R/KQYClcL6ya1AjCuEpoHxLhaQyDdmBuorURoPif6hxdBkmET6j56pcfTD64HpOTj5drnzQc0Ygwa2OXCJ2X51bNKuez8GMm9L8QmlAGDWqwCAd7c91P0O/+a9xJ9GSeL9oX5vAu1IshYAmWWOYCkpDJdROJT30SHojY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=JMj52MHi; arc=none smtp.client-ip=209.85.167.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="JMj52MHi" Received: by mail-oi1-f173.google.com with SMTP id 5614622812f47-3e5ffea2699so1540190b6e.1 for ; Fri, 08 Nov 2024 09:45:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087916; x=1731692716; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=JMj52MHiupzV+QyEZ8xQQuhgCgXAUk4SMreGJBReimFtWo7WbHzumZ8skKIG5nMs8s 8lIjkkEMN2bE2vaekbNtF4hmIhCrPgL7BWlBcejDcpxEnTAnYSXb1coJGpECPxiZ6i44 TGLKfhpxcqJd+ZhogjATbLP8KhGEp1tw2o0UrpioybOGRfcceoOTRoMpIMbehLlDCYDG J5I5bvrpH04WaJ/WQ76sj8WcCzXyrOAGqGCCnY/XdQMpAEJeMBa38QJjyhpIQefp5REn xzb2BDJ9ckAb1q1wqZc8s1t/1cGucPE/skLRTNQubDsewS+d4vLFUWA19uRHpvTrtkch XuFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087916; x=1731692716; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=rbnioEvNkv55ME63fbq3k25yCTeHSSDqWQ918kUM3aTDpk/hhoOslDdYkxRTm82LYH iHjpXaRuRibmzsZ6wMfnEsTxzri6T5Ms53G5CzWIsgALXc1jYLJ7eo3Ua1ctBZEr/60e 6UDsvyjeYXFDezG8/Raxd17J5Q5mbgZ/r5CnQ74YN+LgVy0nxmGE1hAfOkkhQxXb6GZ+ sOnUfNynmSIxKkC7M8xGq0xRb7YQy/1oFqVkG7RfR7AwQL9F6jrL6XKUUaSAb4ZO6Yq6 h4FIeUODFWrBmd0myyQJu6L+To0gltThzDd6D9swhpsRt5892s0Kx0Ap8IGbFYhgeRt1 t69Q== X-Forwarded-Encrypted: i=1; AJvYcCUeMKLEFfGymNBjbDEcgeEFYyI4PcH2Jvt/I9PgsHIo6+3LlTzNP1J0v+4Srwhl98tuxPNEhwc+wNLd6E93@vger.kernel.org X-Gm-Message-State: AOJu0Yz0TsHR7ySSreRsFpYliWe6Kl1Ov2/93DP1F8FHDYJqel2KTnbE z0VFnZ7HdrFyx9k+m4LEjtdhNuilaTQ0YyWYP4uidmGq/s207CKDog6cXCSCCXI6execB/RtLHl ONvg= X-Google-Smtp-Source: AGHT+IE9yyGg7trloxg8/sxB/5Qab0oFpLsLqTwz8oiFcGSIzzHT5q20m9z/5W6tquS0Nz/aDzjCkA== X-Received: by 2002:a05:6808:148f:b0:3e5:f9e9:43c4 with SMTP id 5614622812f47-3e794dc5b57mr2378031b6e.18.1731087916223; Fri, 08 Nov 2024 09:45:16 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:15 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 02/13] mm/readahead: add folio allocation helper Date: Fri, 8 Nov 2024 10:43:25 -0700 Message-ID: <20241108174505.1214230-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index 3dc6c7a128dd..003cfe79880d 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -260,8 +266,8 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, continue; } - folio = filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio = ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; @@ -431,7 +437,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio = filemap_alloc_folio(gfp, order); + struct folio *folio = ractl_alloc_folio(ractl, gfp, order); if (!folio) return -ENOMEM; @@ -753,7 +759,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; @@ -782,7 +788,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; From patchwork Fri Nov 8 17:43:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868784 Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60DCD219E54 for ; Fri, 8 Nov 2024 17:45:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087922; cv=none; b=tpUFxhzYKA2DGfrv1yXwt6nX4gRjsRvhRs3o1yQLhqY9QK3opCs3cPFa1u8lgPk/Z6OwHHimTALeOAcoqDzRPWr41JQ+e+0T2pAO2LQfqS/MMHDOtFDraaeUp7xbzCdvEwJ0oIaJbvK5w8JZdo8TgCIg7TX6F3tNw95oAzOBxtE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087922; c=relaxed/simple; bh=klEANlDo0QcivFe1NIYDNiOFa1lRCa2wf9JjLv/2My8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NfWulqOWqhkUCfK/e/iRT1xx3Gw6UgNs/tiBUM789O/uuE24Jb6qXUKCO+rbIf9OdunMYgQXXY+vYLaAkhmlITQyXNnsJyYK5Fgdc9qKZZm9I0A0q/owLKVoBmuXM7+a02DMIEXOaj/v2NAFzRC6XLoQi0cnVFqvSondYMcR9D8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=T0foBzDD; arc=none smtp.client-ip=209.85.167.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="T0foBzDD" Received: by mail-oi1-f172.google.com with SMTP id 5614622812f47-3e5fef69f2eso1493233b6e.3 for ; Fri, 08 Nov 2024 09:45:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087917; x=1731692717; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=T0foBzDD4OiqyTOmDAvUWSlf8ZX65Rcd4TXeNwMBSGFnyZMpDWYR/Na3JXxZUkFG8f Qvb2zz0xoaCdxGPhKgIkJPVQgR5A5qPOtvMdw8pybkzAzFgzLOLPCdMI2EgxIhGqdv+y PX4s29QLZSFEuWtdYH+BzSPCXf7F2ziNQG9QuCBvEeQ8S3bZWHhQbPZPInu+NeXqC2Ve GAIwRTgRZ4K+QVWRjfWisNu3uR/Czq9+kwDbkFg1ZczTwpPX7cwcKdFqegD6LR8ftX47 CPItpb4+1JW/ZNOarFR/zsiYYACeMFd+oYyCQo7xUDnAekgPtq9cYNm1AjmzhbsopSzp Tdrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087917; x=1731692717; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=gSGrPDrChWscrJD+NX88tCm4HTIPgpCkPy/Z1rPIiXpFd3UM4eVCisR6QyUV3XexJF Qh9mtLcL1m9E7TEo1Hhc0eaQBP27rd6onS8249Mubev3KvV5WTzYXzRn1n7xz4JFN72S v6PA46yhLGkM9KdA3FwIVa4/6Suk8Vrw7a6pexu7UgvWTLeKxwGiwHYk7Pg/6TtERI2d R6XpwS8zUnkjPKrYBXhbmWMfy2+8/Hl0l38IuAenIAW3MHNrnabPCwfGX8VnqygM0ZGV krNUTsZ55Jo5cralHcaz6hmlYBa/0AMbjT6CKBsXzyK8FxMrHwOUG2fAi3QxGRUO+dDM P9Cw== X-Forwarded-Encrypted: i=1; AJvYcCXumb5QCR8Q/P+q/nBnEUEPeBWtilnamH9UliUWnN6/q6SUJlec2ddcO82+mtXoYPDTa898F9bc+LsT9a3Z@vger.kernel.org X-Gm-Message-State: AOJu0Yyoqeb1qq9Lp6cTgadW25Wzerrx9uRIy1TrimCQyMLLjh4vpkK/ 45rQYrX6ri6jKZrUzOOiJ97THiyTjGCN/+rlu1O+2QxEr94DRtjoO4v4qNTI4vQ= X-Google-Smtp-Source: AGHT+IEFLKzY9PDxLtI3q9tzu1PwOWnBFTiZM//bYtu3vxzi/sNq0OsnpZ8TGg2ZJ4XyTsEPnsyfQw== X-Received: by 2002:a05:6808:1789:b0:3e0:7d53:251 with SMTP id 5614622812f47-3e794675decmr4661644b6e.15.1731087917560; Fri, 08 Nov 2024 09:45:17 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:16 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 03/13] mm: add PG_uncached page flag Date: Fri, 8 Nov 2024 10:43:26 -0700 Message-ID: <20241108174505.1214230-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add a page flag that file IO can use to indicate that the IO being done is uncached, as in it should not persist in the page cache after the IO has been completed. Signed-off-by: Jens Axboe --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cc839e4365c1..3c4003495929 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_uncached, /* uncached read/write IO */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) +FOLIO_FLAG(uncached, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(uncached, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(uncached, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is not diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..b60057284102 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(uncached) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ From patchwork Fri Nov 8 17:43:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868783 Received: from mail-oi1-f182.google.com (mail-oi1-f182.google.com [209.85.167.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35C6121A711 for ; Fri, 8 Nov 2024 17:45:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087921; cv=none; b=IwCdKZXq/2VpbL9O6enxB9z8DUL9zM/mCQWChEzSJ2K4eyK8SMalBfvAIQjF0wJ2BgZBoqn9b0lSJPKDskOTgB5urjI3K6BtyzbuQuxeuNxfKxURBPe6VsUphTPqGWX8t9cs41cw+YzM2Pl1ajftLtljNwdYkEM9S/arrQ7MJfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087921; c=relaxed/simple; bh=SpPgvJ1Ucle7567qSYAlsafWxvPkBXjWNV83/9D5Z3U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KXYVI+VVzfqIXZbKAGD8kjQweTeehXvxUAxKLD2Xy6GyCuY115NbZtMRW82UejkEyIHuDiD1CN5U66xTEGCKDu91neyAJff9z0Vy3nS+sQBkYFY1DbM2QDbvn5saCiIdwsufVTrFZ49lb+5GsqA7bj9XeQKz68tnIhYRndez38k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=QgTefese; arc=none smtp.client-ip=209.85.167.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="QgTefese" Received: by mail-oi1-f182.google.com with SMTP id 5614622812f47-3e6104701ffso1630395b6e.0 for ; Fri, 08 Nov 2024 09:45:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087919; x=1731692719; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nPkGwSldMNFXF3cURXd1feXLkclwp0ZCxyEpzT1cPW4=; b=QgTefesezjL6yt0YcDXrIafVcZ4En+JuRAc1rdbDoeHUtEYsG4hAg+fHlAjU1yRh6l DBcLsznGPacdN/HaFGb1Q+8FAFpfSSEwgeTa+9I4y9MzxrhLZDnb08dLNF2bxn+tBTUC FvrklWMXvlCqu7sNkjdnZFPYynBTicRuaMDrEiFBUDGo/e6H1JW93kTp4HhExkdn9owJ 9XIC+pMxrHDuhrHB1U65vagZkZuj66cehY+oe/X1jT43H7nLKwOkq0+v1G/NqKqjRO0b bjUmdL01IHwCcRUsv/e3MjA3YeBnLL85v+NneWmbFIiLdmmmGD0/LK/P6BCfsHVASwvQ GFSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087919; x=1731692719; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nPkGwSldMNFXF3cURXd1feXLkclwp0ZCxyEpzT1cPW4=; b=md0f4+aZMNGp9KxMZTYtMpLT/+wrCXmcjjIzY3whJ7iSKbzd4vS38l2QlchyMu4rs1 moeJuOrmEvLajlujct+YLG7kBbIkMQGUg2GMeC9nEvVDJJGbuDSX0MefpcMERoNWzrYw 8VRhgcdmxYRh9fRs/MdyKLg/zg9lr3WuPM2EFgS0iAWnaqZjmbLqZ2NJ2yJG1Il3y0hU Vsw6Gx903LHgSInylJdW7w1JNe9Jylr1bVhlpZ3OKKSr8obYjtAr+GGkBDgcRtSNUDk5 pdLAvbKC7OvPV5XdwW3AFBTC4m+sJCjDNxmkgmsyKvRw82kKmTXbDBE01iul6JnINeLQ rP+Q== X-Forwarded-Encrypted: i=1; AJvYcCXsU0CmWFOVc+136uVLvlykzgCE6yRX3sVz7TpcqiPBOVMUDgxJ/5lrkYE8HPL4bvyjonO3Ng3C/55mzzvv@vger.kernel.org X-Gm-Message-State: AOJu0Yy6pyg4QYWdwWureT63BcVODBTXxBWrRu2c49S0+hWV3glkUiNu +PcneIp5eGMC8+ahiZqRVquMe9nRUrN60wuZv10Q2wuoIMIjHtbyPhrbmRS6U9o= X-Google-Smtp-Source: AGHT+IF9qnEH6iAmuqzMOZRDZm41rR73Vxyg2vpcvzcsSJsHVKcaTPsimGzR6K5FioRiJT7HzzWlGw== X-Received: by 2002:a05:6808:2119:b0:3e6:134e:3b90 with SMTP id 5614622812f47-3e79470a347mr4589807b6e.30.1731087919330; Fri, 08 Nov 2024 09:45:19 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:18 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 04/13] mm/readahead: add readahead_control->uncached member Date: Fri, 8 Nov 2024 10:43:27 -0700 Message-ID: <20241108174505.1214230-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If ractl->uncached is set to true, then folios created are marked as uncached as well. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 68a5f1ff3301..8afacb7520d4 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1350,6 +1350,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool uncached; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 003cfe79880d..09cddbbfe28f 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio = filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->uncached) + folio_set_uncached(folio); + + return folio; } /** From patchwork Fri Nov 8 17:43:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868785 Received: from mail-oi1-f171.google.com (mail-oi1-f171.google.com [209.85.167.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6260321C183 for ; Fri, 8 Nov 2024 17:45:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087923; cv=none; b=tryhGXd19nRjDX4hwXSZOIKZKzx34lgA8pbCZNeRlfZvOv9aaoozygHkDd4DT2C6Qtei/SOCJKJP+MiL0pQSUsBeyGQcKfnlUGmqbUnu3nwVtqCW3Qks5+T/1fXCiMfBYWpQ6lJFlcatWRycWTSIJ6Peqj549S255BEFhUrtIcM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087923; c=relaxed/simple; bh=xOX9JdqoIpgVbWfGU7LRI5hAxDmBYLlvRERjrD8i98I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kHiul4bfZbc1WaZx7kQ75HFoMCjnQRsZ1TppVf75guZEtlb9JUGnbZrlEFGqkeOM6YbZQgJ9pn672AH3pdrSxGasBJmoyUeKCvMS+uZPzSP4GwMg3tqyplaR7xHFHUYmFNvIHvdlCQDUQQFs3BTflOw6CL5jHsblgnjY2HOt/mc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=2d6wZF+g; arc=none smtp.client-ip=209.85.167.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="2d6wZF+g" Received: by mail-oi1-f171.google.com with SMTP id 5614622812f47-3e5ffea2699so1540224b6e.1 for ; Fri, 08 Nov 2024 09:45:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087921; x=1731692721; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c9RgbH65zDtdD1C8SyJhiChYXVIUYPb7qWGKr+fnitI=; b=2d6wZF+gDH2+unDxPCQtkbtrhYq7FFY3TsMTQYHah6hwQUTgbnKNlzIo7vccH5NLG8 ZkjTFyIWdk3mmKa7jrk7zTomxqdVqby2wQqT8jlDjIpE/Sk7XEkrgPfJrxlxLLctj0Kp mdIkFtbC8noeUYLYvzTyyLl1NDk2JPE+q1/oo9VVOqRGIh6rc/yq9E4lpCcqtradNhFe z4S87HXy+OQLgU7nu4Wf3XBZkVWWTuII7t8MfEiF+BpPb0S3zLqsF3AH+9m8RWm3Ip14 ZL34TRNDjiVOY1VuRJZa4OEvrbUPmpO5Od3mLd1OlX7+tD51eOaCaqjfCvcgPiMxi82I jg7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087921; x=1731692721; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c9RgbH65zDtdD1C8SyJhiChYXVIUYPb7qWGKr+fnitI=; b=DV6zh/n1SEJbym//6tJ6PdJMN8JO0f0fHgl8gxCOqijO+JkrTlc2e6RCcE7oHjRwRw ZeZWDVZa97n6NTRYOSi2WXB1eJLVbfXN1MIut8v5v81j3l7UkmZPLZSePfOVx+etAy3A ooR5HUaZOrRGfPd06wg2RzSHRqNmaobMn1Hy92tb9t9ppbTWNxiY6/d3x0+VDscVkyPF IQxwbpn4XpZAR42CsBl/JMrMHvzKIL0p/5/JqUlH6TtYoFpxKuXJqpExnd/8fw6TBZ5U dAyEfLmxXMwtpyv5E85JV3jGdxngc/xeqo2mfm3N6ejlDyNU+qX+0Y378cySfZ0gMCr4 gc8A== X-Forwarded-Encrypted: i=1; AJvYcCXCnWGDC9lou+7NvFFq3VcHB7CPhG1+zsEGsO0V18N9vxS6HzRAZCvXTBMWAkaucgal4gBZRYcvTDIGWO7o@vger.kernel.org X-Gm-Message-State: AOJu0YzhTOw7Qbrq4evrKclQkF7E92oYBFYk2uZefYVJFYPZADRBbsZV 0yBw8q/swHaywbGzk+ZMS1VaIc2aH26ILrheDGm+qVPfsbDKDqrB9efddVreFXk= X-Google-Smtp-Source: AGHT+IEdm8GyqJXGR6xkfsMSAWw5HHFtOAoa8M2cW5v0Mo0ooZH016xwV9QTHy9Q50Vx5kixVOJjvw== X-Received: by 2002:a05:6808:1391:b0:3dc:1afd:6688 with SMTP id 5614622812f47-3e793f74377mr2161727b6e.20.1731087920676; Fri, 08 Nov 2024 09:45:20 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:19 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 05/13] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Fri, 8 Nov 2024 10:43:28 -0700 Message-ID: <20241108174505.1214230-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 2ae26a0f961b..7f8d13f06c04 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2525,7 +2525,6 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; - struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2540,12 +2539,13 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); From patchwork Fri Nov 8 17:43:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868786 Received: from mail-oi1-f175.google.com (mail-oi1-f175.google.com [209.85.167.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02ADB21A711 for ; Fri, 8 Nov 2024 17:45:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087924; cv=none; b=QXmJ0zYsj8DMIoq6Je5a/ILH2s+h8jzvopNj0+BCuTxOhUunrmDWbl749/RFWa48PldrlrpQLviSWgvPdF1lb5dLkffLtEmzK95UQq/isk0mwJn9sFJaOi927g85kdpkFAmJtG0EjU1y3dh+Nx4HtuUFpO6eqo03bxoakKONq7s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087924; c=relaxed/simple; bh=aEnrAJGEshe44zlb6XbbKSp/FXpeH0RZvF9K0y5SLj4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UM9QgWQD8nThrSsFx12a/3L4ZuTc7YLqZdFvNluG0YcQruXzAbFr+UewJkBX7lve+p1qXhSXSCWrtza/SYd73q4PlrkaU5Rm2Wk+zJxLgz8I4Nm1+M4hcyJtLmUpqXKmOJvaIjwM984Xhz5Iv24XypE3Y2EKr06Ip2jeMQfwzKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=lRrySLls; arc=none smtp.client-ip=209.85.167.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="lRrySLls" Received: by mail-oi1-f175.google.com with SMTP id 5614622812f47-3e600add5dcso1373114b6e.2 for ; Fri, 08 Nov 2024 09:45:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087922; x=1731692722; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SJBisbj/rGvidSFxNEf05dxDvPbQrKvqbgVXmZ2uTrI=; b=lRrySLlsPvkjgWEoIeTcImf5J9L6KPiHTwOJ/WY5bYGrymWCjwzTNLULYyDH3PBYXX BMR8zdThJIsZRHADfIi/j/kT28vGano02RM17EGduF/b6BSfQGoDco2mqTAg1gvItsvh liivkL5JkhRVG8gHZ+OsE2bqNyruxmaDYr4Yc3GTuqeGRfrlsjs3cnFp5qiGgHNFsDaU ti6OXyjw1ARiRySWE08pnz2qkFn1d6hN9WsRU4oqw0M80g5qGixR4wiU3D9BAAcn5vfX Y1JuU8LAWmE2H52GCvP8DekK2p2035wH6JDE/4lDZBM96DgZ/o3IbqJW1Frh9fwRByI8 uDbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087922; x=1731692722; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SJBisbj/rGvidSFxNEf05dxDvPbQrKvqbgVXmZ2uTrI=; b=jD8v/KBOHyec4v/j9H9fUKojZWdxSNzPaqs/65mJTocxO4J9CweVyVb4XYqW3rFoxj 0QPFKSWQsY4m+S/zdLFT1j2JxsyNU/vwEQ/h+2ofpVXtHK9Lm/RFqZI7Tluh9HsgjDxi 3ECqD1XDK+u57xsfpBbW0uzSgE6nDuaX0Y4oerpoYZi0Jg8j2b7N4+5//68PMvHQ4Ga1 rFGq4OiiwkrQXEEqjqeULl46Pgo3X4Q8PqXFkzcuoO297ZSKYSKDaGf/GTm20ZJc1w+X AiouWxkZ5NuErbdvxzA2MI2+EjF9YDX5mUNLWxaYmcvAYNWtoZv+R2+v26PqOE2ONLML /x/w== X-Forwarded-Encrypted: i=1; AJvYcCV6bZtbZMlIN1tKhcboeq3udYwibtNlw9mkAFil3fcE8s7PciWpGg6DbfwzMQqmiaMQ0PtpbpNeuzzdeeW3@vger.kernel.org X-Gm-Message-State: AOJu0YyIGGMDHsajFEWg7xevtffiWRwEDRyoDIgW8WFAYFfs6kl7BKET t1ZlD7UuSEolXE6JnM0F/fMxRqYOt+JWE8I9dTZG5OCv/I7VT110NOXVqGIgNW0= X-Google-Smtp-Source: AGHT+IHAnYXln3VhbkemWI+8mH1zeQAS07vGHF77rvkzzA3Evd8Py7SGOWt6/hgQMWEBNANFNnt9yQ== X-Received: by 2002:a05:6808:1987:b0:3e6:1057:21af with SMTP id 5614622812f47-3e794772f73mr3805781b6e.41.1731087921963; Fri, 08 Nov 2024 09:45:21 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:21 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 06/13] mm/truncate: make invalidate_complete_folio2() public Date: Fri, 8 Nov 2024 10:43:29 -0700 Message-ID: <20241108174505.1214230-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Make invalidate_complete_folio2() be publicly available, and have it take a gfp_t mask as well rather than hardcode GFP_KERNEL. The only caller just passes in GFP_KERNEL, no functional changes in this patch. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/truncate.c | 9 +++++---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 8afacb7520d4..0122b3fbe2ac 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -34,6 +34,8 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); int filemap_invalidate_pages(struct address_space *mapping, loff_t pos, loff_t end, bool nowait); +int invalidate_complete_folio2(struct address_space *mapping, + struct folio *folio, gfp_t gfp_mask); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/truncate.c b/mm/truncate.c index 0668cd340a46..e084f7aa9370 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -546,13 +546,13 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * shrink_folio_list() has a temp ref on them, or because they're transiently * sitting in the folio_add_lru() caches. */ -static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) +int invalidate_complete_folio2(struct address_space *mapping, + struct folio *folio, gfp_t gfp_mask) { if (folio->mapping != mapping) return 0; - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, gfp_mask)) return 0; spin_lock(&mapping->host->i_lock); @@ -650,7 +650,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, ret2 = folio_launder(mapping, folio); if (ret2 == 0) { - if (!invalidate_complete_folio2(mapping, folio)) + if (!invalidate_complete_folio2(mapping, folio, + GFP_KERNEL)) ret2 = -EBUSY; } if (ret2 < 0) From patchwork Fri Nov 8 17:43:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868787 Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 465E121E13E for ; Fri, 8 Nov 2024 17:45:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087926; cv=none; b=fzXnZ9nHzNCHQrCvF+FjqizunUoOOYzAGWuZTHYIEww2Ey1b8sWTHByRiwEnrxW61VlYOVk/l6Dw409xO3Ugig9ut3xXvV637E0gy8qtFiabV7i3S8W9VzEA+7PVkak6o22gzdfFRYA3JUmZWp2Ylw2hUov1UsNGvbgY6nVFJaw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087926; c=relaxed/simple; bh=DyjqDUCBJSj/CnUsQELSU5shQKyR4sQgd2xgO6LEYiQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uawr7+YyXZk4XHDmaUzMZLx363GSOR3nhCQwY6t2VrUo8N8T+WZCrTjkZPAHVH5yQRcJAurKirzQ0rgm2NSCB6eLecwzHP+h6QCRExULFpxNcVAg4AAN9nP2TO0ZWvlEFqNulAjAh3o7peWi7f6bdWLTBVmbeDVnkA2zsX6aN/E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=OnJ2iGzW; arc=none smtp.client-ip=209.85.167.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="OnJ2iGzW" Received: by mail-oi1-f169.google.com with SMTP id 5614622812f47-3e6104701ffso1630437b6e.0 for ; Fri, 08 Nov 2024 09:45:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087923; x=1731692723; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wWDNTRuAGEC8P1PkMAQXcWfNP3UvXVPT3pXPdqP7CT4=; b=OnJ2iGzWog30ADCSfPtO7riVPGsZU0byapv29srIuAWjnUpLjOlWxa8LC//Zm/7mkq MbrmoMuBKhH7AuvxUJr4pVyggg4d024lbbipXJW259Y708NzyTFL0jNTk8DeRp3x2DUy yhkP/rhG9eE1IR4uO+REPsLPNvKhtVHlfEs5VyaXRgbjoNs7UVtahBzKfNgzcCM8J1cm bbrN2XkomvS/zAUCUh8WAMvnQSu5P6HZhnsApD/MbZCk8EuxnpUkcbMup/PGESlb9dgu qO5X7Da9W0lVRl88/J3RiLtlLmIQgZPDXGnoMW6YBPfNJ9jqKMl30gUJ9qy9HqUztVGg 8Suw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087923; x=1731692723; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wWDNTRuAGEC8P1PkMAQXcWfNP3UvXVPT3pXPdqP7CT4=; b=lKdsrbJLlJI9bziBUBKvuA6OdFOxr1XzrwdmtU8oe4QP29lAO3RQNliuoXVYYBmAO+ 9ploMVkW85SFPjAiDzdlrWKAwB4Fy8UEnTTbEEDNgJDN6ERrFIrOXe7dl/BWuMGqUsDk t5ersEzjW3qsUavxCXZl18QRcpCLCsbpNQvVl9ejFsmRV2ts3jv8RDNqrDQypkQwDVku X6BziQ6imJy01b78ZXaoiWfsIc7KyM00IHSaOlVFezTu02EP2W/tJleJ+XuX3rr/RRqx Ce2/lAiCGXa0FUStAvf0KIViPSeh5EK2jjLcGQVS1bHUDkt6Ogpmorm/CkvW58/KjhK1 +G2A== X-Forwarded-Encrypted: i=1; AJvYcCVJtvITd2fMpr8NklXl8NHho13cqddvNETRZPzt54HxkvJHx7mb2k70a0HvFm+iW1I4ZI4v9H2uWP3YqTPP@vger.kernel.org X-Gm-Message-State: AOJu0YzHf50oU2ENBdKBDPLeFt1hGtZiujWkWDHld6zrTwTpDAnMEMbu erPzC5hHccOs2qj5ugpA8UOqTX8qanLAzKZJgRhuL6lQWh705JBkMKp3yXX5FYwN+5uI7y+DWbh lN4s= X-Google-Smtp-Source: AGHT+IEpy29b6bGe+h/CboagPQRbeA16vn/em+bpmlLLfLOrXhNW+1oHUaJS7mfyo/74o97NJpvLFQ== X-Received: by 2002:a05:6808:1525:b0:3e5:f141:1331 with SMTP id 5614622812f47-3e794733f1fmr4465090b6e.37.1731087923389; Fri, 08 Nov 2024 09:45:23 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:22 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 07/13] fs: add FOP_UNCACHED flag Date: Fri, 8 Nov 2024 10:43:30 -0700 Message-ID: <20241108174505.1214230-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If a file system supports uncached buffered IO, it may set FOP_UNCACHED and enable RWF_UNCACHED. If RWF_UNCACHED is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 3559446279c1..491eeb73e725 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2116,6 +2116,8 @@ struct file_operations { #define FOP_HUGE_PAGES ((__force fop_flags_t)(1 << 4)) /* Treat loff_t as unsigned (e.g., /dev/mem) */ #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) +/* File system supports uncached read/write buffered IO */ +#define FOP_UNCACHED ((__force fop_flags_t)(1 << 6)) /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3532,6 +3534,10 @@ static inline int kiocb_set_rw_flags(struct kiocb *ki, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_UNCACHED) { + if (!(ki->ki_filp->f_op->fop_flags & FOP_UNCACHED)) + return -EOPNOTSUPP; + } kiocb_flags |= (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |= IOCB_DSYNC; From patchwork Fri Nov 8 17:43:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868788 Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A14D121EBA4 for ; Fri, 8 Nov 2024 17:45:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087927; cv=none; b=t/vkY/sPCUdiaJusB2142EWNZbL0xYoFsoMIs9Jpu2wM+98Fcpwdj+y9urmSkVRlSIpy8+ydSChmFy77qomDGmJJ4tFS5lf8eb9byq8b41n8RRdMWKy3tLZFdXENPnh5/w3BL/p96lAlIBQloOJCg1UogBoELKheH7pbISQEvLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087927; c=relaxed/simple; bh=HjbXXXZ2QRG/1ZhQXxL5goItns1riItnFEOaINOV5dU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TefuIbFIIqcPI/V29JfNzB0OLP2/LYJbMybT1ddnMX0owwZCqMcT3V5XGgK/6AJrh+ishA8ZCwwLsmji4FvkVvEJ3kPbFdbwRJPGfaLod+3ldcEQj3A9Gc+VCG/zHBTLkqDlnrDFngJSAPVNl0OCbcwOcac/h/649J3FGDMGv00= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=mzGENvrA; arc=none smtp.client-ip=209.85.167.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="mzGENvrA" Received: by mail-oi1-f173.google.com with SMTP id 5614622812f47-3e607556c83so1581620b6e.1 for ; Fri, 08 Nov 2024 09:45:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087924; x=1731692724; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Tq+ywqpEyPaYsCraC6z/Wa/2p0SbC90MEdwC8hVGLhU=; b=mzGENvrAOCh9hnAR00/bEAnahHqDaQJ9ycVQx7MoHypO8YsL3RTCoHfVnApIsib/4U Rb1thvsrXx3tmAUaghDndxj/fw1eFeldg9NJRZOGWOR6cpxG4aroEidfl9cZWz/04php 5YOk0VJiRsJ5c67WGRwYJ9jN0xApw53DmItNNvk5u+P57j7Ne4B2OHHuENjYIOFJbLTQ mwgC6Oe6aZiY3Hr6Co0XZxxXcli+y42GwrneNyP8WRUIPxfUyuLnolDAQO1YM5xEjGKC YXa7cu8c95s5QppvoAoizbPXhtqhUNfvJADyb3g7kj0DRgkxnabnQ5GwpFsPkoHO/wng J56w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087924; x=1731692724; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Tq+ywqpEyPaYsCraC6z/Wa/2p0SbC90MEdwC8hVGLhU=; b=SbcAvwXrRKTMrcJNd5T3fyt/t0eiuNudAU/Pr1R6JAZAjCUqYIutPf5Ftpb9Fom6OT 6S2quFhktuHCK7H6noI/V8jt0BZuHsCn4KplQufym/bAFZt73SXU68WNUGq5ph9wLt34 JicHBgUVMBgDdzgJVcfb995IYY301y98moP+Kn5+YjIhlINIk/cDIm2ihvDPfWUhZ0zf RquL+73+Z2oCRuGK73VUL1DpUos8RzXZj58f+CSgP1A5dcQJCFT1ZpgwgUbbfwyqGCEz kD0cq0EotAW/FX8DpMb0M7ZZcUf6GhrbwX1QbdasAJtLmKfWLn2EMKxt5X6jDgiy12Fs dr/g== X-Forwarded-Encrypted: i=1; AJvYcCXdqzwrlSyIPNSWofXPqWh4VEkOJ+YXBwFIXYmcGGoqMjtn4sCMNJazBlp6VMK3o+KGx5iD1Tq4npwXrzXY@vger.kernel.org X-Gm-Message-State: AOJu0YyI3xEFR8Dj7sDdngpQByIJ2OvtPKCfOpmULmfMq2o8eK629m6u +D/cqI/3gT4Z87db5wG42KJGPeXNKjY7ShmwbqShbl18IAORCoLH9vduwmaejN0= X-Google-Smtp-Source: AGHT+IHYb37IuRL+rD3E4bWkVAGE9ioVB03ZuysIUbDHqe4Odg9KTg4FuyvIzrjyhDZyIdkBKG8ezw== X-Received: by 2002:a05:6808:1819:b0:3e6:3205:1a71 with SMTP id 5614622812f47-3e794699f41mr4284876b6e.15.1731087924649; Fri, 08 Nov 2024 09:45:24 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:24 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 08/13] fs: add read support for RWF_UNCACHED Date: Fri, 8 Nov 2024 10:43:31 -0700 Message-ID: <20241108174505.1214230-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add RWF_UNCACHED as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousing of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_UNCACHED set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- include/linux/fs.h | 4 +++- include/uapi/linux/fs.h | 6 +++++- mm/filemap.c | 18 ++++++++++++++++-- mm/swap.c | 2 ++ 4 files changed, 26 insertions(+), 4 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 491eeb73e725..5abc53991cd0 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -320,6 +320,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_UNCACHED (__force int) RWF_UNCACHED /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -354,7 +355,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_UNCACHED, "UNCACHED" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..dc77cd8ae1a3 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_UNCACHED ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_UNCACHED) #define PROCFS_IOCTL_MAGIC 'f' diff --git a/mm/filemap.c b/mm/filemap.c index 7f8d13f06c04..6f65025782bb 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2471,6 +2471,8 @@ static int filemap_create_folio(struct kiocb *iocb, folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_UNCACHED) + folio_set_uncached(folio); /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2516,6 +2518,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2545,6 +2549,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2705,8 +2711,16 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, } } put_folios: - for (i = 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + + if (folio_test_uncached(folio)) { + folio_lock(folio); + invalidate_complete_folio2(mapping, folio, 0); + folio_unlock(folio); + } + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); diff --git a/mm/swap.c b/mm/swap.c index 835bdf324b76..f2457acae383 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -472,6 +472,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_uncached(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; From patchwork Fri Nov 8 17:43:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868789 Received: from mail-oi1-f178.google.com (mail-oi1-f178.google.com [209.85.167.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FD0417A5BE for ; Fri, 8 Nov 2024 17:45:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087928; cv=none; b=Y5hsmJCKpbY7C3d5gU6RqBDaTKmE2LyNXgyTDF7H9JoNZ6/S9keFTeoV4A9Yrbm3SjgMc6VqK/3cdvAj+I8QcNRVXSYxknhg0ejGj1Dn+mUTNaAovD5YkqMsfuDFhvCYtPWv7XHqKjqCbERK+hM+fWn178hulrfIWqJ+Uc0uT8E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087928; c=relaxed/simple; bh=rnoMCl/wzF1Rdw/7LWpkKe6sGAyHqTjLz9AhJ+JYxUE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dEESsZwQty+RC90//k6+uuJFxn9zuTtj+nh7z76RABVnCz/maBQ6ze8+NXJg50HWZRAUG4lqg0LOEQMx9/gWGY9YljwViPzAAOwNweQF9prRKfs3erHJYfDG/STHy74SEiW7n3rUjaTtWqhgd4BOmM4qLOIxS2OG27PNosNJa18= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=vARexW2p; arc=none smtp.client-ip=209.85.167.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="vARexW2p" Received: by mail-oi1-f178.google.com with SMTP id 5614622812f47-3e6075cba82so1465551b6e.0 for ; Fri, 08 Nov 2024 09:45:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087926; x=1731692726; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8Os07YYPQ7kZ+kUIdYmg3svY/CQbT8cCz0FxxTNIj/Y=; b=vARexW2pfrv6PWUi1sYj02+pH5jThAbg0j8BBW0Rd5hZgmJIId6r5Z63/A1tVYSKk/ aEv60ttUDKkrJnunQZlglUyGpb9AhSiPuJXGuX/dCqnvRVXX2iM5lTgawdakRdkLuEbB GlLy4yZsDI0txpI5qtrNjwNyQ38hsw7XPkOZ15waCh56JdUMV+BMeafJ6gjVRMGCsv19 WDoxouakQV7w6b1BbYxe8Vo5JijxNKj6PYHxK/oPriaKGcPyaIZjMcZ1t52N49smD0wU TrzVyxZRGuGPPX+PMb6tr8JurVBmjsEEMkaFOz5yxWLrOY3XKyMRC+g7OZtKoUi+TJQ2 95qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087926; x=1731692726; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8Os07YYPQ7kZ+kUIdYmg3svY/CQbT8cCz0FxxTNIj/Y=; b=wJMPBPiMndw9XwAmMDdBW22jBcKFhTvX2d0ebxcGG6TjJoUi79BQgCzQfiHwuMNPp2 bape0RmlqeUVowC3bmQTItaQPLv4vCJPIWxpemVXqM3PsZQbHI4cBiTyYbhqMWCXoNLM f6byApjk/5x+97mJfdRoMwgHqv6zJPF+oyIo0GwLOKQYj7rcDyBj7Hwy4XCvnLelUyCH vhaGUguDFHisHmS4JROvNleTwasFMu2nb3BbkKxuCXfy2IySoAtw+hPea2Ll1VXQXAQ5 YJbBUHLitE85lVn0Wy7PSBhlVYZ4N/6tSZD7j05p1/m9gZyDS2dTsy1C/vpWdoMN+HWP JR6g== X-Forwarded-Encrypted: i=1; AJvYcCXPNqfsu/rJERI3e3IRTwLY/2yLESDEBaXzt32wlRYh6KRkExsT4RwDEWRmKJzm0ktlI39wJ/xTsuWpT6cv@vger.kernel.org X-Gm-Message-State: AOJu0YxqBr55pWO44yv8YeBXIcdFh/xyS5aZZn/fb/TK1mkkgnCg19wO kaYde2uQ2J8b/z6nExpe7wTBjsklZhj+yrNBr7gfgJYVCKopdJMhdkTE4duFr7w= X-Google-Smtp-Source: AGHT+IEK3I8pTmLVAcP1tiUKABdaXPcTEUhdzcwYlEN/XSWu4cE0qIXfxmg72hsMc3+DTSEsR4vUEg== X-Received: by 2002:a05:6808:120e:b0:3e0:4db9:8c44 with SMTP id 5614622812f47-3e79470a35bmr4931161b6e.27.1731087926068; Fri, 08 Nov 2024 09:45:26 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:25 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 09/13] mm: drop uncached pages when writeback completes Date: Fri, 8 Nov 2024 10:43:32 -0700 Message-ID: <20241108174505.1214230-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If the folio is marked as uncached, drop pages when writeback completes. Intended to be used with RWF_UNCACHED, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 6f65025782bb..1e455ca872b5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1634,6 +1634,18 @@ void folio_end_writeback(struct folio *folio) if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + /* + * If folio is marked as uncached, then pages should be dropped when + * writeback completes. Do that now. + */ + if (folio_test_uncached(folio)) { + folio_lock(folio); + if (invalidate_complete_folio2(folio->mapping, folio, 0)) + folio_clear_uncached(folio); + folio_unlock(folio); + + } folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); From patchwork Fri Nov 8 17:43:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868790 Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72F9B21EBA4 for ; Fri, 8 Nov 2024 17:45:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087930; cv=none; b=GWJIwfVYx7Crrn0ZncgcLL8vH+HORGKqbonrdLQ/L6yKlg/z1vysgHH9nKE9bm8LA+5fufGswEVP88H7lCrclKjneOnEPm7jPrZl8IB5onllLXNV6EApFDzbGYJizxb7sU7fsnZWrRrH+LmR9l5Ob1Xm/A8PllOw7fpSZd6ElJA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087930; c=relaxed/simple; bh=h/dX79Ug9Oi3tujE3kBQvUZhpb3YBGEldYjvJeJj7aQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=d5s7ZTkgIjQIgEXgdW7a+62drEJs22wm/5vyERjn8B23pnynl4XGMQCd+QHduzJEiEjSuojMuAF0xVyH+6HwDbdEnv4YXLD+rXjcbJba7HmzcpYkE2d0kj3xe67eAObYOz240/XCb6bNzE92U+YYi0GgyZNDiXXuKfu8yJfxft8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=VZU/qT8j; arc=none smtp.client-ip=209.85.167.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="VZU/qT8j" Received: by mail-oi1-f173.google.com with SMTP id 5614622812f47-3e5fbc40239so1444392b6e.3 for ; Fri, 08 Nov 2024 09:45:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087927; x=1731692727; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UUX+L5JwxoRfm7kNG+SFpzZJbOBH//rHSLdNBno0i1Q=; b=VZU/qT8jfM7UKd24qGemIn5WxcnpKaeCOMFNBN8OmayrRBIHFiIwd9+93PuaVMpQBu XMnWCpFPKTbt9mq+1A7H1kijoTdfbL4+T9rII7mrxs6ZoUbE1gp3MuLVfJrlX1gezUC3 yGoUItauFIepN7uRqMKR2p/W90YUDDOpDMjM0aGZlqbsHos3FDCgcfnlBjek900Hl9nF Z/1+IScrJ2TIKAAS/Evcva5gcA1/ANLNPnOUpzeSWt3AshRY54+Lc3WO+Qp4MYNSIS2z gecTAK7aqM7uIoOuf+LzHQXWJVncodbGpeRllHTGBGCftP9vclTsF0TakK2ZeRb/V22T k9BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087927; x=1731692727; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UUX+L5JwxoRfm7kNG+SFpzZJbOBH//rHSLdNBno0i1Q=; b=iPEclkQotTSXyXHox0fkMfvpqP+MfV8arVv2YEPPQ8TZtxhL7ntxU3Inwq0KXKYFxw shXRmUbAJKRozNTP8Q1fCciLUtE09XLYnUXcuOwUMv2ssgJJZqKUdqsRx/p2GpsZ/kGp +EXWysx72nv0yIeJ2n9oHV/FoEeiB6WaEuso6NhUywD1JkpsSQ+b0QNUKe7yMVz3gZu+ 6EdOxtQCDl9zSQ9bv6idhaxDJrEuoC6ysBzAiDLjfaFkED0b3eCZoQGrjWpQa3z8futQ J376eMT5fByTV2/p37UlSeYBI5fxgQRVo9OxLrpfEaNvsaEyPT/HsG9hXNGKSQGVWt1m 4NNw== X-Forwarded-Encrypted: i=1; AJvYcCXoHeatzLfrQAdgCLI4Lf+mge2uhm1j/7Kjr+hUx+KmV94crwyB/IvYnEuLtdBcfSEEGDggdq6KFeQqg3WV@vger.kernel.org X-Gm-Message-State: AOJu0YzOEJfWl+0U8ioKO3ps9CH9wdorspzsOvVhhMZFtZySsm5xY/YH wlUdHGSvOuyvA0bauwrPJya/wHXwd5oDvXjDvCAgxBePn30gA9IXqYeSYB4wH5o= X-Google-Smtp-Source: AGHT+IEeKABK38xxsKod0vZKixk/J7Qylyl5MmSX7mHW/jyfME5Ez4w2eu08IqUOJbkoLHt8qw1oZw== X-Received: by 2002:a05:6808:16a1:b0:3e7:63ce:1f26 with SMTP id 5614622812f47-3e7946a678fmr4543740b6e.24.1731087927472; Fri, 08 Nov 2024 09:45:27 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:26 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 10/13] mm/filemap: make buffered writes work with RWF_UNCACHED Date: Fri, 8 Nov 2024 10:43:33 -0700 Message-ID: <20241108174505.1214230-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If RWF_UNCACHED is set for a write, mark the folios being written with drop_writeback. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest. This provides similar benefits to using RWF_UNCACHED with reads. Testing buffered writes on 32 files: writing bs 65536, uncached 0 1s: 196035MB/sec, MB=196035 2s: 132308MB/sec, MB=328147 3s: 132438MB/sec, MB=460586 4s: 116528MB/sec, MB=577115 5s: 103898MB/sec, MB=681014 6s: 108893MB/sec, MB=789907 7s: 99678MB/sec, MB=889586 8s: 106545MB/sec, MB=996132 9s: 106826MB/sec, MB=1102958 10s: 101544MB/sec, MB=1204503 11s: 111044MB/sec, MB=1315548 12s: 124257MB/sec, MB=1441121 13s: 116031MB/sec, MB=1557153 14s: 114540MB/sec, MB=1671694 15s: 115011MB/sec, MB=1786705 16s: 115260MB/sec, MB=1901966 17s: 116068MB/sec, MB=2018034 18s: 116096MB/sec, MB=2134131 where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages. Running the same test with uncached buffered writes: writing bs 65536, uncached 1 1s: 198974MB/sec 2s: 189618MB/sec 3s: 193601MB/sec 4s: 188582MB/sec 5s: 193487MB/sec 6s: 188341MB/sec 7s: 194325MB/sec 8s: 188114MB/sec 9s: 192740MB/sec 10s: 189206MB/sec 11s: 193442MB/sec 12s: 189659MB/sec 13s: 191732MB/sec 14s: 190701MB/sec 15s: 191789MB/sec 16s: 191259MB/sec 17s: 190613MB/sec 18s: 191951MB/sec and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write. Signed-off-by: Jens Axboe --- mm/filemap.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 1e455ca872b5..d4c5928c5e2a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1610,6 +1610,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_uncached; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1631,6 +1633,7 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + folio_uncached = folio_test_clear_uncached(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); @@ -1639,12 +1642,10 @@ void folio_end_writeback(struct folio *folio) * If folio is marked as uncached, then pages should be dropped when * writeback completes. Do that now. */ - if (folio_test_uncached(folio)) { - folio_lock(folio); - if (invalidate_complete_folio2(folio->mapping, folio, 0)) - folio_clear_uncached(folio); + if (folio_uncached && folio_trylock(folio)) { + if (folio->mapping) + invalidate_complete_folio2(folio->mapping, folio, 0); folio_unlock(folio); - } folio_put(folio); } @@ -4082,6 +4083,9 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) if (unlikely(status < 0)) break; + if (iocb->ki_flags & IOCB_UNCACHED) + folio_set_uncached(folio); + offset = offset_in_folio(folio, pos); if (bytes > folio_size(folio) - offset) bytes = folio_size(folio) - offset; @@ -4122,6 +4126,12 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) if (!written) return status; + if (iocb->ki_flags & IOCB_UNCACHED) { + /* kick off uncached writeback, completion will drop it */ + __filemap_fdatawrite_range(mapping, iocb->ki_pos, + iocb->ki_pos + written, + WB_SYNC_NONE); + } iocb->ki_pos += written; return written; } From patchwork Fri Nov 8 17:43:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868791 Received: from mail-oi1-f171.google.com (mail-oi1-f171.google.com [209.85.167.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A43D721FD96 for ; Fri, 8 Nov 2024 17:45:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087931; cv=none; b=RaiYJfnZ9K8cci1FUn1yYu/GmRQKcWlLYoL5+/416paFPq+T9AXYEalN5i/M4qyPklXhlOLTY7UdBhVjOMNZsL5GwDJ0wDwzyAA9TaDlbcPGeH2/Y78KqS1vIffNQHowWn658ObNb6pb11uXT+BST1dnMkO0TjMih15iKXRhisA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087931; c=relaxed/simple; bh=9YW/2+OE7C3cVBnbovwG8Js+i3sqJ6a5WevlH1jOhvc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hJddOyjl401ZPVVgNgWY7ViAKxtJtb8GTOfihhBSK5Kry4/QCypQuw42/CpYVsCb6ehGEATfCN6u1n+BD7fxyIOSSDJd81lLic9Ax224nwuh7xXYuyGIRTuvPaP8BuXmM1mkmHxiZ7Rs230mxDf7J2d1vYdz/y6IKogSVzty8LA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=K0Cxxqlk; arc=none smtp.client-ip=209.85.167.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="K0Cxxqlk" Received: by mail-oi1-f171.google.com with SMTP id 5614622812f47-3e6104701ffso1630476b6e.0 for ; Fri, 08 Nov 2024 09:45:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087929; x=1731692729; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2fb9FZGyCFOFnaRiTxheAGhz9Js7yp85PwlVOXqpxuQ=; b=K0Cxxqlkqk+ExzN/m+CBwq2f7WR6hgN3h0jjnCXdk4QtblpXFkvnxFZU/lDZfx3z2S nxAshhOmbsLR7RK+VOHk6Nv8DP0y5JXxgmKaVT/wCnozFIgxSQurATQg7WxnM9AK+Fl4 JVKbImvJmKpOM6zm65mlXIXSLxky8djd+RMKEYTEEtmUUU5+JHAbPwXoW3UiKcsccFxx Yjhcg27E77gnsp+ytbc1yDPivqatcWu5YE2qiQ9IVVWiKI9SbiTOnl3OuVEaETJsfTSm E8No3ckE74dUom4T/k0MQYJ7/p2iUTa2YPL9UblsYNAUoXaunRf+a+KfwuUjJ6a9kWNo A3bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087929; x=1731692729; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2fb9FZGyCFOFnaRiTxheAGhz9Js7yp85PwlVOXqpxuQ=; b=HucIcQtUVIO+tlAvK1lLBD7Cmo6bwR6G1J1ej7JhM9Pxk8q7QJnc/Ln9/M9bcT30/5 CdXOBKtq824Io2QPvvzyJqkGkJeydbEK2UZVC2xgXYBfMnEZhB6mpvniCPuHeANMGNFV HaZE6789s6NyeiI4ERDRd9SpxymI4TQcVGJYvZKGmR22tTCtBusKy8FNN9PPlu7V6doo CPhWE5BKfuaZL6jtC0xGfto68q8dvwbbINlxDhLAZsqme/yOgFyEej8ZyOcqR/6LGq6n sNMEGyPPwHVDdee3yuo3zHcQ86gmtbHNslfAT0rOqtWp9SqN2Ww+BLQSOeh/6PAfaiow LBlw== X-Forwarded-Encrypted: i=1; AJvYcCV+R1oOOqVpMfqtJKZTS99nuygLU//fa/UdFEugiK2nQVifrLMH+OCOl0UVgP7cF+AQ7tRG9zcwK/sF9J2x@vger.kernel.org X-Gm-Message-State: AOJu0Yx3Cg443dEwhY+s19XvV/R21szzFSSOcFqMltxdShFjYGZPeMJs D2Hzf+a7KmXiX5YI5RJYt+NwOd/v8VToSxEbwacX1n94AfypjFnK6MNpUPF3kHI= X-Google-Smtp-Source: AGHT+IH8lOHs5F7JgjBYwdnat/pkPGU3i89izTiRydGTFEpVaYP+S65Vq9chKUX3hmqFUZHNf6tOPg== X-Received: by 2002:a05:6808:1922:b0:3e6:61f0:4797 with SMTP id 5614622812f47-3e79477095cmr3867819b6e.40.1731087928735; Fri, 08 Nov 2024 09:45:28 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:28 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 11/13] iomap: make buffered writes work with RWF_UNCACHED Date: Fri, 8 Nov 2024 10:43:34 -0700 Message-ID: <20241108174505.1214230-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add iomap buffered write support for RWF_UNCACHED. If RWF_UNCACHED is set for a write, mark the folios being written with drop_writeback. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest. See the similar patch for the generic filemap handling for performance results, those were in fact done on XFS using this patch. Signed-off-by: Jens Axboe --- fs/iomap/buffered-io.c | 12 +++++++++++- include/linux/iomap.h | 3 ++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index ef0b68bccbb6..609256885094 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -959,6 +959,8 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) } if (iter->iomap.flags & IOMAP_F_STALE) break; + if (iter->flags & IOMAP_UNCACHED) + folio_set_uncached(folio); offset = offset_in_folio(folio, pos); if (bytes > folio_size(folio) - offset) @@ -1023,8 +1025,9 @@ ssize_t iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, const struct iomap_ops *ops, void *private) { + struct address_space *mapping = iocb->ki_filp->f_mapping; struct iomap_iter iter = { - .inode = iocb->ki_filp->f_mapping->host, + .inode = mapping->host, .pos = iocb->ki_pos, .len = iov_iter_count(i), .flags = IOMAP_WRITE, @@ -1034,12 +1037,19 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, if (iocb->ki_flags & IOCB_NOWAIT) iter.flags |= IOMAP_NOWAIT; + if (iocb->ki_flags & IOCB_UNCACHED) + iter.flags |= IOMAP_UNCACHED; while ((ret = iomap_iter(&iter, ops)) > 0) iter.processed = iomap_write_iter(&iter, i); if (unlikely(iter.pos == iocb->ki_pos)) return ret; + if (iocb->ki_flags & IOCB_UNCACHED) { + /* kick off uncached writeback, completion will drop it */ + __filemap_fdatawrite_range(mapping, iocb->ki_pos, iter.pos, + WB_SYNC_NONE); + } ret = iter.pos - iocb->ki_pos; iocb->ki_pos = iter.pos; return ret; diff --git a/include/linux/iomap.h b/include/linux/iomap.h index f61407e3b121..89b24fbb1399 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -173,8 +173,9 @@ struct iomap_folio_ops { #define IOMAP_NOWAIT (1 << 5) /* do not block */ #define IOMAP_OVERWRITE_ONLY (1 << 6) /* only pure overwrites allowed */ #define IOMAP_UNSHARE (1 << 7) /* unshare_file_range */ +#define IOMAP_UNCACHED (1 << 8) /* uncached IO */ #ifdef CONFIG_FS_DAX -#define IOMAP_DAX (1 << 8) /* DAX mapping */ +#define IOMAP_DAX (1 << 9) /* DAX mapping */ #else #define IOMAP_DAX 0 #endif /* CONFIG_FS_DAX */ From patchwork Fri Nov 8 17:43:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868792 Received: from mail-oi1-f178.google.com (mail-oi1-f178.google.com [209.85.167.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F6AE21FDB7 for ; Fri, 8 Nov 2024 17:45:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087932; cv=none; b=CmMDQNAxGoQSfGylABvvXuXB4kQ/nFtRjnyfW+BzlqfR4fo2Iy/rDAXvGfoVc5Renk1+sXR3Auiw5yGTbaFFMdHlb4U+QV9ffawLpRCPc/Fk4oRn9JNW1xqKtlGDc5rvO33FieHK/EH9S9xGpK/4W4NISIgyF65cvzWNPB0lwGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087932; c=relaxed/simple; bh=1VnlS1RCwZciciCsk6/J6hl4e+sRw9JmzWzfIz5VZZI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eg71ypiBjl/tE4YnzSQJI3u1aTq8eCVBi6+WgKmYNpENp307o1+fJxYLEc4qvN5pie+C0wCBoQe7AT7ejWaX9C+ii2+iDCbRFAd4qa3Kx1HbOaINO0A85cfWT8UbfkYS4f57fTqaGWgWz4NPnWUj2B3VU15Ap4mwjhvm/UVJ5Ns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=HZMR+K+U; arc=none smtp.client-ip=209.85.167.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="HZMR+K+U" Received: by mail-oi1-f178.google.com with SMTP id 5614622812f47-3e5f968230bso1270917b6e.3 for ; Fri, 08 Nov 2024 09:45:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087930; x=1731692730; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VQL0Yvui1/uihQbO1amlpE3hTYTghb1QS2rKG5UGAbk=; b=HZMR+K+UmDqm6puflnCGMy7nlolPnAyfUPgt8uND2tZ10eFGflNNgGQkNWw1aIurm9 LcnzgQCdQzPsuYs93NhwYqgnQjG5zSzHtpykr4KJaRlO6ksiZHvo7KI1LMvfKQuLeBYC 7KyX+yjhNh6jX4AGArhnXPb7bFMtOeiHLHSDgbGlv8Ir+fj5WwxFrqJEpceihCDELc6k ieDX9f9zMoKIDho0vcRTLkVXGE+Jm3ZP8BH7mybdJl3mcGhyW2l4+sUjYv+dc3SSxcaQ sRCJzyp6Zb1hCPSeWTJTfxn7/BAx12kiIpbszMcRW/JF2reHktKeJAVUoRD7y/LFJ33y ndEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087930; x=1731692730; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VQL0Yvui1/uihQbO1amlpE3hTYTghb1QS2rKG5UGAbk=; b=vx7dtPzX2o74NmiNOSe+00sYawC03GzVN1HJF75Tu7A7H9PI6oFEyJB2okInP9KdDf 5I3Vyd41XLSIWyqX7S99eQWu1whi5ud0I5StGYGtAJkdGN1N0o4SMwsaHO2Kv/IK9O8w E5hHJXdW1VvYcYMJ9H7K2R4Tr6Ye7i8xAFdr5eNkRBLoirfh8nYLg+78gpPljdDc4h58 QDkoVZ6F+N3H150o2vhcoq9giQGvjDstZm1C9kukDdCLkIfgI2oFBr7N1h+498x9KEdz gIX/WYfhc9gNArGZTJ09yUak3t70+fZCakE0f5V0bLNYuGoJasrwKyXh2ykEKK/P8lQc fTyw== X-Forwarded-Encrypted: i=1; AJvYcCUMX2mzgsl9fc6Bpk2R3r8K9Yxw3cuFI9XurVrDyOcxCIa+m8ukpx61qAeBQ+zkLcFQtMWHLznC0Zx2FlOY@vger.kernel.org X-Gm-Message-State: AOJu0YyUGjxpCGzodKxlKSjd83hmUowjEKWumpodrsL4SAG09p4FKw98 z9nsdgPpxUOywJRG5z0LbiZQnuMdV69QXOW5zf3YDldOvrlvVP3My4f1rl3nYoA= X-Google-Smtp-Source: AGHT+IF8148kIy+GR70FdSYcCZ0fiyujzQfL1vs9+4lT648aDmaAqJC/hHYFUZhM+2GwXQZ5gTEvug== X-Received: by 2002:a05:6808:f89:b0:3e6:62fa:6b04 with SMTP id 5614622812f47-3e794720349mr5084554b6e.32.1731087929940; Fri, 08 Nov 2024 09:45:29 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:29 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 12/13] ext4: flag as supporting FOP_UNCACHED Date: Fri, 8 Nov 2024 10:43:35 -0700 Message-ID: <20241108174505.1214230-13-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 ext4 uses the generic read/write paths, and can fully support FOP_UNCACHED. Set the flag to indicate support, enabling use of RWF_UNCACHED. Signed-off-by: Jens Axboe --- fs/ext4/file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index f14aed14b9cf..0ef39d738598 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -944,7 +944,7 @@ const struct file_operations ext4_file_operations = { .splice_write = iter_file_splice_write, .fallocate = ext4_fallocate, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_DIO_PARALLEL_WRITE, + FOP_DIO_PARALLEL_WRITE | FOP_UNCACHED, }; const struct inode_operations ext4_file_inode_operations = { From patchwork Fri Nov 8 17:43:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13868793 Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 183AA220D66 for ; Fri, 8 Nov 2024 17:45:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087933; cv=none; b=SpWSsv9LFTWpnmJFIGZcnGvhoMwMut1OdpCGaWLWetPFT3dVO+rojEm0KCWnFuXIh0pHNkU9vzKfMYc80qmrMSZGRPgfFDtqEclWcm6StNpkgwuMZLtlAl4pv7Qwp3IjyDdACekWXLEcovRKiMPQ2XbcBO1d7AJrB8culgjrMfM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731087933; c=relaxed/simple; bh=d/CoZrYwTsgzJr5kdSbJuWogGWKOW++I0TYgIw1Kixw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MeC7rryKqgecDRfU3c9S2VqYZ+WIr5BPD5YaZ4JSb096jjS6TC35MOpAw9wv3Ri3tGPX3IcRvECDaZycxcaANef0dNYAiyjdIC2WBlzOnj1ruOzPZKqlRMXvzLaYKmAa+2jJXTxK7Em/hRm4Vu0kBDqQ2NxRctnJoC3FyvkDB+E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=FGPBeUD9; arc=none smtp.client-ip=209.85.167.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="FGPBeUD9" Received: by mail-oi1-f170.google.com with SMTP id 5614622812f47-3e6075cba82so1465607b6e.0 for ; Fri, 08 Nov 2024 09:45:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731087931; x=1731692731; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6OLsZMH/yTrwPfT+lkkEOmz1bcJxf1X+T+9jhpWMtjM=; b=FGPBeUD9CWvRB8454/Ea0PAnycYBt8AlyFYEtmmRW3WW+259zUSK0hEFcnm49RvMwk E7GrOpFpN07yfuA3IghmeKc/PrqIltF/NxNAhd5toeLytE1NaGHh69gkFkdrWDgB875k /O+tFT96QZ6OUPric64N+CWkGvqaa2WbP18rH2OjnJV9ZIys5y37pwfApSI3o6W+Rtub r5BkwJT5etbx4K98QQ9nDwLfF2yUAMQydEnIu4VhL9LzQNrbNaz95c/89ATo6dTe/TnJ S6iaXGDj+T0I5CQ8BdoMbzFUFEPPjwu9trhzIiP4h4dZv3ujW8ETPd4l8YWopkgOT9z7 su6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731087931; x=1731692731; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6OLsZMH/yTrwPfT+lkkEOmz1bcJxf1X+T+9jhpWMtjM=; b=IyYbqS/sWpv4B1zPdwQlcQVMjs9LjPEw1JmgfIDolpu0lA77Dfc1BGFzVWkkmxiQW6 ASTK22Kv0B7GwsCXYatuXpePahd+Ew9LBEBSMv2oi5V3KEV72f92pCXQyVhikDRcP3t5 2qpHMnvRGQmkcwbcrjEf1mFYlMYh4cQUDzxcqFi2XqYzNj8C6M1IhA6RGj39kcUUFXQc LS23qI9jy+2gjYV+7Wk7sV4W422XRNdrDCmbwqPhyCZJTaxTvT7reFf/meBaDiIZvmwe svoAzUu+gk12YFbjD8C7g1R3tWS2jgdSDeX8VID8JsYecWTc/8dk2ruLBEUCVueBDXMC P4uw== X-Forwarded-Encrypted: i=1; AJvYcCWG07Q8RCgC/Svy+qc4OS2pGDpVv4VGlRBdfy0S+z3Xp8pasYLYDWGDjHCdPeQ5/vwRUSStgbgK+0toC4bE@vger.kernel.org X-Gm-Message-State: AOJu0YzKZZfaZJiygBTrIXzgcOx8TxkvHDHRHWNg4klh06hUH8rwCAmo P1YoGyF+TujHMm9h+anzHaQk8i32rgt7XUUWDKaoQzXcL/FCCu2yS23I2ag3Ta8= X-Google-Smtp-Source: AGHT+IHwGGg69OZHlPE1gXTTmbyei/EPohHpGXOl/6pBBWBH6HgFHFaQI6UOOM7LN2ifcEJcud5MBQ== X-Received: by 2002:a05:6808:219f:b0:3e6:366f:8e3b with SMTP id 5614622812f47-3e79475a747mr4137297b6e.39.1731087931319; Fri, 08 Nov 2024 09:45:31 -0800 (PST) Received: from localhost.localdomain ([96.43.243.2]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3e78cd28f80sm780969b6e.39.2024.11.08.09.45.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 09:45:30 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, Jens Axboe Subject: [PATCH 13/13] xfs: flag as supporting FOP_UNCACHED Date: Fri, 8 Nov 2024 10:43:36 -0700 Message-ID: <20241108174505.1214230-14-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241108174505.1214230-1-axboe@kernel.dk> References: <20241108174505.1214230-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 iomap supports uncached IO, enable the use of RWF_UNCACHED with XFS by flagging support with FOP_UNCACHED. Signed-off-by: Jens Axboe --- fs/xfs/xfs_file.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index b19916b11fd5..4fe593896bc5 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -1595,7 +1595,8 @@ const struct file_operations xfs_file_operations = { .fadvise = xfs_file_fadvise, .remap_file_range = xfs_file_remap_range, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE, + FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE | + FOP_UNCACHED, }; const struct file_operations xfs_dir_file_operations = {