From patchwork Fri May 20 18:36:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 502A1C43217 for ; Fri, 20 May 2022 18:37:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5C3A6B0072; Fri, 20 May 2022 14:37:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE5B76B0073; Fri, 20 May 2022 14:37:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5EBB6B0074; Fri, 20 May 2022 14:37:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A49DF6B0072 for ; Fri, 20 May 2022 14:37:28 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8018835437 for ; Fri, 20 May 2022 18:37:28 +0000 (UTC) X-FDA: 79486979376.11.E00126E Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf23.hostedemail.com (Postfix) with ESMTP id 43101140003 for ; Fri, 20 May 2022 18:37:09 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24KHSRRA022480 for ; Fri, 20 May 2022 11:37:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=3zHDjbn72NXSBweb9g6a0xThug/NIwNbovojkJVDB2M=; b=KlpcVBpG1o56WU6V/9+d3pMGsk6wUOQrEUofWrF5HnYQnK+igDdtOfqmgUe0JgLP8v37 MAMknrlhstM4JbckkOK8j7n1/XC39hht0HQCqVO5CgGTmcr61zBzbYzfCtY2rddzpo9p NOgTlVSCNxsibeUh0VAS6sCxAh1Sh1WpUcc= Received: from mail.thefacebook.com ([163.114.132.120]) by m0089730.ppops.net (PPS) with ESMTPS id 3g6bja2auf-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:27 -0700 Received: from twshared14818.18.frc3.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:21d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:25 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 61888F5E5B21; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 01/17] mm: Move starting of background writeback into the main balancing loop Date: Fri, 20 May 2022 11:36:30 -0700 Message-ID: <20220520183646.2002023-2-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: N_euCbccNNqWUowqXP9xFR1kcQBO_Wjq X-Proofpoint-ORIG-GUID: N_euCbccNNqWUowqXP9xFR1kcQBO_Wjq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_05,2022-05-20_02,2022-02-23_01 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=KlpcVBpG; spf=none (imf23.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspam-User: X-Stat-Signature: gxcfybp3hy9udthf838pe887orzforwj X-Rspamd-Queue-Id: 43101140003 X-Rspamd-Server: rspam01 X-HE-Tag: 1653071829-6289 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jan Kara We start background writeback if we are over background threshold after exiting the main loop in balance_dirty_pages(). This may result in basing the decision on already stale values (we may have slept for significant amount of time) and it is also inconvenient for refactoring needed for async dirty throttling. Move the check into the main waiting loop. Signed-off-by: Jan Kara Signed-off-by: Stefan Roesch --- mm/page-writeback.c | 31 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 7e2da284e427..8e5e003f0093 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1618,6 +1618,19 @@ static void balance_dirty_pages(struct bdi_writeback *wb, } } + /* + * In laptop mode, we wait until hitting the higher threshold + * before starting background writeout, and then write out all + * the way down to the lower threshold. So slow writers cause + * minimal disk activity. + * + * In normal mode, we start background writeout at the lower + * background_thresh, to keep the amount of dirty memory low. + */ + if (!laptop_mode && nr_reclaimable > gdtc->bg_thresh && + !writeback_in_progress(wb)) + wb_start_background_writeback(wb); + /* * Throttle it only when the background writeback cannot * catch-up. This avoids (excessively) small writeouts @@ -1648,6 +1661,7 @@ static void balance_dirty_pages(struct bdi_writeback *wb, break; } + /* Start writeback even when in laptop mode */ if (unlikely(!writeback_in_progress(wb))) wb_start_background_writeback(wb); @@ -1814,23 +1828,6 @@ static void balance_dirty_pages(struct bdi_writeback *wb, if (!dirty_exceeded && wb->dirty_exceeded) wb->dirty_exceeded = 0; - - if (writeback_in_progress(wb)) - return; - - /* - * In laptop mode, we wait until hitting the higher threshold before - * starting background writeout, and then write out all the way down - * to the lower threshold. So slow writers cause minimal disk activity. - * - * In normal mode, we start background writeout at the lower - * background_thresh, to keep the amount of dirty memory low. - */ - if (laptop_mode) - return; - - if (nr_reclaimable > gdtc->bg_thresh) - wb_start_background_writeback(wb); } static DEFINE_PER_CPU(int, bdp_ratelimits); From patchwork Fri May 20 18:36:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14349C433F5 for ; Fri, 20 May 2022 18:37:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CA096B0071; Fri, 20 May 2022 14:37:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84FD46B0072; Fri, 20 May 2022 14:37:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A2756B0073; Fri, 20 May 2022 14:37:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 53B5C6B0071 for ; Fri, 20 May 2022 14:37:25 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2582935226 for ; Fri, 20 May 2022 18:37:25 +0000 (UTC) X-FDA: 79486979250.23.4088B75 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf25.hostedemail.com (Postfix) with ESMTP id 1A3B3A0017 for ; Fri, 20 May 2022 18:36:58 +0000 (UTC) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSPmr018150 for ; Fri, 20 May 2022 11:37:22 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=JmNWcJ9S5BuVrDagcKBT8mAi6jQ8Go8fzpG+AeBXTsU=; b=qUZzpfK27enxbJD8TbCXOSUkdo1jEz02hR2C3TBvdksrMSz99UxRgez26GTav6E7cqw3 hX9IIvDPxZwqHQMdsEwgYhTQd9x5MiVa3Q98fZQHM/1cLrEqFBEU0tv/JHtBkrHu8XK3 hmnN4ljOPLE18Rah/iBdaPiKJVNZzyJI3Rs= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g5xexdvw8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:22 -0700 Received: from twshared24024.25.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:21 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 64B10F5E5B23; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 02/17] mm: Move updates of dirty_exceeded into one place Date: Fri, 20 May 2022 11:36:31 -0700 Message-ID: <20220520183646.2002023-3-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: u5CjVyJlv9358gOCN1Kpa3GXzUNl5lwQ X-Proofpoint-GUID: u5CjVyJlv9358gOCN1Kpa3GXzUNl5lwQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_05,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 1A3B3A0017 X-Stat-Signature: p8sdhipype3t1mkfnuoc55d6jexkzpcx X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=qUZzpfK2; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf25.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=6139886598=shr@fb.com" X-HE-Tag: 1653071818-728188 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jan Kara Transition of wb->dirty_exceeded from 0 to 1 happens before we go to sleep in balance_dirty_pages() while transition from 1 to 0 happens when exiting from balance_dirty_pages(), possibly based on old values. This does not make a lot of sense since wb->dirty_exceeded should simply reflect whether wb is over dirty limit and so we should ratelimit entering to balance_dirty_pages() less. Move the two updates together. Signed-off-by: Jan Kara Signed-off-by: Stefan Roesch --- mm/page-writeback.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 8e5e003f0093..89dcc7d8395a 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1720,8 +1720,8 @@ static void balance_dirty_pages(struct bdi_writeback *wb, sdtc = mdtc; } - if (dirty_exceeded && !wb->dirty_exceeded) - wb->dirty_exceeded = 1; + if (dirty_exceeded != wb->dirty_exceeded) + wb->dirty_exceeded = dirty_exceeded; if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + BANDWIDTH_INTERVAL)) @@ -1825,9 +1825,6 @@ static void balance_dirty_pages(struct bdi_writeback *wb, if (fatal_signal_pending(current)) break; } - - if (!dirty_exceeded && wb->dirty_exceeded) - wb->dirty_exceeded = 0; } static DEFINE_PER_CPU(int, bdp_ratelimits); From patchwork Fri May 20 18:36:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7BD6C4167D for ; Fri, 20 May 2022 18:37:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD02E6B0078; Fri, 20 May 2022 14:37:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D2F858D0001; Fri, 20 May 2022 14:37:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98A916B0080; Fri, 20 May 2022 14:37:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6D65E6B0078 for ; Fri, 20 May 2022 14:37:34 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4CD1035226 for ; Fri, 20 May 2022 18:37:34 +0000 (UTC) X-FDA: 79486979628.22.72CEA81 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf29.hostedemail.com (Postfix) with ESMTP id 184F9120006 for ; Fri, 20 May 2022 18:37:23 +0000 (UTC) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSOQG018095 for ; Fri, 20 May 2022 11:37:32 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=NA8OmBU4jiEnjso+uqEe9kZ6Ah3dTXtaz03vpcJX2KM=; b=BfxRSx44mZzcgBh/2gmckixCs+DHoNRq1wHb7rRiN4nNCyhugiHjJmS3dr/8jz0cnAEY DGJBPKFhc/uBEa7Kc/HceyjKqdWyv5zrs4PnEmGmAvunGadjfRXHQzK1Pshbudnkj93R lsGIUZsiB6FIPUli3RwcG7s+B6Wo9L5yico= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g5xexdvwn-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:32 -0700 Received: from twshared8307.18.frc3.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:21d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:26 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 6B6D3F5E5B25; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 03/17] mm: Prepare balance_dirty_pages() for async buffered writes Date: Fri, 20 May 2022 11:36:32 -0700 Message-ID: <20220520183646.2002023-4-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: -QdZs1eOpx-z6bdJgg26rmDAbsZ2fhyy X-Proofpoint-GUID: -QdZs1eOpx-z6bdJgg26rmDAbsZ2fhyy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 184F9120006 X-Stat-Signature: frgzrc3c4sr5mkw4jbnrxjirb63oisx4 X-Rspam-User: Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=BfxRSx44; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf29.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=6139886598=shr@fb.com" X-HE-Tag: 1653071843-363628 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jan Kara If balance_dirty_pages() gets called for async buffered write, we don't want to wait. Instead we need to indicate to the caller that throttling is needed so that it can stop writing and offload the rest of the write to a context that can block. Signed-off-by: Jan Kara Signed-off-by: Stefan Roesch --- include/linux/writeback.h | 4 ++++ mm/page-writeback.c | 12 +++++++++--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/include/linux/writeback.h b/include/linux/writeback.h index fec248ab1fec..a9114c5090e9 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -372,6 +372,10 @@ void global_dirty_limits(unsigned long *pbackground, unsigned long *pdirty); unsigned long wb_calc_thresh(struct bdi_writeback *wb, unsigned long thresh); void wb_update_bandwidth(struct bdi_writeback *wb); + +/* Invoke balance dirty pages in async mode. */ +#define BDP_ASYNC 0x0001 + void balance_dirty_pages_ratelimited(struct address_space *mapping); bool wb_over_bg_thresh(struct bdi_writeback *wb); diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 89dcc7d8395a..7a320fd2ad33 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1545,8 +1545,8 @@ static inline void wb_dirty_limits(struct dirty_throttle_control *dtc) * If we're over `background_thresh' then the writeback threads are woken to * perform some writeout. */ -static void balance_dirty_pages(struct bdi_writeback *wb, - unsigned long pages_dirtied) +static int balance_dirty_pages(struct bdi_writeback *wb, + unsigned long pages_dirtied, unsigned int flags) { struct dirty_throttle_control gdtc_stor = { GDTC_INIT(wb) }; struct dirty_throttle_control mdtc_stor = { MDTC_INIT(wb, &gdtc_stor) }; @@ -1566,6 +1566,7 @@ static void balance_dirty_pages(struct bdi_writeback *wb, struct backing_dev_info *bdi = wb->bdi; bool strictlimit = bdi->capabilities & BDI_CAP_STRICTLIMIT; unsigned long start_time = jiffies; + int ret = 0; for (;;) { unsigned long now = jiffies; @@ -1794,6 +1795,10 @@ static void balance_dirty_pages(struct bdi_writeback *wb, period, pause, start_time); + if (flags & BDP_ASYNC) { + ret = -EAGAIN; + break; + } __set_current_state(TASK_KILLABLE); wb->dirty_sleep = now; io_schedule_timeout(pause); @@ -1825,6 +1830,7 @@ static void balance_dirty_pages(struct bdi_writeback *wb, if (fatal_signal_pending(current)) break; } + return ret; } static DEFINE_PER_CPU(int, bdp_ratelimits); @@ -1906,7 +1912,7 @@ void balance_dirty_pages_ratelimited(struct address_space *mapping) preempt_enable(); if (unlikely(current->nr_dirtied >= ratelimit)) - balance_dirty_pages(wb, current->nr_dirtied); + balance_dirty_pages(wb, current->nr_dirtied, 0); wb_put(wb); } From patchwork Fri May 20 18:36:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E348EC433F5 for ; Fri, 20 May 2022 18:37:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 608DF6B0073; Fri, 20 May 2022 14:37:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 591F66B0074; Fri, 20 May 2022 14:37:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40B176B0075; Fri, 20 May 2022 14:37:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2F5C26B0073 for ; Fri, 20 May 2022 14:37:32 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0740D21D3D for ; Fri, 20 May 2022 18:37:31 +0000 (UTC) X-FDA: 79486979544.03.7A547D3 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf25.hostedemail.com (Postfix) with ESMTP id E5B8AA00C8 for ; Fri, 20 May 2022 18:37:05 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24KHSQ5A022413 for ; Fri, 20 May 2022 11:37:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=4+pNNcv/s7lwCe+89++WipKGwJWYuAjDxvNindOohPA=; b=JU9hG2Y4vcXbFkMMX7XUkpXndS7kbihL0F0ALOTtvizu6TxLA3oue1fzEP9D9zNhKsXg ybCIYG/Aeen2zEayM/MUVt/8hsUi1PHi6aiZBcCQEobZR1ZwJBJXLgBpG3hDha05hbro 9g7pcxDqu6T3amNZx/YKJRQHIhHSTgSS1hM= Received: from mail.thefacebook.com ([163.114.132.120]) by m0089730.ppops.net (PPS) with ESMTPS id 3g6bja2aus-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:30 -0700 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub101.TheFacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:28 -0700 Received: from twshared6696.05.ash7.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:27 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 71B86F5E5B27; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 04/17] mm: Add balance_dirty_pages_ratelimited_flags() function Date: Fri, 20 May 2022 11:36:33 -0700 Message-ID: <20220520183646.2002023-5-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: O-ljwBmY1hnbPwDZZa1KkFRuB_t43HlX X-Proofpoint-ORIG-GUID: O-ljwBmY1hnbPwDZZa1KkFRuB_t43HlX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_05,2022-05-20_02,2022-02-23_01 X-Rspamd-Queue-Id: E5B8AA00C8 X-Stat-Signature: hdpun5uingg1xaqaj6pj56uji9ri5n9d Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=JU9hG2Y4; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf25.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com" X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1653071825-888946 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the helper function balance_dirty_pages_ratelimited_flags(). It adds the parameter flags to balance_dirty_pages_ratelimited(). The flags parameter is passed to balance_dirty_pages(). For async buffered writes the flag value will be BDP_ASYNC. The new helper function is also used by balance_dirty_pages_ratelimited(). Signed-off-by: Stefan Roesch --- include/linux/writeback.h | 3 +++ mm/page-writeback.c | 38 +++++++++++++++++++++++--------------- 2 files changed, 26 insertions(+), 15 deletions(-) diff --git a/include/linux/writeback.h b/include/linux/writeback.h index a9114c5090e9..1bddad86a4f6 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -377,6 +377,9 @@ void wb_update_bandwidth(struct bdi_writeback *wb); #define BDP_ASYNC 0x0001 void balance_dirty_pages_ratelimited(struct address_space *mapping); +int balance_dirty_pages_ratelimited_flags(struct address_space *mapping, + unsigned int flags); + bool wb_over_bg_thresh(struct bdi_writeback *wb); typedef int (*writepage_t)(struct page *page, struct writeback_control *wbc, diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 7a320fd2ad33..3701e813d05f 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1851,28 +1851,18 @@ static DEFINE_PER_CPU(int, bdp_ratelimits); */ DEFINE_PER_CPU(int, dirty_throttle_leaks) = 0; -/** - * balance_dirty_pages_ratelimited - balance dirty memory state - * @mapping: address_space which was dirtied - * - * Processes which are dirtying memory should call in here once for each page - * which was newly dirtied. The function will periodically check the system's - * dirty state and will initiate writeback if needed. - * - * Once we're over the dirty memory limit we decrease the ratelimiting - * by a lot, to prevent individual processes from overshooting the limit - * by (ratelimit_pages) each. - */ -void balance_dirty_pages_ratelimited(struct address_space *mapping) +int balance_dirty_pages_ratelimited_flags(struct address_space *mapping, + unsigned int flags) { struct inode *inode = mapping->host; struct backing_dev_info *bdi = inode_to_bdi(inode); struct bdi_writeback *wb = NULL; int ratelimit; + int ret = 0; int *p; if (!(bdi->capabilities & BDI_CAP_WRITEBACK)) - return; + return ret; if (inode_cgwb_enabled(inode)) wb = wb_get_create_current(bdi, GFP_KERNEL); @@ -1912,9 +1902,27 @@ void balance_dirty_pages_ratelimited(struct address_space *mapping) preempt_enable(); if (unlikely(current->nr_dirtied >= ratelimit)) - balance_dirty_pages(wb, current->nr_dirtied, 0); + balance_dirty_pages(wb, current->nr_dirtied, flags); wb_put(wb); + return ret; +} + +/** + * balance_dirty_pages_ratelimited - balance dirty memory state + * @mapping: address_space which was dirtied + * + * Processes which are dirtying memory should call in here once for each page + * which was newly dirtied. The function will periodically check the system's + * dirty state and will initiate writeback if needed. + * + * Once we're over the dirty memory limit we decrease the ratelimiting + * by a lot, to prevent individual processes from overshooting the limit + * by (ratelimit_pages) each. + */ +void balance_dirty_pages_ratelimited(struct address_space *mapping) +{ + balance_dirty_pages_ratelimited_flags(mapping, 0); } EXPORT_SYMBOL(balance_dirty_pages_ratelimited); From patchwork Fri May 20 18:36:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFF5EC433EF for ; Fri, 20 May 2022 18:37:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 547926B0074; Fri, 20 May 2022 14:37:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4CF4C6B0075; Fri, 20 May 2022 14:37:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D51B6B0078; Fri, 20 May 2022 14:37:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 15F8C6B0074 for ; Fri, 20 May 2022 14:37:33 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id D518981A7B for ; Fri, 20 May 2022 18:37:32 +0000 (UTC) X-FDA: 79486979544.20.999AB90 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf28.hostedemail.com (Postfix) with ESMTP id 0909EC00BF for ; Fri, 20 May 2022 18:37:03 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24KHSQ5E022413 for ; Fri, 20 May 2022 11:37:31 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=UmuWfsTck1J0e+hUP+BRgFN3i31rkvMhPWWM66De5po=; b=Fxg1hdn/CqtmxweUdGOsNRiU7DFAaXTzRjI1M4RAEd2Vp57hnS4ri7TqqNAcFy0lbWgj fRo923G5w2r1JngxnY2VTxxpE25dPWty/ZMDajHlOrwx9t58fc0tqmMc84juGE5fyaLI ck9eRifFCIZSGjB9GhHkkWEm6hLfCw+Ka6k= Received: from mail.thefacebook.com ([163.114.132.120]) by m0089730.ppops.net (PPS) with ESMTPS id 3g6bja2aus-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:31 -0700 Received: from twshared8508.05.ash9.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:30 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 77671F5E5B29; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 05/17] iomap: Add gfp parameter to iomap_page_create() Date: Fri, 20 May 2022 11:36:34 -0700 Message-ID: <20220520183646.2002023-6-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: DfGIXac-oTa_B42aM8NDm4Fysh2c3TOy X-Proofpoint-ORIG-GUID: DfGIXac-oTa_B42aM8NDm4Fysh2c3TOy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_05,2022-05-20_02,2022-02-23_01 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="Fxg1hdn/"; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf28.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com" X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0909EC00BF X-Stat-Signature: z1fgfzwyu9a5w736xr1j45ba5hxz1kcj X-HE-Tag: 1653071823-216797 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the gfp flags parameter to the function iomap_page_create() to be able to specify gfp flags. No intended functional changes in this patch. Signed-off-by: Stefan Roesch --- fs/iomap/buffered-io.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 8ce8720093b9..27e67bfc64f5 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -44,7 +44,7 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio) static struct bio_set iomap_ioend_bioset; static struct iomap_page * -iomap_page_create(struct inode *inode, struct folio *folio) +iomap_page_create(struct inode *inode, struct folio *folio, gfp_t gfp) { struct iomap_page *iop = to_iomap_page(folio); unsigned int nr_blocks = i_blocks_per_folio(inode, folio); @@ -52,8 +52,8 @@ iomap_page_create(struct inode *inode, struct folio *folio) if (iop || nr_blocks <= 1) return iop; - iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)), - GFP_NOFS | __GFP_NOFAIL); + iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)), gfp); + spin_lock_init(&iop->uptodate_lock); if (folio_test_uptodate(folio)) bitmap_fill(iop->uptodate, nr_blocks); @@ -226,7 +226,8 @@ static int iomap_read_inline_data(const struct iomap_iter *iter, if (WARN_ON_ONCE(size > iomap->length)) return -EIO; if (offset > 0) - iop = iomap_page_create(iter->inode, folio); + iop = iomap_page_create(iter->inode, folio, + GFP_NOFS | __GFP_NOFAIL); else iop = to_iomap_page(folio); @@ -264,7 +265,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter, return iomap_read_inline_data(iter, folio); /* zero post-eof blocks as the page may be mapped */ - iop = iomap_page_create(iter->inode, folio); + iop = iomap_page_create(iter->inode, folio, GFP_NOFS | __GFP_NOFAIL); iomap_adjust_read_range(iter->inode, folio, &pos, length, &poff, &plen); if (plen == 0) goto done; @@ -550,17 +551,20 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, size_t len, struct folio *folio) { const struct iomap *srcmap = iomap_iter_srcmap(iter); - struct iomap_page *iop = iomap_page_create(iter->inode, folio); + struct iomap_page *iop = to_iomap_page(folio); loff_t block_size = i_blocksize(iter->inode); loff_t block_start = round_down(pos, block_size); loff_t block_end = round_up(pos + len, block_size); size_t from = offset_in_folio(folio, pos), to = from + len; size_t poff, plen; + gfp_t gfp = GFP_NOFS | __GFP_NOFAIL; if (folio_test_uptodate(folio)) return 0; folio_clear_error(folio); + iop = iomap_page_create(iter->inode, folio, gfp); + do { iomap_adjust_read_range(iter->inode, folio, &block_start, block_end - block_start, &poff, &plen); @@ -1332,7 +1336,8 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, struct writeback_control *wbc, struct inode *inode, struct folio *folio, u64 end_pos) { - struct iomap_page *iop = iomap_page_create(inode, folio); + struct iomap_page *iop = iomap_page_create(inode, folio, + GFP_NOFS | __GFP_NOFAIL); struct iomap_ioend *ioend, *next; unsigned len = i_blocksize(inode); unsigned nblocks = i_blocks_per_folio(inode, folio); From patchwork Fri May 20 18:36:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2962CC433F5 for ; Fri, 20 May 2022 18:37:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA7EE6B0075; Fri, 20 May 2022 14:37:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A970D6B0078; Fri, 20 May 2022 14:37:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DD6D6B007E; Fri, 20 May 2022 14:37:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 61C3A6B0075 for ; Fri, 20 May 2022 14:37:34 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 3BAEE81A7E for ; Fri, 20 May 2022 18:37:34 +0000 (UTC) X-FDA: 79486979628.23.DE86C40 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf23.hostedemail.com (Postfix) with ESMTP id 03BE2140004 for ; Fri, 20 May 2022 18:37:14 +0000 (UTC) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24KHScIZ008454 for ; Fri, 20 May 2022 11:37:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=mtjtBSFBboij+dZeJUy8sfQn/qaPHn8tVpe9eVt5tec=; b=KEtuZFmnc7S13PpMRMlIu/rpwu7L3bcae8H0m3sa51fznvmCJ3fdBSAdtQ87LY2A+4ey /xvfmLgU4JaCvF7nN2C/Tof/yPAm07Yvu8tQpXHnlB4J82N4DUkWWFKtMtWyUnUr/dPq 5TA8/IiS5FWiFhs7HOwiXzJV5GEDVb7eqEw= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0001303.ppops.net (PPS) with ESMTPS id 3g5wkre7my-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:32 -0700 Received: from twshared4937.07.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:30 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 7D316F5E5B2B; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 06/17] iomap: Add async buffered write support Date: Fri, 20 May 2022 11:36:35 -0700 Message-ID: <20220520183646.2002023-7-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: Kfrx4VojBSaHPmHwv-hJUkgN512lg-xO X-Proofpoint-ORIG-GUID: Kfrx4VojBSaHPmHwv-hJUkgN512lg-xO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Stat-Signature: hixwrmcpr9hgbc6yq65tijz457o4ymhf X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=KEtuZFmn; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf23.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com" X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 03BE2140004 X-HE-Tag: 1653071834-728540 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds async buffered write support to iomap. Signed-off-by: Stefan Roesch --- fs/iomap/buffered-io.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 27e67bfc64f5..187f4ddd7ba7 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -555,15 +555,21 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, loff_t block_size = i_blocksize(iter->inode); loff_t block_start = round_down(pos, block_size); loff_t block_end = round_up(pos + len, block_size); + unsigned int nr_blocks = i_blocks_per_folio(iter->inode, folio); size_t from = offset_in_folio(folio, pos), to = from + len; size_t poff, plen; gfp_t gfp = GFP_NOFS | __GFP_NOFAIL; + if (iter->flags & IOMAP_NOWAIT) + gfp = GFP_NOWAIT; + if (folio_test_uptodate(folio)) return 0; folio_clear_error(folio); iop = iomap_page_create(iter->inode, folio, gfp); + if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1) + return -EAGAIN; do { iomap_adjust_read_range(iter->inode, folio, &block_start, @@ -581,7 +587,12 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, return -EIO; folio_zero_segments(folio, poff, from, to, poff + plen); } else { - int status = iomap_read_folio_sync(block_start, folio, + int status; + + if (iter->flags & IOMAP_NOWAIT) + return -EAGAIN; + + status = iomap_read_folio_sync(block_start, folio, poff, plen, srcmap); if (status) return status; @@ -610,6 +621,9 @@ static int iomap_write_begin(const struct iomap_iter *iter, loff_t pos, unsigned fgp = FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE | FGP_NOFS; int status = 0; + if (iter->flags & IOMAP_NOWAIT) + fgp |= FGP_NOWAIT; + BUG_ON(pos + len > iter->iomap.offset + iter->iomap.length); if (srcmap != &iter->iomap) BUG_ON(pos + len > srcmap->offset + srcmap->length); @@ -767,6 +781,10 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) * Otherwise there's a nasty deadlock on copying from the * same page as we're writing to, without it being marked * up-to-date. + * + * For async buffered writes the assumption is that the user + * page has already been faulted in. This can be optimized by + * faulting the user page in the prepare phase of io-uring. */ if (unlikely(fault_in_iov_iter_readable(i, bytes) == bytes)) { status = -EFAULT; @@ -822,6 +840,9 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, }; int ret; + if (iocb->ki_flags & IOCB_NOWAIT) + iter.flags |= IOMAP_NOWAIT; + while ((ret = iomap_iter(&iter, ops)) > 0) iter.processed = iomap_write_iter(&iter, i); if (iter.pos == iocb->ki_pos) From patchwork Fri May 20 18:36:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857268 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83911C41535 for ; Fri, 20 May 2022 18:37:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 45DAB8D0002; Fri, 20 May 2022 14:37:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C21A8D0001; Fri, 20 May 2022 14:37:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 265C38D0002; Fri, 20 May 2022 14:37:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D6A708D0001 for ; Fri, 20 May 2022 14:37:36 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9D2B161D34 for ; Fri, 20 May 2022 18:37:36 +0000 (UTC) X-FDA: 79486979712.03.8D072A2 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf30.hostedemail.com (Postfix) with ESMTP id 90EB680013 for ; Fri, 20 May 2022 18:37:10 +0000 (UTC) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSOQN018095 for ; Fri, 20 May 2022 11:37:35 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=wRT/wXmzHcgMsrcASISsv+kuIl+EmEXecNfLk2i1H+Q=; b=XZrWJQHbaazjv8esO8Kc/bK15skibCixwrQ7jpANZNLfbjUqwyCylvw4FH2ea8UKDX7J EhgVadqEQCPBd+S5SxZhF/mWIt8v7uAY48LtwCoSp0HDD1YnQ9zcb3/ltDtRiDzpm7YK nAs8YkoYA8YxIRYTEguvgDc6hhuDh9J++eE= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g5xexdvwn-11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:34 -0700 Received: from twshared35748.07.ash9.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:34 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 82E0DF5E5B2D; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 07/17] iomap: Use balance_dirty_pages_ratelimited_flags in iomap_write_iter Date: Fri, 20 May 2022 11:36:36 -0700 Message-ID: <20220520183646.2002023-8-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: azSrrCLHgRu6CqrGanCoffgWW-2uojar X-Proofpoint-GUID: azSrrCLHgRu6CqrGanCoffgWW-2uojar X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 90EB680013 X-Stat-Signature: hs6ouqqtr1itcgybxuyijrb63kh7a1nz X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=XZrWJQHb; spf=none (imf30.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653071830-709374 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This replaces the call to balance_dirty_pages_ratelimited() with the call to balance_dirty_pages_ratelimited_flags. This allows to specify if the write request is async or not. In addition this also moves the above function call to the beginning of the function. If the function call is at the end of the function and the decision is made to throttle writes, then there is no request that io-uring can wait on. By moving it to the beginning of the function, the write request is not issued, but returns -EAGAIN instead. io-uring will punt the request and process it in the io-worker. By moving the function call to the beginning of the function, the write throttling will happen one page later. Signed-off-by: Stefan Roesch --- fs/iomap/buffered-io.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 187f4ddd7ba7..020452467ca8 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -765,14 +765,22 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) do { struct folio *folio; struct page *page; + struct address_space *mapping = iter->inode->i_mapping; unsigned long offset; /* Offset into pagecache page */ unsigned long bytes; /* Bytes to write to page */ size_t copied; /* Bytes copied from user */ + unsigned int bdp_flags = + (iter->flags & IOMAP_NOWAIT) ? BDP_ASYNC : 0; offset = offset_in_page(pos); bytes = min_t(unsigned long, PAGE_SIZE - offset, iov_iter_count(i)); again: + status = balance_dirty_pages_ratelimited_flags(mapping, + bdp_flags); + if (unlikely(status)) + break; + if (bytes > length) bytes = length; @@ -796,7 +804,7 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) break; page = folio_file_page(folio, pos >> PAGE_SHIFT); - if (mapping_writably_mapped(iter->inode->i_mapping)) + if (mapping_writably_mapped(mapping)) flush_dcache_page(page); copied = copy_page_from_iter_atomic(page, offset, bytes, i); @@ -821,8 +829,6 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) pos += status; written += status; length -= status; - - balance_dirty_pages_ratelimited(iter->inode->i_mapping); } while (iov_iter_count(i) && length); return written ? written : status; From patchwork Fri May 20 18:36:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E5C2C46467 for ; Fri, 20 May 2022 18:37:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 806198D0005; Fri, 20 May 2022 14:37:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78C988D0001; Fri, 20 May 2022 14:37:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E0AD8D0005; Fri, 20 May 2022 14:37:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3B6408D0001 for ; Fri, 20 May 2022 14:37:40 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0B4C121D4F for ; Fri, 20 May 2022 18:37:40 +0000 (UTC) X-FDA: 79486979880.24.98C64CF Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf31.hostedemail.com (Postfix) with ESMTP id C2F4C2001B for ; Fri, 20 May 2022 18:37:08 +0000 (UTC) Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSK0c010815 for ; Fri, 20 May 2022 11:37:39 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=Z1kN9YKTgEbNSBC9L24GXepXSpkzS0zypoYUBMrIH50=; b=QQazOYI9Fg9iDNGiaOl286kJEqjBASOXM/UPxyBi+8pz4TglXnVDOyKWO3Wziql6WAzd ykjjVwikRsvXNudYOuqXbOm4nHEqKNt7Zr9gkF/p1U3f2Xt4l1YfyhPPzKHPFsKqqaxt M+QqeAMIS3as2Oashn2mfYiAV3T9INbrG+w= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g6341cjes-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:38 -0700 Received: from twshared6696.05.ash7.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:11d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:37 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 88DF4F5E5B2F; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 08/17] fs: Add check for async buffered writes to generic_write_checks Date: Fri, 20 May 2022 11:36:37 -0700 Message-ID: <20220520183646.2002023-9-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: u3de0NPP6MHWVoe6wdZYz0d1HGD9X__K X-Proofpoint-ORIG-GUID: u3de0NPP6MHWVoe6wdZYz0d1HGD9X__K X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: C2F4C2001B X-Stat-Signature: 4uhq7frw9yawenobgnxojynjis38pz1j X-Rspam-User: Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=QQazOYI9; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf31.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com" X-HE-Tag: 1653071828-485603 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This introduces the flag FMODE_BUF_WASYNC. If devices support async buffered writes, this flag can be set. It also modifies the check in generic_write_checks to take async buffered writes into consideration. Signed-off-by: Stefan Roesch --- fs/read_write.c | 4 +++- include/linux/fs.h | 3 +++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/read_write.c b/fs/read_write.c index e643aec2b0ef..175d98713b9a 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -1633,7 +1633,9 @@ int generic_write_checks_count(struct kiocb *iocb, loff_t *count) if (iocb->ki_flags & IOCB_APPEND) iocb->ki_pos = i_size_read(inode); - if ((iocb->ki_flags & IOCB_NOWAIT) && !(iocb->ki_flags & IOCB_DIRECT)) + if ((iocb->ki_flags & IOCB_NOWAIT) && + !((iocb->ki_flags & IOCB_DIRECT) || + (file->f_mode & FMODE_BUF_WASYNC))) return -EINVAL; return generic_write_check_limits(iocb->ki_filp, iocb->ki_pos, count); diff --git a/include/linux/fs.h b/include/linux/fs.h index bbde95387a23..3b479d02e210 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -177,6 +177,9 @@ typedef int (dio_iodone_t)(struct kiocb *iocb, loff_t offset, /* File supports async buffered reads */ #define FMODE_BUF_RASYNC ((__force fmode_t)0x40000000) +/* File supports async nowait buffered writes */ +#define FMODE_BUF_WASYNC ((__force fmode_t)0x80000000) + /* * Attribute flags. These should be or-ed together to figure out what * has been changed! From patchwork Fri May 20 18:36:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E46D8C4332F for ; Fri, 20 May 2022 18:37:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C70C28D0003; Fri, 20 May 2022 14:37:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BABEF8D0001; Fri, 20 May 2022 14:37:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 994EF8D0003; Fri, 20 May 2022 14:37:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 729EF8D0001 for ; Fri, 20 May 2022 14:37:38 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 43BA661AF6 for ; Fri, 20 May 2022 18:37:38 +0000 (UTC) X-FDA: 79486979796.21.AD29F5C Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf25.hostedemail.com (Postfix) with ESMTP id 5241AA00C4 for ; Fri, 20 May 2022 18:37:11 +0000 (UTC) Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHT2cT021415 for ; Fri, 20 May 2022 11:37:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=LjQjeu05KJ6rwhAZwVJii+/tlMhhh5HkUUbhHoJms8c=; b=gC1n0WZzBxoCH+wECZ6OHpXO2phJjsKw0caLbf5hcbDm/QvQFGWdItF+soh3rCPMVpw3 NV9utU1xIasPiv5kK8jKdtsEvOru5f4B1d2whSGYC8uACCB+Uns/faMnYgy6i33432xR D8nCGT0u8IIAyAPo6cVy1QnbDrDmavascTE= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g5rgj8vh7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:36 -0700 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub103.TheFacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:34 -0700 Received: from twshared8307.18.frc3.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:34 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 8E9BDF5E5B31; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 09/17] fs: Split off remove_needs_file_privs() __remove_file_privs() Date: Fri, 20 May 2022 11:36:38 -0700 Message-ID: <20220520183646.2002023-10-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: DviDX8p0HcI-235lVzqxay0Ozcvr5lpo X-Proofpoint-ORIG-GUID: DviDX8p0HcI-235lVzqxay0Ozcvr5lpo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5241AA00C4 X-Stat-Signature: 7nfyb91o4y44jn4p3w4nujyoobh14933 X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=gC1n0WZz; spf=none (imf25.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653071831-449542 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This splits off the function remove_needs_file_privs() from the function __remove_file_privs() from the function file_remove_privs(). No intended functional changes in this patch. Signed-off-by: Stefan Roesch --- fs/inode.c | 75 +++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 55 insertions(+), 20 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 9d9b422504d1..1bb8b7db836f 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2010,17 +2010,8 @@ static int __remove_privs(struct user_namespace *mnt_userns, return notify_change(mnt_userns, dentry, &newattrs, NULL); } -/* - * Remove special file priviledges (suid, capabilities) when file is written - * to or truncated. - */ -int file_remove_privs(struct file *file) +static int file_needs_remove_privs(struct inode *inode, struct dentry *dentry) { - struct dentry *dentry = file_dentry(file); - struct inode *inode = file_inode(file); - int kill; - int error = 0; - /* * Fast path for nothing security related. * As well for non-regular files, e.g. blkdev inodes. @@ -2030,16 +2021,42 @@ int file_remove_privs(struct file *file) if (IS_NOSEC(inode) || !S_ISREG(inode->i_mode)) return 0; - kill = dentry_needs_remove_privs(dentry); - if (kill < 0) - return kill; - if (kill) - error = __remove_privs(file_mnt_user_ns(file), dentry, kill); + return dentry_needs_remove_privs(dentry); +} + +static int __file_remove_privs(struct file *file, struct inode *inode, + struct dentry *dentry, int kill) +{ + int error = 0; + + error = __remove_privs(file_mnt_user_ns(file), dentry, kill); if (!error) inode_has_no_xattr(inode); return error; } + +/** + * file_remove_privs - remove special file privileges (suid, capabilities) + * @file: file to remove privileges from + * + * When file is modified by a write or truncation ensure that special + * file privileges are removed. + * + * Return: 0 on success, negative errno on failure. + */ +int file_remove_privs(struct file *file) +{ + struct dentry *dentry = file_dentry(file); + struct inode *inode = file_inode(file); + int kill; + + kill = file_needs_remove_privs(inode, dentry); + if (kill <= 0) + return kill; + + return __file_remove_privs(file, inode, dentry, kill); +} EXPORT_SYMBOL(file_remove_privs); /** @@ -2090,18 +2107,36 @@ int file_update_time(struct file *file) } EXPORT_SYMBOL(file_update_time); -/* Caller must hold the file's inode lock */ +/** + * file_modified - handle mandated vfs changes when modifying a file + * @file: file that was modified + * + * When file has been modified ensure that special + * file privileges are removed and time settings are updated. + * + * Context: Caller must hold the file's inode lock. + * + * Return: 0 on success, negative errno on failure. + */ int file_modified(struct file *file) { - int err; + int ret; + struct dentry *dentry = file_dentry(file); + struct inode *inode = file_inode(file); /* * Clear the security bits if the process is not being run by root. * This keeps people from modifying setuid and setgid binaries. */ - err = file_remove_privs(file); - if (err) - return err; + ret = file_needs_remove_privs(inode, dentry); + if (ret < 0) + return ret; + + if (ret > 0) { + ret = __file_remove_privs(file, inode, dentry, ret); + if (ret) + return ret; + } if (unlikely(file->f_mode & FMODE_NOCMTIME)) return 0; From patchwork Fri May 20 18:36:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BCE0C43219 for ; Fri, 20 May 2022 18:37:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AEB348D0007; Fri, 20 May 2022 14:37:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A775F8D0001; Fri, 20 May 2022 14:37:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C5DE8D0007; Fri, 20 May 2022 14:37:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 780568D0001 for ; Fri, 20 May 2022 14:37:50 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 5274F81A45 for ; Fri, 20 May 2022 18:37:50 +0000 (UTC) X-FDA: 79486980300.24.B6CD21B Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf16.hostedemail.com (Postfix) with ESMTP id 8100F180013 for ; Fri, 20 May 2022 18:37:37 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24KHSSLq022515 for ; Fri, 20 May 2022 11:37:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=4Gs4vhLGOpR5fcqmlVRzzwre6v2yRJXtVCNsF05UpCw=; b=piBNszL9xwhOP650YDcXtZMudcup701DSRWDsZyjXfYZDZRSeD/kD0RLr/nYfZ1Nonq4 +UHo0MRzqFrB7TX1uKCZDhWxUMyZ4Ry1/FrQH9Wj0/TR+Twls829cKZoqIBYCAgB+irI g0uSBPUp9TyPVvZ7sNNcJf67L+XUiBnCPPk= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0089730.ppops.net (PPS) with ESMTPS id 3g6bja2ayx-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:49 -0700 Received: from twshared11660.23.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:48 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 94D73F5E5B33; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 10/17] fs: Split off file_needs_update_time and __file_update_time Date: Fri, 20 May 2022 11:36:39 -0700 Message-ID: <20220520183646.2002023-11-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: QxcSgrS-dl3a5a_GTyVX6Re84tOPhNm_ X-Proofpoint-ORIG-GUID: QxcSgrS-dl3a5a_GTyVX6Re84tOPhNm_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_05,2022-05-20_02,2022-02-23_01 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=piBNszL9; spf=none (imf16.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspam-User: X-Stat-Signature: t46r5m6y8em5rkft49nkdkcgxex7fd9w X-Rspamd-Queue-Id: 8100F180013 X-Rspamd-Server: rspam01 X-HE-Tag: 1653071857-609929 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This splits off the functions file_needs_update_time() and __file_update_time() from the function file_update_time(). This is required to support async buffered writes. No intended functional changes in this patch. Signed-off-by: Stefan Roesch --- fs/inode.c | 75 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 49 insertions(+), 26 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 1bb8b7db836f..4bb7f583cc6b 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2059,35 +2059,19 @@ int file_remove_privs(struct file *file) } EXPORT_SYMBOL(file_remove_privs); -/** - * file_update_time - update mtime and ctime time - * @file: file accessed - * - * Update the mtime and ctime members of an inode and mark the inode - * for writeback. Note that this function is meant exclusively for - * usage in the file write path of filesystems, and filesystems may - * choose to explicitly ignore update via this function with the - * S_NOCMTIME inode flag, e.g. for network filesystem where these - * timestamps are handled by the server. This can return an error for - * file systems who need to allocate space in order to update an inode. - */ - -int file_update_time(struct file *file) +static int file_needs_update_time(struct inode *inode, struct file *file, + struct timespec64 *now) { - struct inode *inode = file_inode(file); - struct timespec64 now; int sync_it = 0; - int ret; /* First try to exhaust all avenues to not sync */ if (IS_NOCMTIME(inode)) return 0; - now = current_time(inode); - if (!timespec64_equal(&inode->i_mtime, &now)) + if (!timespec64_equal(&inode->i_mtime, now)) sync_it = S_MTIME; - if (!timespec64_equal(&inode->i_ctime, &now)) + if (!timespec64_equal(&inode->i_ctime, now)) sync_it |= S_CTIME; if (IS_I_VERSION(inode) && inode_iversion_need_inc(inode)) @@ -2096,15 +2080,49 @@ int file_update_time(struct file *file) if (!sync_it) return 0; - /* Finally allowed to write? Takes lock. */ - if (__mnt_want_write_file(file)) - return 0; + return sync_it; +} + +static int __file_update_time(struct inode *inode, struct file *file, + struct timespec64 *now, int sync_mode) +{ + int ret = 0; - ret = inode_update_time(inode, &now, sync_it); - __mnt_drop_write_file(file); + /* try to update time settings */ + if (!__mnt_want_write_file(file)) { + ret = inode_update_time(inode, now, sync_mode); + __mnt_drop_write_file(file); + } return ret; } + + /** + * file_update_time - update mtime and ctime time + * @file: file accessed + * + * Update the mtime and ctime members of an inode and mark the inode for + * writeback. Note that this function is meant exclusively for usage in + * the file write path of filesystems, and filesystems may choose to + * explicitly ignore updates via this function with the _NOCMTIME inode + * flag, e.g. for network filesystem where these imestamps are handled + * by the server. This can return an error for file systems who need to + * allocate space in order to update an inode. + * + * Return: 0 on success, negative errno on failure. + */ +int file_update_time(struct file *file) +{ + int ret; + struct inode *inode = file_inode(file); + struct timespec64 now = current_time(inode); + + ret = file_needs_update_time(inode, file, &now); + if (ret <= 0) + return ret; + + return __file_update_time(inode, file, &now, ret); +} EXPORT_SYMBOL(file_update_time); /** @@ -2123,6 +2141,7 @@ int file_modified(struct file *file) int ret; struct dentry *dentry = file_dentry(file); struct inode *inode = file_inode(file); + struct timespec64 now = current_time(inode); /* * Clear the security bits if the process is not being run by root. @@ -2141,7 +2160,11 @@ int file_modified(struct file *file) if (unlikely(file->f_mode & FMODE_NOCMTIME)) return 0; - return file_update_time(file); + ret = file_needs_update_time(inode, file, &now); + if (ret <= 0) + return ret; + + return __file_update_time(inode, file, &now, ret); } EXPORT_SYMBOL(file_modified); From patchwork Fri May 20 18:36:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857276 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A816AC433EF for ; Fri, 20 May 2022 18:37:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A025F8D000B; Fri, 20 May 2022 14:37:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A42D8D0008; Fri, 20 May 2022 14:37:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 742218D000B; Fri, 20 May 2022 14:37:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 514138D0008 for ; Fri, 20 May 2022 14:37:52 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3120B21D41 for ; Fri, 20 May 2022 18:37:52 +0000 (UTC) X-FDA: 79486980384.23.C31A2ED Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf23.hostedemail.com (Postfix) with ESMTP id F3487140021 for ; Fri, 20 May 2022 18:37:32 +0000 (UTC) Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSxUQ001127 for ; Fri, 20 May 2022 11:37:51 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=eRv8uAwss7XyRCk6NE/nwLa4/tzHUoenyM8a0++XiQI=; b=XQ08eS3KSB4jHhZUb7EAR5qmuBmfVV7iCeEhPidFWmvXxq6GHuvBsVPFCNn8qYXGjrim 4GDi8zZXS9MWfcjB60aSQnJBukfiaD09V01DDbE/soswPZ451P6BAUIImlBCn4Tpw74s 7iUnMBBDLoJik3k4YeYKLaeIHpSlGCHIvHs= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g59tbwdth-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:51 -0700 Received: from twshared24024.25.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:49 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 9ACD6F5E5B37; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 11/17] fs: Add async write file modification handling. Date: Fri, 20 May 2022 11:36:40 -0700 Message-ID: <20220520183646.2002023-12-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: AY93RK-I_gqf9bJYMAt36SjU44tu_u9C X-Proofpoint-GUID: AY93RK-I_gqf9bJYMAt36SjU44tu_u9C X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: F3487140021 X-Stat-Signature: a447pc1e8a4dohk8sr6edfgh14ze9y66 X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=XQ08eS3K; spf=none (imf23.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653071852-793353 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds a file_modified_async() function to return -EAGAIN if the request either requires to remove privileges or needs to update the file modification time. This is required for async buffered writes, so the request gets handled in the io worker of io-uring. Signed-off-by: Stefan Roesch --- fs/inode.c | 25 ++++++++++++++++++++++++- fs/xfs/xfs_file.c | 2 +- include/linux/fs.h | 1 + 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 4bb7f583cc6b..3a5d0fa468ab 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2137,6 +2137,27 @@ EXPORT_SYMBOL(file_update_time); * Return: 0 on success, negative errno on failure. */ int file_modified(struct file *file) +{ + return file_modified_async(file, 0); +} +EXPORT_SYMBOL(file_modified); + +/** + * file_modified_async - handle mandated vfs changes when modifying a file + * @file: file that was modified + * @flags: kiocb flags + * + * When file has been modified ensure that special + * file privileges are removed and time settings are updated. + * + * If IOCB_NOWAIT is set, special file privileges will not be removed and + * time settings will not be updated. It will return -EAGAIN. + * + * Context: Caller must hold the file's inode lock. + * + * Return: 0 on success, negative errno on failure. + */ +int file_modified_async(struct file *file, int flags) { int ret; struct dentry *dentry = file_dentry(file); @@ -2163,10 +2184,12 @@ int file_modified(struct file *file) ret = file_needs_update_time(inode, file, &now); if (ret <= 0) return ret; + if (flags & IOCB_NOWAIT) + return -EAGAIN; return __file_update_time(inode, file, &now, ret); } -EXPORT_SYMBOL(file_modified); +EXPORT_SYMBOL(file_modified_async); int inode_needs_sync(struct inode *inode) { diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 5bddb1e9e0b3..793918c83755 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -410,7 +410,7 @@ xfs_file_write_checks( spin_unlock(&ip->i_flags_lock); out: - return file_modified(file); + return file_modified_async(file, iocb->ki_flags); } static int diff --git a/include/linux/fs.h b/include/linux/fs.h index 3b479d02e210..9760283af7dc 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2382,6 +2382,7 @@ static inline void file_accessed(struct file *file) } extern int file_modified(struct file *file); +extern int file_modified_async(struct file *file, int flags); int sync_inode_metadata(struct inode *inode, int wait); From patchwork Fri May 20 18:36:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F472C43217 for ; Fri, 20 May 2022 18:37:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FCA48D0001; Fri, 20 May 2022 14:37:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 804CC8D000A; Fri, 20 May 2022 14:37:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51CCD8D0001; Fri, 20 May 2022 14:37:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2F7068D0008 for ; Fri, 20 May 2022 14:37:51 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CDEB835233 for ; Fri, 20 May 2022 18:37:50 +0000 (UTC) X-FDA: 79486980300.16.CBCEC9A Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf15.hostedemail.com (Postfix) with ESMTP id 4EDF2A000F for ; Fri, 20 May 2022 18:37:33 +0000 (UTC) Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHT46T021516 for ; Fri, 20 May 2022 11:37:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=YjR6bHdDemShrFnRSFoy0SU6vrzP5IeqvXhPCERFUGY=; b=HyrVG9lcrV/CeHxThGHE9eYHZmtVobMA7KM9o6ChVmxZTPqIw9rYWQ0kIAJdLN1JEZsL ZMyHGFbGfmv7xOondbNNrpaHKBRFvMf+NMuYNQ6UcpYtvAYme1A3UBSCmmtSYJQbaBD2 qq/CJEeCPnxARa7Hbv3SW76qcCq1eeU/tnc= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g5rgj8vkm-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:49 -0700 Received: from twshared4937.07.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::d) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:48 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id A0629F5E5B39; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 12/17] fs: Optimization for concurrent file time updates. Date: Fri, 20 May 2022 11:36:41 -0700 Message-ID: <20220520183646.2002023-13-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: sMsCnFX849oEGACZPqp37k2Ue-4t1il_ X-Proofpoint-ORIG-GUID: sMsCnFX849oEGACZPqp37k2Ue-4t1il_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4EDF2A000F X-Stat-Signature: eppqagteq9cquwr3gpqrqgjyp8zrbg5n Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=HyrVG9lc; spf=none (imf15.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspam-User: X-HE-Tag: 1653071853-695984 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This introduces the S_PENDING_TIME flag. If an async buffered write needs to update the time, it cannot be processed in the fast path of io-uring. When a time update is pending this flag is set for async buffered writes. Other concurrent async buffered writes for the same file do not need to wait while this time update is pending. This reduces the number of async buffered writes that need to get punted to the io-workers in io-uring. Signed-off-by: Stefan Roesch --- fs/inode.c | 11 +++++++++-- include/linux/fs.h | 3 +++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 3a5d0fa468ab..5c5021787780 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2184,10 +2184,17 @@ int file_modified_async(struct file *file, int flags) ret = file_needs_update_time(inode, file, &now); if (ret <= 0) return ret; - if (flags & IOCB_NOWAIT) + if (flags & IOCB_NOWAIT) { + if (IS_PENDING_TIME(inode)) + return 0; + + inode->i_flags |= S_PENDING_TIME; return -EAGAIN; + } - return __file_update_time(inode, file, &now, ret); + ret = __file_update_time(inode, file, &now, ret); + inode->i_flags &= ~S_PENDING_TIME; + return ret; } EXPORT_SYMBOL(file_modified_async); diff --git a/include/linux/fs.h b/include/linux/fs.h index 9760283af7dc..5f3aaf61fb4b 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2141,6 +2141,8 @@ struct super_operations { #define S_CASEFOLD (1 << 15) /* Casefolded file */ #define S_VERITY (1 << 16) /* Verity file (using fs/verity/) */ #define S_KERNEL_FILE (1 << 17) /* File is in use by the kernel (eg. fs/cachefiles) */ +#define S_PENDING_TIME (1 << 18) /* File update time is pending */ + /* * Note that nosuid etc flags are inode-specific: setting some file-system @@ -2183,6 +2185,7 @@ static inline bool sb_rdonly(const struct super_block *sb) { return sb->s_flags #define IS_ENCRYPTED(inode) ((inode)->i_flags & S_ENCRYPTED) #define IS_CASEFOLDED(inode) ((inode)->i_flags & S_CASEFOLD) #define IS_VERITY(inode) ((inode)->i_flags & S_VERITY) +#define IS_PENDING_TIME(inode) ((inode)->i_flags & S_PENDING_TIME) #define IS_WHITEOUT(inode) (S_ISCHR(inode->i_mode) && \ (inode)->i_rdev == WHITEOUT_DEV) From patchwork Fri May 20 18:36:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67F65C433FE for ; Fri, 20 May 2022 18:37:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 765038D0009; Fri, 20 May 2022 14:37:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A5D68D0008; Fri, 20 May 2022 14:37:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BE268D0009; Fri, 20 May 2022 14:37:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1964D8D0001 for ; Fri, 20 May 2022 14:37:51 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EE65D35444 for ; Fri, 20 May 2022 18:37:50 +0000 (UTC) X-FDA: 79486980300.17.CBCF2BE Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf25.hostedemail.com (Postfix) with ESMTP id CCC8CA00C4 for ; Fri, 20 May 2022 18:37:24 +0000 (UTC) Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSIOX010648 for ; Fri, 20 May 2022 11:37:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=fdXvE+WWGkZbdDWaaWcycrFeF07NZ6/Rt8YdakSfT7c=; b=Tt5lHOjJNVkIS4uCIUASb50t7kdLJEYq39f8aS7Kd7BjvzyFlnPDFCyONQ/SgWl+LNtl aVgWGF6qI83wP8Lalp4X2OCnPOeoLkhExkXBHvX8OmB7OrDK8hOBqXSNZHbmKfZeK1+G R2ByHDAco573YRxstlhZ2KUbQcZzze94jNg= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g6341cjg9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:49 -0700 Received: from twshared6696.05.ash7.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:47 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id A87FEF5E5B3B; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 13/17] io_uring: Add support for async buffered writes Date: Fri, 20 May 2022 11:36:42 -0700 Message-ID: <20220520183646.2002023-14-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: uEnyFqXLae5V-Xh7xDajSi8XRXwj_1M8 X-Proofpoint-ORIG-GUID: uEnyFqXLae5V-Xh7xDajSi8XRXwj_1M8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CCC8CA00C4 X-Stat-Signature: 9uz6mi9dunfdzcaomumn54bypzbm8xa7 X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=Tt5lHOjJ; spf=none (imf25.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653071844-240017 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This enables the async buffered writes for the filesystems that support async buffered writes in io-uring. Buffered writes are enabled for blocks that are already in the page cache or can be acquired with noio. Signed-off-by: Stefan Roesch --- fs/io_uring.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 91de361ea9ab..f3aaac286509 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3746,7 +3746,7 @@ static inline int io_iter_do_read(struct io_kiocb *req, struct iov_iter *iter) return -EINVAL; } -static bool need_read_all(struct io_kiocb *req) +static bool need_complete_io(struct io_kiocb *req) { return req->flags & REQ_F_ISREG || S_ISBLK(file_inode(req->file)->i_mode); @@ -3875,7 +3875,7 @@ static int io_read(struct io_kiocb *req, unsigned int issue_flags) } else if (ret == -EIOCBQUEUED) { goto out_free; } else if (ret == req->result || ret <= 0 || !force_nonblock || - (req->flags & REQ_F_NOWAIT) || !need_read_all(req)) { + (req->flags & REQ_F_NOWAIT) || !need_complete_io(req)) { /* read all, failed, already did sync or don't want to retry */ goto done; } @@ -3971,9 +3971,10 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags) if (unlikely(!io_file_supports_nowait(req))) goto copy_iov; - /* file path doesn't support NOWAIT for non-direct_IO */ - if (force_nonblock && !(kiocb->ki_flags & IOCB_DIRECT) && - (req->flags & REQ_F_ISREG)) + /* File path supports NOWAIT for non-direct_IO only for block devices. */ + if (!(kiocb->ki_flags & IOCB_DIRECT) && + !(kiocb->ki_filp->f_mode & FMODE_BUF_WASYNC) && + (req->flags & REQ_F_ISREG)) goto copy_iov; kiocb->ki_flags |= IOCB_NOWAIT; @@ -4027,6 +4028,24 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags) /* IOPOLL retry should happen for io-wq threads */ if (ret2 == -EAGAIN && (req->ctx->flags & IORING_SETUP_IOPOLL)) goto copy_iov; + + if (ret2 != req->result && ret2 >= 0 && need_complete_io(req)) { + struct io_async_rw *rw; + + /* This is a partial write. The file pos has already been + * updated, setup the async struct to complete the request + * in the worker. Also update bytes_done to account for + * the bytes already written. + */ + iov_iter_save_state(&s->iter, &s->iter_state); + ret = io_setup_async_rw(req, iovec, s, true); + + rw = req->async_data; + if (rw) + rw->bytes_done += ret2; + + return ret ? ret : -EAGAIN; + } done: kiocb_done(req, ret2, issue_flags); } else { From patchwork Fri May 20 18:36:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 582D2C433F5 for ; Fri, 20 May 2022 18:37:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C2628D000A; Fri, 20 May 2022 14:37:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 126238D0008; Fri, 20 May 2022 14:37:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6A3C8D000A; Fri, 20 May 2022 14:37:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C8B2D8D0008 for ; Fri, 20 May 2022 14:37:51 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A909735422 for ; Fri, 20 May 2022 18:37:51 +0000 (UTC) X-FDA: 79486980342.12.806F4F7 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf23.hostedemail.com (Postfix) with ESMTP id 55E42140018 for ; Fri, 20 May 2022 18:37:32 +0000 (UTC) Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSLUG006625 for ; Fri, 20 May 2022 11:37:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=UfD+h75jLCqmGcJIFVAad9rZN/r0VFdoC8EFOxTjfII=; b=lIv14vGOMRnHDFARgipELlhdPcszdR2I58P/Whn4jzYXTZZ4ec1Vmzom9mFtk4L8Ov5J NFYJi3tVK5QCK2S1AFlFjVd27f/7KUPs6og0H7T7BuN9LZu+9tTtHoPttdCQXljMZ2de wEO78RQ+45FeV1k4bKav7lAGncnk/RDwwXA= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g60b0naar-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:49 -0700 Received: from twshared6696.05.ash7.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:48 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id B0017F5E5B3D; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 14/17] io_uring: Add tracepoint for short writes Date: Fri, 20 May 2022 11:36:43 -0700 Message-ID: <20220520183646.2002023-15-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: uLF5tvdAMpaddT2kxPeQXDwgdN_1RzQ4 X-Proofpoint-GUID: uLF5tvdAMpaddT2kxPeQXDwgdN_1RzQ4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_05,2022-05-20_02,2022-02-23_01 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=lIv14vGO; spf=none (imf23.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspam-User: X-Stat-Signature: cdqbb3bp4gayubba7e7fybsaq3mfrhbc X-Rspamd-Queue-Id: 55E42140018 X-Rspamd-Server: rspam01 X-HE-Tag: 1653071852-585701 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the io_uring_short_write tracepoint to io_uring. A short write is issued if not all pages that are required for a write are in the page cache and the async buffered writes have to return EAGAIN. Signed-off-by: Stefan Roesch --- fs/io_uring.c | 3 +++ include/trace/events/io_uring.h | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index f3aaac286509..7435a9c2007f 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -4032,6 +4032,9 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags) if (ret2 != req->result && ret2 >= 0 && need_complete_io(req)) { struct io_async_rw *rw; + trace_io_uring_short_write(req->ctx, kiocb->ki_pos - ret2, + req->result, ret2); + /* This is a partial write. The file pos has already been * updated, setup the async struct to complete the request * in the worker. Also update bytes_done to account for diff --git a/include/trace/events/io_uring.h b/include/trace/events/io_uring.h index cddf5b6fbeb4..661834361d33 100644 --- a/include/trace/events/io_uring.h +++ b/include/trace/events/io_uring.h @@ -543,6 +543,31 @@ TRACE_EVENT(io_uring_req_failed, (unsigned long long) __entry->pad2, __entry->error) ); +TRACE_EVENT(io_uring_short_write, + + TP_PROTO(void *ctx, u64 fpos, u64 wanted, u64 got), + + TP_ARGS(ctx, fpos, wanted, got), + + TP_STRUCT__entry( + __field(void *, ctx) + __field(u64, fpos) + __field(u64, wanted) + __field(u64, got) + ), + + TP_fast_assign( + __entry->ctx = ctx; + __entry->fpos = fpos; + __entry->wanted = wanted; + __entry->got = got; + ), + + TP_printk("ring %p, fpos %lld, wanted %lld, got %lld", + __entry->ctx, __entry->fpos, + __entry->wanted, __entry->got) +); + #endif /* _TRACE_IO_URING_H */ /* This part must be outside protection */ From patchwork Fri May 20 18:36:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E3E2C433FE for ; Fri, 20 May 2022 18:37:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D0D938D0006; Fri, 20 May 2022 14:37:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C955A8D0001; Fri, 20 May 2022 14:37:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B372A8D0006; Fri, 20 May 2022 14:37:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A0EC58D0001 for ; Fri, 20 May 2022 14:37:48 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7C8283543F for ; Fri, 20 May 2022 18:37:48 +0000 (UTC) X-FDA: 79486980216.15.3AD80DC Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf06.hostedemail.com (Postfix) with ESMTP id 023F3180020 for ; Fri, 20 May 2022 18:37:44 +0000 (UTC) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSejp020165 for ; Fri, 20 May 2022 11:37:47 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=xpR2o1MxE5h1TGY+8JuoVPirJk4Gk5FbraTEzaT2FUw=; b=B+D0y/UOBtUJH2ThPdBKyBqjNoqp7Rs5HNASL+1AjHHDZ/k9j2dF4CH5z+0pQsMCunNp Ovaq6ag0+uprirMFmdaRjr+whE7wyzHYLOfiCOoBsWiqRPE9HHKmDXUh1Ggz2Ak5/eUE bVfkc5UczODbgZLKw4PoDCYBOpAEV0LkTj0= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g60bnn9jt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:46 -0700 Received: from twshared8307.18.frc3.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:45 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id B6C7BF5E5B3F; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 15/17] xfs: Add iomap async buffered write support Date: Fri, 20 May 2022 11:36:44 -0700 Message-ID: <20220520183646.2002023-16-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: cU7kKzCplc566mjbMv4ij5N4QiAuIjax X-Proofpoint-GUID: cU7kKzCplc566mjbMv4ij5N4QiAuIjax X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 023F3180020 X-Stat-Signature: efidm7xj6h41g5348nregz9t69fp9a8b Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="B+D0y/UO"; spf=none (imf06.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspam-User: X-HE-Tag: 1653071864-998157 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the async buffered write support to the iomap layer of XFS. If a lock cannot be acquired or additional reads need to be performed, the request will return -EAGAIN in case this is an async buffered write request. This patch changes the helper function xfs_ilock_for_iomap such that the lock mode needs to be passed in. Signed-off-by: Stefan Roesch --- fs/xfs/xfs_iomap.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index e552ce541ec2..1aea962262ad 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -659,7 +659,7 @@ xfs_ilock_for_iomap( unsigned flags, unsigned *lockmode) { - unsigned mode = XFS_ILOCK_SHARED; + unsigned int mode = *lockmode; bool is_write = flags & (IOMAP_WRITE | IOMAP_ZERO); /* @@ -737,7 +737,7 @@ xfs_direct_write_iomap_begin( int nimaps = 1, error = 0; bool shared = false; u16 iomap_flags = 0; - unsigned lockmode; + unsigned int lockmode = XFS_ILOCK_SHARED; ASSERT(flags & (IOMAP_WRITE | IOMAP_ZERO)); @@ -881,18 +881,22 @@ xfs_buffered_write_iomap_begin( bool eof = false, cow_eof = false, shared = false; int allocfork = XFS_DATA_FORK; int error = 0; + unsigned int lockmode = XFS_ILOCK_EXCL; if (xfs_is_shutdown(mp)) return -EIO; /* we can't use delayed allocations when using extent size hints */ - if (xfs_get_extsz_hint(ip)) + if (xfs_get_extsz_hint(ip)) { return xfs_direct_write_iomap_begin(inode, offset, count, flags, iomap, srcmap); + } ASSERT(!XFS_IS_REALTIME_INODE(ip)); - xfs_ilock(ip, XFS_ILOCK_EXCL); + error = xfs_ilock_for_iomap(ip, flags, &lockmode); + if (error) + return error; if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(&ip->i_df)) || XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { @@ -1167,7 +1171,7 @@ xfs_read_iomap_begin( xfs_fileoff_t end_fsb = xfs_iomap_end_fsb(mp, offset, length); int nimaps = 1, error = 0; bool shared = false; - unsigned lockmode; + unsigned int lockmode = XFS_ILOCK_SHARED; ASSERT(!(flags & (IOMAP_WRITE | IOMAP_ZERO))); From patchwork Fri May 20 18:36:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857278 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3C15C433FE for ; Fri, 20 May 2022 18:38:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D0726B0071; Fri, 20 May 2022 14:38:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4579E6B0072; Fri, 20 May 2022 14:38:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F95B6B0073; Fri, 20 May 2022 14:38:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1A1FF6B0071 for ; Fri, 20 May 2022 14:38:38 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id E5BA561B1B for ; Fri, 20 May 2022 18:38:37 +0000 (UTC) X-FDA: 79486982274.18.821FFE0 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf01.hostedemail.com (Postfix) with ESMTP id 6391C400E1 for ; Fri, 20 May 2022 18:38:36 +0000 (UTC) Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHT53S021531 for ; Fri, 20 May 2022 11:38:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=LWT7/rgVKJvUBvBuLFTZOcg1EE8pCJNmk5mdhmfBzd4=; b=JoS++ya+ly8O9xsG0XzuifGnZeLIEX/gK3kxE7Zir/4A38PbVHGAVoxY/f42lqH5fBq6 mx1mqpCC0T0QkGbdRoxadEjFl/v3wcGwGym4Smjg2FcpJhJ/DBLmctdG/3I9LYi9COVA JczGbxlDbXPhjmOJXb8KIUQHOTODn+oPQ9A= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g5rgj8vs2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:38:36 -0700 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub104.TheFacebook.com (2620:10d:c085:11d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:51 -0700 Received: from twshared6696.05.ash7.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:51 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id BFAF2F5E5B41; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 16/17] xfs: Add async buffered write support Date: Fri, 20 May 2022 11:36:45 -0700 Message-ID: <20220520183646.2002023-17-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: CeCkEXdhsnSqDtENCv5QNh4DrxeGSj57 X-Proofpoint-ORIG-GUID: CeCkEXdhsnSqDtENCv5QNh4DrxeGSj57 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=JoS++ya+; spf=none (imf01.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=6139886598=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspam-User: X-Stat-Signature: 4n3fskfmmmun5rbt4tw7kdbnwkgjyqj5 X-Rspamd-Queue-Id: 6391C400E1 X-Rspamd-Server: rspam01 X-HE-Tag: 1653071916-534980 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the async buffered write support to XFS. For async buffered write requests, the request will return -EAGAIN if the ilock cannot be obtained immediately. This splits off a new helper xfs_ilock_inode from the existing helper xfs_ilock_iocb so it can be used for this function. The exising helper cannot be used as it hardcoded the inode to be used. Signed-off-by: Stefan Roesch --- fs/xfs/xfs_file.c | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 793918c83755..ad3175b7d366 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -190,14 +190,13 @@ xfs_file_fsync( return error; } -static int -xfs_ilock_iocb( - struct kiocb *iocb, +static inline int +xfs_ilock_xfs_inode( + struct xfs_inode *ip, + int flags, unsigned int lock_mode) { - struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp)); - - if (iocb->ki_flags & IOCB_NOWAIT) { + if (flags & IOCB_NOWAIT) { if (!xfs_ilock_nowait(ip, lock_mode)) return -EAGAIN; } else { @@ -222,7 +221,7 @@ xfs_file_dio_read( file_accessed(iocb->ki_filp); - ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + ret = xfs_ilock_xfs_inode(ip, iocb->ki_flags, XFS_IOLOCK_SHARED); if (ret) return ret; ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0, 0); @@ -244,7 +243,7 @@ xfs_file_dax_read( if (!iov_iter_count(to)) return 0; /* skip atime */ - ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + ret = xfs_ilock_xfs_inode(ip, iocb->ki_flags, XFS_IOLOCK_SHARED); if (ret) return ret; ret = dax_iomap_rw(iocb, to, &xfs_read_iomap_ops); @@ -264,7 +263,7 @@ xfs_file_buffered_read( trace_xfs_file_buffered_read(iocb, to); - ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + ret = xfs_ilock_xfs_inode(ip, iocb->ki_flags, XFS_IOLOCK_SHARED); if (ret) return ret; ret = generic_file_read_iter(iocb, to); @@ -343,7 +342,7 @@ xfs_file_write_checks( if (*iolock == XFS_IOLOCK_SHARED && !IS_NOSEC(inode)) { xfs_iunlock(ip, *iolock); *iolock = XFS_IOLOCK_EXCL; - error = xfs_ilock_iocb(iocb, *iolock); + error = xfs_ilock_xfs_inode(ip, iocb->ki_flags, *iolock); if (error) { *iolock = 0; return error; @@ -516,7 +515,7 @@ xfs_file_dio_write_aligned( int iolock = XFS_IOLOCK_SHARED; ssize_t ret; - ret = xfs_ilock_iocb(iocb, iolock); + ret = xfs_ilock_xfs_inode(ip, iocb->ki_flags, iolock); if (ret) return ret; ret = xfs_file_write_checks(iocb, from, &iolock); @@ -583,7 +582,7 @@ xfs_file_dio_write_unaligned( flags = IOMAP_DIO_FORCE_WAIT; } - ret = xfs_ilock_iocb(iocb, iolock); + ret = xfs_ilock_xfs_inode(ip, iocb->ki_flags, iolock); if (ret) return ret; @@ -659,7 +658,7 @@ xfs_file_dax_write( ssize_t ret, error = 0; loff_t pos; - ret = xfs_ilock_iocb(iocb, iolock); + ret = xfs_ilock_xfs_inode(ip, iocb->ki_flags, iolock); if (ret) return ret; ret = xfs_file_write_checks(iocb, from, &iolock); @@ -702,12 +701,11 @@ xfs_file_buffered_write( bool cleared_space = false; int iolock; - if (iocb->ki_flags & IOCB_NOWAIT) - return -EOPNOTSUPP; - write_retry: iolock = XFS_IOLOCK_EXCL; - xfs_ilock(ip, iolock); + ret = xfs_ilock_xfs_inode(ip, iocb->ki_flags, iolock); + if (ret) + return ret; ret = xfs_file_write_checks(iocb, from, &iolock); if (ret) From patchwork Fri May 20 18:36:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12857277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8532EC433F5 for ; Fri, 20 May 2022 18:37:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1EABE8D000C; Fri, 20 May 2022 14:37:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 100638D0008; Fri, 20 May 2022 14:37:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB9BA8D000C; Fri, 20 May 2022 14:37:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D00FF8D0008 for ; Fri, 20 May 2022 14:37:57 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B0BAA3522F for ; Fri, 20 May 2022 18:37:57 +0000 (UTC) X-FDA: 79486980594.20.F8D7A03 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf16.hostedemail.com (Postfix) with ESMTP id B494F180013 for ; Fri, 20 May 2022 18:37:44 +0000 (UTC) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24KHSNMI018068 for ; Fri, 20 May 2022 11:37:56 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=mezo8o29O3Ay9l7JyNHPANH1KNPPbed3umcUOpvhL7Q=; b=jH6HMQZ/qh58RCGqJedbNF1U/zDw6BY2FM3fujn8koRXkX/4BD4Fn3ToH1F1UFcaHQRF w473sTF5VYVvUYFU3PecohX1PjdO3MCUBSz8wtrn2sICyS8dWlQAHd+cAMIFyF9MAOoZ rDltJoufciTpHlpxS8OqdenNLjIlR3T1Zxw= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g5xexdw0y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 May 2022 11:37:56 -0700 Received: from twshared8307.18.frc3.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Fri, 20 May 2022 11:37:55 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id C6F93F5E5B43; Fri, 20 May 2022 11:37:16 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [RFC PATCH v4 17/17] xfs: Enable async buffered write support Date: Fri, 20 May 2022 11:36:46 -0700 Message-ID: <20220520183646.2002023-18-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220520183646.2002023-1-shr@fb.com> References: <20220520183646.2002023-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: p-dNS_G0nh_CYq21NasajAT2h1ymfDY8 X-Proofpoint-GUID: p-dNS_G0nh_CYq21NasajAT2h1ymfDY8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-20_06,2022-05-20_02,2022-02-23_01 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="jH6HMQZ/"; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf16.hostedemail.com: domain of "prvs=6139886598=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=6139886598=shr@fb.com" X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B494F180013 X-Stat-Signature: uisxgwt4kciqkywd83u6i1n5h69qj94k X-HE-Tag: 1653071864-450491 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This turns on the async buffered write support for XFS. Signed-off-by: Stefan Roesch --- fs/xfs/xfs_file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index ad3175b7d366..af4fdc852da5 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -1169,7 +1169,7 @@ xfs_file_open( return -EFBIG; if (xfs_is_shutdown(XFS_M(inode->i_sb))) return -EIO; - file->f_mode |= FMODE_NOWAIT | FMODE_BUF_RASYNC; + file->f_mode |= FMODE_NOWAIT | FMODE_BUF_RASYNC | FMODE_BUF_WASYNC; return 0; }