From patchwork Wed May 25 22:34:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C829C433EF for ; Wed, 25 May 2022 22:34:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 060A08D000C; Wed, 25 May 2022 18:34:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F246D8D000B; Wed, 25 May 2022 18:34:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9C2F8D000A; Wed, 25 May 2022 18:34:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C11358D0001 for ; Wed, 25 May 2022 18:34:52 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8570E31BD2 for ; Wed, 25 May 2022 22:34:52 +0000 (UTC) X-FDA: 79505721624.15.AEDB443 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf31.hostedemail.com (Postfix) with ESMTP id 9CA6A20018 for ; Wed, 25 May 2022 22:34:16 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24PGtbpv009532 for ; Wed, 25 May 2022 15:34:51 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=3zHDjbn72NXSBweb9g6a0xThug/NIwNbovojkJVDB2M=; b=nGh3U+seKW+wqwcOUlFaxxUDiyVe029LznfEhqkjpHqTisFquPfEw+n2SRjBZc4EsXg/ Htx9taWwbAlZwR33WZCZZLOtORytitRPvuO0rVQ9N6pRWM0b8IUAfFBq7e7M8rNa+KV9 R/GiIs3LCmzAIPqqVLghW2HvOhtH/aROyzA= Received: from mail.thefacebook.com ([163.114.132.120]) by m0089730.ppops.net (PPS) with ESMTPS id 3g93tx1dp5-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:51 -0700 Received: from twshared6696.05.ash7.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:49 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id E7293F9E1B40; Wed, 25 May 2022 15:34:34 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 01/16] mm: Move starting of background writeback into the main balancing loop Date: Wed, 25 May 2022 15:34:17 -0700 Message-ID: <20220525223432.205676-2-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: FntW5TSvpleoEdY8uz2ET27uGmlpG15Q X-Proofpoint-GUID: FntW5TSvpleoEdY8uz2ET27uGmlpG15Q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9CA6A20018 X-Stat-Signature: g6bmk9eqhd6omjids93ksqj8j1fwbi7m X-Rspam-User: Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=nGh3U+se; spf=none (imf31.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653518056-939657 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jan Kara We start background writeback if we are over background threshold after exiting the main loop in balance_dirty_pages(). This may result in basing the decision on already stale values (we may have slept for significant amount of time) and it is also inconvenient for refactoring needed for async dirty throttling. Move the check into the main waiting loop. Signed-off-by: Jan Kara Signed-off-by: Stefan Roesch --- mm/page-writeback.c | 31 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 7e2da284e427..8e5e003f0093 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1618,6 +1618,19 @@ static void balance_dirty_pages(struct bdi_writeback *wb, } } + /* + * In laptop mode, we wait until hitting the higher threshold + * before starting background writeout, and then write out all + * the way down to the lower threshold. So slow writers cause + * minimal disk activity. + * + * In normal mode, we start background writeout at the lower + * background_thresh, to keep the amount of dirty memory low. + */ + if (!laptop_mode && nr_reclaimable > gdtc->bg_thresh && + !writeback_in_progress(wb)) + wb_start_background_writeback(wb); + /* * Throttle it only when the background writeback cannot * catch-up. This avoids (excessively) small writeouts @@ -1648,6 +1661,7 @@ static void balance_dirty_pages(struct bdi_writeback *wb, break; } + /* Start writeback even when in laptop mode */ if (unlikely(!writeback_in_progress(wb))) wb_start_background_writeback(wb); @@ -1814,23 +1828,6 @@ static void balance_dirty_pages(struct bdi_writeback *wb, if (!dirty_exceeded && wb->dirty_exceeded) wb->dirty_exceeded = 0; - - if (writeback_in_progress(wb)) - return; - - /* - * In laptop mode, we wait until hitting the higher threshold before - * starting background writeout, and then write out all the way down - * to the lower threshold. So slow writers cause minimal disk activity. - * - * In normal mode, we start background writeout at the lower - * background_thresh, to keep the amount of dirty memory low. - */ - if (laptop_mode) - return; - - if (nr_reclaimable > gdtc->bg_thresh) - wb_start_background_writeback(wb); } static DEFINE_PER_CPU(int, bdp_ratelimits); From patchwork Wed May 25 22:34:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF977C43217 for ; Wed, 25 May 2022 22:34:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 361F48D0008; Wed, 25 May 2022 18:34:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2256F8D0001; Wed, 25 May 2022 18:34:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C42E8D0008; Wed, 25 May 2022 18:34:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id ED9308D0001 for ; Wed, 25 May 2022 18:34:50 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C41D22EC56 for ; Wed, 25 May 2022 22:34:50 +0000 (UTC) X-FDA: 79505721540.25.669F2D5 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf23.hostedemail.com (Postfix) with ESMTP id 823EB140020 for ; Wed, 25 May 2022 22:34:28 +0000 (UTC) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtdXb023267 for ; Wed, 25 May 2022 15:34:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=JmNWcJ9S5BuVrDagcKBT8mAi6jQ8Go8fzpG+AeBXTsU=; b=hnJX3ZfpLapYdQJgGZF0cT4zPgMoIG8AFl1WFlf6Kg74SUCZN93jKBLd+Sh1NN6MIuun 4E5HYyTr3jgK3Z+5pCGxPU8HeRvFtvqc8cJXIoo+JQNq9fZa33y2W9u3CaUDQBg4WEtB ViuZj5w+sfGtRMikgPfbcYBOm3QTT85cxSs= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93upsd8g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:49 -0700 Received: from twshared14818.18.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:46 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id EA399F9E1B44; Wed, 25 May 2022 15:34:34 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 02/16] mm: Move updates of dirty_exceeded into one place Date: Wed, 25 May 2022 15:34:18 -0700 Message-ID: <20220525223432.205676-3-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: LAx-KxPwoL4dNXjNrnhJQTdWPInUpR37 X-Proofpoint-GUID: LAx-KxPwoL4dNXjNrnhJQTdWPInUpR37 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_07,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 823EB140020 X-Stat-Signature: 9e5ws3wg6gid5z3hke9tu6aha59emy3w X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=hnJX3Zfp; spf=none (imf23.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653518068-978935 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jan Kara Transition of wb->dirty_exceeded from 0 to 1 happens before we go to sleep in balance_dirty_pages() while transition from 1 to 0 happens when exiting from balance_dirty_pages(), possibly based on old values. This does not make a lot of sense since wb->dirty_exceeded should simply reflect whether wb is over dirty limit and so we should ratelimit entering to balance_dirty_pages() less. Move the two updates together. Signed-off-by: Jan Kara Signed-off-by: Stefan Roesch --- mm/page-writeback.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 8e5e003f0093..89dcc7d8395a 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1720,8 +1720,8 @@ static void balance_dirty_pages(struct bdi_writeback *wb, sdtc = mdtc; } - if (dirty_exceeded && !wb->dirty_exceeded) - wb->dirty_exceeded = 1; + if (dirty_exceeded != wb->dirty_exceeded) + wb->dirty_exceeded = dirty_exceeded; if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + BANDWIDTH_INTERVAL)) @@ -1825,9 +1825,6 @@ static void balance_dirty_pages(struct bdi_writeback *wb, if (fatal_signal_pending(current)) break; } - - if (!dirty_exceeded && wb->dirty_exceeded) - wb->dirty_exceeded = 0; } static DEFINE_PER_CPU(int, bdp_ratelimits); From patchwork Wed May 25 22:34:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A9E6C433EF for ; Wed, 25 May 2022 22:34:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 171538D0007; Wed, 25 May 2022 18:34:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 11D698D0001; Wed, 25 May 2022 18:34:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00D688D0007; Wed, 25 May 2022 18:34:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E752A8D0001 for ; Wed, 25 May 2022 18:34:49 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id B15FE81425 for ; Wed, 25 May 2022 22:34:49 +0000 (UTC) X-FDA: 79505721498.14.95EC6DB Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf02.hostedemail.com (Postfix) with ESMTP id 132C780020 for ; Wed, 25 May 2022 22:34:45 +0000 (UTC) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24PGtag2012073 for ; Wed, 25 May 2022 15:34:48 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=HPjnCKp2mP5zUhzFD7orqPqCqeAUncMT3i4YP9mUBko=; b=Xeq7UbAJTIf360JtFAeHz5zWRF2O/gjlSpMb/6QAWJ6itPVwmWLEKxk0mwgU1Tpblhv0 N9Wn5ZMt2RqfVHoIQ3z2TLv6rE7wMW/EoWt+m1NwpJPdtDGS/k/lXVplyCJ3aVvoVJVf k/P8cAOLPWXm0+nIta4wqj2A3X5mXRaBf8I= Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net (PPS) with ESMTPS id 3g9qtuafph-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:48 -0700 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub203.TheFacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:46 -0700 Received: from twshared8508.05.ash9.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:45 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id F0FF6F9E1B49; Wed, 25 May 2022 15:34:34 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 03/16] mm: Add balance_dirty_pages_ratelimited_flags() function Date: Wed, 25 May 2022 15:34:19 -0700 Message-ID: <20220525223432.205676-4-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: yoj71-qsqNbJcr_d67SatOCRNAD7lXMD X-Proofpoint-ORIG-GUID: yoj71-qsqNbJcr_d67SatOCRNAD7lXMD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_07,2022-05-25_02,2022-02-23_01 X-Rspam-User: X-Rspamd-Queue-Id: 132C780020 X-Stat-Signature: 1e4zswno1tgddbocs58g5n8m4kq86dkh Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=Xeq7UbAJ; spf=none (imf02.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspamd-Server: rspam03 X-HE-Tag: 1653518085-309670 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jan Kara This adds the helper function balance_dirty_pages_ratelimited_flags(). It adds the parameter flags to balance_dirty_pages_ratelimited(). The flags parameter is passed to balance_dirty_pages(). For async buffered writes the flag value will be BDP_ASYNC. If balance_dirty_pages() gets called for async buffered write, we don't want to wait. Instead we need to indicate to the caller that throttling is needed so that it can stop writing and offload the rest of the write to a context that can block. The new helper function is also used by balance_dirty_pages_ratelimited(). Signed-off-by: Jan Kara Signed-off-by: Stefan Roesch --- include/linux/writeback.h | 7 ++++++ mm/page-writeback.c | 48 +++++++++++++++++++++++++-------------- 2 files changed, 38 insertions(+), 17 deletions(-) diff --git a/include/linux/writeback.h b/include/linux/writeback.h index fec248ab1fec..1bddad86a4f6 100644 --- a/include/linux/writeback.h +++ b/include/linux/writeback.h @@ -372,7 +372,14 @@ void global_dirty_limits(unsigned long *pbackground, unsigned long *pdirty); unsigned long wb_calc_thresh(struct bdi_writeback *wb, unsigned long thresh); void wb_update_bandwidth(struct bdi_writeback *wb); + +/* Invoke balance dirty pages in async mode. */ +#define BDP_ASYNC 0x0001 + void balance_dirty_pages_ratelimited(struct address_space *mapping); +int balance_dirty_pages_ratelimited_flags(struct address_space *mapping, + unsigned int flags); + bool wb_over_bg_thresh(struct bdi_writeback *wb); typedef int (*writepage_t)(struct page *page, struct writeback_control *wbc, diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 89dcc7d8395a..3701e813d05f 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -1545,8 +1545,8 @@ static inline void wb_dirty_limits(struct dirty_throttle_control *dtc) * If we're over `background_thresh' then the writeback threads are woken to * perform some writeout. */ -static void balance_dirty_pages(struct bdi_writeback *wb, - unsigned long pages_dirtied) +static int balance_dirty_pages(struct bdi_writeback *wb, + unsigned long pages_dirtied, unsigned int flags) { struct dirty_throttle_control gdtc_stor = { GDTC_INIT(wb) }; struct dirty_throttle_control mdtc_stor = { MDTC_INIT(wb, &gdtc_stor) }; @@ -1566,6 +1566,7 @@ static void balance_dirty_pages(struct bdi_writeback *wb, struct backing_dev_info *bdi = wb->bdi; bool strictlimit = bdi->capabilities & BDI_CAP_STRICTLIMIT; unsigned long start_time = jiffies; + int ret = 0; for (;;) { unsigned long now = jiffies; @@ -1794,6 +1795,10 @@ static void balance_dirty_pages(struct bdi_writeback *wb, period, pause, start_time); + if (flags & BDP_ASYNC) { + ret = -EAGAIN; + break; + } __set_current_state(TASK_KILLABLE); wb->dirty_sleep = now; io_schedule_timeout(pause); @@ -1825,6 +1830,7 @@ static void balance_dirty_pages(struct bdi_writeback *wb, if (fatal_signal_pending(current)) break; } + return ret; } static DEFINE_PER_CPU(int, bdp_ratelimits); @@ -1845,28 +1851,18 @@ static DEFINE_PER_CPU(int, bdp_ratelimits); */ DEFINE_PER_CPU(int, dirty_throttle_leaks) = 0; -/** - * balance_dirty_pages_ratelimited - balance dirty memory state - * @mapping: address_space which was dirtied - * - * Processes which are dirtying memory should call in here once for each page - * which was newly dirtied. The function will periodically check the system's - * dirty state and will initiate writeback if needed. - * - * Once we're over the dirty memory limit we decrease the ratelimiting - * by a lot, to prevent individual processes from overshooting the limit - * by (ratelimit_pages) each. - */ -void balance_dirty_pages_ratelimited(struct address_space *mapping) +int balance_dirty_pages_ratelimited_flags(struct address_space *mapping, + unsigned int flags) { struct inode *inode = mapping->host; struct backing_dev_info *bdi = inode_to_bdi(inode); struct bdi_writeback *wb = NULL; int ratelimit; + int ret = 0; int *p; if (!(bdi->capabilities & BDI_CAP_WRITEBACK)) - return; + return ret; if (inode_cgwb_enabled(inode)) wb = wb_get_create_current(bdi, GFP_KERNEL); @@ -1906,9 +1902,27 @@ void balance_dirty_pages_ratelimited(struct address_space *mapping) preempt_enable(); if (unlikely(current->nr_dirtied >= ratelimit)) - balance_dirty_pages(wb, current->nr_dirtied); + balance_dirty_pages(wb, current->nr_dirtied, flags); wb_put(wb); + return ret; +} + +/** + * balance_dirty_pages_ratelimited - balance dirty memory state + * @mapping: address_space which was dirtied + * + * Processes which are dirtying memory should call in here once for each page + * which was newly dirtied. The function will periodically check the system's + * dirty state and will initiate writeback if needed. + * + * Once we're over the dirty memory limit we decrease the ratelimiting + * by a lot, to prevent individual processes from overshooting the limit + * by (ratelimit_pages) each. + */ +void balance_dirty_pages_ratelimited(struct address_space *mapping) +{ + balance_dirty_pages_ratelimited_flags(mapping, 0); } EXPORT_SYMBOL(balance_dirty_pages_ratelimited); From patchwork Wed May 25 22:34:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5E49C4167B for ; Wed, 25 May 2022 22:34:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9ACD48D0009; Wed, 25 May 2022 18:34:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89A5C8D0001; Wed, 25 May 2022 18:34:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5DAAB8D000A; Wed, 25 May 2022 18:34:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 37B8E8D0009 for ; Wed, 25 May 2022 18:34:51 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 01D8632DB0 for ; Wed, 25 May 2022 22:34:50 +0000 (UTC) X-FDA: 79505721582.24.331E26D Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf05.hostedemail.com (Postfix) with ESMTP id 35498100020 for ; Wed, 25 May 2022 22:34:21 +0000 (UTC) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24PGtag5012073 for ; Wed, 25 May 2022 15:34:49 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=zeIBIRPPCGXDuSo1H7/8XUmuyxFIlVdisrj7fMF8Y4s=; b=nLg4hrXstjAIFJ2E0axlmr0ufRW7VwNFXhqg6CYVpbaTNemiLCczqSlcgUt6/wz3+NXf YotKU2rjvhou5KTDw8552veHCm/faiiGrz3CANiqRzfRCp2fDJr9DZSyujmJYlZSJAg9 qSW4qeXZWf0IdsMrTJes2aZCS1WFxLqC1aw= Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net (PPS) with ESMTPS id 3g9qtuafph-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:49 -0700 Received: from twshared10276.08.ash9.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:46 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 03BE1F9E1B4E; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 04/16] iomap: Add flags parameter to iomap_page_create() Date: Wed, 25 May 2022 15:34:20 -0700 Message-ID: <20220525223432.205676-5-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: LQgviVJBgs5L0OB6z_UuQR6nHs1pg3s6 X-Proofpoint-ORIG-GUID: LQgviVJBgs5L0OB6z_UuQR6nHs1pg3s6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_07,2022-05-25_02,2022-02-23_01 X-Stat-Signature: 5fxcs9949nue58rh6kgpds8udwuzsotz X-Rspam-User: Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=nLg4hrXs; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf05.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com" X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 35498100020 X-HE-Tag: 1653518061-149641 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add the kiocb flags parameter to the function iomap_page_create(). Depending on the value of the flags parameter it enables different gfp flags. No intended functional changes in this patch. Signed-off-by: Stefan Roesch Reviewed-by: Jan Kara --- fs/iomap/buffered-io.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 8ce8720093b9..8ef77f7287aa 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -44,16 +44,21 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio) static struct bio_set iomap_ioend_bioset; static struct iomap_page * -iomap_page_create(struct inode *inode, struct folio *folio) +iomap_page_create(struct inode *inode, struct folio *folio, unsigned int flags) { struct iomap_page *iop = to_iomap_page(folio); unsigned int nr_blocks = i_blocks_per_folio(inode, folio); + gfp_t gfp = GFP_NOFS | __GFP_NOFAIL; if (iop || nr_blocks <= 1) return iop; + if (flags & IOMAP_NOWAIT) + gfp = GFP_NOWAIT; + iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)), - GFP_NOFS | __GFP_NOFAIL); + gfp); + spin_lock_init(&iop->uptodate_lock); if (folio_test_uptodate(folio)) bitmap_fill(iop->uptodate, nr_blocks); @@ -226,7 +231,7 @@ static int iomap_read_inline_data(const struct iomap_iter *iter, if (WARN_ON_ONCE(size > iomap->length)) return -EIO; if (offset > 0) - iop = iomap_page_create(iter->inode, folio); + iop = iomap_page_create(iter->inode, folio, 0); else iop = to_iomap_page(folio); @@ -264,7 +269,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter, return iomap_read_inline_data(iter, folio); /* zero post-eof blocks as the page may be mapped */ - iop = iomap_page_create(iter->inode, folio); + iop = iomap_page_create(iter->inode, folio, 0); iomap_adjust_read_range(iter->inode, folio, &pos, length, &poff, &plen); if (plen == 0) goto done; @@ -550,7 +555,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, size_t len, struct folio *folio) { const struct iomap *srcmap = iomap_iter_srcmap(iter); - struct iomap_page *iop = iomap_page_create(iter->inode, folio); + struct iomap_page *iop; loff_t block_size = i_blocksize(iter->inode); loff_t block_start = round_down(pos, block_size); loff_t block_end = round_up(pos + len, block_size); @@ -561,6 +566,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, return 0; folio_clear_error(folio); + iop = iomap_page_create(iter->inode, folio, iter->flags); + do { iomap_adjust_read_range(iter->inode, folio, &block_start, block_end - block_start, &poff, &plen); @@ -1332,7 +1339,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, struct writeback_control *wbc, struct inode *inode, struct folio *folio, u64 end_pos) { - struct iomap_page *iop = iomap_page_create(inode, folio); + struct iomap_page *iop = iomap_page_create(inode, folio, 0); struct iomap_ioend *ioend, *next; unsigned len = i_blocksize(inode); unsigned nblocks = i_blocks_per_folio(inode, folio); From patchwork Wed May 25 22:34:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4CC8C433FE for ; Wed, 25 May 2022 22:34:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 40AE18D000A; Wed, 25 May 2022 18:34:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F8538D0001; Wed, 25 May 2022 18:34:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0577F8D000A; Wed, 25 May 2022 18:34:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DC1658D0001 for ; Wed, 25 May 2022 18:34:52 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B26E432EC8 for ; Wed, 25 May 2022 22:34:52 +0000 (UTC) X-FDA: 79505721624.17.F6B25AE Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf09.hostedemail.com (Postfix) with ESMTP id 58039140044 for ; Wed, 25 May 2022 22:34:39 +0000 (UTC) Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtbb6031490 for ; Wed, 25 May 2022 15:34:51 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=OTUuCmx0gIY/WJKJVYl4xz78ATuAORYpOmPdNw/cmIY=; b=pivTYooeZxSKkPiQbYfUxfIcRSmlGk6uCPDoWYooEzOcs7C2Y2XuROh2yEykAOglwcW7 BiBAuFBm+ftBAeoCkMugAP8uw8ZOAdH6IGSbulOS4GwveqcJPSXac3z1qgbUVqOl+2KA Fjm3UdUZnI7TaxiUCJxNs/bC6nh1hlpgJ7Y= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g9bky6g7b-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:51 -0700 Received: from twshared6696.05.ash7.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:49 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 0B892F9E1B55; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 05/16] iomap: Add async buffered write support Date: Wed, 25 May 2022 15:34:21 -0700 Message-ID: <20220525223432.205676-6-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: gJ_PCLyk1yNOV0GpNIG8wSnmoXckAr3e X-Proofpoint-ORIG-GUID: gJ_PCLyk1yNOV0GpNIG8wSnmoXckAr3e X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_07,2022-05-25_02,2022-02-23_01 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=pivTYooe; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf09.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com" X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 58039140044 X-Stat-Signature: 5ufkq8dxysid1ew1h4mw4fjpn3muo3qr X-HE-Tag: 1653518079-66972 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds async buffered write support to iomap. This replaces the call to balance_dirty_pages_ratelimited() with the call to balance_dirty_pages_ratelimited_flags. This allows to specify if the write request is async or not. In addition this also moves the above function call to the beginning of the function. If the function call is at the end of the function and the decision is made to throttle writes, then there is no request that io-uring can wait on. By moving it to the beginning of the function, the write request is not issued, but returns -EAGAIN instead. io-uring will punt the request and process it in the io-worker. By moving the function call to the beginning of the function, the write throttling will happen one page later. Signed-off-by: Stefan Roesch Reviewed-by: Jan Kara --- fs/iomap/buffered-io.c | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 8ef77f7287aa..204ba708c9af 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -559,6 +559,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, loff_t block_size = i_blocksize(iter->inode); loff_t block_start = round_down(pos, block_size); loff_t block_end = round_up(pos + len, block_size); + unsigned int nr_blocks = i_blocks_per_folio(iter->inode, folio); size_t from = offset_in_folio(folio, pos), to = from + len; size_t poff, plen; @@ -567,6 +568,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, folio_clear_error(folio); iop = iomap_page_create(iter->inode, folio, iter->flags); + if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1) + return -EAGAIN; do { iomap_adjust_read_range(iter->inode, folio, &block_start, @@ -584,7 +587,12 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, return -EIO; folio_zero_segments(folio, poff, from, to, poff + plen); } else { - int status = iomap_read_folio_sync(block_start, folio, + int status; + + if (iter->flags & IOMAP_NOWAIT) + return -EAGAIN; + + status = iomap_read_folio_sync(block_start, folio, poff, plen, srcmap); if (status) return status; @@ -613,6 +621,9 @@ static int iomap_write_begin(const struct iomap_iter *iter, loff_t pos, unsigned fgp = FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE | FGP_NOFS; int status = 0; + if (iter->flags & IOMAP_NOWAIT) + fgp |= FGP_NOWAIT; + BUG_ON(pos + len > iter->iomap.offset + iter->iomap.length); if (srcmap != &iter->iomap) BUG_ON(pos + len > srcmap->offset + srcmap->length); @@ -750,6 +761,8 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) loff_t pos = iter->pos; ssize_t written = 0; long status = 0; + struct address_space *mapping = iter->inode->i_mapping; + unsigned int bdp_flags = (iter->flags & IOMAP_NOWAIT) ? BDP_ASYNC : 0; do { struct folio *folio; @@ -762,6 +775,11 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) bytes = min_t(unsigned long, PAGE_SIZE - offset, iov_iter_count(i)); again: + status = balance_dirty_pages_ratelimited_flags(mapping, + bdp_flags); + if (unlikely(status)) + break; + if (bytes > length) bytes = length; @@ -770,6 +788,10 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) * Otherwise there's a nasty deadlock on copying from the * same page as we're writing to, without it being marked * up-to-date. + * + * For async buffered writes the assumption is that the user + * page has already been faulted in. This can be optimized by + * faulting the user page in the prepare phase of io-uring. */ if (unlikely(fault_in_iov_iter_readable(i, bytes) == bytes)) { status = -EFAULT; @@ -781,7 +803,7 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) break; page = folio_file_page(folio, pos >> PAGE_SHIFT); - if (mapping_writably_mapped(iter->inode->i_mapping)) + if (mapping_writably_mapped(mapping)) flush_dcache_page(page); copied = copy_page_from_iter_atomic(page, offset, bytes, i); @@ -806,8 +828,6 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) pos += status; written += status; length -= status; - - balance_dirty_pages_ratelimited(iter->inode->i_mapping); } while (iov_iter_count(i) && length); return written ? written : status; @@ -825,6 +845,9 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, }; int ret; + if (iocb->ki_flags & IOCB_NOWAIT) + iter.flags |= IOMAP_NOWAIT; + while ((ret = iomap_iter(&iter, ops)) > 0) iter.processed = iomap_write_iter(&iter, i); if (iter.pos == iocb->ki_pos) From patchwork Wed May 25 22:34:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1438C433EF for ; Wed, 25 May 2022 22:35:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 521CD8D000F; Wed, 25 May 2022 18:35:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F6CD8D0001; Wed, 25 May 2022 18:35:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D0FB8D000F; Wed, 25 May 2022 18:35:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1B9598D0001 for ; Wed, 25 May 2022 18:35:01 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B08F9216C0 for ; Wed, 25 May 2022 22:35:00 +0000 (UTC) X-FDA: 79505721960.18.14ACF97 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf10.hostedemail.com (Postfix) with ESMTP id CBAC6C0012 for ; Wed, 25 May 2022 22:34:22 +0000 (UTC) Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtYcn017120 for ; Wed, 25 May 2022 15:34:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=CHkVmanun6GHuUP4Cg6nbmQ2wQHDke+Cbgl3CRSd104=; b=BSH0p7gqRVAakwZxYi2DP1KHlDyFrXrPeNbO26lWzNTCIA3jMdQtO5QXpqzNWPOwlRCy UoRqS05LSyMtdB1vrQxfVcwtvYXtfg2bkalfTNcXUFrOFUKqORM62CTWWN6uleJQX82z rxTSUK/5vpNAZ9GQBumxkGx0y/7l8PCldA8= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93vs9ebf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:59 -0700 Received: from twshared5413.23.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:57 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 121D8F9E1B59; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 06/16] fs: Add check for async buffered writes to generic_write_checks Date: Wed, 25 May 2022 15:34:22 -0700 Message-ID: <20220525223432.205676-7-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: KbAqdQuL2JvxkrVchWKKXI204cUJNqqD X-Proofpoint-GUID: KbAqdQuL2JvxkrVchWKKXI204cUJNqqD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: CBAC6C0012 X-Stat-Signature: 4c37n3cns34cx5u8u94193nnp1hqonb9 X-Rspam-User: Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=BSH0p7gq; spf=none (imf10.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653518062-401377 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This introduces the flag FMODE_BUF_WASYNC. If devices support async buffered writes, this flag can be set. It also modifies the check in generic_write_checks to take async buffered writes into consideration. Signed-off-by: Stefan Roesch --- fs/read_write.c | 4 +++- include/linux/fs.h | 3 +++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/read_write.c b/fs/read_write.c index e643aec2b0ef..175d98713b9a 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -1633,7 +1633,9 @@ int generic_write_checks_count(struct kiocb *iocb, loff_t *count) if (iocb->ki_flags & IOCB_APPEND) iocb->ki_pos = i_size_read(inode); - if ((iocb->ki_flags & IOCB_NOWAIT) && !(iocb->ki_flags & IOCB_DIRECT)) + if ((iocb->ki_flags & IOCB_NOWAIT) && + !((iocb->ki_flags & IOCB_DIRECT) || + (file->f_mode & FMODE_BUF_WASYNC))) return -EINVAL; return generic_write_check_limits(iocb->ki_filp, iocb->ki_pos, count); diff --git a/include/linux/fs.h b/include/linux/fs.h index 87b5af1d9fbe..407ad5004f54 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -177,6 +177,9 @@ typedef int (dio_iodone_t)(struct kiocb *iocb, loff_t offset, /* File supports async buffered reads */ #define FMODE_BUF_RASYNC ((__force fmode_t)0x40000000) +/* File supports async nowait buffered writes */ +#define FMODE_BUF_WASYNC ((__force fmode_t)0x80000000) + /* * Attribute flags. These should be or-ed together to figure out what * has been changed! From patchwork Wed May 25 22:34:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DE3CC433EF for ; Wed, 25 May 2022 22:34:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 90D508D000B; Wed, 25 May 2022 18:34:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 896F48D0001; Wed, 25 May 2022 18:34:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 696468D000B; Wed, 25 May 2022 18:34:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 534C48D0001 for ; Wed, 25 May 2022 18:34:54 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 234616D1 for ; Wed, 25 May 2022 22:34:54 +0000 (UTC) X-FDA: 79505721708.25.67E19C5 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf03.hostedemail.com (Postfix) with ESMTP id E7CA22002B for ; Wed, 25 May 2022 22:34:40 +0000 (UTC) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGta4k023116 for ; Wed, 25 May 2022 15:34:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=Fx9Ydqg3gwwtYMpBZFzoM6rR2Mzc7v3vEUHPpZGCjiM=; b=MeDYB2LomSaP4KXdH5Ku/CMgVclc6ihnBQIHfM9xR8smHSftkoy5UdM6XtNdqVNyTXf4 jiT1Adfcz1A31eTDmliv30Tc2irO6Q3rD1MM9C/prXw81w1QmAT0F6H4FoLmhVx1uSbc 7uB2q3tVH96ztdx+lzu9NvucoMbAQrUyfhg= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93upsd8y-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:52 -0700 Received: from twshared10560.18.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:51 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 191D7F9E1B5E; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 07/16] fs: add __remove_file_privs() with flags parameter Date: Wed, 25 May 2022 15:34:23 -0700 Message-ID: <20220525223432.205676-8-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: eTs4p9M-EMttdb8FESQRiixakE-yG6G1 X-Proofpoint-GUID: eTs4p9M-EMttdb8FESQRiixakE-yG6G1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_07,2022-05-25_02,2022-02-23_01 X-Rspam-User: X-Rspamd-Queue-Id: E7CA22002B X-Stat-Signature: f9xk7dwkf6s5otkuxmtoi3k8podzkuaw Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=MeDYB2Lo; spf=none (imf03.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspamd-Server: rspam03 X-HE-Tag: 1653518080-795942 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the function __remove_file_privs, which allows the caller to pass the kiocb flags parameter. No intended functional changes in this patch. Signed-off-by: Stefan Roesch --- fs/inode.c | 57 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 37 insertions(+), 20 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 9d9b422504d1..ac1cf5aa78c8 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2010,36 +2010,43 @@ static int __remove_privs(struct user_namespace *mnt_userns, return notify_change(mnt_userns, dentry, &newattrs, NULL); } -/* - * Remove special file priviledges (suid, capabilities) when file is written - * to or truncated. - */ -int file_remove_privs(struct file *file) +static int __file_remove_privs(struct file *file, unsigned int flags) { struct dentry *dentry = file_dentry(file); struct inode *inode = file_inode(file); + int error; int kill; - int error = 0; - /* - * Fast path for nothing security related. - * As well for non-regular files, e.g. blkdev inodes. - * For example, blkdev_write_iter() might get here - * trying to remove privs which it is not allowed to. - */ if (IS_NOSEC(inode) || !S_ISREG(inode->i_mode)) return 0; kill = dentry_needs_remove_privs(dentry); - if (kill < 0) + if (kill <= 0) return kill; - if (kill) - error = __remove_privs(file_mnt_user_ns(file), dentry, kill); + + if (flags & IOCB_NOWAIT) + return -EAGAIN; + + error = __remove_privs(file_mnt_user_ns(file), dentry, kill); if (!error) inode_has_no_xattr(inode); return error; } + +/** + * file_remove_privs - remove special file privileges (suid, capabilities) + * @file: file to remove privileges from + * + * When file is modified by a write or truncation ensure that special + * file privileges are removed. + * + * Return: 0 on success, negative errno on failure. + */ +int file_remove_privs(struct file *file) +{ + return __file_remove_privs(file, 0); +} EXPORT_SYMBOL(file_remove_privs); /** @@ -2090,18 +2097,28 @@ int file_update_time(struct file *file) } EXPORT_SYMBOL(file_update_time); -/* Caller must hold the file's inode lock */ +/** + * file_modified - handle mandated vfs changes when modifying a file + * @file: file that was modified + * + * When file has been modified ensure that special + * file privileges are removed and time settings are updated. + * + * Context: Caller must hold the file's inode lock. + * + * Return: 0 on success, negative errno on failure. + */ int file_modified(struct file *file) { - int err; + int ret; /* * Clear the security bits if the process is not being run by root. * This keeps people from modifying setuid and setgid binaries. */ - err = file_remove_privs(file); - if (err) - return err; + ret = __file_remove_privs(file, 0); + if (ret) + return ret; if (unlikely(file->f_mode & FMODE_NOCMTIME)) return 0; From patchwork Wed May 25 22:34:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB168C433F5 for ; Wed, 25 May 2022 22:35:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 663128D0012; Wed, 25 May 2022 18:35:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5EF368D0011; Wed, 25 May 2022 18:35:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B53C8D0012; Wed, 25 May 2022 18:35:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2A8D58D0011 for ; Wed, 25 May 2022 18:35:03 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0C5DA328D2 for ; Wed, 25 May 2022 22:35:03 +0000 (UTC) X-FDA: 79505722086.26.7F1CAC4 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf30.hostedemail.com (Postfix) with ESMTP id 0586580007 for ; Wed, 25 May 2022 22:34:32 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24PGtZ5C009395 for ; Wed, 25 May 2022 15:35:01 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=aWGWWAgH7Bi/du0tMMunpKLV8jXk2Gb/WQnwadv9en0=; b=kY67lZCfkPE/FnlR7QiIOhlPC1VyZQsLxT19BvGdo+klKg+7FCnJJdgyrYHGCbhr/MJE QDdjQXtCudByrBDEB8bcMyxQftGbJVCZXrswFtBXL4SJ5svQTdrqSrOKjwOM6WiUTSCj Mwkbz7sKQFkNA5ejCdSY8ABsAXQ+73pCMOE= Received: from mail.thefacebook.com ([163.114.132.120]) by m0089730.ppops.net (PPS) with ESMTPS id 3g93tx1dpk-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:35:01 -0700 Received: from twshared4937.07.ash9.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:58 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 1F4F7F9E1B62; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 08/16] fs: Split off inode_needs_update_time and __file_update_time Date: Wed, 25 May 2022 15:34:24 -0700 Message-ID: <20220525223432.205676-9-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: Zv4uXXaS98WZBDYkuvwW-sT63A66mLsN X-Proofpoint-GUID: Zv4uXXaS98WZBDYkuvwW-sT63A66mLsN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 0586580007 X-Stat-Signature: mw6jxari4s5q7bz7a1umh78a6jkqnq7h X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=kY67lZCf; spf=none (imf30.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653518072-717431 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This splits off the functions inode_needs_update_time() and __file_update_time() from the function file_update_time(). This is required to support async buffered writes. No intended functional changes in this patch. Signed-off-by: Stefan Roesch --- fs/inode.c | 76 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 50 insertions(+), 26 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index ac1cf5aa78c8..c44573a32c6a 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2049,35 +2049,18 @@ int file_remove_privs(struct file *file) } EXPORT_SYMBOL(file_remove_privs); -/** - * file_update_time - update mtime and ctime time - * @file: file accessed - * - * Update the mtime and ctime members of an inode and mark the inode - * for writeback. Note that this function is meant exclusively for - * usage in the file write path of filesystems, and filesystems may - * choose to explicitly ignore update via this function with the - * S_NOCMTIME inode flag, e.g. for network filesystem where these - * timestamps are handled by the server. This can return an error for - * file systems who need to allocate space in order to update an inode. - */ - -int file_update_time(struct file *file) +static int inode_needs_update_time(struct inode *inode, struct timespec64 *now) { - struct inode *inode = file_inode(file); - struct timespec64 now; int sync_it = 0; - int ret; /* First try to exhaust all avenues to not sync */ if (IS_NOCMTIME(inode)) return 0; - now = current_time(inode); - if (!timespec64_equal(&inode->i_mtime, &now)) + if (!timespec64_equal(&inode->i_mtime, now)) sync_it = S_MTIME; - if (!timespec64_equal(&inode->i_ctime, &now)) + if (!timespec64_equal(&inode->i_ctime, now)) sync_it |= S_CTIME; if (IS_I_VERSION(inode) && inode_iversion_need_inc(inode)) @@ -2086,15 +2069,50 @@ int file_update_time(struct file *file) if (!sync_it) return 0; - /* Finally allowed to write? Takes lock. */ - if (__mnt_want_write_file(file)) - return 0; + return sync_it; +} + +static int __file_update_time(struct file *file, struct timespec64 *now, + int sync_mode) +{ + int ret = 0; + struct inode *inode = file_inode(file); - ret = inode_update_time(inode, &now, sync_it); - __mnt_drop_write_file(file); + /* try to update time settings */ + if (!__mnt_want_write_file(file)) { + ret = inode_update_time(inode, now, sync_mode); + __mnt_drop_write_file(file); + } return ret; } + + /** + * file_update_time - update mtime and ctime time + * @file: file accessed + * + * Update the mtime and ctime members of an inode and mark the inode for + * writeback. Note that this function is meant exclusively for usage in + * the file write path of filesystems, and filesystems may choose to + * explicitly ignore updates via this function with the _NOCMTIME inode + * flag, e.g. for network filesystem where these imestamps are handled + * by the server. This can return an error for file systems who need to + * allocate space in order to update an inode. + * + * Return: 0 on success, negative errno on failure. + */ +int file_update_time(struct file *file) +{ + int ret; + struct inode *inode = file_inode(file); + struct timespec64 now = current_time(inode); + + ret = inode_needs_update_time(inode, &now); + if (ret <= 0) + return ret; + + return __file_update_time(file, &now, ret); +} EXPORT_SYMBOL(file_update_time); /** @@ -2111,6 +2129,8 @@ EXPORT_SYMBOL(file_update_time); int file_modified(struct file *file) { int ret; + struct inode *inode = file_inode(file); + struct timespec64 now = current_time(inode); /* * Clear the security bits if the process is not being run by root. @@ -2123,7 +2143,11 @@ int file_modified(struct file *file) if (unlikely(file->f_mode & FMODE_NOCMTIME)) return 0; - return file_update_time(file); + ret = inode_needs_update_time(inode, &now); + if (ret <= 0) + return ret; + + return __file_update_time(file, &now, ret); } EXPORT_SYMBOL(file_modified); From patchwork Wed May 25 22:34:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14385C433FE for ; Wed, 25 May 2022 22:35:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 926BC8D0001; Wed, 25 May 2022 18:35:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8201B8D0012; Wed, 25 May 2022 18:35:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A8178D0010; Wed, 25 May 2022 18:35:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2FAA88D0001 for ; Wed, 25 May 2022 18:35:01 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 10CF132F67 for ; Wed, 25 May 2022 22:35:01 +0000 (UTC) X-FDA: 79505722002.11.24085C7 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf27.hostedemail.com (Postfix) with ESMTP id 3238A4000D for ; Wed, 25 May 2022 22:34:56 +0000 (UTC) Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtd97011313 for ; Wed, 25 May 2022 15:34:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=QH4r+VUDqXluEqZvZRr1nBur/lZ6NB5gKdOPalXnQRE=; b=e7L2o6lK70P4QsXSQgyaUlnr1YCBRUtJig7n3v+Zcxl77YqqfSCu40PzRb1XQGjKPBng roJHalT3B32YTXhuDvLHQBnr3E9RCND2CDRma+AJWy0+Wo7yzfXz2SASwNXlzu4HE8gL Vmb8IdOly246n0HDwsXiReXoG0wVoUAf19M= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93uj1ct3-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:59 -0700 Received: from twshared5413.23.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::e) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:57 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 25281F9E1B67; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 09/16] fs: Add async write file modification handling. Date: Wed, 25 May 2022 15:34:25 -0700 Message-ID: <20220525223432.205676-10-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: rAaSNM4I_cLH_n_tRAHnDiXUhvG19CAo X-Proofpoint-ORIG-GUID: rAaSNM4I_cLH_n_tRAHnDiXUhvG19CAo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspamd-Queue-Id: 3238A4000D X-Stat-Signature: ob5r16kmcguynjnwe7xwgo9yhnx4ck98 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=e7L2o6lK; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf27.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com" X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1653518096-812018 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds a file_modified_async() function to return -EAGAIN if the request either requires to remove privileges or needs to update the file modification time. This is required for async buffered writes, so the request gets handled in the io worker of io-uring. Signed-off-by: Stefan Roesch --- fs/inode.c | 43 +++++++++++++++++++++++++++++++++++++++++-- include/linux/fs.h | 1 + 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index c44573a32c6a..4503bed063e7 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2116,17 +2116,21 @@ int file_update_time(struct file *file) EXPORT_SYMBOL(file_update_time); /** - * file_modified - handle mandated vfs changes when modifying a file + * file_modified_flags - handle mandated vfs changes when modifying a file * @file: file that was modified + * @flags: kiocb flags * * When file has been modified ensure that special * file privileges are removed and time settings are updated. * + * If IOCB_NOWAIT is set, special file privileges will not be removed and + * time settings will not be updated. It will return -EAGAIN. + * * Context: Caller must hold the file's inode lock. * * Return: 0 on success, negative errno on failure. */ -int file_modified(struct file *file) +static int file_modified_flags(struct file *file, int flags) { int ret; struct inode *inode = file_inode(file); @@ -2146,11 +2150,46 @@ int file_modified(struct file *file) ret = inode_needs_update_time(inode, &now); if (ret <= 0) return ret; + if (flags & IOCB_NOWAIT) + return -EAGAIN; return __file_update_time(file, &now, ret); } + +/** + * file_modified - handle mandated vfs changes when modifying a file + * @file: file that was modified + * + * When file has been modified ensure that special + * file privileges are removed and time settings are updated. + * + * Context: Caller must hold the file's inode lock. + * + * Return: 0 on success, negative errno on failure. + */ +int file_modified(struct file *file) +{ + return file_modified_flags(file, 0); +} EXPORT_SYMBOL(file_modified); +/** + * kiocb_modified - handle mandated vfs changes when modifying a file + * @iocb: iocb that was modified + * + * When file has been modified ensure that special + * file privileges are removed and time settings are updated. + * + * Context: Caller must hold the file's inode lock. + * + * Return: 0 on success, negative errno on failure. + */ +int kiocb_modified(struct kiocb *iocb) +{ + return file_modified_flags(iocb->ki_filp, iocb->ki_flags); +} +EXPORT_SYMBOL_GPL(kiocb_modified); + int inode_needs_sync(struct inode *inode) { if (IS_SYNC(inode)) diff --git a/include/linux/fs.h b/include/linux/fs.h index 407ad5004f54..2d9b3afcb4a5 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2384,6 +2384,7 @@ static inline void file_accessed(struct file *file) } extern int file_modified(struct file *file); +int kiocb_modified(struct kiocb *iocb); int sync_inode_metadata(struct inode *inode, int wait); From patchwork Wed May 25 22:34:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67625C433FE for ; Wed, 25 May 2022 22:34:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 976178D000D; Wed, 25 May 2022 18:34:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9241A8D0001; Wed, 25 May 2022 18:34:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74CF88D000D; Wed, 25 May 2022 18:34:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5EFE68D0001 for ; Wed, 25 May 2022 18:34:55 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 28BE060193 for ; Wed, 25 May 2022 22:34:55 +0000 (UTC) X-FDA: 79505721750.01.6479E17 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf28.hostedemail.com (Postfix) with ESMTP id BD784C0025 for ; Wed, 25 May 2022 22:34:21 +0000 (UTC) Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtZxu017198 for ; Wed, 25 May 2022 15:34:53 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=BE5eLO5Z7AZICxMCDTAcJeiKudbeydb7d4UMFp50BLA=; b=I2su/43ctP7xinp5heyI+ieMK5r2Aa9OkURbSwM+dAE8k7Mb3x6fOD61S0gdvEcdg/Bh MuF36VB4cZWS2A+fvxnRbpgPddhdop/wFcEFMIt1mH0n/6BSZptJMARdlk4U5eGA1Fgz xGCt07tFT5Su2+LeYS4VgZF5YfYArBg9UWk= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93vs9eb1-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:53 -0700 Received: from twshared4937.07.ash9.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:11d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:52 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 2AD47F9E1B6C; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 10/16] fs: Optimization for concurrent file time updates. Date: Wed, 25 May 2022 15:34:26 -0700 Message-ID: <20220525223432.205676-11-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: qp80cOHAU429auNQcpnsHouklnczUA86 X-Proofpoint-GUID: qp80cOHAU429auNQcpnsHouklnczUA86 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspam-User: X-Rspamd-Queue-Id: BD784C0025 X-Stat-Signature: rtcjg69ykogwx9r5tstkg75e83rtbz8p Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="I2su/43c"; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf28.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com" X-Rspamd-Server: rspam04 X-HE-Tag: 1653518061-596109 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This introduces the S_PENDING_TIME flag. If an async buffered write needs to update the time, it cannot be processed in the fast path of io-uring. When a time update is pending this flag is set for async buffered writes. Other concurrent async buffered writes for the same file do not need to wait while this time update is pending. This reduces the number of async buffered writes that need to get punted to the io-workers in io-uring. Signed-off-by: Stefan Roesch --- fs/inode.c | 11 +++++++++-- include/linux/fs.h | 3 +++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 4503bed063e7..7185d860d423 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -2150,10 +2150,17 @@ static int file_modified_flags(struct file *file, int flags) ret = inode_needs_update_time(inode, &now); if (ret <= 0) return ret; - if (flags & IOCB_NOWAIT) + if (flags & IOCB_NOWAIT) { + if (IS_PENDING_TIME(inode)) + return 0; + + inode_set_flags(inode, S_PENDING_TIME, S_PENDING_TIME); return -EAGAIN; + } - return __file_update_time(file, &now, ret); + ret = __file_update_time(file, &now, ret); + inode_set_flags(inode, 0, S_PENDING_TIME); + return ret; } /** diff --git a/include/linux/fs.h b/include/linux/fs.h index 2d9b3afcb4a5..5924c90eab1d 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2143,6 +2143,8 @@ struct super_operations { #define S_CASEFOLD (1 << 15) /* Casefolded file */ #define S_VERITY (1 << 16) /* Verity file (using fs/verity/) */ #define S_KERNEL_FILE (1 << 17) /* File is in use by the kernel (eg. fs/cachefiles) */ +#define S_PENDING_TIME (1 << 18) /* File update time is pending */ + /* * Note that nosuid etc flags are inode-specific: setting some file-system @@ -2185,6 +2187,7 @@ static inline bool sb_rdonly(const struct super_block *sb) { return sb->s_flags #define IS_ENCRYPTED(inode) ((inode)->i_flags & S_ENCRYPTED) #define IS_CASEFOLDED(inode) ((inode)->i_flags & S_CASEFOLD) #define IS_VERITY(inode) ((inode)->i_flags & S_VERITY) +#define IS_PENDING_TIME(inode) ((inode)->i_flags & S_PENDING_TIME) #define IS_WHITEOUT(inode) (S_ISCHR(inode->i_mode) && \ (inode)->i_rdev == WHITEOUT_DEV) From patchwork Wed May 25 22:34:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C9E8C433EF for ; Wed, 25 May 2022 22:35:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB9E58D0010; Wed, 25 May 2022 18:35:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D65828D0011; Wed, 25 May 2022 18:35:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA2D58D0010; Wed, 25 May 2022 18:35:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6D8248D0011 for ; Wed, 25 May 2022 18:35:01 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 43F561550 for ; Wed, 25 May 2022 22:35:01 +0000 (UTC) X-FDA: 79505722002.28.30A2AFB Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf21.hostedemail.com (Postfix) with ESMTP id 8F8B61C000A for ; Wed, 25 May 2022 22:34:48 +0000 (UTC) Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtb4S009791 for ; Wed, 25 May 2022 15:35:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=Yh7+qvp7wia1iDmc7QPLBPdsuYBbsRWXU5abdWcnXVM=; b=rZt8rgDGV53gAZwgT+wWkxNjHUlk/JEsDz9LJ13MLJ0N7c+d/xK0vZRuhr0piokzIk+F 4jvx1gPNlxQ+WTUNZMT8cgww+Db3QL2qwpnbiWlcdP5ewz2Bo6FhyrqP/lUiBmvXXcM3 WvD/JWB5I9ffdwC8fBh6YZa3rq8g79WMkFg= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93tvsejd-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:59 -0700 Received: from twshared5413.23.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:57 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 306D7F9E1B71; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 11/16] io_uring: Add support for async buffered writes Date: Wed, 25 May 2022 15:34:27 -0700 Message-ID: <20220525223432.205676-12-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 4IRLuy8HgjBJj7i9uG6zGEtpMPT4FfQD X-Proofpoint-ORIG-GUID: 4IRLuy8HgjBJj7i9uG6zGEtpMPT4FfQD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 8F8B61C000A X-Stat-Signature: bj4wgsrauuke4sqffwsu9cf9rcjszb6j X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=rZt8rgDG; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf21.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com" X-HE-Tag: 1653518088-773644 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This enables the async buffered writes for the filesystems that support async buffered writes in io-uring. Buffered writes are enabled for blocks that are already in the page cache or can be acquired with noio. Signed-off-by: Stefan Roesch --- fs/io_uring.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 9f1c682d7caf..c0771e215669 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -4257,7 +4257,7 @@ static inline int io_iter_do_read(struct io_kiocb *req, struct iov_iter *iter) return -EINVAL; } -static bool need_read_all(struct io_kiocb *req) +static bool need_complete_io(struct io_kiocb *req) { return req->flags & REQ_F_ISREG || S_ISBLK(file_inode(req->file)->i_mode); @@ -4386,7 +4386,7 @@ static int io_read(struct io_kiocb *req, unsigned int issue_flags) } else if (ret == -EIOCBQUEUED) { goto out_free; } else if (ret == req->cqe.res || ret <= 0 || !force_nonblock || - (req->flags & REQ_F_NOWAIT) || !need_read_all(req)) { + (req->flags & REQ_F_NOWAIT) || !need_complete_io(req)) { /* read all, failed, already did sync or don't want to retry */ goto done; } @@ -4482,9 +4482,10 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags) if (unlikely(!io_file_supports_nowait(req))) goto copy_iov; - /* file path doesn't support NOWAIT for non-direct_IO */ - if (force_nonblock && !(kiocb->ki_flags & IOCB_DIRECT) && - (req->flags & REQ_F_ISREG)) + /* File path supports NOWAIT for non-direct_IO only for block devices. */ + if (!(kiocb->ki_flags & IOCB_DIRECT) && + !(kiocb->ki_filp->f_mode & FMODE_BUF_WASYNC) && + (req->flags & REQ_F_ISREG)) goto copy_iov; kiocb->ki_flags |= IOCB_NOWAIT; @@ -4538,6 +4539,24 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags) /* IOPOLL retry should happen for io-wq threads */ if (ret2 == -EAGAIN && (req->ctx->flags & IORING_SETUP_IOPOLL)) goto copy_iov; + + if (ret2 != req->cqe.res && ret2 >= 0 && need_complete_io(req)) { + struct io_async_rw *rw; + + /* This is a partial write. The file pos has already been + * updated, setup the async struct to complete the request + * in the worker. Also update bytes_done to account for + * the bytes already written. + */ + iov_iter_save_state(&s->iter, &s->iter_state); + ret = io_setup_async_rw(req, iovec, s, true); + + rw = req->async_data; + if (rw) + rw->bytes_done += ret2; + + return ret ? ret : -EAGAIN; + } done: kiocb_done(req, ret2, issue_flags); } else { From patchwork Wed May 25 22:34:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7128BC433F5 for ; Wed, 25 May 2022 22:35:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF0198D000E; Wed, 25 May 2022 18:35:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DC31B8D0001; Wed, 25 May 2022 18:35:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C65BA8D000E; Wed, 25 May 2022 18:35:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B2FB38D0001 for ; Wed, 25 May 2022 18:35:00 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 859A8814DE for ; Wed, 25 May 2022 22:35:00 +0000 (UTC) X-FDA: 79505721960.04.2326668 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf07.hostedemail.com (Postfix) with ESMTP id 727DA4002A for ; Wed, 25 May 2022 22:34:49 +0000 (UTC) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtcE9023212 for ; Wed, 25 May 2022 15:34:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=/DrIfFwryF28jpgKYd8aC2BVP4/JsBupN+LpZHKHGXE=; b=pHwPfg8Vp8ExMjHcLu+XeqbDdcvEUOTQ04c2dolXC+9DrPhey/sijLUCwD/vJcoxP+Cf ihP8G15EXQA6wWB+RnP0XT6Q/7XKhulbf8IOVgTMMt+fEFnuoSbkGh/pp2EMiXvGIh2n VVxASgojbvlUMof62MHZhP7dGPu2xGUSOIE= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93upsda7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:34:58 -0700 Received: from twshared35748.07.ash9.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:34:58 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 36285F9E1B76; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 12/16] io_uring: Add tracepoint for short writes Date: Wed, 25 May 2022 15:34:28 -0700 Message-ID: <20220525223432.205676-13-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: om-W_lYUohk2STYxReyxCa3J95ylm2tC X-Proofpoint-GUID: om-W_lYUohk2STYxReyxCa3J95ylm2tC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_07,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 727DA4002A X-Stat-Signature: iy7p3pw5tedd81i8unnzfd74n8jm79un X-Rspam-User: Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=pHwPfg8V; dmarc=pass (policy=reject) header.from=fb.com; spf=none (imf07.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com" X-HE-Tag: 1653518089-754072 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the io_uring_short_write tracepoint to io_uring. A short write is issued if not all pages that are required for a write are in the page cache and the async buffered writes have to return EAGAIN. Signed-off-by: Stefan Roesch --- fs/io_uring.c | 3 +++ include/trace/events/io_uring.h | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index c0771e215669..9ab68138f442 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -4543,6 +4543,9 @@ static int io_write(struct io_kiocb *req, unsigned int issue_flags) if (ret2 != req->cqe.res && ret2 >= 0 && need_complete_io(req)) { struct io_async_rw *rw; + trace_io_uring_short_write(req->ctx, kiocb->ki_pos - ret2, + req->cqe.res, ret2); + /* This is a partial write. The file pos has already been * updated, setup the async struct to complete the request * in the worker. Also update bytes_done to account for diff --git a/include/trace/events/io_uring.h b/include/trace/events/io_uring.h index 66fcc5a1a5b1..25df513660cc 100644 --- a/include/trace/events/io_uring.h +++ b/include/trace/events/io_uring.h @@ -600,6 +600,31 @@ TRACE_EVENT(io_uring_cqe_overflow, __entry->cflags, __entry->ocqe) ); +TRACE_EVENT(io_uring_short_write, + + TP_PROTO(void *ctx, u64 fpos, u64 wanted, u64 got), + + TP_ARGS(ctx, fpos, wanted, got), + + TP_STRUCT__entry( + __field(void *, ctx) + __field(u64, fpos) + __field(u64, wanted) + __field(u64, got) + ), + + TP_fast_assign( + __entry->ctx = ctx; + __entry->fpos = fpos; + __entry->wanted = wanted; + __entry->got = got; + ), + + TP_printk("ring %p, fpos %lld, wanted %lld, got %lld", + __entry->ctx, __entry->fpos, + __entry->wanted, __entry->got) +); + #endif /* _TRACE_IO_URING_H */ /* This part must be outside protection */ From patchwork Wed May 25 22:34:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A0B3C433EF for ; Wed, 25 May 2022 22:35:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2089B8D0011; Wed, 25 May 2022 18:35:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 191DD8D0015; Wed, 25 May 2022 18:35:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D24E18D0011; Wed, 25 May 2022 18:35:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B70598D0011 for ; Wed, 25 May 2022 18:35:06 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9244E216D2 for ; Wed, 25 May 2022 22:35:06 +0000 (UTC) X-FDA: 79505722212.06.9E70F58 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf05.hostedemail.com (Postfix) with ESMTP id B4A91100038 for ; Wed, 25 May 2022 22:34:36 +0000 (UTC) Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGteLd009791 for ; Wed, 25 May 2022 15:35:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=KkKc7JfZWvzdwU4gQJqw11EVWJb6LJMB0THy49VxYKA=; b=k/BfEgt+VinUcvXkmSe0rRGMfXU0cY4QFD0peDfky3/WsAkFaWTw9ezkrjMOzvqx1Mok W4veRs2+V1WDrrFZOnMnryjnsEnh9zEc67AKl3XGgPnOtmlPVd5R7f0piEvEis1jezad zZCpmhnV/FuXGcAadUo/JvIVVw3n9JUyv60= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93uphc08-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:35:05 -0700 Received: from twshared8508.05.ash9.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:11d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:35:04 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 3BE35F9E1B7B; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 13/16] xfs: Specify lockmode when calling xfs_ilock_for_iomap() Date: Wed, 25 May 2022 15:34:29 -0700 Message-ID: <20220525223432.205676-14-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: tCv3vbbgwCPYj9gWv6RsVxcpzvWg27hQ X-Proofpoint-ORIG-GUID: tCv3vbbgwCPYj9gWv6RsVxcpzvWg27hQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspam-User: X-Rspamd-Queue-Id: B4A91100038 X-Stat-Signature: gmixtpcia35xrgkx6wai5g9srpqxgefe Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="k/BfEgt+"; spf=none (imf05.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspamd-Server: rspam03 X-HE-Tag: 1653518076-41235 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch changes the helper function xfs_ilock_for_iomap such that the lock mode must be passed in. Signed-off-by: Stefan Roesch --- fs/xfs/xfs_iomap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index e552ce541ec2..3aa60e53a181 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -659,7 +659,7 @@ xfs_ilock_for_iomap( unsigned flags, unsigned *lockmode) { - unsigned mode = XFS_ILOCK_SHARED; + unsigned int mode = *lockmode; bool is_write = flags & (IOMAP_WRITE | IOMAP_ZERO); /* @@ -737,7 +737,7 @@ xfs_direct_write_iomap_begin( int nimaps = 1, error = 0; bool shared = false; u16 iomap_flags = 0; - unsigned lockmode; + unsigned int lockmode = XFS_ILOCK_SHARED; ASSERT(flags & (IOMAP_WRITE | IOMAP_ZERO)); @@ -1167,7 +1167,7 @@ xfs_read_iomap_begin( xfs_fileoff_t end_fsb = xfs_iomap_end_fsb(mp, offset, length); int nimaps = 1, error = 0; bool shared = false; - unsigned lockmode; + unsigned int lockmode = XFS_ILOCK_SHARED; ASSERT(!(flags & (IOMAP_WRITE | IOMAP_ZERO))); From patchwork Wed May 25 22:34:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CEEBC433FE for ; Wed, 25 May 2022 22:35:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D29598D0014; Wed, 25 May 2022 18:35:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C39648D0015; Wed, 25 May 2022 18:35:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9BDDF8D0014; Wed, 25 May 2022 18:35:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7BE2A8D0011 for ; Wed, 25 May 2022 18:35:06 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 635B0604CD for ; Wed, 25 May 2022 22:35:06 +0000 (UTC) X-FDA: 79505722212.24.9BD2717 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf21.hostedemail.com (Postfix) with ESMTP id ACD5C1C000A for ; Wed, 25 May 2022 22:34:53 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24PGtapp009459 for ; Wed, 25 May 2022 15:35:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=j/dY8M7U0980IK7tXqhTnWDktIso9RCIpG7xYensPsA=; b=p7bHRri7LXOaLHKaj6Fdm0YE8u2VWdobPaat5hVG7PaL0JCX5V53fNaGhI+3yCohtCF1 hO05yklFS7Ug4qr840doLbB3kTjWH3T/7xi3FOQh0zPQEUXMt2w/qAC3+YcGZiMZGOFf CyiZz5gVlQXDltW/HDi0g2oG6xT7n/5GW4s= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0089730.ppops.net (PPS) with ESMTPS id 3g93tx1dr0-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:35:05 -0700 Received: from twshared10560.18.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:35:03 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 4265BF9E1B81; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 14/16] xfs: Change function signature of xfs_ilock_iocb() Date: Wed, 25 May 2022 15:34:30 -0700 Message-ID: <20220525223432.205676-15-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: 9t9FJEjyC5x2JfCS7u09tZLf_gLEmbk0 X-Proofpoint-GUID: 9t9FJEjyC5x2JfCS7u09tZLf_gLEmbk0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: ACD5C1C000A X-Stat-Signature: pw6k963cydutjyqueghnppaifkg67yyw X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=p7bHRri7; spf=none (imf21.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653518093-321599 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This changes the function signature of the function xfs_ilock_iocb(): - the parameter iocb is replaced with ip, passing in an xfs_inode - the parameter iocb_flags is added to be able to pass in the iocb flags This allows to call the function from xfs_file_buffered_writes. All the callers are changed accordingly. Signed-off-by: Stefan Roesch --- fs/xfs/xfs_file.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 5bddb1e9e0b3..50dea07f5e56 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -190,14 +190,13 @@ xfs_file_fsync( return error; } -static int +static inline int xfs_ilock_iocb( - struct kiocb *iocb, + struct xfs_inode *ip, + int iocb_flags, unsigned int lock_mode) { - struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp)); - - if (iocb->ki_flags & IOCB_NOWAIT) { + if (iocb_flags & IOCB_NOWAIT) { if (!xfs_ilock_nowait(ip, lock_mode)) return -EAGAIN; } else { @@ -222,7 +221,7 @@ xfs_file_dio_read( file_accessed(iocb->ki_filp); - ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + ret = xfs_ilock_iocb(ip, iocb->ki_flags, XFS_IOLOCK_SHARED); if (ret) return ret; ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0, 0); @@ -244,7 +243,7 @@ xfs_file_dax_read( if (!iov_iter_count(to)) return 0; /* skip atime */ - ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + ret = xfs_ilock_iocb(ip, iocb->ki_flags, XFS_IOLOCK_SHARED); if (ret) return ret; ret = dax_iomap_rw(iocb, to, &xfs_read_iomap_ops); @@ -264,7 +263,7 @@ xfs_file_buffered_read( trace_xfs_file_buffered_read(iocb, to); - ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); + ret = xfs_ilock_iocb(ip, iocb->ki_flags, XFS_IOLOCK_SHARED); if (ret) return ret; ret = generic_file_read_iter(iocb, to); @@ -343,7 +342,7 @@ xfs_file_write_checks( if (*iolock == XFS_IOLOCK_SHARED && !IS_NOSEC(inode)) { xfs_iunlock(ip, *iolock); *iolock = XFS_IOLOCK_EXCL; - error = xfs_ilock_iocb(iocb, *iolock); + error = xfs_ilock_iocb(ip, iocb->ki_flags, *iolock); if (error) { *iolock = 0; return error; @@ -516,7 +515,7 @@ xfs_file_dio_write_aligned( int iolock = XFS_IOLOCK_SHARED; ssize_t ret; - ret = xfs_ilock_iocb(iocb, iolock); + ret = xfs_ilock_iocb(ip, iocb->ki_flags, iolock); if (ret) return ret; ret = xfs_file_write_checks(iocb, from, &iolock); @@ -583,7 +582,7 @@ xfs_file_dio_write_unaligned( flags = IOMAP_DIO_FORCE_WAIT; } - ret = xfs_ilock_iocb(iocb, iolock); + ret = xfs_ilock_iocb(ip, iocb->ki_flags, iolock); if (ret) return ret; @@ -659,7 +658,7 @@ xfs_file_dax_write( ssize_t ret, error = 0; loff_t pos; - ret = xfs_ilock_iocb(iocb, iolock); + ret = xfs_ilock_iocb(ip, iocb->ki_flags, iolock); if (ret) return ret; ret = xfs_file_write_checks(iocb, from, &iolock); From patchwork Wed May 25 22:34:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39F37C433EF for ; Wed, 25 May 2022 22:35:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 82D548D0013; Wed, 25 May 2022 18:35:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A5E68D0011; Wed, 25 May 2022 18:35:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51E078D0013; Wed, 25 May 2022 18:35:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3C1888D0011 for ; Wed, 25 May 2022 18:35:04 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 1C275601B0 for ; Wed, 25 May 2022 22:35:04 +0000 (UTC) X-FDA: 79505722128.08.C6EF9DE Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf04.hostedemail.com (Postfix) with ESMTP id 512894003F for ; Wed, 25 May 2022 22:34:47 +0000 (UTC) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24PGtaL6012100 for ; Wed, 25 May 2022 15:35:03 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=xrXV6DDKAu8edht2TcHb7XC2Tqf/DAsX1r3VS5rTvoo=; b=nq/FnwgdSIxm3G9FiuG3qyoerWRrW88Z7auP/XV0XDH6CQ1Q4on+mEQUjeHlNPFYE33n YfyLfvGQIFSss8nr2vgfVMvzQY7nXodO3hkRAAC4zhf7eKnoQ8UKQHpX4W6rG+Y6X+1K iOURdEUMw7fswKxKwPRAHTS577uxgBnQAEY= Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net (PPS) with ESMTPS id 3g9qtuafrd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:35:02 -0700 Received: from twshared4937.07.ash9.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:11d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:35:01 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 4A19FF9E1B83; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 15/16] xfs: Add async buffered write support Date: Wed, 25 May 2022 15:34:31 -0700 Message-ID: <20220525223432.205676-16-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 2_fqxl02weTaWOZaQD_Iv0E-fRpjSyjj X-Proofpoint-ORIG-GUID: 2_fqxl02weTaWOZaQD_Iv0E-fRpjSyjj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_07,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 512894003F X-Stat-Signature: x51it63jnn56fuqgq41itjcrc6p3n858 X-Rspam-User: Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b="nq/Fnwgd"; spf=none (imf04.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.153.30) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1653518087-978291 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds the async buffered write support to XFS. For async buffered write requests, the request will return -EAGAIN if the ilock cannot be obtained immediately. Signed-off-by: Stefan Roesch --- fs/xfs/xfs_file.c | 9 ++++----- fs/xfs/xfs_iomap.c | 5 ++++- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 50dea07f5e56..e9d615f4c209 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -409,7 +409,7 @@ xfs_file_write_checks( spin_unlock(&ip->i_flags_lock); out: - return file_modified(file); + return kiocb_modified(iocb); } static int @@ -701,12 +701,11 @@ xfs_file_buffered_write( bool cleared_space = false; int iolock; - if (iocb->ki_flags & IOCB_NOWAIT) - return -EOPNOTSUPP; - write_retry: iolock = XFS_IOLOCK_EXCL; - xfs_ilock(ip, iolock); + ret = xfs_ilock_iocb(ip, iocb->ki_flags, iolock); + if (ret) + return ret; ret = xfs_file_write_checks(iocb, from, &iolock); if (ret) diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 3aa60e53a181..af8c50140f0f 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -881,6 +881,7 @@ xfs_buffered_write_iomap_begin( bool eof = false, cow_eof = false, shared = false; int allocfork = XFS_DATA_FORK; int error = 0; + unsigned int lockmode = XFS_ILOCK_EXCL; if (xfs_is_shutdown(mp)) return -EIO; @@ -892,7 +893,9 @@ xfs_buffered_write_iomap_begin( ASSERT(!XFS_IS_REALTIME_INODE(ip)); - xfs_ilock(ip, XFS_ILOCK_EXCL); + error = xfs_ilock_for_iomap(ip, flags, &lockmode); + if (error) + return error; if (XFS_IS_CORRUPT(mp, !xfs_ifork_has_extents(&ip->i_df)) || XFS_TEST_ERROR(false, mp, XFS_ERRTAG_BMAPIFORMAT)) { From patchwork Wed May 25 22:34:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Roesch X-Patchwork-Id: 12861785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B2D8C433FE for ; Wed, 25 May 2022 22:35:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B7A78D0016; Wed, 25 May 2022 18:35:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7F3A98D0015; Wed, 25 May 2022 18:35:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 669D58D0016; Wed, 25 May 2022 18:35:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 426618D0015 for ; Wed, 25 May 2022 18:35:11 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 13D6C12018A for ; Wed, 25 May 2022 22:35:11 +0000 (UTC) X-FDA: 79505722422.30.C6E41D0 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf06.hostedemail.com (Postfix) with ESMTP id 1FD7F180007 for ; Wed, 25 May 2022 22:35:06 +0000 (UTC) Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24PGtZmF017172 for ; Wed, 25 May 2022 15:35:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=KHrcNgpGoec04+4iaa68lI4ptLIlXN7oy1iPyC/evOA=; b=pSLJ95zwQCcSqAKcJMnQaAr+751tCdKASRfNLWwNM9SDX4vetjajODxve2cqaqs8LwHb HckwoIeOMxEqTVU+oeitSVtAn5q1OpcC30week6UbHRVKihsxewzHpNh3riJcymvw8G2 IfDjsO3qJ94wZAxVTqp4loz8mZ9ec4BDwAU= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g93vs9eck-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 25 May 2022 15:35:09 -0700 Received: from twshared19572.14.frc2.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Wed, 25 May 2022 15:35:06 -0700 Received: by devvm225.atn0.facebook.com (Postfix, from userid 425415) id 5105CF9E1B85; Wed, 25 May 2022 15:34:35 -0700 (PDT) From: Stefan Roesch To: , , , , CC: , , , Subject: [PATCH v5 16/16] xfs: Enable async buffered write support Date: Wed, 25 May 2022 15:34:32 -0700 Message-ID: <20220525223432.205676-17-shr@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220525223432.205676-1-shr@fb.com> References: <20220525223432.205676-1-shr@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: JifsKPxmUqLBgcaNSUbgG62Ktpje8A47 X-Proofpoint-GUID: JifsKPxmUqLBgcaNSUbgG62Ktpje8A47 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.874,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-25_06,2022-05-25_02,2022-02-23_01 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 1FD7F180007 X-Stat-Signature: oztb7oymh4336hggescpscp8ef8ahwhx Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=pSLJ95zw; spf=none (imf06.hostedemail.com: domain of "prvs=61440c7716=shr@fb.com" has no SPF policy when checking 67.231.145.42) smtp.mailfrom="prvs=61440c7716=shr@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-Rspam-User: X-HE-Tag: 1653518106-225524 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This turns on the async buffered write support for XFS. Signed-off-by: Stefan Roesch --- fs/xfs/xfs_file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index e9d615f4c209..2297770364b0 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -1169,7 +1169,7 @@ xfs_file_open( return -EFBIG; if (xfs_is_shutdown(XFS_M(inode->i_sb))) return -EIO; - file->f_mode |= FMODE_NOWAIT | FMODE_BUF_RASYNC; + file->f_mode |= FMODE_NOWAIT | FMODE_BUF_RASYNC | FMODE_BUF_WASYNC; return 0; }