From patchwork Tue Jan 7 00:25:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bernd Schubert X-Patchwork-Id: 13927931 Received: from outbound-ip191b.ess.barracuda.com (outbound-ip191b.ess.barracuda.com [209.222.82.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D073F7082D for ; Tue, 7 Jan 2025 00:25:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=209.222.82.124 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736209545; cv=fail; b=AcKPdmE5p71M9yvUiTZ8pXn5aaxwFylbYEt56/FPC6pY7JEzjFOHvD5ML7y4L78r8R7emGpW8Ak6HDCu/iehJoifEAQnBRUFlCYgUEZpe/Kb3e1S38FNbQmFtCYk8ry4K4PlPS9nhM0atgj7GsyUL6akBmQqG32RHIYCe7ZiILc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736209545; c=relaxed/simple; bh=AvsBUPQfhNKAli9QrA4nArV8nI+dbIIZOg4iPrMRRVQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WzF9FtWL0AmIWsAthlZkzOkX4g5gmhCJSUOZ0TkrL8OuyFtUTJMqjGPpqX4TSQ/yebQsRKEel8L+G+m272RKhtZULhMt1TFf7W1YruVNGiz00piJGMHDO2+VDZocHXv++szOSS7OSLbPxFkb/Yl/z9tFHF4oEZTFJGSQMbZJ3Xw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ddn.com; spf=pass smtp.mailfrom=ddn.com; dkim=pass (1024-bit key) header.d=ddn.com header.i=@ddn.com header.b=RQuHq97C; arc=fail smtp.client-ip=209.222.82.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ddn.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ddn.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ddn.com header.i=@ddn.com header.b="RQuHq97C" Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2168.outbound.protection.outlook.com [104.47.58.168]) by mx-outbound-ea46-177.us-east-2c.ess.aws.cudaops.com (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 07 Jan 2025 00:25:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=AKGzkzuvqCZ3Xii7xuU56+rUbvvyLDJIFN9nOAwBeR7jKD9HysPo5YlKABsvnCkmDAPlmGs4ZSe7be761utUb8NRu0lEA8GFxKKptgc8eojKtF7NwWqV8i1cjSB3aD+L8SxjgV0IPWPKf04Bpz/R39rtveNLpSrORPcpfSYRJ9oaW0ABOQD1oCQTqtmFebdQtxbDAWnvQAU6d1rObt0txfhZ5PmNVk3PDtHZkuMCrnW9DoCOn8ZJYIYIknZSTv5AIjRJl4MUq4TVuCD2MhJHU4InaL75abHrwLMeuu/mSunmt4opQXz4SQvSNy+geqGt+n04X27RiPU02RpRpu8OPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4y8bmvGWbSK87f4CPhigqrP+Vwk8f4NI1+JX7qr0avY=; b=I7r4es8iKLX/J0PNUgwZAwGf6ZbM04Ft/C7LbcZRBm4YsSIiat8SzQi8zeOxBKl/9beEpGjSJZ2V+3h3358jdYfuwxew2gVpazkA3dljDjiQ+Sk5iy4S7KOM3Wb2nyJ7t9H74jclSa0ZhhgR9lojO1rXKNZgTpB50x/QxywrruHBnvjvMzzfZbl/HJ+YRRtssJV33N5kiM/S+aMZ8hXb13yzKsvG9YEeA5kBvp+hB+0ifyqhs38nOEW6o+6Qn6kbLIQ2jWtMx7ur1Hom6jo9WxP4GIhPue24fzRL+rUoddeHwnbyImcyf2WpfbqUHpKvPsL5Xhxw3mePb3W8Y3OohA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 50.222.100.11) smtp.rcpttodomain=bsbernd.com smtp.mailfrom=ddn.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ddn.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddn.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4y8bmvGWbSK87f4CPhigqrP+Vwk8f4NI1+JX7qr0avY=; b=RQuHq97CWlzgwLlzB+YhkDGOaBvZXy8Tvpakv3LedJ9WNijNBOt07Pufw79lpn1UP2kJ3j3McVRo4HxWBJSlWi9lPu8ILUwJperWCKGJQgbe8BHSOe6sQ9vgb2zOgGbH+CTvArhaYqOXMA4ETFoZndusOO/XbC9NhZOIcRgm+I8= Received: from DM6PR02CA0092.namprd02.prod.outlook.com (2603:10b6:5:1f4::33) by SJ2PR19MB7487.namprd19.prod.outlook.com (2603:10b6:a03:4cc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8314.16; Tue, 7 Jan 2025 00:25:28 +0000 Received: from CY4PEPF0000EE31.namprd05.prod.outlook.com (2603:10b6:5:1f4:cafe::9) by DM6PR02CA0092.outlook.office365.com (2603:10b6:5:1f4::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8293.16 via Frontend Transport; Tue, 7 Jan 2025 00:25:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 50.222.100.11) smtp.mailfrom=ddn.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ddn.com; Received-SPF: Pass (protection.outlook.com: domain of ddn.com designates 50.222.100.11 as permitted sender) receiver=protection.outlook.com; client-ip=50.222.100.11; helo=uww-mrp-01.datadirectnet.com; pr=C Received: from uww-mrp-01.datadirectnet.com (50.222.100.11) by CY4PEPF0000EE31.mail.protection.outlook.com (10.167.242.37) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8335.7 via Frontend Transport; Tue, 7 Jan 2025 00:25:27 +0000 Received: from localhost (unknown [10.68.0.8]) by uww-mrp-01.datadirectnet.com (Postfix) with ESMTP id DDD0B1DB; Tue, 7 Jan 2025 00:25:26 +0000 (UTC) From: Bernd Schubert Date: Tue, 07 Jan 2025 01:25:21 +0100 Subject: [PATCH v9 16/17] fuse: block request allocation until io-uring init is complete Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250107-fuse-uring-for-6-10-rfc4-v9-16-9c786f9a7a9d@ddn.com> References: <20250107-fuse-uring-for-6-10-rfc4-v9-0-9c786f9a7a9d@ddn.com> In-Reply-To: <20250107-fuse-uring-for-6-10-rfc4-v9-0-9c786f9a7a9d@ddn.com> To: Miklos Szeredi Cc: Jens Axboe , Pavel Begunkov , linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, Joanne Koong , Josef Bacik , Amir Goldstein , Ming Lei , David Wei , bernd@bsbernd.com, Bernd Schubert X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1736209509; l=2651; i=bschubert@ddn.com; s=20240529; h=from:subject:message-id; bh=O/2YNgt+Vne/zwL4Xkn94aO/pImf6Sji5SRc7zTLOqs=; b=5tbtVDbVpbHE8beRDl0/o2O4rDaOEbLLZiawyLCobl4woFu1FFe610Gl0ZoV81BduxtkeTuMf TmDIa77Xm42DKTVc31vST8YPiFm0i0Nekv/yRLSa6k8F4TvfxSvZBYt X-Developer-Key: i=bschubert@ddn.com; a=ed25519; pk=EZVU4bq64+flgoWFCVQoj0URAs3Urjno+1fIq9ZJx8Y= X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE31:EE_|SJ2PR19MB7487:EE_ X-MS-Office365-Filtering-Correlation-Id: ee87f84e-d356-44d4-2234-08dd2eb1c914 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|7416014|376014|1800799024|7053199007; X-Microsoft-Antispam-Message-Info: =?utf-8?q?JmVNPlpTf0UNcrrZoihYulFeDqV4QeZ?= =?utf-8?q?B8OTXD5i0x5pzrurBH/mi9GFt6vZzcip7oAPG9nvAIrUP8xQZwDhMiMRXN0ZfOs85?= =?utf-8?q?CO3UI4+6v2ecuJNsWf+xpNg0eKAiInY3fk/uocTtpPj3lNhR81fOR/HvqFHnKz6B8?= =?utf-8?q?k2yQjUlpR47/2QuiqeSTmwR8IqLsfHjrcGj+GWYJcDtL1Vi7jR6xu6/45+YjAC7qz?= =?utf-8?q?j4MhnlaGKHVrwWQ0hjFA5IYYDCU6WUh3eO72ZN9SWoWv2KmNMhXy9s7+nXwGenT/E?= =?utf-8?q?3vY84J1jByyNED3ZJheakRX9aL68NbkT3kI14tcBhxYVHxqbdsAcsvErcqqy+jsLE?= =?utf-8?q?z10cF7PB/5r6aBBpotdJ6ZKi1vZXPjWSlEyfe4G0m5utW84jNKzOO5mOb7ujOT5yJ?= =?utf-8?q?CDmNyULqSCQPrE/bhFAMRZNVujSv0vi4oK+Vp1cdYFAEyKIrtnb5HbTRfZURo6k4s?= =?utf-8?q?clbR5qRh2FxZf8dC80v2bV550+Sh5CBq5PJxd/DD1s2msN8SH66eQivmxSQm1qTGQ?= =?utf-8?q?C8MG/FNBJaaGRyK00FlcTRkvo27t2/Jx6OCHstSS0x44evbfBhyBIFF82DZAqNsZI?= =?utf-8?q?8wNTF8/LxiIfQJSwweQh5/WowP+Tx7Pp1ZuXTzZ2tdWppiSFbYaruW/IQh7K0gKym?= =?utf-8?q?LrVKmlEdc2bpVZSAPHvfdfmEalkBrhzqYc59+aKoArW8GD5ddt6X74z3aPPLUn9XK?= =?utf-8?q?OXFAkFXyIo6oOknY0i86gU8pwl0ROKS0/n+8Icr+ABANnpBfhywm6/HVGguuDNsYa?= =?utf-8?q?29Yfjt19sJIYUbCk8oV0EIhqhRsv6diFMr1N+C8SXKxzWV7G5ibhQZuvE6/UWqZ+Q?= =?utf-8?q?XABiVK2lLJnqk8NzAYSCXNafk8VYnpYgqD/IgP+l/0Gu4qaikH2QMbKoKC1myaIw4?= =?utf-8?q?va/wYgx4iZ4IylN9k5CvMRyrvemKR1ALQcfhW4YWjrAz0wHqntyPtdg6ay3sLSeXn?= =?utf-8?q?wlZxMwljHWAHXvlNv4I06+ywHklzmtJTWvbFsWtvqo0q5eNkofycNEKR3Updz1h1v?= =?utf-8?q?wkaVeb+XLyNjgINekU9mgPcucS/0DzRAqvNzejFOOqTB4JexW/0bkko7rRJ+zTk9Z?= =?utf-8?q?kyK13MPW5Q05kgMlK1e8q9yyHlJLfyp7dqIA9YxMKQszNsBG+pBCQd+6MLGu4uhIY?= =?utf-8?q?nfUF3GH8yJrau+w9Bn5x4+pYh5mSmLlPnLM0tEO+EoHmIhcvmamMVaUR/CoDfBxrn?= =?utf-8?q?wwDwvKMH0Nhf2QoY3/ygrgWi7AI9IR5GwCUrTGtRxBX5LrRulS4pr8B8cjb5Sij05?= =?utf-8?q?MNJxQWQQOCxQsVn5BC0V8u//DuSLUyrzGmotM4BKM56Nk9+yLSFlsEBB/sfi8NwJ1?= =?utf-8?q?QidvIGIOKPksRXXPwN0MGty+F27F1BVd1z2xfDskr3C2sDnvMWrixCbtE6XsqoE9C?= =?utf-8?q?oOeL/oSPKt9hBgBUSpnYClJGTvAB9rh+tytcFZPi/vydcEZn0oMICU=3D?= X-Forefront-Antispam-Report: CIP:50.222.100.11;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:uww-mrp-01.datadirectnet.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(7416014)(376014)(1800799024)(7053199007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Use26U0+VBkRoAyBzAIH5ev4gn9sOK+DhkQfc/Y67goXkOKi8tDkoEvafxDWGtQWuoSJ+2yKRsaRHgKG2/oK4+uFdN9yi1LRDClzgEUYE70W836edwtqyw3WUeaD12ttwTNBbwrwPCge7oMcgAzZbymhyDFcrBi1oIbvR4+IqS1842JbjLLSg1afWXRlVC9AzD0xXolJV6SeTCvkbh3CVQ1NR4CLxCXItsUtNGpnT+tZsgNAt9QxahibcsTfuMJCs/PRhA78FYucVoffNBJx6bqBUDfgJZjWcpad3TcVpLNTMpmsEizYkqDtGSKXOkhL7iTQRhegO8CzU0R5Tpfrq/d1iCHsFqrvJXGV46OiGo6qeVNCH1FTazfjky2UGye9bPT3ED28EvTiCSXAjxf9YcWRaSh3wHDhMCQpvN0VmlXmxAT4A5JDMpVW+Q9eZH9UkeiDauAR4PEdOzUvkJuQ5rtSRXFPrAyn10DInF0HlPdpy2VGnxFozzIa79Eg/QLKCnH6v4sFxjWrk2kVkUYKP9yq+INcG08+Lcnu77D06HQu5NGi4olt2CsffcISNiIvZkConz+93p7/vNB7VofTl1VYw4d7zf54pTFSSmHv+t5+6JyHlFXh1r3kscATCVyZqhzhqC1J7cr1lU8fC3vA2Q== X-OriginatorOrg: ddn.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jan 2025 00:25:27.3169 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ee87f84e-d356-44d4-2234-08dd2eb1c914 X-MS-Exchange-CrossTenant-Id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=753b6e26-6fd3-43e6-8248-3f1735d59bb4;Ip=[50.222.100.11];Helo=[uww-mrp-01.datadirectnet.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE31.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR19MB7487 X-BESS-ID: 1736209534-111953-10732-17295-1 X-BESS-VER: 2019.3_20241219.1759 X-BESS-Apparent-Source-IP: 104.47.58.168 X-BESS-Parts: H4sIAAAAAAACA4uuVkqtKFGyUioBkjpK+cVKVkZmhmZAVgZQ0DLFONnMMsksLd E42dzSzNzYMDnJyMDM2BgklGiRolQbCwCCYJjpQQAAAA== X-BESS-Outbound-Spam-Score: 0.50 X-BESS-Outbound-Spam-Report: Code version 3.2, rules version 3.2.2.261635 [from cloudscan22-44.us-east-2b.ess.aws.cudaops.com] Rule breakdown below pts rule name description ---- ---------------------- -------------------------------- 0.50 BSF_RULE7568M META: Custom Rule 7568M 0.00 BSF_BESS_OUTBOUND META: BESS Outbound X-BESS-Outbound-Spam-Status: SCORE=0.50 using account:ESS124931 scores of KILL_LEVEL=7.0 tests=BSF_RULE7568M, BSF_BESS_OUTBOUND X-BESS-BRTS-Status: 1 From: Bernd Schubert Avoid races and block request allocation until io-uring queues are ready. This is a especially important for background requests, as bg request completion might cause lock order inversion of the typical queue->lock and then fc->bg_lock fuse_request_end spin_lock(&fc->bg_lock); flush_bg_queue fuse_send_one fuse_uring_queue_fuse_req spin_lock(&queue->lock); Signed-off-by: Bernd Schubert --- fs/fuse/dev.c | 3 ++- fs/fuse/dev_uring.c | 1 + fs/fuse/fuse_i.h | 3 +++ fs/fuse/inode.c | 2 ++ 4 files changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/fuse/dev.c b/fs/fuse/dev.c index 1b593b23f7b8c319ec38c7e726dabf516965500e..f002e8a096f97ba8b6e039309292942995c901c5 100644 --- a/fs/fuse/dev.c +++ b/fs/fuse/dev.c @@ -76,7 +76,8 @@ void fuse_set_initialized(struct fuse_conn *fc) static bool fuse_block_alloc(struct fuse_conn *fc, bool for_background) { - return !fc->initialized || (for_background && fc->blocked); + return !fc->initialized || (for_background && fc->blocked) || + (fc->io_uring && !fuse_uring_ready(fc)); } static void fuse_drop_waiting(struct fuse_conn *fc) diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c index cdd3917b365f4040c0f147648b09af9a41e2f49e..45815dee2d969650efe0df199cc3bd1f3e2e08f7 100644 --- a/fs/fuse/dev_uring.c +++ b/fs/fuse/dev_uring.c @@ -956,6 +956,7 @@ static void fuse_uring_do_register(struct fuse_ring_ent *ring_ent, if (ready) { WRITE_ONCE(ring->ready, true); fiq->ops = &fuse_io_uring_ops; + wake_up_all(&fc->blocked_waitq); } } } diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h index e71556894bc25808581424ec7bdd4afeebc81f15..886c3af2195892cb2ca0a171cd7b930b6e92484c 100644 --- a/fs/fuse/fuse_i.h +++ b/fs/fuse/fuse_i.h @@ -867,6 +867,9 @@ struct fuse_conn { /* Use pages instead of pointer for kernel I/O */ unsigned int use_pages_for_kvec_io:1; + /* Use io_uring for communication */ + unsigned int io_uring; + /** Maximum stack depth for passthrough backing files */ int max_stack_depth; diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c index 328797b9aac9a816a4ad2c69b6880dc6ef6222b0..e9db2cb8c150878634728685af0fa15e7ade628f 100644 --- a/fs/fuse/inode.c +++ b/fs/fuse/inode.c @@ -1390,6 +1390,8 @@ static void process_init_reply(struct fuse_mount *fm, struct fuse_args *args, else ok = false; } + if (flags & FUSE_OVER_IO_URING && fuse_uring_enabled()) + fc->io_uring = 1; } else { ra_pages = fc->max_read / PAGE_SIZE; fc->no_lock = 1;