From patchwork Thu Oct 24 14:05:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13849106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7F3DCE8E7B for ; Thu, 24 Oct 2024 14:07:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4772F6B00B7; Thu, 24 Oct 2024 10:07:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4015A6B00B8; Thu, 24 Oct 2024 10:07:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 282F76B00B9; Thu, 24 Oct 2024 10:07:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 049086B00B7 for ; Thu, 24 Oct 2024 10:07:38 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 79FEF1A0A1D for ; Thu, 24 Oct 2024 14:07:05 +0000 (UTC) X-FDA: 82708673376.15.996ADD6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf14.hostedemail.com (Postfix) with ESMTP id A275A100016 for ; Thu, 24 Oct 2024 14:07:15 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=S3SQ5SLL; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729778731; a=rsa-sha256; cv=none; b=K5UnmenyF8pxXvs+FyDqZ+Jtxu5Il/IN4faBBA85N9+XMoYn40RtU1dwTF5H9msTEOMV8v hkqKVGXvXYBwIkgb+GU7uNJvrwuVsB9j+i0jyZXGm8KClSWo6IFFNlhpat3Y7EsBe7jGQG IJvPi5cfUvfIBJJ6zf1dP9Xzi9v3G9E= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=S3SQ5SLL; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729778731; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RVoEs9N2N3+Yy5ytz8xwWGrwrjCUqjBQVK6VxNHCBSE=; b=acr6suJWcDzD7tGHgjYAkKnQ3bo1/L6fDErddB703mQ3HnNMQA9i7roPfL0BKel1y7rbva v1yA5jspQxh0Vfznkk8RFLIylKvIDQpadBWsBxjd0+MuGS9dfm9pdL4N5x0h+paEofRcwk U+0VBQGql/gACNXVGKDwspeSLMZu7bU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1729778855; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RVoEs9N2N3+Yy5ytz8xwWGrwrjCUqjBQVK6VxNHCBSE=; b=S3SQ5SLLZKQSVuAQLxQ1KvX6a5pbgWPustccSESVEwnOBqMGdMouBpQVajwNdZgldq4GiB r5vgBvO95yfysuRHpQdcBauyUt31BiDxNEk7jt3/a+b2W1sL1s+3zFJeSdo9I4bbTyVt6G lgAa2ZFGneklNZw3QMC58DdhqPt2pqw= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-631-iZ9hvXLgMfGRtcrtNhal7w-1; Thu, 24 Oct 2024 10:07:30 -0400 X-MC-Unique: iZ9hvXLgMfGRtcrtNhal7w-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 75CC11944F07; Thu, 24 Oct 2024 14:07:27 +0000 (UTC) Received: from warthog.procyon.org.uk.com (unknown [10.42.28.231]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B259D196BB7D; Thu, 24 Oct 2024 14:07:21 +0000 (UTC) From: David Howells To: Christian Brauner , Steve French , Matthew Wilcox Cc: David Howells , Jeff Layton , Gao Xiang , Dominique Martinet , Marc Dionne , Paulo Alcantara , Shyam Prasad N , Tom Talpey , Eric Van Hensbergen , Ilya Dryomov , netfs@lists.linux.dev, linux-afs@lists.infradead.org, linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs@lists.linux.dev, linux-erofs@lists.ozlabs.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/27] afs: Don't use mutex for I/O operation lock Date: Thu, 24 Oct 2024 15:05:10 +0100 Message-ID: <20241024140539.3828093-13-dhowells@redhat.com> In-Reply-To: <20241024140539.3828093-1-dhowells@redhat.com> References: <20241024140539.3828093-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A275A100016 X-Stat-Signature: h5mt7to1w8b64785sosrg77ikk5sxoco X-Rspam-User: X-HE-Tag: 1729778835-168658 X-HE-Meta: U2FsdGVkX1/5aINKvp1dBYpmeEBN60Vx8EBKQtB/lf8CgkWPL7raA4aZqu9mrn2Z131BC7lkRD6EIbJupUZLEz8/u3Q6PzdFL90Sm+W+34B0EAZRPT6Oo2iR0/Wa8dLfrT/ypAuG0+8bCcedwbK4kf7efXyG6/aCHD3IKpeMzR3CHDnVSR+SIGCGa43QbyFBszGFDohNltfcYdQwgVRsRd7X7PfmlMHCNAbLwWf4qmr4BxNlgsiaowBgvexp0DeqwKDZq2eCUR+MICM2ceUuqBVYsMULfer7DZWfdiAy8NZ+ODD+TrjzuOCLxdWTSjIgQOfquIczkI8D8NqFtIagoPO2wTrd/gTN5luqDQB2XU5iZk/j50D1CXOGnRTAkaYxOQcLTqn4QOSRnhflQy29Iy201uUzFzuqxkeMTsL/p1343QBZQJr59fGY/0d+dbCP119Jv8TWUqafPkbkjQkONAt/JhPMMrDGyUhygcU4jOkcvBk+SLAC4riUgNZG135yuDlNba5C9GT5bWjUS/tTS83X3CE7tsjo+rjCk7vXskdWpjsh8AgRE4mh6TnhkmvtcwErsXw7TQwIwAZE6dhbLR8hp1UP0MFBzxKF3UQdJ2BZqFUN9ZQmUe8KV0WNo83LoDAdz9gohWOCetd79hn5Oc1arBt+nD2qa4dT4KOXFFWQSq29UfhOxRCHnrb/AJ3IYtZaH/zkH11Qbem0+Erl6mmZSUKdFxXSA2kSc6/PadBEv1NDFimZcFILMe4BW4qZcN+nILtJyr7OOcMZpx9qXLPE8AYCCPdlSuXlak7YQ7LWr5zJYG2wfjY6pC943ETS9WOvclzbhIrmDT4Gh5Pk9MSZYWytm1VgHsxaMLyfgc51Rf1G0xQpK5UtFKVl3CZiHR852l4wO73vDOE8vHQAozooocnmlMRfwz6i0b7coDvzq3BBOZgdcKXmxRwgGk/uw9XuQO86r5PMZrYHnYK OFXXJ5ye dDYqPmRgb6k5+uibBznZ7wrTxuxiCa4kT3nR1V+oY765R6MAHic6nJpVDx81px9JDa7q1ULdq+RFcnPz1+Pfi7B9U4dXRdCohtTh9UJIuod/T7SZwFtiIZ/Ki8+1EwuEL5oEWI5f/cg8VdlUAH2c0Ujy8gBo9u7iC5Ke/ugm2f32QgfbsO2yn5N9Ok+mlacZmuf/kKz+HPMdn/UBAl58zRF3LpL9fF8HhHZ8+iAAmN2YgsB9BVtzonv2Dw0zQqKZ+QlmX56BcGCZQxVBXg+zak/o81y0VkfIJWh9XxRV94mKPZ+2tTO4XDmYTOc3z4bU+W4ks09eW8biVD4yFXxSHrK7B0KBzr/T0P5mfFTGMP1WWYisEiFf9oyZiI5GbXaBnX1eQNR5IdR5B25nUCr0pS+C3D85+0+z2wJi9wABDzreLun9gR2bQRTpeaacFhTgyBFkv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Don't use the standard mutex for the I/O operation lock, but rather implement our own as the standard mutex must be released in the same thread as locked it. This is a problem when it comes to doing async FetchData where the lock will be dropped from the workqueue that processed the incoming data and not from the issuing thread. Signed-off-by: David Howells cc: Marc Dionne cc: linux-afs@lists.infradead.org --- fs/afs/fs_operation.c | 111 +++++++++++++++++++++++++++++++++++++++--- fs/afs/internal.h | 3 +- fs/afs/super.c | 2 +- 3 files changed, 108 insertions(+), 8 deletions(-) diff --git a/fs/afs/fs_operation.c b/fs/afs/fs_operation.c index 428721bbe4f6..8488ff8183fa 100644 --- a/fs/afs/fs_operation.c +++ b/fs/afs/fs_operation.c @@ -49,6 +49,105 @@ struct afs_operation *afs_alloc_operation(struct key *key, struct afs_volume *vo return op; } +struct afs_io_locker { + struct list_head link; + struct task_struct *task; + unsigned long have_lock; +}; + +/* + * Unlock the I/O lock on a vnode. + */ +static void afs_unlock_for_io(struct afs_vnode *vnode) +{ + struct afs_io_locker *locker; + + spin_lock(&vnode->lock); + locker = list_first_entry_or_null(&vnode->io_lock_waiters, + struct afs_io_locker, link); + if (locker) { + list_del(&locker->link); + smp_store_release(&locker->have_lock, 1); + smp_mb__after_atomic(); /* Store have_lock before task state */ + wake_up_process(locker->task); + } else { + clear_bit(AFS_VNODE_IO_LOCK, &vnode->flags); + } + spin_unlock(&vnode->lock); +} + +/* + * Lock the I/O lock on a vnode uninterruptibly. We can't use an ordinary + * mutex as lockdep will complain if we unlock it in the wrong thread. + */ +static void afs_lock_for_io(struct afs_vnode *vnode) +{ + struct afs_io_locker myself = { .task = current, }; + + spin_lock(&vnode->lock); + + if (!test_and_set_bit(AFS_VNODE_IO_LOCK, &vnode->flags)) { + spin_unlock(&vnode->lock); + return; + } + + list_add_tail(&myself.link, &vnode->io_lock_waiters); + spin_unlock(&vnode->lock); + + for (;;) { + set_current_state(TASK_UNINTERRUPTIBLE); + if (smp_load_acquire(&myself.have_lock)) + break; + schedule(); + } + __set_current_state(TASK_RUNNING); +} + +/* + * Lock the I/O lock on a vnode interruptibly. We can't use an ordinary mutex + * as lockdep will complain if we unlock it in the wrong thread. + */ +static int afs_lock_for_io_interruptible(struct afs_vnode *vnode) +{ + struct afs_io_locker myself = { .task = current, }; + int ret = 0; + + spin_lock(&vnode->lock); + + if (!test_and_set_bit(AFS_VNODE_IO_LOCK, &vnode->flags)) { + spin_unlock(&vnode->lock); + return 0; + } + + list_add_tail(&myself.link, &vnode->io_lock_waiters); + spin_unlock(&vnode->lock); + + for (;;) { + set_current_state(TASK_INTERRUPTIBLE); + if (smp_load_acquire(&myself.have_lock) || + signal_pending(current)) + break; + schedule(); + } + __set_current_state(TASK_RUNNING); + + /* If we got a signal, try to transfer the lock onto the next + * waiter. + */ + if (unlikely(signal_pending(current))) { + spin_lock(&vnode->lock); + if (myself.have_lock) { + spin_unlock(&vnode->lock); + afs_unlock_for_io(vnode); + } else { + list_del(&myself.link); + spin_unlock(&vnode->lock); + } + ret = -ERESTARTSYS; + } + return ret; +} + /* * Lock the vnode(s) being operated upon. */ @@ -60,7 +159,7 @@ static bool afs_get_io_locks(struct afs_operation *op) _enter(""); if (op->flags & AFS_OPERATION_UNINTR) { - mutex_lock(&vnode->io_lock); + afs_lock_for_io(vnode); op->flags |= AFS_OPERATION_LOCK_0; _leave(" = t [1]"); return true; @@ -72,7 +171,7 @@ static bool afs_get_io_locks(struct afs_operation *op) if (vnode2 > vnode) swap(vnode, vnode2); - if (mutex_lock_interruptible(&vnode->io_lock) < 0) { + if (afs_lock_for_io_interruptible(vnode) < 0) { afs_op_set_error(op, -ERESTARTSYS); op->flags |= AFS_OPERATION_STOP; _leave(" = f [I 0]"); @@ -81,10 +180,10 @@ static bool afs_get_io_locks(struct afs_operation *op) op->flags |= AFS_OPERATION_LOCK_0; if (vnode2) { - if (mutex_lock_interruptible_nested(&vnode2->io_lock, 1) < 0) { + if (afs_lock_for_io_interruptible(vnode2) < 0) { afs_op_set_error(op, -ERESTARTSYS); op->flags |= AFS_OPERATION_STOP; - mutex_unlock(&vnode->io_lock); + afs_unlock_for_io(vnode); op->flags &= ~AFS_OPERATION_LOCK_0; _leave(" = f [I 1]"); return false; @@ -104,9 +203,9 @@ static void afs_drop_io_locks(struct afs_operation *op) _enter(""); if (op->flags & AFS_OPERATION_LOCK_1) - mutex_unlock(&vnode2->io_lock); + afs_unlock_for_io(vnode2); if (op->flags & AFS_OPERATION_LOCK_0) - mutex_unlock(&vnode->io_lock); + afs_unlock_for_io(vnode); } static void afs_prepare_vnode(struct afs_operation *op, struct afs_vnode_param *vp, diff --git a/fs/afs/internal.h b/fs/afs/internal.h index c9d620175e80..07b8f7083e73 100644 --- a/fs/afs/internal.h +++ b/fs/afs/internal.h @@ -702,13 +702,14 @@ struct afs_vnode { struct afs_file_status status; /* AFS status info for this file */ afs_dataversion_t invalid_before; /* Child dentries are invalid before this */ struct afs_permits __rcu *permit_cache; /* cache of permits so far obtained */ - struct mutex io_lock; /* Lock for serialising I/O on this mutex */ + struct list_head io_lock_waiters; /* Threads waiting for the I/O lock */ struct rw_semaphore validate_lock; /* lock for validating this vnode */ struct rw_semaphore rmdir_lock; /* Lock for rmdir vs sillyrename */ struct key *silly_key; /* Silly rename key */ spinlock_t wb_lock; /* lock for wb_keys */ spinlock_t lock; /* waitqueue/flags lock */ unsigned long flags; +#define AFS_VNODE_IO_LOCK 0 /* Set if the I/O serialisation lock is held */ #define AFS_VNODE_UNSET 1 /* set if vnode attributes not yet set */ #define AFS_VNODE_DIR_VALID 2 /* Set if dir contents are valid */ #define AFS_VNODE_ZAP_DATA 3 /* set if vnode's data should be invalidated */ diff --git a/fs/afs/super.c b/fs/afs/super.c index f3ba1c3e72f5..7631302c1984 100644 --- a/fs/afs/super.c +++ b/fs/afs/super.c @@ -663,7 +663,7 @@ static void afs_i_init_once(void *_vnode) memset(vnode, 0, sizeof(*vnode)); inode_init_once(&vnode->netfs.inode); - mutex_init(&vnode->io_lock); + INIT_LIST_HEAD(&vnode->io_lock_waiters); init_rwsem(&vnode->validate_lock); spin_lock_init(&vnode->wb_lock); spin_lock_init(&vnode->lock);