From patchwork Wed Mar 5 12:36:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mateusz Guzik X-Patchwork-Id: 14002561 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BA5B24502C; Wed, 5 Mar 2025 12:36:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178217; cv=none; b=J4VeZ6cGyze/DrO+JQeSLVtlL63mMA/wsUKtQBxxAQR6StM5vcdeDFYsVX5YDM+AWcHrMkwq6WBeJFnSdBaFBVm58Px9eIm6ugS+q1sPFFFnFGgWsndRNyZ9qsT6tGPy0h1YntLaPgaj7LAPz3S7ku6rIJw2vlvFHnwkkPXb7cM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178217; c=relaxed/simple; bh=xEVDPl2kN377zrXJhWPCpz+2x0VgvInSaATuymWLGDs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AxRQT5eD2Y2f6qf5cwHQh74p5w/q+mvX0qjX6dGD9aaSRrFc/6mTrqKReXbcNc0cu4zBLAtU62JLaYNrzAcbo1qZ6F9B/W0Jn4YEWDGp0IbCUykVHxpz5IKnQ8Q/t1Eb3uJYJVu2d/s9PhMYN8tch3M6hkPcel1bVOmqRAna9OE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KETT/Vsb; arc=none smtp.client-ip=209.85.208.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KETT/Vsb" Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-5e033c2f106so7892829a12.3; Wed, 05 Mar 2025 04:36:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741178213; x=1741783013; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=anP8qXS9SQZse5TfMTpPl4lEfNtcPXm3bkmpJGZKrtM=; b=KETT/VsbLgQ0Kx/KAhdE0sldW6a7Vp4oqAWvFh23OQ4UHE6XYBGy7TIK23ElgR13w9 VSIvU3gkMZrvuBSO3IDOLB1TJwoKwD2mI9cj+Qijydn0ueEbWtAczYeGu+N1B5pMnGIH u8Y9T0BQqRPbNDxFqhLMFr/T0A8XLlGi0YKSSypS77QQoM/WbFMa0+TDLKKplCT4fLBJ 8sasHgqN2gjVX7CNzMPBnhlOdkMft7rJnVc4SevD7+BVRs0yaX4OKyp6vb5/2izsIQNr 5VRx1EDAg+JiHUAgu+8mj21EwqhcqEbgCEvTny8d5YRWfps2ggTq0j2KevtR62phdU08 9dEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741178213; x=1741783013; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=anP8qXS9SQZse5TfMTpPl4lEfNtcPXm3bkmpJGZKrtM=; b=eU3CGsW4ODSqLLLRql/V0q+jWU0DBKnzIZSTdPxXewlWiC+0MoweRYulLZ9LMMyrly H97ZBCGF6eQrQ0M4704SCYyD3W+kU4Az/+J/vUqrWLhUS/zWdhjD32Rbw8eZu2nYohJ4 qKy7SwBKch2sn/xi5aO0ShwApR3imPPe0e2OHxVlKM5Bb68OzpIyuUWndtuPg64IMlSc xDa+VPXjCm26MzDQLaRlmzqVRXhya8GeX6AmLDCclfvmv4U2dnCtI+ysxBXTjvOVhUe1 sLfhhJEh0NRTK4SUwp0jOKxtdQ61VKvBQFx/NlUzjTvdJME1kAK4inOLPypJENYYwREQ sc4A== X-Forwarded-Encrypted: i=1; AJvYcCXL1KtQgHRNHdVvbce/bkVnvq4MMfVXYYpTGT6g45WcSArmXbOgj6FlJMkdZwYkSVtoysNOok4GQOZuC/FI@vger.kernel.org, AJvYcCXmZJL324MxdROVmCh0pUcCcc0OcXnjDaC5VBmof20CkrkJTr4C3eGsDQPzHS95K/8vVbosqd78Y1Eet/32@vger.kernel.org X-Gm-Message-State: AOJu0Yx63kSNqw7jJLJgws0QpLGUUNBkU+SvFojl8w+47nfWAKEm5WoA k2p6FxJYUkE4+XcSLeqhlHSU8S6bFqzmJN4qHz/ReLV2VrAp90GV X-Gm-Gg: ASbGncur2Wf8y2snHzEElh4C/8zlqDTKsJj7qPz/+OwSnxSgHMhrnkidtpyGDfssGom iH+kAs8Eq8g97KVjZz3rwGPgYU+nzWJeNv8QdJiit+a0JMfUTyjkiLqAKbqZzV2T/9115T7Go9D TO0DkT2wFujwJbLR+sY2X2asBe6vhr+EDs3+CTRGNsvdX7Y1hK448BIQ/mnqemBzfqgdorAOn3H mECe6crgQrD8QcexULd1PQKqOiF3HuD2QMl99AkxXoVtbuGdu/9uWi65Eh/57kL9rQ0KugRDmmZ 5igv7PvrhG2d1E1enD0IvSzwUpPAB0brkt4QXNiKVvrDRnjME+AwCL+2w5pR X-Google-Smtp-Source: AGHT+IEGirpBQDNJgftKFXJOzOjnh5H2UdYautPHPUmHvM339MGGjNyo5C2DAG9e+hg/ATcvESX55w== X-Received: by 2002:a05:6402:5214:b0:5dc:5ada:e0c7 with SMTP id 4fb4d7f45d1cf-5e59f47cdd0mr7071820a12.26.1741178213240; Wed, 05 Mar 2025 04:36:53 -0800 (PST) Received: from f.. (cst-prg-71-44.cust.vodafone.cz. [46.135.71.44]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5e4c3b6cfc4sm9632068a12.18.2025.03.05.04.36.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Mar 2025 04:36:52 -0800 (PST) From: Mateusz Guzik To: brauner@kernel.org Cc: viro@zeniv.linux.org.uk, jack@suse.cz, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 1/4] file: add fput and file_ref_put routines optimized for use when closing a fd Date: Wed, 5 Mar 2025 13:36:41 +0100 Message-ID: <20250305123644.554845-2-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250305123644.554845-1-mjguzik@gmail.com> References: <20250305123644.554845-1-mjguzik@gmail.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Vast majority of the time closing a file descriptor also operates on the last reference, where a regular fput usage will result in 2 atomics. This can be changed to only suffer 1. See commentary above file_ref_put_close() for more information. Signed-off-by: Mateusz Guzik --- fs/file.c | 41 ++++++++++++----------- fs/file_table.c | 70 ++++++++++++++++++++++++++++------------ fs/internal.h | 3 ++ include/linux/file_ref.h | 34 +++++++++++++++++++ 4 files changed, 109 insertions(+), 39 deletions(-) diff --git a/fs/file.c b/fs/file.c index 44efdc8c1e27..6c159ede55f1 100644 --- a/fs/file.c +++ b/fs/file.c @@ -26,6 +26,28 @@ #include "internal.h" +bool __file_ref_put_badval(file_ref_t *ref, unsigned long cnt) +{ + /* + * If the reference count was already in the dead zone, then this + * put() operation is imbalanced. Warn, put the reference count back to + * DEAD and tell the caller to not deconstruct the object. + */ + if (WARN_ONCE(cnt >= FILE_REF_RELEASED, "imbalanced put on file reference count")) { + atomic_long_set(&ref->refcnt, FILE_REF_DEAD); + return false; + } + + /* + * This is a put() operation on a saturated refcount. Restore the + * mean saturation value and tell the caller to not deconstruct the + * object. + */ + if (cnt > FILE_REF_MAXREF) + atomic_long_set(&ref->refcnt, FILE_REF_SATURATED); + return false; +} + /** * __file_ref_put - Slowpath of file_ref_put() * @ref: Pointer to the reference count @@ -67,24 +89,7 @@ bool __file_ref_put(file_ref_t *ref, unsigned long cnt) return true; } - /* - * If the reference count was already in the dead zone, then this - * put() operation is imbalanced. Warn, put the reference count back to - * DEAD and tell the caller to not deconstruct the object. - */ - if (WARN_ONCE(cnt >= FILE_REF_RELEASED, "imbalanced put on file reference count")) { - atomic_long_set(&ref->refcnt, FILE_REF_DEAD); - return false; - } - - /* - * This is a put() operation on a saturated refcount. Restore the - * mean saturation value and tell the caller to not deconstruct the - * object. - */ - if (cnt > FILE_REF_MAXREF) - atomic_long_set(&ref->refcnt, FILE_REF_SATURATED); - return false; + return __file_ref_put_badval(ref, cnt); } EXPORT_SYMBOL_GPL(__file_ref_put); diff --git a/fs/file_table.c b/fs/file_table.c index 5c00dc38558d..5884cc659ea4 100644 --- a/fs/file_table.c +++ b/fs/file_table.c @@ -511,31 +511,37 @@ void flush_delayed_fput(void) } EXPORT_SYMBOL_GPL(flush_delayed_fput); -void fput(struct file *file) +static void __fput_deferred(struct file *file) { - if (file_ref_put(&file->f_ref)) { - struct task_struct *task = current; + struct task_struct *task = current; - if (unlikely(!(file->f_mode & (FMODE_BACKING | FMODE_OPENED)))) { - file_free(file); + if (unlikely(!(file->f_mode & (FMODE_BACKING | FMODE_OPENED)))) { + file_free(file); + return; + } + if (likely(!in_interrupt() && !(task->flags & PF_KTHREAD))) { + init_task_work(&file->f_task_work, ____fput); + if (!task_work_add(task, &file->f_task_work, TWA_RESUME)) return; - } - if (likely(!in_interrupt() && !(task->flags & PF_KTHREAD))) { - init_task_work(&file->f_task_work, ____fput); - if (!task_work_add(task, &file->f_task_work, TWA_RESUME)) - return; - /* - * After this task has run exit_task_work(), - * task_work_add() will fail. Fall through to delayed - * fput to avoid leaking *file. - */ - } - - if (llist_add(&file->f_llist, &delayed_fput_list)) - schedule_delayed_work(&delayed_fput_work, 1); + /* + * After this task has run exit_task_work(), + * task_work_add() will fail. Fall through to delayed + * fput to avoid leaking *file. + */ } + + if (llist_add(&file->f_llist, &delayed_fput_list)) + schedule_delayed_work(&delayed_fput_work, 1); } +void fput(struct file *file) +{ + if (unlikely(file_ref_put(&file->f_ref))) { + __fput_deferred(file); + } +} +EXPORT_SYMBOL(fput); + /* * synchronous analog of fput(); for kernel threads that might be needed * in some umount() (and thus can't use flush_delayed_fput() without @@ -549,10 +555,32 @@ void __fput_sync(struct file *file) if (file_ref_put(&file->f_ref)) __fput(file); } - -EXPORT_SYMBOL(fput); EXPORT_SYMBOL(__fput_sync); +/* + * Equivalent to __fput_sync(), but optimized for being called with the last + * reference. + * + * See file_ref_put_close() for details. + */ +void fput_close_sync(struct file *file) +{ + if (likely(file_ref_put_close(&file->f_ref))) + __fput(file); +} + +/* + * Equivalent to fput(), but optimized for being called with the last + * reference. + * + * See file_ref_put_close() for details. + */ +void fput_close(struct file *file) +{ + if (file_ref_put_close(&file->f_ref)) + __fput_deferred(file); +} + void __init files_init(void) { struct kmem_cache_args args = { diff --git a/fs/internal.h b/fs/internal.h index 3d05a989e4fa..3b3f315394ba 100644 --- a/fs/internal.h +++ b/fs/internal.h @@ -118,6 +118,9 @@ static inline void put_file_access(struct file *file) } } +void fput_close_sync(struct file *); +void fput_close(struct file *); + /* * super.c */ diff --git a/include/linux/file_ref.h b/include/linux/file_ref.h index 9b3a8d9b17ab..6ef92d765a66 100644 --- a/include/linux/file_ref.h +++ b/include/linux/file_ref.h @@ -61,6 +61,7 @@ static inline void file_ref_init(file_ref_t *ref, unsigned long cnt) atomic_long_set(&ref->refcnt, cnt - 1); } +bool __file_ref_put_badval(file_ref_t *ref, unsigned long cnt); bool __file_ref_put(file_ref_t *ref, unsigned long cnt); /** @@ -160,6 +161,39 @@ static __always_inline __must_check bool file_ref_put(file_ref_t *ref) return __file_ref_put(ref, cnt); } +/** + * file_ref_put_close - drop a reference expecting it would transition to FILE_REF_NOREF + * @ref: Pointer to the reference count + * + * Semantically it is equivalent to calling file_ref_put(), but it trades lower + * performance in face of other CPUs also modifying the refcount for higher + * performance when this happens to be the last reference. + * + * For the last reference file_ref_put() issues 2 atomics. One to drop the + * reference and another to transition it to FILE_REF_DEAD. This routine does + * the work in one step, but in order to do it has to pre-read the variable which + * decreases scalability. + * + * Use with close() et al, stick to file_ref_put() by default. + */ +static __always_inline __must_check bool file_ref_put_close(file_ref_t *ref) +{ + long old, new; + + old = atomic_long_read(&ref->refcnt); + do { + if (unlikely(old < 0)) + return __file_ref_put_badval(ref, old); + + if (old == FILE_REF_ONEREF) + new = FILE_REF_DEAD; + else + new = old - 1; + } while (!atomic_long_try_cmpxchg(&ref->refcnt, &old, new)); + + return new == FILE_REF_DEAD; +} + /** * file_ref_read - Read the number of file references * @ref: Pointer to the reference count From patchwork Wed Mar 5 12:36:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mateusz Guzik X-Patchwork-Id: 14002562 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F5502459C0; Wed, 5 Mar 2025 12:36:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178218; cv=none; b=RFytVDQsMK3QURWzjhuLU7l4bVMwe1DSrJRAkUjJAuR/afm2W+DwrWFkDLNs9AbjfHcX1AGEs662F6Y4AltkY4rSaggnn2xnjxKeIDW9WhDnvx4L7qtSXrn1n3ozjffS0qQz6CYnB7DFipF1vpc/9d2c5XqY9Kn3dK6Gqrc22WE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178218; c=relaxed/simple; bh=2ZK91i9XsyeViiZuUTFKex2pNwvDjGUQw0C7bW1KLVA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pF0sL0ILa6c8SCgzmqQ23LqolXJ6y/ZiuDbIJFje/0eDpfWsFKO30/zG+xD09CYxuHpn2C1888noVIWyEAYuAisLuR/4eZ6NnJQ+LG9uM1JsZyJL7Jdwpa6lS8mWLpxclrfJVido/243edpCqmZkwEQ9IPcnvkMOA9VxFa124pg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ByhDDAU0; arc=none smtp.client-ip=209.85.208.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ByhDDAU0" Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-5e535d16180so4942004a12.1; Wed, 05 Mar 2025 04:36:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741178215; x=1741783015; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1nxGNdfaQRrtpdKySkoLLtxRlIKFtjBBmUFqnTGFSs4=; b=ByhDDAU0E/3BRhUXLNbuLiD/NBqZvmYYZhschdCNzkAG/VxF8jMH9Qhg+zN/wQG0V4 Y6D+5wxec7NM/WPwdr8U5zJJolz9Q+I2Hx7fS76RiHtB416ddFji8RDW0IstnOqobWen 260qRkyX0XTn88YUaAfYs5E8X2gecccoiubCMtgmBAEOhL4z/SwFS/YYTM4yMMZa/gT/ teDlTO5j78CIH/94CeGjhAovhMAMeoTYTQ58fCUkRIDTKBCAxdHU/paOCphbB18gx4nf uF0G0v8MClTc3orF5s9VgWeYHAJj3PtJJIuC/GR7zy36VbtXDz/JGXB7rzQ8Osbni+25 y7xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741178215; x=1741783015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1nxGNdfaQRrtpdKySkoLLtxRlIKFtjBBmUFqnTGFSs4=; b=N+0lGVF5fca3Z1OVZ4b4BlFOo7fdI221T14Oc4jFiSAuIBgbRtJDrglKDXS8VjIco6 nrs5ZTzhi5HQUtPI/2ZfaUbOAL0gel9b6OUDmAajQhgXGcl9jNf168XCGqdjR1lB9bfM X7pKcCtbDIBfn84wopxua7WR/Iloks0XZd3LqOgn0/V/Z+1DPpjdmTDatBr3T629nmnt qgGkJ64pNLphy2MmNwZVJ5Ka92wqeFJfbv5o51WlBtqrjzMxruU1sNVwtRd9sV1h0lWV KVLxnfmnSdfxRqkrphtzdbKcM7ZxgQAxi7SkI5m2UC1TkbH82ZQVGPrb+K73YSFril49 9Q3g== X-Forwarded-Encrypted: i=1; AJvYcCUJNkv2UO0ZnZH8gZF9OR9bvbQsq9nw0x7Mq885BRQ/YCD6nOuPoi4lQVt3VEnAvzxjZC93XWnJT01BCbIv@vger.kernel.org, AJvYcCXvDEnfsX2tCq2+Wu9pXpwc7+V7pzCMRrXm6S9HasOTw+Bv5+lD0u/pZ4Yksmrh0STp/u2loPczQt1sIhJJ@vger.kernel.org X-Gm-Message-State: AOJu0YyT9Yt7pzzvjlHjoznk3LX4tO98IHe4vJKvdK9X1ATPuQZvcHEb ZMpNi0RAvDL4D4DKrhL7NTyaqzLHJLucu0sJYQnkAy9JTV040YzVuYJ/xQ== X-Gm-Gg: ASbGncseOM8w4OQrjYJp5QZ+48kKfqfFoaXBTXcPM1Bry4ZP0amxBhegV5dVyGGk4Y3 jtIFSHto9a9wBiok/mCeg2FSq3TWOD8EvOhGSt4sex0YayPu+/ufYM8rtG7cl6TMXLQkzc/WZa+ ZUQGKmwW06KryPWo9Nh+tB/icasE8uaGPu1Ke3xKLqtaXLfIgT7k7wZHWjqXe4IuXo3XhRuTtmI Ub0FRXlDM/9FdglQbnk1THyQAhIIelhwaBUv4a5oO9c4S7vB5Gq/tukWKqw6AICaz6cOE8FEa4U 2XbqjDnbrw74AUHGRLqw3tBp4p+NWBbPywpGCWgiue9meYyxnHk+DHoqQN4C X-Google-Smtp-Source: AGHT+IGvc1WN8cHlOuGOsa9/SGrkXBK3KqfDZQHhVsp4eqeXT+14uDIetFVJ0CuW0AOyU/rLRl1b5g== X-Received: by 2002:a05:6402:40d3:b0:5df:6a:54ea with SMTP id 4fb4d7f45d1cf-5e59f3d49bemr7455657a12.11.1741178215173; Wed, 05 Mar 2025 04:36:55 -0800 (PST) Received: from f.. (cst-prg-71-44.cust.vodafone.cz. [46.135.71.44]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5e4c3b6cfc4sm9632068a12.18.2025.03.05.04.36.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Mar 2025 04:36:54 -0800 (PST) From: Mateusz Guzik To: brauner@kernel.org Cc: viro@zeniv.linux.org.uk, jack@suse.cz, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 2/4] fs: use fput_close_sync() in close() Date: Wed, 5 Mar 2025 13:36:42 +0100 Message-ID: <20250305123644.554845-3-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250305123644.554845-1-mjguzik@gmail.com> References: <20250305123644.554845-1-mjguzik@gmail.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This bumps open+close rate by 1% on Sapphire Rapids by eliding one atomic. It would be higher if it was not for several other slowdowns of the same nature. Signed-off-by: Mateusz Guzik --- fs/open.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/open.c b/fs/open.c index a5def5611b2f..f2fcfaeb2232 100644 --- a/fs/open.c +++ b/fs/open.c @@ -1577,7 +1577,7 @@ SYSCALL_DEFINE1(close, unsigned int, fd) * We're returning to user space. Don't bother * with any delayed fput() cases. */ - __fput_sync(file); + fput_close_sync(file); if (likely(retval == 0)) return 0; From patchwork Wed Mar 5 12:36:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mateusz Guzik X-Patchwork-Id: 14002563 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 634C720551D; Wed, 5 Mar 2025 12:36:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178220; cv=none; b=R1a1xYTnychYfJxVIQUT5rlyVxC0rhVKpUglsT6Qi2ByUzysU6FE0VzdiCotDbxeuvQdmgJYCOckNT9y0cdkcAO4Zgm2A85P3ytRjt//IBV1/2umXzq3ScFi6zYEjX9q8ewocqm8vx3AcsXXepNPdTE8aQUfZpQlLx5pMOjCGpM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178220; c=relaxed/simple; bh=5uPV6u6EoDWFX9T8+OqpJcxy9wCcoz5cTg+5Qc1MY34=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lCtvrhicWqOFivpsutDwnr96WCI2d3uUkX3hY6AXNBYUqxn18rFNKq9J5drQJINK7P/w6BbB1UkIPzgOV41K6qf3tNmav1ahoCl0B8EuZS5Z4vOI1eqb8ED2AiwTNqCqM+Cx/BIsOmmozrOdA1IY1Svd0I4VOMaUh1kWBBqgEk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N7OG+B6O; arc=none smtp.client-ip=209.85.208.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N7OG+B6O" Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-5dccaaca646so1754845a12.0; Wed, 05 Mar 2025 04:36:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741178216; x=1741783016; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ihb5fa+ewtwZTTtF9ovoC44tIaJrnCvIe17/DExrK8c=; b=N7OG+B6OJpRcqVlFKCXvNKuQoPBzEV/I5ZjiU2k864sHF5ZfW5Hmm1b6HlsU3gUO04 xKFWtRRNDDt6T2i8wIeXoEPNXh7oWuqPtDIJ+4bVAWJLfodVbVaLxdqVzX/PJiMpV23B mAJYv1cfBtGaYG8JTIfkcyeqJBAGxkD8y5m4L9kzW1YB3Op7murvRinAGHbTQT7KC/I+ WsoNSS7hBCE84n4j5rC1G6FtUDsfnlXINJF8FcdvmN8fyB3j+YV4VWxJM4MyXLyq6ViH dGQ/ln/0QumINA+0hjGea7qf4wnO2EcMx5G3GKuhHRP7jfT7qa/xGfWoz9vWdubSiuAL rMJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741178216; x=1741783016; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ihb5fa+ewtwZTTtF9ovoC44tIaJrnCvIe17/DExrK8c=; b=j4e7EbFzx8oYpNZNyARlK6RbNIaz6XQalsePkGU6j/YoxcsNK9NtmnGlRsdCcgy77H 3xr6CCKx5qMytm8DIRTxHSXig7tmKfV817VshLkHHKUxIf5BWB8NPbG95ur5r6d6grtO 5anEULziS6XbNqnbI81T6/3gK+weY2UdXGT3r4fWzUCHsUkNk3kIubMXvjVT4IVuCWf7 ixXHmGmaYlH3OLsRJTYZZanUj0MgR/zdjRE0+Y/3zkHAl3PN/Lsp6gphnPfV41nJFznY NzsFfk5Aa+0q9sYjaMqUtqoOOjkC9kkb6x9izbEKWggsY1QI40FCtnb4tyNPGj5p6dgG rC2Q== X-Forwarded-Encrypted: i=1; AJvYcCU7XtKgqzsZPxg4Kp3yJ62VolhUJjzXfkXO9t84VLdD/pn8t8euGFCDrmvzDZImNg+2CLAa3rbIK/XYh7v+@vger.kernel.org, AJvYcCXNCb414S6tHjrrXrRVzLiaQ1k+KjebzTc3I6PTM/LPgpMhEVv81pFqJUouHjrBPu+LT3GId7PWzx9CEXAu@vger.kernel.org X-Gm-Message-State: AOJu0YwdibOIdjaxrf6TEx/JsDinZVStPYBMyP+OyHbF1BJVHpjwOm10 dRceLSnbCSCUIMAxnkKPbEGQTt7uCTufzEV+l3J4AUcoC/Chtc5+ X-Gm-Gg: ASbGncumyp39lGbvmd+2F5Muc5+FUS3By3fhxkkYwZqtDJu879gHQCBa1f4CR0gWhJe /Nh/xO0hXSGj6H/ydx8RXOsm8yCAv11/OHgOMlclq+kJzzqso48CKQik4zcjT8OIM2KhjkllaDe 1WVnUP6Lpi4zBJ+7OgHonAl0MoRC8M4RlwlhG4B8Ub5BHCKy02OKzTfJ7hb8Z6oCQNCPFUkhvMS CKhGwexkVLo81cNqBtOZKzhC5JsAVuwY1LQ3av32wWuU6Yz0rUO8ZYAXp2cquQrfuT818Flu/F/ /qUTBToibwObbI7pVWakmvSofx716Pswy6Z0mM732PQOuemRRr/FiNSinwmt X-Google-Smtp-Source: AGHT+IGa4M++3EToLwZVWWaLP7erRKKGe3l3GeVT3S3aocMyh/TKoZrE6dig4uXC/MDemY1hzVtQ2A== X-Received: by 2002:a05:6402:1cc1:b0:5e4:95fc:d748 with SMTP id 4fb4d7f45d1cf-5e584e2916bmr6730421a12.5.1741178216292; Wed, 05 Mar 2025 04:36:56 -0800 (PST) Received: from f.. (cst-prg-71-44.cust.vodafone.cz. [46.135.71.44]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5e4c3b6cfc4sm9632068a12.18.2025.03.05.04.36.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Mar 2025 04:36:55 -0800 (PST) From: Mateusz Guzik To: brauner@kernel.org Cc: viro@zeniv.linux.org.uk, jack@suse.cz, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 3/4] fs: use fput_close() in filp_close() Date: Wed, 5 Mar 2025 13:36:43 +0100 Message-ID: <20250305123644.554845-4-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250305123644.554845-1-mjguzik@gmail.com> References: <20250305123644.554845-1-mjguzik@gmail.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When tracing a kernel build over refcounts seen this is a wash: @[kprobe:filp_close]: [0] 32195 |@@@@@@@@@@ | [1] 164567 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| I verified vast majority of the skew comes from do_close_on_exec() which could be changed to use a different variant instead. Even without changing that, the 19.5% of calls which got here still can save the extra atomic. Calls here are borderline non-existent compared to fput (over 3.2 mln!), so they should not negatively affect scalability. Signed-off-by: Mateusz Guzik --- fs/open.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/open.c b/fs/open.c index f2fcfaeb2232..bdbf03f799a1 100644 --- a/fs/open.c +++ b/fs/open.c @@ -1551,7 +1551,7 @@ int filp_close(struct file *filp, fl_owner_t id) int retval; retval = filp_flush(filp, id); - fput(filp); + fput_close(filp); return retval; } From patchwork Wed Mar 5 12:36:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mateusz Guzik X-Patchwork-Id: 14002564 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DE7924887A; Wed, 5 Mar 2025 12:37:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178222; cv=none; b=rnf9egDMzqtSpvk5s3HoNlOtJrLj1GzLBQQ53l8rMfPQXkmqcHPfb1io+0SnKLAvnZcvAfnXCh9FcOdzYt7WBUHvPuiCba2i3VU8/fTLKEaRghKPoAJ1A4GkEygKSNljEQDsqxSYrulmCdKDW8vUC7e+X8OgfxDdE+RSbDYKtzo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741178222; c=relaxed/simple; bh=lFarR6NBkdgP4UhOE5fr4mefNelcf7WeVn1JvhvnjeI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JfL+qTJgAVLFWuj507kESOxmif4+F3vupg/aPp8AljQ2vHkYV1CNRBrhJ+68UEI/XErawFb//61qRqwSOCq8+W6bwnM8JyvOt5Y8LdVc9BOHUdFtxOaH+UPmrx4jfm8BzMwzp9YsWiStHUAODJdkQWwSVna9qycN5lC1Rx0KMUg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Rfux4Ti8; arc=none smtp.client-ip=209.85.208.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Rfux4Ti8" Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-5e0b70fb1daso11778994a12.1; Wed, 05 Mar 2025 04:37:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741178218; x=1741783018; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ryYM4FwuDMV/z3F/EjxddAUgiRg0W1++yjtCL16Ctf0=; b=Rfux4Ti8NQRFJTa+hlrURLlsNDXuHYXmPadwl6k9uZ38WMtdpyhXSfqqlXz18wnlL/ ovXbpjeg8zA4IuY/yN6ENZufb6L8lL6QOPueEQzcGzbHZuuYp+dVq8VwYq/CndUjIn1n sMrGOE6wT3Y3lyiGXkbohVDyev4R4+H1I/0zaEH1AHCgdajHy/SkOvdvbtfaN5Trt2Rb YcrJQ4Z/KgsC/akvzsVhmZHxubMzyr5+Q42T1dmjXmCbmZJfyfPO34Ia41A7ESW20FDA qxhsqLka0zbIFUKC0Fpn2OtwXDd93TGmeEYF+dI8fUemATPvCqapE2kYeJPqUHVSM5Ry lQqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741178218; x=1741783018; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ryYM4FwuDMV/z3F/EjxddAUgiRg0W1++yjtCL16Ctf0=; b=Cfr4o2V3RYn8NfrBcN5Hsn4mpczziX8Q/izVW2K3Hcv9gHF03UUzaNYkrtfuyOOAam goHg2D2O70MhrAu+BYjcVzd7AKAJl5KlN09XO0s5wL9LvhccBSJlJn4ycPrHCX0JsKKg 8pbfA2IXAyb9hs2tBmIKMtQ9Jwnyr/7IYfWE0KrThpSx7VBzBF4uTOXFGDxuANAz96R4 nQJeLQrbIUxmValc+EKL6UNyMhflJhGOUSAArhvPBemjX3R51Gc2FjV0MMRwTQQHoAs3 18Csj4bn5z7xiCpvGSGGBLt+i3AUvRSJqUtEj61EsqvBt1l5p+vkkNo/yTKwU+5K+yVy lCVw== X-Forwarded-Encrypted: i=1; AJvYcCUdI/gTWtjkm+HWiQGFVfuD6zwiecdzDEXn2jzJp8w2oRkgAHxLpbBH72GxSGpYSLEMMM+El+2L0UZJXGZb@vger.kernel.org, AJvYcCUj4Ja8uBj9LG5e6wUOOLMJE2weAiMMm5W4alk0cMGOdr0h3ejJSytyeM4RrKjjeI3pr0uXSdL7c5hO52a5@vger.kernel.org X-Gm-Message-State: AOJu0Yyda39p85iv40EfHYM3WpqlAt8lBVP0gueKDSmyjs8eyKVIP9D/ SIeUahU60kVD+uFx84+B+1D5nEEDaHCCrGXCwEr5LAl0cMct/Qv+ X-Gm-Gg: ASbGncvYr2L9m7o5oZDszQlfg4Nw5OMYVsFAm9wobM6z2RJeetdTw/wn89XvmnH4IEx ZBupIumGencb7mE43NMo2n21qIBdGoUe+gF7pxrCttFUGz28mr40vltLjRjuJ+XNNX/om3UJ3IT 7VcBtVvcOXBZgBkSKmtz42fOBBWbJmH7QubNVVB7xaiNKfHN/nJoQU6sdB9u4O0/SjNeBrdjV5A pqDHpDpUd8s/oovECMCdXxdVpPNQ+OavLLENXMtYGSRS4WcVmhZ1HxxmkA0oSdel5BnkvMRXXXJ Epa1E/PcLix1n2vTxUFBvnyTsVaEIsnJhhEcCSfFUEnMA0goW8QXKNFwloo0 X-Google-Smtp-Source: AGHT+IEUhC913u3GMAkY0GYpmBkUBZ+LbShxj3ZYBqm4cB77P0kVFYlIYrinVaEVCFO1M8qLbtALaQ== X-Received: by 2002:a05:6402:13d3:b0:5dc:81b3:5e1a with SMTP id 4fb4d7f45d1cf-5e59f3864ebmr2559563a12.7.1741178218391; Wed, 05 Mar 2025 04:36:58 -0800 (PST) Received: from f.. (cst-prg-71-44.cust.vodafone.cz. [46.135.71.44]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5e4c3b6cfc4sm9632068a12.18.2025.03.05.04.36.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Mar 2025 04:36:57 -0800 (PST) From: Mateusz Guzik To: brauner@kernel.org Cc: viro@zeniv.linux.org.uk, jack@suse.cz, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Mateusz Guzik Subject: [PATCH v3 4/4] fs: use fput_close() in path_openat() Date: Wed, 5 Mar 2025 13:36:44 +0100 Message-ID: <20250305123644.554845-5-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250305123644.554845-1-mjguzik@gmail.com> References: <20250305123644.554845-1-mjguzik@gmail.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This bumps failing open rate by 1.7% on Sapphire Rapids by avoiding one atomic. Signed-off-by: Mateusz Guzik --- fs/namei.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/namei.c b/fs/namei.c index d00443e38d3a..06765d320e7e 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -4005,7 +4005,7 @@ static struct file *path_openat(struct nameidata *nd, WARN_ON(1); error = -EINVAL; } - fput(file); + fput_close(file); if (error == -EOPENSTALE) { if (flags & LOOKUP_RCU) error = -ECHILD;