[RFCv2,3/7] fs: fsnotify: replace memory barrier in __sb_end_write() with RCU
diff mbox

Message ID 20150625001606.E834E08B@viggo.jf.intel.com
State New
Headers show

Commit Message

Dave Hansen June 25, 2015, 12:16 a.m. UTC
From: Dave Hansen <dave.hansen@linux.intel.com>

If I sit in a loop and do write()s to small tmpfs files,
__sb_end_write() is third-hottest kernel function due to its
smp_mb().

__sb_end_write() uses the barrier to avoid races with freeze_super()
and its calls to sb_wait_write().  But, now that freeze_super() is
calling synchronize_rcu() before each sb_wait_write() call, we can
use that to our advantage.

The synchronize_rcu() ensures that all __sb_end_write() will see
freeze_super()'s updates to s_writers.counter.  That, in turn,
guarantees that __sb_end_write() will try to wake up any subsequent
call by freeze_super() to sb_wait_write().

Cc: Jan Kara <jack@suse.cz>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
---

 b/fs/super.c |   17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

Comments

Jan Kara June 26, 2015, 1:07 p.m. UTC | #1
On Wed 24-06-15 17:16:06, Dave Hansen wrote:
> 
> From: Dave Hansen <dave.hansen@linux.intel.com>

This has nothing to do with fsnotify so just remove it from the subject
line please. Thanks!

> If I sit in a loop and do write()s to small tmpfs files,
> __sb_end_write() is third-hottest kernel function due to its
> smp_mb().
> 
> __sb_end_write() uses the barrier to avoid races with freeze_super()
> and its calls to sb_wait_write().  But, now that freeze_super() is
> calling synchronize_rcu() before each sb_wait_write() call, we can
> use that to our advantage.
> 
> The synchronize_rcu() ensures that all __sb_end_write() will see
> freeze_super()'s updates to s_writers.counter.  That, in turn,
> guarantees that __sb_end_write() will try to wake up any subsequent
> call by freeze_super() to sb_wait_write().

What gains does this patch bring?

Otherwise the patch looks good to me. You can add:
Reviewed-by: Jan Kara <jack@suse.cz>

								Honza

> Cc: Jan Kara <jack@suse.cz>
> Cc: Alexander Viro <viro@zeniv.linux.org.uk>
> Cc: linux-fsdevel@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Tim Chen <tim.c.chen@linux.intel.com>
> Cc: Andi Kleen <ak@linux.intel.com>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> ---
> 
>  b/fs/super.c |   17 +++++++++++++----
>  1 file changed, 13 insertions(+), 4 deletions(-)
> 
> diff -puN fs/super.c~selectively-do-barriers-in-__sb_end_write fs/super.c
> --- a/fs/super.c~selectively-do-barriers-in-__sb_end_write	2015-06-24 17:14:35.315142611 -0700
> +++ b/fs/super.c	2015-06-24 17:14:35.318142745 -0700
> @@ -1146,14 +1146,23 @@ out:
>   */
>  void __sb_end_write(struct super_block *sb, int level)
>  {
> +	rcu_read_lock();
>  	percpu_counter_dec(&sb->s_writers.counter[level-1]);
>  	/*
> -	 * Make sure s_writers are updated before we wake up waiters in
> -	 * freeze_super().
> +	 * We are racing here with freeze_super()'s calls to
> +	 * sb_wait_write().  We want to ensure that we call
> +	 * wake_up() whenever one of those calls _might_ be
> +	 * in sb_wait_write().
> +	 *
> +	 * Since freeze_super() does a synchronize_rcu() before
> +	 * each of its sb_wait_write() calls, it can guarantee
> +	 * that it sees our update to s_writers.counter as well
> +	 * as that we see its update to s_writers.frozen.
>  	 */
> -	smp_mb();
> -	if (waitqueue_active(&sb->s_writers.wait))
> +	if (unlikely(sb->s_writers.frozen >= level))
>  		wake_up(&sb->s_writers.wait);
> +	rcu_read_unlock();
> +
>  	rwsem_release(&sb->s_writers.lock_map[level-1], 1, _RET_IP_);
>  }
>  EXPORT_SYMBOL(__sb_end_write);
> _

Patch
diff mbox

diff -puN fs/super.c~selectively-do-barriers-in-__sb_end_write fs/super.c
--- a/fs/super.c~selectively-do-barriers-in-__sb_end_write	2015-06-24 17:14:35.315142611 -0700
+++ b/fs/super.c	2015-06-24 17:14:35.318142745 -0700
@@ -1146,14 +1146,23 @@  out:
  */
 void __sb_end_write(struct super_block *sb, int level)
 {
+	rcu_read_lock();
 	percpu_counter_dec(&sb->s_writers.counter[level-1]);
 	/*
-	 * Make sure s_writers are updated before we wake up waiters in
-	 * freeze_super().
+	 * We are racing here with freeze_super()'s calls to
+	 * sb_wait_write().  We want to ensure that we call
+	 * wake_up() whenever one of those calls _might_ be
+	 * in sb_wait_write().
+	 *
+	 * Since freeze_super() does a synchronize_rcu() before
+	 * each of its sb_wait_write() calls, it can guarantee
+	 * that it sees our update to s_writers.counter as well
+	 * as that we see its update to s_writers.frozen.
 	 */
-	smp_mb();
-	if (waitqueue_active(&sb->s_writers.wait))
+	if (unlikely(sb->s_writers.frozen >= level))
 		wake_up(&sb->s_writers.wait);
+	rcu_read_unlock();
+
 	rwsem_release(&sb->s_writers.lock_map[level-1], 1, _RET_IP_);
 }
 EXPORT_SYMBOL(__sb_end_write);