From patchwork Mon Jan 4 18:21:06 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 7949931 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 257F2BEEE5 for ; Mon, 4 Jan 2016 18:22:00 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 073CE202BE for ; Mon, 4 Jan 2016 18:21:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 20AED202DD for ; Mon, 4 Jan 2016 18:21:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752148AbcADSVx (ORCPT ); Mon, 4 Jan 2016 13:21:53 -0500 Received: from mail-ig0-f177.google.com ([209.85.213.177]:38798 "EHLO mail-ig0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752222AbcADSVv (ORCPT ); Mon, 4 Jan 2016 13:21:51 -0500 Received: by mail-ig0-f177.google.com with SMTP id mw1so109436337igb.1 for ; Mon, 04 Jan 2016 10:21:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=primarydata-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=s+1lgNJ2kIiTNg3GNigPd3Yt5GUt7fliVVRW00To/w8=; b=J8KjXHGdlGuN5PIw5D9rqE1BSEvheR3Ihs9i7GZVMbtyG4KPvCQ7fG2uyCvshB5NeG Q23IBQ7/RpYHPWhPBVgPFX8SvYquDQ/l79DzAdOQ7D4ftCPNjFRiJ5y7CYv3iyVjuWZH 8wwv/lF2Py/Bcq9TxAucAtZscw06Xobb1cdaQ/3BqFJ8QaGfVZBT0IVEyDqGc1mV6m7W M4jAwT20pzvQu/nhrPzTOMtV7TaK5YBbf9lGcl7NU2oHmM9k7aIFSSU8nYarAQn3g0hX t27FWDkoHL7L37jmmVqVUIP6+8F6zyo4pNvosCxhahNGf5T2wOkz4J7dp6cAMYKq6QFr Om/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=s+1lgNJ2kIiTNg3GNigPd3Yt5GUt7fliVVRW00To/w8=; b=kaZJ1+JMPReXFoxTkN4BQ5mM5/QTCtIz+Xd0WJmrPWC7362zYB8UGs8VEPEopettHY 6DTBdQC42gPQMbm8rEC7NZnzSOvdAmGHW4mNvLP9lY3x/SQpad7rDiAParK8giODqBB2 F24+zPzcnqr7KH8x+jWkwNLB3AsZEa2/mnrAbBCqcffpa2jdDnovHXyFMPDmS31YIof6 ZJOZjXvpQ04pva3jgV06usNE/KzAJQO/mWqfzM779bHjFYkJSyEp/lsJaBJ52nisZSpS fOorgUWAlUL3+t0UdRiDqeKubIQqaEyBKCtVEErPY4HPnF6j5u0XScSUNuaedNa5GPVQ YVQQ== X-Gm-Message-State: ALoCoQnQ/hpNDjhpJl/hsSUIsjOe0SKeEIsb6xeVtJyJqcbVksZZsj+MtYsbhFmDxsKi6QP2Lt+csBa1dnpv4XEvswiSa/hyfQ== X-Received: by 10.50.18.4 with SMTP id s4mr37888631igd.21.1451931710996; Mon, 04 Jan 2016 10:21:50 -0800 (PST) Received: from leira.trondhjem.org.localdomain (c-68-49-164-203.hsd1.mi.comcast.net. [68.49.164.203]) by smtp.gmail.com with ESMTPSA id n6sm12760568ige.12.2016.01.04.10.21.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Jan 2016 10:21:50 -0800 (PST) From: Trond Myklebust To: linux-nfs@vger.kernel.org Subject: [PATCH v3 08/18] NFS: Allow multiple commit requests in flight per file Date: Mon, 4 Jan 2016 13:21:06 -0500 Message-Id: <1451931676-68481-8-git-send-email-trond.myklebust@primarydata.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1451931676-68481-7-git-send-email-trond.myklebust@primarydata.com> References: <1451931676-68481-1-git-send-email-trond.myklebust@primarydata.com> <1451931676-68481-2-git-send-email-trond.myklebust@primarydata.com> <1451931676-68481-3-git-send-email-trond.myklebust@primarydata.com> <1451931676-68481-4-git-send-email-trond.myklebust@primarydata.com> <1451931676-68481-5-git-send-email-trond.myklebust@primarydata.com> <1451931676-68481-6-git-send-email-trond.myklebust@primarydata.com> <1451931676-68481-7-git-send-email-trond.myklebust@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow synchronous RPC calls to wait for pending RPC calls to finish, but also allow asynchronous ones to just fire off another commit. With this patch, the xfstests generic/074 test completes in 226s instead of 242s Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 6 ----- fs/nfs/file.c | 2 +- fs/nfs/nfstrace.h | 1 - fs/nfs/pnfs_nfs.c | 5 +--- fs/nfs/write.c | 70 +++++++++++++++++++++++-------------------------- include/linux/nfs_fs.h | 1 - include/linux/nfs_xdr.h | 1 - 7 files changed, 35 insertions(+), 51 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index e73693f75dee..14f77df79c25 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -721,14 +721,8 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) nfs_direct_write_complete(dreq, data->inode); } -static void nfs_direct_error_cleanup(struct nfs_inode *nfsi) -{ - /* There is no lock to clear */ -} - static const struct nfs_commit_completion_ops nfs_direct_commit_completion_ops = { .completion = nfs_direct_commit_complete, - .error_cleanup = nfs_direct_error_cleanup, }; static void nfs_direct_commit_schedule(struct nfs_direct_req *dreq) diff --git a/fs/nfs/file.c b/fs/nfs/file.c index 93e236429c5d..e6ef80ec699c 100644 --- a/fs/nfs/file.c +++ b/fs/nfs/file.c @@ -514,7 +514,7 @@ static void nfs_check_dirty_writeback(struct page *page, * so it will not block due to pages that will shortly be freeable. */ nfsi = NFS_I(mapping->host); - if (test_bit(NFS_INO_COMMIT, &nfsi->flags)) { + if (atomic_read(&nfsi->commit_info.rpcs_out)) { *writeback = true; return; } diff --git a/fs/nfs/nfstrace.h b/fs/nfs/nfstrace.h index 59f838cdc009..9f80a086b612 100644 --- a/fs/nfs/nfstrace.h +++ b/fs/nfs/nfstrace.h @@ -39,7 +39,6 @@ { 1 << NFS_INO_INVALIDATING, "INVALIDATING" }, \ { 1 << NFS_INO_FLUSHING, "FLUSHING" }, \ { 1 << NFS_INO_FSCACHE, "FSCACHE" }, \ - { 1 << NFS_INO_COMMIT, "COMMIT" }, \ { 1 << NFS_INO_LAYOUTCOMMIT, "NEED_LAYOUTCOMMIT" }, \ { 1 << NFS_INO_LAYOUTCOMMITTING, "LAYOUTCOMMIT" }) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 24655b807d44..3c8e3a44e6ea 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -266,17 +266,14 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, } else { nfs_retry_commit(mds_pages, NULL, cinfo, 0); pnfs_generic_retry_commit(cinfo, 0); - cinfo->completion_ops->error_cleanup(NFS_I(inode)); return -ENOMEM; } } nreq += pnfs_generic_alloc_ds_commits(cinfo, &list); - if (nreq == 0) { - cinfo->completion_ops->error_cleanup(NFS_I(inode)); + if (nreq == 0) goto out; - } atomic_add(nreq, &cinfo->mds->rpcs_out); diff --git a/fs/nfs/write.c b/fs/nfs/write.c index 0aa3e6b3db70..ae29f082c9c2 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -21,6 +21,8 @@ #include #include #include +#include +#include #include @@ -1535,27 +1537,29 @@ static void nfs_writeback_result(struct rpc_task *task, } } +static int nfs_wait_atomic_killable(atomic_t *key) +{ + if (fatal_signal_pending(current)) + return -ERESTARTSYS; + freezable_schedule_unsafe(); + return 0; +} -static int nfs_commit_set_lock(struct nfs_inode *nfsi, int may_wait) +static int wait_on_commit(struct nfs_mds_commit_info *cinfo) { - int ret; + return wait_on_atomic_t(&cinfo->rpcs_out, + nfs_wait_atomic_killable, TASK_KILLABLE); +} - if (!test_and_set_bit(NFS_INO_COMMIT, &nfsi->flags)) - return 1; - if (!may_wait) - return 0; - ret = out_of_line_wait_on_bit_lock(&nfsi->flags, - NFS_INO_COMMIT, - nfs_wait_bit_killable, - TASK_KILLABLE); - return (ret < 0) ? ret : 1; +static void nfs_commit_begin(struct nfs_mds_commit_info *cinfo) +{ + atomic_inc(&cinfo->rpcs_out); } -static void nfs_commit_clear_lock(struct nfs_inode *nfsi) +static void nfs_commit_end(struct nfs_mds_commit_info *cinfo) { - clear_bit(NFS_INO_COMMIT, &nfsi->flags); - smp_mb__after_atomic(); - wake_up_bit(&nfsi->flags, NFS_INO_COMMIT); + if (atomic_dec_and_test(&cinfo->rpcs_out)) + wake_up_atomic_t(&cinfo->rpcs_out); } void nfs_commitdata_release(struct nfs_commit_data *data) @@ -1693,7 +1697,6 @@ nfs_commit_list(struct inode *inode, struct list_head *head, int how, data->mds_ops, how, 0); out_bad: nfs_retry_commit(head, NULL, cinfo, 0); - cinfo->completion_ops->error_cleanup(NFS_I(inode)); return -ENOMEM; } @@ -1755,8 +1758,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data) clear_bdi_congested(&nfss->backing_dev_info, BLK_RW_ASYNC); nfs_init_cinfo(&cinfo, data->inode, data->dreq); - if (atomic_dec_and_test(&cinfo.mds->rpcs_out)) - nfs_commit_clear_lock(NFS_I(data->inode)); + nfs_commit_end(cinfo.mds); } static void nfs_commit_release(void *calldata) @@ -1775,7 +1777,6 @@ static const struct rpc_call_ops nfs_commit_ops = { static const struct nfs_commit_completion_ops nfs_commit_completion_ops = { .completion = nfs_commit_release_pages, - .error_cleanup = nfs_commit_clear_lock, }; int nfs_generic_commit_list(struct inode *inode, struct list_head *head, @@ -1794,30 +1795,25 @@ int nfs_commit_inode(struct inode *inode, int how) LIST_HEAD(head); struct nfs_commit_info cinfo; int may_wait = how & FLUSH_SYNC; + int error = 0; int res; - res = nfs_commit_set_lock(NFS_I(inode), may_wait); - if (res <= 0) - goto out_mark_dirty; nfs_init_cinfo_from_inode(&cinfo, inode); + nfs_commit_begin(cinfo.mds); res = nfs_scan_commit(inode, &head, &cinfo); - if (res) { - int error; - + if (res) error = nfs_generic_commit_list(inode, &head, how, &cinfo); - if (error < 0) - return error; - if (!may_wait) - goto out_mark_dirty; - error = wait_on_bit_action(&NFS_I(inode)->flags, - NFS_INO_COMMIT, - nfs_wait_bit_killable, - TASK_KILLABLE); - if (error < 0) - return error; - } else - nfs_commit_clear_lock(NFS_I(inode)); + nfs_commit_end(cinfo.mds); + if (error < 0) + goto out_error; + if (!may_wait) + goto out_mark_dirty; + error = wait_on_commit(cinfo.mds); + if (error < 0) + return error; return res; +out_error: + res = error; /* Note: If we exit without ensuring that the commit is complete, * we must mark the inode as dirty. Otherwise, future calls to * sync_inode() with the WB_SYNC_ALL flag set will fail to ensure diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h index c0e961474a52..ebf0bd72a42b 100644 --- a/include/linux/nfs_fs.h +++ b/include/linux/nfs_fs.h @@ -216,7 +216,6 @@ struct nfs_inode { #define NFS_INO_FLUSHING (4) /* inode is flushing out data */ #define NFS_INO_FSCACHE (5) /* inode can be cached by FS-Cache */ #define NFS_INO_FSCACHE_LOCK (6) /* FS-Cache cookie management lock */ -#define NFS_INO_COMMIT (7) /* inode is committing unstable writes */ #define NFS_INO_LAYOUTCOMMIT (9) /* layoutcommit required */ #define NFS_INO_LAYOUTCOMMITTING (10) /* layoutcommit inflight */ #define NFS_INO_LAYOUTSTATS (11) /* layoutstats inflight */ diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index e89dbb14138c..a8905b7d4d7f 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1423,7 +1423,6 @@ struct nfs_mds_commit_info { struct nfs_commit_data; struct nfs_inode; struct nfs_commit_completion_ops { - void (*error_cleanup) (struct nfs_inode *nfsi); void (*completion) (struct nfs_commit_data *data); };