From patchwork Fri Mar 17 21:03:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olga Kornievskaia X-Patchwork-Id: 9631603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0779C60245 for ; Fri, 17 Mar 2017 21:04:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEEAE28616 for ; Fri, 17 Mar 2017 21:04:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E3E8D28628; Fri, 17 Mar 2017 21:04:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B60A28616 for ; Fri, 17 Mar 2017 21:04:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751098AbdCQVEZ (ORCPT ); Fri, 17 Mar 2017 17:04:25 -0400 Received: from mx141.netapp.com ([216.240.21.12]:8292 "EHLO mx141.netapp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751207AbdCQVET (ORCPT ); Fri, 17 Mar 2017 17:04:19 -0400 X-IronPort-AV: E=Sophos;i="5.36,178,1486454400"; d="scan'208";a="190162771" Received: from vmwexchts03-prd.hq.netapp.com ([10.122.105.31]) by mx141-out.netapp.com with ESMTP; 17 Mar 2017 13:54:17 -0700 Received: from smtp2.corp.netapp.com (10.57.159.114) by VMWEXCHTS03-PRD.hq.netapp.com (10.122.105.31) with Microsoft SMTP Server id 15.0.1210.3; Fri, 17 Mar 2017 14:04:04 -0700 Received: from localhost.localdomain ([10.63.235.168]) by smtp2.corp.netapp.com (8.13.1/8.13.1/NTAP-1.6) with ESMTP id v2HL3wWk019789; Fri, 17 Mar 2017 14:04:04 -0700 (PDT) From: Olga Kornievskaia To: CC: Subject: [RFC v2 07/12] NFS handle COPY reply CB_OFFLOAD call race Date: Fri, 17 Mar 2017 17:03:52 -0400 Message-ID: <20170317210357.48079-8-kolga@netapp.com> X-Mailer: git-send-email 2.10.1 (Apple Git-78) In-Reply-To: <20170317210357.48079-1-kolga@netapp.com> References: <20170317210357.48079-1-kolga@netapp.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It's possible that server replies back with CB_OFFLOAD call and COPY reply at the same time such that client will process CB_OFFLOAD before reply to COPY. For that keep a list of pending callback stateids received and them before waiting on completion check the pending list. Cleanup any pending copies on the client shutdown. Signed-off-by: Olga Kornievskaia --- fs/nfs/callback_proc.c | 17 ++++++++++++++--- fs/nfs/nfs42proc.c | 24 +++++++++++++++++++++--- fs/nfs/nfs4client.c | 15 +++++++++++++++ include/linux/nfs_fs_sb.h | 1 + 4 files changed, 51 insertions(+), 6 deletions(-) diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c index c1b081c..01bcb4b 100644 --- a/fs/nfs/callback_proc.c +++ b/fs/nfs/callback_proc.c @@ -699,11 +699,12 @@ __be32 nfs4_callback_offload(struct cb_offloadargs *args, void *dummy, { struct nfs_server *server; struct nfs4_copy_state *copy; + bool found = false; + spin_lock(&cps->clp->cl_lock); rcu_read_lock(); list_for_each_entry_rcu(server, &cps->clp->cl_superblocks, client_link) { - spin_lock(&server->nfs_client->cl_lock); list_for_each_entry(copy, &server->ss_copies, copies) { if (memcmp(args->coa_stateid.other, copy->stateid.other, @@ -711,13 +712,23 @@ __be32 nfs4_callback_offload(struct cb_offloadargs *args, void *dummy, continue; nfs4_copy_cb_args(copy, args); complete(©->completion); - spin_unlock(&server->nfs_client->cl_lock); + found = true; goto out; } - spin_unlock(&server->nfs_client->cl_lock); } out: rcu_read_unlock(); + if (!found) { + copy = kzalloc(sizeof(struct nfs4_copy_state), GFP_NOFS); + if (!copy) { + spin_unlock(&cps->clp->cl_lock); + return -ENOMEM; + } + memcpy(©->stateid, &args->coa_stateid, NFS4_STATEID_SIZE); + nfs4_copy_cb_args(copy, args); + list_add_tail(©->copies, &cps->clp->pending_cb_stateids); + } + spin_unlock(&cps->clp->cl_lock); return 0; } #endif /* CONFIG_NFS_V4_2 */ diff --git a/fs/nfs/nfs42proc.c b/fs/nfs/nfs42proc.c index 27a6eb8..8ee9fb8 100644 --- a/fs/nfs/nfs42proc.c +++ b/fs/nfs/nfs42proc.c @@ -137,15 +137,32 @@ static int handle_async_copy(struct nfs42_copy_res *res, uint64_t *ret_count) { struct nfs4_copy_state *copy; - int status; + int status = NFS4_OK; + bool found_pending = false; + + spin_lock(&server->nfs_client->cl_lock); + list_for_each_entry(copy, &server->nfs_client->pending_cb_stateids, + copies) { + if (memcmp(&res->write_res.stateid, ©->stateid, + NFS4_STATEID_SIZE)) + continue; + found_pending = true; + list_del(©->copies); + break; + } + if (found_pending) { + spin_unlock(&server->nfs_client->cl_lock); + goto out; + } copy = kzalloc(sizeof(struct nfs4_copy_state), GFP_NOFS); - if (!copy) + if (!copy) { + spin_unlock(&server->nfs_client->cl_lock); return -ENOMEM; + } memcpy(©->stateid, &res->write_res.stateid, NFS4_STATEID_SIZE); init_completion(©->completion); - spin_lock(&server->nfs_client->cl_lock); list_add_tail(©->copies, &server->ss_copies); spin_unlock(&server->nfs_client->cl_lock); @@ -153,6 +170,7 @@ static int handle_async_copy(struct nfs42_copy_res *res, spin_lock(&server->nfs_client->cl_lock); list_del_init(©->copies); spin_unlock(&server->nfs_client->cl_lock); +out: *ret_count = copy->count; status = -copy->error; if (copy->count && copy->verf.committed != NFS_FILE_SYNC) diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c index 5ae9d64..bebb97e1 100644 --- a/fs/nfs/nfs4client.c +++ b/fs/nfs/nfs4client.c @@ -156,9 +156,23 @@ struct rpc_clnt * } } +static void +nfs4_cleanup_callback(struct nfs_client *clp) +{ + struct nfs4_copy_state *cp_state; + + while (!list_empty(&clp->pending_cb_stateids)) { + cp_state = list_entry(clp->pending_cb_stateids.next, + struct nfs4_copy_state, copies); + list_del(&cp_state->copies); + kfree(cp_state); + } +} + void nfs41_shutdown_client(struct nfs_client *clp) { if (nfs4_has_session(clp)) { + nfs4_cleanup_callback(clp); nfs4_shutdown_ds_clients(clp); nfs4_destroy_session(clp->cl_session); nfs4_destroy_clientid(clp); @@ -202,6 +216,7 @@ struct nfs_client *nfs4_alloc_client(const struct nfs_client_initdata *cl_init) #if IS_ENABLED(CONFIG_NFS_V4_1) init_waitqueue_head(&clp->cl_lock_waitq); #endif + INIT_LIST_HEAD(&clp->pending_cb_stateids); return clp; error: diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h index 9b252bb..04f6cb4 100644 --- a/include/linux/nfs_fs_sb.h +++ b/include/linux/nfs_fs_sb.h @@ -118,6 +118,7 @@ struct nfs_client { #endif struct net *cl_net; + struct list_head pending_cb_stateids; }; /*