From patchwork Wed Apr 26 15:55:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 9701615 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D561660245 for ; Wed, 26 Apr 2017 15:56:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5B2F2847F for ; Wed, 26 Apr 2017 15:56:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BABC9285F0; Wed, 26 Apr 2017 15:56:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0ED4E2847F for ; Wed, 26 Apr 2017 15:56:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935732AbdDZPz5 (ORCPT ); Wed, 26 Apr 2017 11:55:57 -0400 Received: from mail-it0-f66.google.com ([209.85.214.66]:35194 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935076AbdDZPzd (ORCPT ); Wed, 26 Apr 2017 11:55:33 -0400 Received: by mail-it0-f66.google.com with SMTP id 70so1016065ita.2 for ; Wed, 26 Apr 2017 08:55:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=emiUU7V9FBNDgN8tDmd1dPvInPtiJBOErn6XGZbGpys=; b=B2fZBl31sHKrIoN8E0iVBtvlwp32wW+6YSSXDrkDebDrZGIYQHavZXCDG6VqAMI1xC 9/HHAsUvvVhOw89mLoZKAnU3RG9mp3qDgFG5F6V0YGGRdm+uZyGZyUMip1UJZBhdR6F+ yTOx4yXk6EC0gC3K7l7IWN3RavGzEcB2djYN7/+pPXT5t7u0cGsvosKv1mNYQcMAnxyh xsET56KxIzr1XVGkEud/ivRkPvH3MdbqBSO8GFzZ9rhmow8sJ33tNp5IBFd2s0YZKCUR 2zqQW58d3gsLOu5w6GnWKJprRZIkRotD/7pqOqkMlcmv3yCxMgd2LqGUXNIcp3eJePUD LFlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=emiUU7V9FBNDgN8tDmd1dPvInPtiJBOErn6XGZbGpys=; b=QsyMb6Ras6ZlyHVvf90oc07+pOF2csuURUQA3tf8ZPK+ibjb1Wo/mduGAr/2C0gls0 peK5apx6QuaRw0T1jBegm0pzLfgTKbGBn3lbwhC3JTlWlQmOg5RPIithAVaYgqOEPdmn JNhvxTmAlg6/C4XKLj+UXQsEyZ08M4qvAPdpuT6otNnEhipgRKSF4dQpHG4dv0kkj0eI e1ztgDc71ZsAnu+2dG6s6ceBYAeqH7y9Eu4FXfJum4Wxg9GX5fj+MhDH/S9HO4jaboDe eYr66uo1bPG7pRkDP9/yaiav46p/VMeU+laSRuh01IwGCiOVpnK/ZYiV53EdKRA/5MoA Wk8g== X-Gm-Message-State: AN3rC/7+kVzZHTnPod8iIvhXKpEr08dpwl3YVhukI/7lqyQKuZz9YCJd /eRspQVbrzekRQ== X-Received: by 10.36.104.72 with SMTP id v69mr11564738itb.27.1493222132752; Wed, 26 Apr 2017 08:55:32 -0700 (PDT) Received: from localhost.localdomain (50-108-104-212.adr01.mskg.mi.frontiernet.net. [50.108.104.212]) by smtp.gmail.com with ESMTPSA id w134sm1741438iod.14.2017.04.26.08.55.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Apr 2017 08:55:32 -0700 (PDT) From: Trond Myklebust To: linux-nfs@vger.kernel.org Cc: "J. Bruce Fields" , Kinglong Mee Subject: [PATCH 2/2] NFSv4: Fix callback server shutdown Date: Wed, 26 Apr 2017 11:55:27 -0400 Message-Id: <20170426155527.92213-3-trond.myklebust@primarydata.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170426155527.92213-2-trond.myklebust@primarydata.com> References: <20170426155527.92213-1-trond.myklebust@primarydata.com> <20170426155527.92213-2-trond.myklebust@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We want to use kthread_stop() in order to ensure the threads are shut down before we tear down the nfs_callback_info in nfs_callback_down. Reported-by: Kinglong Mee Fixes: bb6aeba736ba9 ("NFSv4.x: Switch to using svc_set_num_threads()...") Signed-off-by: Trond Myklebust Cc: J. Bruce Fields --- fs/nfs/callback.c | 24 ++++++++++++++++-------- include/linux/sunrpc/svc.h | 1 + net/sunrpc/svc.c | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 8 deletions(-) diff --git a/fs/nfs/callback.c b/fs/nfs/callback.c index 773774531aff..8cabae9b140e 100644 --- a/fs/nfs/callback.c +++ b/fs/nfs/callback.c @@ -76,7 +76,10 @@ nfs4_callback_svc(void *vrqstp) set_freezable(); - while (!kthread_should_stop()) { + while (!kthread_freezable_should_stop(NULL)) { + + if (signal_pending(current)) + flush_signals(current); /* * Listen for a request on the socket */ @@ -85,6 +88,8 @@ nfs4_callback_svc(void *vrqstp) continue; svc_process(rqstp); } + svc_exit_thread(rqstp); + module_put_and_exit(0); return 0; } @@ -103,9 +108,10 @@ nfs41_callback_svc(void *vrqstp) set_freezable(); - while (!kthread_should_stop()) { - if (try_to_freeze()) - continue; + while (!kthread_freezable_should_stop(NULL)) { + + if (signal_pending(current)) + flush_signals(current); prepare_to_wait(&serv->sv_cb_waitq, &wq, TASK_INTERRUPTIBLE); spin_lock_bh(&serv->sv_cb_lock); @@ -121,11 +127,13 @@ nfs41_callback_svc(void *vrqstp) error); } else { spin_unlock_bh(&serv->sv_cb_lock); - schedule(); + if (!kthread_should_stop()) + schedule(); finish_wait(&serv->sv_cb_waitq, &wq); } - flush_signals(current); } + svc_exit_thread(rqstp); + module_put_and_exit(0); return 0; } @@ -221,14 +229,14 @@ static int nfs_callback_up_net(int minorversion, struct svc_serv *serv, static struct svc_serv_ops nfs40_cb_sv_ops = { .svo_function = nfs4_callback_svc, .svo_enqueue_xprt = svc_xprt_do_enqueue, - .svo_setup = svc_set_num_threads, + .svo_setup = svc_set_num_threads_sync, .svo_module = THIS_MODULE, }; #if defined(CONFIG_NFS_V4_1) static struct svc_serv_ops nfs41_cb_sv_ops = { .svo_function = nfs41_callback_svc, .svo_enqueue_xprt = svc_xprt_do_enqueue, - .svo_setup = svc_set_num_threads, + .svo_setup = svc_set_num_threads_sync, .svo_module = THIS_MODULE, }; diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index e770abeed32d..11cef5a7bc87 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -474,6 +474,7 @@ void svc_pool_map_put(void); struct svc_serv * svc_create_pooled(struct svc_program *, unsigned int, struct svc_serv_ops *); int svc_set_num_threads(struct svc_serv *, struct svc_pool *, int); +int svc_set_num_threads_sync(struct svc_serv *, struct svc_pool *, int); int svc_pool_stats_open(struct svc_serv *serv, struct file *file); void svc_destroy(struct svc_serv *); void svc_shutdown_net(struct svc_serv *, struct net *); diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 98dc33ae738b..bc0f5a0ecbdc 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -795,6 +795,44 @@ svc_set_num_threads(struct svc_serv *serv, struct svc_pool *pool, int nrservs) } EXPORT_SYMBOL_GPL(svc_set_num_threads); +/* destroy old threads */ +static int +svc_stop_kthreads(struct svc_serv *serv, struct svc_pool *pool, int nrservs) +{ + struct task_struct *task; + unsigned int state = serv->sv_nrthreads-1; + + /* destroy old threads */ + do { + task = choose_victim(serv, pool, &state); + if (task == NULL) + break; + kthread_stop(task); + nrservs++; + } while (nrservs < 0); + return 0; +} + +int +svc_set_num_threads_sync(struct svc_serv *serv, struct svc_pool *pool, int nrservs) +{ + if (pool == NULL) { + /* The -1 assumes caller has done a svc_get() */ + nrservs -= (serv->sv_nrthreads-1); + } else { + spin_lock_bh(&pool->sp_lock); + nrservs -= pool->sp_nrthreads; + spin_unlock_bh(&pool->sp_lock); + } + + if (nrservs > 0) + return svc_start_kthreads(serv, pool, nrservs); + if (nrservs < 0) + return svc_stop_kthreads(serv, pool, nrservs); + return 0; +} +EXPORT_SYMBOL_GPL(svc_set_num_threads_sync); + /* * Called from a server thread as it's exiting. Caller must hold the "service * mutex" for the service.