From patchwork Thu Oct 19 18:49:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 10018173 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5BCAD60215 for ; Thu, 19 Oct 2017 18:49:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E9F628D61 for ; Thu, 19 Oct 2017 18:49:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 40A5928E2F; Thu, 19 Oct 2017 18:49:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1520328D61 for ; Thu, 19 Oct 2017 18:49:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752036AbdJSStY (ORCPT ); Thu, 19 Oct 2017 14:49:24 -0400 Received: from us-smtp-delivery-194.mimecast.com ([63.128.21.194]:39522 "EHLO us-smtp-delivery-194.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751466AbdJSStX (ORCPT ); Thu, 19 Oct 2017 14:49:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=primarydata.com; s=mimecast20170802; t=1508438962; h=from:subject:date:message-id:to:cc:mime-version:content-type:content-transfer-encoding:in-reply-to:references; bh=rKCJ6MEkfJoCnAth75MS8WEXZmP30iv2M9HBl5iabs0=; b=TizCXPtLolILysv0fO+uYPJywSsdmIbACRsPPQqDwNeRNFBJV2Hn8Rnor28TA+LKYVv3YhxFUvriI7gwNjxxop065JUUxRylLItg8DH9FYabeAN7HWtjkpsCkAmm/DqL4/PM3Fm8tHY7ZGn5100gA/epwAq5E3u14Rz815Nssmc= Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01lp0177.outbound.protection.outlook.com [216.32.180.177]) (Using TLS) by us-smtp-1.mimecast.com with ESMTP id us-mta-54-kwS1kIylO8KQvlNG5PVzYQ-1; Thu, 19 Oct 2017 14:49:20 -0400 Received: from DM5PR11MB0075.namprd11.prod.outlook.com (10.164.155.144) by DM5PR11MB0073.namprd11.prod.outlook.com (10.164.155.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Thu, 19 Oct 2017 18:49:18 +0000 Received: from DM5PR11MB0075.namprd11.prod.outlook.com ([10.164.155.144]) by DM5PR11MB0075.namprd11.prod.outlook.com ([10.164.155.144]) with mapi id 15.20.0077.022; Thu, 19 Oct 2017 18:49:18 +0000 From: Trond Myklebust To: "lorenzo.pieralisi@arm.com" CC: "anna.schumaker@netapp.com" , "linux-nfs@vger.kernel.org" Subject: Re: [PATCH] SUNRPC: Remove redundant call to cancel_work_sync() in xprt_destroy() Thread-Topic: [PATCH] SUNRPC: Remove redundant call to cancel_work_sync() in xprt_destroy() Thread-Index: AQHTQrr/1UT92IoXz0iB0pzKwvXNp6LgO9UAgAtUogA= Date: Thu, 19 Oct 2017 18:49:18 +0000 Message-ID: <1508438956.72950.1.camel@primarydata.com> References: <20171011180134.52476-1-trond.myklebust@primarydata.com> <20171012134730.GA4803@red-moon> In-Reply-To: <20171012134730.GA4803@red-moon> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [68.49.162.121] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DM5PR11MB0073; 20:IAmmJ3q2bq9GsoJXV3aDjwUdn6OneNPho0WtPqTieu8T6s+JCAzxAHln2jhHa4nCDW2UwG58nz56R1Gxjgou0YQv52fw1QLTxcxOyFGTo76nGLodWixvUGOpS0IZa9B8uepyz4ygcJqPeP6wL9BFkBsdhzdGKKRiUz+NAj+Wsd4= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 4cc04c44-75b3-411e-748d-08d517221ab0 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254172)(2017082002095)(4534020)(4602075)(4603075)(4627075)(201702281549075)(2017052603199)(201703131423095); SRVR:DM5PR11MB0073; x-ms-traffictypediagnostic: DM5PR11MB0073: x-exchange-antispam-report-test: UriScan:(21532816269658); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(93001095)(6041248)(20161123555025)(2016111802025)(20161123564025)(20161123560025)(20161123558100)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(6043046)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM5PR11MB0073; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM5PR11MB0073; x-forefront-prvs: 0465429B7F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(346002)(39830400002)(376002)(199003)(377424004)(189002)(24454002)(8936002)(316002)(6506006)(6916009)(36756003)(66066001)(81156014)(6512007)(106356001)(97736004)(2950100002)(2351001)(53936002)(6436002)(68736007)(50986999)(4326008)(99286003)(4001150100001)(54906003)(101416001)(54356999)(77096006)(76176999)(6486002)(229853002)(81166006)(575784001)(105586002)(2501003)(6246003)(86362001)(8676002)(7736002)(103116003)(25786009)(102836003)(6116002)(5660300001)(2900100001)(2906002)(14454004)(3660700001)(3280700002)(478600001)(3846002)(53546010)(33646002)(5640700003)(305945005)(189998001); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR11MB0073; H:DM5PR11MB0075.namprd11.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: <693FF5A4D4B70C43ACB64AAC2A446DB2@namprd11.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: primarydata.com X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Oct 2017 18:49:18.4870 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 03193ed6-8726-4bb3-a832-18ab0d28adb7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR11MB0073 X-MC-Unique: kwS1kIylO8KQvlNG5PVzYQ-1 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, 2017-10-12 at 14:47 +0100, Lorenzo Pieralisi wrote: > Hi Trond, > > On Wed, Oct 11, 2017 at 02:01:34PM -0400, Trond Myklebust wrote: > > We know that the socket autoclose cannot be queued after we've set > > the XPRT_LOCKED bit, so the call to cancel_work_sync() is > > redundant. > > In addition, it is causing lockdep to complain about a false ABA > > lock dependency. > > > > Signed-off-by: Trond Myklebust > > --- > > net/sunrpc/xprt.c | 1 - > > 1 file changed, 1 deletion(-) > > > > diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c > > index e741ec2b4d8e..5f12fe145f02 100644 > > --- a/net/sunrpc/xprt.c > > +++ b/net/sunrpc/xprt.c > > @@ -1464,7 +1464,6 @@ static void xprt_destroy(struct rpc_xprt > > *xprt) > > rpc_destroy_wait_queue(&xprt->pending); > > rpc_destroy_wait_queue(&xprt->sending); > > rpc_destroy_wait_queue(&xprt->backlog); > > - cancel_work_sync(&xprt->task_cleanup); > > This does not make the lockdep warning go away, actually the lockdep > is triggered by the xs_destroy() cancel_work_sync() call but I do not > know this code path so I can't really comment on it, let me know if > there is any specific test I can carry out. > > Thanks for looking into this, > Lorenzo Sorry for the delay. Does this one help? 8<---------------------------------- From 528fd3547bad0bdd31c8f987e5bd00c83df8af39 Mon Sep 17 00:00:00 2001 From: Trond Myklebust Date: Thu, 19 Oct 2017 12:13:10 -0400 Subject: [PATCH] SUNRPC: Destroy transport from the system workqueue The transport may need to flush transport connect and receive tasks that are running on rpciod. In order to do so safely, we need to ensure that the caller of cancel_work_sync() etc is not itself running on rpciod. Do so by running the destroy task from the system workqueue. Signed-off-by: Trond Myklebust --- net/sunrpc/xprt.c | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) -- 2.13.6 -- Trond Myklebust Linux NFS client maintainer, PrimaryData trond.myklebust@primarydata.com diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 1a39ad14c42f..898485e3ece4 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1445,6 +1445,23 @@ struct rpc_xprt *xprt_create_transport(struct xprt_create *args) return xprt; } +static void xprt_destroy_cb(struct work_struct *work) +{ + struct rpc_xprt *xprt = + container_of(work, struct rpc_xprt, task_cleanup); + + rpc_xprt_debugfs_unregister(xprt); + rpc_destroy_wait_queue(&xprt->binding); + rpc_destroy_wait_queue(&xprt->pending); + rpc_destroy_wait_queue(&xprt->sending); + rpc_destroy_wait_queue(&xprt->backlog); + kfree(xprt->servername); + /* + * Tear down transport state and free the rpc_xprt + */ + xprt->ops->destroy(xprt); +} + /** * xprt_destroy - destroy an RPC transport, killing off all requests. * @xprt: transport to destroy @@ -1454,22 +1471,19 @@ static void xprt_destroy(struct rpc_xprt *xprt) { dprintk("RPC: destroying transport %p\n", xprt); - /* Exclude transport connect/disconnect handlers */ + /* + * Exclude transport connect/disconnect handlers and autoclose + */ wait_on_bit_lock(&xprt->state, XPRT_LOCKED, TASK_UNINTERRUPTIBLE); del_timer_sync(&xprt->timer); - rpc_xprt_debugfs_unregister(xprt); - rpc_destroy_wait_queue(&xprt->binding); - rpc_destroy_wait_queue(&xprt->pending); - rpc_destroy_wait_queue(&xprt->sending); - rpc_destroy_wait_queue(&xprt->backlog); - cancel_work_sync(&xprt->task_cleanup); - kfree(xprt->servername); /* - * Tear down transport state and free the rpc_xprt + * Destroy sockets etc from the system workqueue so they can + * safely flush receive work running on rpciod. */ - xprt->ops->destroy(xprt); + INIT_WORK(&xprt->task_cleanup, xprt_destroy_cb); + schedule_work(&xprt->task_cleanup); } static void xprt_destroy_kref(struct kref *kref)