From patchwork Sat Sep 17 21:45:33 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 9337373 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A0B64601C2 for ; Sat, 17 Sep 2016 21:45:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 97BEF28F7B for ; Sat, 17 Sep 2016 21:45:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B9D328F80; Sat, 17 Sep 2016 21:45:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9730B28F7F for ; Sat, 17 Sep 2016 21:45:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932513AbcIQVpp (ORCPT ); Sat, 17 Sep 2016 17:45:45 -0400 Received: from us-smtp-delivery-194.mimecast.com ([63.128.21.194]:26069 "EHLO us-smtp-delivery-194.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932119AbcIQVpm (ORCPT ); Sat, 17 Sep 2016 17:45:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=PrimaryData.onmicrosoft.com; s=selector1-primarydata-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=k9F7v5G4qe6MQrnw+freltoiv6Fci185LaGFUF0Pu5k=; b=PLeDkIUPhYw9q8ZRc6VCCN26fJWzTP3/G5N+7RfH+YA1zdzuqiCkIuBsnG+iqm2RHmAHKt07pKrZt2FFPXjM7kO+DgrkMwcG+24oUQPYHZNnOl5PhqbYvQeXpHQ0x/q8c/WZU9n6Lr+dB8s4C/n8EJLbHDmu2afpBW3tIyKhuFg= Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03lp0023.outbound.protection.outlook.com [216.32.181.23]) (Using TLS) by us-smtp-1.mimecast.com with ESMTP id us-mta-80-zGsx2q_-PZmOr9Juo-3rGw-1; Sat, 17 Sep 2016 17:45:38 -0400 Received: from BN6PR11MB1570.namprd11.prod.outlook.com (10.172.22.149) by BN6PR11MB1572.namprd11.prod.outlook.com (10.172.23.9) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.619.10; Sat, 17 Sep 2016 21:45:33 +0000 Received: from BN6PR11MB1570.namprd11.prod.outlook.com ([10.172.22.149]) by BN6PR11MB1570.namprd11.prod.outlook.com ([10.172.22.149]) with mapi id 15.01.0619.016; Sat, 17 Sep 2016 21:45:33 +0000 From: Trond Myklebust To: "green@linuxhacker.ru" CC: "anna.schumaker@netapp.com" , "linux-nfs@vger.kernel.org" Subject: Re: [PATCH v5 00/25] Fix delegation behaviour when server revokes some state Thread-Topic: [PATCH v5 00/25] Fix delegation behaviour when server revokes some state Thread-Index: AQHSEKJBrJIeY7mnGES4qGyzQzZ0NqB9+pYAgAAD0oCAABBbgIAABFKAgAAlRoA= Date: Sat, 17 Sep 2016 21:45:33 +0000 Message-ID: <1474148731.7526.3.camel@primarydata.com> References: <1474089213-104014-1-git-send-email-trond.myklebust@primarydata.com> <838E325D-2DA1-4229-A046-76316302F813@linuxhacker.ru> <1474140727.7526.1.camel@primarydata.com> In-Reply-To: <1474140727.7526.1.camel@primarydata.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [68.49.162.121] x-ms-office365-filtering-correlation-id: d1d561e1-2c7d-4a2e-2f9a-08d3df43f3f5 x-microsoft-exchange-diagnostics: 1; BN6PR11MB1572; 20:jClTGGQdB5P+12cCHqvAVjSUbxDbQdfFXseixi6EG5Hu9VdnVTXDelCyLfZf509KQysVGvYDHjms7FHjfxhs3cR3niGnFVmcieR4lluVpWBJr1tet7PwXHerLfzgrSUyqDWg5Un4PwzI2TTP4QThzYRJ/L3cEPKBql1DhRaYMiw= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN6PR11MB1572; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6043046)(6042046); SRVR:BN6PR11MB1572; BCL:0; PCL:0; RULEID:; SRVR:BN6PR11MB1572; x-forefront-prvs: 0068C7E410 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(377424004)(199003)(24454002)(377454003)(189002)(92566002)(7846002)(7736002)(68736007)(8936002)(122556002)(81156014)(8676002)(81166006)(1730700003)(11100500001)(110136003)(19580405001)(86362001)(575784001)(2501003)(87936001)(97736004)(33646002)(93886004)(19580395003)(106356001)(106116001)(77096005)(101416001)(4326007)(105586002)(99286002)(2351001)(3280700002)(5640700001)(66066001)(76176999)(3846002)(586003)(4477795004)(36756003)(6116002)(102836003)(103116003)(10400500002)(54356999)(305945005)(5002640100001)(5660300001)(50986999)(3660700001)(189998001)(2900100001)(2950100001)(2906002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN6PR11MB1572; H:BN6PR11MB1570.namprd11.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: MIME-Version: 1.0 X-OriginatorOrg: primarydata.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Sep 2016 21:45:33.3643 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 03193ed6-8726-4bb3-a832-18ab0d28adb7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1572 X-MC-Unique: zGsx2q_-PZmOr9Juo-3rGw-1 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Sat, 2016-09-17 at 15:32 -0400, Trond Myklebust wrote: > On Sat, 2016-09-17 at 15:16 -0400, Oleg Drokin wrote: > > > > On Sep 17, 2016, at 2:18 PM, Trond Myklebust wrote: > > > > > > > > > > > > > > > > > > > > > > > On Sep 17, 2016, at 14:04, Oleg Drokin > > > > wrote: > > > > > > > > > > > > On Sep 17, 2016, at 1:13 AM, Trond Myklebust wrote: > > > > > > > > > > > > > > > > > > > According to RFC5661, if any of the SEQUENCE status bits > > > > > SEQ4_STATUS_EXPIRED_ALL_STATE_REVOKED, > > > > > SEQ4_STATUS_EXPIRED_SOME_STATE_REVOKED, > > > > > SEQ4_STATUS_ADMIN_STATE_REVOKED, > > > > > or SEQ4_STATUS_RECALLABLE_STATE_REVOKED are set, then we need > > > > > to use > > > > > TEST_STATEID to figure out which stateids have been revoked, > > > > > so > > > > > we > > > > > can acknowledge the loss of state using FREE_STATEID. > > > > > > > > > > While we already do this for open and lock state, we have not > > > > > been doing > > > > > so for all the delegations. > > > > > > > > > > v2: nfs_v4_2_minor_ops needs to set .test_and_free_expired > > > > > too > > > > > v3: Now with added lock revoke fixes and > > > > > close/delegreturn/locku fixes > > > > > v4: Close a bunch of corner cases > > > > > v5: Report revoked delegations as invalid in > > > > > nfs_have_delegation() > > > > >  Fix an infinite loop in nfs_reap_expired_delegations. > > > > >  Fixes for other looping behaviour > > > > > > > > This time around the loop seems to be more tight, > > > > in userspace process: > > > > > > > > [ 9197.256571] --> nfs41_call_sync_prepare data->seq_server > > > > ffff8800a73ce000 > > > > [ 9197.256572] --> nfs41_setup_sequence > > > > [ 9197.256573] --> nfs4_alloc_slot used_slots=0000 > > > > highest_used=4294967295 max_slots=31 > > > > [ 9197.256574] <-- nfs4_alloc_slot used_slots=0001 > > > > highest_used=0 > > > > slotid=0 > > > > [ 9197.256574] <-- nfs41_setup_sequence slotid=0 seqid=14013800 > > > > [ 9197.256582] encode_sequence: sessionid=1474126170:1:2:0 > > > > seqid=14013800 slotid=0 max_slotid=0 cache_this=1 > > > > [ 9197.256755] --> nfs4_alloc_slot used_slots=0001 > > > > highest_used=0 > > > > max_slots=31 > > > > [ 9197.256756] <-- nfs4_alloc_slot used_slots=0003 > > > > highest_used=1 > > > > slotid=1 > > > > [ 9197.256757] nfs4_free_slot: slotid 1 highest_used_slotid 0 > > > > [ 9197.256758] nfs41_sequence_process: Error 0 free the slot  > > > > [ 9197.256760] nfs4_free_slot: slotid 0 highest_used_slotid > > > > 4294967295 > > > > [ 9197.256779] --> nfs_put_client({2}) > > > > > > What operation is the userspace process hanging on? Do you have a > > > stack trace for it? > > > > seems to be open_create->truncate->ssetattr coming from: > > cp /bin/sleep /mnt/nfs2/racer/12 > > > > (gdb) bt > > #0  nfs41_setup_sequence (session=0xffff88005a853800, > > args=0xffff8800a7253b80,  > >     res=0xffff8800a7253b48, task=0xffff8800b0eb0f00) > >     at /home/green/bk/linux-test/fs/nfs/nfs4proc.c:876 > > #1  0xffffffff813a751c in nfs41_call_sync_prepare (task= > out>,  > >     calldata=0xffff8800a7253b80) > >     at /home/green/bk/linux-test/fs/nfs/nfs4proc.c:966 > > #2  0xffffffff8185c639 in rpc_prepare_task (task=) > >     at /home/green/bk/linux-test/net/sunrpc/sched.c:683 > > #3  0xffffffff8185f12b in __rpc_execute (task=0xffff88005a853800) > >     at /home/green/bk/linux-test/net/sunrpc/sched.c:775 > > #4  0xffffffff818617b4 in rpc_execute (task=0xffff88005a853800) > >     at /home/green/bk/linux-test/net/sunrpc/sched.c:843 > > #5  0xffffffff818539b9 in rpc_run_task > > (task_setup_data=0xffff8800a7253a50) > >     at /home/green/bk/linux-test/net/sunrpc/clnt.c:1052 > > #6  0xffffffff813a75e3 in nfs4_call_sync_sequence (clnt= > out>,  > >     server=, msg=, args= > out>,  > >     res=) at /home/green/bk/linux- > > test/fs/nfs/nfs4proc.c:1051 > > #7  0xffffffff813b4645 in nfs4_call_sync (cache_reply= > out>,  > >     res=, args=, msg= > out>,  > >     server=, clnt=) > >     at /home/green/bk/linux-test/fs/nfs/nfs4proc.c:1069 > > #8  _nfs4_do_setattr (state=, cred=,  > >     res=, arg=, inode= > out>) > > ---Type to continue, or q to quit--- > >     at /home/green/bk/linux-test/fs/nfs/nfs4proc.c:2916 > > #9  nfs4_do_setattr (inode=0xffff880079b152a8, cred= > out>,  > >     fattr=, sattr=, > > state=0xffff880060588e00,  > >     ilabel=, olabel=0x0 ) > >     at /home/green/bk/linux-test/fs/nfs/nfs4proc.c:2955 > > #10 0xffffffff813b4a16 in nfs4_proc_setattr (dentry= > out>,  > >     fattr=0xffff8800a7253b80, sattr=0xffff8800a7253b48) > >     at /home/green/bk/linux-test/fs/nfs/nfs4proc.c:3684 > > #11 0xffffffff8138f1cb in nfs_setattr (dentry=0xffff8800740c1000,  > >   > > Cool! Does the following help? Grrr... There is another bug there... 8<----------------------------------------------------------------- From ca5fd2e055cc0fd49d9a5d44f4d01c9ca01fad98 Mon Sep 17 00:00:00 2001 From: Trond Myklebust Date: Sat, 17 Sep 2016 17:37:47 -0400 Subject: [PATCH] NFSv4: nfs4_copy_delegation_stateid() must fail if the delegation is invalid We must not allow the use of delegations that have been revoked or are being returned. Signed-off-by: Trond Myklebust --- fs/nfs/delegation.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c index 62b2215a30b9..0073848b5ad3 100644 --- a/fs/nfs/delegation.c +++ b/fs/nfs/delegation.c @@ -41,6 +41,17 @@ void nfs_mark_delegation_referenced(struct nfs_delegation *delegation) set_bit(NFS_DELEGATION_REFERENCED, &delegation->flags); } +static bool +nfs4_is_valid_delegation(const struct nfs_delegation *delegation, + fmode_t flags) +{ + if (delegation != NULL && (delegation->type & flags) == flags && + !test_bit(NFS_DELEGATION_REVOKED, &delegation->flags) && + !test_bit(NFS_DELEGATION_RETURNING, &delegation->flags)) + return true; + return false; +} + static int nfs4_do_check_delegation(struct inode *inode, fmode_t flags, bool mark) { @@ -50,9 +61,7 @@ nfs4_do_check_delegation(struct inode *inode, fmode_t flags, bool mark) flags &= FMODE_READ|FMODE_WRITE; rcu_read_lock(); delegation = rcu_dereference(NFS_I(inode)->delegation); - if (delegation != NULL && (delegation->type & flags) == flags && - !test_bit(NFS_DELEGATION_REVOKED, &delegation->flags) && - !test_bit(NFS_DELEGATION_RETURNING, &delegation->flags)) { + if (nfs4_is_valid_delegation(delegation, flags)) { if (mark) nfs_mark_delegation_referenced(delegation); ret = 1; @@ -1054,7 +1063,7 @@ bool nfs4_copy_delegation_stateid(struct inode *inode, fmode_t flags, flags &= FMODE_READ|FMODE_WRITE; rcu_read_lock(); delegation = rcu_dereference(nfsi->delegation); - ret = (delegation != NULL && (delegation->type & flags) == flags); + ret = nfs4_is_valid_delegation(delegation, flags); if (ret) { nfs4_stateid_copy(dst, &delegation->stateid); nfs_mark_delegation_referenced(delegation);