From patchwork Thu Mar 9 17:56:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Weston Andros Adamson X-Patchwork-Id: 9613797 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9354B60414 for ; Thu, 9 Mar 2017 17:57:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88123286D5 for ; Thu, 9 Mar 2017 17:57:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C7EB286DA; Thu, 9 Mar 2017 17:57:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9A648286D5 for ; Thu, 9 Mar 2017 17:57:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754865AbdCIR5O (ORCPT ); Thu, 9 Mar 2017 12:57:14 -0500 Received: from mail-it0-f68.google.com ([209.85.214.68]:34851 "EHLO mail-it0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754890AbdCIR5H (ORCPT ); Thu, 9 Mar 2017 12:57:07 -0500 Received: by mail-it0-f68.google.com with SMTP id f203so10818306itf.2 for ; Thu, 09 Mar 2017 09:56:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=WGxn+P1eCXp0hgou6XSq8uyyAxO21neaUGjn+JK5EqY=; b=WslaVYPF/QIxd7weh0hoL/fbMbiGiu357o5LtDherNQEsWLZ07B1K63JqRADcPqAwV 7M+CxD1+u7yVkp8+MznugZQtYJt0SU0daU9SRIZCDlI6W2UAQxIN2SlLDAPB1JekYGd0 zKd5HZ7kLWc8maxT1Tt0smcuL/k4R87bbsXJfvI/IfC30o579rlpCSNTi148r21hvN9H i5l5RDxw47XYWuDsczVg8t4jsjyO6Ozqbm0jqhtiCHa82p/QerBqdbclygm4/feHYN7C ss2/O5n5+mYkpobA5asFhakmdF80lpY5BfBC+ZPBpRcqoplaEWsWMT33cbkQuHoyFkE5 yeIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=WGxn+P1eCXp0hgou6XSq8uyyAxO21neaUGjn+JK5EqY=; b=HoFaPAR1GsKEHVSdG3jzOe4wts5qSg96HZHABcC73Z/RfbgRg9qIpT280OLxFgVscX 5YTVSl1BkGiJtHotcKJoydVZ4LIF3iHlZdd5yO5h2vra8Gx2bwJcJuCHOYoeghaxNsTz Lmpmgfj6PcZw42zNYSqYd8GBcMmhMaB2CCO9X6MQe3FHjOOpIwoAN6GiQacuH3/5q36q wjXigQPp3lYE6MnWAWq8qdGuMhRyhRoQEIXXsX4uxFKQSLkIZS89zsUOD64GptsycrEo WZHBjAO95Iwi/iamFaKFIQKDymboxUo3psmRqUGT56GqzQBO9DRWIlZNvtq9z3rbCsis 5jIQ== X-Gm-Message-State: AMke39l3r/vFAeQfU9JgkEe5KE/ts8N9i+Sc/Z2H1G39wxDFB4X/m3wbBaTIof3CBbKdQg== X-Received: by 10.36.46.5 with SMTP id i5mr13448167ita.23.1489082213131; Thu, 09 Mar 2017 09:56:53 -0800 (PST) Received: from dhcp-51.robotsandstuff.fake (c-68-49-175-159.hsd1.mi.comcast.net. [68.49.175.159]) by smtp.gmail.com with ESMTPSA id y21sm3244475ioi.0.2017.03.09.09.56.52 (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 09 Mar 2017 09:56:52 -0800 (PST) From: Weston Andros Adamson X-Google-Original-From: Weston Andros Adamson To: trond.myklebust@primarydata.com, anna.schumaker@netapp.com Cc: linux-nfs@vger.kernel.org, Weston Andros Adamson Subject: [PATCH 2/2] pNFS/flexfiles: never nfs4_mark_deviceid_unavailable Date: Thu, 9 Mar 2017 12:56:49 -0500 Message-Id: <20170309175649.8914-2-dros@primarydata.com> X-Mailer: git-send-email 2.10.1 (Apple Git-78) In-Reply-To: <20170309175649.8914-1-dros@primarydata.com> References: <20170309175649.8914-1-dros@primarydata.com> Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The flexfiles layout should never mark a device unavailable. Move nfs4_mark_deviceid_unavailable out of nfs4_pnfs_ds_connect and call directly from files layout where it's still needed. The flexfiles driver still handles marked devices in error paths, but will now print a rate limited warning. Signed-off-by: Weston Andros Adamson --- fs/nfs/filelayout/filelayoutdev.c | 1 + fs/nfs/flexfilelayout/flexfilelayout.h | 14 +++++++++++++- fs/nfs/flexfilelayout/flexfilelayoutdev.c | 2 +- fs/nfs/pnfs_nfs.c | 24 ++++++++++++++++-------- 4 files changed, 31 insertions(+), 10 deletions(-) diff --git a/fs/nfs/filelayout/filelayoutdev.c b/fs/nfs/filelayout/filelayoutdev.c index 188120626179..d913e818858f 100644 --- a/fs/nfs/filelayout/filelayoutdev.c +++ b/fs/nfs/filelayout/filelayoutdev.c @@ -282,6 +282,7 @@ nfs4_fl_prepare_ds(struct pnfs_layout_segment *lseg, u32 ds_idx) dataserver_retrans, 4, s->nfs_client->cl_minorversion); if (status) { + nfs4_mark_deviceid_unavailable(devid); ret = NULL; goto out; } diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h index f4f39b0ab09b..98b34c9b0564 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.h +++ b/fs/nfs/flexfilelayout/flexfilelayout.h @@ -175,7 +175,19 @@ ff_layout_no_read_on_rw(struct pnfs_layout_segment *lseg) static inline bool ff_layout_test_devid_unavailable(struct nfs4_deviceid_node *node) { - return nfs4_test_deviceid_unavailable(node); + /* + * Flexfiles should never mark a DS unavailable, but if it does + * print a (ratelimited) warning as this can affect performance. + */ + if (nfs4_test_deviceid_unavailable(node)) { + u32 *p = (u32 *)node->deviceid.data; + + pr_warn_ratelimited("NFS: flexfiles layout referencing an " + "unavailable device [%x%x%x%x]\n", + p[0], p[1], p[2], p[3]); + return true; + } + return false; } static inline int diff --git a/fs/nfs/flexfilelayout/flexfilelayoutdev.c b/fs/nfs/flexfilelayout/flexfilelayoutdev.c index 544e7725e679..85fde93dff77 100644 --- a/fs/nfs/flexfilelayout/flexfilelayoutdev.c +++ b/fs/nfs/flexfilelayout/flexfilelayoutdev.c @@ -421,11 +421,11 @@ nfs4_ff_layout_prepare_ds(struct pnfs_layout_segment *lseg, u32 ds_idx, mirror->mirror_ds->ds_versions[0].wsize = max_payload; goto out; } +out_fail: ff_layout_track_ds_error(FF_LAYOUT_FROM_HDR(lseg->pls_layout), mirror, lseg->pls_range.offset, lseg->pls_range.length, NFS4ERR_NXIO, OP_ILLEGAL, GFP_NOIO); -out_fail: if (fail_return || !ff_layout_has_available_ds(lseg)) pnfs_error_mark_layout_for_return(ino, lseg); ds = NULL; diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index a7691b927af6..7250b95549ec 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -751,9 +751,11 @@ int nfs4_pnfs_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds, struct nfs4_deviceid_node *devid, unsigned int timeo, unsigned int retrans, u32 version, u32 minor_version) { - if (test_and_set_bit(NFS4DS_CONNECTING, &ds->ds_state) == 0) { - int err = 0; + int err; +again: + err = 0; + if (test_and_set_bit(NFS4DS_CONNECTING, &ds->ds_state) == 0) { if (version == 3) { err = _nfs4_pnfs_v3_ds_connect(mds_srv, ds, timeo, retrans); @@ -766,23 +768,29 @@ int nfs4_pnfs_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds, err = -EPROTONOSUPPORT; } - if (err) - nfs4_mark_deviceid_unavailable(devid); nfs4_clear_ds_conn_bit(ds); } else { nfs4_wait_ds_connect(ds); + + /* what was waited on didn't connect AND didn't mark unavail */ + if (!ds->ds_clp && !nfs4_test_deviceid_unavailable(devid)) + goto again; } /* * At this point the ds->ds_clp should be ready, but it might have * hit an error. */ - if (!ds->ds_clp || !nfs_client_init_is_complete(ds->ds_clp)) { - WARN_ON_ONCE(1); - return -EINVAL; + if (!err) { + if (!ds->ds_clp || !nfs_client_init_is_complete(ds->ds_clp)) { + WARN_ON_ONCE(ds->ds_clp || + !nfs4_test_deviceid_unavailable(devid)); + return -EINVAL; + } + err = nfs_client_init_status(ds->ds_clp); } - return nfs_client_init_status(ds->ds_clp); + return err; } EXPORT_SYMBOL_GPL(nfs4_pnfs_ds_connect);