From patchwork Mon Jan 25 01:54:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 12042811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1314C433E0 for ; Mon, 25 Jan 2021 02:23:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A93892168B for ; Mon, 25 Jan 2021 02:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726873AbhAYCWy (ORCPT ); Sun, 24 Jan 2021 21:22:54 -0500 Received: from mail.kernel.org ([198.145.29.99]:42398 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726915AbhAYBzY (ORCPT ); Sun, 24 Jan 2021 20:55:24 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4AE79207AB for ; Mon, 25 Jan 2021 01:54:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611539677; bh=Kq8s+A5QRTVAgmNA6JsSa0lAbw1Txw9em9g6cujZzJY=; h=From:To:Subject:Date:From; b=lP5cuMNPz3ocT0W7ZZ7gDuqgNa+6kiJxuy2aYh8ZdYf+zTE3Tz4rCKtyRVhRR3XOF m76gnbdr7+17TQqMnj/9uuBdtrrXdEgi0G91KzwfvyUUlhOyRpsgpfFgY7X4k7iHZr Ohvx8eD7x2tsEt7RfXheje9tFgK493sSwRc/hZV3s/MoNiIsIsKrfWoajcepEMhk+A ah05MQE9UCa0RsZ9kFkdddPDOmHeO+LeYrt4ETGLzE3nXSqyrSjUrJT96jDKfaCa4q cw0B2jcaWJv7oasTwpOgCjMfDmbUPOZlghdMtmkL0Uf3CuVIRMFxg5KFrpMg46d8pa HF2JhdK4gY0Fg== From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 1/4] pNFS/NFSv4: Fix a layout segment leak in pnfs_layout_process() Date: Sun, 24 Jan 2021 20:54:32 -0500 Message-Id: <20210125015435.45979-1-trondmy@kernel.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust If the server returns a new stateid that does not match the one in our cache, then pnfs_layout_process() will leak the layout segments returned by pnfs_mark_layout_stateid_invalid(). Fixes: 9888d837f3cf ("pNFS: Force a retry of LAYOUTGET if the stateid doesn't match our cache") Signed-off-by: Trond Myklebust --- fs/nfs/pnfs.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 4f274f21c4ab..e68e6f8cb407 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -2417,6 +2417,7 @@ pnfs_layout_process(struct nfs4_layoutget *lgp) spin_unlock(&ino->i_lock); lseg->pls_layout = lo; NFS_SERVER(ino)->pnfs_curr_ld->free_lseg(lseg); + pnfs_free_lseg_list(&free_me); return ERR_PTR(-EAGAIN); } From patchwork Mon Jan 25 01:54:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 12042813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F149BC433E0 for ; Mon, 25 Jan 2021 02:24:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BE2482168B for ; Mon, 25 Jan 2021 02:24:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726863AbhAYCXr (ORCPT ); Sun, 24 Jan 2021 21:23:47 -0500 Received: from mail.kernel.org ([198.145.29.99]:42400 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726921AbhAYBzY (ORCPT ); Sun, 24 Jan 2021 20:55:24 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 14560207B6 for ; Mon, 25 Jan 2021 01:54:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611539678; bh=cBWvWPlPvNudHQcW0ZkEDfh/yrXqrX6wOhwUPt3k7wg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=XgZlYRoQLAps3lfsCNDCpLSRhtgkOLVjctAi2pyoGjlrbJNQGcXKyY9KuuG4VDvnM juxKFjOcOR4TuaBbNG4v4sCyLh8Pih8mbgyLu3nonY+f4NOqw9sT8us5BHJ71K9B+t 9P++xMLpYRWXIvEl8oUpVnewPLLvRry1eKEqpYEMxM5u+Bi3imkgIeATypa3v9bIwq uqznJUtrSqOzW0Iu2M8MmfhLjlFcpYbOsq/yOQIN0DDLREUgI76JPFKA4PychZ6XLD bAmC3jcuv/HRRcSB3D6gOrg+BeNewgtRdJusCCTuQZxrL6GdgNEk4+6aEfAXa9hV9C XTGNRd0J1FFMQ== From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 2/4] pNFS/NFSv4: Try to return invalid layout in pnfs_layout_process() Date: Sun, 24 Jan 2021 20:54:33 -0500 Message-Id: <20210125015435.45979-2-trondmy@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210125015435.45979-1-trondmy@kernel.org> References: <20210125015435.45979-1-trondmy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust If the server returns a new stateid that does not match the one in our cache, then try to return the one we hold instead of just invalidating it on the client side. This ensures that both client and server will agree that the stateid is invalid. Signed-off-by: Trond Myklebust --- fs/nfs/pnfs.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index e68e6f8cb407..d6262289cf4a 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -2398,7 +2398,13 @@ pnfs_layout_process(struct nfs4_layoutget *lgp) * We got an entirely new state ID. Mark all segments for the * inode invalid, and retry the layoutget */ - pnfs_mark_layout_stateid_invalid(lo, &free_me); + struct pnfs_layout_range range = { + .iomode = IOMODE_ANY, + .length = NFS4_MAX_UINT64, + }; + pnfs_set_plh_return_info(lo, IOMODE_ANY, 0); + pnfs_mark_matching_lsegs_return(lo, &lo->plh_return_segs, + &range, 0); goto out_forget; } @@ -2417,7 +2423,6 @@ pnfs_layout_process(struct nfs4_layoutget *lgp) spin_unlock(&ino->i_lock); lseg->pls_layout = lo; NFS_SERVER(ino)->pnfs_curr_ld->free_lseg(lseg); - pnfs_free_lseg_list(&free_me); return ERR_PTR(-EAGAIN); } From patchwork Mon Jan 25 01:54:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 12042815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0775EC433DB for ; Mon, 25 Jan 2021 02:24:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC7342168B for ; Mon, 25 Jan 2021 02:24:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726714AbhAYCYI (ORCPT ); Sun, 24 Jan 2021 21:24:08 -0500 Received: from mail.kernel.org ([198.145.29.99]:42402 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726929AbhAYBzY (ORCPT ); Sun, 24 Jan 2021 20:55:24 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id D2F25224DF for ; Mon, 25 Jan 2021 01:54:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611539679; bh=pq9n19jsRZma9AVdKeOBUawVw72V5gU/vukPoUEihXM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=PCH+rQs3ARfSu2QUDwCpbwJCFdlAKQsBarn5mhSH55Rt9BuZLjFKwo9qLiEyVLujS l0sIXSSgWekpBheYI0t2yP00sRw93prgxuboYiqHYBZUnkaG5LC8YLKL82hvgJoT+x BMfMy0qMsQAiBp1tHQQkAcyJlfRtvpij/uIa6aeTmepfZsM8BxFem5Nwe1X1AUFsrt +ljjhfhXuLpr7HLNHQMlqO4NqeNQ62ngAWZ/3i3zKso3OS8ym9WpudFuXepdwav8xg 7Xjl0q1LBuxUIQli0FosuwA/2Bv9FsNN5AnUhx/mwaEZgrc8y0brNp04ka+HKGhCtR eQxuQl4uBp6Mw== From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 3/4] pNFS/NFSv4: Update the layout barrier when we schedule a layoutreturn Date: Sun, 24 Jan 2021 20:54:34 -0500 Message-Id: <20210125015435.45979-3-trondmy@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210125015435.45979-2-trondmy@kernel.org> References: <20210125015435.45979-1-trondmy@kernel.org> <20210125015435.45979-2-trondmy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust When we're scheduling a layoutreturn, we need to ignore any further incoming layouts with sequence ids that are going to be affected by the layout return. Fixes: 44ea8dfce021 ("NFS/pnfs: Reference the layout cred in pnfs_prepare_layoutreturn()") Signed-off-by: Trond Myklebust --- fs/nfs/pnfs.c | 39 +++++++++++++++++++++------------------ 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index d6262289cf4a..acb63ec00053 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -324,6 +324,21 @@ pnfs_grab_inode_layout_hdr(struct pnfs_layout_hdr *lo) return NULL; } +/* + * Compare 2 layout stateid sequence ids, to see which is newer, + * taking into account wraparound issues. + */ +static bool pnfs_seqid_is_newer(u32 s1, u32 s2) +{ + return (s32)(s1 - s2) > 0; +} + +static void pnfs_barrier_update(struct pnfs_layout_hdr *lo, u32 newseq) +{ + if (pnfs_seqid_is_newer(newseq, lo->plh_barrier)) + lo->plh_barrier = newseq; +} + static void pnfs_set_plh_return_info(struct pnfs_layout_hdr *lo, enum pnfs_iomode iomode, u32 seq) @@ -335,6 +350,7 @@ pnfs_set_plh_return_info(struct pnfs_layout_hdr *lo, enum pnfs_iomode iomode, if (seq != 0) { WARN_ON_ONCE(lo->plh_return_seq != 0 && lo->plh_return_seq != seq); lo->plh_return_seq = seq; + pnfs_barrier_update(lo, seq); } } @@ -639,15 +655,6 @@ static int mark_lseg_invalid(struct pnfs_layout_segment *lseg, return rv; } -/* - * Compare 2 layout stateid sequence ids, to see which is newer, - * taking into account wraparound issues. - */ -static bool pnfs_seqid_is_newer(u32 s1, u32 s2) -{ - return (s32)(s1 - s2) > 0; -} - static bool pnfs_should_free_range(const struct pnfs_layout_range *lseg_range, const struct pnfs_layout_range *recall_range) @@ -984,8 +991,7 @@ pnfs_set_layout_stateid(struct pnfs_layout_hdr *lo, const nfs4_stateid *new, new_barrier = be32_to_cpu(new->seqid); else if (new_barrier == 0) return; - if (pnfs_seqid_is_newer(new_barrier, lo->plh_barrier)) - lo->plh_barrier = new_barrier; + pnfs_barrier_update(lo, new_barrier); } static bool @@ -1183,20 +1189,17 @@ pnfs_prepare_layoutreturn(struct pnfs_layout_hdr *lo, return false; set_bit(NFS_LAYOUT_RETURN, &lo->plh_flags); pnfs_get_layout_hdr(lo); + nfs4_stateid_copy(stateid, &lo->plh_stateid); + *cred = get_cred(lo->plh_lc_cred); if (test_bit(NFS_LAYOUT_RETURN_REQUESTED, &lo->plh_flags)) { - nfs4_stateid_copy(stateid, &lo->plh_stateid); - *cred = get_cred(lo->plh_lc_cred); if (lo->plh_return_seq != 0) stateid->seqid = cpu_to_be32(lo->plh_return_seq); if (iomode != NULL) *iomode = lo->plh_return_iomode; pnfs_clear_layoutreturn_info(lo); - return true; - } - nfs4_stateid_copy(stateid, &lo->plh_stateid); - *cred = get_cred(lo->plh_lc_cred); - if (iomode != NULL) + } else if (iomode != NULL) *iomode = IOMODE_ANY; + pnfs_barrier_update(lo, be32_to_cpu(stateid->seqid)); return true; } From patchwork Mon Jan 25 01:54:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 12042809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C71E7C433DB for ; Mon, 25 Jan 2021 02:23:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 97FB22168B for ; Mon, 25 Jan 2021 02:23:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726855AbhAYCWv (ORCPT ); Sun, 24 Jan 2021 21:22:51 -0500 Received: from mail.kernel.org ([198.145.29.99]:42404 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726934AbhAYBzZ (ORCPT ); Sun, 24 Jan 2021 20:55:25 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9E53B229EF for ; Mon, 25 Jan 2021 01:54:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1611539680; bh=cssEzPA45nM1tOBRahmnYgwGo+srnNnOi9EIslPzwps=; h=From:To:Subject:Date:In-Reply-To:References:From; b=ZM44+ZY7KMoAdkD8k0/7iuZmHFmoF0TyqcmnMjVWseSkeOwP8rYbFTLUfQcudsJI0 MRHktDkwnjO7tDUYX4GlzRbGhSRGcb+vWd1ravqhxFY40I0wgxSnb8t06zG55tuUc2 1RSU9jOORrdC3c5WMizkgHaAROIaZfB4+JFUcXJcr1CV17nTl3NuR7nupkeiICVlmK +nLpEZmELLTSDTqw8uAqBAMWUhedvYicUqSDwvXJT4seY1p5Nf6gFTqnL0PXst/ehY FRqbHK+SGH4bU5/+a/GbA4B9e+UMNizg0sCwrxTeQ5acZIDCPla65u3MrzIBt1hzpJ 2vaxiqw2RpxCg== From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH 4/4] pNFS/NFSv4: Improve rejection of out-of-order layouts Date: Sun, 24 Jan 2021 20:54:35 -0500 Message-Id: <20210125015435.45979-4-trondmy@kernel.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210125015435.45979-3-trondmy@kernel.org> References: <20210125015435.45979-1-trondmy@kernel.org> <20210125015435.45979-2-trondmy@kernel.org> <20210125015435.45979-3-trondmy@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust If a layoutget ends up being reordered w.r.t. a layoutreturn, e.g. due to a layoutget-on-open not knowing a priori which file to lock, then we must assume the layout is no longer being considered valid state by the server. Incrementally improve our ability to reject such states by using the cached old stateid in conjunction with the plh_barrier to try to identify them. Signed-off-by: Trond Myklebust --- fs/nfs/pnfs.c | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index acb63ec00053..af64b4e6fd1f 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -1000,7 +1000,7 @@ pnfs_layout_stateid_blocked(const struct pnfs_layout_hdr *lo, { u32 seqid = be32_to_cpu(stateid->seqid); - return !pnfs_seqid_is_newer(seqid, lo->plh_barrier); + return !pnfs_seqid_is_newer(seqid, lo->plh_barrier) && lo->plh_barrier; } /* lget is set to 1 if called from inside send_layoutget call chain */ @@ -1912,6 +1912,11 @@ static void nfs_layoutget_end(struct pnfs_layout_hdr *lo) wake_up_var(&lo->plh_outstanding); } +static bool pnfs_is_first_layoutget(struct pnfs_layout_hdr *lo) +{ + return test_bit(NFS_LAYOUT_FIRST_LAYOUTGET, &lo->plh_flags); +} + static void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo) { unsigned long *bitlock = &lo->plh_flags; @@ -2386,17 +2391,17 @@ pnfs_layout_process(struct nfs4_layoutget *lgp) goto out_forget; } - if (!pnfs_layout_is_valid(lo)) { - /* We have a completely new layout */ - pnfs_set_layout_stateid(lo, &res->stateid, lgp->cred, true); - } else if (nfs4_stateid_match_other(&lo->plh_stateid, &res->stateid)) { + if (nfs4_stateid_match_other(&lo->plh_stateid, &res->stateid)) { /* existing state ID, make sure the sequence number matches. */ if (pnfs_layout_stateid_blocked(lo, &res->stateid)) { + if (!pnfs_layout_is_valid(lo) && + pnfs_is_first_layoutget(lo)) + lo->plh_barrier = 0; dprintk("%s forget reply due to sequence\n", __func__); goto out_forget; } pnfs_set_layout_stateid(lo, &res->stateid, lgp->cred, false); - } else { + } else if (pnfs_layout_is_valid(lo)) { /* * We got an entirely new state ID. Mark all segments for the * inode invalid, and retry the layoutget @@ -2409,6 +2414,11 @@ pnfs_layout_process(struct nfs4_layoutget *lgp) pnfs_mark_matching_lsegs_return(lo, &lo->plh_return_segs, &range, 0); goto out_forget; + } else { + /* We have a completely new layout */ + if (!pnfs_is_first_layoutget(lo)) + goto out_forget; + pnfs_set_layout_stateid(lo, &res->stateid, lgp->cred, true); } pnfs_get_lseg(lseg);