From patchwork Sat Mar 28 15:31:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16E051667 for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E9FA92073B for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409681; bh=6mPriu0gpOtKj59zFY2FBD+rje0XT523KPDFTm2vWOw=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=bxVtBGih6MFZfPeSmWmU/IcHRVJI8VpDjsXJSDYJ7gBK3P1886y/0098fa7iOmueX jc+xYq4vrBDkBUjOYaFSAmmuWM0LDUquv9j/z4RJDi9lVntNWL2aFIO0lrz6TeWfWE HLAmWYua8dQL3j772kAS860dyIJPEy8+h+ZqrJkg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726389AbgC1Pej (ORCPT ); Sat, 28 Mar 2020 11:34:39 -0400 Received: from mail.kernel.org ([198.145.29.99]:40252 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726482AbgC1Pee (ORCPT ); Sat, 28 Mar 2020 11:34:34 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9B4D52074F for ; Sat, 28 Mar 2020 15:34:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409673; bh=6mPriu0gpOtKj59zFY2FBD+rje0XT523KPDFTm2vWOw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=fS8ndF5pKKxXC9ZMVIqBlrNb2BHZloV3/o9BQTiBUhM6/gIjErPrCJ6IDddGc2tSa g100RZ2oBNMNPdP/PhrgbnJodv/MBf7c8q+EYYo9rcDfwKm2f3b/W7z9kFPYkePIn6 pfXvBhxFNUXWAOPg12sOr31mXzHNM7Oy7z5wqMKk= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 01/22] pNFS/flexfiles: Simplify allocation of the mirror array Date: Sat, 28 Mar 2020 11:31:59 -0400 Message-Id: <20200328153220.1352010-2-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-1-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Just allocate the array at the end of the layout segment structure, instead of allocating it as a separate array of pointers. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 21 +++++---------------- fs/nfs/flexfilelayout/flexfilelayout.h | 2 +- 2 files changed, 6 insertions(+), 17 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index e7d8ae4d0cc5..19728206e9c6 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -248,18 +248,10 @@ static void ff_layout_put_mirror(struct nfs4_ff_layout_mirror *mirror) static void ff_layout_free_mirror_array(struct nfs4_ff_layout_segment *fls) { - int i; + u32 i; - if (fls->mirror_array) { - for (i = 0; i < fls->mirror_array_cnt; i++) { - /* normally mirror_ds is freed in - * .free_deviceid_node but we still do it here - * for .alloc_lseg error path */ - ff_layout_put_mirror(fls->mirror_array[i]); - } - kfree(fls->mirror_array); - fls->mirror_array = NULL; - } + for (i = 0; i < fls->mirror_array_cnt; i++) + ff_layout_put_mirror(fls->mirror_array[i]); } static int ff_layout_check_layout(struct nfs4_layoutget_res *lgr) @@ -400,16 +392,13 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh, goto out_err_free; rc = -ENOMEM; - fls = kzalloc(sizeof(*fls), gfp_flags); + fls = kzalloc(struct_size(fls, mirror_array, mirror_array_cnt), + gfp_flags); if (!fls) goto out_err_free; fls->mirror_array_cnt = mirror_array_cnt; fls->stripe_unit = stripe_unit; - fls->mirror_array = kcalloc(fls->mirror_array_cnt, - sizeof(fls->mirror_array[0]), gfp_flags); - if (fls->mirror_array == NULL) - goto out_err_free; for (i = 0; i < fls->mirror_array_cnt; i++) { struct nfs4_ff_layout_mirror *mirror; diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h index 2f369966abf7..354a031c69b1 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.h +++ b/fs/nfs/flexfilelayout/flexfilelayout.h @@ -99,7 +99,7 @@ struct nfs4_ff_layout_segment { u64 stripe_unit; u32 flags; u32 mirror_array_cnt; - struct nfs4_ff_layout_mirror **mirror_array; + struct nfs4_ff_layout_mirror *mirror_array[]; }; struct nfs4_flexfile_layout { From patchwork Sat Mar 28 15:32:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463585 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0DCA159A for ; Sat, 28 Mar 2020 15:34:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 95D1B20B80 for ; Sat, 28 Mar 2020 15:34:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409679; bh=Ihi7Dit/Q8Thk1RIhBwaphJ7XIANwUrtk7jMQv7GX8s=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=XKb7A6tSuXK7zWfQiDjH8xyHE7oXdBa1tzQZn1n/ucKNHUX8cs/Hr157QxsC/plov 9SSYFmmauaPa2Efugl+KgtiqJ3fnWLSq3TMuqstNKE5umKz0hk36Lu7EO/TlaIm78E JfVNS6W1ygmXzTLe/Mo4gd3oUNacTbcb6lfkfhL8= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727070AbgC1Pef (ORCPT ); Sat, 28 Mar 2020 11:34:35 -0400 Received: from mail.kernel.org ([198.145.29.99]:40258 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727009AbgC1Pee (ORCPT ); Sat, 28 Mar 2020 11:34:34 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0E50E20748 for ; Sat, 28 Mar 2020 15:34:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409674; bh=Ihi7Dit/Q8Thk1RIhBwaphJ7XIANwUrtk7jMQv7GX8s=; h=From:To:Subject:Date:In-Reply-To:References:From; b=aNOQ9tILkQ/oEkBVY6VXe856OcKX38U27IQr7HqlgcH9uE6Vygmfwy/VOvqfn3lLV zSYyQUxWVh1pask+ro2R2dxuRO0a9kmPgXsSR3JRHpqdX63O8WWKA1/tOz2L+L9Kss MTo8BH3oTVOhLk8ZEzfeEpHFHCxozCyfvjcSXcLk= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 02/22] NFS/pNFS: Refactor pnfs_generic_commit_pagelist() Date: Sat, 28 Mar 2020 11:32:00 -0400 Message-Id: <20200328153220.1352010-3-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-2-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Refactor pnfs_generic_commit_pagelist() to simplify the conversion to layout segment based commit lists. Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 164 +++++++++++++++++++--------------------------- fs/nfs/write.c | 13 ++-- 2 files changed, 76 insertions(+), 101 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 8b37e7f8e789..3d0942541618 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -156,103 +156,86 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); -static void pnfs_generic_retry_commit(struct nfs_commit_info *cinfo, int idx) +static struct pnfs_layout_segment * +pnfs_bucket_get_committing(struct list_head *head, + struct pnfs_commit_bucket *bucket, + struct nfs_commit_info *cinfo) { - struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - struct pnfs_commit_bucket *bucket; struct pnfs_layout_segment *freeme; struct list_head *pos; + + list_for_each(pos, &bucket->committing) + cinfo->ds->ncommitting--; + list_splice_init(&bucket->committing, head); + freeme = bucket->clseg; + bucket->clseg = NULL; + return freeme; +} + +static struct nfs_commit_data * +pnfs_bucket_fetch_commitdata(struct pnfs_commit_bucket *bucket, + struct nfs_commit_info *cinfo) +{ + struct nfs_commit_data *data = nfs_commitdata_alloc(false); + + if (!data) + return NULL; + data->lseg = pnfs_bucket_get_committing(&data->pages, bucket, cinfo); + return data; +} + +static void pnfs_generic_retry_commit(struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + struct nfs_commit_info *cinfo, + unsigned int idx) +{ + struct pnfs_commit_bucket *bucket; + struct pnfs_layout_segment *freeme; LIST_HEAD(pages); - int i; - mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); - for (i = idx; i < fl_cinfo->nbuckets; i++) { - bucket = &fl_cinfo->buckets[i]; + for (bucket = buckets; idx < nbuckets; bucket++, idx++) { if (list_empty(&bucket->committing)) continue; - freeme = bucket->clseg; - bucket->clseg = NULL; - list_for_each(pos, &bucket->committing) - cinfo->ds->ncommitting--; - list_splice_init(&bucket->committing, &pages); + mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); + freeme = pnfs_bucket_get_committing(&pages, bucket, cinfo); mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); - nfs_retry_commit(&pages, freeme, cinfo, i); + nfs_retry_commit(&pages, freeme, cinfo, idx); pnfs_put_lseg(freeme); - mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); } - mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); } static unsigned int -pnfs_generic_alloc_ds_commits(struct nfs_commit_info *cinfo, - struct list_head *list) +pnfs_bucket_alloc_ds_commits(struct list_head *list, + struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + struct nfs_commit_info *cinfo) { - struct pnfs_ds_commit_info *fl_cinfo; struct pnfs_commit_bucket *bucket; struct nfs_commit_data *data; - int i; + unsigned int i; unsigned int nreq = 0; - fl_cinfo = cinfo->ds; - bucket = fl_cinfo->buckets; - for (i = 0; i < fl_cinfo->nbuckets; i++, bucket++) { + for (i = 0, bucket = buckets; i < nbuckets; i++, bucket++) { if (list_empty(&bucket->committing)) continue; - data = nfs_commitdata_alloc(false); - if (!data) - break; - data->ds_commit_index = i; - list_add(&data->pages, list); - nreq++; + mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); + if (!list_empty(&bucket->committing)) { + data = pnfs_bucket_fetch_commitdata(bucket, cinfo); + if (!data) + goto out_error; + data->ds_commit_index = i; + list_add_tail(&data->list, list); + atomic_inc(&cinfo->mds->rpcs_out); + nreq++; + } + mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); } - - /* Clean up on error */ - pnfs_generic_retry_commit(cinfo, i); return nreq; -} - -static inline -void pnfs_fetch_commit_bucket_list(struct list_head *pages, - struct nfs_commit_data *data, - struct nfs_commit_info *cinfo) -{ - struct pnfs_commit_bucket *bucket; - struct list_head *pos; - - bucket = &cinfo->ds->buckets[data->ds_commit_index]; - mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); - list_for_each(pos, &bucket->committing) - cinfo->ds->ncommitting--; - list_splice_init(&bucket->committing, pages); - data->lseg = bucket->clseg; - bucket->clseg = NULL; +out_error: mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); - -} - -/* Helper function for pnfs_generic_commit_pagelist to catch an empty - * page list. This can happen when two commits race. - * - * This must be called instead of nfs_init_commit - call one or the other, but - * not both! - */ -static bool -pnfs_generic_commit_cancel_empty_pagelist(struct list_head *pages, - struct nfs_commit_data *data, - struct nfs_commit_info *cinfo) -{ - if (list_empty(pages)) { - if (atomic_dec_and_test(&cinfo->mds->rpcs_out)) - wake_up_var(&cinfo->mds->rpcs_out); - /* don't call nfs_commitdata_release - it tries to put - * the open_context which is not acquired until nfs_init_commit - * which has not been called on @data */ - WARN_ON_ONCE(data->context); - nfs_commit_free(data); - return true; - } - - return false; + /* Clean up on error */ + pnfs_generic_retry_commit(buckets, nbuckets, cinfo, i); + return nreq; } /* This follows nfs_commit_list pretty closely */ @@ -262,6 +245,7 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, int (*initiate_commit)(struct nfs_commit_data *data, int how)) { + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; struct nfs_commit_data *data, *tmp; LIST_HEAD(list); unsigned int nreq = 0; @@ -269,40 +253,26 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, if (!list_empty(mds_pages)) { data = nfs_commitdata_alloc(true); data->ds_commit_index = -1; - list_add(&data->pages, &list); + list_splice_init(mds_pages, &data->pages); + list_add_tail(&data->list, &list); + atomic_inc(&cinfo->mds->rpcs_out); nreq++; } - nreq += pnfs_generic_alloc_ds_commits(cinfo, &list); - + nreq += pnfs_bucket_alloc_ds_commits(&list, fl_cinfo->buckets, + fl_cinfo->nbuckets, cinfo); if (nreq == 0) goto out; - atomic_add(nreq, &cinfo->mds->rpcs_out); - - list_for_each_entry_safe(data, tmp, &list, pages) { - list_del_init(&data->pages); + list_for_each_entry_safe(data, tmp, &list, list) { + list_del(&data->list); if (data->ds_commit_index < 0) { - /* another commit raced with us */ - if (pnfs_generic_commit_cancel_empty_pagelist(mds_pages, - data, cinfo)) - continue; - - nfs_init_commit(data, mds_pages, NULL, cinfo); + nfs_init_commit(data, NULL, NULL, cinfo); nfs_initiate_commit(NFS_CLIENT(inode), data, NFS_PROTO(data->inode), data->mds_ops, how, 0); } else { - LIST_HEAD(pages); - - pnfs_fetch_commit_bucket_list(&pages, data, cinfo); - - /* another commit raced with us */ - if (pnfs_generic_commit_cancel_empty_pagelist(&pages, - data, cinfo)) - continue; - - nfs_init_commit(data, &pages, data->lseg, cinfo); + nfs_init_commit(data, NULL, data->lseg, cinfo); initiate_commit(data, how); } } diff --git a/fs/nfs/write.c b/fs/nfs/write.c index 5544ee6cfda8..1f8108f5a041 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -1746,14 +1746,19 @@ void nfs_init_commit(struct nfs_commit_data *data, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo) { - struct nfs_page *first = nfs_list_entry(head->next); - struct nfs_open_context *ctx = nfs_req_openctx(first); - struct inode *inode = d_inode(ctx->dentry); + struct nfs_page *first; + struct nfs_open_context *ctx; + struct inode *inode; /* Set up the RPC argument and reply structs * NB: take care not to mess about with data->commit et al. */ - list_splice_init(head, &data->pages); + if (head) + list_splice_init(head, &data->pages); + + first = nfs_list_entry(data->pages.next); + ctx = nfs_req_openctx(first); + inode = d_inode(ctx->dentry); data->inode = inode; data->cred = ctx->cred; From patchwork Sat Mar 28 15:32:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463587 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB29592A for ; Sat, 28 Mar 2020 15:34:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CABF82073B for ; Sat, 28 Mar 2020 15:34:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409679; bh=ogmst/gQoqoMT45GiaXzThYRPK322Tm06zT3I5HCDIU=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=p1CUzteyZ9VMSkey5adtL0TDZOwsnL8n4iFZe2tCXni48wy2t3idNYBvoY9VMBrzU h8izFQcX5oH/0q9S9I3ukSNavAhQ/+caWin7ORb+5JwjDnoDkZU+r9PovKbwtjXJJO 9g7BOK7rWmP6eij2Yv4sEwpampQuJxRyNiTIINT0= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727118AbgC1Pef (ORCPT ); Sat, 28 Mar 2020 11:34:35 -0400 Received: from mail.kernel.org ([198.145.29.99]:40264 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725807AbgC1Pef (ORCPT ); Sat, 28 Mar 2020 11:34:35 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 7CB1A207FF for ; Sat, 28 Mar 2020 15:34:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409674; bh=ogmst/gQoqoMT45GiaXzThYRPK322Tm06zT3I5HCDIU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Z6v5ysbpMoGb4+Dmcl9OrrhJnhyXQh4Z8u1m0L/BmF/w7P1xK/iS6die9B7iC6FqM wd9XopnC5uXuShXVNw3I2pbOiobPcPpMbIKhiNM8yNUCFgODjhjmhgsYsV+cUE6EKD AT3cnhN15113I0sn9CKWbSXO6eYCD3mALm62o/tA= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 03/22] pNFS: Add a helper to allocate the array of buckets Date: Sat, 28 Mar 2020 11:32:01 -0400 Message-Id: <20200328153220.1352010-4-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-3-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Signed-off-by: Trond Myklebust --- fs/nfs/pnfs.h | 3 +++ fs/nfs/pnfs_nfs.c | 31 +++++++++++++++++++++++++++++++ include/linux/nfs_xdr.h | 15 ++++++++++++--- 3 files changed, 46 insertions(+), 3 deletions(-) diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 7bfb6970134a..f6b1099aa151 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -366,6 +366,9 @@ bool nfs4_test_deviceid_unavailable(struct nfs4_deviceid_node *node); void nfs4_deviceid_purge_client(const struct nfs_client *); /* pnfs_nfs.c */ +struct pnfs_commit_array *pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags); +void pnfs_free_commit_array(struct pnfs_commit_array *p); + void pnfs_generic_clear_request_commit(struct nfs_page *req, struct nfs_commit_info *cinfo); void pnfs_generic_commit_release(void *calldata); diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 3d0942541618..c8518ce3a4ef 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -87,6 +87,37 @@ pnfs_generic_clear_request_commit(struct nfs_page *req, } EXPORT_SYMBOL_GPL(pnfs_generic_clear_request_commit); +struct pnfs_commit_array * +pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags) +{ + struct pnfs_commit_array *p; + struct pnfs_commit_bucket *b; + + p = kmalloc(struct_size(p, buckets, n), gfp_flags); + if (!p) + return NULL; + p->nbuckets = n; + INIT_LIST_HEAD(&p->cinfo_list); + INIT_LIST_HEAD(&p->lseg_list); + p->lseg = NULL; + for (b = &p->buckets[0]; n != 0; b++, n--) { + INIT_LIST_HEAD(&b->written); + INIT_LIST_HEAD(&b->committing); + b->wlseg = NULL; + b->clseg = NULL; + b->direct_verf.committed = NFS_INVALID_STABLE_HOW; + } + return p; +} +EXPORT_SYMBOL_GPL(pnfs_alloc_commit_array); + +void +pnfs_free_commit_array(struct pnfs_commit_array *p) +{ + kfree_rcu(p, rcu); +} +EXPORT_SYMBOL_GPL(pnfs_free_commit_array); + static int pnfs_generic_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, struct nfs_commit_info *cinfo, diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 94c77ed55ce1..e91c917c9c1c 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1270,10 +1270,19 @@ struct pnfs_commit_bucket { struct nfs_writeverf direct_verf; }; +struct pnfs_commit_array { + struct list_head cinfo_list; + struct list_head lseg_list; + struct pnfs_layout_segment *lseg; + struct rcu_head rcu; + unsigned int nbuckets; + struct pnfs_commit_bucket buckets[]; +}; + struct pnfs_ds_commit_info { - int nwritten; - int ncommitting; - int nbuckets; + unsigned int nwritten; + unsigned int ncommitting; + unsigned int nbuckets; struct pnfs_commit_bucket *buckets; }; From patchwork Sat Mar 28 15:32:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C3111667 for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 01A542082D for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=Typ3nTzxTQRHi7G9/zete+2FBPRAhEJYSVRiAt1mO9k=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=IlVXghekeWXci14/Q1tPY/zFIutoaP6eNZqwb7GvNMs6A2SJegp1fG8gXCX5n1mrz L07qSK4XxU3JANZXmd0d7ztdo+gNNVesahgWlRO7ZD3H8/8pKWPoXE4M1gHOzsUnew 8HRHKUix7+wqNrXfHzwdDA8OxXcBsg/6wXpwfZQg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725807AbgC1Peg (ORCPT ); Sat, 28 Mar 2020 11:34:36 -0400 Received: from mail.kernel.org ([198.145.29.99]:40258 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727009AbgC1Pef (ORCPT ); Sat, 28 Mar 2020 11:34:35 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E6E0A2073B for ; Sat, 28 Mar 2020 15:34:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409675; bh=Typ3nTzxTQRHi7G9/zete+2FBPRAhEJYSVRiAt1mO9k=; h=From:To:Subject:Date:In-Reply-To:References:From; b=r4yWllS1pSCIUw5Xn0Ks/iaD8e2tEuxlruBv6OQOnpS9YOIjc6U+VeGQ9Dk5NB7we EBIp9H07ieEnWL3fonF9K3qwhJh79o9G73cvBWdHM7yti8CeOw0P+BNetSBdtfKVlM 5rt/uImUdMmhatnKhuWaDYm7GM2GRDLXiW0Q6QW0= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 04/22] NFSv4/pnfs: Support a list of commit arrays in struct pnfs_ds_commit_info Date: Sat, 28 Mar 2020 11:32:02 -0400 Message-Id: <20200328153220.1352010-5-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-4-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust When we have multiple layout segments with different lists of mirrored data, we need to track the commits on a per layout segment basis. This patch adds a list to support this tracking in struct pnfs_ds_commit_info. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 1 + fs/nfs/filelayout/filelayout.c | 5 ++++- fs/nfs/flexfilelayout/flexfilelayout.c | 1 + fs/nfs/pnfs.h | 11 +++++++++++ include/linux/nfs_xdr.h | 1 + 5 files changed, 18 insertions(+), 1 deletion(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index ade2435551c8..f9a73febce02 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -305,6 +305,7 @@ static inline struct nfs_direct_req *nfs_direct_req_alloc(void) kref_get(&dreq->kref); init_completion(&dreq->completion); INIT_LIST_HEAD(&dreq->mds_cinfo.list); + pnfs_init_ds_commit_info(&dreq->ds_cinfo); dreq->verf.committed = NFS_INVALID_STABLE_HOW; /* not set yet */ INIT_WORK(&dreq->work, nfs_direct_write_schedule_work); spin_lock_init(&dreq->lock); diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index bd234394a87c..b051d5d320ba 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -1140,7 +1140,10 @@ filelayout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) struct nfs4_filelayout *flo; flo = kzalloc(sizeof(*flo), gfp_flags); - return flo != NULL ? &flo->generic_hdr : NULL; + if (flo == NULL) + return NULL; + pnfs_init_ds_commit_info(&flo->commit_info); + return &flo->generic_hdr; } static void diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 19728206e9c6..c9e79c8e62cd 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -48,6 +48,7 @@ ff_layout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) ffl = kzalloc(sizeof(*ffl), gfp_flags); if (ffl) { + pnfs_init_ds_commit_info(&ffl->commit_info); INIT_LIST_HEAD(&ffl->error_list); INIT_LIST_HEAD(&ffl->mirrors); ffl->last_report_time = ktime_get(); diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index f6b1099aa151..b293afb48d04 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -462,6 +462,12 @@ pnfs_get_ds_info(struct inode *inode) return ld->get_ds_info(inode); } +static inline void +pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo) +{ + INIT_LIST_HEAD(&fl_cinfo->commits); +} + static inline void pnfs_generic_mark_devid_invalid(struct nfs4_deviceid_node *node) { @@ -759,6 +765,11 @@ pnfs_get_ds_info(struct inode *inode) return NULL; } +static inline void +pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo) +{ +} + static inline bool pnfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, u32 ds_commit_idx) diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index e91c917c9c1c..9946787eda72 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1280,6 +1280,7 @@ struct pnfs_commit_array { }; struct pnfs_ds_commit_info { + struct list_head commits; unsigned int nwritten; unsigned int ncommitting; unsigned int nbuckets; From patchwork Sat Mar 28 15:32:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56AA3161F for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 366102082D for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=asZPu00uS55iLq65hnYce26g4HNvsBAKVZEYJOk+hl8=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=0x959RL/Y890h0i1wiJTRvy3iZBKbyik5pAVOhUItcafPfkJ6MB7DpaLTDIAsNGRV zsb3E+VOJJU6ZAxG1pc7aec4eZrrxhfj439kuMvp5ik0BNeSAZxFTwvNz3bR1k/RLd dCiGF1pui7lpdoEPKCW0ZyggqrfQLoFi1XQpjhxA= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727144AbgC1Peg (ORCPT ); Sat, 28 Mar 2020 11:34:36 -0400 Received: from mail.kernel.org ([198.145.29.99]:40264 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727131AbgC1Pef (ORCPT ); Sat, 28 Mar 2020 11:34:35 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5E8C620748 for ; Sat, 28 Mar 2020 15:34:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409675; bh=asZPu00uS55iLq65hnYce26g4HNvsBAKVZEYJOk+hl8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Pz1yZ68U8xAyRvM9UxRvyhqe6tcfbSgmCpLMMayiv5uoupBMuSJzD8XdgDZvczuuw aitY4FdLYHB3MXvHnEFdASP8+GTmFHV0IdmbbyWEjwlIM9UG3qp5fqFnmV2jTknET4 aKBtiM46pOGlOX7ZtSXqsaQ3hkUis01qjVyEiwC0= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 05/22] NFSv4/pNFS: Scan the full list of commit arrays when committing Date: Sat, 28 Mar 2020 11:32:03 -0400 Message-Id: <20200328153220.1352010-6-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-5-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to pnfs_generic_scan_commit_lists() Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 52 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 40 insertions(+), 12 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index c8518ce3a4ef..81fd85e66fd9 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -118,10 +118,14 @@ pnfs_free_commit_array(struct pnfs_commit_array *p) } EXPORT_SYMBOL_GPL(pnfs_free_commit_array); +/* + * Locks the nfs_page requests for commit and moves them to + * @bucket->committing. + */ static int -pnfs_generic_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, - struct nfs_commit_info *cinfo, - int max) +pnfs_bucket_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, + struct nfs_commit_info *cinfo, + int max) { struct list_head *src = &bucket->written; struct list_head *dst = &bucket->committing; @@ -142,20 +146,44 @@ pnfs_generic_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, return ret; } +static int pnfs_bucket_scan_array(struct nfs_commit_info *cinfo, + struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + int max) +{ + unsigned int i; + int rv = 0, cnt; + + for (i = 0; i < nbuckets && max != 0; i++) { + cnt = pnfs_bucket_scan_ds_commit_list(&buckets[i], cinfo, max); + rv += cnt; + max -= cnt; + } + return rv; +} + /* Move reqs from written to committing lists, returning count * of number moved. */ -int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, - int max) +int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) { - int i, rv = 0, cnt; - - lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); - for (i = 0; i < cinfo->ds->nbuckets && max != 0; i++) { - cnt = pnfs_generic_scan_ds_commit_list(&cinfo->ds->buckets[i], - cinfo, max); - max -= cnt; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + struct pnfs_commit_array *array; + int rv = 0, cnt; + + cnt = pnfs_bucket_scan_array(cinfo, fl_cinfo->buckets, + fl_cinfo->nbuckets, max); + rv += cnt; + max -= cnt; + if (!max) + return rv; + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + cnt = pnfs_bucket_scan_array(cinfo, array->buckets, + array->nbuckets, max); rv += cnt; + max -= cnt; + if (!max) + break; } return rv; } From patchwork Sat Mar 28 15:32:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463593 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8151B18A5 for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 60C3C2082E for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=ksX41en9zXztZkK7yJUP1w30ssz69SdnHLr2M+BntI0=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=F50B3KjoKzV0AtOMs6gb/hhAFegQu5QmG7x02YkpoC4VI+g3X51AipVfYx1s/185g ZdbfNod67a7tctV+uuutvmnlG9zxlKaRavjgnnkbVZOZAIyS3o0XFq8vO4PNIH7UDS S1xnSR9SKosLZDllWJXuN3gHh19j+2SA1pfwH/bo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727148AbgC1Peh (ORCPT ); Sat, 28 Mar 2020 11:34:37 -0400 Received: from mail.kernel.org ([198.145.29.99]:40298 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727009AbgC1Peg (ORCPT ); Sat, 28 Mar 2020 11:34:36 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C81892082D for ; Sat, 28 Mar 2020 15:34:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409676; bh=ksX41en9zXztZkK7yJUP1w30ssz69SdnHLr2M+BntI0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=LimI/DbaWwBXBgrHKc5Owta7Jo537FiSr0epkXCcOA9Y54NEaYV8zIKr11HSMmmL+ dw0ME6x2Jbvi4mHYt4HBnFA+Za+5zEbGteQErU/8yG0RWapHXPYmqswIkkZOYexRYN GHvDG7tbnRxG9SWuGkZfCmQQ/1aDSK8hX7Uc4Blw= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 06/22] pNFS: Support per-layout segment commits in pnfs_generic_recover_commit_reqs() Date: Sat, 28 Mar 2020 11:32:04 -0400 Message-Id: <20200328153220.1352010-7-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-6-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to pnfs_generic_recover_commit_reqs(). Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 41 +++++++++++++++++++++++++++++++++-------- 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 81fd85e66fd9..f16bd6d0e830 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -189,22 +189,23 @@ int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) } EXPORT_SYMBOL_GPL(pnfs_generic_scan_commit_lists); -/* Pull everything off the committing lists and dump into @dst. */ -void pnfs_generic_recover_commit_reqs(struct list_head *dst, - struct nfs_commit_info *cinfo) +static unsigned int +pnfs_bucket_recover_commit_reqs(struct list_head *dst, + struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, + struct nfs_commit_info *cinfo) { struct pnfs_commit_bucket *b; struct pnfs_layout_segment *freeme; - int nwritten; - int i; + unsigned int nwritten, ret = 0; + unsigned int i; - lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); restart: - for (i = 0, b = cinfo->ds->buckets; i < cinfo->ds->nbuckets; i++, b++) { + for (i = 0, b = buckets; i < nbuckets; i++, b++) { nwritten = nfs_scan_commit_list(&b->written, dst, cinfo, 0); if (!nwritten) continue; - cinfo->ds->nwritten -= nwritten; + ret += nwritten; if (list_empty(&b->written)) { freeme = b->wlseg; b->wlseg = NULL; @@ -212,6 +213,30 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, goto restart; } } + return ret; +} + +/* Pull everything off the committing lists and dump into @dst. */ +void pnfs_generic_recover_commit_reqs(struct list_head *dst, + struct nfs_commit_info *cinfo) +{ + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + struct pnfs_commit_array *array; + unsigned int nwritten; + + lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); + nwritten = pnfs_bucket_recover_commit_reqs(dst, + fl_cinfo->buckets, + fl_cinfo->nbuckets, + cinfo); + fl_cinfo->nwritten -= nwritten; + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + nwritten = pnfs_bucket_recover_commit_reqs(dst, + array->buckets, + array->nbuckets, + cinfo); + fl_cinfo->nwritten -= nwritten; + } } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); From patchwork Sat Mar 28 15:32:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463595 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B526F92A for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8B7162073B for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=7EnajCoY61wyaI04/xGUaouttgnSwwfGTe62JcRBxjE=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=T/xZo49Ef9dX5o2MTKU5vUP8IvhSNYlMd/n+ApPL1USQeuPTrwNDkOsUYDKXrc4lP I1Dyy+5mAUDZ/SUDCYibJm5JK6y2sJ7GSwAsP756sB1t5hr2HQsYJP2KEsQD7/cPqQ UQJao3WaciWsJw/bybRQq7nXx7dc7ViZc6IyzoDI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727009AbgC1Peh (ORCPT ); Sat, 28 Mar 2020 11:34:37 -0400 Received: from mail.kernel.org ([198.145.29.99]:40304 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727131AbgC1Peg (ORCPT ); Sat, 28 Mar 2020 11:34:36 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 48CFA2073B for ; Sat, 28 Mar 2020 15:34:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409676; bh=7EnajCoY61wyaI04/xGUaouttgnSwwfGTe62JcRBxjE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=MrjiTozTD0XEvIu+XErgUb/KqTrZ+ZprIxMDX7SQpvaujukGiKTKijDSHgmP0v8wW gWtsYUM4bGi7g+Qsq8pAb0fGa1rT4DHbABEfXagtXqggyRmMavyvC21j4XobpY+QVo 5uUnMu2Q7kJd1Wpc46LIRumSNcZhhuXRvp10tGwY= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 07/22] pNFS: Support per-layout segment commits in pnfs_generic_commit_pagelist() Date: Sat, 28 Mar 2020 11:32:05 -0400 Message-Id: <20200328153220.1352010-8-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-7-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to pnfs_generic_commit_pagelist(). Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index f16bd6d0e830..f895a28b1e26 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -322,6 +322,20 @@ pnfs_bucket_alloc_ds_commits(struct list_head *list, return nreq; } +static unsigned int +pnfs_alloc_ds_commits_list(struct list_head *list, + struct pnfs_ds_commit_info *fl_cinfo, + struct nfs_commit_info *cinfo) +{ + struct pnfs_commit_array *array; + unsigned int ret = 0; + + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) + ret += pnfs_bucket_alloc_ds_commits(list, array->buckets, + array->nbuckets, cinfo); + return ret; +} + /* This follows nfs_commit_list pretty closely */ int pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, @@ -345,6 +359,8 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, nreq += pnfs_bucket_alloc_ds_commits(&list, fl_cinfo->buckets, fl_cinfo->nbuckets, cinfo); + + nreq += pnfs_alloc_ds_commits_list(&list, fl_cinfo, cinfo); if (nreq == 0) goto out; From patchwork Sat Mar 28 15:32:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463597 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0FCF159A for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BF3DA2082E for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=vutH3/YReiNFiYTDSRW2/u7raVAoGoT4cTuq6TCiAtk=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=D3okQ8lC5CK4AtcPWQn0aQMDZEv1oWKOyVAsYQFoMP8vZ1fMP03B0jN2yegfhbDdv xqw+LDQWPLv/C/SlFWkdtVQ9V/D6f7+E3cNWWdGASuG73wkDWikHmbzSldC1FsxjS5 GPl5HYYTUOzLE72PEGP+rQJgLD69Cb4T6OMK8OWc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727167AbgC1Pei (ORCPT ); Sat, 28 Mar 2020 11:34:38 -0400 Received: from mail.kernel.org ([198.145.29.99]:40310 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726389AbgC1Peh (ORCPT ); Sat, 28 Mar 2020 11:34:37 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B13FA20748 for ; Sat, 28 Mar 2020 15:34:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409676; bh=vutH3/YReiNFiYTDSRW2/u7raVAoGoT4cTuq6TCiAtk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=miqghEpXJ8w7AZRyxyLvBN6eK0c68MB30GvrswnnsQZy9clekwsNkLrLqPrVAHzFt Fc6ikZD/wEfKgB74Am8OYSBRKAaS+Xe4LC7gCVex24mYqLif+M0aPo8QdnE+3qH+XN +btv3g0wT9aoQOMEw7u3Q2cNEtNluBF4aA2INy24= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 08/22] NFS/pNFS: Allow O_DIRECT to release the DS commitinfo Date: Sat, 28 Mar 2020 11:32:06 -0400 Message-Id: <20200328153220.1352010-9-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-8-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add a pNFS callback to allow the O_DIRECT code to release the DS commitinfo when freeing the dreq. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 1 + fs/nfs/pnfs.h | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index f9a73febce02..7ef7f71ae315 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -317,6 +317,7 @@ static void nfs_direct_req_free(struct kref *kref) { struct nfs_direct_req *dreq = container_of(kref, struct nfs_direct_req, kref); + pnfs_release_ds_info(&dreq->ds_cinfo, dreq->inode); nfs_free_pnfs_ds_cinfo(&dreq->ds_cinfo); if (dreq->l_ctx != NULL) nfs_put_lock_context(dreq->l_ctx); diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index b293afb48d04..2ec97b419b56 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -149,6 +149,8 @@ struct pnfs_layoutdriver_type { const struct nfs_pageio_ops *pg_write_ops; struct pnfs_ds_commit_info *(*get_ds_info) (struct inode *inode); + void (*release_ds_info)(struct pnfs_ds_commit_info *, + struct inode *inode); void (*mark_request_commit) (struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, @@ -468,6 +470,15 @@ pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo) INIT_LIST_HEAD(&fl_cinfo->commits); } +static inline void +pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ + struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + + if (ld != NULL && ld->release_ds_info != NULL) + ld->release_ds_info(fl_cinfo, inode); +} + static inline void pnfs_generic_mark_devid_invalid(struct nfs4_deviceid_node *node) { @@ -770,6 +781,11 @@ pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo) { } +static inline void +pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ +} + static inline bool pnfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, u32 ds_commit_idx) From patchwork Sat Mar 28 15:32:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 756BE92A for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 54A4020748 for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409681; bh=CJ8DY17KMdhfCO9nyiXNNd4crXhjDpelxBt0kju1zOw=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=TeHkQHCIcvWCdJdPLgw07Uoz55H2plDmr4Pkr6wbTJXG4NWPc39qHnSncPJLlmCgH JsgAIu3Mq3kWctzOIwk5qsTMDrnboG4Gsdf2wOlpw3ovE25BzXKdLZKAsoaQcIlnaU G6mfX5InNDQULTDYWV6492Nvc+5D3MR0CcbTSoPk= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726518AbgC1Pej (ORCPT ); Sat, 28 Mar 2020 11:34:39 -0400 Received: from mail.kernel.org ([198.145.29.99]:40320 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727131AbgC1Pei (ORCPT ); Sat, 28 Mar 2020 11:34:38 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 267032082D for ; Sat, 28 Mar 2020 15:34:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409677; bh=CJ8DY17KMdhfCO9nyiXNNd4crXhjDpelxBt0kju1zOw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=1qsdctqpXtSB9W6+jRdk6bVH1DmPnzSmRMEeb1g9UOCpfsua5Xc8ewaDyn12EHLNm 9Q2r9yq/GSSooFABAteKwA/q5hFJNh4a7DFtERR7flQiP+rYoik1ESEwpk0peeu3pI A+QH4f9hzmEVORuX2KNLGucuQnHP1Ao7HRPNyDSA= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 09/22] NFS: commit errors should be fatal Date: Sat, 28 Mar 2020 11:32:07 -0400 Message-Id: <20200328153220.1352010-10-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-9-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Fix the O_DIRECT code to avoid retries if the COMMIT fails with a fatal error. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 7ef7f71ae315..f7bf1181b690 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -94,6 +94,7 @@ struct nfs_direct_req { #define NFS_ODIRECT_RESCHED_WRITES (2) /* write verification failed */ /* for read */ #define NFS_ODIRECT_SHOULD_DIRTY (3) /* dirty user-space page after read */ +#define NFS_ODIRECT_DONE INT_MAX /* write verification failed */ struct nfs_writeverf verf; /* unstable write verifier */ }; @@ -678,8 +679,17 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) struct nfs_page *req; int status = data->task.tk_status; + if (status < 0) { + /* Errors in commit are fatal */ + dreq->error = status; + dreq->max_count = 0; + dreq->count = 0; + dreq->flags = NFS_ODIRECT_DONE; + } else if (dreq->flags == NFS_ODIRECT_DONE) + status = dreq->error; + nfs_init_cinfo_from_dreq(&cinfo, dreq); - if (status < 0 || nfs_direct_cmp_commit_data_verf(dreq, data)) + if (nfs_direct_cmp_commit_data_verf(dreq, data)) dreq->flags = NFS_ODIRECT_RESCHED_WRITES; while (!list_empty(&data->pages)) { @@ -708,7 +718,8 @@ static void nfs_direct_resched_write(struct nfs_commit_info *cinfo, struct nfs_direct_req *dreq = cinfo->dreq; spin_lock(&dreq->lock); - dreq->flags = NFS_ODIRECT_RESCHED_WRITES; + if (dreq->flags != NFS_ODIRECT_DONE) + dreq->flags = NFS_ODIRECT_RESCHED_WRITES; spin_unlock(&dreq->lock); nfs_mark_request_commit(req, NULL, cinfo, 0); } @@ -731,6 +742,22 @@ static void nfs_direct_commit_schedule(struct nfs_direct_req *dreq) nfs_direct_write_reschedule(dreq); } +static void nfs_direct_write_clear_reqs(struct nfs_direct_req *dreq) +{ + struct nfs_commit_info cinfo; + struct nfs_page *req; + LIST_HEAD(reqs); + + nfs_init_cinfo_from_dreq(&cinfo, dreq); + nfs_direct_write_scan_commit_list(dreq->inode, &reqs, &cinfo); + + while (!list_empty(&reqs)) { + req = nfs_list_entry(reqs.next); + nfs_list_remove_request(req); + nfs_unlock_and_release_request(req); + } +} + static void nfs_direct_write_schedule_work(struct work_struct *work) { struct nfs_direct_req *dreq = container_of(work, struct nfs_direct_req, work); @@ -745,6 +772,7 @@ static void nfs_direct_write_schedule_work(struct work_struct *work) nfs_direct_write_reschedule(dreq); break; default: + nfs_direct_write_clear_reqs(dreq); nfs_zap_mapping(dreq->inode, dreq->inode->i_mapping); nfs_direct_complete(dreq); } From patchwork Sat Mar 28 15:32:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463627 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87241161F for ; Sat, 28 Mar 2020 15:34:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6747220748 for ; Sat, 28 Mar 2020 15:34:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409687; bh=4UKHW+bceR/NVQzUHY3TZwKQ1VBLWQyS8R7IzFwQMr8=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=bfvIBN5rnX3FfoVv6cMFO8juIVm6fKGz6hnpC7P3awkhspmRUZoK4W4yarZyp58LC 8uJzwU+lZSB79j5hCE4du9+OvNFKCkIyw7mAMn0XlrIOmTKum3snnWgZ9++ZfQK5N1 5cnKS211RlRKrpCX26bPmRcmMQoONy3HjqtmCiQo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727131AbgC1Peq (ORCPT ); Sat, 28 Mar 2020 11:34:46 -0400 Received: from mail.kernel.org ([198.145.29.99]:40328 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727163AbgC1Pei (ORCPT ); Sat, 28 Mar 2020 11:34:38 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8F0792073B for ; Sat, 28 Mar 2020 15:34:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409677; bh=4UKHW+bceR/NVQzUHY3TZwKQ1VBLWQyS8R7IzFwQMr8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=ofJHEV19qLdVjx1Red+FWOGbdPgGqvLBhtmD/HOpX/dgTye41hselAAPxni9Mn43g j4jkbdf8P8m5sQhHOCuZrcUA44C7TecyJhr1qDzsVUz57K0nuF74IrjtZSyd7qVOVM +SoarJ03rLU+65P3FQ1mc4t4wPQEN08H1+qi0kyw= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 10/22] NFS: Fix O_DIRECT commit verifier handling Date: Sat, 28 Mar 2020 11:32:08 -0400 Message-Id: <20200328153220.1352010-11-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-10-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Instead of trying to save the commit verifiers and checking them against previous writes, adopt the same strategy as for buffered writes, of just checking the verifiers at commit time. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 135 +++++----------------------------------------- fs/nfs/internal.h | 8 +++ fs/nfs/write.c | 3 +- 3 files changed, 22 insertions(+), 124 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index f7bf1181b690..4ee26465b510 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -95,7 +95,6 @@ struct nfs_direct_req { /* for read */ #define NFS_ODIRECT_SHOULD_DIRTY (3) /* dirty user-space page after read */ #define NFS_ODIRECT_DONE INT_MAX /* write verification failed */ - struct nfs_writeverf verf; /* unstable write verifier */ }; static const struct nfs_pgio_completion_ops nfs_direct_write_completion_ops; @@ -152,106 +151,6 @@ nfs_direct_count_bytes(struct nfs_direct_req *dreq, dreq->count = dreq_len; } -/* - * nfs_direct_select_verf - select the right verifier - * @dreq - direct request possibly spanning multiple servers - * @ds_clp - nfs_client of data server or NULL if MDS / non-pnfs - * @commit_idx - commit bucket index for the DS - * - * returns the correct verifier to use given the role of the server - */ -static struct nfs_writeverf * -nfs_direct_select_verf(struct nfs_direct_req *dreq, - struct nfs_client *ds_clp, - int commit_idx) -{ - struct nfs_writeverf *verfp = &dreq->verf; - -#ifdef CONFIG_NFS_V4_1 - /* - * pNFS is in use, use the DS verf except commit_through_mds is set - * for layout segment where nbuckets is zero. - */ - if (ds_clp && dreq->ds_cinfo.nbuckets > 0) { - if (commit_idx >= 0 && commit_idx < dreq->ds_cinfo.nbuckets) - verfp = &dreq->ds_cinfo.buckets[commit_idx].direct_verf; - else - WARN_ON_ONCE(1); - } -#endif - return verfp; -} - - -/* - * nfs_direct_set_hdr_verf - set the write/commit verifier - * @dreq - direct request possibly spanning multiple servers - * @hdr - pageio header to validate against previously seen verfs - * - * Set the server's (MDS or DS) "seen" verifier - */ -static void nfs_direct_set_hdr_verf(struct nfs_direct_req *dreq, - struct nfs_pgio_header *hdr) -{ - struct nfs_writeverf *verfp; - - verfp = nfs_direct_select_verf(dreq, hdr->ds_clp, hdr->ds_commit_idx); - WARN_ON_ONCE(verfp->committed >= 0); - memcpy(verfp, &hdr->verf, sizeof(struct nfs_writeverf)); - WARN_ON_ONCE(verfp->committed < 0); -} - -static int nfs_direct_cmp_verf(const struct nfs_writeverf *v1, - const struct nfs_writeverf *v2) -{ - return nfs_write_verifier_cmp(&v1->verifier, &v2->verifier); -} - -/* - * nfs_direct_cmp_hdr_verf - compare verifier for pgio header - * @dreq - direct request possibly spanning multiple servers - * @hdr - pageio header to validate against previously seen verf - * - * set the server's "seen" verf if not initialized. - * returns result of comparison between @hdr->verf and the "seen" - * verf of the server used by @hdr (DS or MDS) - */ -static int nfs_direct_set_or_cmp_hdr_verf(struct nfs_direct_req *dreq, - struct nfs_pgio_header *hdr) -{ - struct nfs_writeverf *verfp; - - verfp = nfs_direct_select_verf(dreq, hdr->ds_clp, hdr->ds_commit_idx); - if (verfp->committed < 0) { - nfs_direct_set_hdr_verf(dreq, hdr); - return 0; - } - return nfs_direct_cmp_verf(verfp, &hdr->verf); -} - -/* - * nfs_direct_cmp_commit_data_verf - compare verifier for commit data - * @dreq - direct request possibly spanning multiple servers - * @data - commit data to validate against previously seen verf - * - * returns result of comparison between @data->verf and the verf of - * the server used by @data (DS or MDS) - */ -static int nfs_direct_cmp_commit_data_verf(struct nfs_direct_req *dreq, - struct nfs_commit_data *data) -{ - struct nfs_writeverf *verfp; - - verfp = nfs_direct_select_verf(dreq, data->ds_clp, - data->ds_commit_index); - - /* verifier not set so always fail */ - if (verfp->committed < 0 || data->res.verf->committed <= NFS_UNSTABLE) - return 1; - - return nfs_direct_cmp_verf(verfp, data->res.verf); -} - /** * nfs_direct_IO - NFS address space operation for direct I/O * @iocb: target I/O control block @@ -307,7 +206,6 @@ static inline struct nfs_direct_req *nfs_direct_req_alloc(void) init_completion(&dreq->completion); INIT_LIST_HEAD(&dreq->mds_cinfo.list); pnfs_init_ds_commit_info(&dreq->ds_cinfo); - dreq->verf.committed = NFS_INVALID_STABLE_HOW; /* not set yet */ INIT_WORK(&dreq->work, nfs_direct_write_schedule_work); spin_lock_init(&dreq->lock); @@ -637,7 +535,6 @@ static void nfs_direct_write_reschedule(struct nfs_direct_req *dreq) dreq->max_count = 0; list_for_each_entry(req, &reqs, wb_list) dreq->max_count += req->wb_bytes; - dreq->verf.committed = NFS_INVALID_STABLE_HOW; nfs_clear_pnfs_ds_commit_verifiers(&dreq->ds_cinfo); get_dreq(dreq); @@ -674,6 +571,7 @@ static void nfs_direct_write_reschedule(struct nfs_direct_req *dreq) static void nfs_direct_commit_complete(struct nfs_commit_data *data) { + const struct nfs_writeverf *verf = data->res.verf; struct nfs_direct_req *dreq = data->dreq; struct nfs_commit_info cinfo; struct nfs_page *req; @@ -689,21 +587,19 @@ static void nfs_direct_commit_complete(struct nfs_commit_data *data) status = dreq->error; nfs_init_cinfo_from_dreq(&cinfo, dreq); - if (nfs_direct_cmp_commit_data_verf(dreq, data)) - dreq->flags = NFS_ODIRECT_RESCHED_WRITES; while (!list_empty(&data->pages)) { req = nfs_list_entry(data->pages.next); nfs_list_remove_request(req); - if (dreq->flags == NFS_ODIRECT_RESCHED_WRITES) { + if (status >= 0 && !nfs_write_match_verf(verf, req)) { + dreq->flags = NFS_ODIRECT_RESCHED_WRITES; /* * Despite the reboot, the write was successful, * so reset wb_nio. */ req->wb_nio = 0; - /* Note the rewrite will go through mds */ nfs_mark_request_commit(req, NULL, &cinfo, 0); - } else + } else /* Error or match */ nfs_release_request(req); nfs_unlock_and_release_request(req); } @@ -799,20 +695,15 @@ static void nfs_direct_write_completion(struct nfs_pgio_header *hdr) } nfs_direct_count_bytes(dreq, hdr); - if (hdr->good_bytes != 0) { - if (nfs_write_need_commit(hdr)) { - if (dreq->flags == NFS_ODIRECT_RESCHED_WRITES) - request_commit = true; - else if (dreq->flags == 0) { - nfs_direct_set_hdr_verf(dreq, hdr); - request_commit = true; - dreq->flags = NFS_ODIRECT_DO_COMMIT; - } else if (dreq->flags == NFS_ODIRECT_DO_COMMIT) { - request_commit = true; - if (nfs_direct_set_or_cmp_hdr_verf(dreq, hdr)) - dreq->flags = - NFS_ODIRECT_RESCHED_WRITES; - } + if (hdr->good_bytes != 0 && nfs_write_need_commit(hdr)) { + switch (dreq->flags) { + case 0: + dreq->flags = NFS_ODIRECT_DO_COMMIT; + request_commit = true; + break; + case NFS_ODIRECT_RESCHED_WRITES: + case NFS_ODIRECT_DO_COMMIT: + request_commit = true; } } spin_unlock(&dreq->lock); diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 3b6fa9edc9b5..6542411c020f 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -544,6 +544,14 @@ nfs_write_verifier_cmp(const struct nfs_write_verifier *v1, return memcmp(v1->data, v2->data, sizeof(v1->data)); } +static inline bool +nfs_write_match_verf(const struct nfs_writeverf *verf, + struct nfs_page *req) +{ + return verf->committed > NFS_UNSTABLE && + !nfs_write_verifier_cmp(&req->wb_verf, &verf->verifier); +} + /* unlink.c */ extern struct rpc_task * nfs_async_rename(struct inode *old_dir, struct inode *new_dir, diff --git a/fs/nfs/write.c b/fs/nfs/write.c index 1f8108f5a041..03b7f64f7c4f 100644 --- a/fs/nfs/write.c +++ b/fs/nfs/write.c @@ -1874,8 +1874,7 @@ static void nfs_commit_release_pages(struct nfs_commit_data *data) /* Okay, COMMIT succeeded, apparently. Check the verifier * returned by the server against all stored verfs. */ - if (verf->committed > NFS_UNSTABLE && - !nfs_write_verifier_cmp(&req->wb_verf, &verf->verifier)) { + if (nfs_write_match_verf(verf, req)) { /* We have a match */ if (req->wb_page) nfs_inode_remove_request(req); From patchwork Sat Mar 28 15:32:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4A7E218A5 for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 205A220748 for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409681; bh=9TfW3eJyhlN+A0d2XUhAtxTSyb8hkL/qjLtKnnStKMQ=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=TTenvo2v9QHKPWazL1xHLfRaaeKEpZmiDwUxmSyLDH7IdQLhLOajZfykmYF07l7YG ovaT8Ns5wQnT5ofHGrWJWJDok6r8MpxnZR+Co63KyYFGFBpc1L9Aslj53wZZlYG6py 4MrF+tqYw9tXLWy//pgyuWzJgk17B8cgzUjYmd4c= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726482AbgC1Pej (ORCPT ); Sat, 28 Mar 2020 11:34:39 -0400 Received: from mail.kernel.org ([198.145.29.99]:40334 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726518AbgC1Pei (ORCPT ); Sat, 28 Mar 2020 11:34:38 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E93B5207FF for ; Sat, 28 Mar 2020 15:34:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409678; bh=9TfW3eJyhlN+A0d2XUhAtxTSyb8hkL/qjLtKnnStKMQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=slvpnJTV0YW7TtbTTtz1htYtUskXwViSmm4jPEkWuHAoS9whj8yAOToxiW5OnsomB 9TLf1EIwYNDMholVExKoxmz6gI/cm8lW2pNFNOhy9vPDAixyXt+VrCHkm+B7tayiXW Yd/Bi/RYpsufvH5dzkY+KWmez1jYgotect/riwBk= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 11/22] NFS/pNFS: Support commit arrays in nfs_clear_pnfs_ds_commit_verifiers() Date: Sat, 28 Mar 2020 11:32:09 -0400 Message-Id: <20200328153220.1352010-12-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-11-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Add support for scanning the full list of per-layout segment commit arrays to nfs_clear_pnfs_ds_commit_verifiers(). Signed-off-by: Trond Myklebust --- fs/nfs/internal.h | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 6542411c020f..4a1adad3740f 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -517,13 +517,26 @@ int nfs_filemap_write_and_wait_range(struct address_space *mapping, loff_t lstart, loff_t lend); #ifdef CONFIG_NFS_V4_1 +static inline void +pnfs_bucket_clear_pnfs_ds_commit_verifiers(struct pnfs_commit_bucket *buckets, + unsigned int nbuckets) +{ + unsigned int i; + + for (i = 0; i < nbuckets; i++) + buckets[i].direct_verf.committed = NFS_INVALID_STABLE_HOW; +} static inline void nfs_clear_pnfs_ds_commit_verifiers(struct pnfs_ds_commit_info *cinfo) { - int i; + struct pnfs_commit_array *array; + + pnfs_bucket_clear_pnfs_ds_commit_verifiers(cinfo->buckets, + cinfo->nbuckets); - for (i = 0; i < cinfo->nbuckets; i++) - cinfo->buckets[i].direct_verf.committed = NFS_INVALID_STABLE_HOW; + list_for_each_entry(array, &cinfo->commits, cinfo_list) + pnfs_bucket_clear_pnfs_ds_commit_verifiers(array->buckets, + array->nbuckets); } #else static inline From patchwork Sat Mar 28 15:32:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C02A8159A for ; Sat, 28 Mar 2020 15:34:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8C797207FF for ; Sat, 28 Mar 2020 15:34:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409682; bh=kgJwq8VPMwifyd0KwkcA6eKekrxmSxwH7DFb8y8BizM=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=suyIRG5Gp/LZ4k2+7RALGbnIFxLyHmLdBh41OtHcXD1+DpaYsQVFnz5fBpc4uS2PS Bfj/rwVYnxzmPxaBJ/m3OJ2xQn7hBFlb5nmqbjRWCRUAYJkiIlPUj9RHpQC3WR1ndI hHodkL3fxfWI5nMfAE7m7eEBKAI+MGemKOrV9ji8= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726947AbgC1Pel (ORCPT ); Sat, 28 Mar 2020 11:34:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:40346 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727173AbgC1Pej (ORCPT ); Sat, 28 Mar 2020 11:34:39 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5DE3F20748 for ; Sat, 28 Mar 2020 15:34:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409678; bh=kgJwq8VPMwifyd0KwkcA6eKekrxmSxwH7DFb8y8BizM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=cnRBdHQ/YDDFPzYq3GaGFXyXQTUTsU9JcG6NaRsFOCv7yzf9qzRRValUhcA3R/Qis 1ekZB/ZM2cwoxcOHX5tXrUHrfp/OpGtFiqbZU0GW4lFXc07xKMXjfG+h+pvgOquCY9 K4JTnAyEL53+l2+spnb4bdXgntlW4QgIGFjSJTK8= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 12/22] pNFS: Add infrastructure for cleaning up per-layout commit structures Date: Sat, 28 Mar 2020 11:32:10 -0400 Message-Id: <20200328153220.1352010-13-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-12-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Ensure that both the file and flexfiles layout types clean up when freeing the layout segments. Signed-off-by: Trond Myklebust --- fs/nfs/filelayout/filelayout.c | 16 +++++ fs/nfs/flexfilelayout/flexfilelayout.c | 11 ++++ fs/nfs/internal.h | 4 +- fs/nfs/pnfs.c | 1 + fs/nfs/pnfs.h | 4 ++ fs/nfs/pnfs_nfs.c | 88 +++++++++++++++++++++++++- include/linux/nfs_xdr.h | 1 + 7 files changed, 121 insertions(+), 4 deletions(-) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index b051d5d320ba..ffc5e2af1776 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -750,11 +750,16 @@ filelayout_free_lseg(struct pnfs_layout_segment *lseg) /* This assumes a single RW lseg */ if (lseg->pls_range.iomode == IOMODE_RW) { struct nfs4_filelayout *flo; + struct inode *inode; flo = FILELAYOUT_FROM_HDR(lseg->pls_layout); + inode = flo->generic_hdr.plh_inode; + spin_lock(&inode->i_lock); flo->commit_info.nbuckets = 0; kfree(flo->commit_info.buckets); flo->commit_info.buckets = NULL; + pnfs_generic_ds_cinfo_release_lseg(&flo->commit_info, lseg); + spin_unlock(&inode->i_lock); } _filelayout_free_lseg(fl); } @@ -1163,6 +1168,16 @@ filelayout_get_ds_info(struct inode *inode) return &FILELAYOUT_FROM_HDR(layout)->commit_info; } +static void +filelayout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct inode *inode) +{ + spin_lock(&inode->i_lock); + pnfs_generic_ds_cinfo_destroy(fl_cinfo); + spin_unlock(&inode->i_lock); +} + + static struct pnfs_layoutdriver_type filelayout_type = { .id = LAYOUT_NFSV4_1_FILES, .name = "LAYOUT_NFSV4_1_FILES", @@ -1176,6 +1191,7 @@ static struct pnfs_layoutdriver_type filelayout_type = { .pg_read_ops = &filelayout_pg_read_ops, .pg_write_ops = &filelayout_pg_write_ops, .get_ds_info = &filelayout_get_ds_info, + .release_ds_info = filelayout_release_ds_info, .mark_request_commit = filelayout_mark_request_commit, .clear_request_commit = pnfs_generic_clear_request_commit, .scan_commit_lists = pnfs_generic_scan_commit_lists, diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index c9e79c8e62cd..8e1393d75cbc 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -580,6 +580,7 @@ ff_layout_free_lseg(struct pnfs_layout_segment *lseg) kfree(ffl->commit_info.buckets); ffl->commit_info.buckets = NULL; } + pnfs_generic_ds_cinfo_release_lseg(&ffl->commit_info, lseg); spin_unlock(&inode->i_lock); } _ff_layout_free_lseg(fls); @@ -2003,6 +2004,15 @@ ff_layout_get_ds_info(struct inode *inode) return &FF_LAYOUT_FROM_HDR(layout)->commit_info; } +static void +ff_layout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct inode *inode) +{ + spin_lock(&inode->i_lock); + pnfs_generic_ds_cinfo_destroy(fl_cinfo); + spin_unlock(&inode->i_lock); +} + static void ff_layout_free_deviceid_node(struct nfs4_deviceid_node *d) { @@ -2503,6 +2513,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .pg_read_ops = &ff_layout_pg_read_ops, .pg_write_ops = &ff_layout_pg_write_ops, .get_ds_info = ff_layout_get_ds_info, + .release_ds_info = ff_layout_release_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, .mark_request_commit = pnfs_layout_mark_request_commit, .clear_request_commit = pnfs_generic_clear_request_commit, diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 4a1adad3740f..683146a51599 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -534,9 +534,11 @@ void nfs_clear_pnfs_ds_commit_verifiers(struct pnfs_ds_commit_info *cinfo) pnfs_bucket_clear_pnfs_ds_commit_verifiers(cinfo->buckets, cinfo->nbuckets); - list_for_each_entry(array, &cinfo->commits, cinfo_list) + rcu_read_lock(); + list_for_each_entry_rcu(array, &cinfo->commits, cinfo_list) pnfs_bucket_clear_pnfs_ds_commit_verifiers(array->buckets, array->nbuckets); + rcu_read_unlock(); } #else static inline diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index 6b25117fca5f..eba18f137fb0 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -506,6 +506,7 @@ pnfs_init_lseg(struct pnfs_layout_hdr *lo, struct pnfs_layout_segment *lseg, { INIT_LIST_HEAD(&lseg->pls_list); INIT_LIST_HEAD(&lseg->pls_lc_list); + INIT_LIST_HEAD(&lseg->pls_commits); refcount_set(&lseg->pls_refcount, 1); set_bit(NFS_LSEG_VALID, &lseg->pls_flags); lseg->pls_layout = lo; diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 2ec97b419b56..6c48bd7b4640 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -66,6 +66,7 @@ struct nfs4_pnfs_ds { struct pnfs_layout_segment { struct list_head pls_list; struct list_head pls_lc_list; + struct list_head pls_commits; struct pnfs_layout_range pls_range; refcount_t pls_refcount; u32 pls_seq; @@ -370,6 +371,9 @@ void nfs4_deviceid_purge_client(const struct nfs_client *); /* pnfs_nfs.c */ struct pnfs_commit_array *pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags); void pnfs_free_commit_array(struct pnfs_commit_array *p); +void pnfs_generic_ds_cinfo_release_lseg(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg); +void pnfs_generic_ds_cinfo_destroy(struct pnfs_ds_commit_info *fl_cinfo); void pnfs_generic_clear_request_commit(struct nfs_page *req, struct nfs_commit_info *cinfo); diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index f895a28b1e26..edad251a6a48 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -118,6 +118,67 @@ pnfs_free_commit_array(struct pnfs_commit_array *p) } EXPORT_SYMBOL_GPL(pnfs_free_commit_array); +static void +pnfs_release_commit_array_locked(struct pnfs_commit_array *array) +{ + list_del_rcu(&array->cinfo_list); + list_del(&array->lseg_list); + pnfs_free_commit_array(array); +} + +static void +pnfs_put_commit_array_locked(struct pnfs_commit_array *array) +{ + if (refcount_dec_and_test(&array->refcount)) + pnfs_release_commit_array_locked(array); +} + +static void +pnfs_put_commit_array(struct pnfs_commit_array *array, struct inode *inode) +{ + if (refcount_dec_and_lock(&array->refcount, &inode->i_lock)) { + pnfs_release_commit_array_locked(array); + spin_unlock(&inode->i_lock); + } +} + +static struct pnfs_commit_array * +pnfs_get_commit_array(struct pnfs_commit_array *array) +{ + if (refcount_inc_not_zero(&array->refcount)) + return array; + return NULL; +} + +static void +pnfs_remove_and_free_commit_array(struct pnfs_commit_array *array) +{ + array->lseg = NULL; + list_del_init(&array->lseg_list); + pnfs_put_commit_array_locked(array); +} + +void +pnfs_generic_ds_cinfo_release_lseg(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array, *tmp; + + list_for_each_entry_safe(array, tmp, &lseg->pls_commits, lseg_list) + pnfs_remove_and_free_commit_array(array); +} +EXPORT_SYMBOL_GPL(pnfs_generic_ds_cinfo_release_lseg); + +void +pnfs_generic_ds_cinfo_destroy(struct pnfs_ds_commit_info *fl_cinfo) +{ + struct pnfs_commit_array *array, *tmp; + + list_for_each_entry_safe(array, tmp, &fl_cinfo->commits, cinfo_list) + pnfs_remove_and_free_commit_array(array); +} +EXPORT_SYMBOL_GPL(pnfs_generic_ds_cinfo_destroy); + /* * Locks the nfs_page requests for commit and moves them to * @bucket->committing. @@ -177,14 +238,21 @@ int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) max -= cnt; if (!max) return rv; - list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + rcu_read_lock(); + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (!array->lseg || !pnfs_get_commit_array(array)) + continue; + rcu_read_unlock(); cnt = pnfs_bucket_scan_array(cinfo, array->buckets, array->nbuckets, max); + rcu_read_lock(); + pnfs_put_commit_array(array, cinfo->inode); rv += cnt; max -= cnt; if (!max) break; } + rcu_read_unlock(); return rv; } EXPORT_SYMBOL_GPL(pnfs_generic_scan_commit_lists); @@ -230,13 +298,20 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, fl_cinfo->nbuckets, cinfo); fl_cinfo->nwritten -= nwritten; - list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + rcu_read_lock(); + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (!array->lseg || !pnfs_get_commit_array(array)) + continue; + rcu_read_unlock(); nwritten = pnfs_bucket_recover_commit_reqs(dst, array->buckets, array->nbuckets, cinfo); + rcu_read_lock(); + pnfs_put_commit_array(array, cinfo->inode); fl_cinfo->nwritten -= nwritten; } + rcu_read_unlock(); } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); @@ -330,9 +405,16 @@ pnfs_alloc_ds_commits_list(struct list_head *list, struct pnfs_commit_array *array; unsigned int ret = 0; - list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) + rcu_read_lock(); + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (!array->lseg || !pnfs_get_commit_array(array)) + continue; + rcu_read_unlock(); ret += pnfs_bucket_alloc_ds_commits(list, array->buckets, array->nbuckets, cinfo); + rcu_read_lock(); + pnfs_put_commit_array(array, cinfo->inode); + } return ret; } diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 9946787eda72..33be2ee2a248 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1275,6 +1275,7 @@ struct pnfs_commit_array { struct list_head lseg_list; struct pnfs_layout_segment *lseg; struct rcu_head rcu; + refcount_t refcount; unsigned int nbuckets; struct pnfs_commit_bucket buckets[]; }; From patchwork Sat Mar 28 15:32:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1D28159A for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B821E207FF for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409681; bh=AUDm2aMrZiHB7ng37BjudGbZHs+PyvKrbUFlAs00Ecw=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=I5gkSR+mvzQNSk4P7XkJbmpLcKr4yqmkYlTV0//MEn7rQUuLV/mbBQxIAxR2v7hc4 Gcw+9RXR6hKpEDRniIKSnoISPb76XB8XM4jOdzKd7hBjI18ibYYfU0eGULDhxZrAxS rKhoSU+TP6LdXejybr4NAx/C+dC7zOw0R1IfOFEo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727137AbgC1Pek (ORCPT ); Sat, 28 Mar 2020 11:34:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:40334 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727176AbgC1Pek (ORCPT ); Sat, 28 Mar 2020 11:34:40 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D4AAD2082E for ; Sat, 28 Mar 2020 15:34:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409679; bh=AUDm2aMrZiHB7ng37BjudGbZHs+PyvKrbUFlAs00Ecw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=olgreufIIZdUFhdhIW95cKhtVdFAdZuJL5ZwOmGdPTJdYrYbzndqjwAH9MabWgXlN c2RGh/JnuUtKr2oetDNKK0dOx+KBAHv9mF/1Axh0gJxA7Ws9/23PVuWuwCwfC/P+V+ KMxgsdF3xGJCg1aPFn1ySD4X9S4rtanTnu3rKcIM= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 13/22] pNFS: Enable per-layout segment commit structures Date: Sat, 28 Mar 2020 11:32:11 -0400 Message-Id: <20200328153220.1352010-14-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-13-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Enable adding and lookup of per-layout segment commits in filelayout and flexfilelayout. Signed-off-by: Trond Myklebust --- fs/nfs/filelayout/filelayout.c | 21 +++++++ fs/nfs/flexfilelayout/flexfilelayout.c | 19 +++++++ fs/nfs/pnfs.h | 6 ++ fs/nfs/pnfs_nfs.c | 77 ++++++++++++++++++++++++-- 4 files changed, 117 insertions(+), 6 deletions(-) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index ffc5e2af1776..e3cf42c91d80 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -1168,6 +1168,26 @@ filelayout_get_ds_info(struct inode *inode) return &FILELAYOUT_FROM_HDR(layout)->commit_info; } +static void +filelayout_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct nfs4_filelayout_segment *fl = FILELAYOUT_LSEG(lseg); + struct inode *inode = lseg->pls_layout->plh_inode; + struct pnfs_commit_array *array, *new; + unsigned int size = (fl->stripe_type == STRIPE_SPARSE) ? + fl->dsaddr->ds_num : fl->dsaddr->stripe_count; + + new = pnfs_alloc_commit_array(size, GFP_NOIO); + if (new) { + spin_lock(&inode->i_lock); + array = pnfs_add_commit_array(fl_cinfo, new, lseg); + spin_unlock(&inode->i_lock); + if (array != new) + pnfs_free_commit_array(new); + } +} + static void filelayout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) @@ -1191,6 +1211,7 @@ static struct pnfs_layoutdriver_type filelayout_type = { .pg_read_ops = &filelayout_pg_read_ops, .pg_write_ops = &filelayout_pg_write_ops, .get_ds_info = &filelayout_get_ds_info, + .setup_ds_info = filelayout_setup_ds_info, .release_ds_info = filelayout_release_ds_info, .mark_request_commit = filelayout_mark_request_commit, .clear_request_commit = pnfs_generic_clear_request_commit, diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 8e1393d75cbc..f343a241906a 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -2004,6 +2004,24 @@ ff_layout_get_ds_info(struct inode *inode) return &FF_LAYOUT_FROM_HDR(layout)->commit_info; } +static void +ff_layout_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct nfs4_ff_layout_segment *flseg = FF_LAYOUT_LSEG(lseg); + struct inode *inode = lseg->pls_layout->plh_inode; + struct pnfs_commit_array *array, *new; + + new = pnfs_alloc_commit_array(flseg->mirror_array_cnt, GFP_NOIO); + if (new) { + spin_lock(&inode->i_lock); + array = pnfs_add_commit_array(fl_cinfo, new, lseg); + spin_unlock(&inode->i_lock); + if (array != new) + pnfs_free_commit_array(new); + } +} + static void ff_layout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) @@ -2513,6 +2531,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .pg_read_ops = &ff_layout_pg_read_ops, .pg_write_ops = &ff_layout_pg_write_ops, .get_ds_info = ff_layout_get_ds_info, + .setup_ds_info = ff_layout_setup_ds_info, .release_ds_info = ff_layout_release_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, .mark_request_commit = pnfs_layout_mark_request_commit, diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 6c48bd7b4640..9647045a60c2 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -150,6 +150,8 @@ struct pnfs_layoutdriver_type { const struct nfs_pageio_ops *pg_write_ops; struct pnfs_ds_commit_info *(*get_ds_info) (struct inode *inode); + void (*setup_ds_info)(struct pnfs_ds_commit_info *, + struct pnfs_layout_segment *); void (*release_ds_info)(struct pnfs_ds_commit_info *, struct inode *inode); void (*mark_request_commit) (struct nfs_page *req, @@ -371,6 +373,10 @@ void nfs4_deviceid_purge_client(const struct nfs_client *); /* pnfs_nfs.c */ struct pnfs_commit_array *pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags); void pnfs_free_commit_array(struct pnfs_commit_array *p); +struct pnfs_commit_array *pnfs_add_commit_array(struct pnfs_ds_commit_info *, + struct pnfs_commit_array *, + struct pnfs_layout_segment *); + void pnfs_generic_ds_cinfo_release_lseg(struct pnfs_ds_commit_info *fl_cinfo, struct pnfs_layout_segment *lseg); void pnfs_generic_ds_cinfo_destroy(struct pnfs_ds_commit_info *fl_cinfo); diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index edad251a6a48..5b426a090ee3 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -118,6 +118,66 @@ pnfs_free_commit_array(struct pnfs_commit_array *p) } EXPORT_SYMBOL_GPL(pnfs_free_commit_array); +static struct pnfs_commit_array * +pnfs_find_commit_array_by_lseg(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array; + + list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { + if (array->lseg == lseg) + return array; + } + return NULL; +} + +struct pnfs_commit_array * +pnfs_add_commit_array(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_commit_array *new, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array; + + array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); + if (array) + return array; + new->lseg = lseg; + refcount_set(&new->refcount, 1); + list_add_rcu(&new->cinfo_list, &fl_cinfo->commits); + list_add(&new->lseg_list, &lseg->pls_commits); + return new; +} +EXPORT_SYMBOL_GPL(pnfs_add_commit_array); + +static void +pnfs_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct inode *inode = lseg->pls_layout->plh_inode; + struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + + if (ld->setup_ds_info != NULL) + ld->setup_ds_info(fl_cinfo, lseg); +} + +static struct pnfs_commit_array * +pnfs_lookup_commit_array(struct pnfs_ds_commit_info *fl_cinfo, + struct pnfs_layout_segment *lseg) +{ + struct pnfs_commit_array *array; + + rcu_read_lock(); + array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); + if (!array) { + rcu_read_unlock(); + pnfs_setup_ds_info(fl_cinfo, lseg); + rcu_read_lock(); + array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); + } + rcu_read_unlock(); + return array; +} + static void pnfs_release_commit_array_locked(struct pnfs_commit_array *array) { @@ -1082,17 +1142,18 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, u32 ds_commit_idx) { struct list_head *list; + struct pnfs_commit_array *array; struct pnfs_commit_bucket *buckets; mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); - buckets = cinfo->ds->buckets; + array = pnfs_lookup_commit_array(cinfo->ds, lseg); + if (!array) + goto out_resched; + buckets = array->buckets; list = &buckets[ds_commit_idx].written; if (list_empty(list)) { - if (!pnfs_is_valid_lseg(lseg)) { - mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); - cinfo->completion_ops->resched_write(cinfo, req); - return; - } + if (!pnfs_is_valid_lseg(lseg)) + goto out_resched; /* Non-empty buckets hold a reference on the lseg. That ref * is normally transferred to the COMMIT call and released * there. It could also be released if the last req is pulled @@ -1108,6 +1169,10 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, nfs_request_add_commit_list_locked(req, list, cinfo); mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); nfs_mark_page_unstable(req->wb_page, cinfo); + return; +out_resched: + mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); + cinfo->completion_ops->resched_write(cinfo, req); } EXPORT_SYMBOL_GPL(pnfs_layout_mark_request_commit); From patchwork Sat Mar 28 15:32:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 18085161F for ; Sat, 28 Mar 2020 15:34:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EBD112073B for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409682; bh=J1Chf//k42lvRNcoJ1iT6n2WMRpTbOmpYbCBBwJIA4Q=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=g3GQIcsgW/aJ1JrVjlxzDqmM6JMZTjyOwIIVJq5Lh2d0k8WGfxuwLxU10aN16S8zG 05jCd2Wn6s1++3fVysQzCKAI2z48mSRuGmkGaQo0r0OaqDeLmOv3uTk2zGhSMOvZCu fhDnffqc7N4060c53PDaxChZ5vWVTpcvhMLe7/TQ= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727175AbgC1Pek (ORCPT ); Sat, 28 Mar 2020 11:34:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:40410 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726947AbgC1Pek (ORCPT ); Sat, 28 Mar 2020 11:34:40 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 507F3207FF for ; Sat, 28 Mar 2020 15:34:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409679; bh=J1Chf//k42lvRNcoJ1iT6n2WMRpTbOmpYbCBBwJIA4Q=; h=From:To:Subject:Date:In-Reply-To:References:From; b=HsDc2NfJ88QHju5j2wgl4qV69rOYynyeg+joA9eHcP34V9PRXmpltvPzSZ6wIFEx4 5OMb9cRhPE1UBp2S3z6ZjOomul4zRaI/LXuhwW/we+q46UG2MS9QH6Lk7BJ4VnLm3h TRtsObLwHAzBvFLCI0zZNSOgF1WFvrqf8lqoeXpk= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 14/22] NFS/pNFS: Add a helper pnfs_generic_search_commit_reqs() Date: Sat, 28 Mar 2020 11:32:12 -0400 Message-Id: <20200328153220.1352010-15-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-14-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Lift filelayout_search_commit_reqs() into the generic pnfs/nfs code, and add support for commit arrays. Signed-off-by: Trond Myklebust --- fs/nfs/filelayout/filelayout.c | 32 +-------------------- fs/nfs/pnfs.h | 2 ++ fs/nfs/pnfs_nfs.c | 51 ++++++++++++++++++++++++++++++++++ 3 files changed, 54 insertions(+), 31 deletions(-) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index e3cf42c91d80..795508054a4d 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -1083,36 +1083,6 @@ static int filelayout_initiate_commit(struct nfs_commit_data *data, int how) return -EAGAIN; } -/* filelayout_search_commit_reqs - Search lists in @cinfo for the head reqest - * for @page - * @cinfo - commit info for current inode - * @page - page to search for matching head request - * - * Returns a the head request if one is found, otherwise returns NULL. - */ -static struct nfs_page * -filelayout_search_commit_reqs(struct nfs_commit_info *cinfo, struct page *page) -{ - struct nfs_page *freq, *t; - struct pnfs_commit_bucket *b; - int i; - - /* Linearly search the commit lists for each bucket until a matching - * request is found */ - for (i = 0, b = cinfo->ds->buckets; i < cinfo->ds->nbuckets; i++, b++) { - list_for_each_entry_safe(freq, t, &b->written, wb_list) { - if (freq->wb_page == page) - return freq->wb_head; - } - list_for_each_entry_safe(freq, t, &b->committing, wb_list) { - if (freq->wb_page == page) - return freq->wb_head; - } - } - - return NULL; -} - static int filelayout_commit_pagelist(struct inode *inode, struct list_head *mds_pages, int how, struct nfs_commit_info *cinfo) @@ -1217,7 +1187,7 @@ static struct pnfs_layoutdriver_type filelayout_type = { .clear_request_commit = pnfs_generic_clear_request_commit, .scan_commit_lists = pnfs_generic_scan_commit_lists, .recover_commit_reqs = pnfs_generic_recover_commit_reqs, - .search_commit_reqs = filelayout_search_commit_reqs, + .search_commit_reqs = pnfs_generic_search_commit_reqs, .commit_pagelist = filelayout_commit_pagelist, .read_pagelist = filelayout_read_pagelist, .write_pagelist = filelayout_write_pagelist, diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 9647045a60c2..faed9be6e479 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -388,6 +388,8 @@ void pnfs_generic_prepare_to_resend_writes(struct nfs_commit_data *data); void pnfs_generic_rw_release(void *data); void pnfs_generic_recover_commit_reqs(struct list_head *dst, struct nfs_commit_info *cinfo); +struct nfs_page *pnfs_generic_search_commit_reqs(struct nfs_commit_info *cinfo, + struct page *page); int pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, int how, diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 5b426a090ee3..9b55919e64ac 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -375,6 +375,57 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, } EXPORT_SYMBOL_GPL(pnfs_generic_recover_commit_reqs); +static struct nfs_page * +pnfs_bucket_search_commit_reqs(struct pnfs_commit_bucket *buckets, + unsigned int nbuckets, struct page *page) +{ + struct nfs_page *req; + struct pnfs_commit_bucket *b; + unsigned int i; + + /* Linearly search the commit lists for each bucket until a matching + * request is found */ + for (i = 0, b = buckets; i < nbuckets; i++, b++) { + list_for_each_entry(req, &b->written, wb_list) { + if (req->wb_page == page) + return req->wb_head; + } + list_for_each_entry(req, &b->committing, wb_list) { + if (req->wb_page == page) + return req->wb_head; + } + } + return NULL; +} + +/* pnfs_generic_search_commit_reqs - Search lists in @cinfo for the head reqest + * for @page + * @cinfo - commit info for current inode + * @page - page to search for matching head request + * + * Returns a the head request if one is found, otherwise returns NULL. + */ +struct nfs_page * +pnfs_generic_search_commit_reqs(struct nfs_commit_info *cinfo, struct page *page) +{ + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + struct pnfs_commit_array *array; + struct nfs_page *req; + + req = pnfs_bucket_search_commit_reqs(fl_cinfo->buckets, + fl_cinfo->nbuckets, page); + if (req) + return req; + list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { + req = pnfs_bucket_search_commit_reqs(array->buckets, + array->nbuckets, page); + if (req) + return req; + } + return NULL; +} +EXPORT_SYMBOL_GPL(pnfs_generic_search_commit_reqs); + static struct pnfs_layout_segment * pnfs_bucket_get_committing(struct list_head *head, struct pnfs_commit_bucket *bucket, From patchwork Sat Mar 28 15:32:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463611 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E633F159A for ; Sat, 28 Mar 2020 15:34:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BC1D520748 for ; Sat, 28 Mar 2020 15:34:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409683; bh=9b76lu0bXU8xRvfjRRZvPyklMHdMCeF8CDtr3MJQFB8=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=XojJKbDpb4nbPVuLJqlsofE+n7mF24HpcDfdkITbFDzAtG6fa65Cmz1cpW0H6bXl5 kDOEARMtUjApaOyDbLl9Gc9HQRCx04w+/JOYWaoVYFGUf/TgMvjc8gmZWyxf/1ofqj lAfaAj0ivMDgxVv81stGfWzVrgHuhkGpDVLfA9BE= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727173AbgC1Pel (ORCPT ); Sat, 28 Mar 2020 11:34:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:40346 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726977AbgC1Pek (ORCPT ); Sat, 28 Mar 2020 11:34:40 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BEE9620748 for ; Sat, 28 Mar 2020 15:34:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=9b76lu0bXU8xRvfjRRZvPyklMHdMCeF8CDtr3MJQFB8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Kewi1vDL1mv7TJBPHeV1CpsmvLdbzSyb46dmaUPYgQO5CBwwIxki65f8FTUh5mxmJ kDv+Dl3XAK9TkWcs9cYCAzaJfazIZBZ2Hj/q8zLijGhKowbeowIpdykNlEaxXBnE2V BKmRi2U1hzwF4GKf7xAWO+8xUVXT+9bgnznDsBJ8= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 15/22] NFS: Remove bucket array from struct pnfs_ds_commit_info Date: Sat, 28 Mar 2020 11:32:13 -0400 Message-Id: <20200328153220.1352010-16-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-15-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Remove the unused bucket array in struct pnfs_ds_commit_info. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 1 - fs/nfs/filelayout/filelayout.c | 75 +------------------------ fs/nfs/flexfilelayout/flexfilelayout.c | 76 -------------------------- fs/nfs/internal.h | 3 - fs/nfs/pnfs_nfs.c | 18 ------ include/linux/nfs_xdr.h | 13 ----- 6 files changed, 1 insertion(+), 185 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 4ee26465b510..61f93a0fb0e0 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -217,7 +217,6 @@ static void nfs_direct_req_free(struct kref *kref) struct nfs_direct_req *dreq = container_of(kref, struct nfs_direct_req, kref); pnfs_release_ds_info(&dreq->ds_cinfo, dreq->inode); - nfs_free_pnfs_ds_cinfo(&dreq->ds_cinfo); if (dreq->l_ctx != NULL) nfs_put_lock_context(dreq->l_ctx); if (dreq->ctx != NULL) diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 795508054a4d..854f350e2599 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -755,72 +755,12 @@ filelayout_free_lseg(struct pnfs_layout_segment *lseg) flo = FILELAYOUT_FROM_HDR(lseg->pls_layout); inode = flo->generic_hdr.plh_inode; spin_lock(&inode->i_lock); - flo->commit_info.nbuckets = 0; - kfree(flo->commit_info.buckets); - flo->commit_info.buckets = NULL; pnfs_generic_ds_cinfo_release_lseg(&flo->commit_info, lseg); spin_unlock(&inode->i_lock); } _filelayout_free_lseg(fl); } -static int -filelayout_alloc_commit_info(struct pnfs_layout_segment *lseg, - struct nfs_commit_info *cinfo, - gfp_t gfp_flags) -{ - struct nfs4_filelayout_segment *fl = FILELAYOUT_LSEG(lseg); - struct pnfs_commit_bucket *buckets; - int size, i; - - if (fl->commit_through_mds) - return 0; - - size = (fl->stripe_type == STRIPE_SPARSE) ? - fl->dsaddr->ds_num : fl->dsaddr->stripe_count; - - if (cinfo->ds->nbuckets >= size) { - /* This assumes there is only one IOMODE_RW lseg. What - * we really want to do is have a layout_hdr level - * dictionary of keys, each - * associated with a struct list_head, populated by calls - * to filelayout_write_pagelist(). - * */ - return 0; - } - - buckets = kcalloc(size, sizeof(struct pnfs_commit_bucket), - gfp_flags); - if (!buckets) - return -ENOMEM; - for (i = 0; i < size; i++) { - INIT_LIST_HEAD(&buckets[i].written); - INIT_LIST_HEAD(&buckets[i].committing); - /* mark direct verifier as unset */ - buckets[i].direct_verf.committed = NFS_INVALID_STABLE_HOW; - } - - spin_lock(&cinfo->inode->i_lock); - if (cinfo->ds->nbuckets >= size) - goto out; - for (i = 0; i < cinfo->ds->nbuckets; i++) { - list_splice(&cinfo->ds->buckets[i].written, - &buckets[i].written); - list_splice(&cinfo->ds->buckets[i].committing, - &buckets[i].committing); - buckets[i].direct_verf.committed = - cinfo->ds->buckets[i].direct_verf.committed; - buckets[i].wlseg = cinfo->ds->buckets[i].wlseg; - buckets[i].clseg = cinfo->ds->buckets[i].clseg; - } - swap(cinfo->ds->buckets, buckets); - cinfo->ds->nbuckets = size; -out: - spin_unlock(&cinfo->inode->i_lock); - kfree(buckets); - return 0; -} - static struct pnfs_layout_segment * filelayout_alloc_lseg(struct pnfs_layout_hdr *layoutid, struct nfs4_layoutget_res *lgr, @@ -943,9 +883,6 @@ static void filelayout_pg_init_write(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) { - struct nfs_commit_info cinfo; - int status; - pnfs_generic_pg_check_layout(pgio); if (!pgio->pg_lseg) { pgio->pg_lseg = fl_pnfs_update_layout(pgio->pg_inode, @@ -964,17 +901,7 @@ filelayout_pg_init_write(struct nfs_pageio_descriptor *pgio, /* If no lseg, fall back to write through mds */ if (pgio->pg_lseg == NULL) - goto out_mds; - nfs_init_cinfo(&cinfo, pgio->pg_inode, pgio->pg_dreq); - status = filelayout_alloc_commit_info(pgio->pg_lseg, &cinfo, GFP_NOFS); - if (status < 0) { - pnfs_put_lseg(pgio->pg_lseg); - pgio->pg_lseg = NULL; - goto out_mds; - } - return; -out_mds: - nfs_pageio_reset_write_mds(pgio); + nfs_pageio_reset_write_mds(pgio); } static const struct nfs_pageio_ops filelayout_pg_read_ops = { diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index f343a241906a..1a4e36d07eab 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -550,17 +550,6 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh, goto out_free_page; } -static bool ff_layout_has_rw_segments(struct pnfs_layout_hdr *layout) -{ - struct pnfs_layout_segment *lseg; - - list_for_each_entry(lseg, &layout->plh_segs, pls_list) - if (lseg->pls_range.iomode == IOMODE_RW) - return true; - - return false; -} - static void ff_layout_free_lseg(struct pnfs_layout_segment *lseg) { @@ -575,24 +564,12 @@ ff_layout_free_lseg(struct pnfs_layout_segment *lseg) ffl = FF_LAYOUT_FROM_HDR(lseg->pls_layout); inode = ffl->generic_hdr.plh_inode; spin_lock(&inode->i_lock); - if (!ff_layout_has_rw_segments(lseg->pls_layout)) { - ffl->commit_info.nbuckets = 0; - kfree(ffl->commit_info.buckets); - ffl->commit_info.buckets = NULL; - } pnfs_generic_ds_cinfo_release_lseg(&ffl->commit_info, lseg); spin_unlock(&inode->i_lock); } _ff_layout_free_lseg(fls); } -/* Return 1 until we have multiple lsegs support */ -static int -ff_layout_get_lseg_count(struct nfs4_ff_layout_segment *fls) -{ - return 1; -} - static void nfs4_ff_start_busy_timer(struct nfs4_ff_busy_timer *timer, ktime_t now) { @@ -737,52 +714,6 @@ nfs4_ff_layout_stat_io_end_write(struct rpc_task *task, spin_unlock(&mirror->lock); } -static int -ff_layout_alloc_commit_info(struct pnfs_layout_segment *lseg, - struct nfs_commit_info *cinfo, - gfp_t gfp_flags) -{ - struct nfs4_ff_layout_segment *fls = FF_LAYOUT_LSEG(lseg); - struct pnfs_commit_bucket *buckets; - int size; - - if (cinfo->ds->nbuckets != 0) { - /* This assumes there is only one RW lseg per file. - * To support multiple lseg per file, we need to - * change struct pnfs_commit_bucket to allow dynamic - * increasing nbuckets. - */ - return 0; - } - - size = ff_layout_get_lseg_count(fls) * FF_LAYOUT_MIRROR_COUNT(lseg); - - buckets = kcalloc(size, sizeof(struct pnfs_commit_bucket), - gfp_flags); - if (!buckets) - return -ENOMEM; - else { - int i; - - spin_lock(&cinfo->inode->i_lock); - if (cinfo->ds->nbuckets != 0) - kfree(buckets); - else { - cinfo->ds->buckets = buckets; - cinfo->ds->nbuckets = size; - for (i = 0; i < size; i++) { - INIT_LIST_HEAD(&buckets[i].written); - INIT_LIST_HEAD(&buckets[i].committing); - /* mark direct verifier as unset */ - buckets[i].direct_verf.committed = - NFS_INVALID_STABLE_HOW; - } - } - spin_unlock(&cinfo->inode->i_lock); - return 0; - } -} - static void ff_layout_mark_ds_unreachable(struct pnfs_layout_segment *lseg, int idx) { @@ -944,10 +875,8 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, { struct nfs4_ff_layout_mirror *mirror; struct nfs_pgio_mirror *pgm; - struct nfs_commit_info cinfo; struct nfs4_pnfs_ds *ds; int i; - int status; retry: pnfs_generic_pg_check_layout(pgio); @@ -969,11 +898,6 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, if (pgio->pg_lseg == NULL) goto out_mds; - nfs_init_cinfo(&cinfo, pgio->pg_inode, pgio->pg_dreq); - status = ff_layout_alloc_commit_info(pgio->pg_lseg, &cinfo, GFP_NOFS); - if (status < 0) - goto out_mds; - /* Use a direct mapping of ds_idx to pgio mirror_idx */ if (WARN_ON_ONCE(pgio->pg_mirror_count != FF_LAYOUT_MIRROR_COUNT(pgio->pg_lseg))) diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index 683146a51599..78f317fac940 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -531,9 +531,6 @@ void nfs_clear_pnfs_ds_commit_verifiers(struct pnfs_ds_commit_info *cinfo) { struct pnfs_commit_array *array; - pnfs_bucket_clear_pnfs_ds_commit_verifiers(cinfo->buckets, - cinfo->nbuckets); - rcu_read_lock(); list_for_each_entry_rcu(array, &cinfo->commits, cinfo_list) pnfs_bucket_clear_pnfs_ds_commit_verifiers(array->buckets, diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 9b55919e64ac..20f12f3cbe38 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -292,12 +292,6 @@ int pnfs_generic_scan_commit_lists(struct nfs_commit_info *cinfo, int max) struct pnfs_commit_array *array; int rv = 0, cnt; - cnt = pnfs_bucket_scan_array(cinfo, fl_cinfo->buckets, - fl_cinfo->nbuckets, max); - rv += cnt; - max -= cnt; - if (!max) - return rv; rcu_read_lock(); list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { if (!array->lseg || !pnfs_get_commit_array(array)) @@ -353,11 +347,6 @@ void pnfs_generic_recover_commit_reqs(struct list_head *dst, unsigned int nwritten; lockdep_assert_held(&NFS_I(cinfo->inode)->commit_mutex); - nwritten = pnfs_bucket_recover_commit_reqs(dst, - fl_cinfo->buckets, - fl_cinfo->nbuckets, - cinfo); - fl_cinfo->nwritten -= nwritten; rcu_read_lock(); list_for_each_entry_rcu(array, &fl_cinfo->commits, cinfo_list) { if (!array->lseg || !pnfs_get_commit_array(array)) @@ -412,10 +401,6 @@ pnfs_generic_search_commit_reqs(struct nfs_commit_info *cinfo, struct page *page struct pnfs_commit_array *array; struct nfs_page *req; - req = pnfs_bucket_search_commit_reqs(fl_cinfo->buckets, - fl_cinfo->nbuckets, page); - if (req) - return req; list_for_each_entry(array, &fl_cinfo->commits, cinfo_list) { req = pnfs_bucket_search_commit_reqs(array->buckets, array->nbuckets, page); @@ -550,9 +535,6 @@ pnfs_generic_commit_pagelist(struct inode *inode, struct list_head *mds_pages, nreq++; } - nreq += pnfs_bucket_alloc_ds_commits(&list, fl_cinfo->buckets, - fl_cinfo->nbuckets, cinfo); - nreq += pnfs_alloc_ds_commits_list(&list, fl_cinfo, cinfo); if (nreq == 0) goto out; diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 33be2ee2a248..2903597ec88c 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1284,8 +1284,6 @@ struct pnfs_ds_commit_info { struct list_head commits; unsigned int nwritten; unsigned int ncommitting; - unsigned int nbuckets; - struct pnfs_commit_bucket *buckets; }; struct nfs41_state_protection { @@ -1396,22 +1394,11 @@ struct nfs41_free_stateid_res { unsigned int status; }; -static inline void -nfs_free_pnfs_ds_cinfo(struct pnfs_ds_commit_info *cinfo) -{ - kfree(cinfo->buckets); -} - #else struct pnfs_ds_commit_info { }; -static inline void -nfs_free_pnfs_ds_cinfo(struct pnfs_ds_commit_info *cinfo) -{ -} - #endif /* CONFIG_NFS_V4_1 */ #ifdef CONFIG_NFS_V4_2 From patchwork Sat Mar 28 15:32:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463625 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 28C4892A for ; Sat, 28 Mar 2020 15:34:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E831020748 for ; Sat, 28 Mar 2020 15:34:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409687; bh=diUewrALk/h96nTmDtd1aZ+xD4ZCPnwl8rF5Aeil590=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=d7Iis5OOnnGlYjOmHHIZ3IddCCXxlpWRqwH5A6CSuVcmDAf7HLCZeMGtCwomKvgTc VvxgspB9ew95krebZjjtkWyRHGF99y8lVEtbUuh5Iq1q8Ql4L4uGQbeZSenrYlCdsC aQcMlKFrSuuVuu0GSHGm9fFkC6iDkZhOK3EXzT6s= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727178AbgC1Pep (ORCPT ); Sat, 28 Mar 2020 11:34:45 -0400 Received: from mail.kernel.org ([198.145.29.99]:40410 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727131AbgC1Pel (ORCPT ); Sat, 28 Mar 2020 11:34:41 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3615A207FF for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=diUewrALk/h96nTmDtd1aZ+xD4ZCPnwl8rF5Aeil590=; h=From:To:Subject:Date:In-Reply-To:References:From; b=aa8GF51eu/8d0by35iETbBv0CjkYCCU00M0xjJm5DABUEO41BKdo3wMa5tiV9weHI t23kywOg5MejDYHC7QLvP6kRqyNZj6MhjXOPo+BAwTMZw0bXy9n2gPZSyBVviVzJBI sOshd3OomS29qk4JnupZbNkmII5ysP2b6kKedqAU= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 16/22] NFS/pNFS: Clean up pNFS commit operations Date: Sat, 28 Mar 2020 11:32:14 -0400 Message-Id: <20200328153220.1352010-17-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-16-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> <20200328153220.1352010-16-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Move the pNFS commit related operations into a separate structure that can be carried by the pnfs_ds_commit_info. Signed-off-by: Trond Myklebust --- fs/nfs/direct.c | 6 +- fs/nfs/filelayout/filelayout.c | 20 +++-- fs/nfs/flexfilelayout/flexfilelayout.c | 19 +++-- fs/nfs/pnfs.h | 110 ++++++++++++++++--------- fs/nfs/pnfs_nfs.c | 13 +-- include/linux/nfs_xdr.h | 1 + 6 files changed, 98 insertions(+), 71 deletions(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 61f93a0fb0e0..51ab4627c4d6 100644 --- a/fs/nfs/direct.c +++ b/fs/nfs/direct.c @@ -511,10 +511,7 @@ nfs_direct_write_scan_commit_list(struct inode *inode, struct nfs_commit_info *cinfo) { mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); -#ifdef CONFIG_NFS_V4_1 - if (cinfo->ds != NULL && cinfo->ds->nwritten != 0) - NFS_SERVER(inode)->pnfs_curr_ld->recover_commit_reqs(list, cinfo); -#endif + pnfs_recover_commit_reqs(list, cinfo); nfs_scan_commit_list(&cinfo->mds->list, list, cinfo, 0); mutex_unlock(&NFS_I(cinfo->inode)->commit_mutex); } @@ -917,6 +914,7 @@ ssize_t nfs_file_direct_write(struct kiocb *iocb, struct iov_iter *iter) dreq->l_ctx = l_ctx; if (!is_sync_kiocb(iocb)) dreq->iocb = iocb; + pnfs_init_ds_commit_info_ops(&dreq->ds_cinfo, inode); nfs_start_io_direct(inode); diff --git a/fs/nfs/filelayout/filelayout.c b/fs/nfs/filelayout/filelayout.c index 854f350e2599..a13e69009f19 100644 --- a/fs/nfs/filelayout/filelayout.c +++ b/fs/nfs/filelayout/filelayout.c @@ -49,6 +49,7 @@ MODULE_AUTHOR("Dean Hildebrand "); MODULE_DESCRIPTION("The NFSv4 file layout driver"); #define FILELAYOUT_POLL_RETRY_MAX (15*HZ) +static const struct pnfs_commit_ops filelayout_commit_ops; static loff_t filelayout_get_dense_offset(struct nfs4_filelayout_segment *flseg, @@ -1045,6 +1046,7 @@ filelayout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) if (flo == NULL) return NULL; pnfs_init_ds_commit_info(&flo->commit_info); + flo->commit_info.ops = &filelayout_commit_ops; return &flo->generic_hdr; } @@ -1094,6 +1096,16 @@ filelayout_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, spin_unlock(&inode->i_lock); } +static const struct pnfs_commit_ops filelayout_commit_ops = { + .setup_ds_info = filelayout_setup_ds_info, + .release_ds_info = filelayout_release_ds_info, + .mark_request_commit = filelayout_mark_request_commit, + .clear_request_commit = pnfs_generic_clear_request_commit, + .scan_commit_lists = pnfs_generic_scan_commit_lists, + .recover_commit_reqs = pnfs_generic_recover_commit_reqs, + .search_commit_reqs = pnfs_generic_search_commit_reqs, + .commit_pagelist = filelayout_commit_pagelist, +}; static struct pnfs_layoutdriver_type filelayout_type = { .id = LAYOUT_NFSV4_1_FILES, @@ -1108,14 +1120,6 @@ static struct pnfs_layoutdriver_type filelayout_type = { .pg_read_ops = &filelayout_pg_read_ops, .pg_write_ops = &filelayout_pg_write_ops, .get_ds_info = &filelayout_get_ds_info, - .setup_ds_info = filelayout_setup_ds_info, - .release_ds_info = filelayout_release_ds_info, - .mark_request_commit = filelayout_mark_request_commit, - .clear_request_commit = pnfs_generic_clear_request_commit, - .scan_commit_lists = pnfs_generic_scan_commit_lists, - .recover_commit_reqs = pnfs_generic_recover_commit_reqs, - .search_commit_reqs = pnfs_generic_search_commit_reqs, - .commit_pagelist = filelayout_commit_pagelist, .read_pagelist = filelayout_read_pagelist, .write_pagelist = filelayout_write_pagelist, .alloc_deviceid_node = filelayout_alloc_deviceid_node, diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 1a4e36d07eab..d37883a2b51f 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -32,6 +32,7 @@ static unsigned short io_maxretrans; +static const struct pnfs_commit_ops ff_layout_commit_ops; static void ff_layout_read_record_layoutstats_done(struct rpc_task *task, struct nfs_pgio_header *hdr); static int ff_layout_mirror_prepare_stats(struct pnfs_layout_hdr *lo, @@ -52,6 +53,7 @@ ff_layout_alloc_layout_hdr(struct inode *inode, gfp_t gfp_flags) INIT_LIST_HEAD(&ffl->error_list); INIT_LIST_HEAD(&ffl->mirrors); ffl->last_report_time = ktime_get(); + ffl->commit_info.ops = &ff_layout_commit_ops; return &ffl->generic_hdr; } else return NULL; @@ -2440,6 +2442,16 @@ ff_layout_set_layoutdriver(struct nfs_server *server, return 0; } +static const struct pnfs_commit_ops ff_layout_commit_ops = { + .setup_ds_info = ff_layout_setup_ds_info, + .release_ds_info = ff_layout_release_ds_info, + .mark_request_commit = pnfs_layout_mark_request_commit, + .clear_request_commit = pnfs_generic_clear_request_commit, + .scan_commit_lists = pnfs_generic_scan_commit_lists, + .recover_commit_reqs = pnfs_generic_recover_commit_reqs, + .commit_pagelist = ff_layout_commit_pagelist, +}; + static struct pnfs_layoutdriver_type flexfilelayout_type = { .id = LAYOUT_FLEX_FILES, .name = "LAYOUT_FLEX_FILES", @@ -2455,14 +2467,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = { .pg_read_ops = &ff_layout_pg_read_ops, .pg_write_ops = &ff_layout_pg_write_ops, .get_ds_info = ff_layout_get_ds_info, - .setup_ds_info = ff_layout_setup_ds_info, - .release_ds_info = ff_layout_release_ds_info, .free_deviceid_node = ff_layout_free_deviceid_node, - .mark_request_commit = pnfs_layout_mark_request_commit, - .clear_request_commit = pnfs_generic_clear_request_commit, - .scan_commit_lists = pnfs_generic_scan_commit_lists, - .recover_commit_reqs = pnfs_generic_recover_commit_reqs, - .commit_pagelist = ff_layout_commit_pagelist, .read_pagelist = ff_layout_read_pagelist, .write_pagelist = ff_layout_write_pagelist, .alloc_deviceid_node = ff_layout_alloc_deviceid_node, diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index faed9be6e479..b32025553f26 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -150,26 +150,6 @@ struct pnfs_layoutdriver_type { const struct nfs_pageio_ops *pg_write_ops; struct pnfs_ds_commit_info *(*get_ds_info) (struct inode *inode); - void (*setup_ds_info)(struct pnfs_ds_commit_info *, - struct pnfs_layout_segment *); - void (*release_ds_info)(struct pnfs_ds_commit_info *, - struct inode *inode); - void (*mark_request_commit) (struct nfs_page *req, - struct pnfs_layout_segment *lseg, - struct nfs_commit_info *cinfo, - u32 ds_commit_idx); - void (*clear_request_commit) (struct nfs_page *req, - struct nfs_commit_info *cinfo); - int (*scan_commit_lists) (struct nfs_commit_info *cinfo, - int max); - void (*recover_commit_reqs) (struct list_head *list, - struct nfs_commit_info *cinfo); - struct nfs_page * (*search_commit_reqs)(struct nfs_commit_info *cinfo, - struct page *page); - int (*commit_pagelist)(struct inode *inode, - struct list_head *mds_pages, - int how, - struct nfs_commit_info *cinfo); int (*sync)(struct inode *inode, bool datasync); @@ -192,6 +172,29 @@ struct pnfs_layoutdriver_type { int (*prepare_layoutstats) (struct nfs42_layoutstat_args *args); }; +struct pnfs_commit_ops { + void (*setup_ds_info)(struct pnfs_ds_commit_info *, + struct pnfs_layout_segment *); + void (*release_ds_info)(struct pnfs_ds_commit_info *, + struct inode *inode); + int (*commit_pagelist)(struct inode *inode, + struct list_head *mds_pages, + int how, + struct nfs_commit_info *cinfo); + void (*mark_request_commit) (struct nfs_page *req, + struct pnfs_layout_segment *lseg, + struct nfs_commit_info *cinfo, + u32 ds_commit_idx); + void (*clear_request_commit) (struct nfs_page *req, + struct nfs_commit_info *cinfo); + int (*scan_commit_lists) (struct nfs_commit_info *cinfo, + int max); + void (*recover_commit_reqs) (struct list_head *list, + struct nfs_commit_info *cinfo); + struct nfs_page * (*search_commit_reqs)(struct nfs_commit_info *cinfo, + struct page *page); +}; + struct pnfs_layout_hdr { refcount_t plh_refcount; atomic_t plh_outstanding; /* number of RPCs out */ @@ -461,9 +464,11 @@ static inline int pnfs_commit_list(struct inode *inode, struct list_head *mds_pages, int how, struct nfs_commit_info *cinfo) { - if (cinfo->ds == NULL || cinfo->ds->ncommitting == 0) + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + + if (fl_cinfo == NULL || fl_cinfo->ncommitting == 0) return PNFS_NOT_ATTEMPTED; - return NFS_SERVER(inode)->pnfs_curr_ld->commit_pagelist(inode, mds_pages, how, cinfo); + return fl_cinfo->ops->commit_pagelist(inode, mds_pages, how, cinfo); } static inline struct pnfs_ds_commit_info * @@ -476,19 +481,26 @@ pnfs_get_ds_info(struct inode *inode) return ld->get_ds_info(inode); } +static inline void +pnfs_init_ds_commit_info_ops(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ + struct pnfs_ds_commit_info *inode_cinfo = pnfs_get_ds_info(inode); + if (inode_cinfo != NULL) + fl_cinfo->ops = inode_cinfo->ops; +} + static inline void pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo) { INIT_LIST_HEAD(&fl_cinfo->commits); + fl_cinfo->ops = NULL; } static inline void pnfs_release_ds_info(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) { - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; - - if (ld != NULL && ld->release_ds_info != NULL) - ld->release_ds_info(fl_cinfo, inode); + if (fl_cinfo->ops != NULL && fl_cinfo->ops->release_ds_info != NULL) + fl_cinfo->ops->release_ds_info(fl_cinfo, inode); } static inline void @@ -501,24 +513,22 @@ static inline bool pnfs_mark_request_commit(struct nfs_page *req, struct pnfs_layout_segment *lseg, struct nfs_commit_info *cinfo, u32 ds_commit_idx) { - struct inode *inode = d_inode(nfs_req_openctx(req)->dentry); - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - if (lseg == NULL || ld->mark_request_commit == NULL) + if (!lseg || !fl_cinfo->ops->mark_request_commit) return false; - ld->mark_request_commit(req, lseg, cinfo, ds_commit_idx); + fl_cinfo->ops->mark_request_commit(req, lseg, cinfo, ds_commit_idx); return true; } static inline bool pnfs_clear_request_commit(struct nfs_page *req, struct nfs_commit_info *cinfo) { - struct inode *inode = d_inode(nfs_req_openctx(req)->dentry); - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - if (ld == NULL || ld->clear_request_commit == NULL) + if (!fl_cinfo || !fl_cinfo->ops || !fl_cinfo->ops->clear_request_commit) return false; - ld->clear_request_commit(req, cinfo); + fl_cinfo->ops->clear_request_commit(req, cinfo); return true; } @@ -526,21 +536,31 @@ static inline int pnfs_scan_commit_lists(struct inode *inode, struct nfs_commit_info *cinfo, int max) { - if (cinfo->ds == NULL || cinfo->ds->nwritten == 0) + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + + if (!fl_cinfo || fl_cinfo->nwritten == 0) return 0; - else - return NFS_SERVER(inode)->pnfs_curr_ld->scan_commit_lists(cinfo, max); + return fl_cinfo->ops->scan_commit_lists(cinfo, max); +} + +static inline void +pnfs_recover_commit_reqs(struct list_head *head, struct nfs_commit_info *cinfo) +{ + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; + + if (fl_cinfo && fl_cinfo->nwritten != 0) + fl_cinfo->ops->recover_commit_reqs(head, cinfo); } static inline struct nfs_page * pnfs_search_commit_reqs(struct inode *inode, struct nfs_commit_info *cinfo, struct page *page) { - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; + struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds; - if (ld == NULL || ld->search_commit_reqs == NULL) + if (!fl_cinfo->ops || !fl_cinfo->ops->search_commit_reqs) return NULL; - return ld->search_commit_reqs(cinfo, page); + return fl_cinfo->ops->search_commit_reqs(cinfo, page); } /* Should the pNFS client commit and return the layout upon a setattr */ @@ -788,6 +808,11 @@ pnfs_get_ds_info(struct inode *inode) return NULL; } +static inline void +pnfs_init_ds_commit_info_ops(struct pnfs_ds_commit_info *fl_cinfo, struct inode *inode) +{ +} + static inline void pnfs_init_ds_commit_info(struct pnfs_ds_commit_info *fl_cinfo) { @@ -818,6 +843,11 @@ pnfs_scan_commit_lists(struct inode *inode, struct nfs_commit_info *cinfo, return 0; } +static inline void +pnfs_recover_commit_reqs(struct list_head *head, struct nfs_commit_info *cinfo) +{ +} + static inline struct nfs_page * pnfs_search_commit_reqs(struct inode *inode, struct nfs_commit_info *cinfo, struct page *page) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 20f12f3cbe38..06df2e6663dc 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -149,17 +149,6 @@ pnfs_add_commit_array(struct pnfs_ds_commit_info *fl_cinfo, } EXPORT_SYMBOL_GPL(pnfs_add_commit_array); -static void -pnfs_setup_ds_info(struct pnfs_ds_commit_info *fl_cinfo, - struct pnfs_layout_segment *lseg) -{ - struct inode *inode = lseg->pls_layout->plh_inode; - struct pnfs_layoutdriver_type *ld = NFS_SERVER(inode)->pnfs_curr_ld; - - if (ld->setup_ds_info != NULL) - ld->setup_ds_info(fl_cinfo, lseg); -} - static struct pnfs_commit_array * pnfs_lookup_commit_array(struct pnfs_ds_commit_info *fl_cinfo, struct pnfs_layout_segment *lseg) @@ -170,7 +159,7 @@ pnfs_lookup_commit_array(struct pnfs_ds_commit_info *fl_cinfo, array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); if (!array) { rcu_read_unlock(); - pnfs_setup_ds_info(fl_cinfo, lseg); + fl_cinfo->ops->setup_ds_info(fl_cinfo, lseg); rcu_read_lock(); array = pnfs_find_commit_array_by_lseg(fl_cinfo, lseg); } diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index 2903597ec88c..adbbeae9ce5b 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1284,6 +1284,7 @@ struct pnfs_ds_commit_info { struct list_head commits; unsigned int nwritten; unsigned int ncommitting; + const struct pnfs_commit_ops *ops; }; struct nfs41_state_protection { From patchwork Sat Mar 28 15:32:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463613 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B00292A for ; Sat, 28 Mar 2020 15:34:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3A2B82073B for ; Sat, 28 Mar 2020 15:34:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409684; bh=k5rZj86dcDGk+1q5xIdpBhWggFIGK8n7JGoeppfbZaI=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=Dy887/XS4U6Nl1VXX3eHTiyPUC5FrIVnkVgW7IYcqIebTh6b3MmDJrlzWSURi2IKM mOWMqTu+bZj1lnNdlSBRy6EXJVtiUEtEQsisOTTx6rvHvspW+DNcICDy4Lgu+F+n3C xvjzZFftYPZx1XeLc7C+QxR25Bdy6tAXPJ2f1wfg= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727179AbgC1Pen (ORCPT ); Sat, 28 Mar 2020 11:34:43 -0400 Received: from mail.kernel.org ([198.145.29.99]:40334 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727176AbgC1Pel (ORCPT ); Sat, 28 Mar 2020 11:34:41 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A7AD42082D for ; Sat, 28 Mar 2020 15:34:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409680; bh=k5rZj86dcDGk+1q5xIdpBhWggFIGK8n7JGoeppfbZaI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=UzxgNxiyPOFbt3z49ypXgBAcC0WHHxd0zn5JpmjocyMPveMh97kDpmK7tQuIXpDL4 OSQ4weLWSCCVft7ObRl2HQm3m8LtrfRvxX5Z0gJYkGUTHaqf5mY/6ZgLo7y0Yz/p8i 7FiT+tMV9iNMb1gbiiZ5XviR7ZUccUePssa/R/pg= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 17/22] NFS/pNFS: Simplify bucket layout segment reference counting Date: Sat, 28 Mar 2020 11:32:15 -0400 Message-Id: <20200328153220.1352010-18-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-17-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> <20200328153220.1352010-16-trondmy@kernel.org> <20200328153220.1352010-17-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 39 ++++++++++++++++++++------------------- include/linux/nfs_xdr.h | 3 +-- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index 06df2e6663dc..abf16fc98346 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -59,6 +59,17 @@ void pnfs_generic_commit_release(void *calldata) } EXPORT_SYMBOL_GPL(pnfs_generic_commit_release); +static struct pnfs_layout_segment * +pnfs_free_bucket_lseg(struct pnfs_commit_bucket *bucket) +{ + if (list_empty(&bucket->committing) && list_empty(&bucket->written)) { + struct pnfs_layout_segment *freeme = bucket->lseg; + bucket->lseg = NULL; + return freeme; + } + return NULL; +} + /* The generic layer is about to remove the req from the commit list. * If this will make the bucket empty, it will need to put the lseg reference. * Note this must be called holding nfsi->commit_mutex @@ -78,8 +89,7 @@ pnfs_generic_clear_request_commit(struct nfs_page *req, bucket = list_first_entry(&req->wb_list, struct pnfs_commit_bucket, written); - freeme = bucket->wlseg; - bucket->wlseg = NULL; + freeme = pnfs_free_bucket_lseg(bucket); } out: nfs_request_remove_commit_list(req, cinfo); @@ -103,8 +113,7 @@ pnfs_alloc_commit_array(size_t n, gfp_t gfp_flags) for (b = &p->buckets[0]; n != 0; b++, n--) { INIT_LIST_HEAD(&b->written); INIT_LIST_HEAD(&b->committing); - b->wlseg = NULL; - b->clseg = NULL; + b->lseg = NULL; b->direct_verf.committed = NFS_INVALID_STABLE_HOW; } return p; @@ -246,12 +255,6 @@ pnfs_bucket_scan_ds_commit_list(struct pnfs_commit_bucket *bucket, if (ret) { cinfo->ds->nwritten -= ret; cinfo->ds->ncommitting += ret; - if (bucket->clseg == NULL) - bucket->clseg = pnfs_get_lseg(bucket->wlseg); - if (list_empty(src)) { - pnfs_put_lseg(bucket->wlseg); - bucket->wlseg = NULL; - } } return ret; } @@ -317,9 +320,8 @@ pnfs_bucket_recover_commit_reqs(struct list_head *dst, if (!nwritten) continue; ret += nwritten; - if (list_empty(&b->written)) { - freeme = b->wlseg; - b->wlseg = NULL; + freeme = pnfs_free_bucket_lseg(b); + if (freeme) { pnfs_put_lseg(freeme); goto restart; } @@ -405,15 +407,12 @@ pnfs_bucket_get_committing(struct list_head *head, struct pnfs_commit_bucket *bucket, struct nfs_commit_info *cinfo) { - struct pnfs_layout_segment *freeme; struct list_head *pos; list_for_each(pos, &bucket->committing) cinfo->ds->ncommitting--; list_splice_init(&bucket->committing, head); - freeme = bucket->clseg; - bucket->clseg = NULL; - return freeme; + return pnfs_free_bucket_lseg(bucket); } static struct nfs_commit_data * @@ -425,6 +424,8 @@ pnfs_bucket_fetch_commitdata(struct pnfs_commit_bucket *bucket, if (!data) return NULL; data->lseg = pnfs_bucket_get_committing(&data->pages, bucket, cinfo); + if (!data->lseg) + data->lseg = pnfs_get_lseg(bucket->lseg); return data; } @@ -1182,8 +1183,8 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, * off due to a rewrite, in which case it will be done in * pnfs_common_clear_request_commit */ - WARN_ON_ONCE(buckets[ds_commit_idx].wlseg != NULL); - buckets[ds_commit_idx].wlseg = pnfs_get_lseg(lseg); + if (!buckets[ds_commit_idx].lseg) + buckets[ds_commit_idx].lseg = pnfs_get_lseg(lseg); } set_bit(PG_COMMIT_TO_DS, &req->wb_flags); cinfo->ds->nwritten++; diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h index adbbeae9ce5b..7bbb1f6fc1b1 100644 --- a/include/linux/nfs_xdr.h +++ b/include/linux/nfs_xdr.h @@ -1265,8 +1265,7 @@ struct nfstime4 { struct pnfs_commit_bucket { struct list_head written; struct list_head committing; - struct pnfs_layout_segment *wlseg; - struct pnfs_layout_segment *clseg; + struct pnfs_layout_segment *lseg; struct nfs_writeverf direct_verf; }; From patchwork Sat Mar 28 15:32:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463617 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE354159A for ; Sat, 28 Mar 2020 15:34:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BE03620748 for ; Sat, 28 Mar 2020 15:34:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409684; bh=+SFShDxnoX+gVXJT4uPxAaGlDjkIaMd8lZ0ElJY57m0=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=RHcWrNZRMb5D4vlTTAquhn6SKKXQQV47No+aW3fcPJz2/epftQ6734IuduJkEP27V dUHipvFwqNYNZE+81TH1pWzH5hF3q3n/3NqpINyctD+1DkMEDC7Fk1F0I2x+1JHOvC JcyhJ16TNnmLVIBrrcwGYjRPUM7k1t6grD9ImgnA= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727183AbgC1Pem (ORCPT ); Sat, 28 Mar 2020 11:34:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:40346 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727178AbgC1Pel (ORCPT ); Sat, 28 Mar 2020 11:34:41 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1C4CA2073B for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409681; bh=+SFShDxnoX+gVXJT4uPxAaGlDjkIaMd8lZ0ElJY57m0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=oypH7GgjQyt+jYGpGipTemTdSliCmkNLEGUfdnxdNvrQjo0Q6gWhUrOYNcI77jnMT EZxvNeTMJXgm/v0HO7Xd+blFKearldO3IkvklyMy+104WBgYWpmEJEnVRvLou0mtJY 6rqQSdyZ4DuEA/vRlSrkyoSlQTq9qZusUFqOvOIQ= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 18/22] NFS/pNFS: Fix pnfs_layout_mark_request_commit() invalid layout segment handling Date: Sat, 28 Mar 2020 11:32:16 -0400 Message-Id: <20200328153220.1352010-19-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-18-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> <20200328153220.1352010-16-trondmy@kernel.org> <20200328153220.1352010-17-trondmy@kernel.org> <20200328153220.1352010-18-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Fix up pnfs_layout_mark_request_commit() to alway reschedule the write if the layout segment is invalid. Also minor cleanup. Signed-off-by: Trond Myklebust --- fs/nfs/pnfs_nfs.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/fs/nfs/pnfs_nfs.c b/fs/nfs/pnfs_nfs.c index abf16fc98346..25f135572fc8 100644 --- a/fs/nfs/pnfs_nfs.c +++ b/fs/nfs/pnfs_nfs.c @@ -1166,26 +1166,22 @@ pnfs_layout_mark_request_commit(struct nfs_page *req, { struct list_head *list; struct pnfs_commit_array *array; - struct pnfs_commit_bucket *buckets; + struct pnfs_commit_bucket *bucket; mutex_lock(&NFS_I(cinfo->inode)->commit_mutex); array = pnfs_lookup_commit_array(cinfo->ds, lseg); - if (!array) + if (!array || !pnfs_is_valid_lseg(lseg)) goto out_resched; - buckets = array->buckets; - list = &buckets[ds_commit_idx].written; - if (list_empty(list)) { - if (!pnfs_is_valid_lseg(lseg)) - goto out_resched; - /* Non-empty buckets hold a reference on the lseg. That ref - * is normally transferred to the COMMIT call and released - * there. It could also be released if the last req is pulled - * off due to a rewrite, in which case it will be done in - * pnfs_common_clear_request_commit - */ - if (!buckets[ds_commit_idx].lseg) - buckets[ds_commit_idx].lseg = pnfs_get_lseg(lseg); - } + bucket = &array->buckets[ds_commit_idx]; + list = &bucket->written; + /* Non-empty buckets hold a reference on the lseg. That ref + * is normally transferred to the COMMIT call and released + * there. It could also be released if the last req is pulled + * off due to a rewrite, in which case it will be done in + * pnfs_common_clear_request_commit + */ + if (!bucket->lseg) + bucket->lseg = pnfs_get_lseg(lseg); set_bit(PG_COMMIT_TO_DS, &req->wb_flags); cinfo->ds->nwritten++; From patchwork Sat Mar 28 15:32:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463619 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1246592A for ; Sat, 28 Mar 2020 15:34:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E83BC207FF for ; Sat, 28 Mar 2020 15:34:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409685; bh=NuMpFZor33KpgDCooyxC/0nGZA6wuMmyv4qL5T4CILA=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=wRuBaaLv2wUXpIxrnPcua6ldoPbidcr3y5EFDvES8HNRk6vPYtcBrwQw4ZDeqzkCx HYyHIZluSZI0guI0AVIQ/KJhZb5xlc9knm7ctHU4UwgQouXELfFZm7QVu8kMmw9gxn pB9R7e6coC0E5eo71N5ARHC1bMMkAS/O3zJZzsps= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727192AbgC1Pen (ORCPT ); Sat, 28 Mar 2020 11:34:43 -0400 Received: from mail.kernel.org ([198.145.29.99]:40410 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727179AbgC1Pem (ORCPT ); Sat, 28 Mar 2020 11:34:42 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 86C7820748 for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409681; bh=NuMpFZor33KpgDCooyxC/0nGZA6wuMmyv4qL5T4CILA=; h=From:To:Subject:Date:In-Reply-To:References:From; b=aX0P65PJNY4Qbpwv++JoYTRX4jsjeJ88kjBPJC9pczRZDck+mAeAJU/cQo6i4f7vE erBL6B+sgr7MRpyrvCUbdEfWs+2r62880KD1YppMmTVzzJzSuGOe7RfhQphRgNTKc2 plg5Jiy0+K80j7YT35wWJvH4clkkVhmgjo3cMxqU= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 19/22] pNFS/flexfile: Don't merge layout segments if the mirrors don't match Date: Sat, 28 Mar 2020 11:32:17 -0400 Message-Id: <20200328153220.1352010-20-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-19-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> <20200328153220.1352010-16-trondmy@kernel.org> <20200328153220.1352010-17-trondmy@kernel.org> <20200328153220.1352010-18-trondmy@kernel.org> <20200328153220.1352010-19-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Check that the number of mirrors, and the mirror information matches before deciding to merge layout segments in pNFS/flexfiles. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index d37883a2b51f..3221001f2ea1 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -283,6 +283,23 @@ static void _ff_layout_free_lseg(struct nfs4_ff_layout_segment *fls) } } +static bool +ff_lseg_match_mirrors(struct pnfs_layout_segment *l1, + struct pnfs_layout_segment *l2) +{ + const struct nfs4_ff_layout_segment *fl1 = FF_LAYOUT_LSEG(l1); + const struct nfs4_ff_layout_segment *fl2 = FF_LAYOUT_LSEG(l1); + u32 i; + + if (fl1->mirror_array_cnt != fl2->mirror_array_cnt) + return false; + for (i = 0; i < fl1->mirror_array_cnt; i++) { + if (fl1->mirror_array[i] != fl2->mirror_array[i]) + return false; + } + return true; +} + static bool ff_lseg_range_is_after(const struct pnfs_layout_range *l1, const struct pnfs_layout_range *l2) @@ -318,6 +335,8 @@ ff_lseg_merge(struct pnfs_layout_segment *new, new->pls_range.length); if (new_end < old->pls_range.offset) return false; + if (!ff_lseg_match_mirrors(new, old)) + return false; /* Mergeable: copy info from 'old' to 'new' */ if (new_end < old_end) From patchwork Sat Mar 28 15:32:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463615 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F8261667 for ; Sat, 28 Mar 2020 15:34:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 65766207FF for ; Sat, 28 Mar 2020 15:34:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409684; bh=SfThXb5sbELLShXdiZVWMn/D+sIxP3vxICn3SobEKq0=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=bvGN33Jf0o9jbh9OFDvAbl6NR5t7HKZToFdtFfD256nf373JgxoVj6+E3zhuZfbdS fqsK1h0G74cW47z4tElfQ67oHyESF0vb0cqjpV/DpuOTS8kXuK2oFPqqiH/JbRSyYU a/uwwQCD/TbdJicEMHaK3pCTB0obXV8kG0VKg6I4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727067AbgC1Pen (ORCPT ); Sat, 28 Mar 2020 11:34:43 -0400 Received: from mail.kernel.org ([198.145.29.99]:40346 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726977AbgC1Pem (ORCPT ); Sat, 28 Mar 2020 11:34:42 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F0F5820848 for ; Sat, 28 Mar 2020 15:34:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409682; bh=SfThXb5sbELLShXdiZVWMn/D+sIxP3vxICn3SobEKq0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=H04n6K2PSYaLjL/htFFW3k1PyFpz7GmZL8OZkVJAHn6sWyrrpYbfNMmTPMlDoeARw 1Sl1ASISdWkBJYHYEY8lz8+VgO8D0j8P8sdR03IzAsjvNvvz6/yR9K9AIshOtYL/sh rPkKBaMsbqwQOQOvywRQi6MnC50UCoRSt/tABrEo= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 20/22] pNFS/flexfiles: Check the layout segment range before doing I/O Date: Sat, 28 Mar 2020 11:32:18 -0400 Message-Id: <20200328153220.1352010-21-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-20-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> <20200328153220.1352010-16-trondmy@kernel.org> <20200328153220.1352010-17-trondmy@kernel.org> <20200328153220.1352010-18-trondmy@kernel.org> <20200328153220.1352010-19-trondmy@kernel.org> <20200328153220.1352010-20-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust When starting to read or write with a layout segment, check that the range matches our request. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 12 ++++++++++-- fs/nfs/pnfs.c | 3 ++- fs/nfs/pnfs.h | 1 + 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 3221001f2ea1..2b45807a5221 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -830,6 +830,14 @@ ff_layout_pg_get_read(struct nfs_pageio_descriptor *pgio, } } +static void +ff_layout_pg_check_layout(struct nfs_pageio_descriptor *pgio, + struct nfs_page *req) +{ + pnfs_generic_pg_check_layout(pgio); + pnfs_generic_pg_check_range(pgio, req); +} + static void ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) @@ -840,7 +848,7 @@ ff_layout_pg_init_read(struct nfs_pageio_descriptor *pgio, int ds_idx; retry: - pnfs_generic_pg_check_layout(pgio); + ff_layout_pg_check_layout(pgio, req); /* Use full layout for now */ if (!pgio->pg_lseg) { ff_layout_pg_get_read(pgio, req, false); @@ -900,7 +908,7 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, int i; retry: - pnfs_generic_pg_check_layout(pgio); + ff_layout_pg_check_layout(pgio, req); if (!pgio->pg_lseg) { pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, nfs_req_openctx(req), diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index eba18f137fb0..6fcf26b16816 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -2611,7 +2611,7 @@ EXPORT_SYMBOL_GPL(pnfs_generic_pg_check_layout); * Check for any intersection between the request and the pgio->pg_lseg, * and if none, put this pgio->pg_lseg away. */ -static void +void pnfs_generic_pg_check_range(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) { if (pgio->pg_lseg && !pnfs_lseg_request_intersecting(pgio->pg_lseg, req)) { @@ -2619,6 +2619,7 @@ pnfs_generic_pg_check_range(struct nfs_pageio_descriptor *pgio, struct nfs_page pgio->pg_lseg = NULL; } } +EXPORT_SYMBOL_GPL(pnfs_generic_pg_check_range); void pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *req) diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index b32025553f26..8e0ada581b92 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -252,6 +252,7 @@ void pnfs_put_lseg(struct pnfs_layout_segment *lseg); void set_pnfs_layoutdriver(struct nfs_server *, const struct nfs_fh *, struct nfs_fsinfo *); void unset_pnfs_layoutdriver(struct nfs_server *); void pnfs_generic_pg_check_layout(struct nfs_pageio_descriptor *pgio); +void pnfs_generic_pg_check_range(struct nfs_pageio_descriptor *pgio, struct nfs_page *req); void pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *, struct nfs_page *); int pnfs_generic_pg_readpages(struct nfs_pageio_descriptor *desc); void pnfs_generic_pg_init_write(struct nfs_pageio_descriptor *pgio, From patchwork Sat Mar 28 15:32:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463621 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50A1C161F for ; Sat, 28 Mar 2020 15:34:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3018620748 for ; Sat, 28 Mar 2020 15:34:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409685; bh=KLX0w9B6a2wfY6DBgUd9oIry6zclDL0IGW+QbOQtxCo=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=KUB9TpenSvjGaYQnU/fE6kOtB/IBVc9k01+PhbVoTcxQEG/zu5XO5jntuBotYwkPT tEKctHEYOCL2ORfaK39meGSZCnHesoA5G/kwx+HaG1ZbAMvNz0RTaZPgx6jjcYG01s UX774q8XbYFMRch7ii4iW1Rb909+ih/ig6VI+RzU= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726977AbgC1Pen (ORCPT ); Sat, 28 Mar 2020 11:34:43 -0400 Received: from mail.kernel.org ([198.145.29.99]:40410 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727191AbgC1Pen (ORCPT ); Sat, 28 Mar 2020 11:34:43 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 68A862073B for ; Sat, 28 Mar 2020 15:34:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409682; bh=KLX0w9B6a2wfY6DBgUd9oIry6zclDL0IGW+QbOQtxCo=; h=From:To:Subject:Date:In-Reply-To:References:From; b=F7GxnTVMtIThRrT7EAehovr+DQsgMUd/C091wf5xKB/zCdJMhGunUFaSe2wHFFlzJ oYiNvx8IQTchfm2KYfTY5iMwOZ0CUPPDgvkj7xhlnP6EnFZFiZsXnY7IXD3Vq2yS3U 3XWBtIVj+4ff/alAixebn3N46a0nk77Wts6TIc8w= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 21/22] pNFS/flexfiles: remove requirement for whole file layouts Date: Sat, 28 Mar 2020 11:32:19 -0400 Message-Id: <20200328153220.1352010-22-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-21-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> <20200328153220.1352010-16-trondmy@kernel.org> <20200328153220.1352010-17-trondmy@kernel.org> <20200328153220.1352010-18-trondmy@kernel.org> <20200328153220.1352010-19-trondmy@kernel.org> <20200328153220.1352010-20-trondmy@kernel.org> <20200328153220.1352010-21-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Remove the requirement that the server always sends whole file layouts. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 2b45807a5221..42f581e213cc 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -257,24 +257,6 @@ static void ff_layout_free_mirror_array(struct nfs4_ff_layout_segment *fls) ff_layout_put_mirror(fls->mirror_array[i]); } -static int ff_layout_check_layout(struct nfs4_layoutget_res *lgr) -{ - int ret = 0; - - dprintk("--> %s\n", __func__); - - /* FIXME: remove this check when layout segment support is added */ - if (lgr->range.offset != 0 || - lgr->range.length != NFS4_MAX_UINT64) { - dprintk("%s Only whole file layouts supported. Use MDS i/o\n", - __func__); - ret = -EINVAL; - } - - dprintk("--> %s returns %d\n", __func__, ret); - return ret; -} - static void _ff_layout_free_lseg(struct nfs4_ff_layout_segment *fls) { if (fls) { @@ -556,9 +538,6 @@ ff_layout_alloc_lseg(struct pnfs_layout_hdr *lh, out_sort_mirrors: ff_layout_sort_mirrors(fls); - rc = ff_layout_check_layout(lgr); - if (rc) - goto out_err_free; ret = &fls->generic_hdr; dprintk("<-- %s (success)\n", __func__); out_free_page: From patchwork Sat Mar 28 15:32:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 11463623 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACDB1159A for ; Sat, 28 Mar 2020 15:34:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8BA3E20748 for ; Sat, 28 Mar 2020 15:34:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409685; bh=N2F+UWkB32txK8mb0OmAnuzV4O09A/+ZXa4KFe+L4BM=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=bVlgFpBX281u3OrckJIKuaqjQb1jEAK/FTLeNhn0tHKgrdUH+b1CnI9WGLjxRxgq4 i2nHCtlJk2svVELSoiV31wip3WXEsQ+tFKMGNfq/DX61RkM0RTvKbRGWz2rI762NFz hj/xTj439ZgLQKaDxiQjDbM3WHuZ+nKX9KBEVuxw= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727176AbgC1Peo (ORCPT ); Sat, 28 Mar 2020 11:34:44 -0400 Received: from mail.kernel.org ([198.145.29.99]:40334 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727178AbgC1Pen (ORCPT ); Sat, 28 Mar 2020 11:34:43 -0400 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D0E8120748 for ; Sat, 28 Mar 2020 15:34:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585409683; bh=N2F+UWkB32txK8mb0OmAnuzV4O09A/+ZXa4KFe+L4BM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=oFDWD09+IKI5kO6f0E6dbQQbk7n5PyAvBgWp5KYZfidYhDtyq5pvofgdUjGpY+qG1 qkYa+DiXegKDugzSTiF0nbq6d4HQ68Tsgfyc7O2dtZLkZeRkvqkN3KsMEqdtHLUM4+ +2bbH24GKyrjI3fHLmYDK9gO8jxQ+w6raSUkG6MA= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v2 22/22] pNFS/flexfiles: Specify the layout segment range in LAYOUTGET Date: Sat, 28 Mar 2020 11:32:20 -0400 Message-Id: <20200328153220.1352010-23-trondmy@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200328153220.1352010-22-trondmy@kernel.org> References: <20200328153220.1352010-1-trondmy@kernel.org> <20200328153220.1352010-2-trondmy@kernel.org> <20200328153220.1352010-3-trondmy@kernel.org> <20200328153220.1352010-4-trondmy@kernel.org> <20200328153220.1352010-5-trondmy@kernel.org> <20200328153220.1352010-6-trondmy@kernel.org> <20200328153220.1352010-7-trondmy@kernel.org> <20200328153220.1352010-8-trondmy@kernel.org> <20200328153220.1352010-9-trondmy@kernel.org> <20200328153220.1352010-10-trondmy@kernel.org> <20200328153220.1352010-11-trondmy@kernel.org> <20200328153220.1352010-12-trondmy@kernel.org> <20200328153220.1352010-13-trondmy@kernel.org> <20200328153220.1352010-14-trondmy@kernel.org> <20200328153220.1352010-15-trondmy@kernel.org> <20200328153220.1352010-16-trondmy@kernel.org> <20200328153220.1352010-17-trondmy@kernel.org> <20200328153220.1352010-18-trondmy@kernel.org> <20200328153220.1352010-19-trondmy@kernel.org> <20200328153220.1352010-20-trondmy@kernel.org> <20200328153220.1352010-21-trondmy@kernel.org> <20200328153220.1352010-22-trondmy@kernel.org> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust Move from requesting only full file layout segments, to requesting layout segments that match our I/O size. This means the server is still free to return a full file layout, but we will no longer error out if it does not. Signed-off-by: Trond Myklebust --- fs/nfs/flexfilelayout/flexfilelayout.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c index 42f581e213cc..7d399f72ebbb 100644 --- a/fs/nfs/flexfilelayout/flexfilelayout.c +++ b/fs/nfs/flexfilelayout/flexfilelayout.c @@ -798,8 +798,8 @@ ff_layout_pg_get_read(struct nfs_pageio_descriptor *pgio, pnfs_put_lseg(pgio->pg_lseg); pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, nfs_req_openctx(req), - 0, - NFS4_MAX_UINT64, + req_offset(req), + req->wb_bytes, IOMODE_READ, strict_iomode, GFP_KERNEL); @@ -891,8 +891,8 @@ ff_layout_pg_init_write(struct nfs_pageio_descriptor *pgio, if (!pgio->pg_lseg) { pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode, nfs_req_openctx(req), - 0, - NFS4_MAX_UINT64, + req_offset(req), + req->wb_bytes, IOMODE_RW, false, GFP_NOFS);