From patchwork Thu Aug 2 08:47:10 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Idan Kedar X-Patchwork-Id: 1266861 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 6F405DF223 for ; Thu, 2 Aug 2012 08:47:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751980Ab2HBIrX (ORCPT ); Thu, 2 Aug 2012 04:47:23 -0400 Received: from mail-bk0-f46.google.com ([209.85.214.46]:47855 "EHLO mail-bk0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752182Ab2HBIrW (ORCPT ); Thu, 2 Aug 2012 04:47:22 -0400 Received: by mail-bk0-f46.google.com with SMTP id j10so4129902bkw.19 for ; Thu, 02 Aug 2012 01:47:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:subject:date:message-id:x-mailer:in-reply-to:references :x-gm-message-state; bh=/CpIWfQno6zvMTB1ottQXh1YhAG3nX533aT03IOR0u8=; b=dI0AIF/l67xtXL2deW/DToK4c+BxeZQZXvX8Nd+lmXXncSN85UZ5/hWH7ZQalXAMbN j0B3X2/L7PwiQYKR7fXJbzFSvQ0JRRc0ZcrCi/8LjLSg+DhOeHrRyUqZUKA1VtPbLy23 wTBz17BN7Nv5u/dIc7eubhVRSTRDMV93hVfbHiD87W97oTvJ2FaSb13PlLaBXVL1Juac uaAOtYOf/l4qSjl7tFY3sxiQBVfG1g8mj3rOnjDqIet/0mLPvQL58r/YYIw9u2sHvg5f 6TvdQE8n0gPCNbbkN5XFik2B03XX3oDX6hcCYHewq/6ZRqw+PQ/4x9Tm1YPhk/LKjwTl s8Ew== Received: by 10.204.157.135 with SMTP id b7mr7905077bkx.61.1343897241292; Thu, 02 Aug 2012 01:47:21 -0700 (PDT) Received: from tonian.com ([82.166.37.139]) by mx.google.com with ESMTPS id c18sm2864171bkv.8.2012.08.02.01.47.18 (version=SSLv3 cipher=OTHER); Thu, 02 Aug 2012 01:47:20 -0700 (PDT) From: Idan Kedar To: coredev@tonian.com, Trond Myklebust , linux-nfs@vger.kernel.org, Benny Halevy Subject: [PATCH v2 1/2] pnfs: defer release of pages in layoutget Date: Thu, 2 Aug 2012 11:47:10 +0300 Message-Id: <1343897231-30205-2-git-send-email-idank@tonian.com> X-Mailer: git-send-email 1.7.6.5 In-Reply-To: <1343897231-30205-1-git-send-email-idank@tonian.com> References: <1343897231-30205-1-git-send-email-idank@tonian.com> X-Gm-Message-State: ALoCoQnQ8a0u8+UqTdCinftEPaRO+lypPvGRQz7FgSHtuNxnOE4C35M/nMbkpvUKK7jClG3R9nSC Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org we have encountered a bug whereby reading a lot of files (copying fedora's /bin) from a pNFS mount and hitting Ctrl+C in the middle caused a general protection fault in xdr_shrink_bufhead. this function is called when decoding the response from LAYOUTGET. the decoding is done by a worker thread, and the caller of LAYOUTGET waits for the worker thread to complete. hitting Ctrl+C caused the synchronous wait to end and the next thing the caller does is to free the pages, so when the worker thread calls xdr_shrink_bufhead, the pages are gone. therefore, the cleanup of these pages has been moved to nfs4_layoutget_release. Signed-off-by: Idan Kedar Signed-off-by: Benny Halevy --- fs/nfs/nfs4proc.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++++- fs/nfs/pnfs.c | 39 +----------------------------------- fs/nfs/pnfs.h | 2 +- 3 files changed, 58 insertions(+), 40 deletions(-) diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 15fc7e4..1a1776b 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -6164,11 +6164,58 @@ static void nfs4_layoutget_done(struct rpc_task *task, void *calldata) dprintk("<-- %s\n", __func__); } +static size_t max_response_pages(struct nfs_server *server) +{ + u32 max_resp_sz = server->nfs_client->cl_session->fc_attrs.max_resp_sz; + return nfs_page_array_len(0, max_resp_sz); +} + +static void nfs4_free_pages(struct page **pages, size_t size) +{ + int i; + + if (!pages) + return; + + for (i = 0; i < size; i++) { + if (!pages[i]) + break; + __free_page(pages[i]); + } + kfree(pages); +} + +static struct page **nfs4_alloc_pages(size_t size, gfp_t gfp_flags) +{ + struct page **pages; + int i; + + pages = kcalloc(size, sizeof(struct page *), gfp_flags); + if (!pages) { + dprintk("%s: can't alloc array of %zu pages\n", __func__, size); + return NULL; + } + + for (i = 0; i < size; i++) { + pages[i] = alloc_page(gfp_flags); + if (!pages[i]) { + dprintk("%s: failed to allocate page\n", __func__); + nfs4_free_pages(pages, size); + return NULL; + } + } + + return pages; +} + static void nfs4_layoutget_release(void *calldata) { struct nfs4_layoutget *lgp = calldata; + struct nfs_server *server = NFS_SERVER(lgp->args.inode); + size_t max_pages = max_response_pages(server); dprintk("--> %s\n", __func__); + nfs4_free_pages(lgp->args.layout.pages, max_pages); put_nfs_open_context(lgp->args.ctx); kfree(calldata); dprintk("<-- %s\n", __func__); @@ -6180,9 +6227,10 @@ static const struct rpc_call_ops nfs4_layoutget_call_ops = { .rpc_release = nfs4_layoutget_release, }; -int nfs4_proc_layoutget(struct nfs4_layoutget *lgp) +int nfs4_proc_layoutget(struct nfs4_layoutget *lgp, gfp_t gfp_flags) { struct nfs_server *server = NFS_SERVER(lgp->args.inode); + size_t max_pages = max_response_pages(server); struct rpc_task *task; struct rpc_message msg = { .rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_LAYOUTGET], @@ -6200,6 +6248,13 @@ int nfs4_proc_layoutget(struct nfs4_layoutget *lgp) dprintk("--> %s\n", __func__); + lgp->args.layout.pages = nfs4_alloc_pages(max_pages, gfp_flags); + if (!lgp->args.layout.pages) { + nfs4_layoutget_release(lgp); + return -ENOMEM; + } + lgp->args.layout.pglen = max_pages * PAGE_SIZE; + lgp->res.layoutp = &lgp->args.layout; lgp->res.seq_res.sr_slot = NULL; nfs41_init_sequence(&lgp->args.seq_args, &lgp->res.seq_res, 0); diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index bbc49ca..8229a0e 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -583,9 +583,6 @@ send_layoutget(struct pnfs_layout_hdr *lo, struct nfs_server *server = NFS_SERVER(ino); struct nfs4_layoutget *lgp; struct pnfs_layout_segment *lseg = NULL; - struct page **pages = NULL; - int i; - u32 max_resp_sz, max_pages; dprintk("--> %s\n", __func__); @@ -594,20 +591,6 @@ send_layoutget(struct pnfs_layout_hdr *lo, if (lgp == NULL) return NULL; - /* allocate pages for xdr post processing */ - max_resp_sz = server->nfs_client->cl_session->fc_attrs.max_resp_sz; - max_pages = nfs_page_array_len(0, max_resp_sz); - - pages = kcalloc(max_pages, sizeof(struct page *), gfp_flags); - if (!pages) - goto out_err_free; - - for (i = 0; i < max_pages; i++) { - pages[i] = alloc_page(gfp_flags); - if (!pages[i]) - goto out_err_free; - } - lgp->args.minlength = PAGE_CACHE_SIZE; if (lgp->args.minlength > range->length) lgp->args.minlength = range->length; @@ -616,39 +599,19 @@ send_layoutget(struct pnfs_layout_hdr *lo, lgp->args.type = server->pnfs_curr_ld->id; lgp->args.inode = ino; lgp->args.ctx = get_nfs_open_context(ctx); - lgp->args.layout.pages = pages; - lgp->args.layout.pglen = max_pages * PAGE_SIZE; lgp->lsegpp = &lseg; lgp->gfp_flags = gfp_flags; /* Synchronously retrieve layout information from server and * store in lseg. */ - nfs4_proc_layoutget(lgp); + nfs4_proc_layoutget(lgp, gfp_flags); if (!lseg) { /* remember that LAYOUTGET failed and suspend trying */ set_bit(lo_fail_bit(range->iomode), &lo->plh_flags); } - /* free xdr pages */ - for (i = 0; i < max_pages; i++) - __free_page(pages[i]); - kfree(pages); - return lseg; - -out_err_free: - /* free any allocated xdr pages, lgp as it's not used */ - if (pages) { - for (i = 0; i < max_pages; i++) { - if (!pages[i]) - break; - __free_page(pages[i]); - } - kfree(pages); - } - kfree(lgp); - return NULL; } /* Initiates a LAYOUTRETURN(FILE) */ diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h index 64f90d8..9a31ff3 100644 --- a/fs/nfs/pnfs.h +++ b/fs/nfs/pnfs.h @@ -171,7 +171,7 @@ extern int nfs4_proc_getdevicelist(struct nfs_server *server, struct pnfs_devicelist *devlist); extern int nfs4_proc_getdeviceinfo(struct nfs_server *server, struct pnfs_device *dev); -extern int nfs4_proc_layoutget(struct nfs4_layoutget *lgp); +extern int nfs4_proc_layoutget(struct nfs4_layoutget *lgp, gfp_t gfp_flags); extern int nfs4_proc_layoutreturn(struct nfs4_layoutreturn *lrp); /* pnfs.c */