From patchwork Thu Aug 11 16:25:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Grygorii Strashko X-Patchwork-Id: 9275597 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DDE5C600CB for ; Thu, 11 Aug 2016 16:28:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D00DC2871D for ; Thu, 11 Aug 2016 16:28:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C47332871F; Thu, 11 Aug 2016 16:28:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 492CF28727 for ; Thu, 11 Aug 2016 16:28:39 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bXsop-00089i-6Y; Thu, 11 Aug 2016 16:26:55 +0000 Received: from arroyo.ext.ti.com ([198.47.19.12]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bXso4-0007HO-C7 for linux-arm-kernel@lists.infradead.org; Thu, 11 Aug 2016 16:26:35 +0000 Received: from dlelxv90.itg.ti.com ([172.17.2.17]) by arroyo.ext.ti.com (8.13.7/8.13.7) with ESMTP id u7BGPk0f005715; Thu, 11 Aug 2016 11:25:46 -0500 Received: from DFLE72.ent.ti.com (dfle72.ent.ti.com [128.247.5.109]) by dlelxv90.itg.ti.com (8.14.3/8.13.8) with ESMTP id u7BGPk22010732; Thu, 11 Aug 2016 11:25:46 -0500 Received: from dlep33.itg.ti.com (157.170.170.75) by DFLE72.ent.ti.com (128.247.5.109) with Microsoft SMTP Server id 14.3.294.0; Thu, 11 Aug 2016 11:25:45 -0500 Received: from [172.22.233.201] (ileax41-snat.itg.ti.com [10.172.224.153]) by dlep33.itg.ti.com (8.14.3/8.13.8) with ESMTP id u7BGPhAW016828; Thu, 11 Aug 2016 11:25:44 -0500 Subject: Re: Kernel NFS boot failure To: netdev , Trond Myklebust , , Anna Schumaker References: <674e6841-d2c5-ccae-6633-a699e848e6d2@ti.com> <57A1DE28.7020604@arm.com> From: Grygorii Strashko Message-ID: Date: Thu, 11 Aug 2016 19:25:44 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160811_092609_331366_2EA19AE9 X-CRM114-Status: GOOD ( 18.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Vladimir Murzin , Neil Brown , Sekhar Nori , open list , "linux-omap@vger.kernel.org" , linux-arm Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On 08/03/2016 06:04 PM, Grygorii Strashko wrote: > Hi Vladimir, > > On 08/03/2016 03:06 PM, Vladimir Murzin wrote: >> On 03/08/16 12:41, Grygorii Strashko wrote: >>> We observe Kernel boot failure while running NFS boot stress test (1000 iterations): >>> - Linux version 4.7.0 > > I'd like to pay your attention that this issue also reproducible with > Kernel 4.7.0! > The same can be seen from the log I've provided in first e-mail: > [ 0.000000] Linux version 4.7.0 (lcpdbld@dflsdit-build06.dal.design.ti.com) (gcc version 4.9.3 20150413 (prerelease) (Linaro GCC 4.9-2015.05) ) #1 SMP Fri Jul 29 17:41:27 CDT 2016 > > > I've not run the test with current master at it's not been tagged yet. Still in progress. rc1 unstable on my platforms due to other issues :( > >>> - am335x-evm (TI AM335x EVM) >>> - failure rate 10-20 times per test. >>> Originally this issue was reproduced using TI Kernel 4.4 >>> ( git://git.ti.com/ti-linux-kernel/ti-linux-kernel.git, branch: ti-linux-4.4.y) >>> on both am335x-evm and am57xx-beagle-x15(am57xx-evm) platforms. >>> This issues has not been reproduced with TI Kernel 4.1 before. >>> >>> The SysRq shows that system stuck in nfs_fs_mount() >>> >>> [ 207.904632] [] (schedule) from [] (rpc_wait_bit_killable+0x2c/0xd8) >>> [ 207.912996] [] (rpc_wait_bit_killable) from [] (__wait_on_bit+0x84/0xc0) >>> [ 207.921812] [] (__wait_on_bit) from [] (out_of_line_wait_on_bit+0x64/0x70) >>> [ 207.930810] [] (out_of_line_wait_on_bit) from [] (__rpc_execute+0x18c/0x544) >>> [ 207.939988] [] (__rpc_execute) from [] (rpc_run_task+0x13c/0x158) >>> [ 207.948166] [] (rpc_run_task) from [] (rpc_call_sync+0x44/0xc4) >>> [ 207.956163] [] (rpc_call_sync) from [] (rpc_ping+0x48/0x68) >>> [ 207.963796] [] (rpc_ping) from [] (rpc_create_xprt+0xec/0x164) >>> [ 207.971702] [] (rpc_create_xprt) from [] (rpc_create+0xf0/0x1a0) >>> [ 207.979794] [] (rpc_create) from [] (nfs_create_rpc_client+0xd4/0xec) >>> [ 207.988338] [] (nfs_create_rpc_client) from [] (nfs_init_client+0x20/0x78) >>> [ 207.997332] [] (nfs_init_client) from [] (nfs_create_server+0xa0/0x3bc) >>> [ 208.006057] [] (nfs_create_server) from [] (nfs3_create_server+0x8/0x20) >>> [ 208.014879] [] (nfs3_create_server) from [] (nfs_try_mount+0xc4/0x1f0) >>> [ 208.023513] [] (nfs_try_mount) from [] (nfs_fs_mount+0x290/0x910) >>> [ 208.031702] [] (nfs_fs_mount) from [] (mount_fs+0x44/0x168) >>> >>> Has anyone else seen this issue? >>> >>> I'd be appreciated for any help or advice related to this issue? >> >> I did not look at details, but because it is 4.4 and __wait_on_bit >> showed up you might want to look at [1] >> >> [1] https://lkml.org/lkml/2015/11/20/472 > > Thanks. I'll take a look. > I've checked this thread and all three commits mentioned there are present in K4.4 >=3.17 commit 743162013d40 sched: Remove proliferation of wait_on_bit() action functions >=4.4 commit 68985633bccb sched/wait: Fix signal handling in bit wait helpers >=4.4 commit dfd01f026058 sched/wait: Fix the signal handling fix Also, It seems first patch, probably, has copy-past error. I'm not sure and it may be that patch is correct :) Any way, It doesn't help with this issue if I use wait_on_bit_lock_io in nfs_page_group_lock(). 743162013d40 ("sched: Remove proliferation of wait_on_bit() action functions") -- does: TASK_UNINTERRUPTIBLE); } @@ -425,9 +418,8 @@ void nfs_release_request(struct nfs_page *req) int nfs_wait_on_request(struct nfs_page *req) { - return wait_on_bit(&req->wb_flags, PG_BUSY, - nfs_wait_bit_uninterruptible, - TASK_UNINTERRUPTIBLE); + return wait_on_bit_io(&req->wb_flags, PG_BUSY, + TASK_UNINTERRUPTIBLE); } diff --git a/fs/nfs/pagelist.c b/fs/nfs/pagelist.c index b6ee3a6..6104d35 100644 --- a/fs/nfs/pagelist.c +++ b/fs/nfs/pagelist.c @@ -138,12 +138,6 @@ nfs_iocounter_wait(struct nfs_io_counter *c) return __nfs_iocounter_wait(c); } -static int nfs_wait_bit_uninterruptible(void *word) -{ - io_schedule(); - return 0; -} - /* * nfs_page_group_lock - lock the head of the page group * @req - request in group that is to be locked @@ -158,7 +152,6 @@ nfs_page_group_lock(struct nfs_page *req) WARN_ON_ONCE(head != head->wb_head); wait_on_bit_lock(&head->wb_flags, PG_HEADLOCK, - nfs_wait_bit_uninterruptible, [GS] But it seems should be wait_on_bit_lock_io() <----