| Message ID | fd69f2052bc16d5a5de0104ca00ccc59478e3569.1378750714.git.milosz@adfin.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
David, Can I get a sign off on this? Thanks, - Milosz On Mon, Sep 9, 2013 at 2:28 PM, Milosz Tanski <milosz@adfin.com> wrote: > __fscache_check_consistency() does not decrement the count of operations > active after it finishes in the success case. This leads to a hung tasks on > cookie de-registration (commonly in inode eviction). > > INFO: task kworker/1:2:4214 blocked for more than 120 seconds. > kworker/1:2 D ffff880443513fc0 0 4214 2 0x00000000 > Workqueue: ceph-msgr con_work [libceph] > ... > Call Trace: > [<ffffffff81569fc6>] ? _raw_spin_unlock_irqrestore+0x16/0x20 > [<ffffffffa0016570>] ? fscache_wait_bit_interruptible+0x30/0x30 [fscache] > [<ffffffff81568d09>] schedule+0x29/0x70 > [<ffffffffa001657e>] fscache_wait_atomic_t+0xe/0x20 [fscache] > [<ffffffff815665cf>] out_of_line_wait_on_atomic_t+0x9f/0xe0 > [<ffffffff81083560>] ? autoremove_wake_function+0x40/0x40 > [<ffffffffa0015a9c>] __fscache_relinquish_cookie+0x15c/0x310 [fscache] > [<ffffffffa00a4fae>] ceph_fscache_unregister_inode_cookie+0x3e/0x50 [ceph] > [<ffffffffa007e373>] ceph_destroy_inode+0x33/0x200 [ceph] > [<ffffffff811c13ae>] ? __fsnotify_inode_delete+0xe/0x10 > [<ffffffff8119ba1c>] destroy_inode+0x3c/0x70 > [<ffffffff8119bb69>] evict+0x119/0x1b0 > > Signed-off-by: Milosz Tanski <milosz@adfin.com> > --- > fs/fscache/cookie.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c > index 318e843..b2a86e3 100644 > --- a/fs/fscache/cookie.c > +++ b/fs/fscache/cookie.c > @@ -586,7 +586,8 @@ int __fscache_check_consistency(struct fscache_cookie *cookie) > > fscache_operation_init(op, NULL, NULL); > op->flags = FSCACHE_OP_MYTHREAD | > - (1 << FSCACHE_OP_WAITING); > + (1 << FSCACHE_OP_WAITING) | > + (1 << FSCACHE_OP_UNUSE_COOKIE); > > spin_lock(&cookie->lock); > > -- > 1.7.9.5 > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Milosz Tanski <milosz@adfin.com> wrote: > __fscache_check_consistency() does not decrement the count of operations > active after it finishes in the success case. This leads to a hung tasks on > cookie de-registration (commonly in inode eviction). > > INFO: task kworker/1:2:4214 blocked for more than 120 seconds. > kworker/1:2 D ffff880443513fc0 0 4214 2 0x00000000 > Workqueue: ceph-msgr con_work [libceph] > ... > Call Trace: > [<ffffffff81569fc6>] ? _raw_spin_unlock_irqrestore+0x16/0x20 > [<ffffffffa0016570>] ? fscache_wait_bit_interruptible+0x30/0x30 [fscache] > [<ffffffff81568d09>] schedule+0x29/0x70 > [<ffffffffa001657e>] fscache_wait_atomic_t+0xe/0x20 [fscache] > [<ffffffff815665cf>] out_of_line_wait_on_atomic_t+0x9f/0xe0 > [<ffffffff81083560>] ? autoremove_wake_function+0x40/0x40 > [<ffffffffa0015a9c>] __fscache_relinquish_cookie+0x15c/0x310 [fscache] > [<ffffffffa00a4fae>] ceph_fscache_unregister_inode_cookie+0x3e/0x50 [ceph] > [<ffffffffa007e373>] ceph_destroy_inode+0x33/0x200 [ceph] > [<ffffffff811c13ae>] ? __fsnotify_inode_delete+0xe/0x10 > [<ffffffff8119ba1c>] destroy_inode+0x3c/0x70 > [<ffffffff8119bb69>] evict+0x119/0x1b0 > > Signed-off-by: Milosz Tanski <milosz@adfin.com> Acked-by: David Howells <dhowells@redhat.com> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Sage, Can you submit to the upstream next in your next round of fixes (with David's ack). Thanks, - Milosz P.S: Thanks David. On Tue, Sep 10, 2013 at 8:34 AM, David Howells <dhowells@redhat.com> wrote: > Milosz Tanski <milosz@adfin.com> wrote: > >> __fscache_check_consistency() does not decrement the count of operations >> active after it finishes in the success case. This leads to a hung tasks on >> cookie de-registration (commonly in inode eviction). >> >> INFO: task kworker/1:2:4214 blocked for more than 120 seconds. >> kworker/1:2 D ffff880443513fc0 0 4214 2 0x00000000 >> Workqueue: ceph-msgr con_work [libceph] >> ... >> Call Trace: >> [<ffffffff81569fc6>] ? _raw_spin_unlock_irqrestore+0x16/0x20 >> [<ffffffffa0016570>] ? fscache_wait_bit_interruptible+0x30/0x30 [fscache] >> [<ffffffff81568d09>] schedule+0x29/0x70 >> [<ffffffffa001657e>] fscache_wait_atomic_t+0xe/0x20 [fscache] >> [<ffffffff815665cf>] out_of_line_wait_on_atomic_t+0x9f/0xe0 >> [<ffffffff81083560>] ? autoremove_wake_function+0x40/0x40 >> [<ffffffffa0015a9c>] __fscache_relinquish_cookie+0x15c/0x310 [fscache] >> [<ffffffffa00a4fae>] ceph_fscache_unregister_inode_cookie+0x3e/0x50 [ceph] >> [<ffffffffa007e373>] ceph_destroy_inode+0x33/0x200 [ceph] >> [<ffffffff811c13ae>] ? __fsnotify_inode_delete+0xe/0x10 >> [<ffffffff8119ba1c>] destroy_inode+0x3c/0x70 >> [<ffffffff8119bb69>] evict+0x119/0x1b0 >> >> Signed-off-by: Milosz Tanski <milosz@adfin.com> > > Acked-by: David Howells <dhowells@redhat.com> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, 10 Sep 2013, Milosz Tanski wrote: > Sage, > > Can you submit to the upstream next in your next round of fixes (with > David's ack). Yep; it's in the queue. Thanks! sage > > Thanks, > - Milosz > > P.S: Thanks David. > > On Tue, Sep 10, 2013 at 8:34 AM, David Howells <dhowells@redhat.com> wrote: > > Milosz Tanski <milosz@adfin.com> wrote: > > > >> __fscache_check_consistency() does not decrement the count of operations > >> active after it finishes in the success case. This leads to a hung tasks on > >> cookie de-registration (commonly in inode eviction). > >> > >> INFO: task kworker/1:2:4214 blocked for more than 120 seconds. > >> kworker/1:2 D ffff880443513fc0 0 4214 2 0x00000000 > >> Workqueue: ceph-msgr con_work [libceph] > >> ... > >> Call Trace: > >> [<ffffffff81569fc6>] ? _raw_spin_unlock_irqrestore+0x16/0x20 > >> [<ffffffffa0016570>] ? fscache_wait_bit_interruptible+0x30/0x30 [fscache] > >> [<ffffffff81568d09>] schedule+0x29/0x70 > >> [<ffffffffa001657e>] fscache_wait_atomic_t+0xe/0x20 [fscache] > >> [<ffffffff815665cf>] out_of_line_wait_on_atomic_t+0x9f/0xe0 > >> [<ffffffff81083560>] ? autoremove_wake_function+0x40/0x40 > >> [<ffffffffa0015a9c>] __fscache_relinquish_cookie+0x15c/0x310 [fscache] > >> [<ffffffffa00a4fae>] ceph_fscache_unregister_inode_cookie+0x3e/0x50 [ceph] > >> [<ffffffffa007e373>] ceph_destroy_inode+0x33/0x200 [ceph] > >> [<ffffffff811c13ae>] ? __fsnotify_inode_delete+0xe/0x10 > >> [<ffffffff8119ba1c>] destroy_inode+0x3c/0x70 > >> [<ffffffff8119bb69>] evict+0x119/0x1b0 > >> > >> Signed-off-by: Milosz Tanski <milosz@adfin.com> > > > > Acked-by: David Howells <dhowells@redhat.com> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c index 318e843..b2a86e3 100644 --- a/fs/fscache/cookie.c +++ b/fs/fscache/cookie.c @@ -586,7 +586,8 @@ int __fscache_check_consistency(struct fscache_cookie *cookie) fscache_operation_init(op, NULL, NULL); op->flags = FSCACHE_OP_MYTHREAD | - (1 << FSCACHE_OP_WAITING); + (1 << FSCACHE_OP_WAITING) | + (1 << FSCACHE_OP_UNUSE_COOKIE); spin_lock(&cookie->lock);
__fscache_check_consistency() does not decrement the count of operations active after it finishes in the success case. This leads to a hung tasks on cookie de-registration (commonly in inode eviction). INFO: task kworker/1:2:4214 blocked for more than 120 seconds. kworker/1:2 D ffff880443513fc0 0 4214 2 0x00000000 Workqueue: ceph-msgr con_work [libceph] ... Call Trace: [<ffffffff81569fc6>] ? _raw_spin_unlock_irqrestore+0x16/0x20 [<ffffffffa0016570>] ? fscache_wait_bit_interruptible+0x30/0x30 [fscache] [<ffffffff81568d09>] schedule+0x29/0x70 [<ffffffffa001657e>] fscache_wait_atomic_t+0xe/0x20 [fscache] [<ffffffff815665cf>] out_of_line_wait_on_atomic_t+0x9f/0xe0 [<ffffffff81083560>] ? autoremove_wake_function+0x40/0x40 [<ffffffffa0015a9c>] __fscache_relinquish_cookie+0x15c/0x310 [fscache] [<ffffffffa00a4fae>] ceph_fscache_unregister_inode_cookie+0x3e/0x50 [ceph] [<ffffffffa007e373>] ceph_destroy_inode+0x33/0x200 [ceph] [<ffffffff811c13ae>] ? __fsnotify_inode_delete+0xe/0x10 [<ffffffff8119ba1c>] destroy_inode+0x3c/0x70 [<ffffffff8119bb69>] evict+0x119/0x1b0 Signed-off-by: Milosz Tanski <milosz@adfin.com> --- fs/fscache/cookie.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)