From patchwork Fri Dec 18 21:37:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Layton X-Patchwork-Id: 11983001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECAF8C4361B for ; Fri, 18 Dec 2020 21:38:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB39F23B54 for ; Fri, 18 Dec 2020 21:38:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725871AbgLRViR (ORCPT ); Fri, 18 Dec 2020 16:38:17 -0500 Received: from mail.kernel.org ([198.145.29.99]:57870 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725813AbgLRViR (ORCPT ); Fri, 18 Dec 2020 16:38:17 -0500 From: Jeff Layton Authentication-Results: mail.kernel.org; dkim=permerror (bad message/signature format) To: idryomov@gmail.com Cc: ceph-devel@vger.kernel.org, xiubli@redhat.com Subject: [PATCH v2] ceph: allow queueing cap/snap handling after putting cap references Date: Fri, 18 Dec 2020 16:37:34 -0500 Message-Id: <20201218213734.1335788-1-jlayton@kernel.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Testing with the fscache overhaul has triggered some lockdep warnings about circular lock dependencies involving page_mkwrite and the mmap_lock. It'd be better to do the "real work" without the mmap lock being held. Change the skip_checking_caps parameter in __ceph_put_cap_refs to an enum, and use that to determine whether to queue check_caps, do it synchronously or not at all. Change ceph_page_mkwrite to do a ceph_put_cap_refs_async(). Signed-off-by: Jeff Layton --- fs/ceph/addr.c | 2 +- fs/ceph/caps.c | 29 +++++++++++++++++++++++++---- fs/ceph/inode.c | 6 ++++++ fs/ceph/super.h | 19 ++++++++++++++++--- 4 files changed, 48 insertions(+), 8 deletions(-) v2: - get rid of camel case enum labels - minor switch rearrangement - rename "skip" enum member to PUT_CAP_REFS_NO_CHECK diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index 950552944436..26e66436f005 100644 --- a/fs/ceph/addr.c +++ b/fs/ceph/addr.c @@ -1662,7 +1662,7 @@ static vm_fault_t ceph_page_mkwrite(struct vm_fault *vmf) dout("page_mkwrite %p %llu~%zd dropping cap refs on %s ret %x\n", inode, off, len, ceph_cap_string(got), ret); - ceph_put_cap_refs(ci, got); + ceph_put_cap_refs_async(ci, got); out_free: ceph_restore_sigs(&oldset); sb_end_pagefault(inode->i_sb); diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 638d18c198ea..5902e1588228 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -3027,6 +3027,12 @@ static int ceph_try_drop_cap_snap(struct ceph_inode_info *ci, return 0; } +enum put_cap_refs_mode { + PUT_CAP_REFS_SYNC = 0, + PUT_CAP_REFS_NO_CHECK, + PUT_CAP_REFS_ASYNC, +}; + /* * Release cap refs. * @@ -3037,7 +3043,7 @@ static int ceph_try_drop_cap_snap(struct ceph_inode_info *ci, * cap_snap, and wake up any waiters. */ static void __ceph_put_cap_refs(struct ceph_inode_info *ci, int had, - bool skip_checking_caps) + enum put_cap_refs_mode mode) { struct inode *inode = &ci->vfs_inode; int last = 0, put = 0, flushsnaps = 0, wake = 0; @@ -3093,11 +3099,21 @@ static void __ceph_put_cap_refs(struct ceph_inode_info *ci, int had, dout("put_cap_refs %p had %s%s%s\n", inode, ceph_cap_string(had), last ? " last" : "", put ? " put" : ""); - if (!skip_checking_caps) { + switch(mode) { + case PUT_CAP_REFS_SYNC: if (last) ceph_check_caps(ci, 0, NULL); else if (flushsnaps) ceph_flush_snaps(ci, NULL); + break; + case PUT_CAP_REFS_ASYNC: + if (last) + ceph_queue_check_caps(inode); + else if (flushsnaps) + ceph_queue_flush_snaps(inode); + break; + default: + break; } if (wake) wake_up_all(&ci->i_cap_wq); @@ -3107,12 +3123,17 @@ static void __ceph_put_cap_refs(struct ceph_inode_info *ci, int had, void ceph_put_cap_refs(struct ceph_inode_info *ci, int had) { - __ceph_put_cap_refs(ci, had, false); + __ceph_put_cap_refs(ci, had, PUT_CAP_REFS_SYNC); +} + +void ceph_put_cap_refs_async(struct ceph_inode_info *ci, int had) +{ + __ceph_put_cap_refs(ci, had, PUT_CAP_REFS_ASYNC); } void ceph_put_cap_refs_no_check_caps(struct ceph_inode_info *ci, int had) { - __ceph_put_cap_refs(ci, had, true); + __ceph_put_cap_refs(ci, had, PUT_CAP_REFS_NO_CHECK); } /* diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c index e64ccd93350f..5d20a620e96c 100644 --- a/fs/ceph/inode.c +++ b/fs/ceph/inode.c @@ -1965,6 +1965,12 @@ static void ceph_inode_work(struct work_struct *work) if (test_and_clear_bit(CEPH_I_WORK_VMTRUNCATE, &ci->i_work_mask)) __ceph_do_pending_vmtruncate(inode); + if (test_and_clear_bit(CEPH_I_WORK_CHECK_CAPS, &ci->i_work_mask)) + ceph_check_caps(ci, 0, NULL); + + if (test_and_clear_bit(CEPH_I_WORK_FLUSH_SNAPS, &ci->i_work_mask)) + ceph_flush_snaps(ci, NULL); + iput(inode); } diff --git a/fs/ceph/super.h b/fs/ceph/super.h index 59153ee201c0..13b02887b085 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -562,9 +562,11 @@ static inline struct inode *ceph_find_inode(struct super_block *sb, /* * Masks of ceph inode work. */ -#define CEPH_I_WORK_WRITEBACK 0 /* writeback */ -#define CEPH_I_WORK_INVALIDATE_PAGES 1 /* invalidate pages */ -#define CEPH_I_WORK_VMTRUNCATE 2 /* vmtruncate */ +#define CEPH_I_WORK_WRITEBACK 0 +#define CEPH_I_WORK_INVALIDATE_PAGES 1 +#define CEPH_I_WORK_VMTRUNCATE 2 +#define CEPH_I_WORK_CHECK_CAPS 3 +#define CEPH_I_WORK_FLUSH_SNAPS 4 /* * We set the ERROR_WRITE bit when we start seeing write errors on an inode @@ -982,6 +984,16 @@ static inline void ceph_queue_writeback(struct inode *inode) ceph_queue_inode_work(inode, CEPH_I_WORK_WRITEBACK); } +static inline void ceph_queue_check_caps(struct inode *inode) +{ + ceph_queue_inode_work(inode, CEPH_I_WORK_CHECK_CAPS); +} + +static inline void ceph_queue_flush_snaps(struct inode *inode) +{ + ceph_queue_inode_work(inode, CEPH_I_WORK_FLUSH_SNAPS); +} + extern int __ceph_do_getattr(struct inode *inode, struct page *locked_page, int mask, bool force); static inline int ceph_do_getattr(struct inode *inode, int mask, bool force) @@ -1120,6 +1132,7 @@ extern void ceph_take_cap_refs(struct ceph_inode_info *ci, int caps, bool snap_rwsem_locked); extern void ceph_get_cap_refs(struct ceph_inode_info *ci, int caps); extern void ceph_put_cap_refs(struct ceph_inode_info *ci, int had); +extern void ceph_put_cap_refs_async(struct ceph_inode_info *ci, int had); extern void ceph_put_cap_refs_no_check_caps(struct ceph_inode_info *ci, int had); extern void ceph_put_wrbuffer_cap_refs(struct ceph_inode_info *ci, int nr,