From patchwork Wed Jan 15 20:59:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeffrey Layton X-Patchwork-Id: 11335703 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A384D1398 for ; Wed, 15 Jan 2020 20:59:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 822E9207FF for ; Wed, 15 Jan 2020 20:59:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1579121969; bh=svNmx6hp03Y3zMwqmVXglg6eeIM9J50G+e9QNevUp2k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=QHKG+QhGN+RtwcJcpktl8zTdtH1z+/SCGhCeKdTvYTT54+SpwcygHYBKRCCumb34O BWRdAj2zfZcLf0TAE4Rqa3aE6S9KdetKxgcvZ+6Wt6WDoQCNsm/SHqU69X0nL2zxO2 45Zwo5TlDGPxwXycTVzMNgNuPAlvypIg+QyI+5U8= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729637AbgAOU72 (ORCPT ); Wed, 15 Jan 2020 15:59:28 -0500 Received: from mail.kernel.org ([198.145.29.99]:58238 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729587AbgAOU7X (ORCPT ); Wed, 15 Jan 2020 15:59:23 -0500 Received: from tleilax.poochiereds.net (68-20-15-154.lightspeed.rlghnc.sbcglobal.net [68.20.15.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0DA99222C3; Wed, 15 Jan 2020 20:59:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1579121962; bh=svNmx6hp03Y3zMwqmVXglg6eeIM9J50G+e9QNevUp2k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RksfBJvnr8/GYAZ+rfHxFQnVPwIY++D09GtlKAi+IyaVFwv0EvsxsNCjk45bmkUBO G3Pp/JUPw9moMREv6cXM9jc9oWPCEH4z4jMebeHh+T82RSTNUlwbx0WQp7tIXpjEEj qewElMs1JwTrSkqDTrvSK2QVH7W98io22MEi0EoQ= From: Jeff Layton To: ceph-devel@vger.kernel.org Cc: zyan@redhat.com, sage@redhat.com, idryomov@gmail.com, pdonnell@redhat.com, xiubli@redhat.com Subject: [RFC PATCH v2 09/10] ceph: cache layout in parent dir on first sync create Date: Wed, 15 Jan 2020 15:59:11 -0500 Message-Id: <20200115205912.38688-10-jlayton@kernel.org> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200115205912.38688-1-jlayton@kernel.org> References: <20200115205912.38688-1-jlayton@kernel.org> MIME-Version: 1.0 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org It doesn't do much good to do an asynchronous create unless we can do I/O to it before the create reply comes in. That means we need layout info the new file before we've gotten the response from the MDS. All files created in a directory will initially inherit the same layout, so copy off the requisite info from the first synchronous create in the directory, and save it in a new i_cached_layout field. Zero out the layout when we lose Dc caps in the dir. Signed-off-by: Jeff Layton --- fs/ceph/caps.c | 13 ++++++++++--- fs/ceph/file.c | 22 +++++++++++++++++++++- fs/ceph/inode.c | 2 ++ fs/ceph/super.h | 1 + 4 files changed, 34 insertions(+), 4 deletions(-) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 9d1a3d6831f7..70bf50f00c27 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -561,14 +561,14 @@ static void __cap_delay_cancel(struct ceph_mds_client *mdsc, spin_unlock(&mdsc->cap_delay_lock); } -/* - * Common issue checks for add_cap, handle_cap_grant. - */ +/* Common issue checks for add_cap, handle_cap_grant. */ static void __check_cap_issue(struct ceph_inode_info *ci, struct ceph_cap *cap, unsigned issued) { unsigned had = __ceph_caps_issued(ci, NULL); + lockdep_assert_held(&ci->i_ceph_lock); + /* * Each time we receive FILE_CACHE anew, we increment * i_rdcache_gen. @@ -593,6 +593,13 @@ static void __check_cap_issue(struct ceph_inode_info *ci, struct ceph_cap *cap, __ceph_dir_clear_complete(ci); } } + + /* Wipe saved layout if we're losing DIR_CREATE caps */ + if (S_ISDIR(ci->vfs_inode.i_mode) && (had & CEPH_CAP_DIR_CREATE) && + !(issued & CEPH_CAP_DIR_CREATE)) { + ceph_put_string(rcu_dereference_raw(ci->i_cached_layout.pool_ns)); + memset(&ci->i_cached_layout, 0, sizeof(ci->i_cached_layout)); + } } /* diff --git a/fs/ceph/file.c b/fs/ceph/file.c index 1e6cdf2dfe90..b44ccbc85fe4 100644 --- a/fs/ceph/file.c +++ b/fs/ceph/file.c @@ -430,6 +430,23 @@ int ceph_open(struct inode *inode, struct file *file) return err; } +/* Clone the layout from a synchronous create, if the dir now has Dc caps */ +static void +cache_file_layout(struct inode *dst, struct inode *src) +{ + struct ceph_inode_info *cdst = ceph_inode(dst); + struct ceph_inode_info *csrc = ceph_inode(src); + + spin_lock(&cdst->i_ceph_lock); + if ((__ceph_caps_issued(cdst, NULL) & CEPH_CAP_DIR_CREATE) && + !ceph_file_layout_is_valid(&cdst->i_cached_layout)) { + memcpy(&cdst->i_cached_layout, &csrc->i_layout, + sizeof(cdst->i_cached_layout)); + rcu_assign_pointer(cdst->i_cached_layout.pool_ns, + ceph_try_get_string(csrc->i_layout.pool_ns)); + } + spin_unlock(&cdst->i_ceph_lock); +} /* * Do a lookup + open with a single request. If we get a non-existent @@ -518,7 +535,10 @@ int ceph_atomic_open(struct inode *dir, struct dentry *dentry, } else { dout("atomic_open finish_open on dn %p\n", dn); if (req->r_op == CEPH_MDS_OP_CREATE && req->r_reply_info.has_create_ino) { - ceph_init_inode_acls(d_inode(dentry), &as_ctx); + struct inode *newino = d_inode(dentry); + + cache_file_layout(dir, newino); + ceph_init_inode_acls(newino, &as_ctx); file->f_mode |= FMODE_CREATED; } err = finish_open(file, dentry, ceph_open); diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c index 4056c7968b86..73f986efb1fd 100644 --- a/fs/ceph/inode.c +++ b/fs/ceph/inode.c @@ -447,6 +447,7 @@ struct inode *ceph_alloc_inode(struct super_block *sb) ci->i_max_files = 0; memset(&ci->i_dir_layout, 0, sizeof(ci->i_dir_layout)); + memset(&ci->i_cached_layout, 0, sizeof(ci->i_cached_layout)); RCU_INIT_POINTER(ci->i_layout.pool_ns, NULL); ci->i_fragtree = RB_ROOT; @@ -587,6 +588,7 @@ void ceph_evict_inode(struct inode *inode) ceph_buffer_put(ci->i_xattrs.prealloc_blob); ceph_put_string(rcu_dereference_raw(ci->i_layout.pool_ns)); + ceph_put_string(rcu_dereference_raw(ci->i_cached_layout.pool_ns)); } static inline blkcnt_t calc_inode_blocks(u64 size) diff --git a/fs/ceph/super.h b/fs/ceph/super.h index 86dd4a2163e0..09bd4b71de91 100644 --- a/fs/ceph/super.h +++ b/fs/ceph/super.h @@ -326,6 +326,7 @@ struct ceph_inode_info { struct ceph_dir_layout i_dir_layout; struct ceph_file_layout i_layout; + struct ceph_file_layout i_cached_layout; // for async creates char *i_symlink; /* for dirs */