From patchwork Sat Feb 8 00:30:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966168 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8CFB9C02199 for ; Sat, 8 Feb 2025 00:30:41 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWw021kFz1x5M; Fri, 07 Feb 2025 16:30:40 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWvv1961z1wsW for ; Fri, 07 Feb 2025 16:30:35 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm5-e204-208.ccs.ornl.gov [160.91.203.29]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id E47CC88F9D4; Fri, 7 Feb 2025 19:30:32 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id D9A6A106BE16; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:07 -0500 Message-ID: <20250208003027.180076-2-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 01/21] lustre: llite: implicit default LMV inherit X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lai Siyao , Hongchao Zhang , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Lai Siyao With implicit default LMV inherit, the inherited default LMV is not stored on disk, but maintained on client side. Benefits: * change of directory default LMV will be populated to all sub levels at runtime. * default LMV will be packed into mkdir request, therefore MDT doesn't need to read it from disk, as will improve mkdir performance. Caveats: * to disable inherited default LMV on subdir, a default LMV need to be set on this subdir explicitly like this: "lfs setdirstripe -D -i --max-inherit 0" Changes on client side: * update inherited default LMV after lookup/open/revalidate. * pack default LMV in mkdir request. * add "--raw" option for "lfs getdirstripe -D" to print default LMV stored in inode, if directory doesn't have default LMV, or its default LMV is implicitly inherited, nothing will be printed. Changes on MDT side: * use the default LMV from client in lod_ah_init() to mkdir. * don't save inherited default LMV in mkdir. WC-bug-id: https://jira.whamcloud.com/browse/LU-15971 Lustre-commit: 388a185eace0e6b16 ("LU-15971 llite: implicit default LMV inherit") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/47789 Reviewed-by: Andreas Dilger Reviewed-by: Hongchao Zhang Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_lmv.h | 64 ++++++---- fs/lustre/llite/dcache.c | 11 +- fs/lustre/llite/dir.c | 129 ++++++++++--------- fs/lustre/llite/file.c | 9 +- fs/lustre/llite/llite_internal.h | 9 +- fs/lustre/llite/llite_lib.c | 158 ++++++++++++++++++++---- fs/lustre/llite/namei.c | 58 +++++++-- fs/lustre/llite/statahead.c | 6 +- fs/lustre/mdc/mdc_lib.c | 7 ++ fs/lustre/obdclass/lprocfs_status.c | 2 +- fs/lustre/ptlrpc/wiretest.c | 2 + include/uapi/linux/lustre/lustre_idl.h | 2 + include/uapi/linux/lustre/lustre_user.h | 6 +- 13 files changed, 344 insertions(+), 119 deletions(-) diff --git a/fs/lustre/include/lustre_lmv.h b/fs/lustre/include/lustre_lmv.h index 5e3dbc52e846..c3b55d4b63e0 100644 --- a/fs/lustre/include/lustre_lmv.h +++ b/fs/lustre/include/lustre_lmv.h @@ -82,6 +82,33 @@ static inline bool lmv_dir_bad_hash(const struct lmv_stripe_md *lsm) return !lmv_is_known_hash_type(lsm->lsm_md_hash_type); } +static inline u8 lmv_inherit_next(u8 inherit) +{ + if (inherit == LMV_INHERIT_END || inherit == LMV_INHERIT_NONE) + return LMV_INHERIT_NONE; + + if (inherit == LMV_INHERIT_UNLIMITED || inherit > LMV_INHERIT_MAX) + return inherit; + + return inherit - 1; +} + +static inline u8 lmv_inherit_rr_next(u8 inherit_rr) +{ + if (inherit_rr == LMV_INHERIT_RR_NONE || + inherit_rr == LMV_INHERIT_RR_UNLIMITED || + inherit_rr > LMV_INHERIT_RR_MAX) + return inherit_rr; + + return inherit_rr - 1; +} + +static inline bool lmv_is_inheritable(u8 inherit) +{ + return inherit == LMV_INHERIT_UNLIMITED || + (inherit > LMV_INHERIT_END && inherit <= LMV_INHERIT_MAX); +} + static inline bool lsm_md_eq(const struct lmv_stripe_md *lsm1, const struct lmv_stripe_md *lsm2) { @@ -145,6 +172,22 @@ static inline void lsm_md_dump(int mask, const struct lmv_stripe_md *lsm) i, PFID(&lsm->lsm_md_oinfo[i].lmo_fid)); } +static inline bool +lsm_md_inherited(const struct lmv_stripe_md *plsm, + const struct lmv_stripe_md *clsm) +{ + return plsm && clsm && + plsm->lsm_md_magic == clsm->lsm_md_magic && + plsm->lsm_md_stripe_count == clsm->lsm_md_stripe_count && + plsm->lsm_md_master_mdt_index == + clsm->lsm_md_master_mdt_index && + plsm->lsm_md_hash_type == clsm->lsm_md_hash_type && + lmv_inherit_next(plsm->lsm_md_max_inherit) == + clsm->lsm_md_max_inherit && + lmv_inherit_rr_next(plsm->lsm_md_max_inherit_rr) == + clsm->lsm_md_max_inherit_rr; +} + union lmv_mds_md; void lmv_free_memmd(struct lmv_stripe_md *lsm); @@ -517,25 +560,4 @@ static inline bool lmv_is_layout_changing(const struct lmv_mds_md_v1 *lmv) lmv_hash_is_migrating(cpu_to_le32(lmv->lmv_hash_type)); } -static inline u8 lmv_inherit_next(u8 inherit) -{ - if (inherit == LMV_INHERIT_END || inherit == LMV_INHERIT_NONE) - return LMV_INHERIT_NONE; - - if (inherit == LMV_INHERIT_UNLIMITED || inherit > LMV_INHERIT_MAX) - return inherit; - - return inherit - 1; -} - -static inline u8 lmv_inherit_rr_next(u8 inherit_rr) -{ - if (inherit_rr == LMV_INHERIT_RR_NONE || - inherit_rr == LMV_INHERIT_RR_UNLIMITED || - inherit_rr > LMV_INHERIT_RR_MAX) - return inherit_rr; - - return inherit_rr - 1; -} - #endif diff --git a/fs/lustre/llite/dcache.c b/fs/lustre/llite/dcache.c index d9fb0cdc409f..1abb134f13d2 100644 --- a/fs/lustre/llite/dcache.c +++ b/fs/lustre/llite/dcache.c @@ -219,8 +219,9 @@ int ll_revalidate_it_finish(struct ptlrpc_request *request, ll_set_lock_data(ll_i2sbi(inode)->ll_md_exp, inode, it, &bits); if (bits & MDS_INODELOCK_LOOKUP) { - ll_update_dir_depth(de->d_parent->d_inode, inode); d_lustre_revalidate(de); + if (S_ISDIR(inode->i_mode)) + ll_update_dir_depth_dmv(de->d_parent->d_inode, de); } return rc; @@ -263,8 +264,11 @@ static int ll_revalidate_dentry(struct dentry *dentry, * to this dentry, then its lock has not been revoked and the * path component is valid. */ - if (lookup_flags & LOOKUP_PARENT) + if (lookup_flags & LOOKUP_PARENT) { + if (dentry->d_inode && S_ISDIR(dentry->d_inode->i_mode)) + ll_update_dir_depth_dmv(dir, dentry); return 1; + } /* Symlink - always valid as long as the dentry was found */ /* only special case is to prevent ELOOP error from VFS during open @@ -298,6 +302,9 @@ static int ll_revalidate_dentry(struct dentry *dentry, if (dentry_may_statahead(dir, dentry)) ll_revalidate_statahead(dir, &dentry, !d_inode(dentry)); + if (dentry->d_inode && S_ISDIR(dentry->d_inode->i_mode)) + ll_update_dir_depth_dmv(dir, dentry); + return 1; } diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index 0f4dc2f1ce5e..25ea565a2f0b 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -655,6 +655,64 @@ int ll_dir_setstripe(struct inode *inode, struct lov_user_md *lump, return rc; } +/* get default LMV from client cache */ +static int ll_dir_get_default_lmv(struct inode *inode, struct lmv_user_md *lum) +{ + struct ll_inode_info *lli = ll_i2info(inode); + const struct lmv_stripe_md *lsm; + bool fs_dmv_got = false; + int rc = -ENODATA; + +retry: + if (lli->lli_default_lsm_md) { + down_read(&lli->lli_lsm_sem); + lsm = lli->lli_default_lsm_md; + if (lsm) { + lum->lum_magic = lsm->lsm_md_magic; + lum->lum_stripe_count = lsm->lsm_md_stripe_count; + lum->lum_stripe_offset = lsm->lsm_md_master_mdt_index; + lum->lum_hash_type = lsm->lsm_md_hash_type; + lum->lum_max_inherit = lsm->lsm_md_max_inherit; + lum->lum_max_inherit_rr = lsm->lsm_md_max_inherit_rr; + rc = 0; + } + up_read(&lli->lli_lsm_sem); + } + + if (rc == -ENODATA && !is_root_inode(inode) && !fs_dmv_got) { + lli = ll_i2info(inode->i_sb->s_root->d_inode); + fs_dmv_got = true; + goto retry; + } + + if (!rc && fs_dmv_got) { + lli = ll_i2info(inode); + if (lum->lum_max_inherit != LMV_INHERIT_UNLIMITED) { + if (lum->lum_max_inherit == LMV_INHERIT_NONE || + lum->lum_max_inherit < LMV_INHERIT_END || + lum->lum_max_inherit > LMV_INHERIT_MAX || + lum->lum_max_inherit <= lli->lli_dir_depth) { + rc = -ENODATA; + goto out; + } + lum->lum_max_inherit -= lli->lli_dir_depth; + } + + if (lum->lum_max_inherit_rr != LMV_INHERIT_RR_UNLIMITED) { + if (lum->lum_max_inherit_rr == LMV_INHERIT_NONE || + lum->lum_max_inherit_rr < LMV_INHERIT_RR_END || + lum->lum_max_inherit_rr > LMV_INHERIT_RR_MAX || + lum->lum_max_inherit_rr <= lli->lli_dir_depth) + lum->lum_max_inherit_rr = LMV_INHERIT_RR_NONE; + + if (lum->lum_max_inherit_rr > lli->lli_dir_depth) + lum->lum_max_inherit_rr -= lli->lli_dir_depth; + } + } +out: + return rc; +} + int ll_dir_get_default_layout(struct inode *inode, void **plmm, int *plmm_size, struct ptlrpc_request **request, u64 valid, enum get_default_layout_type type) @@ -1634,7 +1692,6 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) struct lmv_user_md __user *ulmv = uarg; struct lmv_user_md lum; struct ptlrpc_request *request = NULL; - struct ptlrpc_request *root_request = NULL; struct lmv_user_md *tmp = NULL; union lmv_mds_md *lmm = NULL; u64 valid = 0; @@ -1650,6 +1707,19 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) if (copy_from_user(&lum, ulmv, sizeof(*ulmv))) return -EFAULT; + /* get default LMV */ + if (lum.lum_magic == LMV_USER_MAGIC && + lum.lum_type != LMV_TYPE_RAW) { + rc = ll_dir_get_default_lmv(inode, &lum); + if (rc) + return rc; + + if (copy_to_user(ulmv, &lum, sizeof(lum))) + return -EFAULT; + + return 0; + } + max_stripe_count = lum.lum_stripe_count; /* * lum_magic will indicate which stripe the ioctl will like @@ -1664,64 +1734,12 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) return -EINVAL; rc = ll_dir_getstripe_default(inode, (void **)&lmm, &lmmsize, - &request, &root_request, valid); + &request, NULL, valid); if (rc) goto finish_req; - /* Get default LMV EA */ + /* Get default LMV EA in raw mode */ if (lum.lum_magic == LMV_USER_MAGIC) { - struct lmv_user_md *lum; - struct ll_inode_info *lli; - - if (lmmsize > sizeof(*ulmv)) { - rc = -EINVAL; - goto finish_req; - } - - lum = (struct lmv_user_md *)lmm; - if (lum->lum_max_inherit == LMV_INHERIT_NONE) { - rc = -ENODATA; - goto finish_req; - } - - if (root_request) { - lli = ll_i2info(inode); - if (lum->lum_max_inherit != - LMV_INHERIT_UNLIMITED) { - if (lum->lum_max_inherit < - LMV_INHERIT_END || - lum->lum_max_inherit > - LMV_INHERIT_MAX || - lum->lum_max_inherit <= - lli->lli_dir_depth) { - rc = -ENODATA; - goto finish_req; - } - - lum->lum_max_inherit -= - lli->lli_dir_depth; - } - - if (lum->lum_max_inherit_rr != - LMV_INHERIT_RR_UNLIMITED) { - if (lum->lum_max_inherit_rr == - LMV_INHERIT_NONE || - lum->lum_max_inherit_rr < - LMV_INHERIT_RR_END || - lum->lum_max_inherit_rr > - LMV_INHERIT_RR_MAX || - lum->lum_max_inherit_rr <= - lli->lli_dir_depth) { - lum->lum_max_inherit_rr = - LMV_INHERIT_RR_NONE; - goto out_copy; - } - - lum->lum_max_inherit_rr -= - lli->lli_dir_depth; - } - } -out_copy: if (copy_to_user(ulmv, lmm, lmmsize)) rc = -EFAULT; @@ -1812,7 +1830,6 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) kfree(tmp); finish_req: ptlrpc_req_finished(request); - ptlrpc_req_finished(root_request); return rc; } case LL_IOC_RMFID: diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 7899ffdd92fb..aee529098497 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -693,10 +693,8 @@ static int ll_intent_file_open(struct dentry *de, void *lmm, int lmmsize, * of kernel will deal with that later. */ ll_set_lock_data(sbi->ll_md_exp, inode, itp, &bits); - if (bits & MDS_INODELOCK_LOOKUP) { + if (bits & MDS_INODELOCK_LOOKUP) d_lustre_revalidate(de); - ll_update_dir_depth(parent->d_inode, d_inode(de)); - } /* if DoM bit returned along with LAYOUT bit then there * can be read-on-open data returned. @@ -704,6 +702,11 @@ static int ll_intent_file_open(struct dentry *de, void *lmm, int lmmsize, if (bits & MDS_INODELOCK_DOM && bits & MDS_INODELOCK_LAYOUT) ll_dom_finish_open(inode, req); } + /* open may not fetch LOOKUP lock, update dir depth and default LMV + * anyway. + */ + if (!rc && S_ISDIR(de->d_inode->i_mode)) + ll_update_dir_depth_dmv(parent->d_inode, de); out: ptlrpc_req_finished(req); diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index 93743d27739c..746ef983edb8 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -196,7 +196,12 @@ struct ll_inode_info { * statahead hit ratio is too low, or start statahead * thread failed. */ - unsigned short lli_sa_enabled:1; + unsigned short lli_sa_enabled:1, + /* default LMV is explicitly set in inode on MDT, this + * is for old server, or default LMV is set by + * "lfs setdirstripe -D". + */ + lli_default_lmv_set:1; /* generation for statahead */ unsigned int lli_sa_generation; /* rw lock protects lli_lsm_md */ @@ -1293,7 +1298,7 @@ int ll_statfs_internal(struct ll_sb_info *sbi, struct obd_statfs *osfs, u32 flags); int ll_update_inode(struct inode *inode, struct lustre_md *md); void ll_update_inode_flags(struct inode *inode, unsigned int ext_flags); -void ll_update_dir_depth(struct inode *dir, struct inode *inode); +void ll_update_dir_depth_dmv(struct inode *dir, struct dentry *de); int ll_read_inode2(struct inode *inode, void *opaque); void ll_truncate_inode_pages_final(struct inode *inode, struct cl_io *io); void ll_delete_inode(struct inode *inode); diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index b158eccc16c8..afb231ecd70b 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -327,7 +327,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt) OBD_CONNECT2_DOM_LVB | OBD_CONNECT2_REP_MBITS | OBD_CONNECT2_ATOMIC_OPEN_LOCK | - OBD_CONNECT2_BATCH_RPC; + OBD_CONNECT2_BATCH_RPC | + OBD_CONNECT2_DMV_IMP_INHERIT; if (test_bit(LL_SBI_LRU_RESIZE, sbi->ll_flags)) data->ocd_connect_flags |= OBD_CONNECT_LRU_RESIZE; @@ -1607,13 +1608,15 @@ static void ll_update_default_lsm_md(struct inode *inode, struct lustre_md *md) if (!md->default_lmv) { /* clear default lsm */ - if (lli->lli_default_lsm_md) { + if (lli->lli_default_lsm_md && lli->lli_default_lmv_set) { down_write(&lli->lli_lsm_sem); - if (lli->lli_default_lsm_md) { + if (lli->lli_default_lsm_md && + lli->lli_default_lmv_set) { lmv_free_memmd(lli->lli_default_lsm_md); lli->lli_default_lsm_md = NULL; + lli->lli_inherit_depth = 0; + lli->lli_default_lmv_set = 0; } - lli->lli_inherit_depth = 0; up_write(&lli->lli_lsm_sem); } return; @@ -1634,6 +1637,7 @@ static void ll_update_default_lsm_md(struct inode *inode, struct lustre_md *md) if (lli->lli_default_lsm_md) lmv_free_memmd(lli->lli_default_lsm_md); lli->lli_default_lsm_md = md->default_lmv; + lli->lli_default_lmv_set = 1; lsm_md_dump(D_INODE, md->default_lmv); md->default_lmv = NULL; up_write(&lli->lli_lsm_sem); @@ -2735,38 +2739,150 @@ static inline bool ll_default_lmv_inherited(struct lmv_stripe_md *pdmv, return true; } -/* update directory depth to ROOT, called after LOOKUP lock is fetched. */ -void ll_update_dir_depth(struct inode *dir, struct inode *inode) +/* if default LMV is implicitly inherited, subdir default LMV is maintained on + * client side. + */ +int ll_dir_default_lmv_inherit(struct inode *dir, struct inode *inode) { + struct ll_inode_info *plli = ll_i2info(dir); + struct ll_inode_info *lli = ll_i2info(inode); + struct lmv_stripe_md *plsm; + struct lmv_stripe_md *lsm; + int rc = 0; + + /* ROOT default LMV is not inherited */ + if (is_root_inode(dir) || + !(exp_connect_flags2(ll_i2mdexp(dir)) & + OBD_CONNECT2_DMV_IMP_INHERIT)) + return 0; + + /* nothing to do if no default LMV on both */ + if (!plli->lli_default_lsm_md && !lli->lli_default_lsm_md) + return 0; + + /* subdir default LMV comes from disk */ + if (lli->lli_default_lsm_md && lli->lli_default_lmv_set) + return 0; + + /* delete subdir default LMV if parent's is deleted or becomes + * uninheritable. + */ + down_read(&plli->lli_lsm_sem); + plsm = plli->lli_default_lsm_md; + if (!plsm || !lmv_is_inheritable(plsm->lsm_md_max_inherit)) { + if (lli->lli_default_lsm_md && !lli->lli_default_lmv_set) { + down_write(&lli->lli_lsm_sem); + if (lli->lli_default_lsm_md && + !lli->lli_default_lmv_set) { + lmv_free_memmd(lli->lli_default_lsm_md); + lli->lli_default_lsm_md = NULL; + lli->lli_inherit_depth = 0; + } + up_write(&lli->lli_lsm_sem); + } + rc = 0; + goto unlock_parent; + } + + /* do nothing if inherited LMV is unchanged */ + if (lli->lli_default_lsm_md) { + rc = 1; + down_read(&lli->lli_lsm_sem); + if (!lli->lli_default_lmv_set) + rc = lsm_md_inherited(plsm, lli->lli_default_lsm_md); + up_read(&lli->lli_lsm_sem); + if (rc == 1) { + rc = 0; + goto unlock_parent; + } + } + + /* inherit default LMV */ + down_write(&lli->lli_lsm_sem); + if (lli->lli_default_lsm_md) { + /* checked above, but in case of race, check again with lock */ + if (lli->lli_default_lmv_set) { + rc = 0; + goto unlock_child; + } + /* always update subdir default LMV in case parent's changed */ + lsm = lli->lli_default_lsm_md; + } else { + lsm = kzalloc(sizeof(*lsm), GFP_NOFS); + if (!lsm) { + rc = -ENOMEM; + goto unlock_child; + } + lli->lli_default_lsm_md = lsm; + } + + *lsm = *plsm; + lsm->lsm_md_max_inherit = lmv_inherit_next(plsm->lsm_md_max_inherit); + lsm->lsm_md_max_inherit_rr = + lmv_inherit_rr_next(plsm->lsm_md_max_inherit_rr); + lli->lli_inherit_depth = plli->lli_inherit_depth + 1; + + lsm_md_dump(D_INODE, lsm); + +unlock_child: + up_write(&lli->lli_lsm_sem); +unlock_parent: + up_read(&plli->lli_lsm_sem); + + return rc; +} + +/** + * Update directory depth and default LMV + * + * Update directory depth to ROOT and inherit default LMV from parent if + * parent's default LMV is inheritable. The default LMV set with command + * "lfs setdirstripe -D ..." is stored on MDT, while the inherited default LMV + * is generated at runtime on client side. + * + * \param[in] dir parent directory inode + * \param[in] de dentry + */ +void ll_update_dir_depth_dmv(struct inode *dir, struct dentry *de) +{ + struct inode *inode = de->d_inode; struct ll_inode_info *plli; struct ll_inode_info *lli; - if (!S_ISDIR(inode->i_mode)) - return; - + LASSERT(S_ISDIR(inode->i_mode)); if (inode == dir) return; plli = ll_i2info(dir); lli = ll_i2info(inode); lli->lli_dir_depth = plli->lli_dir_depth + 1; - if (plli->lli_default_lsm_md && lli->lli_default_lsm_md) { - down_read(&plli->lli_lsm_sem); - down_read(&lli->lli_lsm_sem); - if (ll_default_lmv_inherited(plli->lli_default_lsm_md, + if (lli->lli_default_lsm_md && lli->lli_default_lmv_set) { + if (plli->lli_default_lsm_md) { + down_read(&plli->lli_lsm_sem); + down_read(&lli->lli_lsm_sem); + if (lsm_md_inherited(plli->lli_default_lsm_md, lli->lli_default_lsm_md)) - lli->lli_inherit_depth = - plli->lli_inherit_depth + 1; - else + lli->lli_inherit_depth = + plli->lli_inherit_depth + 1; + else + /* in case parent default LMV changed */ + lli->lli_inherit_depth = 0; + up_read(&lli->lli_lsm_sem); + up_read(&plli->lli_lsm_sem); + } else { + /* in case parent default LMV deleted */ lli->lli_inherit_depth = 0; - up_read(&lli->lli_lsm_sem); - up_read(&plli->lli_lsm_sem); + } } else { - lli->lli_inherit_depth = 0; + ll_dir_default_lmv_inherit(dir, inode); } - CDEBUG(D_INODE, DFID" depth %hu default LMV depth %hu\n", - PFID(&lli->lli_fid), lli->lli_dir_depth, lli->lli_inherit_depth); + if (lli->lli_default_lsm_md) + CDEBUG(D_INODE, + "%s "DFID" depth %hu %s default LMV inherit depth %hu\n", + de->d_name.name, PFID(&lli->lli_fid), lli->lli_dir_depth, + lli->lli_default_lmv_set ? "server" : "client", + lli->lli_inherit_depth); } void ll_truncate_inode_pages_final(struct inode *inode, struct cl_io *io) diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index 85a5902b0598..657ad6fd961d 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -726,10 +726,13 @@ static int ll_lookup_it_finish(struct ptlrpc_request *request, if (!it_disposition(it, DISP_LOOKUP_NEG)) { /* We have the "lookup" lock, so unhide dentry */ - if (bits & MDS_INODELOCK_LOOKUP) { + if (bits & MDS_INODELOCK_LOOKUP) d_lustre_revalidate(*de); - ll_update_dir_depth(parent, d_inode(*de)); - } + /* open may not fetch LOOKUP lock, update dir depth/dmv anyway + * in case it's used uninitialized. + */ + if (S_ISDIR(inode->i_mode)) + ll_update_dir_depth_dmv(parent, *de); if (encrypt) { rc = fscrypt_get_encryption_info(inode); @@ -1424,10 +1427,11 @@ static int ll_create_it(struct inode *dir, struct dentry *dentry, ll_set_lock_data(ll_i2sbi(dir)->ll_md_exp, inode, it, &bits); if (bits & MDS_INODELOCK_LOOKUP) { d_lustre_revalidate(dentry); - ll_update_dir_depth(dir, inode); + if (S_ISDIR(inode->i_mode)) + ll_update_dir_depth_dmv(dir, dentry); } - return rc; + return 0; } void ll_update_times(struct ptlrpc_request *request, struct inode *inode) @@ -1517,6 +1521,9 @@ static int ll_new_node(struct inode *dir, struct dentry *dchild, struct ll_sb_info *sbi = ll_i2sbi(dir); struct fscrypt_str *disk_link = NULL; bool encrypt = false; + struct lmv_user_md *lum = NULL; + const void *data = NULL; + size_t datalen = 0; int err; if (unlikely(tgt)) { @@ -1524,6 +1531,8 @@ static int ll_new_node(struct inode *dir, struct dentry *dchild, rdev = 0; if (!disk_link) return -EINVAL; + data = disk_link->name; + datalen = disk_link->len; } again: @@ -1534,8 +1543,37 @@ static int ll_new_node(struct inode *dir, struct dentry *dchild, goto err_exit; } - if (S_ISDIR(mode)) + if (S_ISDIR(mode)) { ll_qos_mkdir_prep(op_data, dir); + if ((exp_connect_flags2(ll_i2mdexp(dir)) & + OBD_CONNECT2_DMV_IMP_INHERIT) && + op_data->op_default_mea1 && !lum) { + const struct lmv_stripe_md *lsm; + + /* once DMV_IMP_INHERIT is set, pack default LMV in + * create request. + */ + lum = kzalloc(sizeof(*lum), GFP_NOFS); + if (!lum) { + err = -ENOMEM; + goto err_exit; + } + lsm = op_data->op_default_mea1; + lum->lum_magic = cpu_to_le32(lsm->lsm_md_magic); + lum->lum_stripe_count = + cpu_to_le32(lsm->lsm_md_stripe_count); + lum->lum_stripe_offset = + cpu_to_le32(lsm->lsm_md_master_mdt_index); + lum->lum_hash_type = + cpu_to_le32(lsm->lsm_md_hash_type); + lum->lum_max_inherit = lsm->lsm_md_max_inherit; + lum->lum_max_inherit_rr = lsm->lsm_md_max_inherit_rr; + lum->lum_pool_name[0] = 0; + op_data->op_bias |= MDS_CREATE_DEFAULT_LMV; + data = lum; + datalen = sizeof(*lum); + } + } if (test_bit(LL_SBI_FILE_SECCTX, sbi->ll_flags)) { err = ll_dentry_init_security(dchild, mode, &dchild->d_name, @@ -1596,11 +1634,13 @@ static int ll_new_node(struct inode *dir, struct dentry *dchild, dchild->d_sb->s_op->destroy_inode(fakeinode); if (err) goto err_exit; + + data = disk_link->name; + datalen = disk_link->len; } } - err = md_create(sbi->ll_md_exp, op_data, tgt ? disk_link->name : NULL, - tgt ? disk_link->len : 0, mode, + err = md_create(sbi->ll_md_exp, op_data, data, datalen, mode, from_kuid(&init_user_ns, current_fsuid()), from_kgid(&init_user_ns, current_fsgid()), current_cap(), rdev, &request); @@ -1727,9 +1767,9 @@ static int ll_new_node(struct inode *dir, struct dentry *dchild, err_exit: if (request) ptlrpc_req_finished(request); - if (!IS_ERR_OR_NULL(op_data)) ll_finish_md_op_data(op_data); + kfree(lum); return err; } diff --git a/fs/lustre/llite/statahead.c b/fs/lustre/llite/statahead.c index c820455cc3af..9399b676e582 100644 --- a/fs/lustre/llite/statahead.c +++ b/fs/lustre/llite/statahead.c @@ -1612,10 +1612,10 @@ static int revalidate_statahead_dentry(struct inode *dir, goto out_unplug; } - if ((bits & MDS_INODELOCK_LOOKUP) && - d_lustre_invalid(*dentryp)) { + if (bits & MDS_INODELOCK_LOOKUP) { d_lustre_revalidate(*dentryp); - ll_update_dir_depth(dir, (*dentryp)->d_inode); + if (S_ISDIR(inode->i_mode)) + ll_update_dir_depth_dmv(dir, *dentryp); } ll_intent_release(&it); diff --git a/fs/lustre/mdc/mdc_lib.c b/fs/lustre/mdc/mdc_lib.c index 077639d5127c..a0d35f0b4932 100644 --- a/fs/lustre/mdc/mdc_lib.c +++ b/fs/lustre/mdc/mdc_lib.c @@ -219,6 +219,13 @@ void mdc_create_pack(struct req_capsule *pill, struct md_op_data *op_data, * create only, and don't restripe if object exists. */ flags |= MDS_OPEN_CREAT; + if (op_data->op_bias & MDS_CREATE_DEFAULT_LMV) { + /* borrow MDS_OPEN_DEFAULT_LMV flag to indicate parent default + * LMV is packed in create request. + */ + flags |= MDS_OPEN_DEFAULT_LMV; + LASSERT(data); + } set_mrc_cr_flags(rec, flags); rec->cr_bias = op_data->op_bias; rec->cr_umask = current_umask(); diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index 0d669f4dde15..bde287bbb793 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -138,7 +138,7 @@ static const char *const obd_connect_names[] = { "atomic_open_lock", /* 0x4000000 */ "name_encryption", /* 0x8000000 */ "mkdir_replay", /* 0x10000000 */ - "dmv_inherit", /* 0x20000000 */ + "dmv_imp_inherit", /* 0x20000000 */ "encryption_fid2path", /* 0x40000000 */ NULL }; diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 45adb33a9c56..8cec6a35c692 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -2164,6 +2164,8 @@ void lustre_assert_wire_constants(void) (unsigned int)MDS_FID_OP); LASSERTF(MDS_MIGRATE_NSONLY == 0x00800000UL, "found 0x%.8xUL\n", (unsigned int)MDS_MIGRATE_NSONLY); + LASSERTF(MDS_CREATE_DEFAULT_LMV == 0x01000000UL, "found 0x%.8xUL\n", + (unsigned int)MDS_CREATE_DEFAULT_LMV); /* Checks for struct mdt_body */ LASSERTF((int)sizeof(struct mdt_body) == 216, "found %lld\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 187a807d4809..2a7299e6ce46 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -1702,6 +1702,8 @@ enum mds_op_bias { MDS_FID_OP = 1 << 22, /* migrate dirent only */ MDS_MIGRATE_NSONLY = 1 << 23, + /* create with default LMV from client */ + MDS_CREATE_DEFAULT_LMV = 1 << 24, }; #define MDS_CLOSE_INTENT (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP | \ diff --git a/include/uapi/linux/lustre/lustre_user.h b/include/uapi/linux/lustre/lustre_user.h index 4b49bb9c370a..397d93705d98 100644 --- a/include/uapi/linux/lustre/lustre_user.h +++ b/include/uapi/linux/lustre/lustre_user.h @@ -901,6 +901,8 @@ static inline __u32 lmv_foreign_to_md_stripes(__u32 size) */ enum lmv_type { LMV_TYPE_DEFAULT = 0x0000, + /* fetch raw default LMV set on directory inode */ + LMV_TYPE_RAW = 0x0001, }; /* lum_max_inherit will be decreased by 1 after each inheritance if it's not @@ -1303,7 +1305,9 @@ enum la_valid { * for newly created file */ #define MDS_OP_WITH_FID 020000000000000ULL /* operation carried out by FID */ -#define MDS_OPEN_DEFAULT_LMV 040000000000000ULL /* open fetches default LMV */ +#define MDS_OPEN_DEFAULT_LMV 040000000000000ULL /* open fetches default LMV, + * or mkdir with default LMV + */ /* lustre internal open flags, which should not be set from user space */ #define MDS_OPEN_FL_INTERNAL (MDS_OPEN_HAS_EA | MDS_OPEN_HAS_OBJS | \ From patchwork Sat Feb 8 00:30:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84396C02199 for ; Sat, 8 Feb 2025 00:32:57 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWw62DJwz1xdK; Fri, 07 Feb 2025 16:30:46 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWvx6K0Tz1x5M for ; Fri, 07 Feb 2025 16:30:37 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm2-e204-208.ccs.ornl.gov [160.91.203.3]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id E5B4C17B5B8; Fri, 7 Feb 2025 19:30:32 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id DA3EF106BE17; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:08 -0500 Message-ID: <20250208003027.180076-3-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 02/21] lnet: report actual timeout used by lnd X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Frank Sehr , Cyril Bordage , Serguei Smirnov , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Frank Sehr lnd_timeout value reported by lnetctl may be different from what is actually used. There's an lnd_timeout calculated as a function of transaction timeout and retry_count. This is the value displayed by "lnetctl global show". However, each LND may define its own timeout by setting timeout module parameter to a positive value, which overrides the higher-level lnd_timeout defined by LNet. lnetctl net show -v will show the timeout value in the lnd_tunables section. The timeout for socklnd and o2iblnd is implemented. WC-bug-id: https://jira.whamcloud.com/browse/LU-16548 Lustre-commit: 56097c490465cb67a ("LU-16548 lnet: report actual timeout used by lnd") Signed-off-by: Frank Sehr Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50620 Reviewed-by: Neil Brown Reviewed-by: Serguei Smirnov Reviewed-by: Cyril Bordage Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lnet/lnet-dlc.h | 2 ++ net/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 2 ++ net/lnet/klnds/socklnd/socklnd_modparams.c | 2 ++ 3 files changed, 6 insertions(+) diff --git a/include/uapi/linux/lnet/lnet-dlc.h b/include/uapi/linux/lnet/lnet-dlc.h index d7592f3fd1f8..b4851b193a90 100644 --- a/include/uapi/linux/lnet/lnet-dlc.h +++ b/include/uapi/linux/lnet/lnet-dlc.h @@ -105,12 +105,14 @@ struct lnet_ioctl_config_o2iblnd_tunables { __u32 lnd_fmr_cache; __u16 lnd_conns_per_peer; __u16 lnd_ntx; + __u32 lnd_timeout; }; struct lnet_ioctl_config_socklnd_tunables { __u32 lnd_version; __u16 lnd_conns_per_peer; __u16 lnd_pad; + __u32 lnd_timeout; }; struct lnet_lnd_tunables { diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c index 04286e18c7ba..e26bedee00ba 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c @@ -316,6 +316,8 @@ int kiblnd_tunables_setup(struct lnet_ni *ni) conns_per_peer : 1; } + tunables->lnd_timeout = kiblnd_timeout(); + return 0; } diff --git a/net/lnet/klnds/socklnd/socklnd_modparams.c b/net/lnet/klnds/socklnd/socklnd_modparams.c index 031fe13d2a38..ed6c60e88c84 100644 --- a/net/lnet/klnds/socklnd/socklnd_modparams.c +++ b/net/lnet/klnds/socklnd/socklnd_modparams.c @@ -332,4 +332,6 @@ void ksocknal_tunables_setup(struct lnet_ni *ni) if (!tunables->lnd_conns_per_peer) tunables->lnd_conns_per_peer = ksocklnd_lookup_conns_per_peer(ni); + + tunables->lnd_timeout = ksocknal_timeout(); } From patchwork Sat Feb 8 00:30:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9AFA4C02199 for ; Sat, 8 Feb 2025 00:31:00 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWw46Ymhz1xRx; Fri, 07 Feb 2025 16:30:44 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWvw2Bkvz1wsW for ; Fri, 07 Feb 2025 16:30:36 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm4-e204-208.ccs.ornl.gov [160.91.203.22]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id E9BFF88F9DB; Fri, 7 Feb 2025 19:30:32 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id DEF3D106BE18; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:09 -0500 Message-ID: <20250208003027.180076-4-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 03/21] lustre: ptlrpc: replace OBD_ -> CFS_ macros X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arshad Hussain , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Timothy Day Replace OBD macros that are simply redefinitions of CFS macros. WC-bug-id: https://jira.whamcloud.com/browse/LU-12610 Lustre-commit: 3e24d50046e59a552 ("LU-12610 ptlrpc: replace OBD_ -> CFS_ macros") Signed-off-by: Timothy Day Signed-off-by: Ben Evans Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50684 Reviewed-by: Andreas Dilger Reviewed-by: Arshad Hussain Reviewed-by: Neil Brown Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/client.c | 20 ++++++++++---------- fs/lustre/ptlrpc/import.c | 12 ++++++------ fs/lustre/ptlrpc/niobuf.c | 21 +++++++++++---------- fs/lustre/ptlrpc/pinger.c | 2 +- fs/lustre/ptlrpc/ptlrpcd.c | 2 +- fs/lustre/ptlrpc/recover.c | 4 ++-- fs/lustre/ptlrpc/sec_plain.c | 4 ++-- fs/lustre/ptlrpc/service.c | 16 ++++++++-------- 8 files changed, 41 insertions(+), 40 deletions(-) diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 7a267e67e45c..80eed1b67b9b 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -1829,13 +1829,13 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) LASSERT(req->rq_next_phase != RQ_PHASE_UNDEFINED); if (req->rq_req_deadline && - !OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REQ_UNLINK)) + !CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REQ_UNLINK)) req->rq_req_deadline = 0; if (req->rq_reply_deadline && - !OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK)) + !CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK)) req->rq_reply_deadline = 0; if (req->rq_bulk_deadline && - !OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK)) + !CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK)) req->rq_bulk_deadline = 0; /* @@ -1856,13 +1856,13 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) * Turn fail_loc off to prevent it from looping * forever. */ - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK)) { - OBD_FAIL_CHECK_ORSET(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK, - OBD_FAIL_ONCE); + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK)) { + CFS_FAIL_CHECK_ORSET(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK, + CFS_FAIL_ONCE); } - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK)) { - OBD_FAIL_CHECK_ORSET(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK, - OBD_FAIL_ONCE); + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK)) { + CFS_FAIL_CHECK_ORSET(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK, + CFS_FAIL_ONCE); } /* Move to next phase if reply was successfully @@ -2702,7 +2702,7 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) LASSERT(!in_interrupt()); /* Let's setup deadline for reply unlink. */ - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK) && + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK) && async && request->rq_reply_deadline == 0 && cfs_fail_val == 0) request->rq_reply_deadline = ktime_get_real_seconds() + PTLRPC_REQ_LONG_UNLINK; diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 697b3c36fb1b..47e805b773ab 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -298,8 +298,8 @@ void ptlrpc_invalidate_import(struct obd_import *imp) if (!imp->imp_invalid || imp->imp_obd->obd_no_recov) ptlrpc_deactivate_import(imp); - if (OBD_FAIL_PRECHECK(OBD_FAIL_PTLRPC_CONNECT_RACE)) { - OBD_RACE(OBD_FAIL_PTLRPC_CONNECT_RACE); + if (CFS_FAIL_PRECHECK(OBD_FAIL_PTLRPC_CONNECT_RACE)) { + CFS_RACE(OBD_FAIL_PTLRPC_CONNECT_RACE); msleep(10 * MSEC_PER_SEC); } CFS_FAIL_TIMEOUT(OBD_FAIL_MGS_CONNECT_NET, 3 * cfs_fail_val / 2); @@ -316,7 +316,7 @@ void ptlrpc_invalidate_import(struct obd_import *imp) * out. Use obd_timeout if calculated value is smaller * than it. */ - if (!OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK)) { + if (!CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK)) { timeout = ptlrpc_inflight_timeout(imp); timeout += div_u64(timeout, 3); @@ -1436,7 +1436,7 @@ static int signal_completed_replay(struct obd_import *imp) { struct ptlrpc_request *req; - if (unlikely(OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_FINISH_REPLAY))) + if (unlikely(CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_FINISH_REPLAY))) return 0; if (!atomic_add_unless(&imp->imp_replay_inflight, 1, 1)) @@ -1835,12 +1835,12 @@ int ptlrpc_disconnect_and_idle_import(struct obd_import *imp) req->rq_interpret_reply = ptlrpc_disconnect_idle_interpret; - if (OBD_FAIL_PRECHECK(OBD_FAIL_PTLRPC_IDLE_RACE)) { + if (CFS_FAIL_PRECHECK(OBD_FAIL_PTLRPC_IDLE_RACE)) { u32 idx; server_name2index(imp->imp_obd->obd_name, &idx, NULL); if (idx == 0) - OBD_RACE(OBD_FAIL_PTLRPC_IDLE_RACE); + CFS_RACE(OBD_FAIL_PTLRPC_IDLE_RACE); } spin_lock(&imp->imp_lock); diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index ccc2caab3876..d426d3c678b7 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -68,10 +68,11 @@ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, md.options |= LNET_MD_BULK_HANDLE; } - if (unlikely(ack == LNET_ACK_REQ && - OBD_FAIL_CHECK_ORSET(OBD_FAIL_PTLRPC_ACK, OBD_FAIL_ONCE))) + if (CFS_FAIL_CHECK_ORSET(OBD_FAIL_PTLRPC_ACK, OBD_FAIL_ONCE) && + ack == LNET_ACK_REQ) { /* don't ask for the ack to simulate failing client */ ack = LNET_NOACK_REQ; + } rc = LNetMDBind(&md, LNET_UNLINK, mdh); if (unlikely(rc != 0)) { @@ -128,7 +129,7 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) struct lnet_me *me; struct lnet_md md; - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_BULK_GET_NET)) + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_BULK_GET_NET)) return 0; /* NB no locking required until desc is on the network */ @@ -183,7 +184,7 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) ptlrpc_fill_bulk_md(&md, desc, posted_md); if (posted_md > 0 && posted_md + 1 == desc->bd_md_count && - OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_BULK_ATTACH)) { + CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_BULK_ATTACH)) { rc = -ENOMEM; } else { me = LNetMEAttach(desc->bd_portal, &peer, mbits, 0, @@ -255,7 +256,7 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) desc->bd_registered = 0; /* Let's setup deadline for reply unlink. */ - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK) && + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK) && async && req->rq_bulk_deadline == 0 && cfs_fail_val == 0) req->rq_bulk_deadline = ktime_get_real_seconds() + PTLRPC_REQ_LONG_UNLINK; @@ -504,7 +505,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) bulk_cookie.cookie = LNET_WIRE_HANDLE_COOKIE_NONE; - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_DROP_RPC)) + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_DROP_RPC)) return 0; LASSERT(request->rq_type == PTL_RPC_MSG_REQUEST); @@ -640,7 +641,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) peer = connection->c_peer; if (request->rq_bulk && - OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_BULK_REPLY_ATTACH)) { + CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_BULK_REPLY_ATTACH)) { reply_me = ERR_PTR(-ENOMEM); } else { reply_me = LNetMEAttach(request->rq_reply_portal, @@ -715,7 +716,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) lprocfs_counter_add(obd->obd_svc_stats, PTLRPC_REQACTIVE_CNTR, atomic_read(&imp->imp_inflight)); - OBD_FAIL_TIMEOUT(OBD_FAIL_PTLRPC_DELAY_SEND, request->rq_timeout + 5); + CFS_FAIL_TIMEOUT(OBD_FAIL_PTLRPC_DELAY_SEND, request->rq_timeout + 5); request->rq_sent_ns = ktime_get_real(); request->rq_sent = ktime_get_real_seconds(); @@ -726,7 +727,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) ptlrpc_at_get_net_latency(request); if (unlikely(opc == OBD_PING && - OBD_FAIL_TIMEOUT(OBD_FAIL_PTLRPC_DELAY_SEND_FAIL, cfs_fail_val))) + CFS_FAIL_TIMEOUT(OBD_FAIL_PTLRPC_DELAY_SEND_FAIL, cfs_fail_val))) goto skip_send; DEBUG_REQ(D_INFO, request, "send flags=%x", @@ -790,7 +791,7 @@ int ptlrpc_register_rqbd(struct ptlrpc_request_buffer_desc *rqbd) CDEBUG(D_NET, "%s: registering portal %d\n", service->srv_name, service->srv_req_portal); - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_RQBD)) + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_RQBD)) return -ENOMEM; /* NB: CPT affinity service should use new LNet flag LNET_INS_LOCAL, diff --git a/fs/lustre/ptlrpc/pinger.c b/fs/lustre/ptlrpc/pinger.c index 76a08442c870..5bcf00469c12 100644 --- a/fs/lustre/ptlrpc/pinger.c +++ b/fs/lustre/ptlrpc/pinger.c @@ -155,7 +155,7 @@ static int ptlrpc_ping(struct obd_import *imp) static inline int imp_is_deactive(struct obd_import *imp) { return (imp->imp_deactive || - OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_IMP_DEACTIVE)); + CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_IMP_DEACTIVE)); } static inline time64_t ptlrpc_next_reconnect(struct obd_import *imp) diff --git a/fs/lustre/ptlrpc/ptlrpcd.c b/fs/lustre/ptlrpc/ptlrpcd.c index 7342db8e56a3..f9152752f4dc 100644 --- a/fs/lustre/ptlrpc/ptlrpcd.c +++ b/fs/lustre/ptlrpc/ptlrpcd.c @@ -60,7 +60,7 @@ #include #include #include /* for obd_zombie */ -#include /* for OBD_FAIL_CHECK */ +#include #include /* cl_env_{get,put}() */ #include diff --git a/fs/lustre/ptlrpc/recover.c b/fs/lustre/ptlrpc/recover.c index 6ba31c133ed0..4af8388d1cbc 100644 --- a/fs/lustre/ptlrpc/recover.c +++ b/fs/lustre/ptlrpc/recover.c @@ -200,7 +200,7 @@ int ptlrpc_resend(struct obd_import *imp) } spin_unlock(&imp->imp_lock); - OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_ENQUEUE_OLD_EXPORT, 2); + CFS_FAIL_TIMEOUT(OBD_FAIL_LDLM_ENQUEUE_OLD_EXPORT, 2); return 0; } @@ -330,7 +330,7 @@ int ptlrpc_recover_import(struct obd_import *imp, char *new_uuid, int async) if (rc) goto out; - OBD_RACE(OBD_FAIL_PTLRPC_CONNECT_RACE); + CFS_RACE(OBD_FAIL_PTLRPC_CONNECT_RACE); rc = ptlrpc_connect_import(imp); if (rc) diff --git a/fs/lustre/ptlrpc/sec_plain.c b/fs/lustre/ptlrpc/sec_plain.c index d546722ce398..858d71d7f432 100644 --- a/fs/lustre/ptlrpc/sec_plain.c +++ b/fs/lustre/ptlrpc/sec_plain.c @@ -313,7 +313,7 @@ int plain_cli_wrap_bulk(struct ptlrpc_cli_ctx *ctx, * for sending we only compute the wrong checksum instead * of corrupting the data so it is still correct on a redo */ - if (OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_SEND) && + if (CFS_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_SEND) && req->rq_flvr.u_bulk.hash.hash_alg != BULK_HASH_ALG_NULL) token->pbt_hash[0] ^= 0x1; } @@ -944,7 +944,7 @@ int plain_svc_wrap_bulk(struct ptlrpc_request *req, CERROR("bulk read: server failed to compute checksum: %d\n", rc); } else { - if (OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_RECEIVE)) + if (CFS_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_RECEIVE)) corrupt_bulk_data(desc); } diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index bf7627227a5a..3acf1bbed0ef 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -1610,7 +1610,7 @@ static int ptlrpc_server_handle_req_in(struct ptlrpc_service_part *svcpt, } opc = lustre_msg_get_opc(req->rq_reqmsg); - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_DROP_REQ_OPC) && + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_DROP_REQ_OPC) && opc == cfs_fail_val) { CERROR("drop incoming rpc opc %u, x%llu\n", cfs_fail_val, req->rq_xid); @@ -1691,10 +1691,10 @@ static int ptlrpc_server_handle_req_in(struct ptlrpc_service_part *svcpt, } - if (unlikely(OBD_FAIL_PRECHECK(OBD_FAIL_PTLRPC_ENQ_RESEND) && + if (unlikely(CFS_FAIL_PRECHECK(OBD_FAIL_PTLRPC_ENQ_RESEND) && (opc == LDLM_ENQUEUE) && (lustre_msg_get_flags(req->rq_reqmsg) & MSG_RESENT))) - OBD_FAIL_TIMEOUT(OBD_FAIL_PTLRPC_ENQ_RESEND, 6); + CFS_FAIL_TIMEOUT(OBD_FAIL_PTLRPC_ENQ_RESEND, 6); ptlrpc_at_add_timed(req); @@ -1738,19 +1738,19 @@ static int ptlrpc_server_handle_request(struct ptlrpc_service_part *svcpt, if (!request) return 0; - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_HPREQ_NOTIMEOUT)) + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_HPREQ_NOTIMEOUT)) fail_opc = OBD_FAIL_PTLRPC_HPREQ_NOTIMEOUT; - else if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_HPREQ_TIMEOUT)) + else if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_HPREQ_TIMEOUT)) fail_opc = OBD_FAIL_PTLRPC_HPREQ_TIMEOUT; if (unlikely(fail_opc)) { if (request->rq_export && request->rq_ops) - OBD_FAIL_TIMEOUT(fail_opc, 4); + CFS_FAIL_TIMEOUT(fail_opc, 4); } ptlrpc_rqphase_move(request, RQ_PHASE_INTERPRET); - if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_DUMP_LOG)) + if (CFS_FAIL_CHECK(OBD_FAIL_PTLRPC_DUMP_LOG)) libcfs_debug_dumplog(); work_start = ktime_get_real(); @@ -2633,7 +2633,7 @@ static int ptlrpc_start_thread(struct ptlrpc_service_part *svcpt, int wait) return -ESRCH; if (!ptlrpc_threads_increasable(svcpt) || - (OBD_FAIL_CHECK(OBD_FAIL_TGT_TOOMANY_THREADS) && + (CFS_FAIL_CHECK(OBD_FAIL_TGT_TOOMANY_THREADS) && svcpt->scp_nthrs_running == svc->srv_nthrs_cpt_init - 1)) return -EMFILE; From patchwork Sat Feb 8 00:30:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69F50C02199 for ; Sat, 8 Feb 2025 00:33:39 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWw72lJdz1xf0; Fri, 07 Feb 2025 16:30:47 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw12H5pz1xNt for ; Fri, 07 Feb 2025 16:30:41 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm5-e204-208.ccs.ornl.gov [160.91.203.29]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id EB84017B5BA; Fri, 7 Feb 2025 19:30:32 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id E3CB6106BE19; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:10 -0500 Message-ID: <20250208003027.180076-5-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 04/21] lustre: misc: remove OBD_ -> CFS_ macros X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Timothy Day Remove OBD macros that are simply redefinitions of CFS macros. WC-bug-id: https://jira.whamcloud.com/browse/LU-12610 Lustre-commit: 424fa1bff070d0382 ("LU-12610 misc: remove OBD_ -> CFS_ macros") Signed-off-by: Timothy Day Signed-off-by: Ben Evans Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50809 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Neil Brown Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/fid/fid_request.c | 2 +- fs/lustre/fld/fld_request.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/lustre/fid/fid_request.c b/fs/lustre/fid/fid_request.c index c41932fadc7d..78525bac252f 100644 --- a/fs/lustre/fid/fid_request.c +++ b/fs/lustre/fid/fid_request.c @@ -210,7 +210,7 @@ int seq_client_alloc_fid(const struct lu_env *env, spin_lock(&seq->lcs_lock); - if (OBD_FAIL_CHECK(OBD_FAIL_SEQ_EXHAUST)) + if (CFS_FAIL_CHECK(OBD_FAIL_SEQ_EXHAUST)) seq->lcs_fid.f_oid = seq->lcs_width; wait_event_cmd(seq->lcs_waitq, diff --git a/fs/lustre/fld/fld_request.c b/fs/lustre/fld/fld_request.c index bafd5a9f4aad..0fe8cef6e945 100644 --- a/fs/lustre/fld/fld_request.c +++ b/fs/lustre/fld/fld_request.c @@ -364,7 +364,7 @@ int fld_client_rpc(struct obd_export *exp, req->rq_reply_portal = MDC_REPLY_PORTAL; ptlrpc_at_set_req_timeout(req); - if (OBD_FAIL_CHECK(OBD_FAIL_FLD_QUERY_REQ) && req->rq_no_delay) { + if (CFS_FAIL_CHECK(OBD_FAIL_FLD_QUERY_REQ) && req->rq_no_delay) { /* the same error returned by ptlrpc_import_delay_req */ rc = -EAGAIN; req->rq_status = rc; From patchwork Sat Feb 8 00:30:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CC845C02199 for ; Sat, 8 Feb 2025 00:34:13 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWw86qn5z1xGG; Fri, 07 Feb 2025 16:30:48 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWvx6FG2z1x3n for ; Fri, 07 Feb 2025 16:30:37 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm-e204-208.ccs.ornl.gov [160.91.203.12]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id F29D288F9DC; Fri, 7 Feb 2025 19:30:32 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id E7B47106BE1A; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:11 -0500 Message-ID: <20250208003027.180076-6-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 05/21] lnet: libcfs: move percpt_lock into lnet X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arshad Hussain , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown lnet is the only users of percpt_lock - and there are only two such locks! So move the code into lnet, as part of deprecating libcfs. WC-bug-id: https://jira.whamcloud.com/browse/LU-9859 Lustre-commit: c4e2563ff3bfa84ab ("LU-9859 libcfs: move percpt_lock into lnet") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50832 Reviewed-by: James Simmons Reviewed-by: Arshad Hussain Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/libcfs/libcfs_cpu.h | 62 ------------------ include/linux/lnet/lib-lnet.h | 3 +- include/linux/lnet/lock.h | 64 +++++++++++++++++++ net/lnet/libcfs/Makefile | 2 +- net/lnet/lnet/Makefile | 9 +-- .../{libcfs/libcfs_lock.c => lnet/lock.c} | 5 +- 6 files changed, 73 insertions(+), 72 deletions(-) create mode 100644 include/linux/lnet/lock.h rename net/lnet/{libcfs/libcfs_lock.c => lnet/lock.c} (96%) diff --git a/include/linux/libcfs/libcfs_cpu.h b/include/linux/libcfs/libcfs_cpu.h index b4f1b5879e73..bd7ea98cb38f 100644 --- a/include/linux/libcfs/libcfs_cpu.h +++ b/include/linux/libcfs/libcfs_cpu.h @@ -333,68 +333,6 @@ int cfs_percpt_number(void *vars); for (i = 0; i < cfs_percpt_number(vars) && \ ((var) = (vars)[i]) != NULL; i++) -/* - * percpu partition lock - * - * There are some use-cases like this in Lustre: - * . each CPU partition has it's own private data which is frequently changed, - * and mostly by the local CPU partition. - * . all CPU partitions share some global data, these data are rarely changed. - * - * LNet is typical example. - * CPU partition lock is designed for this kind of use-cases: - * . each CPU partition has it's own private lock - * . change on private data just needs to take the private lock - * . read on shared data just needs to take _any_ of private locks - * . change on shared data needs to take _all_ private locks, - * which is slow and should be really rare. - */ -enum { - CFS_PERCPT_LOCK_EX = -1, /* negative */ -}; - -struct cfs_percpt_lock { - /* cpu-partition-table for this lock */ - struct cfs_cpt_table *pcl_cptab; - /* exclusively locked */ - unsigned int pcl_locked; - /* private lock table */ - spinlock_t **pcl_locks; -}; - -/* return number of private locks */ -#define cfs_percpt_lock_num(pcl) cfs_cpt_number(pcl->pcl_cptab) - -/* - * create a cpu-partition lock based on CPU partition table @cptab, - * each private lock has extra @psize bytes padding data - */ -struct cfs_percpt_lock *cfs_percpt_lock_create(struct cfs_cpt_table *cptab, - struct lock_class_key *keys); -/* destroy a cpu-partition lock */ -void cfs_percpt_lock_free(struct cfs_percpt_lock *pcl); - -/* lock private lock @index of @pcl */ -void cfs_percpt_lock(struct cfs_percpt_lock *pcl, int index); - -/* unlock private lock @index of @pcl */ -void cfs_percpt_unlock(struct cfs_percpt_lock *pcl, int index); - -#define CFS_PERCPT_LOCK_KEYS 256 - -/* NB: don't allocate keys dynamically, lockdep needs them to be in ".data" */ -#define cfs_percpt_lock_alloc(cptab) \ -({ \ - static struct lock_class_key ___keys[CFS_PERCPT_LOCK_KEYS]; \ - struct cfs_percpt_lock *___lk; \ - \ - if (cfs_cpt_number(cptab) > CFS_PERCPT_LOCK_KEYS) \ - ___lk = cfs_percpt_lock_create(cptab, NULL); \ - else \ - ___lk = cfs_percpt_lock_create(cptab, ___keys); \ - ___lk; \ -}) - /** * iterate over all CPU partitions in @cptab */ diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 84bb3f2966a7..09cf42995f39 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -49,6 +49,8 @@ #include #include +#include "lock.h" + /* LNET has 0xeXXX */ #define CFS_FAIL_PTLRPC_OST_BULK_CB2 0xe000 @@ -1208,7 +1210,6 @@ u32 lnet_sum_stats(struct lnet_element_stats *stats, void lnet_usr_translate_stats(struct lnet_ioctl_element_msg_stats *msg_stats, struct lnet_element_stats *stats); - static inline void lnet_set_route_aliveness(struct lnet_route *route, bool alive) { diff --git a/include/linux/lnet/lock.h b/include/linux/lnet/lock.h new file mode 100644 index 000000000000..e5e431ccf7c0 --- /dev/null +++ b/include/linux/lnet/lock.h @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* This file is part of Lustre, http://www.lustre.org/ + * + * percpu partition lock + * + * There are some use-cases like this in Lustre: + * . each CPU partition has it's own private data which is frequently changed, + * and mostly by the local CPU partition. + * . all CPU partitions share some global data, these data are rarely changed. + * + * LNet is typical example. + * CPU partition lock is designed for this kind of use-cases: + * . each CPU partition has it's own private lock + * . change on private data just needs to take the private lock + * . read on shared data just needs to take _any_ of private locks + * . change on shared data needs to take _all_ private locks, + * which is slow and should be really rare. + */ + +enum { + CFS_PERCPT_LOCK_EX = -1, /* negative */ +}; + +struct cfs_percpt_lock { + /* cpu-partition-table for this lock */ + struct cfs_cpt_table *pcl_cptab; + /* exclusively locked */ + unsigned int pcl_locked; + /* private lock table */ + spinlock_t **pcl_locks; +}; + +/* return number of private locks */ +#define cfs_percpt_lock_num(pcl) cfs_cpt_number(pcl->pcl_cptab) + +/* create a cpu-partition lock based on CPU partition table \a cptab, + * each private lock has extra \a psize bytes padding data + */ +struct cfs_percpt_lock *cfs_percpt_lock_create(struct cfs_cpt_table *cptab, + struct lock_class_key *keys); +/* destroy a cpu-partition lock */ +void cfs_percpt_lock_free(struct cfs_percpt_lock *pcl); + +/* lock private lock \a index of \a pcl */ +void cfs_percpt_lock(struct cfs_percpt_lock *pcl, int index); + +/* unlock private lock \a index of \a pcl */ +void cfs_percpt_unlock(struct cfs_percpt_lock *pcl, int index); + +#define CFS_PERCPT_LOCK_KEYS 256 + +/* NB: don't allocate keys dynamically, lockdep needs them to be in ".data" */ +#define cfs_percpt_lock_alloc(cptab) \ +({ \ + static struct lock_class_key ___keys[CFS_PERCPT_LOCK_KEYS]; \ + struct cfs_percpt_lock *___lk; \ + \ + if (cfs_cpt_number(cptab) > CFS_PERCPT_LOCK_KEYS) \ + ___lk = cfs_percpt_lock_create(cptab, NULL); \ + else \ + ___lk = cfs_percpt_lock_create(cptab, ___keys); \ + ___lk; \ +}) diff --git a/net/lnet/libcfs/Makefile b/net/lnet/libcfs/Makefile index 15a686d76873..0753ea8e618a 100644 --- a/net/lnet/libcfs/Makefile +++ b/net/lnet/libcfs/Makefile @@ -8,6 +8,6 @@ libcfs-obj-y += linux-crypto-adler.o libcfs-obj-y += debug.o fail.o module.o tracefile.o libcfs-obj-y += libcfs_string.o hash.o libcfs-obj-$(CONFIG_SMP) += libcfs_cpu.o -libcfs-obj-y += libcfs_mem.o libcfs_lock.o +libcfs-obj-y += libcfs_mem.o libcfs-objs := $(libcfs-obj-y) diff --git a/net/lnet/lnet/Makefile b/net/lnet/lnet/Makefile index 9918008ad2d4..e22114bcd817 100644 --- a/net/lnet/lnet/Makefile +++ b/net/lnet/lnet/Makefile @@ -2,7 +2,8 @@ obj-$(CONFIG_LNET) += lnet.o -lnet-y := api-ni.o config.o nidstrings.o net_fault.o udsp.o \ - lib-me.o lib-msg.o lib-md.o lib-ptl.o \ - lib-socket.o lib-move.o module.o lo.o \ - router.o router_proc.o acceptor.o peer.o +lnet-y := api-ni.o config.o nidstrings.o lock.o \ + lib-me.o lib-msg.o lib-md.o lib-ptl.o \ + lib-socket.o lib-move.o module.o lo.o \ + router.o router_proc.o acceptor.o peer.o \ + net_fault.o udsp.o diff --git a/net/lnet/libcfs/libcfs_lock.c b/net/lnet/lnet/lock.c similarity index 96% rename from net/lnet/libcfs/libcfs_lock.c rename to net/lnet/lnet/lock.c index 8af77b14ab00..d9d2af597233 100644 --- a/net/lnet/libcfs/libcfs_lock.c +++ b/net/lnet/lnet/lock.c @@ -27,10 +27,7 @@ #define DEBUG_SUBSYSTEM S_LNET -#include -#include -#include -#include +#include /** destroy cpu-partition lock, see libcfs_private.h for more detail */ void From patchwork Sat Feb 8 00:30:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17D9CC02199 for ; Sat, 8 Feb 2025 00:34:37 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWwC4cZLz1xt4; Fri, 07 Feb 2025 16:30:51 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw21Cr0z1xPK for ; Fri, 07 Feb 2025 16:30:42 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm3-e204-208.ccs.ornl.gov [160.91.203.26]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id F24F6179486; Fri, 7 Feb 2025 19:30:32 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id EB704106BE1B; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:12 -0500 Message-ID: <20250208003027.180076-7-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 06/21] lustre: osc: Ensure there's no eviction with long cache discard X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Nunez , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Oleg Drokin Just pause execution while doing page processing for discard if appropriate failloc is set. WC-bug-id: https://jira.whamcloud.com/browse/LU-14711 Lustre-commit: c0a7f78529e21c9ca ("LU-14711 tests: Ensure there's no eviction with long cache discard") Signed-off-by: Oleg Drokin Reviewed-on: https://review.whamcloud.com/43869 Reviewed-by: Andreas Dilger Reviewed-by: James Nunez Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 1 + fs/lustre/osc/osc_cache.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index bd5ccea30b2a..f1e2e3bbb247 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -336,6 +336,7 @@ extern char obd_jobid_var[]; #define OBD_FAIL_OSC_DELAY_IO 0x414 #define OBD_FAIL_OSC_NO_SIZE_DATA 0x415 #define OBD_FAIL_OSC_DELAY_CANCEL 0x416 +#define OBD_FAIL_OSC_SLOW_PAGE_EVICT 0x417 #define OBD_FAIL_PTLRPC 0x500 #define OBD_FAIL_PTLRPC_ACK 0x501 diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index f1f21c28765e..37624a7a99d6 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -3196,6 +3196,9 @@ bool osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io, if (!res) break; + OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_SLOW_PAGE_EVICT, + cfs_fail_val ?: 20); + if (io->ci_type == CIT_MISC && io->u.ci_misc.lm_next_rpc_time && ktime_get_seconds() > io->u.ci_misc.lm_next_rpc_time) { From patchwork Sat Feb 8 00:30:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966170 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5CCA5C02199 for ; Sat, 8 Feb 2025 00:30:54 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWw30y9bz1xPb; Fri, 07 Feb 2025 16:30:43 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw12S7pz1xNy for ; Fri, 07 Feb 2025 16:30:41 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm3-e204-208.ccs.ornl.gov [160.91.203.26]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 011AE88F9DD; Fri, 7 Feb 2025 19:30:32 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id F09D1106BE14; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:13 -0500 Message-ID: <20250208003027.180076-8-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 07/21] lustre: osc: remove OBD_ -> CFS_ macros X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Timothy Day Remove OBD macros that are simply redefinitions of CFS macros. WC-bug-id: https://jira.whamcloud.com/browse/LU-12610 Lustre-commit: ef01e035b3307e20e ("LU-12610 osc: remove OBD_ -> CFS_ macros") Signed-off-by: Timothy Day Signed-off-by: Ben Evans Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51124 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/osc/osc_cache.c | 4 ++-- fs/lustre/osc/osc_io.c | 2 +- fs/lustre/osc/osc_lock.c | 8 ++++---- fs/lustre/osc/osc_request.c | 26 +++++++++++++------------- 4 files changed, 20 insertions(+), 20 deletions(-) diff --git a/fs/lustre/osc/osc_cache.c b/fs/lustre/osc/osc_cache.c index 37624a7a99d6..41a07e5d1d07 100644 --- a/fs/lustre/osc/osc_cache.c +++ b/fs/lustre/osc/osc_cache.c @@ -1638,7 +1638,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, /* force the caller to try sync io. this can jump the list * of queued writes and create a discontiguous rpc stream */ - if (OBD_FAIL_CHECK(OBD_FAIL_OSC_NO_GRANT) || + if (CFS_FAIL_CHECK(OBD_FAIL_OSC_NO_GRANT) || !cli->cl_dirty_max_pages || cli->cl_ar.ar_force_sync || loi->loi_ar.ar_force_sync) { OSC_DUMP_GRANT(D_CACHE, cli, "forced sync i/o\n"); @@ -3196,7 +3196,7 @@ bool osc_page_gang_lookup(const struct lu_env *env, struct cl_io *io, if (!res) break; - OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_SLOW_PAGE_EVICT, + CFS_FAIL_TIMEOUT(OBD_FAIL_OSC_SLOW_PAGE_EVICT, cfs_fail_val ?: 20); if (io->ci_type == CIT_MISC && diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index 71bfe60ad40a..900edf8b9a41 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -899,7 +899,7 @@ int osc_io_write_start(const struct lu_env *env, struct cl_attr *attr = &osc_env_info(env)->oti_attr; int rc = 0; - OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_SETTIME, 1); + CFS_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_SETTIME, 1); cl_object_attr_lock(obj); attr->cat_ctime = ktime_get_real_seconds(); attr->cat_mtime = attr->cat_ctime; diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index 181edf286739..3eac1d44fb2d 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -412,7 +412,7 @@ static int __osc_dlm_blocking_ast(const struct lu_env *env, unlock_res_and_lock(dlmlock); - OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_CANCEL, 5); + CFS_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_CANCEL, 5); /* if l_ast_data is NULL, the dlmlock was enqueued by AGL or * the object has been destroyed. @@ -586,7 +586,7 @@ int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) dlmlock = NULL; if (!obj && res->lr_type == LDLM_EXTENT) { - if (OBD_FAIL_CHECK(OBD_FAIL_OSC_NO_SIZE_DATA)) + if (CFS_FAIL_CHECK(OBD_FAIL_OSC_NO_SIZE_DATA)) break; lock_res(res); @@ -1025,8 +1025,8 @@ static int osc_lock_enqueue(const struct lu_env *env, if (osc_lock_is_lockless(oscl)) { oio->oi_lockless = 1; } else if (!async) { - if (OBD_FAIL_PRECHECK(OBD_FAIL_PTLRPC_IDLE_RACE)) { - OBD_RACE(OBD_FAIL_PTLRPC_IDLE_RACE); + if (CFS_FAIL_PRECHECK(OBD_FAIL_PTLRPC_IDLE_RACE)) { + CFS_RACE(OBD_FAIL_PTLRPC_IDLE_RACE); set_current_state(TASK_UNINTERRUPTIBLE); schedule_timeout(HZ / 2); } diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 128b7d15d760..3298a9923ea2 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -1243,7 +1243,7 @@ static int osc_checksum_bulk_t10pi(const char *obd_name, int nob, * simulate an OST->client data error */ if (unlikely(i == 0 && opc == OST_READ && - OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_RECEIVE))) { + CFS_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_RECEIVE))) { unsigned char *ptr = kmap(pga[i]->pg); memcpy(ptr + off, "bad1", min_t(typeof(nob), 4, nob)); @@ -1292,7 +1292,7 @@ static int osc_checksum_bulk_t10pi(const char *obd_name, int nob, * of corrupting the data so it is still correct on a redo */ if (opc == OST_WRITE && - OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_SEND)) + CFS_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_SEND)) cksum++; *check_sum = cksum; @@ -1328,7 +1328,7 @@ static int osc_checksum_bulk(int nob, u32 pg_count, * simulate an OST->client data error */ if (i == 0 && opc == OST_READ && - OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_RECEIVE)) { + CFS_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_RECEIVE)) { unsigned char *ptr = kmap(pga[i]->pg); int off = pga[i]->off & ~PAGE_MASK; @@ -1356,7 +1356,7 @@ static int osc_checksum_bulk(int nob, u32 pg_count, /* For sending we only compute the wrong checksum instead * of corrupting the data so it is still correct on a redo */ - if (opc == OST_WRITE && OBD_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_SEND)) + if (opc == OST_WRITE && CFS_FAIL_CHECK(OBD_FAIL_OSC_CHECKSUM_SEND)) (*cksum)++; return 0; @@ -1537,9 +1537,9 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, if (clpage->cp_type == CPT_TRANSIENT) directio = true; } - if (OBD_FAIL_CHECK(OBD_FAIL_OSC_BRW_PREP_REQ)) + if (CFS_FAIL_CHECK(OBD_FAIL_OSC_BRW_PREP_REQ)) return -ENOMEM; /* Recoverable */ - if (OBD_FAIL_CHECK(OBD_FAIL_OSC_BRW_PREP_REQ2)) + if (CFS_FAIL_CHECK(OBD_FAIL_OSC_BRW_PREP_REQ2)) return -EINVAL; /* Fatal */ if ((cmd & OBD_BRW_WRITE) != 0) { @@ -1804,7 +1804,7 @@ static int osc_brw_prep_request(int cmd, struct client_obd *cli, if (inode && IS_ENCRYPTED(inode) && fscrypt_has_encryption_key(inode) && - !OBD_FAIL_CHECK(OBD_FAIL_LFSCK_NO_ENCFLAG)) { + !CFS_FAIL_CHECK(OBD_FAIL_LFSCK_NO_ENCFLAG)) { if ((body->oa.o_valid & OBD_MD_FLFLAGS) == 0) { body->oa.o_valid |= OBD_MD_FLFLAGS; body->oa.o_flags = 0; @@ -2846,7 +2846,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, cmd == OBD_BRW_READ ? "read" : "write", starting_offset, ending_offset, cli->cl_r_in_flight, cli->cl_w_in_flight); } - OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_IO, cfs_fail_val); + CFS_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_IO, cfs_fail_val); ptlrpcd_add_req(req); rc = 0; @@ -2987,10 +2987,10 @@ int osc_enqueue_interpret(const struct lu_env *env, struct ptlrpc_request *req, ldlm_lock_addref(lockh, mode); /* Let cl_lock_state_wait fail with -ERESTARTSYS to unuse sublocks. */ - OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_ENQUEUE_HANG, 2); + CFS_FAIL_TIMEOUT(OBD_FAIL_LDLM_ENQUEUE_HANG, 2); /* Let CP AST to grant the lock first. */ - OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 1); + CFS_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_ENQ_RACE, 1); if (aa->oa_speculative) { LASSERT(!aa->oa_lvb); @@ -3006,7 +3006,7 @@ int osc_enqueue_interpret(const struct lu_env *env, struct ptlrpc_request *req, rc = osc_enqueue_fini(req, aa->oa_upcall, aa->oa_cookie, lockh, mode, aa->oa_flags, aa->oa_speculative, rc); - OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_CANCEL_RACE, 10); + CFS_FAIL_TIMEOUT(OBD_FAIL_OSC_CP_CANCEL_RACE, 10); ldlm_lock_decref(lockh, mode); LDLM_LOCK_PUT(lock); @@ -3165,7 +3165,7 @@ int osc_match_base(const struct lu_env *env, struct obd_export *exp, u64 lflags = *flags; enum ldlm_mode rc; - if (OBD_FAIL_CHECK(OBD_FAIL_OSC_MATCH)) + if (CFS_FAIL_CHECK(OBD_FAIL_OSC_MATCH)) return -EIO; /* Filesystem lock extents are extended to page boundaries so that @@ -3432,7 +3432,7 @@ int osc_set_info_async(const struct lu_env *env, struct obd_export *exp, char *tmp; int rc; - OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_SHUTDOWN, 10); + CFS_FAIL_TIMEOUT(OBD_FAIL_OSC_SHUTDOWN, 10); if (KEY_IS(KEY_CHECKSUM)) { if (vallen != sizeof(int)) From patchwork Sat Feb 8 00:30:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53B51C02199 for ; Sat, 8 Feb 2025 00:40:10 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWxX2cMyz1yD4; Fri, 07 Feb 2025 16:32:00 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw32l5cz1xPd for ; Fri, 07 Feb 2025 16:30:43 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm2-e204-208.ccs.ornl.gov [160.91.203.3]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 039F5179487; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 006BC106BE16; Fri, 7 Feb 2025 19:30:32 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:14 -0500 Message-ID: <20250208003027.180076-9-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 08/21] lustre: update version to 2.15.56 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Oleg Drokin New tag 2.15.56 Signed-off-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lustre/lustre_ver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/lustre/lustre_ver.h b/include/uapi/linux/lustre/lustre_ver.h index 4a72d705db68..39d228286498 100644 --- a/include/uapi/linux/lustre/lustre_ver.h +++ b/include/uapi/linux/lustre/lustre_ver.h @@ -3,9 +3,9 @@ #define LUSTRE_MAJOR 2 #define LUSTRE_MINOR 15 -#define LUSTRE_PATCH 55 +#define LUSTRE_PATCH 56 #define LUSTRE_FIX 0 -#define LUSTRE_VERSION_STRING "2.15.55" +#define LUSTRE_VERSION_STRING "2.15.56" #define OBD_OCD_VERSION(major, minor, patch, fix) \ (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix)) From patchwork Sat Feb 8 00:30:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 29247C02199 for ; Sat, 8 Feb 2025 00:37:12 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWwG4FtWz1y26; Fri, 07 Feb 2025 16:30:54 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw21XQcz1xPM for ; Fri, 07 Feb 2025 16:30:42 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm-e204-208.ccs.ornl.gov [160.91.203.12]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 078D388F9E9; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 04586106BE17; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:15 -0500 Message-ID: <20250208003027.180076-10-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 09/21] lnet: socklnd: remove ksnr_myiface from ksock_conn_cb X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Frank Sehr , Cyril Bordage , Serguei Smirnov , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Serguei Smirnov Drop ksnr_myiface: it is no longer needed since socklnd TCP bonding got removed. There's one interface per connection cb per peer_ni, and it can be accessed as net->ksnn_interface.ksni_index. Fix setting of ksni_nroutes accordingly. Duplication of interface index in conn_cb and ksnn_interface was causing the assertion ASSERTION( net->ksnn_interface.ksni_nroutes == 0 ) in ksocknal_shutdown() to fail if the corresponding device is deregistered before lnd shutdown. Fixes: a669fa61 ("lnet: handles unregister/register events) WC-bug-id: https://jira.whamcloud.com/browse/LU-16850 Lustre-commit: f6be07c457385cfac ("LU-16850 socklnd: remove ksnr_myiface from ksock_conn_cb") Signed-off-by: Serguei Smirnov Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51148 Reviewed-by: Frank Sehr Reviewed-by: Cyril Bordage Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/socklnd/socklnd.c | 130 ++-------------------------- net/lnet/klnds/socklnd/socklnd.h | 1 - net/lnet/klnds/socklnd/socklnd_cb.c | 4 +- 3 files changed, 12 insertions(+), 123 deletions(-) diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index 435762f84f27..6028520308ff 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -47,79 +47,6 @@ static struct lnet_lnd the_ksocklnd; struct ksock_nal_data ksocknal_data; -static struct ksock_interface * -ksocknal_index2iface(struct lnet_ni *ni, int index) -{ - struct ksock_net *net = ni->ni_data; - struct ksock_interface *iface; - - iface = &net->ksnn_interface; - - if (iface->ksni_index == index) - return iface; - - return NULL; -} - -static int ksocknal_ip2index(struct sockaddr *addr, struct lnet_ni *ni) -{ - struct net_device *dev; - int ret = -1; - const struct in_ifaddr *ifa; - - if (addr->sa_family != AF_INET && addr->sa_family != AF_INET6) - return ret; - - rcu_read_lock(); - for_each_netdev(ni->ni_net_ns, dev) { - int flags = dev_get_flags(dev); - struct in_device *in_dev; - - if (flags & IFF_LOOPBACK) /* skip the loopback IF */ - continue; - - if (!(flags & IFF_UP)) - continue; - - switch (addr->sa_family) { - case AF_INET: - in_dev = __in_dev_get_rcu(dev); - if (!in_dev) - continue; - - in_dev_for_each_ifa_rcu(ifa, in_dev) { - if (ifa->ifa_local == - ((struct sockaddr_in *)addr)->sin_addr.s_addr) - ret = dev->ifindex; - } - break; -#if IS_ENABLED(CONFIG_IPV6) - case AF_INET6: { - struct inet6_dev *in6_dev; - const struct inet6_ifaddr *ifa6; - struct sockaddr_in6 *addr6 = (struct sockaddr_in6 *)addr; - - in6_dev = __in6_dev_get(dev); - if (!in6_dev) - continue; - - list_for_each_entry_rcu(ifa6, &in6_dev->addr_list, if_list) { - if (ipv6_addr_cmp(&ifa6->addr, - &addr6->sin6_addr) == 0) - ret = dev->ifindex; - } - break; - } -#endif /* IS_ENABLED(CONFIG_IPV6) */ - } - if (ret >= 0) - break; - } - rcu_read_unlock(); - - return ret; -} - static struct ksock_conn_cb * ksocknal_create_conn_cb(struct sockaddr *addr) { @@ -135,7 +62,6 @@ ksocknal_create_conn_cb(struct sockaddr *addr) rpc_copy_addr((struct sockaddr *)&conn_cb->ksnr_addr, addr); rpc_set_port((struct sockaddr *)&conn_cb->ksnr_addr, rpc_get_port(addr)); - conn_cb->ksnr_myiface = -1; conn_cb->ksnr_scheduled = 0; conn_cb->ksnr_connecting = 0; conn_cb->ksnr_connected = 0; @@ -333,6 +259,7 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index, struct ksock_conn_cb *conn_cb; int i; int rc = -ENOENT; + struct ksock_net *net; read_lock(&ksocknal_data.ksnd_global_lock); @@ -358,8 +285,9 @@ ksocknal_get_peer_info(struct lnet_ni *ni, int index, struct sockaddr_in *sa; sa = (void *)&conn_cb->ksnr_addr; + net = ni->ni_data; rc = choose_ipv4_src(myip, - conn_cb->ksnr_myiface, + net->ksnn_interface.ksni_index, ntohl(sa->sin_addr.s_addr), ni->ni_net_ns); *peer_ip = ntohl(sa->sin_addr.s_addr); @@ -500,45 +428,10 @@ static void ksocknal_associate_cb_conn_locked(struct ksock_conn_cb *conn_cb, struct ksock_conn *conn) { - struct ksock_peer_ni *peer_ni = conn_cb->ksnr_peer; int type = conn->ksnc_type; - struct ksock_interface *iface; - int conn_iface; - conn_iface = ksocknal_ip2index((struct sockaddr *)&conn->ksnc_myaddr, - peer_ni->ksnp_ni); conn->ksnc_conn_cb = conn_cb; ksocknal_conn_cb_addref(conn_cb); - - if (conn_cb->ksnr_myiface != conn_iface) { - if (conn_cb->ksnr_myiface < 0) { - /* conn_cb wasn't bound locally yet (the initial - * conn_cb) - */ - CDEBUG(D_NET, "Binding %s %pISc to interface %d\n", - libcfs_idstr(&peer_ni->ksnp_id), - &conn_cb->ksnr_addr, - conn_iface); - } else { - CDEBUG(D_NET, - "Rebinding %s %pISc from interface %d to %d\n", - libcfs_idstr(&peer_ni->ksnp_id), - &conn_cb->ksnr_addr, - conn_cb->ksnr_myiface, - conn_iface); - - iface = ksocknal_index2iface(peer_ni->ksnp_ni, - conn_cb->ksnr_myiface); - if (iface) - iface->ksni_nroutes--; - } - conn_cb->ksnr_myiface = conn_iface; - iface = ksocknal_index2iface(peer_ni->ksnp_ni, - conn_cb->ksnr_myiface); - if (iface) - iface->ksni_nroutes++; - } - ksocknal_incr_conn_count(conn_cb, type); /* Successful connection => further attempts can @@ -563,18 +456,16 @@ ksocknal_add_conn_cb_locked(struct ksock_peer_ni *peer_ni, conn_cb->ksnr_peer = peer_ni; ksocknal_peer_addref(peer_ni); - /* set the conn_cb's interface to the current net's interface */ - conn_cb->ksnr_myiface = net->ksnn_interface.ksni_index; - net->ksnn_interface.ksni_nroutes++; - /* peer_ni's route list takes over my ref on 'route' */ peer_ni->ksnp_conn_cb = conn_cb; + net->ksnn_interface.ksni_nroutes++; list_for_each_entry(conn, &peer_ni->ksnp_conns, ksnc_list) { if (!rpc_cmp_addr((struct sockaddr *)&conn->ksnc_peeraddr, (struct sockaddr *)&conn_cb->ksnr_addr)) continue; + CDEBUG(D_NET, "call ksocknal_associate_cb_conn_locked\n"); ksocknal_associate_cb_conn_locked(conn_cb, conn); /* keep going (typed conns) */ } @@ -584,9 +475,9 @@ static void ksocknal_del_conn_cb_locked(struct ksock_conn_cb *conn_cb) { struct ksock_peer_ni *peer_ni = conn_cb->ksnr_peer; - struct ksock_interface *iface; struct ksock_conn *conn; struct ksock_conn *cnxt; + struct ksock_net *net; LASSERT(!conn_cb->ksnr_deleted); @@ -598,12 +489,9 @@ ksocknal_del_conn_cb_locked(struct ksock_conn_cb *conn_cb) ksocknal_close_conn_locked(conn, 0); } - if (conn_cb->ksnr_myiface >= 0) { - iface = ksocknal_index2iface(peer_ni->ksnp_ni, - conn_cb->ksnr_myiface); - if (iface) - iface->ksni_nroutes--; - } + net = (struct ksock_net *)(peer_ni->ksnp_ni->ni_data); + net->ksnn_interface.ksni_nroutes--; + LASSERT(net->ksnn_interface.ksni_nroutes >= 0); conn_cb->ksnr_deleted = 1; ksocknal_conn_cb_decref(conn_cb); /* drop peer_ni's ref */ diff --git a/net/lnet/klnds/socklnd/socklnd.h b/net/lnet/klnds/socklnd/socklnd.h index 50892b1af344..1c95cfe4bb50 100644 --- a/net/lnet/klnds/socklnd/socklnd.h +++ b/net/lnet/klnds/socklnd/socklnd.h @@ -398,7 +398,6 @@ struct ksock_conn_cb { * can happen next */ time64_t ksnr_retry_interval; /* how long between retries */ - int ksnr_myiface; /* my IP */ struct sockaddr_storage ksnr_addr; /* IP address to connect to */ unsigned int ksnr_scheduled:1; /* scheduled for attention */ unsigned int ksnr_connecting:1; /* connection establishment in diff --git a/net/lnet/klnds/socklnd/socklnd_cb.c b/net/lnet/klnds/socklnd/socklnd_cb.c index 17ea0cca9255..f04e87a444ba 100644 --- a/net/lnet/klnds/socklnd/socklnd_cb.c +++ b/net/lnet/klnds/socklnd/socklnd_cb.c @@ -1795,6 +1795,7 @@ ksocknal_connect(struct ksock_conn_cb *conn_cb) time64_t deadline; bool retry_later = 0; int rc = 0; + struct ksock_net *net; deadline = ktime_get_seconds() + ksocknal_timeout(); @@ -1850,8 +1851,9 @@ ksocknal_connect(struct ksock_conn_cb *conn_cb) goto failed; } + net = (struct ksock_net *)(peer_ni->ksnp_ni->ni_data); sock = lnet_connect(&peer_ni->ksnp_id.nid, - conn_cb->ksnr_myiface, + net->ksnn_interface.ksni_index, (struct sockaddr *)&conn_cb->ksnr_addr, peer_ni->ksnp_ni->ni_net_ns); if (IS_ERR(sock)) { From patchwork Sat Feb 8 00:30:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5226CC02199 for ; Sat, 8 Feb 2025 00:39:22 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWwr09jfz1y8n; Fri, 07 Feb 2025 16:31:24 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw45z69z1xRt for ; Fri, 07 Feb 2025 16:30:44 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm3-e204-208.ccs.ornl.gov [160.91.203.26]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 0A76217948A; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 08946106BE18; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:16 -0500 Message-ID: <20250208003027.180076-11-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 10/21] lustre: llite: Fix style issues for dir.c X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arshad Hussain , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Arshad Hussain This patch fixes issues reported by checkpatch for file lustre/llite/dir.c. We are also in sync with the OpenSFS version of this file. WC-bug-id: https://jira.whamcloud.com/browse/LU-6142 Lustre-commit: 20bc026cfc928511d ("LU-6142 llite: Fix style issues for dir.c") Signed-off-by: Arshad Hussain Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50724 Reviewed-by: Andreas Dilger Reviewed-by: Timothy Day Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/dir.c | 245 +++++++++++++++++++----------------------- 1 file changed, 110 insertions(+), 135 deletions(-) diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index 25ea565a2f0b..93f8a27aace3 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -103,26 +103,27 @@ * returned page, page hash collision has to be handled. Pages in the * hash chain, except first one, are termed "overflow pages". * - * Solution to index uniqueness problem is to not cache overflow - * pages. Instead, when page hash collision is detected, all overflow pages - * from emerging chain are immediately requested from the server and placed in - * a special data structure (struct ll_dir_chain). This data structure is used - * by ll_readdir() to process entries from overflow pages. When readdir - * invocation finishes, overflow pages are discarded. If page hash collision - * chain weren't completely processed, next call to readdir will again detect - * page hash collision, again read overflow pages in, process next portion of - * entries and again discard the pages. This is not as wasteful as it looks, - * because, given reasonable hash, page hash collisions are extremely rare. + * Proposed (unimplimented) solution to index uniqueness problem is to + * not cache overflow pages. Instead, when page hash collision is + * detected, all overflow pages from emerging chain should be + * immediately requested from the server and placed in a special data + * structure. This data structure can be used by ll_readdir() to + * process entries from overflow pages. When readdir invocation + * finishes, overflow pages are discarded. If page hash collision chain + * weren't completely processed, next call to readdir will again detect + * page hash collision, again read overflow pages in, process next + * portion of entries and again discard the pages. This is not as + * wasteful as it looks, because, given reasonable hash, page hash + * collisions are extremely rare. * * 1. directory positioning * - * When seekdir(hash) is called, original - * - * - * - * - * + * When seekdir(hash) is called. * + * seekdir() sets the location in the directory stream from which the next + * readdir() call will start. mdc_page_locate() is used to find page with + * starting hash and will issue RPC to fetch that page. If there is a hash + * collision the concerned page is removed. * * * Server. @@ -135,8 +136,9 @@ * a header lu_dirpage which describes the start/end hash, and whether this * page is empty (contains no dir entry) or hash collide with next page. * After client receives reply, several pages will be integrated into dir page - * in PAGE_SIZE (if PAGE_SIZE greater than LU_PAGE_SIZE), and the lu_dirpage - * for this integrated page will be adjusted. See lmv_adjust_dirpages(). + * in PAGE_SIZE (if PAGE_SIZE greater than LU_PAGE_SIZE), and the + * lu_dirpage for this integrated page will be adjusted. See + * mdc_adjust_dirpages(). * */ struct page *ll_get_dir_page(struct inode *dir, struct md_op_data *op_data, @@ -149,7 +151,7 @@ struct page *ll_get_dir_page(struct inode *dir, struct md_op_data *op_data, int rc; rc = md_read_page(ll_i2mdexp(dir), op_data, &mrinfo, offset, &page); - if (rc) + if (rc != 0) return ERR_PTR(rc); if (partial_readdir_rc && mrinfo.mr_partial_readdir_rc) @@ -162,11 +164,11 @@ void ll_release_page(struct inode *inode, struct page *page, bool remove) { kunmap(page); - /* - * Always remove the page for striped dir, because the page is + /* Always remove the page for striped dir, because the page is * built from temporarily in LMV layer */ - if (inode && ll_dir_striped(inode)) { + if (inode && S_ISDIR(inode->i_mode) && + lmv_dir_striped(ll_i2info(inode)->lli_lsm_md)) { __free_page(page); return; } @@ -222,13 +224,11 @@ int ll_dir_read(struct inode *inode, u64 *ppos, struct md_op_data *op_data, u64 ino; hash = le64_to_cpu(ent->lde_hash); - if (hash < pos) - /* Skip until we find target hash */ + if (hash < pos) /* Skip until we find target hash */ continue; namelen = le16_to_cpu(ent->lde_namelen); - if (namelen == 0) - /* Skip dummy record. */ + if (namelen == 0) /* Skip dummy record */ continue; if (is_api32 && is_hash64) @@ -237,13 +237,12 @@ int ll_dir_read(struct inode *inode, u64 *ppos, struct md_op_data *op_data, lhash = hash; fid_le_to_cpu(&fid, &ent->lde_fid); ino = cl_fid_build_ino(&fid, is_api32); - type = IFTODT(lu_dirent_type_get(ent)); - ctx->pos = lhash; - /* For 'll_nfs_get_name_filldir()', it will try - * to access the 'ent' through its 'lde_name', - * so the parameter 'name' for 'ctx->actor()' - * must be part of the 'ent'. + type = S_DT(lu_dirent_type_get(ent)); + /* For ll_nfs_get_name_filldir(), it will try to access + * 'ent' through 'lde_name', so the parameter 'name' + * for 'dir_emit()' must be part of the 'ent'. */ + ctx->pos = lhash; if (!IS_ENCRYPTED(inode)) { done = !dir_emit(ctx, ent->lde_name, namelen, ino, type); @@ -275,16 +274,11 @@ int ll_dir_read(struct inode *inode, u64 *ppos, struct md_op_data *op_data, next = le64_to_cpu(dp->ldp_hash_end); pos = next; if (pos == MDS_DIR_END_OFF) { - /* - * End of directory reached. - */ + /* End of directory reached. */ done = 1; ll_release_page(inode, page, false); } else { - /* - * Normal case: continue to the next - * page. - */ + /* Normal case: continue to the next page.*/ ll_release_page(inode, page, le32_to_cpu(dp->ldp_flags) & LDF_COLLIDE); @@ -303,8 +297,7 @@ static int ll_readdir(struct file *filp, struct dir_context *ctx) { struct inode *inode = file_inode(filp); struct ll_file_data *lfd = filp->private_data; - struct ll_sb_info *sbi = ll_i2sbi(inode); - u64 pos = lfd ? lfd->lfd_pos : 0; + struct ll_sb_info *sbi = ll_i2sbi(inode); bool hash64 = test_bit(LL_SBI_64BIT_HASH, sbi->ll_flags); bool api32 = ll_need_32bit_api(sbi); struct md_op_data *op_data; @@ -312,13 +305,14 @@ static int ll_readdir(struct file *filp, struct dir_context *ctx) ktime_t kstart = ktime_get(); /* result of possible partial readdir */ int partial_readdir_rc = 0; + u64 pos; int rc; LASSERT(lfd); pos = lfd->lfd_pos; CDEBUG(D_VFSTRACE, - "VFS Op:inode=" DFID "(%p) pos/size %lu/%llu 32bit_api %d\n", + "VFS Op:inode="DFID"(%p) pos/size %lu/%llu 32bit_api %d\n", PFID(ll_inode2fid(inode)), inode, (unsigned long)pos, i_size_read(inode), api32); @@ -329,9 +323,7 @@ static int ll_readdir(struct file *filp, struct dir_context *ctx) } if (pos == MDS_DIR_END_OFF) { - /* - * end-of-file. - */ + /* end-of-file. */ rc = 0; goto out; } @@ -352,13 +344,12 @@ static int ll_readdir(struct file *filp, struct dir_context *ctx) pfid = *ll_inode2fid(parent); } - /* - * If it can not find in cache, do lookup .. on the master + /* If it can not find in cache, do lookup .. on the master * object */ if (fid_is_zero(&pfid)) { rc = ll_dir_get_parent_fid(inode, &pfid); - if (rc) + if (rc != 0) return rc; } } @@ -444,7 +435,7 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, if (lump->lum_magic != LMV_MAGIC_FOREIGN) { CDEBUG(D_VFSTRACE, - "VFS Op:inode=" DFID "(%p) name %s stripe_offset %d stripe_count: %u, hash_type=%x\n", + "VFS Op:inode="DFID"(%p) name %s stripe_offset %d stripe_count: %u, hash_type=%x\n", PFID(ll_inode2fid(parent)), parent, dirname, (int)lump->lum_stripe_offset, lump->lum_stripe_count, lump->lum_hash_type); @@ -452,7 +443,7 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, struct lmv_foreign_md *lfm = (struct lmv_foreign_md *)lump; CDEBUG(D_VFSTRACE, - "VFS Op:inode=" DFID "(%p) name %s foreign, length %u, value '%.*s'\n", + "VFS Op:inode="DFID"(%p) name %s foreign, length %u, value '%.*s'\n", PFID(ll_inode2fid(parent)), parent, dirname, lfm->lfm_length, lfm->lfm_length, lfm->lfm_value); } @@ -556,6 +547,8 @@ static int ll_dir_setdirstripe(struct dentry *dparent, struct lmv_user_md *lump, op_data->op_file_secctx_size); else err = ll_inode_init_security(&dentry, inode, parent); + if (err) + goto out_inode; if (encrypt) err = ll_set_encflags(inode, op_data->op_file_encctx, @@ -630,8 +623,7 @@ int ll_dir_setstripe(struct inode *inode, struct lov_user_md *lump, return -EINVAL; } - /* - * This is coming from userspace, so should be in + /* This is coming from userspace, so should be in * local endian. But the MDS would like it in little * endian, so we swab it before we send it. */ @@ -721,12 +713,12 @@ int ll_dir_get_default_layout(struct inode *inode, void **plmm, int *plmm_size, struct mdt_body *body; struct lov_mds_md *lmm = NULL; struct ptlrpc_request *req = NULL; - int lmmsize = OBD_MAX_DEFAULT_EA_SIZE; + int lmm_size = OBD_MAX_DEFAULT_EA_SIZE; struct md_op_data *op_data; struct lu_fid fid; int rc; - op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, lmmsize, + op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, lmm_size, LUSTRE_OPC_ANY, NULL); if (IS_ERR(op_data)) return PTR_ERR(op_data); @@ -743,27 +735,27 @@ int ll_dir_get_default_layout(struct inode *inode, void **plmm, int *plmm_size, rc = md_getattr(sbi->ll_md_exp, op_data, &req); ll_finish_md_op_data(op_data); if (rc < 0) { - CDEBUG(D_INFO, "md_getattr failed on inode " DFID ": rc %d\n", + CDEBUG(D_INFO, "md_getattr failed on inode "DFID": rc %d\n", PFID(&fid), rc); goto out; } body = req_capsule_server_get(&req->rq_pill, &RMF_MDT_BODY); + LASSERT(body); - lmmsize = body->mbo_eadatasize; + lmm_size = body->mbo_eadatasize; if (!(body->mbo_valid & (OBD_MD_FLEASIZE | OBD_MD_FLDIREA)) || - lmmsize == 0) { + lmm_size == 0) { rc = -ENODATA; goto out; } lmm = req_capsule_server_sized_get(&req->rq_pill, - &RMF_MDT_MD, lmmsize); + &RMF_MDT_MD, lmm_size); LASSERT(lmm); - /* - * This is coming from the MDS, so is probably in + /* This is coming from the MDS, so is probably in * little endian. We convert it to host endian before * passing it to userspace. */ @@ -796,12 +788,13 @@ int ll_dir_get_default_layout(struct inode *inode, void **plmm, int *plmm_size, break; } default: - CERROR("unknown magic: %lX\n", (unsigned long)lmm->lmm_magic); rc = -EPROTO; + CERROR("%s: unknown magic: %lX: rc = %d\n", sbi->ll_fsname, + (unsigned long)lmm->lmm_magic, rc); } out: *plmm = lmm; - *plmm_size = lmmsize; + *plmm_size = lmm_size; *request = req; return rc; } @@ -947,7 +940,7 @@ static int ll_ioc_copy_start(struct super_block *sb, struct hsm_copy *copy) iput(inode); if (rc != 0) { CDEBUG(D_HSM, - "Could not read file data version of " DFID " (rc = %d). Archive request (%#llx) could not be done.\n", + "Could not read file data version of "DFID" (rc = %d). Archive request (%#llx) could not be done.\n", PFID(©->hc_hai.hai_fid), rc, copy->hc_hai.hai_cookie); hpk.hpk_flags |= HP_FLAG_RETRY; @@ -966,9 +959,11 @@ static int ll_ioc_copy_start(struct super_block *sb, struct hsm_copy *copy) /* On error, the request should be considered as completed */ if (hpk.hpk_errval > 0) hpk.hpk_flags |= HP_FLAG_COMPLETED; + rc2 = obd_iocontrol(LL_IOC_HSM_PROGRESS, sbi->ll_md_exp, sizeof(hpk), &hpk, NULL); + /* Return first error */ return rc ? rc : rc2; } @@ -1235,7 +1230,8 @@ int quotactl_ioctl(struct super_block *sb, struct if_quotactl *qctl) case LUSTRE_Q_GETINFOPOOL: break; default: - CERROR("unsupported quotactl op: %#x\n", cmd); + CERROR("%s: unsupported quotactl op: %#x: rc = %d\n", + sbi->ll_fsname, cmd, -EOPNOTSUPP); return -ENOTSUPP; } @@ -1374,6 +1370,7 @@ int ll_rmfid(struct file *file, void __user *arg) lfa = kzalloc(size, GFP_NOFS); if (!lfa) return -ENOMEM; + rcs = kcalloc(nr, sizeof(int), GFP_NOFS); if (!rcs) { rc = -ENOMEM; @@ -1492,36 +1489,33 @@ int ll_rmfid(struct file *file, void __user *arg) */ static char *ll_getname(const char __user *filename) { - int ret = 0, len; char *tmp; - tmp = kzalloc(NAME_MAX + 1, GFP_KERNEL); - if (!tmp) - return ERR_PTR(-ENOMEM); + tmp = strndup_user(filename, NAME_MAX + 1); + if (IS_ERR(tmp)) { + int ret = PTR_ERR(tmp); - len = strncpy_from_user(tmp, filename, NAME_MAX + 1); - if (len < 0) - ret = len; - else if (len == 0) - ret = -ENOENT; - else if (len > NAME_MAX && tmp[NAME_MAX] != 0) - ret = -ENAMETOOLONG; - - if (ret) { - kfree(tmp); - tmp = ERR_PTR(ret); + switch (ret) { + case -EFAULT: /* zero length */ + tmp = ERR_PTR(-ENOENT); + break; + case -EINVAL: + tmp = ERR_PTR(-ENAMETOOLONG); + fallthrough; + default: + break; + } } + return tmp; } -#define ll_putname(filename) kfree(filename) - static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { struct dentry *dentry = file_dentry(file); struct inode *inode = file_inode(file); struct ll_sb_info *sbi = ll_i2sbi(inode); - struct obd_ioctl_data *data; + struct obd_ioctl_data *data = NULL; void __user *uarg = (void __user *)arg; int rc = 0; @@ -1539,12 +1533,11 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) char *filename; rc = obd_ioctl_getdata(&data, &len, uarg); - if (rc) + if (rc != 0) return rc; filename = data->ioc_inlbuf1; namelen = strlen(filename); - if (namelen < 1) { CDEBUG(D_INFO, "IOC_MDC_LOOKUP missing filename\n"); rc = -EINVAL; @@ -1647,7 +1640,6 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) struct lov_user_md_v1 __user *lumv1p = uarg; struct lov_user_md_v3 __user *lumv3p = uarg; int lum_size = 0; - int set_default = 0; BUILD_BUG_ON(sizeof(struct lov_user_md_v3) <= @@ -1703,7 +1695,6 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) int rc; int i; - ulmv = (struct lmv_user_md __user *)arg; if (copy_from_user(&lum, ulmv, sizeof(*ulmv))) return -EFAULT; @@ -1721,8 +1712,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) } max_stripe_count = lum.lum_stripe_count; - /* - * lum_magic will indicate which stripe the ioctl will like + /* lum_magic will indicate which stripe the ioctl will like * to get, LMV_MAGIC_V1 is for normal LMV stripe, LMV_USER_MAGIC * is for default LMV stripe */ @@ -1738,7 +1728,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) if (rc) goto finish_req; - /* Get default LMV EA in raw mode */ + /* get default LMV in raw mode */ if (lum.lum_magic == LMV_USER_MAGIC) { if (copy_to_user(ulmv, lmm, lmmsize)) rc = -EFAULT; @@ -1758,8 +1748,9 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) stripe_count = lmv_foreign_to_md_stripes(size); } else { - CERROR("invalid %d foreign size returned\n", - lfm->lfm_length); + CERROR("%s: invalid %d foreign size returned: rc = %d\n", + sbi->ll_fsname, lfm->lfm_length, + -EINVAL); return -EINVAL; } } else { @@ -1833,7 +1824,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) return rc; } case LL_IOC_RMFID: - return ll_rmfid(file, (void __user *)arg); + return ll_rmfid(file, uarg); case LL_IOC_LOV_SWAP_LAYOUTS: return -EPERM; case LL_IOC_LOV_GETSTRIPE: @@ -1894,18 +1885,16 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) if (cmd == IOC_MDC_GETFILESTRIPE || cmd == LL_IOC_LOV_GETSTRIPE || cmd == LL_IOC_LOV_GETSTRIPE_NEW) { - lump = (struct lov_user_md __user *)arg; + lump = uarg; } else if (cmd == IOC_MDC_GETFILEINFO_V1 || cmd == LL_IOC_MDC_GETINFO_V1) { - struct lov_user_mds_data_v1 __user *lmdp; + struct lov_user_mds_data_v1 __user *lmdp = uarg; - lmdp = (struct lov_user_mds_data_v1 __user *)arg; statp = &lmdp->lmd_st; lump = &lmdp->lmd_lmm; } else { - struct lov_user_mds_data __user *lmdp; + struct lov_user_mds_data __user *lmdp = uarg; - lmdp = (struct lov_user_mds_data __user *)arg; fidp = &lmdp->lmd_fid; stxp = &lmdp->lmd_stx; flagsp = &lmdp->lmd_flags; @@ -1985,8 +1974,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) stx.stx_dev_minor = MINOR(inode->i_sb->s_dev); stx.stx_mask |= STATX_BASIC_STATS | STATX_BTIME; - /* - * For a striped directory, the size and blocks returned + /* For a striped directory, the size and blocks returned * from MDT is not correct. * The size and blocks are aggregated by client across * all stripes. @@ -2031,8 +2019,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) out_req: ptlrpc_req_finished(request); ptlrpc_req_finished(root_request); - if (filename) - ll_putname(filename); + kfree(filename); return rc; } case OBD_IOC_QUOTACTL: { @@ -2049,7 +2036,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) } if (LUSTRE_Q_CMD_IS_POOL(qctl->qc_cmd)) { - char __user *from = (char __user *)arg + + char __user *from = uarg + offsetof(typeof(*qctl), qc_poolname); if (copy_from_user(qctl->qc_poolname, from, @@ -2068,10 +2055,10 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) return rc; } case LL_IOC_GETOBDCOUNT: { - int count, vallen; + u32 count, vallen; struct obd_export *exp; - if (copy_from_user(&count, (int __user *)arg, sizeof(int))) + if (copy_from_user(&count, uarg, sizeof(count))) return -EFAULT; /* get ost count when count is zero, get mdt count otherwise */ @@ -2085,31 +2072,24 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) return rc; } - if (copy_to_user((int __user *)arg, &count, sizeof(int))) + if (copy_to_user(uarg, &count, sizeof(count))) return -EFAULT; return 0; } - case LL_IOC_GET_CONNECT_FLAGS: { - return obd_iocontrol(cmd, sbi->ll_md_exp, 0, NULL, - uarg); - } - case OBD_IOC_FID2PATH: - return ll_fid2path(inode, uarg); - case LL_IOC_GETPARENT: - return ll_getparent(file, uarg); + case LL_IOC_GET_CONNECT_FLAGS: + return obd_iocontrol(cmd, sbi->ll_md_exp, 0, NULL, uarg); case LL_IOC_FID2MDTIDX: { struct obd_export *exp = ll_i2mdexp(inode); struct lu_fid fid; u32 index; - if (copy_from_user(&fid, (const struct lu_fid __user *)arg, - sizeof(fid))) + if (copy_from_user(&fid, uarg, sizeof(fid))) return -EFAULT; /* Call mdc_iocontrol */ rc = obd_iocontrol(LL_IOC_FID2MDTIDX, exp, sizeof(fid), &fid, - &index); + (u32 __user *)&index); if (rc) return rc; @@ -2139,8 +2119,8 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) /* Copy the whole struct */ if (copy_from_user(hur, uarg, totalsize)) { - kvfree(hur); - return -EFAULT; + rc = -EFAULT; + goto out_hur; } if (hur->hur_request.hr_action == HUA_RELEASE) { @@ -2165,7 +2145,7 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) rc = obd_iocontrol(cmd, ll_i2mdexp(inode), totalsize, hur, NULL); } - +out_hur: kvfree(hur); return rc; @@ -2195,20 +2175,19 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) if (!capable(CAP_SYS_ADMIN)) return -EPERM; - rc = copy_and_ct_start(cmd, sbi->ll_md_exp, - (struct lustre_kernelcomm __user *)arg); + rc = copy_and_ct_start(cmd, sbi->ll_md_exp, uarg); return rc; case LL_IOC_HSM_COPY_START: { struct hsm_copy *copy; int rc; - copy = memdup_user((char __user *)arg, sizeof(*copy)); + copy = memdup_user(uarg, sizeof(*copy)); if (IS_ERR(copy)) return PTR_ERR(copy); rc = ll_ioc_copy_start(inode->i_sb, copy); - if (copy_to_user((char __user *)arg, copy, sizeof(*copy))) + if (copy_to_user(uarg, copy, sizeof(*copy))) rc = -EFAULT; kfree(copy); @@ -2218,12 +2197,12 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) struct hsm_copy *copy; int rc; - copy = memdup_user((char __user *)arg, sizeof(*copy)); + copy = memdup_user(uarg, sizeof(*copy)); if (IS_ERR(copy)) return PTR_ERR(copy); rc = ll_ioc_copy_end(inode->i_sb, copy); - if (copy_to_user((char __user *)arg, copy, sizeof(*copy))) + if (copy_to_user(uarg, copy, sizeof(*copy))) rc = -EFAULT; kfree(copy); @@ -2278,16 +2257,10 @@ static long ll_dir_ioctl(struct file *file, unsigned int cmd, unsigned long arg) struct inode *inode2; unsigned long ino; - detach = kzalloc(sizeof(*detach), GFP_KERNEL); - if (!detach) - return -ENOMEM; + detach = memdup_user(uarg, sizeof(*detach)); + if (IS_ERR(detach)) + return PTR_ERR(detach); - if (copy_from_user(detach, - (const struct lu_pcc_detach_fid __user *)arg, - sizeof(*detach))) { - rc = -EFAULT; - goto out_detach; - } fid = &detach->pccd_fid; ino = cl_fid_build_ino(fid, ll_need_32bit_api(sbi)); @@ -2334,6 +2307,7 @@ static loff_t ll_dir_seek(struct file *file, loff_t offset, int origin) bool api32 = ll_need_32bit_api(sbi); loff_t ret = -EINVAL; + inode_lock(inode); switch (origin) { case SEEK_SET: break; @@ -2367,12 +2341,13 @@ static loff_t ll_dir_seek(struct file *file, loff_t offset, int origin) else fd->lfd_pos = offset; file->f_pos = offset; + file->f_version = 0; } ret = offset; } - goto out; out: + inode_unlock(inode); return ret; } From patchwork Sat Feb 8 00:30:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8096EC0219E for ; Sat, 8 Feb 2025 00:37:17 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWwL0fbFz1y63; Fri, 07 Feb 2025 16:30:58 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw42G5pz1xPn for ; Fri, 07 Feb 2025 16:30:44 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm-e204-208.ccs.ornl.gov [160.91.203.12]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 0E4E288F9EB; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 0C5FE106BE19; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:17 -0500 Message-ID: <20250208003027.180076-12-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 11/21] lustre: csdc: reserve connect bits for compressed layout X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bobi Jam Add connect data bit for compressed layout (OBD_CONNECT2_COMPRESS) and another connect data bit to be used (OBD_CONNECT2_LARGE_NID). Also reserve obd_connect_data::ocd_compr_type which is a bitmask of supported compression type to be negotiated between client and MDS. WC-bug-id: https://jira.whamcloud.com/browse/LU-16837 Lustre-commit: 83189aef3b23f18cb ("LU-16837 csdc: reserve connect bits for compressed layout") Signed-off-by: Bobi Jam Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51108 Reviewed-by: Sebastien Buisson Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/obdclass/lprocfs_status.c | 2 ++ fs/lustre/ptlrpc/pack_generic.c | 6 ++++-- fs/lustre/ptlrpc/wiretest.c | 13 +++++++++---- include/uapi/linux/lustre/lustre_idl.h | 4 +++- 4 files changed, 18 insertions(+), 7 deletions(-) diff --git a/fs/lustre/obdclass/lprocfs_status.c b/fs/lustre/obdclass/lprocfs_status.c index bde287bbb793..8ecce8901941 100644 --- a/fs/lustre/obdclass/lprocfs_status.c +++ b/fs/lustre/obdclass/lprocfs_status.c @@ -140,6 +140,8 @@ static const char *const obd_connect_names[] = { "mkdir_replay", /* 0x10000000 */ "dmv_imp_inherit", /* 0x20000000 */ "encryption_fid2path", /* 0x40000000 */ + "large_nid", /* 0x100000000 */ + "compressed_file", /* 0x200000000 */ NULL }; diff --git a/fs/lustre/ptlrpc/pack_generic.c b/fs/lustre/ptlrpc/pack_generic.c index cb7a8a39a380..e1692986dd4c 100644 --- a/fs/lustre/ptlrpc/pack_generic.c +++ b/fs/lustre/ptlrpc/pack_generic.c @@ -1768,9 +1768,11 @@ void lustre_swab_connect(struct obd_connect_data *ocd) __swab16s(&ocd->ocd_maxmodrpcs); BUILD_BUG_ON(!offsetof(typeof(*ocd), padding0)); BUILD_BUG_ON(offsetof(typeof(*ocd), padding1) == 0); - if (ocd->ocd_connect_flags & OBD_CONNECT_FLAGS2) + if (ocd->ocd_connect_flags & OBD_CONNECT_FLAGS2) { __swab64s(&ocd->ocd_connect_flags2); - BUILD_BUG_ON(offsetof(typeof(*ocd), padding3) == 0); + if (ocd->ocd_connect_flags2 & OBD_CONNECT2_COMPRESS) + __swab64s(&ocd->ocd_compr_type); + } BUILD_BUG_ON(offsetof(typeof(*ocd), padding4) == 0); BUILD_BUG_ON(offsetof(typeof(*ocd), padding5) == 0); BUILD_BUG_ON(offsetof(typeof(*ocd), padding6) == 0); diff --git a/fs/lustre/ptlrpc/wiretest.c b/fs/lustre/ptlrpc/wiretest.c index 8cec6a35c692..cc81d4d6717d 100644 --- a/fs/lustre/ptlrpc/wiretest.c +++ b/fs/lustre/ptlrpc/wiretest.c @@ -1025,10 +1025,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct obd_connect_data, ocd_connect_flags2)); LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_connect_flags2) == 8, "found %lld\n", (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_connect_flags2)); - LASSERTF((int)offsetof(struct obd_connect_data, padding3) == 88, "found %lld\n", - (long long)(int)offsetof(struct obd_connect_data, padding3)); - LASSERTF((int)sizeof(((struct obd_connect_data *)0)->padding3) == 8, "found %lld\n", - (long long)(int)sizeof(((struct obd_connect_data *)0)->padding3)); + LASSERTF((int)offsetof(struct obd_connect_data, ocd_compr_type) == 88, "found %lld\n", + (long long)(int)offsetof(struct obd_connect_data, ocd_compr_type)); + LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_compr_type) == 8, "found %lld\n", + (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_compr_type)); LASSERTF((int)offsetof(struct obd_connect_data, padding4) == 96, "found %lld\n", (long long)(int)offsetof(struct obd_connect_data, padding4)); LASSERTF((int)sizeof(((struct obd_connect_data *)0)->padding4) == 8, "found %lld\n", @@ -1255,6 +1255,11 @@ void lustre_assert_wire_constants(void) OBD_CONNECT2_DMV_IMP_INHERIT); LASSERTF(OBD_CONNECT2_ENCRYPT_FID2PATH == 0x40000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT2_ENCRYPT_FID2PATH); + LASSERTF(OBD_CONNECT2_LARGE_NID == 0x100000000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT2_LARGE_NID); + LASSERTF(OBD_CONNECT2_COMPRESS == 0x200000000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT2_COMPRESS); + LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n", (unsigned int)OBD_CKSUM_CRC32); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n", diff --git a/include/uapi/linux/lustre/lustre_idl.h b/include/uapi/linux/lustre/lustre_idl.h index 2a7299e6ce46..b61fa5c6d9a1 100644 --- a/include/uapi/linux/lustre/lustre_idl.h +++ b/include/uapi/linux/lustre/lustre_idl.h @@ -788,6 +788,8 @@ struct ptlrpc_body_v2 { #define OBD_CONNECT2_ENCRYPT_NAME 0x8000000ULL /* name encrypt */ #define OBD_CONNECT2_DMV_IMP_INHERIT 0x20000000ULL /* client handle DMV inheritance */ #define OBD_CONNECT2_ENCRYPT_FID2PATH 0x40000000ULL /* fid2path enc file */ +#define OBD_CONNECT2_LARGE_NID 0x100000000ULL /* understands large/IPv6 NIDs */ +#define OBD_CONNECT2_COMPRESS 0x200000000ULL /* compressed file */ /* XXX README XXX README XXX README XXX README XXX README XXX README XXX * Please DO NOT add OBD_CONNECT flags before first ensuring that this value * is not in use by some other branch/patch. Email adilger@whamcloud.com @@ -839,7 +841,7 @@ struct obd_connect_data { __u16 padding0; /* READ BELOW! also fix lustre_swab_connect */ __u32 padding1; /* READ BELOW! also fix lustre_swab_connect */ __u64 ocd_connect_flags2;/* OBD_CONNECT2_* per above */ - __u64 padding3; /* READ BELOW! also fix lustre_swab_connect */ + __u64 ocd_compr_type; /* bitmask of supported compression types */ __u64 padding4; /* READ BELOW! also fix lustre_swab_connect */ __u64 padding5; /* READ BELOW! also fix lustre_swab_connect */ __u64 padding6; /* READ BELOW! also fix lustre_swab_connect */ From patchwork Sat Feb 8 00:30:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C37F8C02199 for ; Sat, 8 Feb 2025 00:41:23 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWyD4qrJz1xTw; Fri, 07 Feb 2025 16:32:36 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw51kCKz1xS5 for ; Fri, 07 Feb 2025 16:30:45 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm3-e204-208.ccs.ornl.gov [160.91.203.26]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 11D3A893E88; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 10163106BE14; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:18 -0500 Message-ID: <20250208003027.180076-13-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 12/21] lnet: udsp: lnetctl udsp improvements X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Horn , Cyril Bordage , Frank Sehr , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn lnet_udsp_del_policy() did not previously return non-zero, but its single caller would check for a non-zero and call lnet_udsp_apply_policies(). This code is removed. lnet_udsp_del_policy() will now return non-zero but only in the case where there is no matching policy index. In this case the policies are not modified and thus we needn't re-apply them. HPE-bug-id: LUS-11490 WC-bug-id: https://jira.whamcloud.com/browse/LU-16574 Lustre-commit: 9e6c0915bb73a04e2 ("LU-16574 udsp: lnetctl udsp improvements") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51087 Reviewed-by: Cyril Bordage Reviewed-by: Frank Sehr Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 6 ------ net/lnet/lnet/udsp.c | 3 +++ 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index d3d51acdf2ad..6884c5d3ad86 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -4550,12 +4550,6 @@ LNetCtl(unsigned int cmd, void *arg) mutex_lock(&the_lnet.ln_api_mutex); rc = lnet_udsp_del_policy(idx); - if (!rc) { - rc = lnet_udsp_apply_policies(NULL, false); - CDEBUG(D_NET, "policy re-application returned %d\n", - rc); - rc = 0; - } mutex_unlock(&the_lnet.ln_api_mutex); return rc; diff --git a/net/lnet/lnet/udsp.c b/net/lnet/lnet/udsp.c index eb9a614a0ec1..9f6ff5baf118 100644 --- a/net/lnet/lnet/udsp.c +++ b/net/lnet/lnet/udsp.c @@ -982,6 +982,9 @@ lnet_udsp_del_policy(int idx) } } + if (!removed) + return -ENOENT; + return 0; } From patchwork Sat Feb 8 00:30:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966178 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 181BEC02199 for ; Sat, 8 Feb 2025 00:37:16 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWwH5B7vz1y56; Fri, 07 Feb 2025 16:30:55 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw55VNjz1xTv for ; Fri, 07 Feb 2025 16:30:45 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm2-e204-208.ccs.ornl.gov [160.91.203.3]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 160E418232E; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 13D3B106BE16; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:19 -0500 Message-ID: <20250208003027.180076-14-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 13/21] lustre: llite: make lsm_sem nested for ll_update_dir_depth X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arshad Hussain , Lai Siyao , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Lockdep is reporting: chmod/16751 is trying to acquire lock: (&lli->lli_lsm_sem){++++}-{3:3}, at: ll_update_dir_depth+0x8b/0x280 but task is already holding lock: (&lli->lli_lsm_sem){++++}-{3:3}, at: ll_update_dir_depth+0x7b/0x280 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(&lli->lli_lsm_sem); lock(&lli->lli_lsm_sem); *** DEADLOCK *** May be due to missing lock nesting notation Lockdep sees acquiring more than one lock of the same lock class as a potential dead lock. The execption is if the locks are used for objects that belong to a hierarchy. For our case of the lsm_sem we do have a hierarchy since a lsm for a child directory is related to the parent directory lsm that is being protected as well. Create new lock classes for the lsm_sem with proper ordering. WC-bug-id: https://jira.whamcloud.com/browse/LU-16807 Lustre-commit: fe44198b7fb710fce ("LU-16807 llite: make lsm_sem nested for ll_update_dir_depth") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51192 Reviewed-by: Arshad Hussain Reviewed-by: Oleg Drokin Reviewed-by: Lai Siyao Reviewed-by: Andreas Dilger --- fs/lustre/llite/llite_lib.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index afb231ecd70b..99b2982fa322 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -2832,6 +2832,11 @@ int ll_dir_default_lmv_inherit(struct inode *dir, struct inode *inode) return rc; } +enum lsm_sem_class { + LSM_SEM_PARENT, + LSM_SEM_CHILD, +}; + /** * Update directory depth and default LMV * @@ -2858,8 +2863,8 @@ void ll_update_dir_depth_dmv(struct inode *dir, struct dentry *de) lli->lli_dir_depth = plli->lli_dir_depth + 1; if (lli->lli_default_lsm_md && lli->lli_default_lmv_set) { if (plli->lli_default_lsm_md) { - down_read(&plli->lli_lsm_sem); - down_read(&lli->lli_lsm_sem); + down_read_nested(&plli->lli_lsm_sem, LSM_SEM_PARENT); + down_read_nested(&lli->lli_lsm_sem, LSM_SEM_CHILD); if (lsm_md_inherited(plli->lli_default_lsm_md, lli->lli_default_lsm_md)) lli->lli_inherit_depth = From patchwork Sat Feb 8 00:30:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CA2AFC02199 for ; Sat, 8 Feb 2025 00:41:19 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWxW5Qf1z1yCv; Fri, 07 Feb 2025 16:31:59 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw60mHBz1xYp for ; Fri, 07 Feb 2025 16:30:46 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm-e204-208.ccs.ornl.gov [160.91.203.12]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 19C78893E8A; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 17937106BE17; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:20 -0500 Message-ID: <20250208003027.180076-15-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 14/21] lustre: gss: fix ptlrpc_gss automatic loading X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Aurelien Degremont , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson ptlrpc_gss kernel module is automatically loaded when a GSS security flavor is enforced. Loading success is recorded in a static variable in the ptlrpc module, which prevents further reloading in case ptlrpc_gss is unloaded while keeping ptlrpc loaded. Get rid of this static variable as it is not required in order to avoid calling request_module("ptlrpc_gss") when not needed. Indeed, once loaded, the static array policies[] has an entry at the SPTLRPC_POLICY_GSS index, indicating that the ptlrpc_gss module is loaded. WC-bug-id: https://jira.whamcloud.com/browse/LU-16888 Lustre-commit: b80d6defb7b018250 ("LU-16888 gss: fix ptlrpc_gss automatic loading") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51264 Reviewed-by: Aurelien Degremont Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/sec.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/fs/lustre/ptlrpc/sec.c b/fs/lustre/ptlrpc/sec.c index 576e6480cda8..c9ad27617836 100644 --- a/fs/lustre/ptlrpc/sec.c +++ b/fs/lustre/ptlrpc/sec.c @@ -116,10 +116,9 @@ static struct ptlrpc_sec_policy *sptlrpc_wireflavor2policy(u32 flavor) { static DEFINE_MUTEX(load_mutex); - static atomic_t loaded = ATOMIC_INIT(0); struct ptlrpc_sec_policy *policy; u16 number = SPTLRPC_FLVR_POLICY(flavor); - u16 flag = 0; + int rc; if (number >= SPTLRPC_POLICY_MAX) return NULL; @@ -129,25 +128,26 @@ struct ptlrpc_sec_policy *sptlrpc_wireflavor2policy(u32 flavor) policy = policies[number]; if (policy && !try_module_get(policy->sp_owner)) policy = NULL; - if (!policy) - flag = atomic_read(&loaded); read_unlock(&policy_lock); - if (policy || flag != 0 || - number != SPTLRPC_POLICY_GSS) + if (policy || number != SPTLRPC_POLICY_GSS) break; - /* try to load gss module, once */ + /* try to load gss module, happens only if policy at index + * SPTLRPC_POLICY_GSS is not already referenced in + * global array policies[] + */ mutex_lock(&load_mutex); - if (atomic_read(&loaded) == 0) { - if (request_module("ptlrpc_gss") == 0) - CDEBUG(D_SEC, - "module ptlrpc_gss loaded on demand\n"); - else - CERROR("Unable to load module ptlrpc_gss\n"); - - atomic_set(&loaded, 1); - } + /* The fact that request_module() returns 0 does not guarantee + * the module has done its job. So we must check that the + * requested policy is now available. This is done by checking + * again for policies[number] in the loop. + */ + rc = request_module("ptlrpc_gss"); + if (rc == 0) + CDEBUG(D_SEC, "module ptlrpc_gss loaded on demand\n"); + else + CERROR("Unable to load module ptlrpc_gss: rc %d\n", rc); mutex_unlock(&load_mutex); } From patchwork Sat Feb 8 00:30:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70F61C02199 for ; Sat, 8 Feb 2025 00:37:22 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWwQ4MG8z1y6d; Fri, 07 Feb 2025 16:31:02 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw64rFFz1xdb for ; Fri, 07 Feb 2025 16:30:46 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm3-e204-208.ccs.ornl.gov [160.91.203.26]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 1E6BF182335; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 1B6DC106BE18; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:21 -0500 Message-ID: <20250208003027.180076-16-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 15/21] lnet: libcfs: add unlikely to CFS_ macros X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arshad Hussain , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Timothy Day Fix the (hopefully) last few OBD_ users to use CFS_ macros instead. Add an 'unlikely()' to CFS_ macros. Some of the OBD_ macros included this hint. Once those macros are removed, the hint will be lost. Add it to the CFS_ macros instead. The libcfs_fail.h only has a couple style issues left. Just fix them in this patch. Also have the cfs_race code have a timeout. WC-bug-id: https://jira.whamcloud.com/browse/LU-12610 Lustre-commit: bbfe5c8d211c8285a ("LU-12610 cfs: add unlikely to CFS_ macros") Signed-off-by: Timothy Day Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51291 Reviewed-by: Andreas Dilger Reviewed-by: Arshad Hussain Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/rw.c | 2 +- fs/lustre/llite/vvp_page.c | 2 +- include/linux/libcfs/libcfs_fail.h | 66 +++++++++++++++++++----------- 3 files changed, 43 insertions(+), 27 deletions(-) diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index d2e400182ecf..276cad2b6eb7 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -2061,7 +2061,7 @@ int ll_readpage(struct file *file, struct page *vmpage) * races with the page being unlocked after readpage() but before it's * used by the caller */ - OBD_FAIL_TIMEOUT(OBD_FAIL_LLITE_READPAGE_PAUSE2, cfs_fail_val); + CFS_FAIL_TIMEOUT(OBD_FAIL_LLITE_READPAGE_PAUSE2, cfs_fail_val); return result; } diff --git a/fs/lustre/llite/vvp_page.c b/fs/lustre/llite/vvp_page.c index 9994c3d292a9..9153e2d9656f 100644 --- a/fs/lustre/llite/vvp_page.c +++ b/fs/lustre/llite/vvp_page.c @@ -116,7 +116,7 @@ static void vvp_vmpage_error(struct inode *inode, struct page *vmpage, } else { SetPageError(vmpage); if (ioret != -ENOSPC && - OBD_FAIL_CHECK(OBD_FAIL_LLITE_PANIC_ON_ESTALE)) + CFS_FAIL_CHECK(OBD_FAIL_LLITE_PANIC_ON_ESTALE)) LBUG(); mapping_set_error(inode->i_mapping, ioret); diff --git a/include/linux/libcfs/libcfs_fail.h b/include/linux/libcfs/libcfs_fail.h index 552aad66c5a6..2da3982534af 100644 --- a/include/linux/libcfs/libcfs_fail.h +++ b/include/linux/libcfs/libcfs_fail.h @@ -48,16 +48,17 @@ int __cfs_fail_check_set(u32 id, u32 value, int set); int __cfs_fail_timeout_set(u32 id, u32 value, int ms, int set); enum { - CFS_FAIL_LOC_NOSET = 0, - CFS_FAIL_LOC_ORSET = 1, - CFS_FAIL_LOC_RESET = 2, - CFS_FAIL_LOC_VALUE = 3 + CFS_FAIL_LOC_NOSET = 0, + CFS_FAIL_LOC_ORSET = 1, + CFS_FAIL_LOC_RESET = 2, + CFS_FAIL_LOC_VALUE = 3 }; -/* Failure ranges - * "0x0100 - 0x3fff" for Lustre - * "0xe000 - 0xefff" for LNet - * "0xf000 - 0xffff" for LNDs +/* + * Failure ranges: + * "0x0100 - 0x3fff" for Lustre + * "0xe000 - 0xefff" for LNet + * "0xf000 - 0xffff" for LNDs */ /* Failure injection control */ @@ -78,6 +79,7 @@ enum { #define CFS_FAIL_RAND 0x08000000 /* fail 1/N of the times */ #define CFS_FAIL_USR1 0x04000000 /* user flag */ +/* CFS_FAULT may be combined with any one of the above flags. */ #define CFS_FAULT 0x02000000 /* match any CFS_FAULT_CHECK */ static inline bool CFS_FAIL_PRECHECK(u32 id) @@ -87,6 +89,9 @@ static inline bool CFS_FAIL_PRECHECK(u32 id) (cfs_fail_loc & id & CFS_FAULT)); } +#define UNLIKELY_CHECK_SET(id, value, set, quiet) \ + (unlikely(cfs_fail_check_set(id, value, set, quiet))) + static inline int cfs_fail_check_set(u32 id, u32 value, int set, int quiet) { unsigned long failed_once = cfs_fail_loc & CFS_FAILED; /* ok if racy */ @@ -108,38 +113,43 @@ static inline int cfs_fail_check_set(u32 id, u32 value, int set, int quiet) return ret; } -/* If id hit cfs_fail_loc, return 1, otherwise return 0 */ +/* + * If id hit cfs_fail_loc, return 1, otherwise return 0 + */ #define CFS_FAIL_CHECK(id) \ - cfs_fail_check_set(id, 0, CFS_FAIL_LOC_NOSET, 0) + UNLIKELY_CHECK_SET(id, 0, CFS_FAIL_LOC_NOSET, 0) #define CFS_FAIL_CHECK_QUIET(id) \ - cfs_fail_check_set(id, 0, CFS_FAIL_LOC_NOSET, 1) + UNLIKELY_CHECK_SET(id, 0, CFS_FAIL_LOC_NOSET, 1) /* * If id hit cfs_fail_loc and cfs_fail_val == (-1 or value) return 1, * otherwise return 0 */ #define CFS_FAIL_CHECK_VALUE(id, value) \ - cfs_fail_check_set(id, value, CFS_FAIL_LOC_VALUE, 0) + UNLIKELY_CHECK_SET(id, value, CFS_FAIL_LOC_VALUE, 0) #define CFS_FAIL_CHECK_VALUE_QUIET(id, value) \ - cfs_fail_check_set(id, value, CFS_FAIL_LOC_VALUE, 1) + UNLIKELY_CHECK_SET(id, value, CFS_FAIL_LOC_VALUE, 1) /* * If id hit cfs_fail_loc, cfs_fail_loc |= value and return 1, * otherwise return 0 */ #define CFS_FAIL_CHECK_ORSET(id, value) \ - cfs_fail_check_set(id, value, CFS_FAIL_LOC_ORSET, 0) + UNLIKELY_CHECK_SET(id, value, CFS_FAIL_LOC_ORSET, 0) #define CFS_FAIL_CHECK_ORSET_QUIET(id, value) \ - cfs_fail_check_set(id, value, CFS_FAIL_LOC_ORSET, 1) + UNLIKELY_CHECK_SET(id, value, CFS_FAIL_LOC_ORSET, 1) /* * If id hit cfs_fail_loc, cfs_fail_loc = value and return 1, * otherwise return 0 */ #define CFS_FAIL_CHECK_RESET(id, value) \ - cfs_fail_check_set(id, value, CFS_FAIL_LOC_RESET, 0) + UNLIKELY_CHECK_SET(id, value, CFS_FAIL_LOC_RESET, 0) #define CFS_FAIL_CHECK_RESET_QUIET(id, value) \ - cfs_fail_check_set(id, value, CFS_FAIL_LOC_RESET, 1) + UNLIKELY_CHECK_SET(id, value, CFS_FAIL_LOC_RESET, 1) + +#define UNLIKELY_TIMEOUT_SET(id, value, ms, set) \ + (unlikely(cfs_fail_timeout_set(id, value, ms, set))) static inline int cfs_fail_timeout_set(u32 id, u32 value, int ms, int set) { @@ -150,23 +160,23 @@ static inline int cfs_fail_timeout_set(u32 id, u32 value, int ms, int set) /* If id hit cfs_fail_loc, sleep for seconds or milliseconds */ #define CFS_FAIL_TIMEOUT(id, secs) \ - cfs_fail_timeout_set(id, 0, (secs) * 1000, CFS_FAIL_LOC_NOSET) + UNLIKELY_TIMEOUT_SET(id, 0, (secs) * 1000, CFS_FAIL_LOC_NOSET) #define CFS_FAIL_TIMEOUT_MS(id, ms) \ - cfs_fail_timeout_set(id, 0, ms, CFS_FAIL_LOC_NOSET) + UNLIKELY_TIMEOUT_SET(id, 0, ms, CFS_FAIL_LOC_NOSET) /* * If id hit cfs_fail_loc, cfs_fail_loc |= value and * sleep seconds or milliseconds */ #define CFS_FAIL_TIMEOUT_ORSET(id, value, secs) \ - cfs_fail_timeout_set(id, value, (secs) * 1000, CFS_FAIL_LOC_ORSET) + UNLIKELY_TIMEOUT_SET(id, value, (secs) * 1000, CFS_FAIL_LOC_ORSET) #define CFS_FAIL_TIMEOUT_RESET(id, value, secs) \ - cfs_fail_timeout_set(id, value, (secs) * 1000, CFS_FAIL_LOC_RESET) + UNLIKELY_TIMEOUT_SET(id, value, (secs) * 1000, CFS_FAIL_LOC_RESET) #define CFS_FAIL_TIMEOUT_MS_ORSET(id, value, ms) \ - cfs_fail_timeout_set(id, value, ms, CFS_FAIL_LOC_ORSET) + UNLIKELY_TIMEOUT_SET(id, value, ms, CFS_FAIL_LOC_ORSET) #define CFS_FAULT_CHECK(id) \ CFS_FAIL_CHECK(CFS_FAULT | (id)) @@ -185,8 +195,14 @@ static inline void cfs_race(u32 id) cfs_race_state = 0; CERROR("cfs_race id %x sleeping\n", id); - rc = wait_event_interruptible(cfs_race_waitq, - !!cfs_race_state); + /* + * XXX: don't wait forever as there is no guarantee + * that this branch is executed first. for testing + * purposes this construction works good enough + */ + rc = wait_event_interruptible_timeout(cfs_race_waitq, + !!cfs_race_state, + 5 * HZ); CERROR("cfs_fail_race id %x awake: rc=%d\n", id, rc); } else { CERROR("cfs_fail_race id %x waking\n", id); @@ -219,7 +235,7 @@ static inline void cfs_race_wait(u32 id) * purposes this construction works good enough */ rc = wait_event_interruptible_timeout(cfs_race_waitq, - cfs_race_state != 0, + !!cfs_race_state, 5 * HZ); CERROR("cfs_fail_race id %x awake: rc=%d\n", id, rc); } From patchwork Sat Feb 8 00:30:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39BB4C0219D for ; Sat, 8 Feb 2025 00:34:17 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWw96WVYz1xNq; Fri, 07 Feb 2025 16:30:49 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw675RVz1xdj for ; Fri, 07 Feb 2025 16:30:46 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm-e204-208.ccs.ornl.gov [160.91.203.12]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 247BA893E8D; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 1F10C106BE14; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:22 -0500 Message-ID: <20250208003027.180076-17-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 16/21] lustre: osc: Don't include lock for srvlock X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell When doing server side locking, it doesn't make sense to do the 'search for covering lock and send it to the server' step when building an RPC, because we will not use that lock. This can disguise issues on the client, because prolonging a lock is supposed to let a client avoid eviction if it is still doing IO under the lock, but we are not. This can result in delaying an eviction which should be occurring because the client can't give the lock back. WC-bug-id: https://jira.whamcloud.com/browse/LU-13805 Lustre-commit: 9a34ec2b09864a933 ("LU-13805 osc: Don't include lock for srvlock") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50067 Reviewed-by: Oleg Drokin Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- fs/lustre/osc/osc_object.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/fs/lustre/osc/osc_object.c b/fs/lustre/osc/osc_object.c index efb053323dda..dd2820b8049a 100644 --- a/fs/lustre/osc/osc_object.c +++ b/fs/lustre/osc/osc_object.c @@ -344,10 +344,12 @@ static void osc_req_attr_set(const struct lu_env *env, struct cl_object *obj, struct lov_oinfo *oinfo; struct ost_lvb *lvb; struct obdo *oa; + struct osc_page *opg; oinfo = cl2osc(obj)->oo_oinfo; lvb = &oinfo->loi_lvb; oa = attr->cra_oa; + opg = osc_cl_page_osc(attr->cra_page, cl2osc(obj)); if (flags & OBD_MD_FLMTIME) { oa->o_mtime = lvb->lvb_mtime; @@ -381,14 +383,15 @@ static void osc_req_attr_set(const struct lu_env *env, struct cl_object *obj, } oa->o_valid |= OBD_MD_FLID; } - if (flags & OBD_MD_FLHANDLE) { + /* if srvlock is set, don't look for a local lock, since we won't use + * it and shouldn't note it in the RPC + */ + if (flags & OBD_MD_FLHANDLE && !opg->ops_srvlock) { struct ldlm_lock *lock; - struct osc_page *opg; - opg = osc_cl_page_osc(attr->cra_page, cl2osc(obj)); lock = osc_dlmlock_at_pgoff(env, cl2osc(obj), osc_index(opg), OSC_DAP_FL_TEST_LOCK | OSC_DAP_FL_CANCELING); - if (!lock && !opg->ops_srvlock) { + if (!lock) { struct ldlm_namespace *ns; struct ldlm_resource *res; struct ldlm_res_id *resname; @@ -405,12 +408,9 @@ static void osc_req_attr_set(const struct lu_env *env, struct cl_object *obj, LBUG(); } - /* check for lockless io. */ - if (lock) { - oa->o_handle = lock->l_remote_handle; - oa->o_valid |= OBD_MD_FLHANDLE; - LDLM_LOCK_PUT(lock); - } + oa->o_handle = lock->l_remote_handle; + oa->o_valid |= OBD_MD_FLHANDLE; + LDLM_LOCK_PUT(lock); } } From patchwork Sat Feb 8 00:30:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E429C02199 for ; Sat, 8 Feb 2025 00:42:54 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWyz3YPrz20t5; Fri, 07 Feb 2025 16:33:15 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw84CVQz1xfS for ; Fri, 07 Feb 2025 16:30:48 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm3-e204-208.ccs.ornl.gov [160.91.203.26]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 245FD893E8C; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 22ECE106BE16; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:23 -0500 Message-ID: <20250208003027.180076-18-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 17/21] lustre: clio: Trivial DIO cleanups X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell This is some minor DIO refactoring and an additional debug message discovered while working on this. Extremely minor. WC-bug-id: https://jira.whamcloud.com/browse/LU-13805 Lustre-commit: a667cfac0e70cd4da ("LU-13805 clio: Trivial DIO cleanups") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49987 Reviewed-by: Oleg Drokin Reviewed-by: Andreas Dilger Reviewed-by: Sebastien Buisson Signed-off-by: James Simmons --- fs/lustre/obdclass/cl_io.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/fs/lustre/obdclass/cl_io.c b/fs/lustre/obdclass/cl_io.c index ee82260ab0f5..d6518e9f9c66 100644 --- a/fs/lustre/obdclass/cl_io.c +++ b/fs/lustre/obdclass/cl_io.c @@ -1273,14 +1273,11 @@ void cl_sync_io_note(const struct lu_env *env, struct cl_sync_io *anchor, spin_unlock(&anchor->csi_waitq.lock); - if (csi_dio_aio) { - if (end_io == cl_dio_aio_end) { - if (!creator_free) - cl_dio_aio_free(env, dio_aio); - } else if (end_io == cl_sub_dio_end) { - if (!creator_free) - cl_sub_dio_free(sub_dio_aio); - } + if (csi_dio_aio && !creator_free) { + if (end_io == cl_dio_aio_end) + cl_dio_aio_free(env, dio_aio); + else if (end_io == cl_sub_dio_end) + cl_sub_dio_free(sub_dio_aio); } } } From patchwork Sat Feb 8 00:30:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1C8B5C02199 for ; Sat, 8 Feb 2025 00:37:40 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWwV6j9mz1y7B; Fri, 07 Feb 2025 16:31:06 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw74PrZz1xf6 for ; Fri, 07 Feb 2025 16:30:47 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm2-e204-208.ccs.ornl.gov [160.91.203.3]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 27D80182337; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 26B7B106BE17; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:24 -0500 Message-ID: <20250208003027.180076-19-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 18/21] lustre: osc: Add debug X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Patrick Farrell This adds some minor additional debug for unaligned IO. The purpose here is just to shorten the length of the main patch by pulling out supporting bits. WC-bug-id: https://jira.whamcloud.com/browse/LU-13805 Lustre-commit: 6d1045d63a5bf33a1 ("LU-13805 osc: Add debug") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/49988 Reviewed-by: Oleg Drokin Reviewed-by: Andreas Dilger Reviewed-by: Sebastien Buisson Signed-off-by: James Simmons --- fs/lustre/osc/osc_page.c | 2 ++ fs/lustre/osc/osc_request.c | 13 +++++++++---- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index d4b3baf227b6..c428c3626676 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -201,6 +201,8 @@ static void osc_page_clip(const struct lu_env *env, struct osc_page *opg = cl2osc_page(slice); struct osc_async_page *oap = &opg->ops_oap; + CDEBUG(D_CACHE, "from %d, to %d\n", from, to); + opg->ops_from = from; /* argument @to is exclusive, but @ops_to is inclusive */ opg->ops_to = to - 1; diff --git a/fs/lustre/osc/osc_request.c b/fs/lustre/osc/osc_request.c index 3298a9923ea2..5c9f93b91609 100644 --- a/fs/lustre/osc/osc_request.c +++ b/fs/lustre/osc/osc_request.c @@ -2737,16 +2737,21 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, list_add_tail(&oap->oap_rpc_item, &rpc_list); if (starting_offset == OBD_OBJECT_EOF || - starting_offset > oap->oap_obj_off) + starting_offset > oap->oap_obj_off) { starting_offset = oap->oap_obj_off; - else + } else { + CDEBUG(D_CACHE, + "page i:%d, oap->oap_obj_off %llu, oap->oap_page_off %u\n", + i, oap->oap_obj_off, oap->oap_page_off); LASSERT(!oap->oap_page_off); - if (ending_offset < oap->oap_obj_off + oap->oap_count) + } + if (ending_offset < oap->oap_obj_off + oap->oap_count) { ending_offset = oap->oap_obj_off + oap->oap_count; - else + } else { LASSERT(oap->oap_page_off + oap->oap_count == PAGE_SIZE); + } } if (ext->oe_ndelay) ndelay = true; From patchwork Sat Feb 8 00:30:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966182 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FB37C0219D for ; Sat, 8 Feb 2025 00:38:05 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWws2CNfz1y8w; Fri, 07 Feb 2025 16:31:25 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw943B3z1xp7 for ; Fri, 07 Feb 2025 16:30:49 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm-e204-208.ccs.ornl.gov [160.91.203.12]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 2C854899AC0; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 2A7D2106BE18; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:25 -0500 Message-ID: <20250208003027.180076-20-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 19/21] lustre: misc: fix clang build errors X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Timothy Day Fix several format security errors by explicitly giving the format to the affected functions. Explicitly cast ll_nfs_get_name_filldir as a filldir_t and NR_WRITEBACK as a zone_stat_item. This silences some implicit cast errors. These casts can likely be removed when older kernel support is dropped. Refactor some code to avoid strncat, which was being used incorrectly anyway. Adjust some variables to use more appropriate types. Inline some functions which are only sometimes used. WC-bug-id: https://jira.whamcloud.com/browse/LU-16510 Lustre-commit: 03ddd706d99b09790 ("LU-16518 misc: fix clang build errors") Signed-off-by: Timothy Day Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50332 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_nfs.c | 2 +- fs/lustre/mgc/mgc_request.c | 3 +-- fs/lustre/obdclass/genops.c | 8 +++++--- fs/lustre/osc/osc_lock.c | 2 +- fs/lustre/osc/osc_page.c | 6 ++++-- 5 files changed, 12 insertions(+), 9 deletions(-) diff --git a/fs/lustre/llite/llite_nfs.c b/fs/lustre/llite/llite_nfs.c index 232b2b311151..d505c206a62e 100644 --- a/fs/lustre/llite/llite_nfs.c +++ b/fs/lustre/llite/llite_nfs.c @@ -197,7 +197,7 @@ static int ll_get_name(struct dentry *dentry, char *name, struct ll_getname_data lgd = { .lgd_name = name, .lgd_fid = ll_i2info(d_inode(child))->lli_fid, - .ctx.actor = ll_nfs_get_name_filldir, + .ctx.actor = (filldir_t)ll_nfs_get_name_filldir, }; struct md_op_data *op_data; u64 pos = 0; diff --git a/fs/lustre/mgc/mgc_request.c b/fs/lustre/mgc/mgc_request.c index 56fb50aedbd9..c8ce8d4c1738 100644 --- a/fs/lustre/mgc/mgc_request.c +++ b/fs/lustre/mgc/mgc_request.c @@ -255,9 +255,8 @@ config_recover_log_add(struct obd_device *obd, char *fsname, * where only clients are notified if one of cmd server restarts */ LASSERT(strlen(fsname) < sizeof(logname) / 2); - strcpy(logname, fsname); LASSERT(lcfg.cfg_instance); - strcat(logname, "-cliir"); + scnprintf(logname, sizeof(logname), "%s-cliir", fsname); cld = do_config_log_add(obd, logname, MGS_CFG_T_RECOVER, &lcfg, sb); return cld; diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index 43772aac3776..a3f7b59db47f 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -1277,6 +1277,7 @@ int obd_get_request_slot(struct client_obd *cli) else list_del(&orsw.orsw_entry); } + rc = -EINTR; } if (orsw.orsw_signaled) { @@ -1328,7 +1329,7 @@ int obd_set_max_rpcs_in_flight(struct client_obd *cli, u32 max) if (max > OBD_MAX_RIF_MAX || max < 1) return -ERANGE; - CDEBUG(D_INFO, "%s: max = %hu max_mod = %u rif = %u\n", + CDEBUG(D_INFO, "%s: max = %u max_mod = %u rif = %u\n", cli->cl_import->imp_obd->obd_name, max, cli->cl_max_mod_rpcs_in_flight, cli->cl_max_rpcs_in_flight); @@ -1384,6 +1385,7 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, u16 max) if (max > OBD_MAX_RIF_MAX || max < 1) return -ERANGE; + ocd = &cli->cl_import->imp_connect_data; CDEBUG(D_INFO, "%s: max = %hu flags = %llx, max_mod = %u rif = %u\n", cli->cl_import->imp_obd->obd_name, max, ocd->ocd_connect_flags, ocd->ocd_maxmodrpcs, cli->cl_max_rpcs_in_flight); @@ -1396,9 +1398,9 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, u16 max) */ if (max >= cli->cl_max_rpcs_in_flight) { CDEBUG(D_INFO, - "%s: increasing max_rpcs_in_flight=%hu to allow larger max_mod_rpcs_in_flight=%u\n", + "%s: increasing max_rpcs_in_flight=%u to allow larger max_mod_rpcs_in_flight=%u\n", cli->cl_import->imp_obd->obd_name, max + 1, max); - return -ERANGE; + obd_set_max_rpcs_in_flight(cli, max + 1); } /* cannot exceed max modify RPCs in flight supported by the server, diff --git a/fs/lustre/osc/osc_lock.c b/fs/lustre/osc/osc_lock.c index 3eac1d44fb2d..c7ae1ab3e3d3 100644 --- a/fs/lustre/osc/osc_lock.c +++ b/fs/lustre/osc/osc_lock.c @@ -65,7 +65,7 @@ static struct ldlm_lock *osc_handle_ptr(struct lustre_handle *handle) /** * Invariant that has to be true all of the time. */ -static int osc_lock_invariant(struct osc_lock *ols) +static inline int osc_lock_invariant(struct osc_lock *ols) { struct ldlm_lock *lock = osc_handle_ptr(&ols->ols_handle); struct ldlm_lock *olock = ols->ols_dlmlock; diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index c428c3626676..38903eb53a92 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -912,7 +912,8 @@ static inline void unstable_page_accounting(struct ptlrpc_bulk_desc *desc, } if (count > 0) { - mod_node_page_state(pgdat, NR_WRITEBACK, + mod_node_page_state(pgdat, + (enum zone_stat_item)NR_WRITEBACK, factor * count); count = 0; } @@ -920,7 +921,8 @@ static inline void unstable_page_accounting(struct ptlrpc_bulk_desc *desc, ++count; } if (count > 0) - mod_node_page_state(last, NR_WRITEBACK, factor * count); + mod_node_page_state(last, (enum zone_stat_item)NR_WRITEBACK, + factor * count); } static inline void add_unstable_page_accounting(struct ptlrpc_bulk_desc *desc, From patchwork Sat Feb 8 00:30:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 54B5AC02199 for ; Sat, 8 Feb 2025 00:44:01 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWzg2tRpz20tl; Fri, 07 Feb 2025 16:33:51 -0800 (PST) Received: from smtp4.ccs.ornl.gov (smtp4.ccs.ornl.gov [160.91.203.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWw85cFGz1xnj for ; Fri, 07 Feb 2025 16:30:48 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm3-e204-208.ccs.ornl.gov [160.91.203.26]) by smtp4.ccs.ornl.gov (Postfix) with ESMTP id 30CB6188AC7; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 2E4E9106BE19; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:26 -0500 Message-ID: <20250208003027.180076-21-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 20/21] lnet: fix locking multiple NIDs of the MR peer X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Frank Sehr , Chris Horn , Serguei Smirnov , Cyril Bordage , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Serguei Smirnov If Lustre identifies the same peer with multiple NIDs, as a result of peer discovery it is possible that the discovered peer is found to contain a NID which is locked as primary by a different existing peer record. In this case it is safe to merge the peer records, but the NID which got locked the earliest should be kept as primary. This allows for the first of the two locked NIDs to stay primary as intended for the purpose of communicating with Lustre even if peer discovery succeeded using a different NID of MR peer. Fixes: cede069b76 ("lnet: Lock primary NID logic") WC-bug-id: https://jira.whamcloud.com/browse/LU-16709 Lustre-commit: 3b7a02ee4d656b7b3 ("LU-16709 lnet: fix locking multiple NIDs of the MR peer") Signed-off-by: Serguei Smirnov Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/50530 Reviewed-by: Cyril Bordage Reviewed-by: Frank Sehr Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-types.h | 3 ++ include/uapi/linux/lnet/libcfs_ioctl.h | 3 +- net/lnet/lnet/api-ni.c | 24 +++++++++++ net/lnet/lnet/peer.c | 59 ++++++++++++++++++++------ 4 files changed, 75 insertions(+), 14 deletions(-) diff --git a/include/linux/lnet/lib-types.h b/include/linux/lnet/lib-types.h index 0de7654e3a67..c17c55ff88e0 100644 --- a/include/linux/lnet/lib-types.h +++ b/include/linux/lnet/lib-types.h @@ -987,6 +987,9 @@ struct lnet_peer { * nets/NIs */ u32 lp_send_seq; + + /* timestamp (ns) of primary nid lock */ + u64 lp_prim_lock_ts; }; /* diff --git a/include/uapi/linux/lnet/libcfs_ioctl.h b/include/uapi/linux/lnet/libcfs_ioctl.h index 98b61b1f1292..2a067cdafc48 100644 --- a/include/uapi/linux/lnet/libcfs_ioctl.h +++ b/include/uapi/linux/lnet/libcfs_ioctl.h @@ -156,6 +156,7 @@ struct libcfs_ioctl_data { #define IOC_LIBCFS_GET_CONST_UDSP_INFO _IOWR(IOC_LIBCFS_TYPE, 109, IOCTL_CONFIG_SIZE) #define IOC_LIBCFS_RESET_LNET_STATS _IOWR(IOC_LIBCFS_TYPE, 110, IOCTL_CONFIG_SIZE) #define IOC_LIBCFS_SET_CONNS_PER_PEER _IOWR(IOC_LIBCFS_TYPE, 111, IOCTL_CONFIG_SIZE) -#define IOC_LIBCFS_MAX_NR 111 +#define IOC_LIBCFS_SET_PEER _IOWR(IOC_LIBCFS_TYPE, 112, IOCTL_CONFIG_SIZE) +#define IOC_LIBCFS_MAX_NR 112 #endif /* __LIBCFS_IOCTL_H__ */ diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 6884c5d3ad86..870ae4312d78 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -4384,6 +4384,30 @@ LNetCtl(unsigned int cmd, void *arg) return 0; } + case IOC_LIBCFS_SET_PEER: { + struct lnet_ioctl_peer_cfg *cfg = arg; + struct lnet_peer *lp; + + if (cfg->prcfg_hdr.ioc_len < sizeof(*cfg)) + return -EINVAL; + + mutex_lock(&the_lnet.ln_api_mutex); + lnet_nid4_to_nid(cfg->prcfg_prim_nid, &nid); + lp = lnet_find_peer(&nid); + if (!lp) { + mutex_unlock(&the_lnet.ln_api_mutex); + return -ENOENT; + } + spin_lock(&lp->lp_lock); + lp->lp_state = cfg->prcfg_state; + spin_unlock(&lp->lp_lock); + lnet_peer_decref_locked(lp); + mutex_unlock(&the_lnet.ln_api_mutex); + CDEBUG(D_NET, "Set peer %s state to %u\n", + libcfs_nid2str(cfg->prcfg_prim_nid), cfg->prcfg_state); + return 0; + } + case IOC_LIBCFS_SET_CONNS_PER_PEER: { struct lnet_ioctl_reset_conns_per_peer_cfg *cfg = arg; int value; diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 916864100bee..0aa4a77ab0d4 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -1429,8 +1429,10 @@ void LNetPrimaryNID(struct lnet_nid *nid) */ again: spin_lock(&lp->lp_lock); - if (!(lp->lp_state & LNET_PEER_LOCK_PRIMARY) && lock_prim_nid) + if (!(lp->lp_state & LNET_PEER_LOCK_PRIMARY) && lock_prim_nid) { lp->lp_state |= LNET_PEER_LOCK_PRIMARY; + lp->lp_prim_lock_ts = ktime_get_ns(); + } /* DD disabled, nothing to do */ if (lnet_peer_discovery_disabled) { @@ -1577,8 +1579,10 @@ lnet_peer_attach_peer_ni(struct lnet_peer *lp, lnet_peer_clr_non_mr_pref_nids(lp); } } - if (flags & LNET_PEER_LOCK_PRIMARY) + if (flags & LNET_PEER_LOCK_PRIMARY) { lp->lp_state |= LNET_PEER_LOCK_PRIMARY; + lp->lp_prim_lock_ts = ktime_get_ns(); + } spin_unlock(&lp->lp_lock); lp->lp_nnis++; @@ -1762,24 +1766,53 @@ lnet_peer_add_nid(struct lnet_peer *lp, struct lnet_nid *nid, struct lnet_peer *lp2 = lpni->lpni_peer_net->lpn_peer; int rtr_refcount = lp2->lp_rtr_refcount; - - /* If the new peer that this NID belongs to is - * a primary NID for another peer which we're - * suppose to preserve the Primary for then we - * don't want to mess with it. But the - * configuration is wrong at this point, so we - * should flag both of these peers as in a bad + unsigned int peer2_state; + u64 peer2_prim_lock_ts; + + /* If there's another peer that this NID belongs to + * and the primary NID for that peer is locked, + * then, unless it is the only NID, we don't want + * to mess with it. + * But the configuration is wrong at this point, + * so we should flag both of these peers as in a bad * state */ - if (lp2->lp_state & LNET_PEER_LOCK_PRIMARY) { + spin_lock(&lp2->lp_lock); + if (lp2->lp_state & LNET_PEER_LOCK_PRIMARY && + lp2->lp_nnis > 1) { + lp2->lp_state |= LNET_PEER_BAD_CONFIG; + spin_unlock(&lp2->lp_lock); spin_lock(&lp->lp_lock); lp->lp_state |= LNET_PEER_BAD_CONFIG; spin_unlock(&lp->lp_lock); - spin_lock(&lp2->lp_lock); - lp2->lp_state |= LNET_PEER_BAD_CONFIG; - spin_unlock(&lp2->lp_lock); + CERROR("Peer %s NID %s is already locked with peer %s\n", + libcfs_nidstr(&lp->lp_primary_nid), + libcfs_nidstr(nid), + libcfs_nidstr(&lp2->lp_primary_nid)); goto out_free_lpni; } + peer2_state = lp2->lp_state; + peer2_prim_lock_ts = lp2->lp_prim_lock_ts; + spin_unlock(&lp2->lp_lock); + + /* NID which got locked the earliest should be + * kept as primary. In case if the peers were + * created by Lustre, this allows the + * first listed NID to stay primary as intended + * for the purpose of communicating with Lustre + * even if peer discovery succeeded using + * a different NID of MR peer. + */ + spin_lock(&lp->lp_lock); + if (peer2_state & LNET_PEER_LOCK_PRIMARY && + ((lp->lp_state & LNET_PEER_LOCK_PRIMARY && + peer2_prim_lock_ts < lp->lp_prim_lock_ts) || + !(lp->lp_state & LNET_PEER_LOCK_PRIMARY))) { + lp->lp_prim_lock_ts = peer2_prim_lock_ts; + lp->lp_primary_nid = *nid; + lp->lp_state |= LNET_PEER_LOCK_PRIMARY; + } + spin_unlock(&lp->lp_lock); /* if we're trying to delete a router it means * we're moving this peer NI to a new peer so must * transfer router properties to the new peer From patchwork Sat Feb 8 00:30:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 13966186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from pdx1-mailman-customer002.dreamhost.com (listserver-buz.dreamhost.com [69.163.136.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0BAFCC0219D for ; Sat, 8 Feb 2025 00:41:20 +0000 (UTC) Received: from pdx1-mailman-customer002.dreamhost.com (localhost [127.0.0.1]) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTP id 4YqWyB48fTz1yWh; Fri, 07 Feb 2025 16:32:34 -0800 (PST) Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by pdx1-mailman-customer002.dreamhost.com (Postfix) with ESMTPS id 4YqWwB3HXRz1xqB for ; Fri, 07 Feb 2025 16:30:50 -0800 (PST) Received: from star2.ccs.ornl.gov (ltm-e204-208.ccs.ornl.gov [160.91.203.12]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 346C8899AD6; Fri, 7 Feb 2025 19:30:33 -0500 (EST) Received: by star2.ccs.ornl.gov (Postfix, from userid 2004) id 324AD106BE14; Fri, 7 Feb 2025 19:30:33 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Fri, 7 Feb 2025 19:30:27 -0500 Message-ID: <20250208003027.180076-22-jsimmons@infradead.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20250208003027.180076-1-jsimmons@infradead.org> References: <20250208003027.180076-1-jsimmons@infradead.org> MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 21/21] lnet: ensure dev notification on lnd startup X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Frank Sehr , Cyril Bordage , Serguei Smirnov , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Serguei Smirnov Look up device and link state on lnd startup so that the initial NI state may be set properly. Reduce code duplication by adding lnet_set_link_fatal_state() and lnet_get_link_status() functions which are shared across LNDs. LND-specific versions of these are removed. This fixes the issue with adding LNet NI using an interface with cable unplugged which results in the NI state initialized as "up". Fixes: 91f28da387 ("lnet: use discovered ni status to set initial health") WC-bug-id: https://jira.whamcloud.com/browse/LU-16836 Lustre-commit: 09c6e2b872287c847 ("LU-16836 lnet: ensure dev notification on lnd startup") Signed-off-by: Serguei Smirnov Reviewed-on: https://review.whamcloud.com/c/fs/lustre-release/+/51057 Reviewed-by: Frank Sehr Reviewed-by: Cyril Bordage Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 2 + net/lnet/klnds/o2iblnd/o2iblnd.c | 76 ++++++++---------- net/lnet/klnds/socklnd/socklnd.c | 131 ++++++++++++++++++++++--------- net/lnet/lnet/config.c | 32 ++++++++ 4 files changed, 161 insertions(+), 80 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 09cf42995f39..ce4ad5ae7eb7 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -880,6 +880,8 @@ void lnet_swap_pinginfo(struct lnet_ping_buffer *pbuf); int lnet_ping_info_validate(struct lnet_ping_info *pinfo); struct lnet_ping_buffer *lnet_ping_buffer_alloc(int bytes, gfp_t gfp); void lnet_ping_buffer_free(struct lnet_ping_buffer *pbuf); +int lnet_get_link_status(struct net_device *dev); +u32 lnet_set_link_fatal_state(struct lnet_ni *ni, unsigned int link_state); static inline void lnet_ping_buffer_addref(struct lnet_ping_buffer *pbuf) { diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.c b/net/lnet/klnds/o2iblnd/o2iblnd.c index fc59f88f0801..c954df32ab50 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd.c @@ -1673,21 +1673,6 @@ static void kiblnd_fini_fmr_poolset(struct kib_fmr_poolset *fps) } } -static int kiblnd_get_link_status(struct net_device *dev) -{ - int ret = -1; - - LASSERT(dev); - - if (!netif_running(dev)) - ret = 0; - /* Some devices may not be providing link settings */ - else if (dev->ethtool_ops->get_link) - ret = dev->ethtool_ops->get_link(dev); - - return ret; -} - static int kiblnd_init_fmr_poolset(struct kib_fmr_poolset *fps, int cpt, int ncpts, struct kib_net *net, @@ -2385,15 +2370,19 @@ kiblnd_set_ni_fatal_on(struct kib_hca_dev *hdev, int val) struct kib_net *net; u32 ni_state_before; bool update_ping_buf = false; + struct lnet_ni *ni = NULL; /* for health check */ list_for_each_entry(net, &hdev->ibh_dev->ibd_nets, ibn_list) { + ni = net->ibn_ni; if (val) CDEBUG(D_NETERROR, "Fatal device error for NI %s\n", - libcfs_nidstr(&net->ibn_ni->ni_nid)); - ni_state_before = atomic_xchg(&net->ibn_ni->ni_fatal_error_on, - val); - if (!update_ping_buf && val != ni_state_before) + libcfs_nidstr(&ni->ni_nid)); + ni_state_before = lnet_set_link_fatal_state(ni, val); + + if (!update_ping_buf && + ni->ni_state == LNET_NI_STATE_ACTIVE && + val != ni_state_before) update_ping_buf = true; } @@ -2689,7 +2678,7 @@ int kiblnd_dev_failover(struct kib_dev *dev, struct net *ns) if (set_fatal) { rcu_read_lock(); netdev = dev_get_by_name_rcu(ns, dev->ibd_ifname); - if (netdev && (kiblnd_get_link_status(netdev) == 1)) + if (netdev && (lnet_get_link_status(netdev) == 1)) kiblnd_set_ni_fatal_on(dev->ibd_hdev, 0); rcu_read_unlock(); } @@ -2759,6 +2748,7 @@ kiblnd_handle_link_state_change(struct net_device *dev, u32 ni_state_before; bool update_ping_buf = false; const struct in_ifaddr *ifa; + int state; event_kibdev = kiblnd_dev_search(dev->name); @@ -2773,10 +2763,7 @@ kiblnd_handle_link_state_change(struct net_device *dev, if (!in_dev) { CDEBUG(D_NET, "Interface %s has no IPv4 status.\n", dev->name); - CDEBUG(D_NET, "%s: set link fatal state to 1\n", - libcfs_nidstr(&net->ibn_ni->ni_nid)); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - 1); + ni_state_before = lnet_set_link_fatal_state(ni, 1); goto ni_done; } in_dev_for_each_ifa_rtnl(ifa, in_dev) { @@ -2787,27 +2774,20 @@ kiblnd_handle_link_state_change(struct net_device *dev, if (!found_ip) { CDEBUG(D_NET, "Interface %s has no matching ip\n", dev->name); - CDEBUG(D_NET, "%s: set link fatal state to 1\n", - libcfs_nidstr(&net->ibn_ni->ni_nid)); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - 1); + ni_state_before = lnet_set_link_fatal_state(ni, 1); goto ni_done; } if (link_down) { - CDEBUG(D_NET, "%s: set link fatal state to 1\n", - libcfs_nidstr(&net->ibn_ni->ni_nid)); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - link_down); + ni_state_before = lnet_set_link_fatal_state(ni, 1); } else { - CDEBUG(D_NET, "%s: set link fatal state to %u\n", - libcfs_nidstr(&net->ibn_ni->ni_nid), - (kiblnd_get_link_status(dev) == 0)); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - (kiblnd_get_link_status(dev) == 0)); + state = (lnet_get_link_status(dev) == 0); + ni_state_before = lnet_set_link_fatal_state(ni, + state); } ni_done: if (!update_ping_buf && + (ni->ni_state == LNET_NI_STATE_ACTIVE) && (atomic_read(&ni->ni_fatal_error_on) != ni_state_before)) update_ping_buf = true; } @@ -2827,6 +2807,8 @@ kiblnd_handle_inetaddr_change(struct in_ifaddr *ifa, unsigned long event) struct net_device *event_netdev = ifa->ifa_dev->dev; u32 ni_state_before; bool update_ping_buf = false; + struct lnet_ni *ni = NULL; + bool link_down; event_kibdev = kiblnd_dev_search(event_netdev->name); @@ -2838,12 +2820,11 @@ kiblnd_handle_inetaddr_change(struct in_ifaddr *ifa, unsigned long event) list_for_each_entry_safe(net, cnxt, &event_kibdev->ibd_nets, ibn_list) { - CDEBUG(D_NET, "%s: set link fatal state to %u\n", - libcfs_nidstr(&net->ibn_ni->ni_nid), - (event == NETDEV_DOWN)); - ni_state_before = atomic_xchg(&net->ibn_ni->ni_fatal_error_on, - (event == NETDEV_DOWN)); + ni = net->ibn_ni; + link_down = (event == NETDEV_DOWN); + ni_state_before = lnet_set_link_fatal_state(ni, link_down); if (!update_ping_buf && + (ni->ni_state == LNET_NI_STATE_ACTIVE) && ((event == NETDEV_DOWN) != ni_state_before)) update_ping_buf = true; } @@ -3199,6 +3180,7 @@ static int kiblnd_startup(struct lnet_ni *ni) int rc; int i; bool newdev; + struct net_device *netdev; LASSERT(ni->ni_net->net_lnd == &the_o2iblnd); @@ -3312,6 +3294,16 @@ static int kiblnd_startup(struct lnet_ni *ni) /* for health check */ if (ibdev->ibd_hdev->ibh_state == IBLND_DEV_PORT_DOWN) kiblnd_set_ni_fatal_on(ibdev->ibd_hdev, 1); + + rcu_read_lock(); + netdev = dev_get_by_name_rcu(ni->ni_net_ns, net->ibn_dev->ibd_ifname); + if ((netdev->reg_state == NETREG_UNREGISTERING || + netdev->operstate != IF_OPER_UP) || + lnet_get_link_status(netdev) == 0) { + kiblnd_set_ni_fatal_on(ibdev->ibd_hdev, 1); + } + rcu_read_unlock(); + write_unlock_irqrestore(&kiblnd_data.kib_global_lock, flags); net->ibn_init = IBLND_INIT_ALL; diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index 6028520308ff..d8fc20d18d44 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -47,6 +47,77 @@ static struct lnet_lnd the_ksocklnd; struct ksock_nal_data ksocknal_data; +static int ksocknal_ip2index(struct sockaddr *addr, struct lnet_ni *ni, + int *dev_status) +{ + struct net_device *dev; + int ret = -1; + const struct in_ifaddr *ifa; + + *dev_status = -1; + + if (addr->sa_family != AF_INET && addr->sa_family != AF_INET6) + return ret; + + rcu_read_lock(); + for_each_netdev(ni->ni_net_ns, dev) { + int flags = dev_get_flags(dev); + struct in_device *in_dev; + + if (flags & IFF_LOOPBACK) /* skip the loopback IF */ + continue; + + if (!(flags & IFF_UP)) + continue; + + switch (addr->sa_family) { + case AF_INET: + in_dev = __in_dev_get_rcu(dev); + if (!in_dev) + continue; + + in_dev_for_each_ifa_rcu(ifa, in_dev) { + if (ifa->ifa_local == + ((struct sockaddr_in *)addr)->sin_addr.s_addr) + ret = dev->ifindex; + } + break; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: { + struct inet6_dev *in6_dev; + const struct inet6_ifaddr *ifa6; + struct sockaddr_in6 *addr6 = (struct sockaddr_in6 *)addr; + + in6_dev = __in6_dev_get(dev); + if (!in6_dev) + continue; + + list_for_each_entry_rcu(ifa6, &in6_dev->addr_list, if_list) { + if (ipv6_addr_cmp(&ifa6->addr, + &addr6->sin6_addr) == 0) + ret = dev->ifindex; + } + break; + } +#endif /* IS_ENABLED(CONFIG_IPV6) */ + } + if (ret >= 0) + break; + } + + rcu_read_unlock(); + if (ret >= 0) + *dev_status = 1; + + if ((ret == -1) || + (dev->reg_state == NETREG_UNREGISTERING || + dev->operstate != IF_OPER_UP) || + (lnet_get_link_status(dev) == 0)) + *dev_status = 0; + + return ret; +} + static struct ksock_conn_cb * ksocknal_create_conn_cb(struct sockaddr *addr) { @@ -1856,25 +1927,6 @@ ksocknal_free_buffers(void) } } -static int ksocknal_get_link_status(struct net_device *dev) -{ - int ret = -1; - - LASSERT(dev); - - if (!netif_running(dev)) { - ret = 0; - CDEBUG(D_NET, "device not running\n"); - } - /* Some devices may not be providing link settings */ - else if (dev->ethtool_ops->get_link) { - ret = dev->ethtool_ops->get_link(dev); - CDEBUG(D_NET, "get_link returns %u\n", ret); - } - - return ret; -} - static int ksocknal_handle_link_state_change(struct net_device *dev, unsigned char operstate) @@ -1891,6 +1943,7 @@ ksocknal_handle_link_state_change(struct net_device *dev, u32 ni_state_before; bool update_ping_buf = false; const struct in_ifaddr *ifa; + int state; ifindex = dev->ifindex; @@ -1921,7 +1974,7 @@ ksocknal_handle_link_state_change(struct net_device *dev, continue; if (dev->reg_state == NETREG_UNREGISTERING) { - /* Device is being unregitering, we need to clear the + /* Device is being unregitered, we need to clear the * index, it can change when device will be back */ ksi->ksni_index = -1; @@ -1934,9 +1987,7 @@ ksocknal_handle_link_state_change(struct net_device *dev, if (!in_dev) { CDEBUG(D_NET, "Interface %s has no IPv4 status.\n", dev->name); - CDEBUG(D_NET, "set link fatal state to 1\n"); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - 1); + ni_state_before = lnet_set_link_fatal_state(ni, 1); goto ni_done; } in_dev_for_each_ifa_rtnl(ifa, in_dev) { @@ -1947,24 +1998,20 @@ ksocknal_handle_link_state_change(struct net_device *dev, if (!found_ip) { CDEBUG(D_NET, "Interface %s has no matching ip\n", dev->name); - CDEBUG(D_NET, "set link fatal state to 1\n"); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - 1); + ni_state_before = lnet_set_link_fatal_state(ni, 1); goto ni_done; } if (link_down) { - CDEBUG(D_NET, "set link fatal state to 1\n"); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - 1); + ni_state_before = lnet_set_link_fatal_state(ni, 1); } else { - CDEBUG(D_NET, "set link fatal state to %u\n", - (ksocknal_get_link_status(dev) == 0)); - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - (ksocknal_get_link_status(dev) == 0)); + state = (lnet_get_link_status(dev) == 0); + ni_state_before = lnet_set_link_fatal_state(ni, + state); } ni_done: if (!update_ping_buf && + (ni->ni_state == LNET_NI_STATE_ACTIVE) && (atomic_read(&ni->ni_fatal_error_on) != ni_state_before)) update_ping_buf = true; } @@ -1979,7 +2026,7 @@ ksocknal_handle_link_state_change(struct net_device *dev, static int ksocknal_handle_inetaddr_change(struct in_ifaddr *ifa, unsigned long event) { - struct lnet_ni *ni; + struct lnet_ni *ni = NULL; struct ksock_net *net; struct ksock_net *cnxt; struct net_device *event_netdev = ifa->ifa_dev->dev; @@ -1988,6 +2035,7 @@ ksocknal_handle_inetaddr_change(struct in_ifaddr *ifa, unsigned long event) struct sockaddr_in *sa; u32 ni_state_before; bool update_ping_buf = false; + bool link_down; if (!ksocknal_data.ksnd_nnets) goto out; @@ -2005,12 +2053,13 @@ ksocknal_handle_inetaddr_change(struct in_ifaddr *ifa, unsigned long event) continue; if (sa->sin_addr.s_addr == ifa->ifa_local) { - CDEBUG(D_NET, "set link fatal state to %u\n", - (event == NETDEV_DOWN)); ni = net->ksnn_ni; - ni_state_before = atomic_xchg(&ni->ni_fatal_error_on, - (event == NETDEV_DOWN)); + link_down = (event == NETDEV_DOWN); + ni_state_before = lnet_set_link_fatal_state(ni, + link_down); + if (!update_ping_buf && + (ni->ni_state == LNET_NI_STATE_ACTIVE) && ((event == NETDEV_DOWN) != ni_state_before)) update_ping_buf = true; } @@ -2455,6 +2504,7 @@ ksocknal_startup(struct lnet_ni *ni) struct ksock_interface *ksi = NULL; struct lnet_inetdev *ifaces = NULL; int rc, if_idx; + int dev_status; LASSERT(ni->ni_net->net_lnd == &the_ksocklnd); @@ -2521,6 +2571,11 @@ ksocknal_startup(struct lnet_ni *ni) if (rc) goto out_net; + if (ksocknal_ip2index((struct sockaddr *)&ksi->ksni_addr, + ni, &dev_status) < 0 || + dev_status <= 0) + lnet_set_link_fatal_state(ni, 1); + list_add(&net->ksnn_list, &ksocknal_data.ksnd_nets); net->ksnn_ni = ni; ksocknal_data.ksnd_nnets++; diff --git a/net/lnet/lnet/config.c b/net/lnet/lnet/config.c index c239f9caa0f5..ca498b0670e5 100644 --- a/net/lnet/lnet/config.c +++ b/net/lnet/lnet/config.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -1489,6 +1490,37 @@ lnet_match_networks(const char **networksp, const char *ip2nets, return count; } +u32 lnet_set_link_fatal_state(struct lnet_ni *ni, unsigned int link_state) +{ + CDEBUG(D_NET, "%s: set link fatal state to %u\n", + libcfs_nidstr(&ni->ni_nid), link_state); + return atomic_xchg(&ni->ni_fatal_error_on, link_state); +} +EXPORT_SYMBOL(lnet_set_link_fatal_state); + +int lnet_get_link_status(struct net_device *dev) +{ + int ret = -1; + + if (!dev) + return -1; + + if (!netif_running(dev)) { + ret = 0; + CDEBUG(D_NET, "device idx %d not running\n", dev->ifindex); + } + /* Some devices may not be providing link settings */ + else if (dev->ethtool_ops->get_link) { + ret = dev->ethtool_ops->get_link(dev); + CDEBUG(D_NET, "device idx %d get_link %u\n", + ret, + dev->ifindex); + } + + return ret; +} +EXPORT_SYMBOL(lnet_get_link_status); + int lnet_inet_enumerate(struct lnet_inetdev **dev_list, struct net *ns, bool v6) { struct lnet_inetdev *ifaces = NULL;