From patchwork Thu Jun 9 12:32:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5118CCA482 for ; Thu, 9 Jun 2022 12:33:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238198AbiFIMdf (ORCPT ); Thu, 9 Jun 2022 08:33:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245125AbiFIMdX (ORCPT ); Thu, 9 Jun 2022 08:33:23 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58311186FB for ; Thu, 9 Jun 2022 05:33:21 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 318BBEF0; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2A58BD438A; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, "John L. Hammond" , James Simmons Subject: [PATCH 01/18] lustre: llite: reenable fast_read by default Date: Thu, 9 Jun 2022 08:32:57 -0400 Message-Id: <1654777994-29806-2-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: "John L. Hammond" Reenable fast_read by default. WC-bug-id: https://jira.whamcloud.com/browse/LU-14541 Lustre-commit: a94e28fda44077f77 ("LU-14541 llite: reenable fast_read by default") Signed-off-by: John L. Hammond Reviewed-on: https://review.whamcloud.com/47298 Reviewed-by: Andreas Dilger Reviewed-by: Patrick Farrell Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/llite_lib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 4578a9e..ad77ef0 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -169,7 +169,7 @@ static struct ll_sb_info *ll_init_sbi(void) atomic_set(&sbi->ll_sa_running, 0); atomic_set(&sbi->ll_agl_total, 0); set_bit(LL_SBI_AGL_ENABLED, sbi->ll_flags); - /* Disable LL_SBI_FAST_READ by default, see LU-15815. */ + set_bit(LL_SBI_FAST_READ, sbi->ll_flags); set_bit(LL_SBI_TINY_WRITE, sbi->ll_flags); set_bit(LL_SBI_PARALLEL_DIO, sbi->ll_flags); ll_sbi_set_encrypt(sbi, true); From patchwork Thu Jun 9 12:32:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E76D8CCA47F for ; Thu, 9 Jun 2022 12:33:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244796AbiFIMde (ORCPT ); Thu, 9 Jun 2022 08:33:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245234AbiFIMd0 (ORCPT ); Thu, 9 Jun 2022 08:33:26 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB5D1186FB for ; Thu, 9 Jun 2022 05:33:23 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 39621EF1; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2D622D439B; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Patrick Farrell , James Simmons Subject: [PATCH 02/18] lustre: llite: Check vmpage in releasepage Date: Thu, 9 Jun 2022 08:32:58 -0400 Message-Id: <1654777994-29806-3-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Patrick Farrell We cannot release a page if the vmpage reference count is >1, otherwise we will detach a vmpage from Lustre when the page is still referenced in the VM. This creates a situation where page discard for lock cancellation will not find the page, so we can get stale data reads. This re-introduces the LU-12587 issue where direct I/O on a client falls back to buffered I/O if there are pages in cache, since it cannot flush them. This is annoying but not a huge problem. WC-bug-id: https://jira.whamcloud.com/browse/LU-14541 Lustre-commit: c524079f4f59a39b9 ("LU-14541 llite: Check vmpage in releasepage") Signed-off-by: Patrick Farrell Reviewed-on: https://review.whamcloud.com/47262 Reviewed-by: Andreas Dilger Reviewed-by: John L. Hammond Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 9 +++++++++ fs/lustre/llite/rw26.c | 19 +++++++++++++------ fs/lustre/osc/osc_page.c | 9 +++++++-- 3 files changed, 29 insertions(+), 8 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index ab7f0f2..b98109d 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -91,6 +91,7 @@ #include #include #include +#include #include #include #include @@ -1071,6 +1072,14 @@ static inline bool __page_in_use(const struct cl_page *page, int refc) */ #define cl_page_in_use_noref(pg) __page_in_use(pg, 0) +/* references: cl_page, page cache, optional + refcount for caller reference + * (always 0 or 1 currently) + */ +static inline int vmpage_in_use(struct page *vmpage, int refcount) +{ + return (page_count(vmpage) - page_mapcount(vmpage) > 2 + refcount); +} + /** @} cl_page */ /** \addtogroup cl_lock cl_lock diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index a5cdb01..8b379ca 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -102,7 +102,7 @@ static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) { struct lu_env *env; struct cl_object *obj; - struct cl_page *page; + struct cl_page *clpage; struct address_space *mapping; int result = 0; @@ -118,16 +118,23 @@ static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) if (!obj) return 1; - page = cl_vmpage_page(vmpage, obj); - if (!page) + clpage = cl_vmpage_page(vmpage, obj); + if (!clpage) return 1; env = cl_env_percpu_get(); LASSERT(!IS_ERR(env)); - if (!cl_page_in_use(page)) { + /* we must not delete the cl_page if the vmpage is in use, otherwise we + * disconnect the vmpage from Lustre while it's still alive(!), which + * means we won't find it to discard on lock cancellation. + * + * References here are: caller + cl_page + page cache. + * Any other references are potentially transient and must be ignored. + */ + if (!cl_page_in_use(clpage) && !vmpage_in_use(vmpage, 1)) { result = 1; - cl_page_delete(env, page); + cl_page_delete(env, clpage); } /* To use percpu env array, the call path can not be rescheduled; @@ -144,7 +151,7 @@ static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) * that we won't get into object delete path. */ LASSERT(cl_object_refc(obj) > 1); - cl_page_put(env, page); + cl_page_put(env, clpage); cl_env_percpu_put(env); return result; diff --git a/fs/lustre/osc/osc_page.c b/fs/lustre/osc/osc_page.c index f46b4e7..b56bc1a 100644 --- a/fs/lustre/osc/osc_page.c +++ b/fs/lustre/osc/osc_page.c @@ -539,8 +539,13 @@ static inline bool lru_page_busy(struct client_obd *cli, struct cl_page *page) if (cli->cl_cache->ccc_unstable_check) { struct page *vmpage = cl_page_vmpage(page); - /* vmpage have two known users: cl_page and VM page cache */ - if (page_count(vmpage) - page_mapcount(vmpage) > 2) + /* this check is racy because the vmpage is not locked, but + * that's OK - the code which does the actual page release + * checks this again before releasing + * + * vmpage have two known users: cl_page and VM page cache + */ + if (vmpage_in_use(vmpage, 0)) return true; } return false; From patchwork Thu Jun 9 12:32:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F0C2CCA487 for ; Thu, 9 Jun 2022 12:33:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245125AbiFIMdg (ORCPT ); Thu, 9 Jun 2022 08:33:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245380AbiFIMd1 (ORCPT ); Thu, 9 Jun 2022 08:33:27 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1A74186FB for ; Thu, 9 Jun 2022 05:33:26 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 3A850EF2; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2FA3ED43A0; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Oleg Drokin , James Simmons Subject: [PATCH 03/18] lustre: update version to 2.15.50 Date: Thu, 9 Jun 2022 08:32:59 -0400 Message-Id: <1654777994-29806-4-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Oleg Drokin New tag 2.15.50 Signed-off-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lustre/lustre_ver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/lustre/lustre_ver.h b/include/uapi/linux/lustre/lustre_ver.h index a17edd6..bcee87c 100644 --- a/include/uapi/linux/lustre/lustre_ver.h +++ b/include/uapi/linux/lustre/lustre_ver.h @@ -3,9 +3,9 @@ #define LUSTRE_MAJOR 2 #define LUSTRE_MINOR 15 -#define LUSTRE_PATCH 0 +#define LUSTRE_PATCH 50 #define LUSTRE_FIX 0 -#define LUSTRE_VERSION_STRING "2.15.0" +#define LUSTRE_VERSION_STRING "2.15.50" #define OBD_OCD_VERSION(major, minor, patch, fix) \ (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix)) From patchwork Thu Jun 9 12:33:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4933ECCA473 for ; Thu, 9 Jun 2022 12:33:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235786AbiFIMdd (ORCPT ); Thu, 9 Jun 2022 08:33:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245425AbiFIMda (ORCPT ); Thu, 9 Jun 2022 08:33:30 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12501192AB for ; Thu, 9 Jun 2022 05:33:27 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 3D405EF3; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 31CF9D43A3; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Etienne AUJAMES , James Simmons Subject: [PATCH 04/18] lustre: llog: read canceled records in llog_backup Date: Thu, 9 Jun 2022 08:33:00 -0400 Message-Id: <1654777994-29806-5-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Etienne AUJAMES llog_backup() do not reproduce index "holes" in the generated copy. This could result to a llog copy indexes different from the source. Then it might confuse the configuration update mechanism that rely on indexes between the MGS source and the target copy. This index gaps can be caused by "lctl --device MGS llog_cancel". This patch add "raw" read mode to llog_process* to read canceled records. WC-bug-id: https://jira.whamcloud.com/browse/LU-15000 Lustre-commit: d8e2723b4e9409954 ("LU-15000 llog: read canceled records in llog_backup") Signed-off-by: Etienne AUJAMES Reviewed-on: https://review.whamcloud.com/46552 Reviewed-by: Dominique Martinet Reviewed-by: DELBARY Gael Reviewed-by: Stephane Thiell Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/lustre_log.h | 11 +++++++++++ fs/lustre/obdclass/llog.c | 22 +++++++++++++++++----- fs/lustre/obdclass/llog_cat.c | 5 ++++- fs/lustre/obdclass/obd_config.c | 1 + 4 files changed, 33 insertions(+), 6 deletions(-) diff --git a/fs/lustre/include/lustre_log.h b/fs/lustre/include/lustre_log.h index 1fc7729..2e43d56 100644 --- a/fs/lustre/include/lustre_log.h +++ b/fs/lustre/include/lustre_log.h @@ -94,6 +94,13 @@ int llog_open(const struct lu_env *env, struct llog_ctxt *ctxt, /* llog_process flags */ #define LLOG_FLAG_NODEAMON 0x0001 +/* llog read mode, LLOG_READ_MODE_RAW will process llog canceled records */ +enum llog_read_mode { + LLOG_READ_MODE_NORMAL = 0x0000, + LLOG_READ_MODE_RAW = 0x0001, +}; + + /* llog_cat.c - catalog api */ struct llog_process_data { /** @@ -122,6 +129,10 @@ struct llog_process_cat_data { * Temporary stored last_idx while scanning log. */ int lpcd_last_idx; + /** + * llog read mode + */ + enum llog_read_mode lpcd_read_mode; }; struct thandle; diff --git a/fs/lustre/obdclass/llog.c b/fs/lustre/obdclass/llog.c index acede87..0cc64ce 100644 --- a/fs/lustre/obdclass/llog.c +++ b/fs/lustre/obdclass/llog.c @@ -256,6 +256,15 @@ int llog_verify_record(const struct llog_handle *llh, struct llog_rec_hdr *rec) return 0; } +static inline bool llog_is_index_skipable(int idx, struct llog_log_hdr *llh, + struct llog_process_cat_data *cd) +{ + if (cd && (cd->lpcd_read_mode & LLOG_READ_MODE_RAW)) + return false; + + return !test_bit_le(idx, LLOG_HDR_BITMAP(llh)); +} + static int llog_process_thread(void *arg) { struct llog_process_info *lpi = arg; @@ -291,6 +300,8 @@ static int llog_process_thread(void *arg) } if (cd && cd->lpcd_last_idx) last_index = cd->lpcd_last_idx; + else if (cd && (cd->lpcd_read_mode & LLOG_READ_MODE_RAW)) + last_index = loghandle->lgh_last_idx; else last_index = LLOG_HDR_BITMAP_SIZE(llh) - 1; @@ -303,7 +314,7 @@ static int llog_process_thread(void *arg) /* skip records not set in bitmap */ while (index <= last_index && - !test_bit_le(index, LLOG_HDR_BITMAP(llh))) + llog_is_index_skipable(index, llh, cd)) ++index; if (index > last_index) @@ -451,8 +462,8 @@ static int llog_process_thread(void *arg) loghandle->lgh_cur_offset = (char *)rec - (char *)buf + chunk_offset; - /* if set, process the callback on this record */ - if (test_bit_le(index, LLOG_HDR_BITMAP(llh))) { + /* if needed, process the callback on this record */ + if (!llog_is_index_skipable(index, llh, cd)) { rc = lpi->lpi_cb(lpi->lpi_env, loghandle, rec, lpi->lpi_cbdata); last_called_index = index; @@ -522,11 +533,12 @@ int llog_process_or_fork(const struct lu_env *env, lpi->lpi_catdata = catdata; CDEBUG(D_OTHER, - "Processing " DFID " flags 0x%03x startcat %d startidx %d first_idx %d last_idx %d\n", + "Processing " DFID " flags 0x%03x startcat %d startidx %d first_idx %d last_idx %d read_mode %d\n", PFID(&loghandle->lgh_id.lgl_oi.oi_fid), flags, (flags & LLOG_F_IS_CAT) && d ? d->lpd_startcat : -1, (flags & LLOG_F_IS_CAT) && d ? d->lpd_startidx : -1, - cd ? cd->lpcd_first_idx : -1, cd ? cd->lpcd_last_idx : -1); + cd ? cd->lpcd_first_idx : -1, cd ? cd->lpcd_last_idx : -1, + cd ? cd->lpcd_read_mode : -1); if (fork) { struct task_struct *task; diff --git a/fs/lustre/obdclass/llog_cat.c b/fs/lustre/obdclass/llog_cat.c index 7f55895..753422b 100644 --- a/fs/lustre/obdclass/llog_cat.c +++ b/fs/lustre/obdclass/llog_cat.c @@ -197,6 +197,7 @@ static int llog_cat_process_cb(const struct lu_env *env, else if (d->lpd_startidx > 0) { struct llog_process_cat_data cd; + cd.lpcd_read_mode = LLOG_READ_MODE_NORMAL; cd.lpcd_first_idx = d->lpd_startidx; cd.lpcd_last_idx = 0; rc = llog_process_or_fork(env, llh, d->lpd_cb, d->lpd_data, @@ -231,7 +232,9 @@ static int llog_cat_process_or_fork(const struct lu_env *env, d.lpd_startidx = startidx; if (llh->llh_cat_idx > cat_llh->lgh_last_idx) { - struct llog_process_cat_data cd; + struct llog_process_cat_data cd = { + .lpcd_read_mode = LLOG_READ_MODE_NORMAL + }; CWARN("%s: catlog " DFID " crosses index zero\n", loghandle2name(cat_llh), diff --git a/fs/lustre/obdclass/obd_config.c b/fs/lustre/obdclass/obd_config.c index cb70ed5..4db7399 100644 --- a/fs/lustre/obdclass/obd_config.c +++ b/fs/lustre/obdclass/obd_config.c @@ -1401,6 +1401,7 @@ int class_config_parse_llog(const struct lu_env *env, struct llog_ctxt *ctxt, { struct llog_process_cat_data cd = { .lpcd_first_idx = 0, + .lpcd_read_mode = LLOG_READ_MODE_NORMAL, }; struct llog_handle *llh; llog_cb_t callback; From patchwork Thu Jun 9 12:33:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 981A3CCA489 for ; Thu, 9 Jun 2022 12:33:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238023AbiFIMdh (ORCPT ); Thu, 9 Jun 2022 08:33:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245441AbiFIMdc (ORCPT ); Thu, 9 Jun 2022 08:33:32 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 008F5192AB for ; Thu, 9 Jun 2022 05:33:30 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 3ECD6EF4; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 36142D43AC; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 05/18] lnet: change LNetPrimaryNID to use struct lnet_nid Date: Thu, 9 Jun 2022 08:33:01 -0400 Message-Id: <1654777994-29806-6-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown Rather than taking and returning a 4-byte-addr nid, LNetPrimaryNID now takes a pointer to a struct lnet_nid, and updates it in-place. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: ac881498fa19e6b04 ("LU-10391 lnet: change LNetPrimaryNID to use struct lnet_nid") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/43616 Reviewed-by: Serguei Smirnov Reviewed-by: Chris Horn Reviewed-by: Cyril Bordage Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/connection.c | 2 +- include/linux/lnet/api.h | 2 +- net/lnet/lnet/peer.c | 21 +++++++++++---------- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/fs/lustre/ptlrpc/connection.c b/fs/lustre/ptlrpc/connection.c index d1f53c6..58161fe 100644 --- a/fs/lustre/ptlrpc/connection.c +++ b/fs/lustre/ptlrpc/connection.c @@ -82,8 +82,8 @@ struct ptlrpc_connection * struct ptlrpc_connection *conn, *conn2; struct lnet_processid peer; - peer4.nid = LNetPrimaryNID(peer4.nid); lnet_pid4_to_pid(peer4, &peer); + LNetPrimaryNID(&peer.nid); conn = rhashtable_lookup_fast(&conn_hash, &peer, conn_hash_params); if (conn) { ptlrpc_connection_addref(conn); diff --git a/include/linux/lnet/api.h b/include/linux/lnet/api.h index 6d8e915..447b41d 100644 --- a/include/linux/lnet/api.h +++ b/include/linux/lnet/api.h @@ -77,7 +77,7 @@ */ int LNetGetId(unsigned int index, struct lnet_processid *id); int LNetDist(lnet_nid_t nid, lnet_nid_t *srcnid, u32 *order); -lnet_nid_t LNetPrimaryNID(lnet_nid_t nid); +void LNetPrimaryNID(struct lnet_nid *nid); /** @} lnet_addr */ diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 714326a..2055f31 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -1430,18 +1430,20 @@ struct lnet_peer_ni * } EXPORT_SYMBOL(LNetAddPeer); -/* FIXME support large-addr nid */ -lnet_nid_t -LNetPrimaryNID(lnet_nid_t nid) +void LNetPrimaryNID(struct lnet_nid *nid) { struct lnet_peer *lp; struct lnet_peer_ni *lpni; - lnet_nid_t primary_nid = nid; + struct lnet_nid orig; int rc = 0; int cpt; + if (!nid || nid_is_lo0(nid)) + return; + orig = *nid; + cpt = lnet_net_lock_current(); - lpni = lnet_nid2peerni_locked(nid, LNET_NID_ANY, cpt); + lpni = lnet_peerni_by_nid_locked(nid, NULL, cpt); if (IS_ERR(lpni)) { rc = PTR_ERR(lpni); goto out_unlock; @@ -1468,7 +1470,7 @@ struct lnet_peer_ni * * and lookup the lpni again */ lnet_peer_ni_decref_locked(lpni); - lpni = lnet_find_peer_ni_locked(nid); + lpni = lnet_peer_ni_find_locked(nid); if (!lpni) { rc = -ENOENT; goto out_unlock; @@ -1483,15 +1485,14 @@ struct lnet_peer_ni * if (lnet_is_discovery_disabled(lp)) break; } - primary_nid = lnet_nid_to_nid4(&lp->lp_primary_nid); + *nid = lp->lp_primary_nid; out_decref: lnet_peer_ni_decref_locked(lpni); out_unlock: lnet_net_unlock(cpt); - CDEBUG(D_NET, "NID %s primary NID %s rc %d\n", libcfs_nid2str(nid), - libcfs_nid2str(primary_nid), rc); - return primary_nid; + CDEBUG(D_NET, "NID %s primary NID %s rc %d\n", libcfs_nidstr(&orig), + libcfs_nidstr(nid), rc); } EXPORT_SYMBOL(LNetPrimaryNID); From patchwork Thu Jun 9 12:33:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875522 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9C99CCA473 for ; Thu, 9 Jun 2022 12:33:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245364AbiFIMdh (ORCPT ); Thu, 9 Jun 2022 08:33:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245165AbiFIMde (ORCPT ); Thu, 9 Jun 2022 08:33:34 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 716EB186FB for ; Thu, 9 Jun 2022 05:33:32 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 429FEEF5; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 38701D4403; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 06/18] lnet: alter lnet_drop_rule_match() to take lnet_nid Date: Thu, 9 Jun 2022 08:33:02 -0400 Message-Id: <1654777994-29806-7-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown The local nid passed to lnet_drop_rule_match() is now a 16-byte nid. Various support functions are also changed to embrace 'struct lnet_nid'. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 57b7b3d36f5fa1527 ("LU-10391 lnet: alter lnet_drop_rule_match() to take lnet_nid") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/43617 Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 2 +- net/lnet/lnet/lib-move.c | 3 +-- net/lnet/lnet/lib-msg.c | 3 +-- net/lnet/lnet/net_fault.c | 40 +++++++++++++++++++++++----------------- 4 files changed, 26 insertions(+), 22 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index b6a7a54..2e3c391 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -751,7 +751,7 @@ void lnet_drop_message(struct lnet_ni *ni, int cpt, void *private, int lnet_fault_init(void); void lnet_fault_fini(void); -bool lnet_drop_rule_match(struct lnet_hdr *hdr, lnet_nid_t local_nid, +bool lnet_drop_rule_match(struct lnet_hdr *hdr, struct lnet_nid *local_nid, enum lnet_msg_hstatus *hstatus); int lnet_delay_rule_add(struct lnet_fault_attr *attr); diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 0496bf5..080bfe6 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -4379,9 +4379,8 @@ void lnet_monitor_thr_stop(void) goto drop; } - /* FIXME need to support large-addr nid */ if (!list_empty(&the_lnet.ln_drop_rules) && - lnet_drop_rule_match(hdr, lnet_nid_to_nid4(&ni->ni_nid), NULL)) { + lnet_drop_rule_match(hdr, &ni->ni_nid, NULL)) { CDEBUG(D_NET, "%s, src %s, dst %s: Dropping %s to simulate silent message loss\n", libcfs_nidstr(from_nid), libcfs_nidstr(&src_nid), libcfs_nidstr(&dest_nid), lnet_msgtyp2str(type)); diff --git a/net/lnet/lnet/lib-msg.c b/net/lnet/lnet/lib-msg.c index f476975..95695b2 100644 --- a/net/lnet/lnet/lib-msg.c +++ b/net/lnet/lnet/lib-msg.c @@ -1115,8 +1115,7 @@ return false; /* match only health rules */ - if (!lnet_drop_rule_match(&msg->msg_hdr, LNET_NID_ANY, - hstatus)) + if (!lnet_drop_rule_match(&msg->msg_hdr, NULL, hstatus)) return false; CDEBUG(D_NET, diff --git a/net/lnet/lnet/net_fault.c b/net/lnet/lnet/net_fault.c index 1f08b38..fe7a07c 100644 --- a/net/lnet/lnet/net_fault.c +++ b/net/lnet/lnet/net_fault.c @@ -65,12 +65,16 @@ struct lnet_drop_rule { }; static bool -lnet_fault_nid_match(lnet_nid_t nid, lnet_nid_t msg_nid) +lnet_fault_nid_match(lnet_nid_t nid, struct lnet_nid *msg_nid) { - if (nid == msg_nid || nid == LNET_NID_ANY) + if (nid == LNET_NID_ANY) + return true; + if (!msg_nid) + return false; + if (lnet_nid_to_nid4(msg_nid) == nid) return true; - if (LNET_NIDNET(nid) != LNET_NIDNET(msg_nid)) + if (LNET_NIDNET(nid) != LNET_NID_NET(msg_nid)) return false; /* 255.255.255.255@net is wildcard for all addresses in a network */ @@ -78,8 +82,10 @@ struct lnet_drop_rule { } static bool -lnet_fault_attr_match(struct lnet_fault_attr *attr, lnet_nid_t src, - lnet_nid_t local_nid, lnet_nid_t dst, +lnet_fault_attr_match(struct lnet_fault_attr *attr, + struct lnet_nid *src, + struct lnet_nid *local_nid, + struct lnet_nid *dst, unsigned int type, unsigned int portal) { if (!lnet_fault_nid_match(attr->fa_src, src) || @@ -339,8 +345,10 @@ struct lnet_drop_rule { * decide whether should drop this message or not */ static bool -drop_rule_match(struct lnet_drop_rule *rule, lnet_nid_t src, - lnet_nid_t local_nid, lnet_nid_t dst, +drop_rule_match(struct lnet_drop_rule *rule, + struct lnet_nid *src, + struct lnet_nid *local_nid, + struct lnet_nid *dst, unsigned int type, unsigned int portal, enum lnet_msg_hstatus *hstatus) { @@ -424,11 +432,9 @@ struct lnet_drop_rule { */ bool lnet_drop_rule_match(struct lnet_hdr *hdr, - lnet_nid_t local_nid, + struct lnet_nid *local_nid, enum lnet_msg_hstatus *hstatus) { - lnet_nid_t src = lnet_nid_to_nid4(&hdr->src_nid); - lnet_nid_t dst = lnet_nid_to_nid4(&hdr->dest_nid); unsigned int typ = hdr->type; struct lnet_drop_rule *rule; unsigned int ptl = -1; @@ -446,7 +452,8 @@ struct lnet_drop_rule { cpt = lnet_net_lock_current(); list_for_each_entry(rule, &the_lnet.ln_drop_rules, dr_link) { - drop = drop_rule_match(rule, src, local_nid, dst, typ, ptl, + drop = drop_rule_match(rule, &hdr->src_nid, local_nid, + &hdr->dest_nid, typ, ptl, hstatus); if (drop) break; @@ -530,15 +537,15 @@ struct delay_daemon_data { * decide whether should delay this message or not */ static bool -delay_rule_match(struct lnet_delay_rule *rule, lnet_nid_t src, - lnet_nid_t dst, unsigned int type, unsigned int portal, +delay_rule_match(struct lnet_delay_rule *rule, struct lnet_nid *src, + struct lnet_nid *dst, unsigned int type, unsigned int portal, struct lnet_msg *msg) { struct lnet_fault_attr *attr = &rule->dl_attr; bool delay; time64_t now = ktime_get_seconds(); - if (!lnet_fault_attr_match(attr, src, LNET_NID_ANY, + if (!lnet_fault_attr_match(attr, src, NULL, dst, type, portal)) return false; @@ -605,8 +612,6 @@ struct delay_daemon_data { lnet_delay_rule_match_locked(struct lnet_hdr *hdr, struct lnet_msg *msg) { struct lnet_delay_rule *rule; - lnet_nid_t src = lnet_nid_to_nid4(&hdr->src_nid); - lnet_nid_t dst = lnet_nid_to_nid4(&hdr->dest_nid); unsigned int typ = hdr->type; unsigned int ptl = -1; @@ -622,7 +627,8 @@ struct delay_daemon_data { ptl = le32_to_cpu(hdr->msg.get.ptl_index); list_for_each_entry(rule, &the_lnet.ln_delay_rules, dl_link) { - if (delay_rule_match(rule, src, dst, typ, ptl, msg)) + if (delay_rule_match(rule, &hdr->src_nid, &hdr->dest_nid, + typ, ptl, msg)) return true; } From patchwork Thu Jun 9 12:33:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8E3CCCA47B for ; Thu, 9 Jun 2022 12:33:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245165AbiFIMdi (ORCPT ); Thu, 9 Jun 2022 08:33:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245355AbiFIMdg (ORCPT ); Thu, 9 Jun 2022 08:33:36 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB105186FB for ; Thu, 9 Jun 2022 05:33:34 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 473E9EF7; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3CCFBD4381; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 07/18] lnet: Change LNetDist to work with struct lnet_nid Date: Thu, 9 Jun 2022 08:33:03 -0400 Message-Id: <1654777994-29806-8-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown LNetDist now takes and returns 'struct lnet_nid' lustre_uuid_to_peer() is also updated. The 'dst' and 'src' parameters to LNetDist are now both pointers, and that can point to the same 'struct lnet_nid'. Code needs to be careful not to set *src until after the last use of *dst. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: c87f70acd86c59425 ("LU-10391 lnet: Change LNetDist to work with struct lnet_nid") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/43618 Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/obd_class.h | 3 ++- fs/lustre/obdclass/lustre_peer.c | 4 ++-- fs/lustre/ptlrpc/events.c | 12 ++++++------ include/linux/lnet/api.h | 2 +- net/lnet/lnet/api-ni.c | 4 +++- net/lnet/lnet/lib-move.c | 35 ++++++++++++++++------------------- 6 files changed, 30 insertions(+), 30 deletions(-) diff --git a/fs/lustre/include/obd_class.h b/fs/lustre/include/obd_class.h index 3f444b0..f603140 100644 --- a/fs/lustre/include/obd_class.h +++ b/fs/lustre/include/obd_class.h @@ -1688,7 +1688,8 @@ struct lwp_register_item { int lustre_check_exclusion(struct super_block *sb, char *svname); /* lustre_peer.c */ -int lustre_uuid_to_peer(const char *uuid, lnet_nid_t *peer_nid, int index); +int lustre_uuid_to_peer(const char *uuid, struct lnet_nid *peer_nid, + int index); int class_add_uuid(const char *uuid, u64 nid); int class_del_uuid(const char *uuid); int class_add_nids_to_uuid(struct obd_uuid *uuid, lnet_nid_t *nids, diff --git a/fs/lustre/obdclass/lustre_peer.c b/fs/lustre/obdclass/lustre_peer.c index f7e6a0f..5eae2eb 100644 --- a/fs/lustre/obdclass/lustre_peer.c +++ b/fs/lustre/obdclass/lustre_peer.c @@ -51,7 +51,7 @@ struct uuid_nid_data { static LIST_HEAD(g_uuid_list); static DEFINE_SPINLOCK(g_uuid_lock); -int lustre_uuid_to_peer(const char *uuid, lnet_nid_t *peer_nid, int index) +int lustre_uuid_to_peer(const char *uuid, struct lnet_nid *peer_nid, int index) { struct uuid_nid_data *data; struct obd_uuid tmp; @@ -65,7 +65,7 @@ int lustre_uuid_to_peer(const char *uuid, lnet_nid_t *peer_nid, int index) break; rc = 0; - *peer_nid = data->un_nids[index]; + lnet_nid4_to_nid(data->un_nids[index], peer_nid); break; } } diff --git a/fs/lustre/ptlrpc/events.c b/fs/lustre/ptlrpc/events.c index 385a6f2..140ea85 100644 --- a/fs/lustre/ptlrpc/events.c +++ b/fs/lustre/ptlrpc/events.c @@ -476,18 +476,18 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, int rc = -ENOENT; int dist; u32 order; - lnet_nid_t dst_nid; - lnet_nid_t src_nid; + struct lnet_nid dst_nid; + struct lnet_nid src_nid; peer->pid = LNET_PID_LUSTRE; /* Choose the matching UUID that's closest */ while (lustre_uuid_to_peer(uuid->uuid, &dst_nid, count++) == 0) { if (peer->nid != LNET_NID_ANY && LNET_NIDADDR(peer->nid) == 0 && - LNET_NIDNET(dst_nid) != LNET_NIDNET(peer->nid)) + LNET_NID_NET(&dst_nid) != LNET_NIDNET(peer->nid)) continue; - dist = LNetDist(dst_nid, &src_nid, &order); + dist = LNetDist(&dst_nid, &src_nid, &order); if (dist < 0) continue; @@ -503,8 +503,8 @@ int ptlrpc_uuid_to_peer(struct obd_uuid *uuid, best_dist = dist; best_order = order; - peer->nid = dst_nid; - *self = src_nid; + peer->nid = lnet_nid_to_nid4(&dst_nid); + *self = lnet_nid_to_nid4(&src_nid); rc = 0; } } diff --git a/include/linux/lnet/api.h b/include/linux/lnet/api.h index 447b41d..3657c13 100644 --- a/include/linux/lnet/api.h +++ b/include/linux/lnet/api.h @@ -76,7 +76,7 @@ * @{ */ int LNetGetId(unsigned int index, struct lnet_processid *id); -int LNetDist(lnet_nid_t nid, lnet_nid_t *srcnid, u32 *order); +int LNetDist(struct lnet_nid *nid, struct lnet_nid *srcnid, u32 *order); void LNetPrimaryNID(struct lnet_nid *nid); /** @} lnet_addr */ diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 44d5014..c977b47 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -4261,10 +4261,12 @@ u32 lnet_get_dlc_seq_locked(void) } case IOC_LIBCFS_LNET_DIST: - rc = LNetDist(data->ioc_nid, &data->ioc_nid, &data->ioc_u32[1]); + lnet_nid4_to_nid(data->ioc_nid, &nid); + rc = LNetDist(&nid, &nid, &data->ioc_u32[1]); if (rc < 0 && rc != -EHOSTUNREACH) return rc; + data->ioc_nid = lnet_nid_to_nid4(&nid); data->ioc_u32[0] = rc; return 0; diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 080bfe6..bca33bf 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -5072,55 +5072,51 @@ struct lnet_msg * * -EHOSTUNREACH If @dstnid is not reachable. */ int -LNetDist(lnet_nid_t dstnid, lnet_nid_t *srcnidp, u32 *orderp) +LNetDist(struct lnet_nid *dstnid, struct lnet_nid *srcnid, u32 *orderp) { struct lnet_ni *ni = NULL; struct lnet_remotenet *rnet; - u32 dstnet = LNET_NIDNET(dstnid); + u32 dstnet = LNET_NID_NET(dstnid); int hops; int cpt; u32 order = 2; struct list_head *rn_list; - bool matched_dstnet = false; + struct lnet_ni *matched_dstnet = NULL; - /* - * if !local_nid_dist_zero, I don't return a distance of 0 ever + /* if !local_nid_dist_zero, I don't return a distance of 0 ever * (when lustre sees a distance of 0, it substitutes 0@lo), so I * keep order 0 free for 0@lo and order 1 free for a local NID * match + * WARNING: dstnid and srcnid might point to same place. + * Don't set *srcnid until late. */ LASSERT(the_lnet.ln_refcount > 0); cpt = lnet_net_lock_current(); while ((ni = lnet_get_next_ni_locked(NULL, ni))) { - /* FIXME support large-addr nid */ - if (lnet_nid_to_nid4(&ni->ni_nid) == dstnid) { - if (srcnidp) - *srcnidp = dstnid; + if (nid_same(&ni->ni_nid, dstnid)) { if (orderp) { - if (dstnid == LNET_NID_LO_0) + if (nid_is_lo0(dstnid)) *orderp = 0; else *orderp = 1; } + if (srcnid) + *srcnid = *dstnid; lnet_net_unlock(cpt); return local_nid_dist_zero ? 0 : 1; } if (!matched_dstnet && LNET_NID_NET(&ni->ni_nid) == dstnet) { - matched_dstnet = true; + matched_dstnet = ni; /* We matched the destination net, but we may have * additional local NIs to inspect. * - * We record the nid and order as appropriate, but + * We record the order as appropriate, but * they may be overwritten if we match local NI above. */ - if (srcnidp) - /* FIXME support large-addr nids */ - *srcnidp = lnet_nid_to_nid4(&ni->ni_nid); - if (orderp) { /* Check if ni was originally created in * current net namespace. @@ -5140,6 +5136,8 @@ struct lnet_msg * } if (matched_dstnet) { + if (srcnid) + *srcnid = matched_dstnet->ni_nid; lnet_net_unlock(cpt); return 1; } @@ -5168,14 +5166,13 @@ struct lnet_msg * LASSERT(shortest); hops = shortest_hops; - if (srcnidp) { + if (srcnid) { struct lnet_net *net; net = lnet_get_net_locked(shortest->lr_lnet); LASSERT(net); ni = lnet_get_next_ni_locked(net, NULL); - /* FIXME support large-addr nids */ - *srcnidp = lnet_nid_to_nid4(&ni->ni_nid); + *srcnid = ni->ni_nid; } if (orderp) *orderp = order; From patchwork Thu Jun 9 12:33:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875524 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25811C433EF for ; Thu, 9 Jun 2022 12:33:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245465AbiFIMds (ORCPT ); Thu, 9 Jun 2022 08:33:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235883AbiFIMdi (ORCPT ); Thu, 9 Jun 2022 08:33:38 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 075F4186FB for ; Thu, 9 Jun 2022 05:33:36 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 46CD1EF6; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 3FF05D438A; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 08/18] lnet: convert LNetPut to take 16byte nid and pid. Date: Thu, 9 Jun 2022 08:33:04 -0400 Message-Id: <1654777994-29806-9-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown LNetPut() now takes a 16byte nid for self and similar process_id for target. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 50f6bb62987c54ea9 ("LU-10391 lnet: convert LNetPut to take 16byte nid and pid.") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/43619 Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/niobuf.c | 15 ++++++++++----- include/linux/lnet/api.h | 4 ++-- net/lnet/lnet/lib-move.c | 28 +++++++++++----------------- net/lnet/lnet/peer.c | 10 +++++----- net/lnet/selftest/rpc.c | 17 +++++++++++------ 5 files changed, 39 insertions(+), 35 deletions(-) diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index afe83ad..94a0329 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -46,15 +46,20 @@ */ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, enum lnet_ack_req ack, struct ptlrpc_cb_id *cbid, - lnet_nid_t self, struct lnet_process_id peer_id, + lnet_nid_t self4, struct lnet_process_id peer_id4, int portal, u64 xid, unsigned int offset, struct lnet_handle_md *bulk_cookie) { int rc; struct lnet_md md; + struct lnet_nid self; + struct lnet_processid peer_id; + + lnet_nid4_to_nid(self4, &self); + lnet_pid4_to_pid(peer_id4, &peer_id); LASSERT(portal != 0); - CDEBUG(D_INFO, "peer_id %s\n", libcfs_id2str(peer_id)); + CDEBUG(D_INFO, "peer_id %s\n", libcfs_id2str(peer_id4)); md.start = base; md.length = len; md.threshold = (ack == LNET_ACK_REQ) ? 2 : 1; @@ -85,8 +90,8 @@ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, percpu_ref_get(&ptlrpc_pending); - rc = LNetPut(self, *mdh, ack, - peer_id, portal, xid, offset, 0); + rc = LNetPut(&self, *mdh, ack, + &peer_id, portal, xid, offset, 0); if (unlikely(rc != 0)) { int rc2; /* We're going to get an UNLINK event when I unlink below, @@ -94,7 +99,7 @@ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, * I fall through and return success here! */ CERROR("LNetPut(%s, %d, %lld) failed: %d\n", - libcfs_id2str(peer_id), portal, xid, rc); + libcfs_id2str(peer_id4), portal, xid, rc); rc2 = LNetMDUnlink(*mdh); LASSERTF(rc2 == 0, "rc2 = %d\n", rc2); } diff --git a/include/linux/lnet/api.h b/include/linux/lnet/api.h index 3657c13..514cbe7 100644 --- a/include/linux/lnet/api.h +++ b/include/linux/lnet/api.h @@ -137,10 +137,10 @@ int LNetMDBind(const struct lnet_md *md_in, * and LNetGet(). * @{ */ -int LNetPut(lnet_nid_t self, +int LNetPut(struct lnet_nid *self, struct lnet_handle_md md_in, enum lnet_ack_req ack_req_in, - struct lnet_process_id target_in, + struct lnet_processid *target_in, unsigned int portal_in, u64 match_bits_in, unsigned int offset_in, diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index bca33bf..55a001e 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -4707,36 +4707,30 @@ void lnet_monitor_thr_stop(void) * \see lnet_event::hdr_data and lnet_event_kind. */ int -LNetPut(lnet_nid_t self4, struct lnet_handle_md mdh, enum lnet_ack_req ack, - struct lnet_process_id target4, unsigned int portal, +LNetPut(struct lnet_nid *self, struct lnet_handle_md mdh, enum lnet_ack_req ack, + struct lnet_processid *target, unsigned int portal, u64 match_bits, unsigned int offset, u64 hdr_data) { struct lnet_rsp_tracker *rspt = NULL; - struct lnet_processid target; struct lnet_msg *msg; struct lnet_libmd *md; - struct lnet_nid self; int cpt; int rc; LASSERT(the_lnet.ln_refcount > 0); - lnet_nid4_to_nid(self4, &self); - lnet_nid4_to_nid(target4.nid, &target.nid); - target.pid = target4.pid; - if (!list_empty(&the_lnet.ln_test_peers) && /* normally we don't */ - fail_peer(&target.nid, 1)) { /* shall we now? */ + fail_peer(&target->nid, 1)) { /* shall we now? */ CERROR("Dropping PUT to %s: simulated failure\n", - libcfs_id2str(target4)); + libcfs_idstr(target)); return -EIO; } msg = kmem_cache_zalloc(lnet_msg_cachep, GFP_NOFS); if (!msg) { CERROR("Dropping PUT to %s: ENOMEM on struct lnet_msg\n", - libcfs_id2str(target4)); + libcfs_idstr(target)); return -ENOMEM; } msg->msg_vmflush = !!(current->flags & PF_MEMALLOC); @@ -4747,7 +4741,7 @@ void lnet_monitor_thr_stop(void) rspt = lnet_rspt_alloc(cpt); if (!rspt) { CERROR("Dropping PUT to %s: ENOMEM on response tracker\n", - libcfs_id2str(target4)); + libcfs_idstr(target)); return -ENOMEM; } INIT_LIST_HEAD(&rspt->rspt_on_list); @@ -4758,7 +4752,7 @@ void lnet_monitor_thr_stop(void) md = lnet_handle2md(&mdh); if (!md || !md->md_threshold || md->md_me) { CERROR("Dropping PUT (%llu:%d:%s): MD (%d) invalid\n", - match_bits, portal, libcfs_id2str(target4), + match_bits, portal, libcfs_idstr(target), !md ? -1 : md->md_threshold); if (md && md->md_me) CERROR("Source MD also attached to portal %d\n", @@ -4772,11 +4766,11 @@ void lnet_monitor_thr_stop(void) return -ENOENT; } - CDEBUG(D_NET, "%s -> %s\n", __func__, libcfs_id2str(target4)); + CDEBUG(D_NET, "%s -> %s\n", __func__, libcfs_idstr(target)); lnet_msg_attach_md(msg, md, 0, 0); - lnet_prep_send(msg, LNET_MSG_PUT, &target, 0, md->md_length); + lnet_prep_send(msg, LNET_MSG_PUT, target, 0, md->md_length); msg->msg_hdr.msg.put.match_bits = cpu_to_le64(match_bits); msg->msg_hdr.msg.put.ptl_index = cpu_to_le32(portal); @@ -4810,10 +4804,10 @@ void lnet_monitor_thr_stop(void) CFS_FAIL_ONCE)) rc = -EIO; else - rc = lnet_send(&self, msg, NULL); + rc = lnet_send(self, msg, NULL); if (rc) { CNETERR("Error sending PUT to %s: %d\n", - libcfs_id2str(target4), rc); + libcfs_idstr(target), rc); msg->msg_no_resend = true; lnet_finalize(msg, rc); } diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 2055f31..6c35901 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -3559,7 +3559,7 @@ static int lnet_peer_send_push(struct lnet_peer *lp) __must_hold(&lp->lp_lock) { struct lnet_ping_buffer *pbuf; - struct lnet_process_id id; + struct lnet_processid id; struct lnet_md md; int cpt; int rc; @@ -3606,13 +3606,13 @@ static int lnet_peer_send_push(struct lnet_peer *lp) lnet_peer_addref_locked(lp); id.pid = LNET_PID_LUSTRE; if (!LNET_NID_IS_ANY(&lp->lp_disc_dst_nid)) - id.nid = lnet_nid_to_nid4(&lp->lp_disc_dst_nid); + id.nid = lp->lp_disc_dst_nid; else - id.nid = lnet_nid_to_nid4(&lp->lp_primary_nid); + id.nid = lp->lp_primary_nid; lnet_net_unlock(cpt); - rc = LNetPut(lnet_nid_to_nid4(&lp->lp_disc_src_nid), lp->lp_push_mdh, - LNET_ACK_REQ, id, LNET_RESERVED_PORTAL, + rc = LNetPut(&lp->lp_disc_src_nid, lp->lp_push_mdh, + LNET_ACK_REQ, &id, LNET_RESERVED_PORTAL, LNET_PROTO_PING_MATCHBITS, 0, 0); /* reset the discovery nid. There is no need to restrict sending * from that source, if we call lnet_push_update_to_peers(). It'll diff --git a/net/lnet/selftest/rpc.c b/net/lnet/selftest/rpc.c index d1538be..b16711a 100644 --- a/net/lnet/selftest/rpc.c +++ b/net/lnet/selftest/rpc.c @@ -397,12 +397,17 @@ struct srpc_bulk * static int srpc_post_active_rdma(int portal, u64 matchbits, void *buf, int len, - int options, struct lnet_process_id peer, - lnet_nid_t self, struct lnet_handle_md *mdh, + int options, struct lnet_process_id peer4, + lnet_nid_t self4, struct lnet_handle_md *mdh, struct srpc_event *ev) { int rc; struct lnet_md md; + struct lnet_nid self; + struct lnet_processid peer; + + lnet_nid4_to_nid(self4, &self); + lnet_pid4_to_pid(peer4, &peer); md.user_ptr = ev; md.start = buf; @@ -424,18 +429,18 @@ struct srpc_bulk * * buffers... */ if (options & LNET_MD_OP_PUT) { - rc = LNetPut(self, *mdh, LNET_NOACK_REQ, peer, + rc = LNetPut(&self, *mdh, LNET_NOACK_REQ, &peer, portal, matchbits, 0, 0); } else { LASSERT(options & LNET_MD_OP_GET); - rc = LNetGet(self, *mdh, peer, portal, matchbits, 0, false); + rc = LNetGet(self4, *mdh, peer4, portal, matchbits, 0, false); } if (rc) { CERROR("LNet%s(%s, %d, %lld) failed: %d\n", options & LNET_MD_OP_PUT ? "Put" : "Get", - libcfs_id2str(peer), portal, matchbits, rc); + libcfs_id2str(peer4), portal, matchbits, rc); /* * The forthcoming unlink event will complete this operation @@ -446,7 +451,7 @@ struct srpc_bulk * } else { CDEBUG(D_NET, "Posted active RDMA: peer %s, portal %u, matchbits %#llx\n", - libcfs_id2str(peer), portal, matchbits); + libcfs_id2str(peer4), portal, matchbits); } return 0; } From patchwork Thu Jun 9 12:33:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E28BBCCA47B for ; Thu, 9 Jun 2022 12:33:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245448AbiFIMdu (ORCPT ); Thu, 9 Jun 2022 08:33:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245424AbiFIMdk (ORCPT ); Thu, 9 Jun 2022 08:33:40 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4106321E38 for ; Thu, 9 Jun 2022 05:33:38 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 49E9AEF8; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 43316D4404; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 09/18] lnet: change LNetGet to take 16byte nid and pid. Date: Thu, 9 Jun 2022 08:33:05 -0400 Message-Id: <1654777994-29806-10-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown "self" is now passed to LNetGet as a pointer to a 16-byte-addr nid, or NULL for "ANY". "target" is passed as a 16-bytes-addr process_id. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: d727ec56b26cbd1b8 ("LU-10391 lnet: change LNetGet to take 16byte nid and pid.") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/43620 Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/api.h | 4 ++-- net/lnet/lnet/api-ni.c | 25 +++++++++++++------------ net/lnet/lnet/lib-move.c | 34 ++++++++++++++-------------------- net/lnet/selftest/rpc.c | 2 +- 4 files changed, 30 insertions(+), 35 deletions(-) diff --git a/include/linux/lnet/api.h b/include/linux/lnet/api.h index 514cbe7..7ea61cb 100644 --- a/include/linux/lnet/api.h +++ b/include/linux/lnet/api.h @@ -146,9 +146,9 @@ int LNetPut(struct lnet_nid *self, unsigned int offset_in, u64 hdr_data_in); -int LNetGet(lnet_nid_t self, +int LNetGet(struct lnet_nid *self, struct lnet_handle_md md_in, - struct lnet_process_id target_in, + struct lnet_processid *target_in, unsigned int portal_in, u64 match_bits_in, unsigned int offset_in, diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index c977b47..8643ac8d 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -205,7 +205,7 @@ static void lnet_set_lnd_timeout(void) */ static atomic_t lnet_dlc_seq_no = ATOMIC_INIT(0); -static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, +static int lnet_ping(struct lnet_process_id id4, struct lnet_nid *src_nid, signed long timeout, struct lnet_process_id __user *ids, int n_ids); @@ -4562,7 +4562,7 @@ struct ping_data { complete(&pd->completion); } -static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, +static int lnet_ping(struct lnet_process_id id4, struct lnet_nid *src_nid, signed long timeout, struct lnet_process_id __user *ids, int n_ids) { @@ -4570,13 +4570,14 @@ static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, struct ping_data pd = { 0 }; struct lnet_ping_buffer *pbuf; struct lnet_process_id tmpid; + struct lnet_processid id; int i; int nob; int rc; int rc2; /* n_ids limit is arbitrary */ - if (n_ids <= 0 || id.nid == LNET_NID_ANY) + if (n_ids <= 0 || id4.nid == LNET_NID_ANY) return -EINVAL; /* if the user buffer has more space than the lnet_interfaces_max @@ -4585,8 +4586,8 @@ static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, if (n_ids > lnet_interfaces_max) n_ids = lnet_interfaces_max; - if (id.pid == LNET_PID_ANY) - id.pid = LNET_PID_LUSTRE; + if (id4.pid == LNET_PID_ANY) + id4.pid = LNET_PID_LUSTRE; pbuf = lnet_ping_buffer_alloc(n_ids, GFP_NOFS); if (!pbuf) @@ -4609,8 +4610,8 @@ static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, goto fail_ping_buffer_decref; } - rc = LNetGet(lnet_nid_to_nid4(src_nid), pd.mdh, id, - LNET_RESERVED_PORTAL, + lnet_pid4_to_pid(id4, &id); + rc = LNetGet(src_nid, pd.mdh, &id, LNET_RESERVED_PORTAL, LNET_PROTO_PING_MATCHBITS, 0, false); if (rc) { /* Don't CERROR; this could be deliberate! */ @@ -4637,7 +4638,7 @@ static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, if (nob < 8) { CERROR("%s: ping info too short %d\n", - libcfs_id2str(id), nob); + libcfs_id2str(id4), nob); goto fail_ping_buffer_decref; } @@ -4645,19 +4646,19 @@ static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, lnet_swap_pinginfo(pbuf); } else if (pbuf->pb_info.pi_magic != LNET_PROTO_PING_MAGIC) { CERROR("%s: Unexpected magic %08x\n", - libcfs_id2str(id), pbuf->pb_info.pi_magic); + libcfs_id2str(id4), pbuf->pb_info.pi_magic); goto fail_ping_buffer_decref; } if (!(pbuf->pb_info.pi_features & LNET_PING_FEAT_NI_STATUS)) { CERROR("%s: ping w/o NI status: 0x%x\n", - libcfs_id2str(id), pbuf->pb_info.pi_features); + libcfs_id2str(id4), pbuf->pb_info.pi_features); goto fail_ping_buffer_decref; } if (nob < LNET_PING_INFO_SIZE(0)) { CERROR("%s: Short reply %d(%d min)\n", - libcfs_id2str(id), + libcfs_id2str(id4), nob, (int)LNET_PING_INFO_SIZE(0)); goto fail_ping_buffer_decref; } @@ -4667,7 +4668,7 @@ static int lnet_ping(struct lnet_process_id id, struct lnet_nid *src_nid, if (nob < LNET_PING_INFO_SIZE(n_ids)) { CERROR("%s: Short reply %d(%d expected)\n", - libcfs_id2str(id), + libcfs_id2str(id4), nob, (int)LNET_PING_INFO_SIZE(n_ids)); goto fail_ping_buffer_decref; } diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 55a001e..9ee1075 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -3630,7 +3630,7 @@ struct lnet_mt_event_info { void *user_data, lnet_handler_t handler, bool recovery) { struct lnet_md md = { NULL }; - struct lnet_process_id id; + struct lnet_processid id; struct lnet_ping_buffer *pbuf; int rc; @@ -3662,9 +3662,9 @@ struct lnet_mt_event_info { goto fail_error; } id.pid = LNET_PID_LUSTRE; - id.nid = lnet_nid_to_nid4(dest_nid); + id.nid = *dest_nid; - rc = LNetGet(LNET_NID_ANY, *mdh, id, + rc = LNetGet(NULL, *mdh, &id, LNET_RESERVED_PORTAL, LNET_PROTO_PING_MATCHBITS, 0, recovery); if (rc) @@ -4948,35 +4948,29 @@ struct lnet_msg * * -ENOENT Invalid MD object. */ int -LNetGet(lnet_nid_t self4, struct lnet_handle_md mdh, - struct lnet_process_id target4, unsigned int portal, +LNetGet(struct lnet_nid *self, struct lnet_handle_md mdh, + struct lnet_processid *target, unsigned int portal, u64 match_bits, unsigned int offset, bool recovery) { struct lnet_rsp_tracker *rspt; - struct lnet_processid target; struct lnet_msg *msg; struct lnet_libmd *md; - struct lnet_nid self; int cpt; int rc; LASSERT(the_lnet.ln_refcount > 0); - lnet_nid4_to_nid(self4, &self); - lnet_nid4_to_nid(target4.nid, &target.nid); - target.pid = target4.pid; - if (!list_empty(&the_lnet.ln_test_peers) && /* normally we don't */ - fail_peer(&target.nid, 1)) { /* shall we now? */ + fail_peer(&target->nid, 1)) { /* shall we now? */ CERROR("Dropping GET to %s: simulated failure\n", - libcfs_id2str(target4)); + libcfs_idstr(target)); return -EIO; } msg = kmem_cache_zalloc(lnet_msg_cachep, GFP_NOFS); if (!msg) { CERROR("Dropping GET to %s: ENOMEM on struct lnet_msg\n", - libcfs_id2str(target4)); + libcfs_idstr(target)); return -ENOMEM; } @@ -4985,7 +4979,7 @@ struct lnet_msg * rspt = lnet_rspt_alloc(cpt); if (!rspt) { CERROR("Dropping GET to %s: ENOMEM on response tracker\n", - libcfs_id2str(target4)); + libcfs_idstr(target)); return -ENOMEM; } INIT_LIST_HEAD(&rspt->rspt_on_list); @@ -4997,7 +4991,7 @@ struct lnet_msg * md = lnet_handle2md(&mdh); if (!md || !md->md_threshold || md->md_me) { CERROR("Dropping GET (%llu:%d:%s): MD (%d) invalid\n", - match_bits, portal, libcfs_id2str(target4), + match_bits, portal, libcfs_idstr(target), !md ? -1 : md->md_threshold); if (md && md->md_me) CERROR("REPLY MD also attached to portal %d\n", @@ -5010,11 +5004,11 @@ struct lnet_msg * return -ENOENT; } - CDEBUG(D_NET, "%s -> %s\n", __func__, libcfs_id2str(target4)); + CDEBUG(D_NET, "%s -> %s\n", __func__, libcfs_idstr(target)); lnet_msg_attach_md(msg, md, 0, 0); - lnet_prep_send(msg, LNET_MSG_GET, &target, 0, 0); + lnet_prep_send(msg, LNET_MSG_GET, target, 0, 0); msg->msg_hdr.msg.get.match_bits = cpu_to_le64(match_bits); msg->msg_hdr.msg.get.ptl_index = cpu_to_le32(portal); @@ -5036,10 +5030,10 @@ struct lnet_msg * else lnet_rspt_free(rspt, cpt); - rc = lnet_send(&self, msg, NULL); + rc = lnet_send(self, msg, NULL); if (rc < 0) { CNETERR("Error sending GET to %s: %d\n", - libcfs_id2str(target4), rc); + libcfs_idstr(target), rc); msg->msg_no_resend = true; lnet_finalize(msg, rc); } diff --git a/net/lnet/selftest/rpc.c b/net/lnet/selftest/rpc.c index b16711a..17277b8 100644 --- a/net/lnet/selftest/rpc.c +++ b/net/lnet/selftest/rpc.c @@ -434,7 +434,7 @@ struct srpc_bulk * } else { LASSERT(options & LNET_MD_OP_GET); - rc = LNetGet(self4, *mdh, peer4, portal, matchbits, 0, false); + rc = LNetGet(&self, *mdh, &peer, portal, matchbits, 0, false); } if (rc) { From patchwork Thu Jun 9 12:33:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97DF1C43334 for ; Thu, 9 Jun 2022 12:33:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245510AbiFIMdu (ORCPT ); Thu, 9 Jun 2022 08:33:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245448AbiFIMds (ORCPT ); Thu, 9 Jun 2022 08:33:48 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C993F220F6 for ; Thu, 9 Jun 2022 05:33:41 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 4AC73EF9; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 46891D439B; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 10/18] lnet: socklnd: pass large processid to ksocknal_add_peer Date: Thu, 9 Jun 2022 08:33:06 -0400 Message-Id: <1654777994-29806-11-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown Teach ksocknal_add_peer() to handle large-address processid, and now ksocknal_launch_packet() can support IPv6 addresses as well as IPv4. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 6deddc3d46704643d ("LU-10391 socklnd: pass large processid to ksocknal_add_peer") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44621 Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/socklnd/socklnd.c | 22 +++++++++------------- net/lnet/klnds/socklnd/socklnd.h | 2 +- net/lnet/klnds/socklnd/socklnd_cb.c | 28 ++++++++++++++++++---------- 3 files changed, 28 insertions(+), 24 deletions(-) diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index 4267832..2b6fa18 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -611,23 +611,19 @@ struct ksock_peer_ni * } int -ksocknal_add_peer(struct lnet_ni *ni, struct lnet_process_id id4, +ksocknal_add_peer(struct lnet_ni *ni, struct lnet_processid *id, struct sockaddr *addr) { struct ksock_peer_ni *peer_ni; struct ksock_peer_ni *peer2; struct ksock_conn_cb *conn_cb; - struct lnet_processid id; - if (id4.nid == LNET_NID_ANY || - id4.pid == LNET_PID_ANY) + if (LNET_NID_IS_ANY(&id->nid) || + id->pid == LNET_PID_ANY) return -EINVAL; - id.pid = id4.pid; - lnet_nid4_to_nid(id4.nid, &id.nid); - /* Have a brand new peer_ni ready... */ - peer_ni = ksocknal_create_peer(ni, &id); + peer_ni = ksocknal_create_peer(ni, id); if (IS_ERR(peer_ni)) return PTR_ERR(peer_ni); @@ -642,14 +638,14 @@ struct ksock_peer_ni * /* always called with a ref on ni, so shutdown can't have started */ LASSERT(atomic_read(&((struct ksock_net *)ni->ni_data)->ksnn_npeers) >= 0); - peer2 = ksocknal_find_peer_locked(ni, &id); + peer2 = ksocknal_find_peer_locked(ni, id); if (peer2) { ksocknal_peer_decref(peer_ni); peer_ni = peer2; } else { /* peer_ni table takes my ref on peer_ni */ hash_add(ksocknal_data.ksnd_peers, &peer_ni->ksnp_list, - nidhash(&id.nid)); + nidhash(&id->nid)); } ksocknal_add_conn_cb_locked(peer_ni, conn_cb); @@ -1830,11 +1826,11 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_processid *id) case IOC_LIBCFS_ADD_PEER: { struct sockaddr_in sa = {.sin_family = AF_INET}; - id4.nid = data->ioc_nid; - id4.pid = LNET_PID_LUSTRE; + id.pid = LNET_PID_LUSTRE; + lnet_nid4_to_nid(data->ioc_nid, &id.nid); sa.sin_addr.s_addr = htonl(data->ioc_u32[0]); sa.sin_port = htons(data->ioc_u32[1]); - return ksocknal_add_peer(ni, id4, (struct sockaddr *)&sa); + return ksocknal_add_peer(ni, &id, (struct sockaddr *)&sa); } case IOC_LIBCFS_DEL_PEER: id4.nid = data->ioc_nid; diff --git a/net/lnet/klnds/socklnd/socklnd.h b/net/lnet/klnds/socklnd/socklnd.h index 13abe20..93368bd 100644 --- a/net/lnet/klnds/socklnd/socklnd.h +++ b/net/lnet/klnds/socklnd/socklnd.h @@ -627,7 +627,7 @@ int ksocknal_recv(struct lnet_ni *ni, void *private, struct lnet_msg *lntmsg, int delayed, struct iov_iter *to, unsigned int rlen); int ksocknal_accept(struct lnet_ni *ni, struct socket *sock); -int ksocknal_add_peer(struct lnet_ni *ni, struct lnet_process_id id, +int ksocknal_add_peer(struct lnet_ni *ni, struct lnet_processid *id, struct sockaddr *addr); struct ksock_peer_ni *ksocknal_find_peer_locked(struct lnet_ni *ni, struct lnet_processid *id); diff --git a/net/lnet/klnds/socklnd/socklnd_cb.c b/net/lnet/klnds/socklnd/socklnd_cb.c index adec183..94600f3 100644 --- a/net/lnet/klnds/socklnd/socklnd_cb.c +++ b/net/lnet/klnds/socklnd/socklnd_cb.c @@ -808,7 +808,7 @@ struct ksock_conn_cb * { struct ksock_peer_ni *peer_ni; struct ksock_conn *conn; - struct sockaddr_in sa; + struct sockaddr_storage sa; rwlock_t *g_lock; int retry; int rc; @@ -859,16 +859,24 @@ struct ksock_conn_cb * } memset(&sa, 0, sizeof(sa)); - sa.sin_family = AF_INET; - sa.sin_addr.s_addr = id->nid.nid_addr[0]; - sa.sin_port = htons(lnet_acceptor_port()); - { - struct lnet_process_id id4 = { - .pid = id->pid, - .nid = lnet_nid_to_nid4(&id->nid), - }; - rc = ksocknal_add_peer(ni, id4, (struct sockaddr *)&sa); + switch (NID_ADDR_BYTES(&id->nid)) { + struct sockaddr_in *sin; + struct sockaddr_in6 *sin6; + case 4: + sin = (void *)&sa; + sin->sin_family = AF_INET; + sin->sin_addr.s_addr = id->nid.nid_addr[0]; + sin->sin_port = htons(lnet_acceptor_port()); + break; + case 16: + sin6 = (void *)&sa; + sin6->sin6_family = AF_INET6; + memcpy(&sin6->sin6_addr, id->nid.nid_addr, + sizeof(sin6->sin6_addr)); + sin6->sin6_port = htons(lnet_acceptor_port()); + break; } + rc = ksocknal_add_peer(ni, id, (struct sockaddr *)&sa); if (rc) { CERROR("Can't add peer_ni %s: %d\n", libcfs_idstr(id), rc); From patchwork Thu Jun 9 12:33:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 590D1C433EF for ; Thu, 9 Jun 2022 12:34:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245532AbiFIMeB (ORCPT ); Thu, 9 Jun 2022 08:34:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245480AbiFIMds (ORCPT ); Thu, 9 Jun 2022 08:33:48 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E9AC22520 for ; Thu, 9 Jun 2022 05:33:44 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 4B4B6EFA; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 498CCD43A0; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 11/18] lnet: socklnd: large processid for ksocknal_get_peer_info Date: Thu, 9 Jun 2022 08:33:07 -0400 Message-Id: <1654777994-29806-12-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown Have ksocknal_launch_packet() report a 'struct lnet_processid' with a large address. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: e4a49294530a5d5f7 ("LU-10391 socklnd: large processid for ksocknal_get_peer_info") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44622 Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/socklnd/socklnd.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index 2b6fa18..857aa05 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -292,7 +292,7 @@ struct ksock_peer_ni * static int ksocknal_get_peer_info(struct lnet_ni *ni, int index, - struct lnet_process_id *id, u32 *myip, u32 *peer_ip, + struct lnet_processid *id, u32 *myip, u32 *peer_ip, int *port, int *conn_count, int *share_count) { struct ksock_peer_ni *peer_ni; @@ -312,8 +312,7 @@ struct ksock_peer_ni * if (index-- > 0) continue; - id->pid = peer_ni->ksnp_id.pid; - id->nid = lnet_nid_to_nid4(&peer_ni->ksnp_id.nid); + *id = peer_ni->ksnp_id; *myip = 0; *peer_ip = 0; *port = 0; @@ -327,8 +326,7 @@ struct ksock_peer_ni * if (index-- > 0) continue; - id->pid = peer_ni->ksnp_id.pid; - id->nid = lnet_nid_to_nid4(&peer_ni->ksnp_id.nid); + *id = peer_ni->ksnp_id; *myip = peer_ni->ksnp_passive_ips[j]; *peer_ip = 0; *port = 0; @@ -344,8 +342,7 @@ struct ksock_peer_ni * conn_cb = peer_ni->ksnp_conn_cb; - id->pid = peer_ni->ksnp_id.pid; - id->nid = lnet_nid_to_nid4(&peer_ni->ksnp_id.nid); + *id = peer_ni->ksnp_id; if (conn_cb->ksnr_addr.ss_family == AF_INET) { struct sockaddr_in *sa; @@ -1808,18 +1805,20 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_processid *id) int share_count = 0; rc = ksocknal_get_peer_info(ni, data->ioc_count, - &id4, &myip, &ip, &port, + &id, &myip, &ip, &port, &conn_count, &share_count); if (rc) return rc; - data->ioc_nid = id4.nid; + if (!nid_is_nid4(&id.nid)) + return -EINVAL; + data->ioc_nid = lnet_nid_to_nid4(&id.nid); data->ioc_count = share_count; data->ioc_u32[0] = ip; data->ioc_u32[1] = port; data->ioc_u32[2] = myip; data->ioc_u32[3] = conn_count; - data->ioc_u32[4] = id4.pid; + data->ioc_u32[4] = id.pid; return 0; } From patchwork Thu Jun 9 12:33:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4353C43334 for ; Thu, 9 Jun 2022 12:34:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235883AbiFIMeC (ORCPT ); Thu, 9 Jun 2022 08:34:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245500AbiFIMdu (ORCPT ); Thu, 9 Jun 2022 08:33:50 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE2E922B28 for ; Thu, 9 Jun 2022 05:33:45 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 4F0DCEFB; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4C91ED43A3; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, James Simmons Subject: [PATCH 12/18] lnet: socklnd: switch ksocknal_del_peer to lnet_processid Date: Thu, 9 Jun 2022 08:33:08 -0400 Message-Id: <1654777994-29806-13-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Mr NeilBrown ksocknal_del_peer now takes a pointer to a lnet_processid, with room for a large address. A NULL means "ANY NID, AND PID". The "ip" argument was completely unused, so has been removed. This was the last use of 'struct lnet_process_id' in ksocklnd. WC-bug-id: https://jira.whamcloud.com/browse/LU-10391 Lustre-commit: 782e37e54f5c54886 ("LU-10391 socklnd: switch ksocknal_del_peer to lnet_processid") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/44623 Reviewed-by: James Simmons Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/socklnd/socklnd.c | 36 +++++++++++++----------------------- 1 file changed, 13 insertions(+), 23 deletions(-) diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index 857aa05..01b434f 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -660,7 +660,7 @@ struct ksock_peer_ni * } static void -ksocknal_del_peer_locked(struct ksock_peer_ni *peer_ni, u32 ip) +ksocknal_del_peer_locked(struct ksock_peer_ni *peer_ni) { struct ksock_conn *conn; struct ksock_conn *cnxt; @@ -683,7 +683,7 @@ struct ksock_peer_ni * } static int -ksocknal_del_peer(struct lnet_ni *ni, struct lnet_process_id id4, u32 ip) +ksocknal_del_peer(struct lnet_ni *ni, struct lnet_processid *id) { LIST_HEAD(zombies); struct hlist_node *pnxt; @@ -692,15 +692,11 @@ struct ksock_peer_ni * int hi; int i; int rc = -ENOENT; - struct lnet_processid id; - - id.pid = id4.pid; - lnet_nid4_to_nid(id4.nid, &id.nid); write_lock_bh(&ksocknal_data.ksnd_global_lock); - if (!LNET_NID_IS_ANY(&id.nid)) { - lo = hash_min(nidhash(&id.nid), + if (id && !LNET_NID_IS_ANY(&id->nid)) { + lo = hash_min(nidhash(&id->nid), HASH_BITS(ksocknal_data.ksnd_peers)); hi = lo; } else { @@ -715,15 +711,15 @@ struct ksock_peer_ni * if (peer_ni->ksnp_ni != ni) continue; - if (!((LNET_NID_IS_ANY(&id.nid) || - nid_same(&peer_ni->ksnp_id.nid, &id.nid)) && - (id.pid == LNET_PID_ANY || - peer_ni->ksnp_id.pid == id.pid))) + if (!((!id || LNET_NID_IS_ANY(&id->nid) || + nid_same(&peer_ni->ksnp_id.nid, &id->nid)) && + (!id || id->pid == LNET_PID_ANY || + peer_ni->ksnp_id.pid == id->pid))) continue; ksocknal_peer_addref(peer_ni); /* a ref for me... */ - ksocknal_del_peer_locked(peer_ni, ip); + ksocknal_del_peer_locked(peer_ni); if (peer_ni->ksnp_closing && !list_empty(&peer_ni->ksnp_tx_queue)) { @@ -1764,7 +1760,6 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_processid *id) int ksocknal_ctl(struct lnet_ni *ni, unsigned int cmd, void *arg) { - struct lnet_process_id id4 = {}; struct lnet_processid id = {}; struct libcfs_ioctl_data *data = arg; int rc; @@ -1832,10 +1827,9 @@ static int ksocknal_push(struct lnet_ni *ni, struct lnet_processid *id) return ksocknal_add_peer(ni, &id, (struct sockaddr *)&sa); } case IOC_LIBCFS_DEL_PEER: - id4.nid = data->ioc_nid; - id4.pid = LNET_PID_ANY; - return ksocknal_del_peer(ni, id4, - data->ioc_u32[0]); /* IP */ + lnet_nid4_to_nid(data->ioc_nid, &id.nid); + id.pid = LNET_PID_ANY; + return ksocknal_del_peer(ni, &id); case IOC_LIBCFS_GET_CONN: { int txmem; @@ -2347,10 +2341,6 @@ static int ksocknal_inetaddr_event(struct notifier_block *unused, ksocknal_shutdown(struct lnet_ni *ni) { struct ksock_net *net = ni->ni_data; - struct lnet_process_id anyid = { 0 }; - - anyid.nid = LNET_NID_ANY; - anyid.pid = LNET_PID_ANY; LASSERT(ksocknal_data.ksnd_init == SOCKNAL_INIT_ALL); LASSERT(ksocknal_data.ksnd_nnets > 0); @@ -2359,7 +2349,7 @@ static int ksocknal_inetaddr_event(struct notifier_block *unused, atomic_add(SOCKNAL_SHUTDOWN_BIAS, &net->ksnn_npeers); /* Delete all peers */ - ksocknal_del_peer(ni, anyid, 0); + ksocknal_del_peer(ni, NULL); /* Wait for all peer_ni state to clean up */ wait_var_event_warning(&net->ksnn_npeers, From patchwork Thu Jun 9 12:33:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75846CCA473 for ; Thu, 9 Jun 2022 12:34:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245475AbiFIMeE (ORCPT ); Thu, 9 Jun 2022 08:34:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245363AbiFIMeA (ORCPT ); Thu, 9 Jun 2022 08:34:00 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6AE622BC2 for ; Thu, 9 Jun 2022 05:33:47 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 51DBAEFC; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 4FA1BD43AC; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Lai Siyao , James Simmons Subject: [PATCH 13/18] lustre: llite: access lli_lsm_md with lock in all places Date: Thu, 9 Jun 2022 08:33:09 -0400 Message-Id: <1654777994-29806-14-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Lai Siyao lli_lsm_md should be accessed with lock in all places. Among all the changes, ll_rease_page() is inside lock already, except statahead code. WC-bug-id: https://jira.whamcloud.com/browse/LU-15284 Lustre-commit: 1dfae156d1dbc11cf ("LU-15284 llite: access lli_lsm_md with lock in all places") Signed-off-by: Lai Siyao Reviewed-on: https://review.whamcloud.com/46355 Reviewed-by: Mike Pershin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/dir.c | 6 +++--- fs/lustre/llite/file.c | 8 +++++--- fs/lustre/llite/llite_internal.h | 17 +++++++++++++++-- fs/lustre/llite/llite_lib.c | 20 +++++++++++--------- fs/lustre/llite/namei.c | 7 +++++-- fs/lustre/llite/statahead.c | 2 ++ fs/lustre/lmv/lmv_obd.c | 3 ++- 7 files changed, 43 insertions(+), 20 deletions(-) diff --git a/fs/lustre/llite/dir.c b/fs/lustre/llite/dir.c index cfd8184..29d7e44 100644 --- a/fs/lustre/llite/dir.c +++ b/fs/lustre/llite/dir.c @@ -162,11 +162,11 @@ void ll_release_page(struct inode *inode, struct page *page, bool remove) { kunmap(page); - /* - * Always remove the page for striped dir, because the page is + /* Always remove the page for striped dir, because the page is * built from temporarily in LMV layer */ - if (inode && ll_dir_striped(inode)) { + if (inode && S_ISDIR(inode->i_mode) && + lmv_dir_striped(ll_i2info(inode)->lli_lsm_md)) { __free_page(page); return; } diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 1ac3e4f..30f522b 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -5066,12 +5066,14 @@ static int ll_merge_md_attr(struct inode *inode) struct cl_attr attr = { 0 }; int rc; - LASSERT(lli->lli_lsm_md); - - if (!lmv_dir_striped(lli->lli_lsm_md)) + if (!lli->lli_lsm_md) return 0; down_read(&lli->lli_lsm_sem); + if (!lmv_dir_striped(lli->lli_lsm_md)) { + up_read(&lli->lli_lsm_sem); + return 0; + } rc = md_merge_attr(ll_i2mdexp(inode), lli->lli_lsm_md, &attr, ll_md_blocking_ast); up_read(&lli->lli_lsm_sem); diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index f51ab19..bc50169 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -1373,9 +1373,22 @@ static inline struct lu_fid *ll_inode2fid(struct inode *inode) static inline bool ll_dir_striped(struct inode *inode) { + struct ll_inode_info *lli; + bool rc; + LASSERT(inode); - return S_ISDIR(inode->i_mode) && - lmv_dir_striped(ll_i2info(inode)->lli_lsm_md); + if (!S_ISDIR(inode->i_mode)) + return false; + + lli = ll_i2info(inode); + if (!lli->lli_lsm_md) + return false; + + down_read(&lli->lli_lsm_sem); + rc = lmv_dir_striped(lli->lli_lsm_md); + up_read(&lli->lli_lsm_sem); + + return rc; } static inline loff_t ll_file_maxbytes(struct inode *inode) diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index ad77ef0..99ab9ac 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -1626,23 +1626,25 @@ static int ll_update_lsm_md(struct inode *inode, struct lustre_md *md) } rc = ll_init_lsm_md(inode, md); - up_write(&lli->lli_lsm_sem); - - if (rc) + if (rc) { + up_write(&lli->lli_lsm_sem); return rc; + } + + /* md_merge_attr() may take long, since lsm is already set, switch to + * read lock. + */ + downgrade_write(&lli->lli_lsm_sem); /* set md->lmv to NULL, so the following free lustre_md will not free * this lsm. */ md->lmv = NULL; - /* md_merge_attr() may take long, since lsm is already set, switch to - * read lock. - */ - down_read(&lli->lli_lsm_sem); - - if (!lmv_dir_striped(lli->lli_lsm_md)) + if (!lmv_dir_striped(lli->lli_lsm_md)) { + rc = 0; goto unlock; + } attr = kzalloc(sizeof(*attr), GFP_NOFS); if (!attr) { diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index c05e3bf..f7e900d 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -765,14 +765,17 @@ static int ll_lookup_it_finish(struct ptlrpc_request *request, .it_op = IT_GETATTR, .it_lock_handle = 0 }; - struct lu_fid fid = ll_i2info(parent)->lli_fid; + struct ll_inode_info *lli = ll_i2info(parent); + struct lu_fid fid = lli->lli_fid; /* If it is striped directory, get the real stripe parent */ if (unlikely(ll_dir_striped(parent))) { + down_read(&lli->lli_lsm_sem); rc = md_get_fid_from_lsm(ll_i2mdexp(parent), - ll_i2info(parent)->lli_lsm_md, + lli->lli_lsm_md, (*de)->d_name.name, (*de)->d_name.len, &fid); + up_read(&lli->lli_lsm_sem); if (rc) return rc; } diff --git a/fs/lustre/llite/statahead.c b/fs/lustre/llite/statahead.c index c781e49..3043a51 100644 --- a/fs/lustre/llite/statahead.c +++ b/fs/lustre/llite/statahead.c @@ -1164,8 +1164,10 @@ static int ll_statahead_thread(void *arg) } pos = le64_to_cpu(dp->ldp_hash_end); + down_read(&lli->lli_lsm_sem); ll_release_page(dir, page, le32_to_cpu(dp->ldp_flags) & LDF_COLLIDE); + up_read(&lli->lli_lsm_sem); if (sa_low_hit(sai)) { rc = -EFAULT; diff --git a/fs/lustre/lmv/lmv_obd.c b/fs/lustre/lmv/lmv_obd.c index 5b43cfd..d83ba41ff 100644 --- a/fs/lustre/lmv/lmv_obd.c +++ b/fs/lustre/lmv/lmv_obd.c @@ -3654,7 +3654,8 @@ static int lmv_revalidate_lock(struct obd_export *exp, struct lookup_intent *it, { const struct lmv_oinfo *oinfo; - LASSERT(lmv_dir_striped(lsm)); + if (!lmv_dir_striped(lsm)) + return -ESTALE; oinfo = lsm_name_to_stripe_info(lsm, name, namelen, false); if (IS_ERR(oinfo)) From patchwork Thu Jun 9 12:33:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02BCACCA47B for ; Thu, 9 Jun 2022 12:34:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245507AbiFIMeD (ORCPT ); Thu, 9 Jun 2022 08:34:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245520AbiFIMeA (ORCPT ); Thu, 9 Jun 2022 08:34:00 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E035A22B30 for ; Thu, 9 Jun 2022 05:33:49 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 55970EFD; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 52B30D4381; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Arshad Hussain , James Simmons Subject: [PATCH 14/18] lustre: quota: fallocate does not increase projectid usage Date: Thu, 9 Jun 2022 08:33:10 -0400 Message-Id: <1654777994-29806-15-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Arshad Hussain fallocate() was not accounting for projectid quota usage. This was happening due to two reasons. 1) the projectid was not properly passed to md_op_data in ll_set_project() and 2) the OBD_MD_FLPROJID flag was not set receive the projctid. This patch addresses the above reasons. Fixes: d748d2ffa1bc ("lustre: fallocate: Implement fallocate preallocate operation") WC-bug-id: https://jira.whamcloud.com/browse/LU-15519 Lustre-commit: 5fc934ebbbe665f24 ("LU-15519 quota: fallocate does not increase projectid usage") Signed-off-by: Arshad Hussain Reviewed-on: https://review.whamcloud.com/46676 Reviewed-by: Andreas Dilger Reviewed-by: Hongchao Zhang Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 1 + fs/lustre/llite/file.c | 18 +++++++++++------- fs/lustre/llite/vvp_object.c | 3 ++- fs/lustre/lov/lov_io.c | 2 ++ fs/lustre/osc/osc_io.c | 9 ++++++--- 5 files changed, 22 insertions(+), 11 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index b98109d..06f03b4 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -1889,6 +1889,7 @@ struct cl_io { loff_t sa_falloc_end; uid_t sa_falloc_uid; gid_t sa_falloc_gid; + u32 sa_falloc_projid; } ci_setattr; struct cl_data_version_io { u64 dv_data_version; diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 30f522b..5be77e8 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -2629,7 +2629,7 @@ static int ll_do_fiemap(struct inode *inode, struct fiemap *fiemap, goto out; } - fmkey.lfik_oa.o_valid = OBD_MD_FLID | OBD_MD_FLGROUP; + fmkey.lfik_oa.o_valid = OBD_MD_FLID | OBD_MD_FLGROUP | OBD_MD_FLPROJID; obdo_from_inode(&fmkey.lfik_oa, inode, OBD_MD_FLSIZE); obdo_set_parent_fid(&fmkey.lfik_oa, &ll_i2info(inode)->lli_fid); @@ -3412,10 +3412,12 @@ static int ll_set_project(struct inode *inode, u32 xflags, u32 projid) op_data->op_attr_flags = ll_inode_to_ext_flags(inode_flags); if (xflags & FS_XFLAG_PROJINHERIT) op_data->op_attr_flags |= LUSTRE_PROJINHERIT_FL; + + /* pass projid to md_op_data */ op_data->op_projid = projid; - op_data->op_xvalid |= OP_XVALID_PROJID; - rc = md_setattr(ll_i2sbi(inode)->ll_md_exp, op_data, NULL, - 0, &req); + + op_data->op_xvalid |= OP_XVALID_PROJID | OP_XVALID_FLAGS; + rc = md_setattr(ll_i2sbi(inode)->ll_md_exp, op_data, NULL, 0, &req); ptlrpc_req_finished(req); if (rc) goto out_fsxattr; @@ -5262,11 +5264,11 @@ int ll_getattr(const struct path *path, struct kstat *stat, int cl_falloc(struct file *file, struct inode *inode, int mode, loff_t offset, loff_t len) { + loff_t size = i_size_read(inode); struct lu_env *env; struct cl_io *io; u16 refcheck; int rc; - loff_t size = i_size_read(inode); env = cl_env_get(&refcheck); if (IS_ERR(env)) @@ -5283,12 +5285,14 @@ int cl_falloc(struct file *file, struct inode *inode, int mode, loff_t offset, io->u.ci_setattr.sa_falloc_end = offset + len; io->u.ci_setattr.sa_subtype = CL_SETATTR_FALLOCATE; - CDEBUG(D_INODE, "UID %u GID %u\n", + CDEBUG(D_INODE, "UID %u GID %u PRJID %u\n", from_kuid(&init_user_ns, inode->i_uid), - from_kgid(&init_user_ns, inode->i_gid)); + from_kgid(&init_user_ns, inode->i_gid), + ll_i2info(inode)->lli_projid); io->u.ci_setattr.sa_falloc_uid = from_kuid(&init_user_ns, inode->i_uid); io->u.ci_setattr.sa_falloc_gid = from_kgid(&init_user_ns, inode->i_gid); + io->u.ci_setattr.sa_falloc_projid = ll_i2info(inode)->lli_projid; if (io->u.ci_setattr.sa_falloc_end > size) { loff_t newsize = io->u.ci_setattr.sa_falloc_end; diff --git a/fs/lustre/llite/vvp_object.c b/fs/lustre/llite/vvp_object.c index 8a53458..64ecdb9 100644 --- a/fs/lustre/llite/vvp_object.c +++ b/fs/lustre/llite/vvp_object.c @@ -190,7 +190,8 @@ static int vvp_object_glimpse(const struct lu_env *env, static void vvp_req_attr_set(const struct lu_env *env, struct cl_object *obj, struct cl_req_attr *attr) { - u64 valid_flags = OBD_MD_FLTYPE | OBD_MD_FLUID | OBD_MD_FLGID; + u64 valid_flags = OBD_MD_FLTYPE | OBD_MD_FLUID | OBD_MD_FLGID | + OBD_MD_FLPROJID; struct inode *inode; struct obdo *oa; diff --git a/fs/lustre/lov/lov_io.c b/fs/lustre/lov/lov_io.c index 38dacd35..b535092 100644 --- a/fs/lustre/lov/lov_io.c +++ b/fs/lustre/lov/lov_io.c @@ -685,6 +685,8 @@ static void lov_io_sub_inherit(struct lov_io_sub *sub, struct lov_io *lio, parent->u.ci_setattr.sa_falloc_uid; io->u.ci_setattr.sa_falloc_gid = parent->u.ci_setattr.sa_falloc_gid; + io->u.ci_setattr.sa_falloc_projid = + parent->u.ci_setattr.sa_falloc_projid; } if (cl_io_is_trunc(io)) { loff_t new_size = parent->u.ci_setattr.sa_attr.lvb_size; diff --git a/fs/lustre/osc/osc_io.c b/fs/lustre/osc/osc_io.c index db91bf2..1361d7f 100644 --- a/fs/lustre/osc/osc_io.c +++ b/fs/lustre/osc/osc_io.c @@ -671,11 +671,14 @@ static int osc_io_setattr_start(const struct lu_env *env, oa->o_blocks = io->u.ci_setattr.sa_falloc_end; oa->o_uid = io->u.ci_setattr.sa_falloc_uid; oa->o_gid = io->u.ci_setattr.sa_falloc_gid; + oa->o_projid = io->u.ci_setattr.sa_falloc_projid; oa->o_valid |= OBD_MD_FLSIZE | OBD_MD_FLBLOCKS | - OBD_MD_FLUID | OBD_MD_FLGID; + OBD_MD_FLUID | OBD_MD_FLGID | OBD_MD_FLPROJID; - CDEBUG(D_INODE, "size %llu blocks %llu uid %u gid %u\n", - oa->o_size, oa->o_blocks, oa->o_uid, oa->o_gid); + CDEBUG(D_INODE, + "size %llu blocks %llu uid %u gid %u prjid %u\n", + oa->o_size, oa->o_blocks, oa->o_uid, oa->o_gid, + oa->o_projid); result = osc_fallocate_base(osc_export(cl2osc(obj)), oa, osc_async_upcall, cbargs, falloc_mode); From patchwork Thu Jun 9 12:33:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 992AAC43334 for ; Thu, 9 Jun 2022 12:34:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245372AbiFIMeF (ORCPT ); Thu, 9 Jun 2022 08:34:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245554AbiFIMeC (ORCPT ); Thu, 9 Jun 2022 08:34:02 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30088237C6 for ; Thu, 9 Jun 2022 05:33:52 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 56F15EFE; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 55D8CD438A; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Alexey Lyashkov , James Simmons Subject: [PATCH 15/18] lnet: selftest: improve lnet_selftest speed Date: Thu, 9 Jun 2022 08:33:11 -0400 Message-Id: <1654777994-29806-16-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Alexey Lyashkov lets replace a global spinlock with atomic variables, to avoid cpu power limit in testing. HPE-bug-id: LUS-10812 WC-bug-id: https://jira.whamcloud.com/browse/LU-15718 Lustre-commit: dd5aa640781d8c6dc ("LU-15718 lnet: improve lnet_selftest speed") Signed-off-by: Alexey Lyashkov Reviewed-on: https://review.whamcloud.com/47002 Reviewed-by: Cyril Bordage Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/selftest/rpc.c | 105 ++++++++++++++++++++++++------------------- net/lnet/selftest/selftest.h | 1 - 2 files changed, 58 insertions(+), 48 deletions(-) diff --git a/net/lnet/selftest/rpc.c b/net/lnet/selftest/rpc.c index 17277b8..c376019 100644 --- a/net/lnet/selftest/rpc.c +++ b/net/lnet/selftest/rpc.c @@ -50,15 +50,41 @@ enum srpc_state { SRPC_STATE_STOPPING, }; +enum rpc_counter_32 { + SRPC_ERROR, + SRPC_RPC_SENT, + SRPC_RPC_RCVD, + SRPC_RPC_DROP, + SRPC_RPC_EXPIRED, + SRPC_COUNTER32_MAX, +}; + +enum rpc_counter_64 { + SRPC_BULK_GET, + SRPC_BULK_PUT, + SRPC_COUNTER64_MAX, +}; + static struct smoketest_rpc { - spinlock_t rpc_glock; /* global lock */ - struct srpc_service *rpc_services[SRPC_SERVICE_MAX_ID + 1]; - lnet_handler_t rpc_lnet_handler; /* _the_ LNet event queue */ - enum srpc_state rpc_state; - struct srpc_counters rpc_counters; - u64 rpc_matchbits; /* matchbits counter */ + spinlock_t rpc_glock; /* global lock */ + struct srpc_service *rpc_services[SRPC_SERVICE_MAX_ID + 1]; + lnet_handler_t rpc_lnet_handler; /* _the_ LNet event queue */ + enum srpc_state rpc_state; + struct srpc_counters rpc_counters; + atomic_t rpc_counters32[SRPC_COUNTER32_MAX]; + atomic64_t rpc_counters64[SRPC_COUNTER64_MAX]; + atomic64_t rpc_matchbits; /* matchbits counter */ } srpc_data; +#define RPC_STAT32(a) \ + srpc_data.rpc_counters32[(a)] + +#define GET_RPC_STAT32(a) \ + atomic_read(&srpc_data.rpc_counters32[(a)]) + +#define GET_RPC_STAT64(a) \ + atomic64_read(&srpc_data.rpc_counters64[(a)]) + static inline int srpc_serv_portal(int svc_id) { @@ -69,18 +95,17 @@ enum srpc_state { /* forward ref's */ void srpc_handle_rpc(struct swi_workitem *wi); -void srpc_get_counters(struct srpc_counters *cnt) -{ - spin_lock(&srpc_data.rpc_glock); - *cnt = srpc_data.rpc_counters; - spin_unlock(&srpc_data.rpc_glock); -} -void srpc_set_counters(const struct srpc_counters *cnt) +void srpc_get_counters(struct srpc_counters *cnt) { - spin_lock(&srpc_data.rpc_glock); - srpc_data.rpc_counters = *cnt; - spin_unlock(&srpc_data.rpc_glock); + cnt->errors = GET_RPC_STAT32(SRPC_ERROR); + cnt->rpcs_sent = GET_RPC_STAT32(SRPC_RPC_SENT); + cnt->rpcs_rcvd = GET_RPC_STAT32(SRPC_RPC_RCVD); + cnt->rpcs_dropped = GET_RPC_STAT32(SRPC_RPC_DROP); + cnt->rpcs_expired = GET_RPC_STAT32(SRPC_RPC_EXPIRED); + + cnt->bulk_get = GET_RPC_STAT64(SRPC_BULK_GET); + cnt->bulk_put = GET_RPC_STAT64(SRPC_BULK_PUT); } static int @@ -162,12 +187,7 @@ struct srpc_bulk * static inline u64 srpc_next_id(void) { - u64 id; - - spin_lock(&srpc_data.rpc_glock); - id = srpc_data.rpc_matchbits++; - spin_unlock(&srpc_data.rpc_glock); - return id; + return atomic64_inc_return(&srpc_data.rpc_matchbits); } static void @@ -922,11 +942,8 @@ struct srpc_bulk * rpc, sv->sv_name, libcfs_id2str(rpc->srpc_peer), swi_state2str(rpc->srpc_wi.swi_state), status); - if (status) { - spin_lock(&srpc_data.rpc_glock); - srpc_data.rpc_counters.rpcs_dropped++; - spin_unlock(&srpc_data.rpc_glock); - } + if (status) + atomic_inc(&RPC_STAT32(SRPC_RPC_DROP)); if (rpc->srpc_done) (*rpc->srpc_done) (rpc); @@ -1096,9 +1113,7 @@ struct srpc_bulk * spin_unlock(&rpc->crpc_lock); - spin_lock(&srpc_data.rpc_glock); - srpc_data.rpc_counters.rpcs_expired++; - spin_unlock(&srpc_data.rpc_glock); + atomic_inc(&RPC_STAT32(SRPC_RPC_EXPIRED)); } static void @@ -1431,11 +1446,10 @@ struct srpc_client_rpc * if (ev->status) { u32 errors; - spin_lock(&srpc_data.rpc_glock); if (ev->status != -ECANCELED) /* cancellation is not error */ - srpc_data.rpc_counters.errors++; - errors = srpc_data.rpc_counters.errors; - spin_unlock(&srpc_data.rpc_glock); + errors = atomic_inc_return(&RPC_STAT32(SRPC_ERROR)); + else + errors = atomic_read(&RPC_STAT32(SRPC_ERROR)); CNETERR("LNet event status %d type %d, RPC errors %u\n", ev->status, ev->type, errors); @@ -1449,11 +1463,9 @@ struct srpc_client_rpc * rpcev->ev_status, rpcev->ev_type, rpcev->ev_lnet); LBUG(); case SRPC_REQUEST_SENT: - if (!ev->status && ev->type != LNET_EVENT_UNLINK) { - spin_lock(&srpc_data.rpc_glock); - srpc_data.rpc_counters.rpcs_sent++; - spin_unlock(&srpc_data.rpc_glock); - } + if (!ev->status && ev->type != LNET_EVENT_UNLINK) + atomic_inc(&RPC_STAT32(SRPC_RPC_SENT)); + /* fall through */ case SRPC_REPLY_RCVD: case SRPC_BULK_REQ_RCVD: @@ -1566,9 +1578,7 @@ struct srpc_client_rpc * spin_unlock(&scd->scd_lock); - spin_lock(&srpc_data.rpc_glock); - srpc_data.rpc_counters.rpcs_rcvd++; - spin_unlock(&srpc_data.rpc_glock); + atomic_inc(&RPC_STAT32(SRPC_RPC_RCVD)); break; case SRPC_BULK_GET_RPLD: @@ -1581,14 +1591,14 @@ struct srpc_client_rpc * /* fall through */ case SRPC_BULK_PUT_SENT: if (!ev->status && ev->type != LNET_EVENT_UNLINK) { - spin_lock(&srpc_data.rpc_glock); + atomic64_t *data; if (rpcev->ev_type == SRPC_BULK_GET_RPLD) - srpc_data.rpc_counters.bulk_get += ev->mlength; + data = &srpc_data.rpc_counters64[SRPC_BULK_GET]; else - srpc_data.rpc_counters.bulk_put += ev->mlength; + data = &srpc_data.rpc_counters64[SRPC_BULK_PUT]; - spin_unlock(&srpc_data.rpc_glock); + atomic64_add(ev->mlength, data); } /* fall through */ case SRPC_REPLY_SENT: @@ -1619,7 +1629,8 @@ struct srpc_client_rpc * /* 1 second pause to avoid timestamp reuse */ schedule_timeout_uninterruptible(HZ); - srpc_data.rpc_matchbits = ((u64)ktime_get_real_seconds()) << 48; + atomic64_set(&srpc_data.rpc_matchbits, + ((u64)ktime_get_real_seconds() << 48)); srpc_data.rpc_state = SRPC_STATE_NONE; diff --git a/net/lnet/selftest/selftest.h b/net/lnet/selftest/selftest.h index 26202c1..223a432 100644 --- a/net/lnet/selftest/selftest.h +++ b/net/lnet/selftest/selftest.h @@ -452,7 +452,6 @@ struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int off, int srpc_service_add_buffers(struct srpc_service *sv, int nbuffer); void srpc_service_remove_buffers(struct srpc_service *sv, int nbuffer); void srpc_get_counters(struct srpc_counters *cnt); -void srpc_set_counters(const struct srpc_counters *cnt); extern struct workqueue_struct *lst_serial_wq; extern struct workqueue_struct **lst_test_wq; From patchwork Thu Jun 9 12:33:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1B7FCCA473 for ; Thu, 9 Jun 2022 12:34:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245344AbiFIMeL (ORCPT ); Thu, 9 Jun 2022 08:34:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245363AbiFIMeI (ORCPT ); Thu, 9 Jun 2022 08:34:08 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C748D2252D for ; Thu, 9 Jun 2022 05:34:01 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5A5AFEFF; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 58E6DD439B; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Etienne AUJAMES , James Simmons Subject: [PATCH 16/18] lustre: mdc: Use early cancels for hsm requests Date: Thu, 9 Jun 2022 08:33:12 -0400 Message-Id: <1654777994-29806-17-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Etienne AUJAMES HSM RELEASE and RESTORE requests take EX layout lock on the MDT side. So the client can use early cancel for its local lock on the resource to limit the contention (mdt side). This patch does not pack ldlm request inside the hsm request because the field (RMF_DLM_REQ) does not exist in the request. Adding this field inside the request would break compatibility with _old_ servers. WC-bug-id: https://jira.whamcloud.com/browse/LU-15132 Lustre-commit: 60d2a4b0efa4a944b ("LU-15132 mdc: Use early cancels for hsm requests") Signed-off-by: Etienne AUJAMES Reviewed-on: https://review.whamcloud.com/47181 Reviewed-by: Nikitas Angelinas Reviewed-by: Sergey Cheremencev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/mdc/mdc_request.c | 37 +++++++++++++++++++++++++++++++++---- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/fs/lustre/mdc/mdc_request.c b/fs/lustre/mdc/mdc_request.c index f553d44..bb51878 100644 --- a/fs/lustre/mdc/mdc_request.c +++ b/fs/lustre/mdc/mdc_request.c @@ -2000,6 +2000,32 @@ static int mdc_ioc_hsm_state_set(struct obd_export *exp, return rc; } +/* For RESTORE and RELEASE the mdt will take EX lock on the file layout. + * So we can use early cancel on client side locks for that resource. + */ +static inline int mdc_hsm_request_lock_to_cancel(struct obd_export *exp, + struct hsm_user_request *hur, + struct list_head *cancels) +{ + struct hsm_user_item *hui = &hur->hur_user_item[0]; + struct hsm_request *req_hr = &hur->hur_request; + int count = 0; + int i; + + if (req_hr->hr_action != HUA_RESTORE && + req_hr->hr_action != HUA_RELEASE) + return 0; + + for (i = 0; i < req_hr->hr_itemcount; i++, hui++) { + if (!fid_is_sane(&hui->hui_fid)) + continue; + count += mdc_resource_get_unused(exp, &hui->hui_fid, cancels, + LCK_EX, MDS_INODELOCK_LAYOUT); + } + + return count; +} + static int mdc_ioc_hsm_request(struct obd_export *exp, struct hsm_user_request *hur) { @@ -2008,13 +2034,13 @@ static int mdc_ioc_hsm_request(struct obd_export *exp, struct hsm_request *req_hr; struct hsm_user_item *req_hui; char *req_opaque; + LIST_HEAD(cancels); + int count; int rc; req = ptlrpc_request_alloc(imp, &RQF_MDS_HSM_REQUEST); - if (!req) { - rc = -ENOMEM; - goto out; - } + if (!req) + return -ENOMEM; req_capsule_set_size(&req->rq_pill, &RMF_MDS_HSM_USER_ITEM, RCL_CLIENT, hur->hur_request.hr_itemcount @@ -2028,6 +2054,9 @@ static int mdc_ioc_hsm_request(struct obd_export *exp, return rc; } + /* Cancel existing locks */ + count = mdc_hsm_request_lock_to_cancel(exp, hur, &cancels); + ldlm_cli_cancel_list(&cancels, count, NULL, 0); mdc_pack_body(&req->rq_pill, NULL, 0, 0, -1, 0); /* Copy hsm_request struct */ From patchwork Thu Jun 9 12:33:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA087C43334 for ; Thu, 9 Jun 2022 12:34:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240794AbiFIMeR (ORCPT ); Thu, 9 Jun 2022 08:34:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245443AbiFIMeJ (ORCPT ); Thu, 9 Jun 2022 08:34:09 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E44E621E28 for ; Thu, 9 Jun 2022 05:34:08 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5E31D1020; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5C522D43A0; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Alexander Boyko , James Simmons Subject: [PATCH 17/18] lustre: ptlrpc: send disconnected events Date: Thu, 9 Jun 2022 08:33:13 -0400 Message-Id: <1654777994-29806-18-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Alexander Boyko Failover of node does not produce disconnect event, only inactive. Add sending disconnected events. HPE-bug-id: LUS-10750 WC-bug-id: https://jira.whamcloud.com/browse/LU-15724 Lustre-commit: e55fc043679cdfadf ("LU-15724 osp: wakeup all precreate threads") Signed-off-by: Alexander Boyko Reviewed-on: https://review.whamcloud.com/47005 Reviewed-by: Alexey Lyashkov Reviewed-by: Sergey Cheremencev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/import.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/lustre/ptlrpc/import.c b/fs/lustre/ptlrpc/import.c index 3dc987cf..c0aee34 100644 --- a/fs/lustre/ptlrpc/import.c +++ b/fs/lustre/ptlrpc/import.c @@ -1726,6 +1726,10 @@ int ptlrpc_disconnect_import(struct obd_import *imp, int noclose) memset(&imp->imp_remote_handle, 0, sizeof(imp->imp_remote_handle)); spin_unlock(&imp->imp_lock); + obd_import_event(imp->imp_obd, imp, IMP_EVENT_DISCON); + if (!noclose) + obd_import_event(imp->imp_obd, imp, IMP_EVENT_INACTIVE); + if (rc == -ETIMEDOUT || rc == -ENOTCONN || rc == -ESHUTDOWN) rc = 0; From patchwork Thu Jun 9 12:33:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 12875534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3045C433EF for ; Thu, 9 Jun 2022 12:34:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245497AbiFIMeP (ORCPT ); Thu, 9 Jun 2022 08:34:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245537AbiFIMeL (ORCPT ); Thu, 9 Jun 2022 08:34:11 -0400 Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AD5A22520 for ; Thu, 9 Jun 2022 05:34:10 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 61DAC1021; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5F4E9D43A3; Thu, 9 Jun 2022 08:33:16 -0400 (EDT) From: James Simmons To: Eric Biggers , Andreas Dilger , NeilBrown Cc: linux-fscrypt@vger.kernel.org, Chris Horn , James Simmons Subject: [PATCH 18/18] lnet: Avoid redundant peer NI lookups Date: Thu, 9 Jun 2022 08:33:14 -0400 Message-Id: <1654777994-29806-19-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> References: <1654777994-29806-1-git-send-email-jsimmons@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org From: Chris Horn Each caller of lnet_peer_ni_traffic_add() performs a subsequent call to lnet_peer_ni_find_locked(). We can avoid the extra lookup by having lnet_peer_ni_traffic_add() return a peer NI pointer (or ERR_PTR as appropriate). lnet_peer_ni_traffic_add() now takes a ref on the peer NI to mimic the behavior of lnet_peer_ni_find_locked(). lnet_nid2peerni_ex() only has a single caller that always passes LNET_LOCK_EX for the cpt argument, so this function argument is removed. Some duplicate code dealing with ln_state handling is removed from lnet_peerni_by_nid_locked() WC-bug-id: https://jira.whamcloud.com/browse/LU-12756 Lustre-commit: b00ac5f7038434a33 ("LU-12756 lnet: Avoid redundant peer NI lookups") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/36623 Reviewed-by: Alexey Lyashkov Reviewed-by: Serguei Smirnov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 2 +- net/lnet/lnet/peer.c | 74 ++++++++++++++++++------------------------- net/lnet/lnet/router.c | 6 ++-- 3 files changed, 35 insertions(+), 47 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index 2e3c391..ca2a565 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -901,7 +901,7 @@ struct lnet_peer_ni *lnet_nid2peerni_locked(lnet_nid_t nid, lnet_nid_t pref, struct lnet_peer_ni *lnet_peerni_by_nid_locked(struct lnet_nid *nid, struct lnet_nid *pref, int cpt); -struct lnet_peer_ni *lnet_nid2peerni_ex(struct lnet_nid *nid, int cpt); +struct lnet_peer_ni *lnet_nid2peerni_ex(struct lnet_nid *nid); struct lnet_peer_ni *lnet_peer_get_ni_locked(struct lnet_peer *lp, lnet_nid_t nid); struct lnet_peer_ni *lnet_peer_ni_get_locked(struct lnet_peer *lp, diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 6c35901..57d137c 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -1836,19 +1836,22 @@ struct lnet_peer_net * /* * lpni creation initiated due to traffic either sending or receiving. + * Callers must hold ln_api_mutex + * Ref taken on lnet_peer_ni returned by this function */ -static int +static struct lnet_peer_ni * lnet_peer_ni_traffic_add(struct lnet_nid *nid, struct lnet_nid *pref) +__must_hold(&the_lnet.ln_api_mutex) { - struct lnet_peer *lp; - struct lnet_peer_net *lpn; + struct lnet_peer *lp = NULL; + struct lnet_peer_net *lpn = NULL; struct lnet_peer_ni *lpni; unsigned int flags = 0; int rc = 0; if (LNET_NID_IS_ANY(nid)) { rc = -EINVAL; - goto out; + goto out_err; } /* lnet_net_lock is not needed here because ln_api_lock is held */ @@ -1860,7 +1863,6 @@ struct lnet_peer_net * * traffic, we just assume everything is ok and * return. */ - lnet_peer_ni_decref_locked(lpni); goto out; } @@ -1868,24 +1870,29 @@ struct lnet_peer_net * rc = -ENOMEM; lp = lnet_peer_alloc(nid); if (!lp) - goto out; + goto out_err; lpn = lnet_peer_net_alloc(LNET_NID_NET(nid)); if (!lpn) - goto out_free_lp; + goto out_err; lpni = lnet_peer_ni_alloc(nid); if (!lpni) - goto out_free_lpn; + goto out_err; lnet_peer_ni_set_non_mr_pref_nid(lpni, pref); - return lnet_peer_attach_peer_ni(lp, lpn, lpni, flags); + /* lnet_peer_attach_peer_ni() always returns 0 */ + rc = lnet_peer_attach_peer_ni(lp, lpn, lpni, flags); -out_free_lpn: - kfree(lpn); -out_free_lp: - kfree(lp); + lnet_peer_ni_addref_locked(lpni); + +out_err: + if (rc) { + kfree(lpn); + kfree(lp); + lpni = ERR_PTR(rc); + } out: CDEBUG(D_NET, "peer %s: %d\n", libcfs_nidstr(nid), rc); - return rc; + return lpni; } /* @@ -2054,10 +2061,10 @@ struct lnet_peer_net * } struct lnet_peer_ni * -lnet_nid2peerni_ex(struct lnet_nid *nid, int cpt) +lnet_nid2peerni_ex(struct lnet_nid *nid) +__must_hold(&the_lnet.ln_api_mutex) { struct lnet_peer_ni *lpni = NULL; - int rc; if (the_lnet.ln_state != LNET_STATE_RUNNING) return ERR_PTR(-ESHUTDOWN); @@ -2070,19 +2077,11 @@ struct lnet_peer_ni * if (lpni) return lpni; - lnet_net_unlock(cpt); - - rc = lnet_peer_ni_traffic_add(nid, NULL); - if (rc) { - lpni = ERR_PTR(rc); - goto out_net_relock; - } + lnet_net_unlock(LNET_LOCK_EX); - lpni = lnet_peer_ni_find_locked(nid); - LASSERT(lpni); + lpni = lnet_peer_ni_traffic_add(nid, NULL); -out_net_relock: - lnet_net_lock(cpt); + lnet_net_lock(LNET_LOCK_EX); return lpni; } @@ -2096,7 +2095,6 @@ struct lnet_peer_ni * struct lnet_nid *pref, int cpt) { struct lnet_peer_ni *lpni = NULL; - int rc; if (the_lnet.ln_state != LNET_STATE_RUNNING) return ERR_PTR(-ESHUTDOWN); @@ -2124,30 +2122,18 @@ struct lnet_peer_ni * lnet_net_unlock(cpt); mutex_lock(&the_lnet.ln_api_mutex); /* - * Shutdown is only set under the ln_api_lock, so a single + * the_lnet.ln_state is only modified under the ln_api_lock, so a single * check here is sufficent. */ - if (the_lnet.ln_state != LNET_STATE_RUNNING) { - lpni = ERR_PTR(-ESHUTDOWN); - goto out_mutex_unlock; - } - - rc = lnet_peer_ni_traffic_add(nid, pref); - if (rc) { - lpni = ERR_PTR(rc); - goto out_mutex_unlock; - } - - lpni = lnet_peer_ni_find_locked(nid); - LASSERT(lpni); + if (the_lnet.ln_state == LNET_STATE_RUNNING) + lpni = lnet_peer_ni_traffic_add(nid, pref); -out_mutex_unlock: mutex_unlock(&the_lnet.ln_api_mutex); lnet_net_lock(cpt); /* Lock has been dropped, check again for shutdown. */ if (the_lnet.ln_state != LNET_STATE_RUNNING) { - if (!IS_ERR(lpni)) + if (!IS_ERR_OR_NULL(lpni)) lnet_peer_ni_decref_locked(lpni); lpni = ERR_PTR(-ESHUTDOWN); } diff --git a/net/lnet/lnet/router.c b/net/lnet/lnet/router.c index 60ae15d..b4f7aaa 100644 --- a/net/lnet/lnet/router.c +++ b/net/lnet/lnet/router.c @@ -702,7 +702,7 @@ static void lnet_shuffle_seed(void) /* lnet_nid2peerni_ex() grabs a ref on the lpni. We will need to * lose that once we're done */ - lpni = lnet_nid2peerni_ex(gateway, LNET_LOCK_EX); + lpni = lnet_nid2peerni_ex(gateway); if (IS_ERR(lpni)) { lnet_net_unlock(LNET_LOCK_EX); @@ -716,7 +716,9 @@ static void lnet_shuffle_seed(void) return rc; } - LASSERT(lpni->lpni_peer_net && lpni->lpni_peer_net->lpn_peer); + LASSERT(lpni); + LASSERT(lpni->lpni_peer_net); + LASSERT(lpni->lpni_peer_net->lpn_peer); gw = lpni->lpni_peer_net->lpn_peer; route->lr_gateway = gw;