From patchwork Wed Jun 3 00:59:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584731 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D88881392 for ; Wed, 3 Jun 2020 01:00:15 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C112A2072F for ; Wed, 3 Jun 2020 01:00:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C112A2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 651B121F6F8; Tue, 2 Jun 2020 18:00:10 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 38EBE21F2E0 for ; Tue, 2 Jun 2020 18:00:05 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5B98A5E2; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 559642C5; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:40 -0400 Message-Id: <1591146001-27171-2-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 01/22] lnet: libcfs: fix CPT handling for UP systems X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" If a NULL nodemask or NULL a cpumask are passed to certain Linux kernel functions can result in a crash. So for the UP case return the default cpu_online_mask, which should only have one entry, and the system nodemask defined by node_online_map. We can also remove some unused inline functions for the UP case. Fixes: 7b8e2026fb18 ("staging: lustre: remove conditional compilation from libcfs_cpu.c") WC-bug-id: https://jira.whamcloud.com/browse/LU-9859 Lustre-commit: 83a2dbe9e50d3 ("83a2dbe9e50d3a60312fa40f41052b23a3edb3ca") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/37881 Reviewed-by: Shaun Tancheff Reviewed-by: Oleg Drokin --- include/linux/libcfs/libcfs_cpu.h | 82 ++++++++++----------------------------- net/lnet/libcfs/libcfs_cpu.c | 5 +-- 2 files changed, 23 insertions(+), 64 deletions(-) diff --git a/include/linux/libcfs/libcfs_cpu.h b/include/linux/libcfs/libcfs_cpu.h index c354dbd..4b0604a 100644 --- a/include/linux/libcfs/libcfs_cpu.h +++ b/include/linux/libcfs/libcfs_cpu.h @@ -85,9 +85,13 @@ extern struct cfs_cpt_table *cfs_cpt_tab; /** - * return cpumask of CPU partition @cpt + * destroy a CPU partition table */ -cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt); +void cfs_cpt_table_free(struct cfs_cpt_table *cptab); +/** + * create a cfs_cpt_table with @ncpt number of partitions + */ +struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt); /** * print string information of cpt-table */ @@ -109,6 +113,10 @@ */ int cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt); /** + * return cpumask of CPU partition @cpt + */ +cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt); +/** * return nodemask of CPU partition @cpt */ nodemask_t *cfs_cpt_nodemask(struct cfs_cpt_table *cptab, int cpt); @@ -185,6 +193,15 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, #define cfs_cpt_tab ((struct cfs_cpt_table *)NULL) +static inline void cfs_cpt_table_free(struct cfs_cpt_table *cptab) +{ +} + +static inline struct cfs_cpt_table *cfs_cpt_table_alloc(int ncpt) +{ + return NULL; +} + static inline int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len) { @@ -214,7 +231,7 @@ static inline int cfs_cpt_distance_print(struct cfs_cpt_table *cptab, static inline cpumask_var_t *cfs_cpt_cpumask(struct cfs_cpt_table *cptab, int cpt) { - return NULL; + return (cpumask_var_t *) cpu_online_mask; } static inline int cfs_cpt_number(struct cfs_cpt_table *cptab) @@ -227,15 +244,10 @@ static inline int cfs_cpt_weight(struct cfs_cpt_table *cptab, int cpt) return 1; } -static inline int cfs_cpt_online(struct cfs_cpt_table *cptab, int cpt) -{ - return 1; -} - static inline nodemask_t *cfs_cpt_nodemask(struct cfs_cpt_table *cptab, int cpt) { - return NULL; + return &node_online_map; } static inline unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, @@ -244,50 +256,12 @@ static inline unsigned int cfs_cpt_distance(struct cfs_cpt_table *cptab, return 1; } -static inline int cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, - int cpu) -{ - return 1; -} - -static inline void cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, - int cpu) -{ -} - -static inline int cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt, - const cpumask_t *mask) -{ - return 1; -} - -static inline void cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt, - const cpumask_t *mask) -{ -} - static inline int cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node) { return 1; } -static inline void cfs_cpt_unset_node(struct cfs_cpt_table *cptab, int cpt, - int node) -{ -} - -static inline int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt, - const nodemask_t *mask) -{ - return 1; -} - -static inline void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, - int cpt, const nodemask_t *mask) -{ -} - static inline int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt) { return 0; @@ -298,11 +272,6 @@ static inline int cfs_cpt_current(struct cfs_cpt_table *cptab, int remap) return 0; } -static inline int cfs_cpt_of_cpu(struct cfs_cpt_table *cptab, int cpu) -{ - return 0; -} - static inline int cfs_cpt_of_node(struct cfs_cpt_table *cptab, int node) { return 0; @@ -324,15 +293,6 @@ static inline void cfs_cpu_fini(void) #endif /* CONFIG_SMP */ -/** - * destroy a CPU partition table - */ -void cfs_cpt_table_free(struct cfs_cpt_table *cptab); -/** - * create a cfs_cpt_table with @ncpt number of partitions - */ -struct cfs_cpt_table *cfs_cpt_table_alloc(unsigned int ncpt); - /* * allocate per-cpu-partition data, returned value is an array of pointers, * variable can be indexed by CPU ID. diff --git a/net/lnet/libcfs/libcfs_cpu.c b/net/lnet/libcfs/libcfs_cpu.c index 6363fd7..8969e1f 100644 --- a/net/lnet/libcfs/libcfs_cpu.c +++ b/net/lnet/libcfs/libcfs_cpu.c @@ -35,7 +35,6 @@ #include #include #include -#include #include #include @@ -1162,13 +1161,13 @@ int cfs_cpu_init(void) #ifdef CONFIG_HOTPLUG_CPU ret = cpuhp_setup_state_nocalls(CPUHP_LUSTRE_CFS_DEAD, - "staging/lustre/cfe:dead", NULL, + "fs/lustre/cfe:dead", NULL, cfs_cpu_dead); if (ret < 0) goto failed_cpu_dead; ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, - "staging/lustre/cfe:online", + "fs/lustre/cfe:online", cfs_cpu_online, NULL); if (ret < 0) goto failed_cpu_online; From patchwork Wed Jun 3 00:59:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584727 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2314B912 for ; Wed, 3 Jun 2020 01:00:09 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0B8F62072F for ; Wed, 3 Jun 2020 01:00:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B8F62072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 483D121F3ED; Tue, 2 Jun 2020 18:00:07 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8FC1221F2E0 for ; Tue, 2 Jun 2020 18:00:05 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 5F1505E3; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 57B7A2C6; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:41 -0400 Message-Id: <1591146001-27171-3-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 02/22] lustre: use BIT() macro where appropriate in include X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown When accessing a bit in a bitmap/mask/flags-word it can be more readable to use BIT(num) rather than "1 << num". This patch makes that change to various places in lustre/include. WC-bug-id: https://jira.whamcloud.com/browse/LU-6142 Lustre-commit: c438fba7f068b ("LU-6142 lustre: use BIT() macro where appropriate in include") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/38376 Reviewed-by: Olaf Faaland-LLNL Reviewed-by: Gian-Carlo DeFazio Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/include/cl_object.h | 14 +++++++------- fs/lustre/include/lu_object.h | 36 ++++++++++++++++++------------------ fs/lustre/include/lustre_dlm.h | 8 ++++---- fs/lustre/include/lustre_net.h | 32 ++++++++++++++++---------------- fs/lustre/include/lustre_sec.h | 12 ++++++------ fs/lustre/include/obd.h | 2 +- 6 files changed, 52 insertions(+), 52 deletions(-) diff --git a/fs/lustre/include/cl_object.h b/fs/lustre/include/cl_object.h index cde89f67..a0b9e87 100644 --- a/fs/lustre/include/cl_object.h +++ b/fs/lustre/include/cl_object.h @@ -176,13 +176,13 @@ struct cl_attr { * Fields in cl_attr that are being set. */ enum cl_attr_valid { - CAT_SIZE = 1 << 0, - CAT_KMS = 1 << 1, - CAT_MTIME = 1 << 3, - CAT_ATIME = 1 << 4, - CAT_CTIME = 1 << 5, - CAT_BLOCKS = 1 << 6, - CAT_UID = 1 << 7, + CAT_SIZE = BIT(0), + CAT_KMS = BIT(1), + CAT_MTIME = BIT(3), + CAT_ATIME = BIT(4), + CAT_CTIME = BIT(5), + CAT_BLOCKS = BIT(6), + CAT_UID = BIT(7), CAT_GID = BIT(8), CAT_PROJID = BIT(9), }; diff --git a/fs/lustre/include/lu_object.h b/fs/lustre/include/lu_object.h index 6886177..2a2f38e 100644 --- a/fs/lustre/include/lu_object.h +++ b/fs/lustre/include/lu_object.h @@ -299,11 +299,11 @@ struct lu_device { */ enum lu_device_tag { /** this is meta-data device */ - LU_DEVICE_MD = (1 << 0), + LU_DEVICE_MD = BIT(0), /** this is data device */ - LU_DEVICE_DT = (1 << 1), + LU_DEVICE_DT = BIT(1), /** data device in the client stack */ - LU_DEVICE_CL = (1 << 2) + LU_DEVICE_CL = BIT(2) }; /** @@ -477,8 +477,8 @@ enum lu_object_header_flags { }; enum lu_object_header_attr { - LOHA_EXISTS = 1 << 0, - LOHA_REMOTE = 1 << 1, + LOHA_EXISTS = BIT(0), + LOHA_REMOTE = BIT(1), /** * UNIX file type is stored in S_IFMT bits. */ @@ -874,7 +874,7 @@ struct lu_rdpg { }; enum lu_xattr_flags { - LU_XATTR_REPLACE = (1 << 0), + LU_XATTR_REPLACE = BIT(0), LU_XATTR_CREATE = BIT(1), LU_XATTR_MERGE = BIT(2), LU_XATTR_SPLIT = BIT(3), @@ -964,36 +964,36 @@ enum lu_context_tag { /** * Thread on md server */ - LCT_MD_THREAD = 1 << 0, + LCT_MD_THREAD = BIT(0), /** * Thread on dt server */ - LCT_DT_THREAD = 1 << 1, + LCT_DT_THREAD = BIT(1), /** * Context for transaction handle */ - LCT_TX_HANDLE = 1 << 2, + LCT_TX_HANDLE = BIT(2), /** * Thread on client */ - LCT_CL_THREAD = 1 << 3, + LCT_CL_THREAD = BIT(3), /** * A per-request session on a server, and a per-system-call session on * a client. */ - LCT_SESSION = 1 << 4, + LCT_SESSION = BIT(4), /** * A per-request data on OSP device */ - LCT_OSP_THREAD = 1 << 5, + LCT_OSP_THREAD = BIT(5), /** * MGS device thread */ - LCT_MG_THREAD = 1 << 6, + LCT_MG_THREAD = BIT(6), /** * Context for local operations */ - LCT_LOCAL = 1 << 7, + LCT_LOCAL = BIT(7), /** * session for server thread **/ @@ -1003,20 +1003,20 @@ enum lu_context_tag { * non-NULL lu_context_key::lct_exit() method. This is used to * optimize lu_context_exit() call. */ - LCT_HAS_EXIT = 1 << 28, + LCT_HAS_EXIT = BIT(28), /** * Don't add references for modules creating key values in that context. * This is only for contexts used internally by lu_object framework. */ - LCT_NOREF = 1 << 29, + LCT_NOREF = BIT(29), /** * Key is being prepared for retiring, don't create new values for it. */ - LCT_QUIESCENT = 1 << 30, + LCT_QUIESCENT = BIT(30), /** * Context should be remembered. */ - LCT_REMEMBER = 1 << 31, + LCT_REMEMBER = BIT(31), /** * Contexts usable in cache shrinker thread. */ diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index 174b314..74c380e 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -91,8 +91,8 @@ enum ldlm_error { * first speaking to a server. */ enum ldlm_side { - LDLM_NAMESPACE_SERVER = 1 << 0, - LDLM_NAMESPACE_CLIENT = 1 << 1 + LDLM_NAMESPACE_SERVER = 0x01, + LDLM_NAMESPACE_CLIENT = 0x02 }; /** @@ -293,8 +293,8 @@ struct ldlm_valblock_ops { * Greedy means release cached locks aggressively */ enum ldlm_appetite { - LDLM_NAMESPACE_GREEDY = 1 << 0, - LDLM_NAMESPACE_MODEST = 1 << 1 + LDLM_NAMESPACE_GREEDY = BIT(0), + LDLM_NAMESPACE_MODEST = BIT(1), }; struct ldlm_ns_bucket { diff --git a/fs/lustre/include/lustre_net.h b/fs/lustre/include/lustre_net.h index cad69a0..a94d826 100644 --- a/fs/lustre/include/lustre_net.h +++ b/fs/lustre/include/lustre_net.h @@ -386,8 +386,8 @@ struct ptlrpc_client { /** state flags of requests */ /* XXX only ones left are those used by the bulk descs as well! */ -#define PTL_RPC_FL_INTR (1 << 0) /* reply wait was interrupted by user */ -#define PTL_RPC_FL_TIMEOUT (1 << 7) /* request timed out waiting for reply */ +#define PTL_RPC_FL_INTR BIT(0) /* reply wait was interrupted by user */ +#define PTL_RPC_FL_TIMEOUT BIT(7) /* request timed out waiting for reply */ #define REQ_MAX_ACK_LOCKS 8 @@ -1013,7 +1013,7 @@ static inline bool ptlrpc_nrs_req_can_move(struct ptlrpc_request *req) static inline bool lustre_req_swabbed(struct ptlrpc_request *req, size_t index) { LASSERT(index < sizeof(req->rq_req_swab_mask) * 8); - return req->rq_req_swab_mask & (1 << index); + return req->rq_req_swab_mask & BIT(index); } /** @@ -1022,7 +1022,7 @@ static inline bool lustre_req_swabbed(struct ptlrpc_request *req, size_t index) static inline bool lustre_rep_swabbed(struct ptlrpc_request *req, size_t index) { LASSERT(index < sizeof(req->rq_rep_swab_mask) * 8); - return req->rq_rep_swab_mask & (1 << index); + return req->rq_rep_swab_mask & BIT(index); } /** @@ -1048,8 +1048,8 @@ static inline void lustre_set_req_swabbed(struct ptlrpc_request *req, size_t index) { LASSERT(index < sizeof(req->rq_req_swab_mask) * 8); - LASSERT((req->rq_req_swab_mask & (1 << index)) == 0); - req->rq_req_swab_mask |= 1 << index; + LASSERT((req->rq_req_swab_mask & BIT(index)) == 0); + req->rq_req_swab_mask |= BIT(index); } /** @@ -1059,8 +1059,8 @@ static inline void lustre_set_rep_swabbed(struct ptlrpc_request *req, size_t index) { LASSERT(index < sizeof(req->rq_rep_swab_mask) * 8); - LASSERT((req->rq_rep_swab_mask & (1 << index)) == 0); - req->rq_rep_swab_mask |= 1 << index; + LASSERT((req->rq_rep_swab_mask & BIT(index)) == 0); + req->rq_rep_swab_mask |= BIT(index); } /** @@ -1274,10 +1274,10 @@ struct ptlrpc_bulk_desc { }; enum { - SVC_STOPPED = 1 << 0, - SVC_STOPPING = 1 << 1, - SVC_STARTING = 1 << 2, - SVC_RUNNING = 1 << 3, + SVC_STOPPED = BIT(0), + SVC_STOPPING = BIT(1), + SVC_STARTING = BIT(2), + SVC_RUNNING = BIT(3), }; #define PTLRPC_THR_NAME_LEN 32 @@ -1694,21 +1694,21 @@ enum ptlrpcd_ctl_flags { /** * Ptlrpc thread start flag. */ - LIOD_START = 1 << 0, + LIOD_START = BIT(0), /** * Ptlrpc thread stop flag. */ - LIOD_STOP = 1 << 1, + LIOD_STOP = BIT(1), /** * Ptlrpc thread force flag (only stop force so far). * This will cause aborting any inflight rpcs handled * by thread if LIOD_STOP is specified. */ - LIOD_FORCE = 1 << 2, + LIOD_FORCE = BIT(2), /** * This is a recovery ptlrpc thread. */ - LIOD_RECOVERY = 1 << 3, + LIOD_RECOVERY = BIT(3), }; /** diff --git a/fs/lustre/include/lustre_sec.h b/fs/lustre/include/lustre_sec.h index e265868..aec30bc 100644 --- a/fs/lustre/include/lustre_sec.h +++ b/fs/lustre/include/lustre_sec.h @@ -467,12 +467,12 @@ struct ptlrpc_ctx_ops { #define PTLRPC_CTX_CACHED_BIT (8) /* in ctx cache (hash etc.) */ #define PTLRPC_CTX_ETERNAL_BIT (9) /* always valid */ -#define PTLRPC_CTX_NEW (1 << PTLRPC_CTX_NEW_BIT) -#define PTLRPC_CTX_UPTODATE (1 << PTLRPC_CTX_UPTODATE_BIT) -#define PTLRPC_CTX_DEAD (1 << PTLRPC_CTX_DEAD_BIT) -#define PTLRPC_CTX_ERROR (1 << PTLRPC_CTX_ERROR_BIT) -#define PTLRPC_CTX_CACHED (1 << PTLRPC_CTX_CACHED_BIT) -#define PTLRPC_CTX_ETERNAL (1 << PTLRPC_CTX_ETERNAL_BIT) +#define PTLRPC_CTX_NEW BIT(PTLRPC_CTX_NEW_BIT) +#define PTLRPC_CTX_UPTODATE BIT(PTLRPC_CTX_UPTODATE_BIT) +#define PTLRPC_CTX_DEAD BIT(PTLRPC_CTX_DEAD_BIT) +#define PTLRPC_CTX_ERROR BIT(PTLRPC_CTX_ERROR_BIT) +#define PTLRPC_CTX_CACHED BIT(PTLRPC_CTX_CACHED_BIT) +#define PTLRPC_CTX_ETERNAL BIT(PTLRPC_CTX_ETERNAL_BIT) #define PTLRPC_CTX_STATUS_MASK (PTLRPC_CTX_NEW_BIT | \ PTLRPC_CTX_UPTODATE | \ diff --git a/fs/lustre/include/obd.h b/fs/lustre/include/obd.h index bf3273a..0ff19c8 100644 --- a/fs/lustre/include/obd.h +++ b/fs/lustre/include/obd.h @@ -702,7 +702,7 @@ enum md_cli_flags { CLI_HASH64 = BIT(2), CLI_API32 = BIT(3), CLI_MIGRATE = BIT(4), - CLI_DIRTY_DATA = 1 << 5, + CLI_DIRTY_DATA = BIT(5), }; enum md_op_code { From patchwork Wed Jun 3 00:59:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584729 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B1BF1391 for ; Wed, 3 Jun 2020 01:00:15 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 830192072F for ; Wed, 3 Jun 2020 01:00:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 830192072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 6395C21F6F2; Tue, 2 Jun 2020 18:00:10 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id F057921F2E0 for ; Tue, 2 Jun 2020 18:00:05 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 644355E5; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5B8BD2CD; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:42 -0400 Message-Id: <1591146001-27171-4-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 03/22] lustre: use BIT() macro where appropriate X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown When accessing a bit in a bitmap/mask/flags-word it can be more readable to use BIT(num) rather than "1 << num". This patch makes that change to various places in lustre WC-bug-id: https://jira.whamcloud.com/browse/LU-6142 Lustre-commit: ff09273feadc9 ("LU-6142 lustre: use BIT() macro where appropriate") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/38377 Reviewed-by: Olaf Faaland-LLNL Reviewed-by: Gian-Carlo DeFazio Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ldlm/ldlm_extent.c | 4 ++-- fs/lustre/ldlm/ldlm_resource.c | 2 +- fs/lustre/llite/file.c | 2 +- fs/lustre/lov/lov_cl_internal.h | 2 +- fs/lustre/osc/lproc_osc.c | 4 ++-- fs/lustre/ptlrpc/layout.c | 6 +++--- 6 files changed, 10 insertions(+), 10 deletions(-) diff --git a/fs/lustre/ldlm/ldlm_extent.c b/fs/lustre/ldlm/ldlm_extent.c index 3e0f7a2..7474c85 100644 --- a/fs/lustre/ldlm/ldlm_extent.c +++ b/fs/lustre/ldlm/ldlm_extent.c @@ -157,7 +157,7 @@ void ldlm_extent_add_lock(struct ldlm_resource *res, LASSERT(RB_EMPTY_NODE(&lock->l_rb)); idx = lock_mode_to_index(lock->l_granted_mode); - LASSERT(lock->l_granted_mode == 1 << idx); + LASSERT(lock->l_granted_mode == BIT(idx)); LASSERT(lock->l_granted_mode == res->lr_itree[idx].lit_mode); tree = &res->lr_itree[idx]; @@ -202,7 +202,7 @@ void ldlm_extent_unlink_lock(struct ldlm_lock *lock) return; idx = lock_mode_to_index(lock->l_granted_mode); - LASSERT(lock->l_granted_mode == 1 << idx); + LASSERT(lock->l_granted_mode == BIT(idx)); tree = &res->lr_itree[idx]; tree->lit_size--; diff --git a/fs/lustre/ldlm/ldlm_resource.c b/fs/lustre/ldlm/ldlm_resource.c index a6572af..a461ca7 100644 --- a/fs/lustre/ldlm/ldlm_resource.c +++ b/fs/lustre/ldlm/ldlm_resource.c @@ -1023,7 +1023,7 @@ static struct ldlm_resource *ldlm_resource_new(enum ldlm_type ldlm_type) /* Initialize interval trees for each lock mode. */ for (idx = 0; idx < LCK_MODE_NUM; idx++) { res->lr_itree[idx].lit_size = 0; - res->lr_itree[idx].lit_mode = 1 << idx; + res->lr_itree[idx].lit_mode = BIT(idx); res->lr_itree[idx].lit_root = RB_ROOT_CACHED; } } diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 871fa59..ce50fd9 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -4410,7 +4410,7 @@ int ll_have_md_lock(struct inode *inode, u64 *bits, flags = LDLM_FL_BLOCK_GRANTED | LDLM_FL_CBPENDING | LDLM_FL_TEST_LOCK; for (i = 0; i < MDS_INODELOCK_NUMBITS && *bits != 0; i++) { - policy.l_inodebits.bits = *bits & (1 << i); + policy.l_inodebits.bits = *bits & BIT(i); if (policy.l_inodebits.bits == 0) continue; diff --git a/fs/lustre/lov/lov_cl_internal.h b/fs/lustre/lov/lov_cl_internal.h index 9dd523f..6796d88 100644 --- a/fs/lustre/lov/lov_cl_internal.h +++ b/fs/lustre/lov/lov_cl_internal.h @@ -83,7 +83,7 @@ struct lovsub_object; enum lov_device_flags { - LOV_DEV_INITIALIZED = 1 << 0 + LOV_DEV_INITIALIZED = BIT(0), }; /* diff --git a/fs/lustre/osc/lproc_osc.c b/fs/lustre/osc/lproc_osc.c index de35d32..9b43710 100644 --- a/fs/lustre/osc/lproc_osc.c +++ b/fs/lustre/osc/lproc_osc.c @@ -398,9 +398,9 @@ static int osc_checksum_type_seq_show(struct seq_file *m, void *v) return 0; for (i = 0; i < ARRAY_SIZE(cksum_name); i++) { - if (((1 << i) & obd->u.cli.cl_supp_cksum_types) == 0) + if ((BIT(i) & obd->u.cli.cl_supp_cksum_types) == 0) continue; - if (obd->u.cli.cl_cksum_type == (1 << i)) + if (obd->u.cli.cl_cksum_type == BIT(i)) seq_printf(m, "[%s] ", cksum_name[i]); else seq_printf(m, "%s ", cksum_name[i]); diff --git a/fs/lustre/ptlrpc/layout.c b/fs/lustre/ptlrpc/layout.c index 754c87d..1286547 100644 --- a/fs/lustre/ptlrpc/layout.c +++ b/fs/lustre/ptlrpc/layout.c @@ -817,17 +817,17 @@ enum rmf_flags { /** * The field is a string, must be NUL-terminated. */ - RMF_F_STRING = BIT(0), + RMF_F_STRING = BIT(0), /** * The field's buffer size need not match the declared @rmf_size. */ - RMF_F_NO_SIZE_CHECK = BIT(1), + RMF_F_NO_SIZE_CHECK = BIT(1), /** * The field's buffer size must be a whole multiple of the declared * @rmf_size and the @rmf_swabber function must work on the declared * @rmf_size worth of bytes. */ - RMF_F_STRUCT_ARRAY = BIT(2) + RMF_F_STRUCT_ARRAY = BIT(2), }; struct req_capsule; From patchwork Wed Jun 3 00:59:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584745 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 04E4A1391 for ; Wed, 3 Jun 2020 01:00:41 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E15EB2072F for ; Wed, 3 Jun 2020 01:00:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E15EB2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 69A0921FB09; Tue, 2 Jun 2020 18:00:23 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 5274721F2E1 for ; Tue, 2 Jun 2020 18:00:06 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 666245E6; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 5EEE22CE; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:43 -0400 Message-Id: <1591146001-27171-5-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 04/22] lustre: ptlrpc: change LONG_UNLINK to PTLRPC_REQ_LONG_UNLINK X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown The name "LONG_UNLINK" is vague and generic. Change it to PTLRPC_REQ_LONG_UNLINK to make it clear it is about requests taking a long time, and of interest to PTLRPC. WC-bug-id: https://jira.whamcloud.com/browse/LU-10467 Lustre-commit: 7c549da4014a7 ("LU-10467 ptlrpc: change LONG_UNLINK to PTLRPC_REQ_LONG_UNLINK") Suggested-by: Andreas Dilger Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/38405 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Signed-off-by: James Simmons --- fs/lustre/include/obd_support.h | 3 ++- fs/lustre/ptlrpc/client.c | 13 +++++++------ fs/lustre/ptlrpc/niobuf.c | 7 ++++--- fs/lustre/ptlrpc/service.c | 2 +- 4 files changed, 14 insertions(+), 11 deletions(-) diff --git a/fs/lustre/include/obd_support.h b/fs/lustre/include/obd_support.h index ce0aa8c..b706a20 100644 --- a/fs/lustre/include/obd_support.h +++ b/fs/lustre/include/obd_support.h @@ -100,7 +100,8 @@ #define OBD_IR_FACTOR_DEFAULT (OBD_IR_FACTOR_MAX / 2) /* default timeout for the MGS to become IR_FULL */ #define OBD_IR_MGS_TIMEOUT (4 * obd_timeout) -#define LONG_UNLINK 300 /* Unlink should happen before now */ +/* Unlink should happen within this many seconds. */ +#define PTLRPC_REQ_LONG_UNLINK 300 /** * Time interval of shrink, if the client is "idle" more than this interval, diff --git a/fs/lustre/ptlrpc/client.c b/fs/lustre/ptlrpc/client.c index 4865b04..22022ff 100644 --- a/fs/lustre/ptlrpc/client.c +++ b/fs/lustre/ptlrpc/client.c @@ -791,11 +791,12 @@ int ptlrpc_request_bufs_pack(struct ptlrpc_request *request, } if (fail_t) { - *fail_t = ktime_get_real_seconds() + LONG_UNLINK; + *fail_t = ktime_get_real_seconds() + + PTLRPC_REQ_LONG_UNLINK; if (fail2_t) *fail2_t = ktime_get_real_seconds() + - LONG_UNLINK; + PTLRPC_REQ_LONG_UNLINK; /* The RPC is infected, let the test change the * fail_loc @@ -2559,8 +2560,8 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) /* Let's setup deadline for reply unlink. */ if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_REPL_UNLINK) && async && request->rq_reply_deadline == 0 && cfs_fail_val == 0) - request->rq_reply_deadline = - ktime_get_real_seconds() + LONG_UNLINK; + request->rq_reply_deadline = ktime_get_real_seconds() + + PTLRPC_REQ_LONG_UNLINK; /* Nothing left to do. */ if (!ptlrpc_client_recv_or_unlink(request)) @@ -2583,12 +2584,12 @@ static int ptlrpc_unregister_reply(struct ptlrpc_request *request, int async) wait_queue_head_t *wq = (request->rq_set) ? &request->rq_set->set_waitq : &request->rq_reply_waitq; - int seconds = LONG_UNLINK; + int seconds = PTLRPC_REQ_LONG_UNLINK; /* * Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish NALs */ - while (seconds > LONG_UNLINK && + while (seconds > PTLRPC_REQ_LONG_UNLINK && (wait_event_idle_timeout(*wq, !ptlrpc_client_recv_or_unlink(request), HZ)) == 0) diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index a15c125..c59fc7f 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -253,7 +253,8 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) /* Let's setup deadline for reply unlink. */ if (OBD_FAIL_CHECK(OBD_FAIL_PTLRPC_LONG_BULK_UNLINK) && async && req->rq_bulk_deadline == 0 && cfs_fail_val == 0) - req->rq_bulk_deadline = ktime_get_real_seconds() + LONG_UNLINK; + req->rq_bulk_deadline = ktime_get_real_seconds() + + PTLRPC_REQ_LONG_UNLINK; if (ptlrpc_client_bulk_active(req) == 0) /* completed or */ return 1; /* never registered */ @@ -286,9 +287,9 @@ int ptlrpc_unregister_bulk(struct ptlrpc_request *req, int async) * Network access will complete in finite time but the HUGE * timeout lets us CWARN for visibility of sluggish LNDs */ - int seconds = LONG_UNLINK; + int seconds = PTLRPC_REQ_LONG_UNLINK; - while (seconds > LONG_UNLINK && + while (seconds > PTLRPC_REQ_LONG_UNLINK && wait_event_idle_timeout(*wq, !ptlrpc_client_bulk_active(req), HZ) == 0) diff --git a/fs/lustre/ptlrpc/service.c b/fs/lustre/ptlrpc/service.c index 2f1cd35..4d5e6b3 100644 --- a/fs/lustre/ptlrpc/service.c +++ b/fs/lustre/ptlrpc/service.c @@ -2859,7 +2859,7 @@ static void ptlrpc_wait_replies(struct ptlrpc_service_part *svcpt) * of sluggish LNDs */ cnt = 0; - while (cnt < LONG_UNLINK && + while (cnt < PTLRPC_REQ_LONG_UNLINK && (rc = wait_event_idle_timeout(svcpt->scp_waitq, svcpt->scp_nrqbds_posted == 0, HZ)) == 0) From patchwork Wed Jun 3 00:59:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584733 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E05B912 for ; Wed, 3 Jun 2020 01:00:22 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 163F52072F for ; Wed, 3 Jun 2020 01:00:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 163F52072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A228721F88B; Tue, 2 Jun 2020 18:00:13 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EC8A221F37D for ; Tue, 2 Jun 2020 18:00:06 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 6BD925E9; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 61F182D0; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:44 -0400 Message-Id: <1591146001-27171-6-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 05/22] lustre: llite: use %pd to report dentry names. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown Since Linux 3.12, it has been possible use the "%pd" format specifier to print a dentry name, so use that instead of "%.*s" and having to pass both the length and the name. WC-bug-id: https://jira.whamcloud.com/browse/LU-6142 Lustre-commit: ef9718e8e9cf8 ("LU-6142 llite: use %pd to report dentry names.") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/38442 Reviewed-by: James Simmons Reviewed-by: Olaf Faaland-LLNL Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/file.c | 4 ++-- fs/lustre/llite/pcc.c | 20 ++++++-------------- 2 files changed, 8 insertions(+), 16 deletions(-) diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index ce50fd9..52f0865 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -2810,8 +2810,8 @@ int ll_file_lock_ahead(struct file *file, struct llapi_lu_ladvise *ladvise) int result; CDEBUG(D_VFSTRACE, - "Lock request: file=%.*s, inode=%p, mode=%s start=%llu, end=%llu\n", - dentry->d_name.len, dentry->d_name.name, dentry->d_inode, + "Lock request: file=%pd, inode=%p, mode=%s start=%llu, end=%llu\n", + dentry, dentry->d_inode, user_lockname[ladvise->lla_lockahead_mode], (__u64) start, end); cl_mode = cl_mode_user_to_kernel(ladvise->lla_lockahead_mode); diff --git a/fs/lustre/llite/pcc.c b/fs/lustre/llite/pcc.c index 391733e..2adde68 100644 --- a/fs/lustre/llite/pcc.c +++ b/fs/lustre/llite/pcc.c @@ -1466,7 +1466,6 @@ int pcc_file_open(struct inode *inode, struct file *file) struct pcc_file *pccf = &fd->fd_pcc_file; struct file *pcc_file; struct path *path; - struct qstr *dname; bool cached = false; int rc = 0; @@ -1494,9 +1493,7 @@ int pcc_file_open(struct inode *inode, struct file *file) WARN_ON(pccf->pccf_file); path = &pcci->pcci_path; - dname = &path->dentry->d_name; - CDEBUG(D_CACHE, "opening pcc file '%.*s'\n", dname->len, - dname->name); + CDEBUG(D_CACHE, "opening pcc file '%pd'\n", path->dentry); pcc_file = dentry_open(path, file->f_flags, pcc_super_cred(inode->i_sb)); @@ -1519,7 +1516,6 @@ void pcc_file_release(struct inode *inode, struct file *file) struct ll_file_data *fd = file->private_data; struct pcc_file *pccf; struct path *path; - struct qstr *dname; if (!S_ISREG(inode->i_mode) || !fd) return; @@ -1532,9 +1528,7 @@ void pcc_file_release(struct inode *inode, struct file *file) pcci = ll_i2pcci(inode); LASSERT(pcci); path = &pcci->pcci_path; - dname = &path->dentry->d_name; - CDEBUG(D_CACHE, "releasing pcc file \"%.*s\"\n", dname->len, - dname->name); + CDEBUG(D_CACHE, "releasing pcc file \"%pd\"\n", path->dentry); pcc_inode_put(pcci); fput(pccf->pccf_file); pccf->pccf_file = NULL; @@ -2005,9 +1999,8 @@ static int pcc_inode_remove(struct inode *inode, struct dentry *pcc_dentry) rc = vfs_unlink(pcc_dentry->d_parent->d_inode, pcc_dentry, NULL); if (rc) - CWARN("%s: failed to unlink PCC file %.*s, rc = %d\n", - ll_i2sbi(inode)->ll_fsname, pcc_dentry->d_name.len, - pcc_dentry->d_name.name, rc); + CWARN("%s: failed to unlink PCC file %pd: rc = %d\n", + ll_i2sbi(inode)->ll_fsname, pcc_dentry, rc); return rc; } @@ -2239,9 +2232,8 @@ void pcc_create_attach_cleanup(struct super_block *sb, rc = vfs_unlink(pca->pca_dentry->d_parent->d_inode, pca->pca_dentry, NULL); if (rc) - CWARN("failed to unlink PCC file %.*s, rc = %d\n", - pca->pca_dentry->d_name.len, - pca->pca_dentry->d_name.name, rc); + CWARN("%s: failed to unlink PCC file %pd: rc = %d\n", + ll_s2sbi(sb)->ll_fsname, pca->pca_dentry, rc); /* ignore the unlink failure */ revert_creds(old_cred); dput(pca->pca_dentry); From patchwork Wed Jun 3 00:59:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584737 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB3651391 for ; Wed, 3 Jun 2020 01:00:27 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A3DDD2072F for ; Wed, 3 Jun 2020 01:00:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3DDD2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 886C621F98D; Tue, 2 Jun 2020 18:00:16 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 4FD3521F37D for ; Tue, 2 Jun 2020 18:00:07 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 6C50D60A; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 652702D1; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:45 -0400 Message-Id: <1591146001-27171-7-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 06/22] lnet: tidy lnet_discover and fix mem accounting bug. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown A recent patch introduce a memory accounting bug because "n_ids" can change between the ALLOC call and the FREE call. With this patch we fix that by ensuring n_ids doesn't change - the current change is not needed. Also: - discard 'max_intf' var. It is always exactly lnet_interfaces_max, so just use that directly. - only copy back the number of interfaces found - report the number of interfaces actually copied. - Move the copy_to_user until after all locks and references are dropped so there is no need to re-take any locks. WC-bug-id: https://jira.whamcloud.com/browse/LU-9679 Lustre-commit: 45722de60a8fb ("LU-9679 lnet: tidy lnet_discover and fix mem accounting bug.") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/38644 Reviewed-by: Shaun Tancheff Reviewed-by: Chris Horn Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 28 ++++++++++------------------ 1 file changed, 10 insertions(+), 18 deletions(-) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index e2c364b..629a597 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -4163,7 +4163,6 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, int cpt; int i; int rc; - int max_intf = lnet_interfaces_max; if (n_ids <= 0 || id.nid == LNET_NID_ANY) @@ -4172,11 +4171,11 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, if (id.pid == LNET_PID_ANY) id.pid = LNET_PID_LUSTRE; - /* if the user buffer has more space than the max_intf - * then only fill it up to max_intf + /* If the user buffer has more space than the lnet_interfaces_max, + * then only fill it up to lnet_interfaces_max. */ - if (n_ids > max_intf) - n_ids = max_intf; + if (n_ids > lnet_interfaces_max) + n_ids = lnet_interfaces_max; buf = kcalloc(n_ids, sizeof(*buf), GFP_KERNEL); if (!buf) @@ -4204,11 +4203,6 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, if (rc) goto out_decref; - /* Peer may have changed. */ - lp = lpni->lpni_peer_net->lpn_peer; - if (lp->lp_nnis < n_ids) - n_ids = lp->lp_nnis; - i = 0; p = NULL; while ((p = lnet_get_next_peer_ni_locked(lp, NULL, p)) != NULL) { @@ -4217,20 +4211,18 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, if (++i >= n_ids) break; } + rc = i; - lnet_net_unlock(cpt); - - rc = -EFAULT; - if (copy_to_user(ids, buf, n_ids * sizeof(*buf))) - goto out_relock; - rc = n_ids; -out_relock: - lnet_net_lock(cpt); out_decref: lnet_peer_ni_decref_locked(lpni); out: lnet_net_unlock(cpt); + + if (rc >= 0) + if (copy_to_user(ids, buf, rc * sizeof(*buf))) + rc = -EFAULT; kfree(buf); + return rc; } From patchwork Wed Jun 3 00:59:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584735 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 68D3F1392 for ; Wed, 3 Jun 2020 01:00:22 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51A5E2072F for ; Wed, 3 Jun 2020 01:00:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51A5E2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D548521F493; Tue, 2 Jun 2020 18:00:13 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id A760921F2E1 for ; Tue, 2 Jun 2020 18:00:06 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 6B8035E7; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6942F2C5; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:46 -0400 Message-Id: <1591146001-27171-8-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 07/22] lustre: llite: prevent MAX_DIO_SIZE 32-bit truncation X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Sebastien Buisson On 4kB PAGE_SIZE systems, kmalloc can allocate up to 4MB, which makes MAX_DIO_SIZE up to 682MB. This number can fit into 32 bits. But on 64kB PAGE_SIZE systems, kmalloc can allocate up to 512MB, which then makes MAX_DIO_SIZE up to 1365GB. This needs 64 bits to fit. Make sure that for every platform MAX_DIO_SIZE is not abusively truncated, by casting it to size_t. WC-bug-id: https://jira.whamcloud.com/browse/LU-13528 Lustre-commit: 8cfd5be8b04bd ("LU-13528 llite: prevent MAX_DIO_SIZE 32-bit truncation") Signed-off-by: Sebastien Buisson Reviewed-on: https://review.whamcloud.com/38526 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Signed-off-by: James Simmons --- fs/lustre/llite/rw26.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/lustre/llite/rw26.c b/fs/lustre/llite/rw26.c index 7abf3fc..5e7aa6e 100644 --- a/fs/lustre/llite/rw26.c +++ b/fs/lustre/llite/rw26.c @@ -276,7 +276,7 @@ struct ll_dio_pages { * up to 22MB for 128kB kmalloc and up to 682MB for 4MB kmalloc. */ #define MAX_DIO_SIZE ((KMALLOC_MAX_SIZE / sizeof(struct brw_page) * PAGE_SIZE) & \ - ~(DT_MAX_BRW_SIZE - 1)) + ~((size_t)DT_MAX_BRW_SIZE - 1)) static ssize_t ll_direct_IO(struct kiocb *iocb, struct iov_iter *iter) { From patchwork Wed Jun 3 00:59:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584741 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1D84912 for ; Wed, 3 Jun 2020 01:00:33 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA27A2072F for ; Wed, 3 Jun 2020 01:00:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA27A2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 917EA21FA46; Tue, 2 Jun 2020 18:00:19 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9052D21F4B7 for ; Tue, 2 Jun 2020 18:00:07 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 7132F6B5; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6C48F2C6; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:47 -0400 Message-Id: <1591146001-27171-9-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 08/22] lustre: llite: integrate statx() API with Lustre X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Qian Yingjin , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Qian Yingjin System call statx() interface can specify a bitmask to fetch specific attributes from a file (e.g. st_uid, st_gid, st_mode, and st_btime = file creation time), rather than fetching all of the normal stat() attributes (such as st_size and st_blocks). It also has a AT_STATX_DONT_SYNC mode which allows the kernel to return cached attributes without flushing all of the client data and fetching an accurate result from the server. The conditions for adding statx() API for Lustre are mature: 1. statx() is added to Linux 4.11+; 2. glibc supports statx() (glibc 2.28+ -> RHEL 8, Ubuntun 18.10+) 3. The support for stat(1) and ls(1) to use statx(3) to fetch only the required attributes has landed to the upstream GNU coreutils package. This patch integrates statx() API with Lustre so that we can take advantage of the efficiencies available: - Only fetch MDS attributes if STATX_SIZE, STATX_BLOCKS and STATX_MTIME are not requested, and avoid OSS glimpse RPCs completely; - Hook this into statahead to avoid async glimpse locks (AGL) if OST information not needed; - Enhance the MDS RPC interface to return the file creation time stored in both ldiskfs and ZFS already, and enable STATX_BTIME; - Better support with AT_STATX_DONT_SYNC mode. Return the "lazy" attributes or cached attributes (even stale) on a client if available without any RPCs to servers (MDS and OSS). - statx (lustre/test/statx): port coreutils ls/stat by using statx(3) system call if OS supported it. - Test scripts. Using statx() to verify btime attribute and the advantage described above. WC-bug-id: https://jira.whamcloud.com/browse/LU-10934 Lustre-commit: 3f7853b31ef6f5 ("LU-10934 llite: integrate statx() API with Lustre") Signed-off-by: Qian Yingjin Reviewed-on: https://review.whamcloud.com/36674 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Signed-off-by: James Simmons --- fs/lustre/llite/dcache.c | 4 +- fs/lustre/llite/file.c | 80 ++++++++++++++++++++++++++++++++++++---- fs/lustre/llite/llite_internal.h | 12 +++++- fs/lustre/llite/llite_lib.c | 9 +++++ fs/lustre/llite/namei.c | 2 +- fs/lustre/llite/pcc.c | 5 ++- fs/lustre/llite/pcc.h | 3 +- fs/lustre/llite/statahead.c | 59 +++++++++++++++++++++++++---- 8 files changed, 152 insertions(+), 22 deletions(-) diff --git a/fs/lustre/llite/dcache.c b/fs/lustre/llite/dcache.c index edcc1a7..e8b6fe8 100644 --- a/fs/lustre/llite/dcache.c +++ b/fs/lustre/llite/dcache.c @@ -270,7 +270,9 @@ static int ll_revalidate_dentry(struct dentry *dentry, if (lookup_flags & LOOKUP_RCU) return -ECHILD; - ll_statahead(dir, &dentry, !d_inode(dentry)); + if (dentry_may_statahead(dir, dentry)) + ll_revalidate_statahead(dir, &dentry, !d_inode(dentry)); + return 1; } diff --git a/fs/lustre/llite/file.c b/fs/lustre/llite/file.c index 52f0865..8264b86 100644 --- a/fs/lustre/llite/file.c +++ b/fs/lustre/llite/file.c @@ -4555,23 +4555,66 @@ static int ll_merge_md_attr(struct inode *inode) int ll_getattr(const struct path *path, struct kstat *stat, u32 request_mask, unsigned int flags) { - struct inode *inode = d_inode(path->dentry); + struct dentry *de = path->dentry; + struct inode *inode = d_inode(de); struct ll_sb_info *sbi = ll_i2sbi(inode); struct ll_inode_info *lli = ll_i2info(inode); + struct inode *dir = de->d_parent->d_inode; + bool need_glimpse = true; ktime_t kstart = ktime_get(); int rc; - rc = ll_inode_revalidate(path->dentry, IT_GETATTR); + /* The OST object(s) determine the file size, blocks and mtime. */ + if (!(request_mask & STATX_SIZE || request_mask & STATX_BLOCKS || + request_mask & STATX_MTIME)) + need_glimpse = false; + + if (dentry_may_statahead(dir, de)) + ll_start_statahead(dir, de, need_glimpse && + !(flags & AT_STATX_DONT_SYNC)); + + if (flags & AT_STATX_DONT_SYNC) { + rc = 0; + goto fill_attr; + } + + rc = ll_inode_revalidate(de, IT_GETATTR); if (rc < 0) return rc; if (S_ISREG(inode->i_mode)) { bool cached; - rc = pcc_inode_getattr(inode, &cached); + if (!need_glimpse) + goto fill_attr; + + rc = pcc_inode_getattr(inode, request_mask, flags, &cached); if (cached && rc < 0) return rc; + if (cached) + goto fill_attr; + + /* + * If the returned attr is masked with OBD_MD_FLSIZE & + * OBD_MD_FLBLOCKS & OBD_MD_FLMTIME, it means that the file size + * or blocks obtained from MDT is strictly correct, and the file + * is usually not being modified by clients, and the [a|m|c]time + * got from MDT is also strictly correct. + * Under this circumstance, it does not need to send glimpse + * RPCs to OSTs for file attributes such as the size and blocks. + */ + if (lli->lli_attr_valid & OBD_MD_FLSIZE && + lli->lli_attr_valid & OBD_MD_FLBLOCKS && + lli->lli_attr_valid & OBD_MD_FLMTIME) { + inode->i_mtime.tv_sec = lli->lli_mtime; + if (lli->lli_attr_valid & OBD_MD_FLATIME) + inode->i_atime.tv_sec = lli->lli_atime; + if (lli->lli_attr_valid & OBD_MD_FLCTIME) + inode->i_ctime.tv_sec = lli->lli_ctime; + goto fill_attr; + } + /* In case of restore, the MDT has the right size and has * already send it back without granting the layout lock, * inode is up-to-date so glimpse is useless. @@ -4579,8 +4622,7 @@ int ll_getattr(const struct path *path, struct kstat *stat, * restore the MDT holds the layout lock so the glimpse will * block up to the end of restore (getattr will block) */ - if (!cached && !test_bit(LLIF_FILE_RESTORING, - &lli->lli_flags)) { + if (!test_bit(LLIF_FILE_RESTORING, &lli->lli_flags)) { rc = ll_glimpse_size(inode); if (rc < 0) return rc; @@ -4593,11 +4635,15 @@ int ll_getattr(const struct path *path, struct kstat *stat, return rc; } - inode->i_atime.tv_sec = lli->lli_atime; - inode->i_mtime.tv_sec = lli->lli_mtime; - inode->i_ctime.tv_sec = lli->lli_ctime; + if (lli->lli_attr_valid & OBD_MD_FLATIME) + inode->i_atime.tv_sec = lli->lli_atime; + if (lli->lli_attr_valid & OBD_MD_FLMTIME) + inode->i_mtime.tv_sec = lli->lli_mtime; + if (lli->lli_attr_valid & OBD_MD_FLCTIME) + inode->i_ctime.tv_sec = lli->lli_ctime; } +fill_attr: OBD_FAIL_TIMEOUT(OBD_FAIL_GETATTR_DELAY, 30); stat->dev = inode->i_sb->s_dev; @@ -4631,6 +4677,24 @@ int ll_getattr(const struct path *path, struct kstat *stat, stat->size = i_size_read(inode); stat->blocks = inode->i_blocks; + if (flags & AT_STATX_DONT_SYNC) { + if (stat->size == 0 && + lli->lli_attr_valid & OBD_MD_FLLAZYSIZE) + stat->size = lli->lli_lazysize; + if (stat->blocks == 0 && + lli->lli_attr_valid & OBD_MD_FLLAZYBLOCKS) + stat->blocks = lli->lli_lazyblocks; + } + + if (lli->lli_attr_valid & OBD_MD_FLBTIME) { + stat->result_mask |= STATX_BTIME; + stat->btime.tv_sec = lli->lli_btime; + } + + stat->attributes_mask = STATX_ATTR_IMMUTABLE | STATX_ATTR_APPEND; + stat->attributes |= ll_inode_to_ext_flags(inode->i_flags); + stat->result_mask &= request_mask; + ll_stats_ops_tally(sbi, LPROC_LL_GETATTR, ktime_us_delta(ktime_get(), kstart)); diff --git a/fs/lustre/llite/llite_internal.h b/fs/lustre/llite/llite_internal.h index 2544a40..be3a0b0 100644 --- a/fs/lustre/llite/llite_internal.h +++ b/fs/lustre/llite/llite_internal.h @@ -143,6 +143,7 @@ struct ll_inode_info { s64 lli_atime; s64 lli_mtime; s64 lli_ctime; + s64 lli_btime; spinlock_t lli_agl_lock; /* Try to make the d::member and f::member are aligned. Before using @@ -233,6 +234,10 @@ struct ll_inode_info { struct mutex lli_group_mutex; u64 lli_group_users; unsigned long lli_group_gid; + + u64 lli_attr_valid; + u64 lli_lazysize; + u64 lli_lazyblocks; }; }; @@ -1349,7 +1354,9 @@ struct ll_statahead_info { atomic_t sai_cache_count; /* entry count in cache */ }; -int ll_statahead(struct inode *dir, struct dentry **dentry, bool unplug); +int ll_revalidate_statahead(struct inode *dir, struct dentry **dentry, + bool unplug); +int ll_start_statahead(struct inode *dir, struct dentry *dentry, bool agl); void ll_authorize_statahead(struct inode *dir, void *key); void ll_deauthorize_statahead(struct inode *dir, void *key); @@ -1433,7 +1440,8 @@ static inline int ll_glimpse_size(struct inode *inode) * 'lld_sa_generation == lli->lli_sa_generation'. */ ldd = ll_d2d(dentry); - if (ldd->lld_sa_generation == lli->lli_sa_generation) + if (lli->lli_sa_generation && + ldd->lld_sa_generation == lli->lli_sa_generation) return false; return true; diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index da7604f..70e839b 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -2049,6 +2049,9 @@ int ll_update_inode(struct inode *inode, struct lustre_md *md) lli->lli_ctime = body->mbo_ctime; } + if (body->mbo_valid & OBD_MD_FLBTIME) + lli->lli_btime = body->mbo_btime; + /* Clear i_flags to remove S_NOSEC before permissions are updated */ if (body->mbo_valid & OBD_MD_FLFLAGS) ll_update_inode_flags(inode, body->mbo_flags); @@ -2085,6 +2088,7 @@ int ll_update_inode(struct inode *inode, struct lustre_md *md) LASSERT(fid_seq(&lli->lli_fid) != 0); + lli->lli_attr_valid = body->mbo_valid; if (body->mbo_valid & OBD_MD_FLSIZE) { i_size_write(inode, body->mbo_size); @@ -2094,6 +2098,11 @@ int ll_update_inode(struct inode *inode, struct lustre_md *md) if (body->mbo_valid & OBD_MD_FLBLOCKS) inode->i_blocks = body->mbo_blocks; + } else { + if (body->mbo_valid & OBD_MD_FLLAZYSIZE) + lli->lli_lazysize = body->mbo_size; + if (body->mbo_valid & OBD_MD_FLLAZYBLOCKS) + lli->lli_lazyblocks = body->mbo_blocks; } if (body->mbo_valid & OBD_MD_TSTATE) { diff --git a/fs/lustre/llite/namei.c b/fs/lustre/llite/namei.c index 16c3bc5..aa2dd13 100644 --- a/fs/lustre/llite/namei.c +++ b/fs/lustre/llite/namei.c @@ -751,7 +751,7 @@ static struct dentry *ll_lookup_it(struct inode *parent, struct dentry *dentry, it = &lookup_it; if (it->it_op == IT_GETATTR && dentry_may_statahead(parent, dentry)) { - rc = ll_statahead(parent, &dentry, 0); + rc = ll_revalidate_statahead(parent, &dentry, 0); if (rc == 1) { if (dentry == save) retval = NULL; diff --git a/fs/lustre/llite/pcc.c b/fs/lustre/llite/pcc.c index 2adde68..5a4bb33 100644 --- a/fs/lustre/llite/pcc.c +++ b/fs/lustre/llite/pcc.c @@ -1673,7 +1673,8 @@ int pcc_inode_setattr(struct inode *inode, struct iattr *attr, return rc; } -int pcc_inode_getattr(struct inode *inode, bool *cached) +int pcc_inode_getattr(struct inode *inode, u32 request_mask, + unsigned int flags, bool *cached) { struct ll_inode_info *lli = ll_i2info(inode); const struct cred *old_cred; @@ -1694,7 +1695,7 @@ int pcc_inode_getattr(struct inode *inode, bool *cached) old_cred = override_creds(pcc_super_cred(inode->i_sb)); rc = vfs_getattr(&ll_i2pcci(inode)->pcci_path, &stat, - STATX_BASIC_STATS, AT_STATX_SYNC_AS_STAT); + request_mask, flags); revert_creds(old_cred); if (rc) goto out; diff --git a/fs/lustre/llite/pcc.h b/fs/lustre/llite/pcc.h index 60f9bea..b13f9da8 100644 --- a/fs/lustre/llite/pcc.h +++ b/fs/lustre/llite/pcc.h @@ -238,7 +238,8 @@ ssize_t pcc_file_read_iter(struct kiocb *iocb, struct iov_iter *iter, bool *cached); ssize_t pcc_file_write_iter(struct kiocb *iocb, struct iov_iter *iter, bool *cached); -int pcc_inode_getattr(struct inode *inode, bool *cached); +int pcc_inode_getattr(struct inode *inode, u32 request_mask, + unsigned int flags, bool *cached); int pcc_inode_setattr(struct inode *inode, struct iattr *attr, bool *cached); ssize_t pcc_file_splice_read(struct file *in_file, loff_t *ppos, struct pipe_inode_info *pipe, size_t count, diff --git a/fs/lustre/llite/statahead.c b/fs/lustre/llite/statahead.c index 04e013f..fb25520 100644 --- a/fs/lustre/llite/statahead.c +++ b/fs/lustre/llite/statahead.c @@ -965,10 +965,12 @@ static void ll_start_agl(struct dentry *parent, struct ll_statahead_info *sai) plli->lli_opendir_pid); if (IS_ERR(task)) { CERROR("can't start ll_agl thread, rc: %ld\n", PTR_ERR(task)); + sai->sai_agl_valid = 0; return; } sai->sai_agl_task = task; + LASSERT(sai->sai_agl_valid == 1); atomic_inc(&ll_i2sbi(d_inode(parent))->ll_agl_total); spin_lock(&plli->lli_agl_lock); sai->sai_agl_valid = 1; @@ -1521,6 +1523,7 @@ static int revalidate_statahead_dentry(struct inode *dir, * @dir: parent directory * @dentry: dentry that triggers statahead, normally the first * dirent under @dir + * @agl indicate whether AGL is needed * * Returns: -EAGAIN on success, because when this function is * called, it's already in lookup call, so client should @@ -1529,7 +1532,8 @@ static int revalidate_statahead_dentry(struct inode *dir, * * negative number upon error */ -static int start_statahead_thread(struct inode *dir, struct dentry *dentry) +static int start_statahead_thread(struct inode *dir, struct dentry *dentry, + bool agl) { struct ll_inode_info *lli = ll_i2info(dir); struct ll_statahead_info *sai = NULL; @@ -1562,6 +1566,8 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) } sai->sai_ls_all = (first == LS_FIRST_DOT_DE); + sai->sai_agl_valid = agl; + /* * if current lli_opendir_key was deauthorized, or dir re-opened by * another process, don't start statahead, otherwise the newly spawned @@ -1592,7 +1598,7 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) goto out; } - if (ll_i2sbi(parent->d_inode)->ll_flags & LL_SBI_AGL_ENABLED) + if (ll_i2sbi(parent->d_inode)->ll_flags & LL_SBI_AGL_ENABLED && agl) ll_start_agl(parent, sai); atomic_inc(&ll_i2sbi(parent->d_inode)->ll_sa_total); @@ -1615,6 +1621,7 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) if (lli->lli_opendir_pid == current->pid) lli->lli_sa_enabled = 0; spin_unlock(&lli->lli_sa_lock); + if (sai) ll_sai_free(sai); if (first != LS_NOT_FIRST_DE) @@ -1623,11 +1630,50 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) return rc; } +/* + * Check whether statahead for @dir was started. + */ +static inline bool ll_statahead_started(struct inode *dir, bool agl) +{ + struct ll_inode_info *lli = ll_i2info(dir); + struct ll_statahead_info *sai; + + spin_lock(&lli->lli_sa_lock); + sai = lli->lli_sai; + if (sai && sai->sai_agl_valid != agl) + CDEBUG(D_READA, + "%s: Statahead AGL hint changed from %d to %d\n", + ll_i2sbi(dir)->ll_fsname, sai->sai_agl_valid, agl); + spin_unlock(&lli->lli_sa_lock); + + return !!sai; +} + /** * statahead entry function, this is called when client getattr on a file, it * will start statahead thread if this is the first dir entry, else revalidate * dentry from statahead cache. * + * @dir parent directory + * @dentryp dentry to getattr + * @agl whether start the agl thread + * + * Return: 1 on success + * 0 revalidation from statahead cache failed, caller needs + * to getattr from server directly + * negative number on error, caller often ignores this and + * then getattr from server + */ +int ll_start_statahead(struct inode *dir, struct dentry *dentry, bool agl) +{ + if (!ll_statahead_started(dir, agl)) + return start_statahead_thread(dir, dentry, agl); + return 0; +} + +/** + * revalidate dentry from statahead cache. + * * @dir: parent directory * @dentryp: dentry to getattr * @unplug: unplug statahead window only (normally for negative @@ -1638,19 +1684,18 @@ static int start_statahead_thread(struct inode *dir, struct dentry *dentry) * negative number on error, caller often ignores this and * then getattr from server */ -int ll_statahead(struct inode *dir, struct dentry **dentryp, bool unplug) +int ll_revalidate_statahead(struct inode *dir, struct dentry **dentryp, + bool unplug) { struct ll_statahead_info *sai; + int rc = 0; sai = ll_sai_get(dir); if (sai) { - int rc; - rc = revalidate_statahead_dentry(dir, sai, dentryp, unplug); CDEBUG(D_READA, "revalidate statahead %pd: %d.\n", *dentryp, rc); ll_sai_put(sai); - return rc; } - return start_statahead_thread(dir, *dentryp); + return rc; } From patchwork Wed Jun 3 00:59:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584749 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C4A71391 for ; Wed, 3 Jun 2020 01:00:47 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8C7B2072F for ; Wed, 3 Jun 2020 01:00:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8C7B2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9392A21FB87; Tue, 2 Jun 2020 18:00:26 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E58AB21F563 for ; Tue, 2 Jun 2020 18:00:07 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 732DD6B6; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 6F5602D2; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:48 -0400 Message-Id: <1591146001-27171-10-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 09/22] lustre: ldlm: no current source if lu_ref_del not in same tsk X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Bruno Faccini Running with USE_LU_REF ("configure --enable-lu_ref") configured triggers a LBUG (because "ref->lf_failed > 0" condition false) due to to using "current" as the lu_ref source, but in some cases lu_ref_del() occurs within a different task context. To avoid this, lu_ref source is changed to ldlm_lock address by this patch. WC-bug-id: https://jira.whamcloud.com/browse/LU-13262 Lustre-commit: 419325a2c9a29e ("LU-13262 ldlm: no current source if lu_ref_del not in same tsk") Signed-off-by: Bruno Faccini Reviewed-on: https://review.whamcloud.com/37624 Reviewed-by: Andreas Dilger Reviewed-by: Mike Pershin Signed-off-by: James Simmons --- fs/lustre/include/lustre_dlm.h | 2 +- fs/lustre/ldlm/ldlm_lock.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/lustre/include/lustre_dlm.h b/fs/lustre/include/lustre_dlm.h index 74c380e..28e766b 100644 --- a/fs/lustre/include/lustre_dlm.h +++ b/fs/lustre/include/lustre_dlm.h @@ -1085,7 +1085,7 @@ static inline struct ldlm_lock *ldlm_handle2lock(const struct lustre_handle *h) } #define LDLM_LOCK_REF_DEL(lock) \ - lu_ref_del(&lock->l_reference, "handle", current) + lu_ref_del(&lock->l_reference, "handle", lock) static inline struct ldlm_lock * ldlm_handle2lock_long(const struct lustre_handle *h, u64 flags) diff --git a/fs/lustre/ldlm/ldlm_lock.c b/fs/lustre/ldlm/ldlm_lock.c index 3e71e4e..7fda7b8 100644 --- a/fs/lustre/ldlm/ldlm_lock.c +++ b/fs/lustre/ldlm/ldlm_lock.c @@ -528,7 +528,7 @@ struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle, * destroyed after we did handle2object on it */ if (flags == 0 && !ldlm_is_destroyed(lock)) { - lu_ref_add_atomic(&lock->l_reference, "handle", current); + lu_ref_add_atomic(&lock->l_reference, "handle", lock); return lock; } @@ -536,7 +536,7 @@ struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle, LASSERT(lock->l_resource); - lu_ref_add_atomic(&lock->l_reference, "handle", current); + lu_ref_add_atomic(&lock->l_reference, "handle", lock); if (unlikely(ldlm_is_destroyed(lock))) { unlock_res_and_lock(lock); CDEBUG(D_INFO, "lock already destroyed: lock %p\n", lock); From patchwork Wed Jun 3 00:59:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584753 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 67C7F912 for ; Wed, 3 Jun 2020 01:00:53 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 506982072F for ; Wed, 3 Jun 2020 01:00:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 506982072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 95D0321FBDB; Tue, 2 Jun 2020 18:00:29 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 31BF221F563 for ; Tue, 2 Jun 2020 18:00:08 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 742086B7; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7279F2CD; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:49 -0400 Message-Id: <1591146001-27171-11-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 10/22] lnet: always pass struct lnet_md by reference. X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Mr NeilBrown Both LNetMDAttach and LNetMDBind expected a struct lnet_md to be passed by value. This requires copying the data structure onto the stack, which is a waste of stack space and brings no value. So change them to expect a reference, and declare it 'const' to be sure it doesn't get changed. WC-bug-id: https://jira.whamcloud.com/browse/LU-13004 Lustre-commit: fb3ed0fe68e32 ("LU-13004 lnet: always pass struct lnet_md by reference.") Signed-off-by: Mr NeilBrown Reviewed-on: https://review.whamcloud.com/37853 Reviewed-by: Serguei Smirnov Reviewed-by: Chris Horn Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/ptlrpc/niobuf.c | 8 ++++---- include/linux/lnet/api.h | 4 ++-- net/lnet/lnet/api-ni.c | 6 +++--- net/lnet/lnet/lib-md.c | 24 ++++++++++++------------ net/lnet/lnet/lib-move.c | 2 +- net/lnet/lnet/peer.c | 2 +- net/lnet/selftest/rpc.c | 4 ++-- 7 files changed, 25 insertions(+), 25 deletions(-) diff --git a/fs/lustre/ptlrpc/niobuf.c b/fs/lustre/ptlrpc/niobuf.c index c59fc7f..d62629a 100644 --- a/fs/lustre/ptlrpc/niobuf.c +++ b/fs/lustre/ptlrpc/niobuf.c @@ -74,7 +74,7 @@ static int ptl_send_buf(struct lnet_handle_md *mdh, void *base, int len, /* don't ask for the ack to simulate failing client */ ack = LNET_NOACK_REQ; - rc = LNetMDBind(md, LNET_UNLINK, mdh); + rc = LNetMDBind(&md, LNET_UNLINK, mdh); if (unlikely(rc != 0)) { CERROR("LNetMDBind failed: %d\n", rc); LASSERT(rc == -ENOMEM); @@ -197,7 +197,7 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) percpu_ref_get(&ptlrpc_pending); /* About to let the network at it... */ - rc = LNetMDAttach(me, md, LNET_UNLINK, + rc = LNetMDAttach(me, &md, LNET_UNLINK, &desc->bd_mds[posted_md]); if (rc != 0) { CERROR("%s: LNetMDAttach failed x%llu/%d: rc = %d\n", @@ -656,7 +656,7 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) /* We must see the unlink callback to set rq_reply_unlinked, * so we can't auto-unlink */ - rc = LNetMDAttach(reply_me, reply_md, LNET_RETAIN, + rc = LNetMDAttach(reply_me, &reply_md, LNET_RETAIN, &request->rq_reply_md_h); if (rc != 0) { CERROR("LNetMDAttach failed: %d\n", rc); @@ -786,7 +786,7 @@ int ptlrpc_register_rqbd(struct ptlrpc_request_buffer_desc *rqbd) md.user_ptr = &rqbd->rqbd_cbid; md.handler = ptlrpc_handler; - rc = LNetMDAttach(me, md, LNET_UNLINK, &rqbd->rqbd_md_h); + rc = LNetMDAttach(me, &md, LNET_UNLINK, &rqbd->rqbd_md_h); if (rc == 0) { percpu_ref_get(&ptlrpc_pending); return 0; diff --git a/include/linux/lnet/api.h b/include/linux/lnet/api.h index bdfc8d3..24115eb 100644 --- a/include/linux/lnet/api.h +++ b/include/linux/lnet/api.h @@ -119,11 +119,11 @@ struct lnet_me * * @{ */ int LNetMDAttach(struct lnet_me *current_in, - struct lnet_md md_in, + const struct lnet_md *md_in, enum lnet_unlink unlink_in, struct lnet_handle_md *md_handle_out); -int LNetMDBind(struct lnet_md md_in, +int LNetMDBind(const struct lnet_md *md_in, enum lnet_unlink unlink_in, struct lnet_handle_md *md_handle_out); diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 629a597..6f19e63 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -1646,7 +1646,7 @@ struct lnet_ping_buffer * md.handler = the_lnet.ln_ping_target_handler; md.user_ptr = *ppbuf; - rc = LNetMDAttach(me, md, LNET_RETAIN, ping_mdh); + rc = LNetMDAttach(me, &md, LNET_RETAIN, ping_mdh); if (rc) { CERROR("Can't attach ping target MD: %d\n", rc); goto fail_unlink_ping_me; @@ -1856,7 +1856,7 @@ int lnet_push_target_post(struct lnet_ping_buffer *pbuf, md.user_ptr = pbuf; md.handler = the_lnet.ln_push_target_handler; - rc = LNetMDAttach(me, md, LNET_UNLINK, mdhp); + rc = LNetMDAttach(me, &md, LNET_UNLINK, mdhp); if (rc) { CERROR("Can't attach push MD: %d\n", rc); LNetMEUnlink(me); @@ -4066,7 +4066,7 @@ static int lnet_ping(struct lnet_process_id id, signed long timeout, init_completion(&pd.completion); - rc = LNetMDBind(md, LNET_UNLINK, &pd.mdh); + rc = LNetMDBind(&md, LNET_UNLINK, &pd.mdh); if (rc) { CERROR("Can't bind MD: %d\n", rc); goto fail_ping_buffer_decref; diff --git a/net/lnet/lnet/lib-md.c b/net/lnet/lnet/lib-md.c index cfa285a..e80dc6f 100644 --- a/net/lnet/lnet/lib-md.c +++ b/net/lnet/lnet/lib-md.c @@ -124,7 +124,7 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) } static struct lnet_libmd * -lnet_md_build(struct lnet_md *umd, int unlink) +lnet_md_build(const struct lnet_md *umd, int unlink) { int i; unsigned int niov; @@ -269,7 +269,7 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) } static int -lnet_md_validate(struct lnet_md *umd) +lnet_md_validate(const struct lnet_md *umd) { if (!umd->start && umd->length) { CERROR("MD start pointer can not be NULL with length %u\n", @@ -314,7 +314,7 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) * a MD. */ int -LNetMDAttach(struct lnet_me *me, struct lnet_md umd, +LNetMDAttach(struct lnet_me *me, const struct lnet_md *umd, enum lnet_unlink unlink, struct lnet_handle_md *handle) { LIST_HEAD(matches); @@ -325,15 +325,15 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) LASSERT(the_lnet.ln_refcount > 0); - if (lnet_md_validate(&umd)) + if (lnet_md_validate(umd)) return -EINVAL; - if (!(umd.options & (LNET_MD_OP_GET | LNET_MD_OP_PUT))) { + if (!(umd->options & (LNET_MD_OP_GET | LNET_MD_OP_PUT))) { CERROR("Invalid option: no MD_OP set\n"); return -EINVAL; } - md = lnet_md_build(&umd, unlink); + md = lnet_md_build(umd, unlink); if (IS_ERR(md)) return PTR_ERR(md); @@ -344,7 +344,7 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) if (me->me_md) rc = -EBUSY; else - rc = lnet_md_link(md, umd.handler, cpt); + rc = lnet_md_link(md, umd->handler, cpt); if (rc) goto out_unlock; @@ -388,7 +388,7 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) * calling LNetInvalidateHandle() on it. */ int -LNetMDBind(struct lnet_md umd, enum lnet_unlink unlink, +LNetMDBind(const struct lnet_md *umd, enum lnet_unlink unlink, struct lnet_handle_md *handle) { struct lnet_libmd *md; @@ -397,15 +397,15 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) LASSERT(the_lnet.ln_refcount > 0); - if (lnet_md_validate(&umd)) + if (lnet_md_validate(umd)) return -EINVAL; - if ((umd.options & (LNET_MD_OP_GET | LNET_MD_OP_PUT))) { + if ((umd->options & (LNET_MD_OP_GET | LNET_MD_OP_PUT))) { CERROR("Invalid option: GET|PUT illegal on active MDs\n"); return -EINVAL; } - md = lnet_md_build(&umd, unlink); + md = lnet_md_build(umd, unlink); if (IS_ERR(md)) return PTR_ERR(md); @@ -418,7 +418,7 @@ int lnet_cpt_of_md(struct lnet_libmd *md, unsigned int offset) cpt = lnet_res_lock_current(); - rc = lnet_md_link(md, umd.handler, cpt); + rc = lnet_md_link(md, umd->handler, cpt); if (rc) goto out_unlock; diff --git a/net/lnet/lnet/lib-move.c b/net/lnet/lnet/lib-move.c index 51b8191..2f3ef8c 100644 --- a/net/lnet/lnet/lib-move.c +++ b/net/lnet/lnet/lib-move.c @@ -3305,7 +3305,7 @@ struct lnet_mt_event_info { md.user_ptr = user_data; md.handler = handler; - rc = LNetMDBind(md, LNET_UNLINK, mdh); + rc = LNetMDBind(&md, LNET_UNLINK, mdh); if (rc) { lnet_ping_buffer_decref(pbuf); CERROR("Can't bind MD: %d\n", rc); diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index ae70033..b2065bd 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -3073,7 +3073,7 @@ static int lnet_peer_send_push(struct lnet_peer *lp) md.handler = the_lnet.ln_dc_handler; md.user_ptr = lp; - rc = LNetMDBind(md, LNET_UNLINK, &lp->lp_push_mdh); + rc = LNetMDBind(&md, LNET_UNLINK, &lp->lp_push_mdh); if (rc) { lnet_ping_buffer_decref(pbuf); CERROR("Can't bind push source MD: %d\n", rc); diff --git a/net/lnet/selftest/rpc.c b/net/lnet/selftest/rpc.c index e8667b1..799ad99 100644 --- a/net/lnet/selftest/rpc.c +++ b/net/lnet/selftest/rpc.c @@ -378,7 +378,7 @@ struct srpc_bulk * md.options = options; md.handler = srpc_data.rpc_lnet_handler; - rc = LNetMDAttach(me, md, LNET_UNLINK, mdh); + rc = LNetMDAttach(me, &md, LNET_UNLINK, mdh); if (rc) { CERROR("LNetMDAttach failed: %d\n", rc); LASSERT(rc == -ENOMEM); @@ -409,7 +409,7 @@ struct srpc_bulk * md.threshold = options & LNET_MD_OP_GET ? 2 : 1; md.options = options & ~(LNET_MD_OP_PUT | LNET_MD_OP_GET); - rc = LNetMDBind(md, LNET_UNLINK, mdh); + rc = LNetMDBind(&md, LNET_UNLINK, mdh); if (rc) { CERROR("LNetMDBind failed: %d\n", rc); LASSERT(rc == -ENOMEM); From patchwork Wed Jun 3 00:59:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584739 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6385912 for ; Wed, 3 Jun 2020 01:00:30 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9EA002072F for ; Wed, 3 Jun 2020 01:00:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9EA002072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3C3ED21F6CE; Tue, 2 Jun 2020 18:00:18 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8793021F563 for ; Tue, 2 Jun 2020 18:00:08 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 77A606B8; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7576F2CE; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:50 -0400 Message-Id: <1591146001-27171-12-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 11/22] lustre: llite: fix read if readahead window smaller than rpc size X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wang Shilong , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Wang Shilong Readahead always try to align readahead with RPC size, but this could introduce a problem if readahead window is smaller than RPC size. With current codes, it will fallback a lot of 4k read because RPC aligned window start plus window pages will be behind of current read. Fix this to align with readahead window rather than RPC size in this case. WC-bug-id: https://jira.whamcloud.com/browse/LU-13412 Lustre-commit: 33d08805f27bb ("LU-13412 llite: fix read if readahead window smaller than rpc size") Signed-off-by: Wang Shilong Reviewed-on: https://review.whamcloud.com/38132 Reviewed-by: Andreas Dilger Reviewed-by: Gu Zheng Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- fs/lustre/llite/rw.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fs/lustre/llite/rw.c b/fs/lustre/llite/rw.c index fd0bed6..ff8f3c6 100644 --- a/fs/lustre/llite/rw.c +++ b/fs/lustre/llite/rw.c @@ -350,7 +350,11 @@ static unsigned long ria_page_count(struct ra_io_arg *ria) static pgoff_t ras_align(struct ll_readahead_state *ras, pgoff_t index) { - return index - (index % ras->ras_rpc_pages); + unsigned opt_size = min(ras->ras_window_pages, ras->ras_rpc_pages); + + if (opt_size == 0) + opt_size = 1; + return index - (index % opt_size); } /* Check whether the index is in the defined ra-window */ From patchwork Wed Jun 3 00:59:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584757 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98EF1912 for ; Wed, 3 Jun 2020 01:00:59 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 812DF2072F for ; Wed, 3 Jun 2020 01:00:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 812DF2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9AEDD21FC52; Tue, 2 Jun 2020 18:00:32 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id CA32E21F563 for ; Tue, 2 Jun 2020 18:00:08 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 797F36B9; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 788232C5; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:51 -0400 Message-Id: <1591146001-27171-13-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 12/22] lustre: obdclass: bind zombie export cleanup workqueue X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" Lustre uses a workqueue to clear out stale exports. Bind this workqueue to the cores used by Lustre defined by the CPT setup. Move the code handling workqueue binding to libcfs so it can be used by everyone. WC-bug-id: https://jira.whamcloud.com/browse/LU-13258 Lustre-commit: 76b602c2bfe9d ("LU-13258 obdclass: bind zombie export cleanup workqueue") Signed-off-by: James Simmons Reviewed-on: https://review.whamcloud.com/38212 Reviewed-by: Shaun Tancheff Reviewed-by: Wang Shilong Reviewed-by: Oleg Drokin --- fs/lustre/llite/llite_lib.c | 21 +++++---------------- fs/lustre/obdclass/genops.c | 9 +++++---- fs/lustre/ptlrpc/pinger.c | 19 ++++--------------- include/linux/libcfs/libcfs_cpu.h | 24 ++++++++++++++++++++++++ net/lnet/libcfs/libcfs_cpu.c | 2 +- 5 files changed, 39 insertions(+), 36 deletions(-) diff --git a/fs/lustre/llite/llite_lib.c b/fs/lustre/llite/llite_lib.c index 70e839b..05d949b 100644 --- a/fs/lustre/llite/llite_lib.c +++ b/fs/lustre/llite/llite_lib.c @@ -81,8 +81,6 @@ static inline unsigned int ll_get_ra_async_max_active(void) static struct ll_sb_info *ll_init_sbi(void) { - struct workqueue_attrs attrs = { }; - cpumask_var_t *mask; struct ll_sb_info *sbi = NULL; unsigned long pages; unsigned long lru_page_max; @@ -111,23 +109,14 @@ static struct ll_sb_info *ll_init_sbi(void) sbi->ll_ra_info.ra_async_max_active = ll_get_ra_async_max_active(); sbi->ll_ra_info.ll_readahead_wq = - alloc_workqueue("ll-readahead-wq", WQ_UNBOUND, - sbi->ll_ra_info.ra_async_max_active); - if (!sbi->ll_ra_info.ll_readahead_wq) { - rc = -ENOMEM; + cfs_cpt_bind_workqueue("ll-readahead-wq", cfs_cpt_tab, + 0, CFS_CPT_ANY, + sbi->ll_ra_info.ra_async_max_active); + if (IS_ERR(sbi->ll_ra_info.ll_readahead_wq)) { + rc = PTR_ERR(sbi->ll_ra_info.ll_readahead_wq); goto out_pcc; } - mask = cfs_cpt_cpumask(cfs_cpt_tab, CFS_CPT_ANY); - if (mask && alloc_cpumask_var(&attrs.cpumask, GFP_KERNEL)) { - cpumask_copy(attrs.cpumask, *mask); - cpus_read_lock(); - apply_workqueue_attrs(sbi->ll_ra_info.ll_readahead_wq, - &attrs); - cpus_read_unlock(); - free_cpumask_var(attrs.cpumask); - } - sbi->ll_cache = cl_cache_init(lru_page_max); if (!sbi->ll_cache) { rc = -ENOMEM; diff --git a/fs/lustre/obdclass/genops.c b/fs/lustre/obdclass/genops.c index e4ad53c..607f0d6 100644 --- a/fs/lustre/obdclass/genops.c +++ b/fs/lustre/obdclass/genops.c @@ -37,6 +37,7 @@ */ #define DEBUG_SUBSYSTEM S_CLASS +#include #include #include #include @@ -1207,11 +1208,11 @@ void obd_zombie_barrier(void) */ int obd_zombie_impexp_init(void) { - zombie_wq = alloc_workqueue("obd_zombid", 0, 0); - if (!zombie_wq) - return -ENOMEM; + zombie_wq = cfs_cpt_bind_workqueue("obd_zombid", cfs_cpt_tab, + 0, CFS_CPT_ANY, + cfs_cpt_number(cfs_cpt_tab)); - return 0; + return IS_ERR(zombie_wq) ? PTR_ERR(zombie_wq) : 0; } /** diff --git a/fs/lustre/ptlrpc/pinger.c b/fs/lustre/ptlrpc/pinger.c index 63bd132..ec4c51a 100644 --- a/fs/lustre/ptlrpc/pinger.c +++ b/fs/lustre/ptlrpc/pinger.c @@ -298,25 +298,14 @@ static void ptlrpc_pinger_main(struct work_struct *ws) int ptlrpc_start_pinger(void) { #ifdef CONFIG_LUSTRE_FS_PINGER - struct workqueue_attrs attrs = { }; - cpumask_var_t *mask; - if (pinger_wq) return -EALREADY; - pinger_wq = alloc_workqueue("ptlrpc_pinger", WQ_UNBOUND, 1); - if (!pinger_wq) { + pinger_wq = cfs_cpt_bind_workqueue("ptlrpc_pinger", cfs_cpt_tab, + 0, CFS_CPT_ANY, 1); + if (IS_ERR(pinger_wq)) { CERROR("cannot start pinger workqueue\n"); - return -ENOMEM; - } - - mask = cfs_cpt_cpumask(cfs_cpt_tab, CFS_CPT_ANY); - if (mask && alloc_cpumask_var(&attrs.cpumask, GFP_KERNEL)) { - cpumask_copy(attrs.cpumask, *mask); - cpus_read_lock(); - apply_workqueue_attrs(pinger_wq, &attrs); - cpus_read_unlock(); - free_cpumask_var(attrs.cpumask); + return PTR_ERR(pinger_wq); } queue_delayed_work(pinger_wq, &ping_work, 0); diff --git a/include/linux/libcfs/libcfs_cpu.h b/include/linux/libcfs/libcfs_cpu.h index 4b0604a..310b25c 100644 --- a/include/linux/libcfs/libcfs_cpu.h +++ b/include/linux/libcfs/libcfs_cpu.h @@ -293,6 +293,30 @@ static inline void cfs_cpu_fini(void) #endif /* CONFIG_SMP */ +static inline +struct workqueue_struct *cfs_cpt_bind_workqueue(const char *wq_name, + struct cfs_cpt_table *tbl, + int flags, int cpt, int nthrs) +{ + cpumask_var_t *mask = cfs_cpt_cpumask(tbl, cpt); + struct workqueue_attrs attrs = { }; + struct workqueue_struct *wq; + + wq = alloc_workqueue(wq_name, WQ_UNBOUND | flags, nthrs); + if (!wq) + return ERR_PTR(-ENOMEM); + + if (mask && alloc_cpumask_var(&attrs.cpumask, GFP_KERNEL)) { + cpumask_copy(attrs.cpumask, *mask); + cpus_read_lock(); + apply_workqueue_attrs(wq, &attrs); + cpus_read_unlock(); + free_cpumask_var(attrs.cpumask); + } + + return wq; +} + /* * allocate per-cpu-partition data, returned value is an array of pointers, * variable can be indexed by CPU ID. diff --git a/net/lnet/libcfs/libcfs_cpu.c b/net/lnet/libcfs/libcfs_cpu.c index 8969e1f..8e4fdb1 100644 --- a/net/lnet/libcfs/libcfs_cpu.c +++ b/net/lnet/libcfs/libcfs_cpu.c @@ -36,9 +36,9 @@ #include #include -#include #include #include +#include /** virtual processing unit */ struct cfs_cpu_partition { From patchwork Wed Jun 3 00:59:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584743 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8E6FA912 for ; Wed, 3 Jun 2020 01:00:37 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 76FF72072F for ; Wed, 3 Jun 2020 01:00:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76FF72072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 909B221FA82; Tue, 2 Jun 2020 18:00:21 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2D49D21F563 for ; Tue, 2 Jun 2020 18:00:09 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 7D4566BA; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7B84B2D0; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:52 -0400 Message-Id: <1591146001-27171-14-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 13/22] lnet: handle discovery off properly X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata Peers need to only be updated when discovery is toggled from on to off. This way the peers don't attempt to send to a non-primary NID of the node. However, when discovery is toggled from off to on, the peer will attempt rediscovery and the peer information will eventually consolidate. In order to properly delete the peer only when it makes sense we have to differentiate between the case when we get the initial message and when we get a push for an already discovered peer. We only want to delete our local representation if the peer is one we have already had in our records. WC-bug-id: https://jira.whamcloud.com/browse/LU-13478 Lustre-commit: adae4295b62b1 ("LU-13478 lnet: handle discovery off properly") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/38321 Reviewed-by: Chris Horn Reviewed-by: Serguei Smirnov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/api-ni.c | 26 +++++++------------------- net/lnet/lnet/peer.c | 12 ++++++++---- 2 files changed, 15 insertions(+), 23 deletions(-) diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index 6f19e63..a966e64 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -270,7 +270,7 @@ static int lnet_discover(struct lnet_process_id id, u32 force, discovery_set(const char *val, const struct kernel_param *kp) { int rc; - unsigned int *discovery = (unsigned int *)kp->arg; + unsigned int *discovery_off = (unsigned int *)kp->arg; unsigned long value; struct lnet_ping_buffer *pbuf; @@ -288,7 +288,7 @@ static int lnet_discover(struct lnet_process_id id, u32 force, */ mutex_lock(&the_lnet.ln_api_mutex); - if (value == *discovery) { + if (value == *discovery_off) { mutex_unlock(&the_lnet.ln_api_mutex); return 0; } @@ -300,7 +300,7 @@ static int lnet_discover(struct lnet_process_id id, u32 force, * updating the peers */ if (the_lnet.ln_state == LNET_STATE_SHUTDOWN) { - *discovery = value; + *discovery_off = value; mutex_unlock(&the_lnet.ln_api_mutex); return 0; } @@ -314,22 +314,10 @@ static int lnet_discover(struct lnet_process_id id, u32 force, pbuf->pb_info.pi_features |= LNET_PING_FEAT_DISCOVERY; lnet_net_unlock(LNET_LOCK_EX); - /* Always update the peers. This will result in a push to the - * peers with the updated capabilities feature mask. The peer can - * then take appropriate action to update its representation of - * the node. - * - * If discovery is already off, turn it on first before pushing - * the update. The discovery flag must be on before pushing. - * otherwise if the flag is on and we're turning it off then push - * first before turning the flag off. In the former case the flag - * is being set twice, but I find it's better to do that rather - * than have duplicate code in an if/else statement. - */ - if (*discovery > 0 && value == 0) - *discovery = value; - lnet_push_update_to_peers(1); - *discovery = value; + /* only send a push when we're turning off discovery */ + if (*discovery_off <= 0 && value > 0) + lnet_push_update_to_peers(1); + *discovery_off = value; mutex_unlock(&the_lnet.ln_api_mutex); diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index b2065bd..60749da 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -2018,13 +2018,17 @@ void lnet_peer_push_event(struct lnet_event *ev) if (!(pbuf->pb_info.pi_features & LNET_PING_FEAT_DISCOVERY)) { CDEBUG(D_NET, "Peer %s has discovery disabled\n", libcfs_nid2str(lp->lp_primary_nid)); - /* If the peer is going from discovery enabled to - * discovery disabled, we need to reflect that in our - * representation of the peer. + /* Mark the peer for deletion if we already know about it + * and it's going from discovery set to no discovery set */ if (!(lp->lp_state & (LNET_PEER_NO_DISCOVERY | - LNET_PEER_DISCOVERING))) + LNET_PEER_DISCOVERING)) && + lp->lp_state & LNET_PEER_DISCOVERED) { + CDEBUG(D_NET, "Marking %s:0x%x for deletion\n", + libcfs_nid2str(lp->lp_primary_nid), + lp->lp_state); lp->lp_state |= LNET_PEER_MARK_DELETION; + } lp->lp_state |= LNET_PEER_NO_DISCOVERY; } else if (lp->lp_state & LNET_PEER_NO_DISCOVERY) { CDEBUG(D_NET, "Peer %s has discovery enabled\n", From patchwork Wed Jun 3 00:59:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584747 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A68C912 for ; Wed, 3 Jun 2020 01:00:43 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 62A572072F for ; Wed, 3 Jun 2020 01:00:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 62A572072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 7B7CE21FB46; Tue, 2 Jun 2020 18:00:24 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 82B3D21F563 for ; Tue, 2 Jun 2020 18:00:09 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 7FB506BB; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 7E9F32C6; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:53 -0400 Message-Id: <1591146001-27171-15-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 14/22] lnet: Force full discovery cycle X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata There are scenarios where there could be a discrepancy between cached peer information and reality. In these cases what could end-up happening is incomplete interface information might be cached because one side determined that the peer didn't require a PUSH. This will lead to undesired MR behavior, where not all the interfaces are used for a period of time. Therefore, it is safer to always force a full discovery cycle: GET/PUSH to ensure both sides are up-to-date. In the NMR case, when discovery is turned off, make sure to flag discovery as complete to avoid stalling the state machine. WC-bug-id: https://jira.whamcloud.com/browse/LU-13477 Lustre-commit: 9bafd530d7858 ("LU-13477 lnet: Force full discovery cycle") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/38322 Reviewed-by: Chris Horn Reviewed-by: Serguei Smirnov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/peer.c | 40 ++++++++++++++++++++++++---------------- net/lnet/lnet/router.c | 5 +++-- 2 files changed, 27 insertions(+), 18 deletions(-) diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 60749da..5869a20 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -1179,6 +1179,11 @@ struct lnet_peer_ni * lp = lpni->lpni_peer_net->lpn_peer; while (!lnet_peer_is_uptodate(lp)) { + spin_lock(&lp->lp_lock); + /* force a full discovery cycle */ + lp->lp_state |= LNET_PEER_FORCE_PING | LNET_PEER_FORCE_PUSH; + spin_unlock(&lp->lp_lock); + rc = lnet_discover_peer_locked(lpni, cpt, true); if (rc) goto out_decref; @@ -3043,6 +3048,19 @@ static int lnet_peer_push_failed(struct lnet_peer *lp) return rc ? rc : LNET_REDISCOVER_PEER; } +/* Mark the peer as discovered. */ +static int lnet_peer_discovered(struct lnet_peer *lp) +__must_hold(&lp->lp_lock) +{ + lp->lp_state |= LNET_PEER_DISCOVERED; + lp->lp_state &= ~(LNET_PEER_DISCOVERING | + LNET_PEER_REDISCOVER); + + CDEBUG(D_NET, "peer %s\n", libcfs_nid2str(lp->lp_primary_nid)); + + return 0; +} + /* Active side of push. */ static int lnet_peer_send_push(struct lnet_peer *lp) __must_hold(&lp->lp_lock) @@ -3056,6 +3074,12 @@ static int lnet_peer_send_push(struct lnet_peer *lp) /* Don't push to a non-multi-rail peer. */ if (!(lp->lp_state & LNET_PEER_MULTI_RAIL)) { lp->lp_state &= ~LNET_PEER_FORCE_PUSH; + /* if peer's NIDs are uptodate then peer is discovered */ + if (lp->lp_state & LNET_PEER_NIDS_UPTODATE) { + rc = lnet_peer_discovered(lp); + return rc; + } + return 0; } @@ -3147,22 +3171,6 @@ static void lnet_peer_discovery_error(struct lnet_peer *lp, int error) } /* - * Mark the peer as discovered. - */ -static int lnet_peer_discovered(struct lnet_peer *lp) -__must_hold(&lp->lp_lock) -{ - lp->lp_state |= LNET_PEER_DISCOVERED; - lp->lp_state &= ~(LNET_PEER_DISCOVERING | - LNET_PEER_REDISCOVER); - - CDEBUG(D_NET, "peer %s\n", libcfs_nid2str(lp->lp_primary_nid)); - - return 0; -} - - -/* * Discovering this peer is taking too long. Cancel any Ping or Push * that discovery is waiting on by unlinking the relevant MDs. The * lnet_discovery_event_handler() will proceed from here and complete diff --git a/net/lnet/lnet/router.c b/net/lnet/lnet/router.c index af50e51..903d027 100644 --- a/net/lnet/lnet/router.c +++ b/net/lnet/lnet/router.c @@ -1154,9 +1154,10 @@ bool lnet_router_checker_active(void) spin_unlock(&rtr->lp_lock); continue; } - /* make sure we actively discover the router */ + /* make sure we fully discover the router */ rtr->lp_state &= ~LNET_PEER_NIDS_UPTODATE; - rtr->lp_state |= LNET_PEER_RTR_DISCOVERY; + rtr->lp_state |= LNET_PEER_FORCE_PING | LNET_PEER_FORCE_PUSH | + LNET_PEER_RTR_DISCOVERY; spin_unlock(&rtr->lp_lock); /* find the peer_ni associated with the primary NID */ From patchwork Wed Jun 3 00:59:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584751 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 038541391 for ; Wed, 3 Jun 2020 01:00:50 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E04C32072F for ; Wed, 3 Jun 2020 01:00:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E04C32072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1160D21F8DC; Tue, 2 Jun 2020 18:00:28 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D863221F563 for ; Tue, 2 Jun 2020 18:00:09 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 831E16BC; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 819A22CD; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:54 -0400 Message-Id: <1591146001-27171-16-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 15/22] lnet: set route aliveness properly X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata In the case when the discover is toggled from on to off, the route aliveness might become stale due to not updating the route->lr_alive variable correctly. It will get updated once the gateway is pinged. However, there is a period of max alive_router_check_interval where the route can be down. WC-bug-id: https://jira.whamcloud.com/browse/LU-13472 Lustre-commit: eee4358d9d467 ("LU-13472 lnet: set route aliveness properly") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/38323 Reviewed-by: Chris Horn Reviewed-by: Serguei Smirnov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/lnet/router.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/lnet/lnet/router.c b/net/lnet/lnet/router.c index 903d027..c0578d9 100644 --- a/net/lnet/lnet/router.c +++ b/net/lnet/lnet/router.c @@ -513,8 +513,10 @@ bool lnet_is_route_alive(struct lnet_route *route) * on the gateway we assume this is intentional and we mark the * gateway as multi-hop */ - list_for_each_entry(route, &lp->lp_routes, lr_gwlist) + list_for_each_entry(route, &lp->lp_routes, lr_gwlist) { + lnet_set_route_aliveness(route, true); lnet_set_route_hop_type(lp, route); + } return; } From patchwork Wed Jun 3 00:59:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584761 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F381C1391 for ; Wed, 3 Jun 2020 01:01:05 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DBE572072F for ; Wed, 3 Jun 2020 01:01:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DBE572072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id B94DB21FCBD; Tue, 2 Jun 2020 18:00:35 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2880321F563 for ; Tue, 2 Jun 2020 18:00:10 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 860936BD; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 849EF2C5; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:55 -0400 Message-Id: <1591146001-27171-17-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 16/22] lnet: Correct the default LND timeout X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn Default LND timeout is currently too low. To allow for lnet_retry_count resend attempts within a single lnet_transaction_timeout window, the LND timeout needs to be less than lnet_transaction_timeout / lnet_retry_count. If the retry count is 0, we still want LND timeout to be less than the LNet transaction timeout. Also, be sure to update the LND timeout when health is toggled on or off. WC-bug-id: https://jira.whamcloud.com/browse/LU-13510 Lustre-commit: 0127d64b8cadd ("LU-13510 lnet: Correct the default LND timeout") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/38481 Reviewed-by: Serguei Smirnov Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 1 - net/lnet/lnet/api-ni.c | 28 +++++++++++++++++++--------- 2 files changed, 19 insertions(+), 10 deletions(-) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index a4a323c..a7825f9 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -83,7 +83,6 @@ /* default timeout */ #define DEFAULT_PEER_TIMEOUT 180 -#define LNET_LND_DEFAULT_TIMEOUT 5 int choose_ipv4_src(u32 *ret, int interface, u32 dst_ipaddr, struct net *ns); diff --git a/net/lnet/lnet/api-ni.c b/net/lnet/lnet/api-ni.c index a966e64..62b4fa7 100644 --- a/net/lnet/lnet/api-ni.c +++ b/net/lnet/lnet/api-ni.c @@ -170,7 +170,15 @@ static int recovery_interval_set(const char *val, MODULE_PARM_DESC(lnet_retry_count, "Maximum number of times to retry transmitting a message"); -unsigned int lnet_lnd_timeout = LNET_LND_DEFAULT_TIMEOUT; +#define LNET_LND_TIMEOUT_DEFAULT ((LNET_TRANSACTION_TIMEOUT_HEALTH_DEFAULT - 1) / \ + (LNET_RETRY_COUNT_HEALTH_DEFAULT + 1)) +unsigned int lnet_lnd_timeout = LNET_LND_TIMEOUT_DEFAULT; +static void lnet_set_lnd_timeout(void) +{ + lnet_lnd_timeout = (lnet_transaction_timeout - 1) / + (lnet_retry_count + 1); +} + unsigned int lnet_current_net_count; /* @@ -220,6 +228,7 @@ static int lnet_discover(struct lnet_process_id id, u32 force, lnet_transaction_timeout = LNET_TRANSACTION_TIMEOUT_HEALTH_DEFAULT; lnet_retry_count = LNET_RETRY_COUNT_HEALTH_DEFAULT; + lnet_set_lnd_timeout(); /* if we're turning off health then use the no health timeout * default. */ @@ -227,6 +236,7 @@ static int lnet_discover(struct lnet_process_id id, u32 force, lnet_transaction_timeout = LNET_TRANSACTION_TIMEOUT_NO_HEALTH_DEFAULT; lnet_retry_count = 0; + lnet_set_lnd_timeout(); } *sensitivity = value; @@ -385,10 +395,10 @@ static int lnet_discover(struct lnet_process_id id, u32 force, } *transaction_to = value; - if (lnet_retry_count == 0) - lnet_lnd_timeout = value; - else - lnet_lnd_timeout = value / lnet_retry_count; + /* Update the lnet_lnd_timeout now that we've modified the + * transaction timeout + */ + lnet_set_lnd_timeout(); mutex_unlock(&the_lnet.ln_api_mutex); @@ -428,10 +438,10 @@ static int lnet_discover(struct lnet_process_id id, u32 force, *retry_count = value; - if (value == 0) - lnet_lnd_timeout = lnet_transaction_timeout; - else - lnet_lnd_timeout = lnet_transaction_timeout / value; + /* Update the lnet_lnd_timeout now that we've modified the + * transaction timeout + */ + lnet_set_lnd_timeout(); mutex_unlock(&the_lnet.ln_api_mutex); From patchwork Wed Jun 3 00:59:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584755 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 701591391 for ; Wed, 3 Jun 2020 01:00:56 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 582882072F for ; Wed, 3 Jun 2020 01:00:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 582882072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1868E21F9C5; Tue, 2 Jun 2020 18:00:31 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 800DE21F714 for ; Tue, 2 Jun 2020 18:00:10 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 894436BE; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 87B132CE; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:56 -0400 Message-Id: <1591146001-27171-18-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 17/22] lnet: Add lnet_lnd_timeout to sysfs X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn Allow lnet_lnd_timeout to be read (only) from sysfs. WC-bug-id: https://jira.whamcloud.com/browse/LU-13510 Lustre-commit: 17802b329cdea ("LU-13510 lnet: Add lnet_lnd_timeout to sysfs") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/38482 Reviewed-by: Serguei Smirnov Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- include/linux/lnet/lib-lnet.h | 1 + net/lnet/lnet/router_proc.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/include/linux/lnet/lib-lnet.h b/include/linux/lnet/lib-lnet.h index a7825f9..def0923 100644 --- a/include/linux/lnet/lib-lnet.h +++ b/include/linux/lnet/lib-lnet.h @@ -417,6 +417,7 @@ struct lnet_ni * extern unsigned int lnet_transaction_timeout; extern unsigned int lnet_retry_count; +extern unsigned int lnet_lnd_timeout; extern unsigned int lnet_numa_range; extern unsigned int lnet_health_sensitivity; extern unsigned int lnet_recovery_interval; diff --git a/net/lnet/lnet/router_proc.c b/net/lnet/lnet/router_proc.c index 180bbde..7fe8d33 100644 --- a/net/lnet/lnet/router_proc.c +++ b/net/lnet/lnet/router_proc.c @@ -881,6 +881,13 @@ static int proc_lnet_portal_rotor(struct ctl_table *table, int write, .proc_handler = &proc_lnet_portal_rotor, }, { + .procname = "lnet_lnd_timeout", + .data = &lnet_lnd_timeout, + .maxlen = sizeof(lnet_lnd_timeout), + .mode = 0444, + .proc_handler = &proc_dointvec, + }, + { } }; From patchwork Wed Jun 3 00:59:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584759 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90389912 for ; Wed, 3 Jun 2020 01:01:02 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 786C32072F for ; Wed, 3 Jun 2020 01:01:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 786C32072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 1E4AF21FC8E; Tue, 2 Jun 2020 18:00:34 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id C07F421F714 for ; Tue, 2 Jun 2020 18:00:10 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 8BE406BF; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8AC2D2C6; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:57 -0400 Message-Id: <1591146001-27171-19-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 18/22] lnet: lnd: Allow independent ko2iblnd timeout X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn Allow ko2iblnd timeout parameter to be set independent of the lnet_transaction_timeout and retry_count. WC-bug-id: https://jira.whamcloud.com/browse/LU-13510 Lustre-commit: 0c038ee2f613e ("LU-13510 lnd: Allow independent ko2iblnd timeout") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/38459 Reviewed-by: Serguei Smirnov Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd.h | 6 ++++++ net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 10 +++++----- net/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 2 +- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.h b/net/lnet/klnds/o2iblnd/o2iblnd.h index 8aa79d5..f60a69d 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.h +++ b/net/lnet/klnds/o2iblnd/o2iblnd.h @@ -642,6 +642,12 @@ struct kib_peer_ni { int kiblnd_msg_queue_size(int version, struct lnet_ni *ni); +static inline int kiblnd_timeout(void) +{ + return *kiblnd_tunables.kib_timeout ? *kiblnd_tunables.kib_timeout : + lnet_get_lnd_timeout(); +} + static inline int kiblnd_concurrent_sends(int version, struct lnet_ni *ni) { diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index f7c802f..09a46d6 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -1160,7 +1160,7 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, return; } - timeout_ns = lnet_get_lnd_timeout() * NSEC_PER_SEC; + timeout_ns = kiblnd_timeout() * NSEC_PER_SEC; tx->tx_queued = 1; tx->tx_deadline = ktime_add_ns(ktime_get(), timeout_ns); @@ -1289,12 +1289,12 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, if (*kiblnd_tunables.kib_use_priv_port) { rc = kiblnd_resolve_addr(cmid, &srcaddr, &dstaddr, - lnet_get_lnd_timeout() * 1000); + kiblnd_timeout() * 1000); } else { rc = rdma_resolve_addr(cmid, (struct sockaddr *)&srcaddr, (struct sockaddr *)&dstaddr, - lnet_get_lnd_timeout() * 1000); + kiblnd_timeout() * 1000); } if (rc != 0) { /* Can't initiate address resolution: */ @@ -3021,7 +3021,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, rc = event->status; } else { rc = rdma_resolve_route(cmid, - lnet_get_lnd_timeout() * 1000); + kiblnd_timeout() * 1000); if (!rc) { struct kib_net *net = peer_ni->ibp_ni->ni_data; struct kib_dev *dev = net->ibn_dev; @@ -3437,7 +3437,7 @@ static int kiblnd_resolve_addr(struct rdma_cm_id *cmid, * interval. */ - lnd_timeout = lnet_get_lnd_timeout(); + lnd_timeout = kiblnd_timeout(); if (lnd_timeout > n * p) chunk = (chunk * n * p) / lnd_timeout; if (!chunk) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c index c9e14ec..7407ced 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_modparams.c @@ -45,7 +45,7 @@ module_param(cksum, int, 0644); MODULE_PARM_DESC(cksum, "set non-zero to enable message (not RDMA) checksums"); -static int timeout = 50; +static int timeout; module_param(timeout, int, 0644); MODULE_PARM_DESC(timeout, "timeout (seconds)"); From patchwork Wed Jun 3 00:59:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584763 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E83B1391 for ; Wed, 3 Jun 2020 01:01:09 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 072B42072F for ; Wed, 3 Jun 2020 01:01:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 072B42072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 31C6B21FCCF; Tue, 2 Jun 2020 18:00:37 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 2131E21F714 for ; Tue, 2 Jun 2020 18:00:11 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 907B96C0; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 8DEE02D0; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:58 -0400 Message-Id: <1591146001-27171-20-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 19/22] lnet: lnd: Allow independent socklnd timeout X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn Allow the socklnd timeout to be set independent of lnet_transaction_timeout and retry_count. WC-bug-id: https://jira.whamcloud.com/browse/LU-13510 Lustre-commit: 5c2a1267f9471 ("LU-13510 lnd: Allow independent socklnd timeout") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/38460 Reviewed-by: Serguei Smirnov Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/socklnd/socklnd.c | 4 ++-- net/lnet/klnds/socklnd/socklnd.h | 7 +++++++ net/lnet/klnds/socklnd/socklnd_cb.c | 16 ++++++++-------- net/lnet/klnds/socklnd/socklnd_modparams.c | 2 +- 4 files changed, 18 insertions(+), 11 deletions(-) diff --git a/net/lnet/klnds/socklnd/socklnd.c b/net/lnet/klnds/socklnd/socklnd.c index b5d92d3..444b90b 100644 --- a/net/lnet/klnds/socklnd/socklnd.c +++ b/net/lnet/klnds/socklnd/socklnd.c @@ -1311,7 +1311,7 @@ struct ksock_peer_ni * /* Set the deadline for the outgoing HELLO to drain */ conn->ksnc_tx_bufnob = sock->sk->sk_wmem_queued; conn->ksnc_tx_deadline = ktime_get_seconds() + - lnet_get_lnd_timeout(); + ksocknal_timeout(); mb(); /* order with adding to peer_ni's conn list */ list_add(&conn->ksnc_list, &peer_ni->ksnp_conns); @@ -1699,7 +1699,7 @@ struct ksock_peer_ni * switch (conn->ksnc_rx_state) { case SOCKNAL_RX_LNET_PAYLOAD: last_rcv = conn->ksnc_rx_deadline - - lnet_get_lnd_timeout(); + ksocknal_timeout(); CERROR("Completing partial receive from %s[%d], ip %pI4h:%d, with error, wanted: %zd, left: %d, last alive is %lld secs ago\n", libcfs_id2str(conn->ksnc_peer->ksnp_id), conn->ksnc_type, &conn->ksnc_ipaddr, conn->ksnc_port, diff --git a/net/lnet/klnds/socklnd/socklnd.h b/net/lnet/klnds/socklnd/socklnd.h index 6c77b75..7d49fff 100644 --- a/net/lnet/klnds/socklnd/socklnd.h +++ b/net/lnet/klnds/socklnd/socklnd.h @@ -610,6 +610,13 @@ struct ksock_proto { ksocknal_destroy_peer(peer_ni); } +static inline int ksocknal_timeout(void) +{ + return *ksocknal_tunables.ksnd_timeout ? + *ksocknal_tunables.ksnd_timeout : + lnet_get_lnd_timeout(); +} + int ksocknal_startup(struct lnet_ni *ni); void ksocknal_shutdown(struct lnet_ni *ni); int ksocknal_ctl(struct lnet_ni *ni, unsigned int cmd, void *arg); diff --git a/net/lnet/klnds/socklnd/socklnd_cb.c b/net/lnet/klnds/socklnd/socklnd_cb.c index c03f91c7..2759455 100644 --- a/net/lnet/klnds/socklnd/socklnd_cb.c +++ b/net/lnet/klnds/socklnd/socklnd_cb.c @@ -218,7 +218,7 @@ struct ksock_tx * * something got ACKed */ conn->ksnc_tx_deadline = ktime_get_seconds() + - lnet_get_lnd_timeout(); + ksocknal_timeout(); conn->ksnc_peer->ksnp_last_alive = ktime_get_seconds(); conn->ksnc_tx_bufnob = bufnob; mb(); @@ -264,7 +264,7 @@ struct ksock_tx * conn->ksnc_peer->ksnp_last_alive = ktime_get_seconds(); conn->ksnc_rx_deadline = ktime_get_seconds() + - lnet_get_lnd_timeout(); + ksocknal_timeout(); mb(); /* order with setting rx_started */ conn->ksnc_rx_started = 1; @@ -419,7 +419,7 @@ struct ksock_tx * /* ZC_REQ is going to be pinned to the peer_ni */ tx->tx_deadline = ktime_get_seconds() + - lnet_get_lnd_timeout(); + ksocknal_timeout(); LASSERT(!tx->tx_msg.ksm_zc_cookies[0]); @@ -711,7 +711,7 @@ struct ksock_conn * if (list_empty(&conn->ksnc_tx_queue) && !bufnob) { /* First packet starts the timeout */ conn->ksnc_tx_deadline = ktime_get_seconds() + - lnet_get_lnd_timeout(); + ksocknal_timeout(); if (conn->ksnc_tx_bufnob > 0) /* something got ACKed */ conn->ksnc_peer->ksnp_last_alive = ktime_get_seconds(); conn->ksnc_tx_bufnob = 0; @@ -887,7 +887,7 @@ struct ksock_route * ksocknal_find_connecting_route_locked(peer_ni)) { /* the message is going to be pinned to the peer_ni */ tx->tx_deadline = ktime_get_seconds() + - lnet_get_lnd_timeout(); + ksocknal_timeout(); /* Queue the message until a connection is established */ list_add_tail(&tx->tx_list, &peer_ni->ksnp_tx_queue); @@ -1652,7 +1652,7 @@ void ksocknal_write_callback(struct ksock_conn *conn) /* socket type set on active connections - not set on passive */ LASSERT(!active == !(conn->ksnc_type != SOCKLND_CONN_NONE)); - timeout = active ? lnet_get_lnd_timeout() : + timeout = active ? ksocknal_timeout() : lnet_acceptor_timeout(); rc = lnet_sock_read(sock, &hello->kshm_magic, @@ -1790,7 +1790,7 @@ void ksocknal_write_callback(struct ksock_conn *conn) int retry_later = 0; int rc = 0; - deadline = ktime_get_seconds() + lnet_get_lnd_timeout(); + deadline = ktime_get_seconds() + ksocknal_timeout(); write_lock_bh(&ksocknal_data.ksnd_global_lock); @@ -2550,7 +2550,7 @@ void ksocknal_write_callback(struct ksock_conn *conn) * timeout interval. */ - lnd_timeout = lnet_get_lnd_timeout(); + lnd_timeout = ksocknal_timeout(); if (lnd_timeout > n * p) chunk = (chunk * n * p) / lnd_timeout; if (!chunk) diff --git a/net/lnet/klnds/socklnd/socklnd_modparams.c b/net/lnet/klnds/socklnd/socklnd_modparams.c index 35b71ba..b511e54 100644 --- a/net/lnet/klnds/socklnd/socklnd_modparams.c +++ b/net/lnet/klnds/socklnd/socklnd_modparams.c @@ -24,7 +24,7 @@ #include #endif -static int sock_timeout = 50; +static int sock_timeout; module_param(sock_timeout, int, 0644); MODULE_PARM_DESC(sock_timeout, "dead socket timeout (seconds)"); From patchwork Wed Jun 3 00:59:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584765 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A6E01392 for ; Wed, 3 Jun 2020 01:01:13 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0313C2072F for ; Wed, 3 Jun 2020 01:01:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0313C2072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id EF81421FAAB; Tue, 2 Jun 2020 18:00:38 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 76FCA21F2E1 for ; Tue, 2 Jun 2020 18:00:11 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 92B086C1; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 9137F2C5; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 20:59:59 -0400 Message-Id: <1591146001-27171-21-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 20/22] lnet: lnd: gracefully handle unexpected events X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Amir Shehata , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Amir Shehata When a tx completes kiblnd_tx_complete() callback is invoked. We ensure: LASSERT (tx->tx_sending > 0); However this assert is being triggered in some rare scenarios. The reason tx_sending would be 0 at this point is because: 1. ib_post_send() failed but OFED stack is still sending a tx complete event. 2. We're getting two different events for the same tx Instead of asserting, ignore that tx_complete event and print the tx pointer and its status. WC-bug-id: https://jira.whamcloud.com/browse/LU-13553 Lustre-commit: 60f9f539e686f ("LU-13553 lnd: gracefully handle unexpected events") Signed-off-by: Amir Shehata Reviewed-on: https://review.whamcloud.com/38669 Reviewed-by: Andreas Dilger Reviewed-by: Serguei Smirnov Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd_cb.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c index 09a46d6..40e196d 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -969,7 +969,11 @@ static int kiblnd_map_tx(struct lnet_ni *ni, struct kib_tx *tx, struct kib_conn *conn = tx->tx_conn; int idle; - LASSERT(tx->tx_sending > 0); + if (tx->tx_sending <= 0) { + CERROR("Received an event on a freed tx: %p status %d\n", + tx, tx->tx_status); + return; + } if (failed) { if (conn->ibc_state == IBLND_CONN_ESTABLISHED) From patchwork Wed Jun 3 01:00:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584769 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23566912 for ; Wed, 3 Jun 2020 01:01:19 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C4552072F for ; Wed, 3 Jun 2020 01:01:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C4552072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id D6F7021FD78; Tue, 2 Jun 2020 18:00:41 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id BA61421F77A for ; Tue, 2 Jun 2020 18:00:11 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 95A176C2; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 9458B2CD; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 21:00:00 -0400 Message-Id: <1591146001-27171-22-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 21/22] lustre: update version to 2.13.54 X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Oleg Drokin New tag 2.13.54 Signed-off-by: Oleg Drokin Signed-off-by: James Simmons --- include/uapi/linux/lustre/lustre_ver.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/lustre/lustre_ver.h b/include/uapi/linux/lustre/lustre_ver.h index a5f96be..b862cf6 100644 --- a/include/uapi/linux/lustre/lustre_ver.h +++ b/include/uapi/linux/lustre/lustre_ver.h @@ -3,9 +3,9 @@ #define LUSTRE_MAJOR 2 #define LUSTRE_MINOR 13 -#define LUSTRE_PATCH 53 +#define LUSTRE_PATCH 54 #define LUSTRE_FIX 0 -#define LUSTRE_VERSION_STRING "2.13.53" +#define LUSTRE_VERSION_STRING "2.13.54" #define OBD_OCD_VERSION(major, minor, patch, fix) \ (((major) << 24) + ((minor) << 16) + ((patch) << 8) + (fix)) From patchwork Wed Jun 3 01:00:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11584767 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5ED81391 for ; Wed, 3 Jun 2020 01:01:15 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AE9A72072F for ; Wed, 3 Jun 2020 01:01:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE9A72072F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 3B9FD21FD3A; Tue, 2 Jun 2020 18:00:40 -0700 (PDT) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 0797521F791 for ; Tue, 2 Jun 2020 18:00:12 -0700 (PDT) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 983516C4; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 974312C6; Tue, 2 Jun 2020 21:00:02 -0400 (EDT) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Tue, 2 Jun 2020 21:00:01 -0400 Message-Id: <1591146001-27171-23-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> References: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 22/22] lnet: procs: print new line based on distro X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Yang Sheng , Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Yang Sheng Since upstream changed to print new line in module parameter callback instead of kernel self. So we need test output of param_get_byte to determine whether output the new line. (upstream: v4.14-rc3-148-g96802e6b1dbf) Also output filename and file content when test failed for santy 133h. Lustre-commit: 0130d70195346 ("LU-10401 procs: print new line based on distro") Signed-off-by: Yang Sheng Reviewed-on: https://review.whamcloud.com/38699 Reviewed-by: Jian Yu Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- net/lnet/libcfs/debug.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/lnet/libcfs/debug.c b/net/lnet/libcfs/debug.c index 41a0a5e..dd03520 100644 --- a/net/lnet/libcfs/debug.c +++ b/net/lnet/libcfs/debug.c @@ -129,7 +129,9 @@ static int param_get_delay(char *buffer, const struct kernel_param *kp) { unsigned int d = *(unsigned int *)kp->arg; - return sprintf(buffer, "%lu", jiffies_to_msecs(d * 10) / MSEC_PER_SEC); + param_get_byte(buffer, kp); + return sprintf(buffer, "%lu%c", jiffies_to_msecs(d * 10) / MSEC_PER_SEC, + strnchr(buffer, PAGE_SIZE, '\n') ? '\n' : '\0'); } unsigned int libcfs_console_max_delay;