From patchwork Fri Nov 23 07:15:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 10695083 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9F8718F0 for ; Fri, 23 Nov 2018 07:19:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B04672B079 for ; Fri, 23 Nov 2018 07:19:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A38372BAE4; Fri, 23 Nov 2018 07:19:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2CB352B0B6 for ; Fri, 23 Nov 2018 07:19:38 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id E80A021F6EF; Thu, 22 Nov 2018 23:19:37 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 22B0521F647 for ; Thu, 22 Nov 2018 23:19:36 -0800 (PST) X-Virus-Scanned: by amavisd-new at test-mx.suse.de X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "Cc" Received: from relay1.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 42D0DAC70; Fri, 23 Nov 2018 07:19:35 +0000 (UTC) From: NeilBrown To: Andreas Dilger , James Simmons , Oleg Drokin Date: Fri, 23 Nov 2018 18:15:28 +1100 Message-ID: <154295732806.2850.603181458106225374.stgit@noble> In-Reply-To: <154295730810.2850.961218355189474016.stgit@noble> References: <154295730810.2850.961218355189474016.stgit@noble> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Subject: [lustre-devel] [PATCH 7/9] lustre: lnet: Stop MLX5 triggering a dump_cqe X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Doug Oucharek , Lustre Development List Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Doug Oucharek We have found that MLX5 will trigger a dump_cqe if we don't invalidate the rkey on a newly allocated MR for FastReg usage. This fix just tags the MR as invalid on its creation if we are using FastReg and that will force it to do an invalidate of the rkey on first usage. Signed-off-by: Doug Oucharek WC-bug-id: https://jira.whamcloud.com/browse/LU-8752 Reviewed-on: https://review.whamcloud.com/24306 Reviewed-by: Dmitry Eremin Reviewed-by: Amir Shehata Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: NeilBrown --- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index ecdf4dee533d..a5eada8ee354 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -1483,7 +1483,12 @@ static int kiblnd_alloc_freg_pool(struct kib_fmr_poolset *fps, goto out_middle; } - frd->frd_valid = true; + /* + * There appears to be a bug in MLX5 code where you must + * invalidate the rkey of a new FastReg pool before first + * using it. Thus, I am marking the FRD invalid here. + */ + frd->frd_valid = false; list_add_tail(&frd->frd_list, &fpo->fast_reg.fpo_pool_list); fpo->fast_reg.fpo_pool_size++;