From patchwork Thu Oct 19 06:55:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 10016067 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 42E3460215 for ; Thu, 19 Oct 2017 06:55:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34D8828C7E for ; Thu, 19 Oct 2017 06:55:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 29AD428CA6; Thu, 19 Oct 2017 06:55:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0786328C7E for ; Thu, 19 Oct 2017 06:55:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751628AbdJSGzv (ORCPT ); Thu, 19 Oct 2017 02:55:51 -0400 Received: from mail-wr0-f178.google.com ([209.85.128.178]:49214 "EHLO mail-wr0-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751579AbdJSGzu (ORCPT ); Thu, 19 Oct 2017 02:55:50 -0400 Received: by mail-wr0-f178.google.com with SMTP id g90so7192701wrd.6 for ; Wed, 18 Oct 2017 23:55:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Yf9SFv6R6PHHfkb+XWAv6Lx1bZQ5I1FHC2FRjKb43Eg=; b=IGfEjnjYqti5B+dDF0iDLtHSoHAI81wNUVm/TcZl7qxBhy9iYRy+c4GYrkaeRoWMvx x/Z7XFke7tDr6ObXYUdw5as6DJeBsRfHKGu7Ft3iSiWwaytxq14M9E8okfTpqBvTsy5o ADwQUSOzJUEoWbVO9uxSuVPRmuTowLYipdkQ5zGKzboDCX6IcPgpNqdulwCVYKrBEn4+ oLdSFc6OOQM8D7YqXbxqxNUioVvXwPrCkOYgN+wXNEWIkZ+neURMtR+JyKhszPiMSU1U xZSx3CfFmQcZO0BBdRhlOT9mgNswcSru5uFto0qP/beU1XlzMrAoz+e7ybpemmxB0HJ5 P59Q== X-Gm-Message-State: AMCzsaUEWbKNZMXSNZEO5n+Ux1kAvY6kzL0drW45lds+I96dUvEBqMqp NF9suI8tGhpAfyPZprGtUUc= X-Google-Smtp-Source: ABhQp+SpdxeR/gF15N5IpUfOjZQE0I9WOrS08y0Hls0awxF6b4bixNgnuTJ308rePyRfJTrqn8jqNA== X-Received: by 10.223.131.65 with SMTP id 59mr512828wrd.66.1508396149619; Wed, 18 Oct 2017 23:55:49 -0700 (PDT) Received: from [192.168.64.169] (bzq-82-81-101-184.red.bezeqint.net. [82.81.101.184]) by smtp.gmail.com with ESMTPSA id r44sm32411450wrb.37.2017.10.18.23.55.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Oct 2017 23:55:48 -0700 (PDT) Subject: Re: nvmeof rdma regression issue on 4.14.0-rc1 (or maybe mlx4?) To: Yi Zhang , Christoph Hellwig Cc: linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org, Leon Romanovsky , Tariq Toukan References: <1735134433.8514119.1505997532669.JavaMail.zimbra@redhat.com> <1215229914.8516804.1505998051674.JavaMail.zimbra@redhat.com> <20170921144421.GA15285@infradead.org> <47493aa0-4cad-721b-4ea2-c3b2293340aa@grimberg.me> <20170924103426.GB25094@mtr-leonro.local> <4d0e27f2-99e4-ffc7-18a5-e5757c27dad4@redhat.com> <4faccebf-7fcf-58b9-1605-82ee9acc652b@grimberg.me> <729c512f-55ff-25e4-6dd9-8b4dcc31bb8d@grimberg.me> <6d639dd4-9366-2ecf-b49e-751c7a167528@grimberg.me> <64135330-a62d-87b8-c805-6b52fd40bf81@redhat.com> From: Sagi Grimberg Message-ID: <62eea88b-caa4-5799-3d8f-8d8789879aa8@grimberg.me> Date: Thu, 19 Oct 2017 09:55:46 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <64135330-a62d-87b8-c805-6b52fd40bf81@redhat.com> Content-Language: en-US Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP >> Hi Yi, >> >> I was referring to the bug you reported on a simple create_ctrl failed: >> https://pastebin.com/7z0XSGSd >> >> Does it reproduce? >> > yes, this issue was reproduced during "git bisect" with below patch OK, if this does not reproduce with the latest code, lets put it aside for now. So as for the error you see, can you please try the following patch? --- } @@ -739,8 +744,6 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl, static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl, bool remove) { - nvme_rdma_free_qe(ctrl->queues[0].device->dev, &ctrl->async_event_sqe, - sizeof(struct nvme_command), DMA_TO_DEVICE); nvme_rdma_stop_queue(&ctrl->queues[0]); if (remove) { blk_cleanup_queue(ctrl->ctrl.admin_q); -- -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 405895b1dff2..916658e010ff 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -572,6 +572,11 @@ static void nvme_rdma_free_queue(struct nvme_rdma_queue *queue) if (!test_and_clear_bit(NVME_RDMA_Q_ALLOCATED, &queue->flags)) return; + if(nvme_rdma_queue_idx(queue) == 0) + nvme_rdma_free_qe(queue->device->dev, + &queue->ctrl->async_event_sqe, + sizeof(struct nvme_command), DMA_TO_DEVICE); + nvme_rdma_destroy_queue_ib(queue); rdma_destroy_id(queue->cm_id);