From patchwork Fri Aug 23 07:50:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng Xu X-Patchwork-Id: 13774725 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B62913D51D for ; Fri, 23 Aug 2024 07:51:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399475; cv=none; b=c1yGtIp5wE0zlaLi2mYI+wDkFdyDMCTIirEbEuruyDnxoE17EfB7Qw/0jGYauhVU46EPWpb7zs6DApl/SH2sv/0mV+J0ZXCfDxuyNGcBkx14wpOGI+2JEI94C3Rw92/Ri7ucFLtDtJqsmi+zHdA/yttu1+32kROxk+2GC/5WGao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399475; c=relaxed/simple; bh=kllXqj7/n7qr5ooiWLb4gkupRQZfpra1DehjCKInps0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GdXMj/8qWTzULFYXc7zj0GnaM6NqPvp037TbcabUs718XRFb7irf7KJkdqfzpegkaq5fyzymIn3Q//Z/hGoo55ORSZy9DH8YNVcJGlz/yquvNs0n0bxQwKOriwsbFXvYy6HHzpQcHUuvWJQvXiUyj1oSQq83qECHccsL1v4nodg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=KBUN56Ll; arc=none smtp.client-ip=115.124.30.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="KBUN56Ll" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724399463; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=3B0UdIQF1Wd35JDJ4rkg/HdwUySaqmCTJOmNmTkeSkU=; b=KBUN56LlXh6oMzAF9kBNCJskwcVhikQGIEpp4MhwWs2ZG6M+h8opRUt2Eg11rfQwACQD9Cwqu0aRUxMoq8JdHSiRL/nqWj6i1faMu6gixnwqRSscAsUJL+3Le8DUlcVluTCvXpj4ftPrkBh4mWwXlTTOGYkr4dSPdWlpkrH2tmA= Received: from localhost(mailfrom:chengyou@linux.alibaba.com fp:SMTPD_---0WDSn1VO_1724399462) by smtp.aliyun-inc.com; Fri, 23 Aug 2024 15:51:03 +0800 From: Cheng Xu To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, KaiShen@linux.alibaba.com Subject: [PATCH for-next 1/4] RDMA/erdma: Make the device probe process more robust Date: Fri, 23 Aug 2024 15:50:55 +0800 Message-Id: <20240823075058.89488-2-chengyou@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240823075058.89488-1-chengyou@linux.alibaba.com> References: <20240823075058.89488-1-chengyou@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Driver may probe again while hardware is destroying the internal resources allocated for previous probing, which will fail the device probe. To make it more robust, we always issue a reset at the beginning of the device probe process. Signed-off-by: Cheng Xu --- drivers/infiniband/hw/erdma/erdma.h | 1 + drivers/infiniband/hw/erdma/erdma_main.c | 44 +++++++++++++++++++----- 2 files changed, 36 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/hw/erdma/erdma.h b/drivers/infiniband/hw/erdma/erdma.h index c8bd698e21b0..b5c258f77ca0 100644 --- a/drivers/infiniband/hw/erdma/erdma.h +++ b/drivers/infiniband/hw/erdma/erdma.h @@ -94,6 +94,7 @@ enum { #define ERDMA_CMDQ_TIMEOUT_MS 15000 #define ERDMA_REG_ACCESS_WAIT_MS 20 +#define ERDMA_WAIT_DEV_REST_CNT 50 #define ERDMA_WAIT_DEV_DONE_CNT 500 struct erdma_cmdq { diff --git a/drivers/infiniband/hw/erdma/erdma_main.c b/drivers/infiniband/hw/erdma/erdma_main.c index 7080f8a71ec4..9199058a0b29 100644 --- a/drivers/infiniband/hw/erdma/erdma_main.c +++ b/drivers/infiniband/hw/erdma/erdma_main.c @@ -209,11 +209,30 @@ static void erdma_device_uninit(struct erdma_dev *dev) dma_pool_destroy(dev->resp_pool); } -static void erdma_hw_reset(struct erdma_dev *dev) +static int erdma_hw_reset(struct erdma_dev *dev, bool wait) { u32 ctrl = FIELD_PREP(ERDMA_REG_DEV_CTRL_RESET_MASK, 1); + int i; erdma_reg_write32(dev, ERDMA_REGS_DEV_CTRL_REG, ctrl); + + if (!wait) + return 0; + + for (i = 0; i < ERDMA_WAIT_DEV_REST_CNT; i++) { + if (erdma_reg_read32_filed(dev, ERDMA_REGS_DEV_ST_REG, + ERDMA_REG_DEV_ST_RESET_DONE_MASK)) + break; + + msleep(ERDMA_REG_ACCESS_WAIT_MS); + } + + if (i == ERDMA_WAIT_DEV_REST_CNT) { + dev_err(&dev->pdev->dev, "wait reset done timeout.\n"); + return -ETIME; + } + + return 0; } static int erdma_wait_hw_init_done(struct erdma_dev *dev) @@ -239,6 +258,17 @@ static int erdma_wait_hw_init_done(struct erdma_dev *dev) return 0; } +static int erdma_preinit_check(struct erdma_dev *dev) +{ + u32 version = erdma_reg_read32(dev, ERDMA_REGS_VERSION_REG); + + /* we knows that it is a non-functional function. */ + if (version == 0) + return -ENODEV; + + return erdma_hw_reset(dev, true); +} + static const struct pci_device_id erdma_pci_tbl[] = { { PCI_DEVICE(PCI_VENDOR_ID_ALIBABA, 0x107f) }, {} @@ -248,7 +278,6 @@ static int erdma_probe_dev(struct pci_dev *pdev) { struct erdma_dev *dev; int bars, err; - u32 version; err = pci_enable_device(pdev); if (err) { @@ -287,12 +316,9 @@ static int erdma_probe_dev(struct pci_dev *pdev) goto err_release_bars; } - version = erdma_reg_read32(dev, ERDMA_REGS_VERSION_REG); - if (version == 0) { - /* we knows that it is a non-functional function. */ - err = -ENODEV; + err = erdma_preinit_check(dev); + if (err) goto err_iounmap_func_bar; - } err = erdma_device_init(dev, pdev); if (err) @@ -327,7 +353,7 @@ static int erdma_probe_dev(struct pci_dev *pdev) return 0; err_reset_hw: - erdma_hw_reset(dev); + erdma_hw_reset(dev, false); err_uninit_cmdq: erdma_cmdq_destroy(dev); @@ -364,7 +390,7 @@ static void erdma_remove_dev(struct pci_dev *pdev) struct erdma_dev *dev = pci_get_drvdata(pdev); erdma_ceqs_uninit(dev); - erdma_hw_reset(dev); + erdma_hw_reset(dev, false); erdma_cmdq_destroy(dev); erdma_aeq_destroy(dev); erdma_comm_irq_uninit(dev); From patchwork Fri Aug 23 07:50:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng Xu X-Patchwork-Id: 13774726 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A585613D635 for ; Fri, 23 Aug 2024 07:51:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.112 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399475; cv=none; b=BMpgWraB6bu+thtXoCM46UiP4a3mk9u//T1YseWBhp1Ni+ceb/TgfGS1gUZxHJWCNlLgKt+sNhVxk4NZAqkxT4LfERX46pxV9XGdU0bONP6jDh8W75n47/OeFhE1LE6OP3qGM4qhMJNi0OnpdTcCgRPrQH4tEcrhzfjsmducSSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399475; c=relaxed/simple; bh=E9tBNIv20W9VMIxsRNaGKcU3HzppGjyEj/zxRGeq+EU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r0tw3YR96JhGXicjjCP43TXjBJgDxYOjiiCy02QkSRIVRY5m0tUZwQ8Lu/MpDDAse3dPVQqfsAxNSuBqAsY+S/31piMWSGcDnLJPDUgD+DQQb+3DZEcjvlYCYAr6kJ6xcMgrcHol2ErE7DSu+0oKXKbKXPHXUJsC3LLwZjPiBVY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=A9WRhfNl; arc=none smtp.client-ip=115.124.30.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="A9WRhfNl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724399464; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=7J1VEBiRlVh31m5C1XJjnqvs7H1YW9A3Er5xclysqDQ=; b=A9WRhfNlxGN9jFu2VLWE6F6t1qqqhW8FflTL8Q7+x5aIixAvc28MysCUy1x7P7KMTPcYzKNoXUN/tl0DcXIJAYcpNAjG4A1vG9ICUczGe17P25WsRt3+r/wHzkWf7jD1c8m2biAfWrrq0KN5bvNWyabeydryfp7S9q4QGWRaKA0= Received: from localhost(mailfrom:chengyou@linux.alibaba.com fp:SMTPD_---0WDSj-TL_1724399463) by smtp.aliyun-inc.com; Fri, 23 Aug 2024 15:51:03 +0800 From: Cheng Xu To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, KaiShen@linux.alibaba.com Subject: [PATCH for-next 2/4] RDMA/erdma: Refactor the initialization and destruction of EQ Date: Fri, 23 Aug 2024 15:50:56 +0800 Message-Id: <20240823075058.89488-3-chengyou@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240823075058.89488-1-chengyou@linux.alibaba.com> References: <20240823075058.89488-1-chengyou@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 We extracted the common parts of the initialization/destruction process to make the code cleaner. Signed-off-by: Cheng Xu --- drivers/infiniband/hw/erdma/erdma.h | 3 +- drivers/infiniband/hw/erdma/erdma_cmdq.c | 26 ++------ drivers/infiniband/hw/erdma/erdma_eq.c | 83 +++++++++++++----------- drivers/infiniband/hw/erdma/erdma_main.c | 4 +- 4 files changed, 55 insertions(+), 61 deletions(-) diff --git a/drivers/infiniband/hw/erdma/erdma.h b/drivers/infiniband/hw/erdma/erdma.h index b5c258f77ca0..f3b648dcf4b6 100644 --- a/drivers/infiniband/hw/erdma/erdma.h +++ b/drivers/infiniband/hw/erdma/erdma.h @@ -275,7 +275,8 @@ void notify_eq(struct erdma_eq *eq); void *get_next_valid_eqe(struct erdma_eq *eq); int erdma_aeq_init(struct erdma_dev *dev); -void erdma_aeq_destroy(struct erdma_dev *dev); +int erdma_eq_common_init(struct erdma_dev *dev, struct erdma_eq *eq, u32 depth); +void erdma_eq_destroy(struct erdma_dev *dev, struct erdma_eq *eq); void erdma_aeq_event_handler(struct erdma_dev *dev); void erdma_ceq_completion_handler(struct erdma_eq_cb *ceq_cb); diff --git a/drivers/infiniband/hw/erdma/erdma_cmdq.c b/drivers/infiniband/hw/erdma/erdma_cmdq.c index 43ff40b5a09d..a3d8922d1ad1 100644 --- a/drivers/infiniband/hw/erdma/erdma_cmdq.c +++ b/drivers/infiniband/hw/erdma/erdma_cmdq.c @@ -158,20 +158,13 @@ static int erdma_cmdq_eq_init(struct erdma_dev *dev) { struct erdma_cmdq *cmdq = &dev->cmdq; struct erdma_eq *eq = &cmdq->eq; + int ret; - eq->depth = cmdq->max_outstandings; - eq->qbuf = dma_alloc_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, - &eq->qbuf_dma_addr, GFP_KERNEL); - if (!eq->qbuf) - return -ENOMEM; - - spin_lock_init(&eq->lock); - atomic64_set(&eq->event_num, 0); + ret = erdma_eq_common_init(dev, eq, cmdq->max_outstandings); + if (ret) + return ret; eq->db = dev->func_bar + ERDMA_REGS_CEQ_DB_BASE_REG; - eq->dbrec = dma_pool_zalloc(dev->db_pool, GFP_KERNEL, &eq->dbrec_dma); - if (!eq->dbrec) - goto err_out; erdma_reg_write32(dev, ERDMA_REGS_CMDQ_EQ_ADDR_H_REG, upper_32_bits(eq->qbuf_dma_addr)); @@ -181,12 +174,6 @@ static int erdma_cmdq_eq_init(struct erdma_dev *dev) erdma_reg_write64(dev, ERDMA_CMDQ_EQ_DB_HOST_ADDR_REG, eq->dbrec_dma); return 0; - -err_out: - dma_free_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, eq->qbuf, - eq->qbuf_dma_addr); - - return -ENOMEM; } int erdma_cmdq_init(struct erdma_dev *dev) @@ -247,10 +234,7 @@ void erdma_cmdq_destroy(struct erdma_dev *dev) clear_bit(ERDMA_CMDQ_STATE_OK_BIT, &cmdq->state); - dma_free_coherent(&dev->pdev->dev, cmdq->eq.depth << EQE_SHIFT, - cmdq->eq.qbuf, cmdq->eq.qbuf_dma_addr); - - dma_pool_free(dev->db_pool, cmdq->eq.dbrec, cmdq->eq.dbrec_dma); + erdma_eq_destroy(dev, &cmdq->eq); dma_free_coherent(&dev->pdev->dev, cmdq->sq.depth << SQEBB_SHIFT, cmdq->sq.qbuf, cmdq->sq.qbuf_dma_addr); diff --git a/drivers/infiniband/hw/erdma/erdma_eq.c b/drivers/infiniband/hw/erdma/erdma_eq.c index 84ccdd8144c9..227c742c43df 100644 --- a/drivers/infiniband/hw/erdma/erdma_eq.c +++ b/drivers/infiniband/hw/erdma/erdma_eq.c @@ -80,25 +80,53 @@ void erdma_aeq_event_handler(struct erdma_dev *dev) notify_eq(&dev->aeq); } -int erdma_aeq_init(struct erdma_dev *dev) +int erdma_eq_common_init(struct erdma_dev *dev, struct erdma_eq *eq, u32 depth) { - struct erdma_eq *eq = &dev->aeq; - - eq->depth = ERDMA_DEFAULT_EQ_DEPTH; + u32 buf_size = depth << EQE_SHIFT; - eq->qbuf = dma_alloc_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, - &eq->qbuf_dma_addr, GFP_KERNEL); + eq->qbuf = dma_alloc_coherent(&dev->pdev->dev, buf_size, + &eq->qbuf_dma_addr, + GFP_KERNEL | __GFP_ZERO); if (!eq->qbuf) return -ENOMEM; + eq->dbrec = dma_pool_alloc(dev->db_pool, GFP_KERNEL | __GFP_ZERO, + &eq->dbrec_dma); + if (!eq->dbrec) + goto err_free_qbuf; + spin_lock_init(&eq->lock); atomic64_set(&eq->event_num, 0); atomic64_set(&eq->notify_num, 0); + eq->ci = 0; + eq->depth = depth; + + return 0; + +err_free_qbuf: + dma_free_coherent(&dev->pdev->dev, buf_size, eq->qbuf, + eq->qbuf_dma_addr); + + return -ENOMEM; +} + +void erdma_eq_destroy(struct erdma_dev *dev, struct erdma_eq *eq) +{ + dma_pool_free(dev->db_pool, eq->dbrec, eq->dbrec_dma); + dma_free_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, eq->qbuf, + eq->qbuf_dma_addr); +} + +int erdma_aeq_init(struct erdma_dev *dev) +{ + struct erdma_eq *eq = &dev->aeq; + int ret; + + ret = erdma_eq_common_init(dev, &dev->aeq, ERDMA_DEFAULT_EQ_DEPTH); + if (ret) + return ret; eq->db = dev->func_bar + ERDMA_REGS_AEQ_DB_REG; - eq->dbrec = dma_pool_zalloc(dev->db_pool, GFP_KERNEL, &eq->dbrec_dma); - if (!eq->dbrec) - goto err_out; erdma_reg_write32(dev, ERDMA_REGS_AEQ_ADDR_H_REG, upper_32_bits(eq->qbuf_dma_addr)); @@ -108,12 +136,6 @@ int erdma_aeq_init(struct erdma_dev *dev) erdma_reg_write64(dev, ERDMA_AEQ_DB_HOST_ADDR_REG, eq->dbrec_dma); return 0; - -err_out: - dma_free_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, eq->qbuf, - eq->qbuf_dma_addr); - - return -ENOMEM; } void erdma_aeq_destroy(struct erdma_dev *dev) @@ -234,32 +256,21 @@ static int erdma_ceq_init_one(struct erdma_dev *dev, u16 ceqn) struct erdma_eq *eq = &dev->ceqs[ceqn].eq; int ret; - eq->depth = ERDMA_DEFAULT_EQ_DEPTH; - eq->qbuf = dma_alloc_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, - &eq->qbuf_dma_addr, GFP_KERNEL); - if (!eq->qbuf) - return -ENOMEM; - - spin_lock_init(&eq->lock); - atomic64_set(&eq->event_num, 0); - atomic64_set(&eq->notify_num, 0); + ret = erdma_eq_common_init(dev, eq, ERDMA_DEFAULT_EQ_DEPTH); + if (ret) + return ret; eq->db = dev->func_bar + ERDMA_REGS_CEQ_DB_BASE_REG + (ceqn + 1) * ERDMA_DB_SIZE; - - eq->dbrec = dma_pool_zalloc(dev->db_pool, GFP_KERNEL, &eq->dbrec_dma); - if (!eq->dbrec) { - dma_free_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, - eq->qbuf, eq->qbuf_dma_addr); - return -ENOMEM; - } - - eq->ci = 0; dev->ceqs[ceqn].dev = dev; + dev->ceqs[ceqn].ready = true; /* CEQ indexed from 1, 0 rsvd for CMDQ-EQ. */ ret = create_eq_cmd(dev, ceqn + 1, eq); - dev->ceqs[ceqn].ready = ret ? false : true; + if (ret) { + erdma_eq_destroy(dev, eq); + dev->ceqs[ceqn].ready = false; + } return ret; } @@ -283,9 +294,7 @@ static void erdma_ceq_uninit_one(struct erdma_dev *dev, u16 ceqn) if (err) return; - dma_free_coherent(&dev->pdev->dev, eq->depth << EQE_SHIFT, eq->qbuf, - eq->qbuf_dma_addr); - dma_pool_free(dev->db_pool, eq->dbrec, eq->dbrec_dma); + erdma_eq_destroy(dev, eq); } int erdma_ceqs_init(struct erdma_dev *dev) diff --git a/drivers/infiniband/hw/erdma/erdma_main.c b/drivers/infiniband/hw/erdma/erdma_main.c index 9199058a0b29..d1cb488e7ad4 100644 --- a/drivers/infiniband/hw/erdma/erdma_main.c +++ b/drivers/infiniband/hw/erdma/erdma_main.c @@ -359,7 +359,7 @@ static int erdma_probe_dev(struct pci_dev *pdev) erdma_cmdq_destroy(dev); err_uninit_aeq: - erdma_aeq_destroy(dev); + erdma_eq_destroy(dev, &dev->aeq); err_uninit_comm_irq: erdma_comm_irq_uninit(dev); @@ -392,7 +392,7 @@ static void erdma_remove_dev(struct pci_dev *pdev) erdma_ceqs_uninit(dev); erdma_hw_reset(dev, false); erdma_cmdq_destroy(dev); - erdma_aeq_destroy(dev); + erdma_eq_destroy(dev, &dev->aeq); erdma_comm_irq_uninit(dev); pci_free_irq_vectors(dev->pdev); erdma_device_uninit(dev); From patchwork Fri Aug 23 07:50:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng Xu X-Patchwork-Id: 13774727 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25E1013D8A0 for ; Fri, 23 Aug 2024 07:51:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399476; cv=none; b=iGxIz3zXkz1fJYWkR8zUR+qPPYbaUmm81USgTJViYgpzKWLtCxTkXASmPRftfi1HpnEXWqaZFLml8PGqv2iu4xJEdjuM3ReBt5n2nyrgvrhx4iahdR4RLOggvIByZDNtmywJgFWswLiFD9r8MU2EZy8RpiHgXrpptTOParI03jw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399476; c=relaxed/simple; bh=ckTvPKiI5QEhksb4cEarFV1MS9B0zHPtT2TxBBKjiVg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=poG8m+PThsML2LN6Qde94t9NEo8WW8rQqno3ktDYlZ0XeWddzH9bW7cDUwLFPKb+UuEB6jM7gbyvQHIOGdLp6gxkuDvsEtbxpyLJi5wluhsnG7m8rY+9ny7FmYNJoQdYivtlw8KrubLCQ0vMcHKZKD8JuV4807SSNJ45cZSl2+w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=SzPwrroG; arc=none smtp.client-ip=115.124.30.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="SzPwrroG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724399465; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=ILEVsIxtwCVK9Ex+2VK/+fPKofqndLxjLXuNjQIFhMc=; b=SzPwrroGm/NY92q/A1qIE055umEozTqvewdwLtYS+8vlwAXOMZLRMtDKb2A63C/oPUKKAk0HDdnrW5oBtWl5dz64k0kZozoG+WF3WZFKX7hixu/8yisQ9sOgqe/ZN8WfvRdJ81kInGsz6mZo2GuPeCSGQHm4Amvd/qDG/3Ya1Zg= Received: from localhost(mailfrom:chengyou@linux.alibaba.com fp:SMTPD_---0WDSmI8._1724399464) by smtp.aliyun-inc.com; Fri, 23 Aug 2024 15:51:04 +0800 From: Cheng Xu To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, KaiShen@linux.alibaba.com Subject: [PATCH for-next 3/4] RDMA/erdma: Add disassociate ucontext support Date: Fri, 23 Aug 2024 15:50:57 +0800 Message-Id: <20240823075058.89488-4-chengyou@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240823075058.89488-1-chengyou@linux.alibaba.com> References: <20240823075058.89488-1-chengyou@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 All IO pages mapped to user space are handled by rdma_user_mmap_io, so add empty stub for disassociate ucontext. Signed-off-by: Cheng Xu --- drivers/infiniband/hw/erdma/erdma_main.c | 1 + drivers/infiniband/hw/erdma/erdma_verbs.c | 4 ++++ drivers/infiniband/hw/erdma/erdma_verbs.h | 1 + 3 files changed, 6 insertions(+) diff --git a/drivers/infiniband/hw/erdma/erdma_main.c b/drivers/infiniband/hw/erdma/erdma_main.c index d1cb488e7ad4..1ccf1b65c02c 100644 --- a/drivers/infiniband/hw/erdma/erdma_main.c +++ b/drivers/infiniband/hw/erdma/erdma_main.c @@ -516,6 +516,7 @@ static const struct ib_device_ops erdma_device_ops = { .dereg_mr = erdma_dereg_mr, .destroy_cq = erdma_destroy_cq, .destroy_qp = erdma_destroy_qp, + .disassociate_ucontext = erdma_disassociate_ucontext, .get_dma_mr = erdma_get_dma_mr, .get_hw_stats = erdma_get_hw_stats, .get_port_immutable = erdma_get_port_immutable, diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.c b/drivers/infiniband/hw/erdma/erdma_verbs.c index 40c9b6e46b82..48b08a15e6a8 100644 --- a/drivers/infiniband/hw/erdma/erdma_verbs.c +++ b/drivers/infiniband/hw/erdma/erdma_verbs.c @@ -1700,6 +1700,10 @@ int erdma_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, return ret; } +void erdma_disassociate_ucontext(struct ib_ucontext *ibcontext) +{ +} + void erdma_set_mtu(struct erdma_dev *dev, u32 mtu) { struct erdma_cmdq_config_mtu_req req; diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.h b/drivers/infiniband/hw/erdma/erdma_verbs.h index 4f02ba06b210..b7478376eb80 100644 --- a/drivers/infiniband/hw/erdma/erdma_verbs.h +++ b/drivers/infiniband/hw/erdma/erdma_verbs.h @@ -344,6 +344,7 @@ int erdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int mask, struct ib_udata *data); int erdma_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata); int erdma_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata); +void erdma_disassociate_ucontext(struct ib_ucontext *ibcontext); int erdma_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len, u64 virt, int access, struct ib_udata *udata); From patchwork Fri Aug 23 07:50:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng Xu X-Patchwork-Id: 13774728 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5226313FD99 for ; Fri, 23 Aug 2024 07:51:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.118 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399477; cv=none; b=tCQ4LT7TpewD4oZdOSApdGXHMBfFctVA+oIgtcSKAEOwwavM25sU81t7trvT1leSMJ1XLA9EiH1XFeFrj5vpWLd5XyFn4+8hl0WNJOUtFkU+2LR3CYENnEpUEpLxrlnTizpLZnii6Fnm6dNxqfFaMwDrRLprQ2TftYgbuBUVJNc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724399477; c=relaxed/simple; bh=atrAcBXqGYOwK9IhJ5JVPvzdvvQ4IF74G4Aivl+S04U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WkP01ahbBtzz+M1SDboDkmYIzf7s8HTeNjMZ+OdnV3AZNXjw6N9goDRmZp8Kyakxn0w9dVFcbEl01cmSqljQma+wu3OOnThP3uMPUhMyRqWCb2bsOVxq1bcXJ8Aqa4dP2GfTH6a4QtjP5YTt1q3njy5DRP1Ay7XYZf8aOeJVWbM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=L2dmY7os; arc=none smtp.client-ip=115.124.30.118 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="L2dmY7os" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724399466; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=PKiV/AnZhD8XWBkIWGSe+NiegrdQamtp1efkIRSVEOQ=; b=L2dmY7osU93HlKIKa2lxC+BmhXfjox8d+8spy17xc3jwuW/jPaiAdUgXf4RrE7+1zgj3lyFtlB4EHzMrDwf/h2bEoIstwKi8pjmoQICiWeDksbfD/CQ06GiaLJG8Y/ANyOxlrCxtYybr0yaUqJp8c9vCdWgXGXnkS3HAg5LMyG8= Received: from localhost(mailfrom:chengyou@linux.alibaba.com fp:SMTPD_---0WDSj-UC_1724399465) by smtp.aliyun-inc.com; Fri, 23 Aug 2024 15:51:06 +0800 From: Cheng Xu To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, KaiShen@linux.alibaba.com Subject: [PATCH for-next 4/4] RDMA/erdma: Return QP state in erdma_query_qp Date: Fri, 23 Aug 2024 15:50:58 +0800 Message-Id: <20240823075058.89488-5-chengyou@linux.alibaba.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240823075058.89488-1-chengyou@linux.alibaba.com> References: <20240823075058.89488-1-chengyou@linux.alibaba.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Fix qp_state and cur_qp_state to return correct values in struct ib_qp_attr. Fixes: 155055771704 ("RDMA/erdma: Add verbs implementation") Signed-off-by: Cheng Xu --- drivers/infiniband/hw/erdma/erdma_verbs.c | 25 ++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.c b/drivers/infiniband/hw/erdma/erdma_verbs.c index 48b08a15e6a8..de11f0f1adb1 100644 --- a/drivers/infiniband/hw/erdma/erdma_verbs.c +++ b/drivers/infiniband/hw/erdma/erdma_verbs.c @@ -1544,11 +1544,31 @@ int erdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask, return ret; } +static inline enum ib_qp_state query_qp_state(struct erdma_qp *qp) +{ + switch (qp->attrs.state) { + case ERDMA_QP_STATE_IDLE: + return IB_QPS_INIT; + case ERDMA_QP_STATE_RTR: + return IB_QPS_RTR; + case ERDMA_QP_STATE_RTS: + return IB_QPS_RTS; + case ERDMA_QP_STATE_CLOSING: + return IB_QPS_ERR; + case ERDMA_QP_STATE_TERMINATE: + return IB_QPS_ERR; + case ERDMA_QP_STATE_ERROR: + return IB_QPS_ERR; + default: + return IB_QPS_ERR; + } +} + int erdma_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr) { - struct erdma_qp *qp; struct erdma_dev *dev; + struct erdma_qp *qp; if (ibqp && qp_attr && qp_init_attr) { qp = to_eqp(ibqp); @@ -1575,6 +1595,9 @@ int erdma_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, qp_init_attr->cap = qp_attr->cap; + qp_attr->qp_state = query_qp_state(qp); + qp_attr->cur_qp_state = query_qp_state(qp); + return 0; }