diff mbox

[net] qed: Free RoCE ILT Memory on rmmod qedr

Message ID 1520240651-11308-1-git-send-email-Michal.Kalderon@cavium.com (mailing list archive)
State Superseded
Headers show

Commit Message

Kalderon, Michal March 5, 2018, 9:04 a.m. UTC
Rdma requires ILT Memory to be allocated for it's QPs.
Each ILT entry points to a page used by several Rdma QPs.
To avoid allocating all the memory in advance, the rdma
implementation dynamically allocates memory as more QPs are
added, however it does not dynamically free the memory.
The memory should have been freed on rmmod qedr, but isn't.
This patch adds the memory freeing on rmmod qedr (currently
it will be freed with qed is removed).

An outcome of this bug, is that if qedr is unloaded and loaded
without unloaded qed, there will be no more RoCE traffic.

The reason these are related, is that the logic of detecting the
first QP ever opened is by asking whether ILT memory for RoCE has
been allocated.

In addition, this patch modifies freeing of the Task context to
always use the PROTOCOLID_ROCE and not the protocol passed,
this is because task context for iWARP and ROCE both use the
ROCE protocol id, as opposed to the connection context.

Fixes: dbb799c39717e7b7
Signed-off-by: Michal Kalderon <Michal.Kalderon@cavium.com>
Signed-off-by: Ariel Elior <Ariel.Elior@cavium.com>
---
 drivers/net/ethernet/qlogic/qed/qed_cxt.c  | 5 ++++-
 drivers/net/ethernet/qlogic/qed/qed_rdma.c | 1 +
 2 files changed, 5 insertions(+), 1 deletion(-)

Comments

Leon Romanovsky March 5, 2018, 9:41 a.m. UTC | #1
On Mon, Mar 05, 2018 at 11:04:11AM +0200, Michal Kalderon wrote:
> Rdma requires ILT Memory to be allocated for it's QPs.
> Each ILT entry points to a page used by several Rdma QPs.
> To avoid allocating all the memory in advance, the rdma
> implementation dynamically allocates memory as more QPs are
> added, however it does not dynamically free the memory.
> The memory should have been freed on rmmod qedr, but isn't.
> This patch adds the memory freeing on rmmod qedr (currently
> it will be freed with qed is removed).
>
> An outcome of this bug, is that if qedr is unloaded and loaded
> without unloaded qed, there will be no more RoCE traffic.
>
> The reason these are related, is that the logic of detecting the
> first QP ever opened is by asking whether ILT memory for RoCE has
> been allocated.
>
> In addition, this patch modifies freeing of the Task context to
> always use the PROTOCOLID_ROCE and not the protocol passed,
> this is because task context for iWARP and ROCE both use the
> ROCE protocol id, as opposed to the connection context.
>
> Fixes: dbb799c39717e7b7

Your fixes line was truncated.

Thanks
Kalderon, Michal March 5, 2018, 9:50 a.m. UTC | #2
> From: Leon Romanovsky [mailto:leon@kernel.org]
> Sent: Monday, March 05, 2018 11:41 AM
> To: Kalderon, Michal <Michal.Kalderon@cavium.com>
> Cc: davem@davemloft.net; netdev@vger.kernel.org; dledford@redhat.com;
> jgg@mellanox.com; linux-rdma@vger.kernel.org; Elior, Ariel
> <Ariel.Elior@cavium.com>
> Subject: Re: [PATCH net] qed: Free RoCE ILT Memory on rmmod qedr
> 
> On Mon, Mar 05, 2018 at 11:04:11AM +0200, Michal Kalderon wrote:
> > Rdma requires ILT Memory to be allocated for it's QPs.
> > Each ILT entry points to a page used by several Rdma QPs.
> > To avoid allocating all the memory in advance, the rdma
> > implementation dynamically allocates memory as more QPs are
> > added, however it does not dynamically free the memory.
> > The memory should have been freed on rmmod qedr, but isn't.
> > This patch adds the memory freeing on rmmod qedr (currently
> > it will be freed with qed is removed).
> >
> > An outcome of this bug, is that if qedr is unloaded and loaded
> > without unloaded qed, there will be no more RoCE traffic.
> >
> > The reason these are related, is that the logic of detecting the
> > first QP ever opened is by asking whether ILT memory for RoCE has
> > been allocated.
> >
> > In addition, this patch modifies freeing of the Task context to
> > always use the PROTOCOLID_ROCE and not the protocol passed,
> > this is because task context for iWARP and ROCE both use the
> > ROCE protocol id, as opposed to the connection context.
> >
> > Fixes: dbb799c39717e7b7
> 
> Your fixes line was truncated.
> 
> Thanks
Thanks  for noticing, sending V2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/net/ethernet/qlogic/qed/qed_cxt.c b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
index 6f546e8..b6f55bc 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_cxt.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
@@ -2480,7 +2480,10 @@  int qed_cxt_free_proto_ilt(struct qed_hwfn *p_hwfn, enum protocol_type proto)
 	if (rc)
 		return rc;
 
-	/* Free Task CXT */
+	/* Free Task CXT ( Intentionally RoCE as task-id is shared between
+	 * RoCE and iWARP
+	 */
+	proto = PROTOCOLID_ROCE;
 	rc = qed_cxt_free_ilt_range(p_hwfn, QED_ELEM_TASK, 0,
 				    qed_cxt_get_proto_tid_count(p_hwfn, proto));
 	if (rc)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
index 5d040b8..f3ee653 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
@@ -380,6 +380,7 @@  static void qed_rdma_free(struct qed_hwfn *p_hwfn)
 
 	qed_rdma_free_reserved_lkey(p_hwfn);
 	qed_rdma_resc_free(p_hwfn);
+	qed_cxt_free_proto_ilt(p_hwfn, p_hwfn->p_rdma_info->proto);
 }
 
 static void qed_rdma_get_guid(struct qed_hwfn *p_hwfn, u8 *guid)