From patchwork Sun Jan 19 17:28:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13944548 Received: from out-171.mta0.migadu.com (out-171.mta0.migadu.com [91.218.175.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29B041DE3C0 for ; Sun, 19 Jan 2025 17:28:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737307731; cv=none; b=WIqcF+YJBW4XADkXwOrs94L92kMzClkfAeSN7pN3KA+TQMgDDT+kalr39J2YQtAA77xj2MgaGJaiwEJAkrmF4QWPwSxMBHdXqJPWOCPTKsVVUIKIhy6jo3SiS5CH+hIIRFT+Xa4lxWHSlih/U1/QB9af+YuH+1OqBId8/0NtW6c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737307731; c=relaxed/simple; bh=UYbyGJxbae39QdCHBlx82fxQ3OIJ/siHvqgNQcw55s4=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=egoBgxWRZK4kPFqZrIe0X76dXJ0mBT5+ZUPG1eZfApEjmIvTexu0EbBqx8emoF784l6hpgL3v0XIdqXzcIwNhFfexSAFTtrUtMWzOChgvFi5sDdcup9XN5laWHR8GyDD+Vl0i5LHmustU3hr4P98VdM5KDK0w+/F++PxzIXKJlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=T1aDKv2F; arc=none smtp.client-ip=91.218.175.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="T1aDKv2F" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1737307726; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oxFtoELoh2q96fSTltOY5jcoD8o3DkX8pBrT/0Q3xAg=; b=T1aDKv2FagUZqqIxdC7+9AGDYBnoR99+UvgFO4B72ScHW4lvx+04SF+bjOVn21oh4Kt80p Qxq7L7fG7rSVBK3+mFY7jAE1axcQhS5uZTQSUX914FNRtO7rDpKDhDiica+JaLHHI3kYWl nGveyEf3owJARHyI5w2CJZ8mbRYaIMw= From: Zhu Yanjun To: yanjun.zhu@linux.dev, jgg@ziepe.ca, leon@kernel.org, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Subject: [PATCH 1/3] RDMA/rxe: Replace netdev dev addr with raw_gid Date: Sun, 19 Jan 2025 18:28:29 +0100 Message-Id: <20250119172831.3123110-2-yanjun.zhu@linux.dev> In-Reply-To: <20250119172831.3123110-1-yanjun.zhu@linux.dev> References: <20250119172831.3123110-1-yanjun.zhu@linux.dev> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Because TUN device does not have dev_addr, but a gid in rdma is needed, as such, a raw_gid is generated to act as the gid. The similar commit is in SIW. This commit learns from the similar commit bad5b6e34ffb ("RDMA/siw: Fabricate a GID on tun and loopback devices") in SIW. Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 15 +++++++++++++-- drivers/infiniband/sw/rxe/rxe_verbs.c | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.h | 4 +++- 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 1ba4a0c8726a..432e54a29b99 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -78,8 +78,19 @@ static void rxe_init_device_param(struct rxe_dev *rxe) if (!ndev) return; + if (ndev->addr_len) { + memcpy(rxe->raw_gid, ndev->dev_addr, + min_t(unsigned int, ndev->addr_len, ETH_ALEN)); + } else { + /* + * This device does not have a HW address, but + * connection mangagement requires a unique gid. + */ + eth_random_addr(rxe->raw_gid); + } + addrconf_addr_eui48((unsigned char *)&rxe->attr.sys_image_guid, - ndev->dev_addr); + rxe->raw_gid); dev_put(ndev); @@ -125,7 +136,7 @@ static void rxe_init_ports(struct rxe_dev *rxe) if (!ndev) return; addrconf_addr_eui48((unsigned char *)&port->port_guid, - ndev->dev_addr); + rxe->raw_gid); dev_put(ndev); spin_lock_init(&port->port_lock); } diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 8a5fc20fd186..837566cd682c 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -1523,7 +1523,7 @@ int rxe_register_device(struct rxe_dev *rxe, const char *ibdev_name, dev->num_comp_vectors = num_possible_cpus(); dev->local_dma_lkey = 0; addrconf_addr_eui48((unsigned char *)&dev->node_guid, - ndev->dev_addr); + rxe->raw_gid); dev->uverbs_cmd_mask |= BIT_ULL(IB_USER_VERBS_CMD_POST_SEND) | BIT_ULL(IB_USER_VERBS_CMD_REQ_NOTIFY_CQ); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 6573ceec0ef5..729a6ada46af 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -376,7 +376,9 @@ struct rxe_dev { struct ib_device_attr attr; int max_ucontext; int max_inline_data; - struct mutex usdev_lock; + struct mutex usdev_lock; + + char raw_gid[ETH_ALEN]; struct rxe_pool uc_pool; struct rxe_pool pd_pool; From patchwork Sun Jan 19 17:28:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13944549 Received: from out-186.mta0.migadu.com (out-186.mta0.migadu.com [91.218.175.186]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FBD61B21AA for ; Sun, 19 Jan 2025 17:28:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.186 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737307732; cv=none; b=uLXgPtFo7bF2Sq3IVArWxeN2oe3Z4/wtnh2BVz5ldttvsLB6+LkOr3Gh0U5boD2AIXCzybH8aKxbLZOdeM1nGPISFXUlQ58O8I4ni6ZESsg5IFWsGJRmKF9J5QYsqQ6n3PRrti8dG3cdnSzJmyrI7KtWwk2Y4fRul7qqub9682A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737307732; c=relaxed/simple; bh=eLgGF/rdRLdff8SHS8qMTtB3HhlOTPEAofy+UBrU1uk=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AK2u7JM8qDCin5Bt3kAIsd3QhWajWqNOrLAsWtYEoALsqdTQJ9IGk6xSW24JBFWaVBOR4fO8T/oiO7cI2Y+vXudkg5McYFJ0DgBBN6iQ0z4nmo1PkrJJCVExamANEPJpHKppF1PJi7+/UN6GglITKh4TjKlpOpHvYI2DtxLvgLw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=FQ4D1UAW; arc=none smtp.client-ip=91.218.175.186 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="FQ4D1UAW" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1737307728; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NYVKceX/q6XRL3z/pd1rWkpqPx+frEy0JzAYOQYG/Bw=; b=FQ4D1UAWACBdFT2xLEDYQASy3BRA/JLm8AGGUViVh1mFSje1dMTGEvTeQWiUOGL6yEBU5Y x7lsl4E4vV5pAQUqnuofSjNw2774H0AstrVvzLObhe71SUeUqDa2GXSEQ2/3X08x903Bpi 8rV4Bm2crZH2RblADhuOWVIu8VN47vs= From: Zhu Yanjun To: yanjun.zhu@linux.dev, jgg@ziepe.ca, leon@kernel.org, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Subject: [PATCH 2/3] RDMA/rxe: Add query_gid support Date: Sun, 19 Jan 2025 18:28:30 +0100 Message-Id: <20250119172831.3123110-3-yanjun.zhu@linux.dev> In-Reply-To: <20250119172831.3123110-1-yanjun.zhu@linux.dev> References: <20250119172831.3123110-1-yanjun.zhu@linux.dev> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT The query_gid is not implemented in RXE. After the raw_gid is added, this query_gid should be implemented in RXE. Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe_verbs.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 837566cd682c..22d6312ecbc7 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -79,6 +79,18 @@ static int rxe_query_port(struct ib_device *ibdev, return err; } +static int rxe_query_gid(struct ib_device *ibdev, u32 port, int idx, + union ib_gid *gid) +{ + struct rxe_dev *rxe = to_rdev(ibdev); + + /* subnet_prefix == interface_id == 0; */ + memset(gid, 0, sizeof(*gid)); + memcpy(gid->raw, rxe->raw_gid, ETH_ALEN); + + return 0; +} + static int rxe_query_pkey(struct ib_device *ibdev, u32 port_num, u16 index, u16 *pkey) { @@ -1493,6 +1505,7 @@ static const struct ib_device_ops rxe_dev_ops = { .query_ah = rxe_query_ah, .query_device = rxe_query_device, .query_pkey = rxe_query_pkey, + .query_gid = rxe_query_gid, .query_port = rxe_query_port, .query_qp = rxe_query_qp, .query_srq = rxe_query_srq, From patchwork Sun Jan 19 17:28:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13944550 Received: from out-172.mta0.migadu.com (out-172.mta0.migadu.com [91.218.175.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D9F91DF27D for ; Sun, 19 Jan 2025 17:28:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737307733; cv=none; b=keIqpTQP3WUf7Zd97vRtmupmjD4MloQX2nLdzEwAOJ+XCSSnzKEWegwiWuPkcXP5gpNGUKM2XDWYhl8k/qv7SaO5cVKpsOuvTbHl8doDOAFO05gvE8w6nLMebLhiDCWPNrh9sc0ZSYJu5lS1gUxsW9ZeuKAJWy0OYQy5vhcKyDo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737307733; c=relaxed/simple; bh=OBFEUoexhOVTYJLFAprcRGF5Hj49H1Lk4VDuruGhOgg=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UzxgNs9JPY2xDv7+1wFCqxxk39mjsOQu/eKfWGzW5NR5me0Y7jpaIL4YXRxWZ5mhL5RkePLUZ1UQ34ZlBbrmuyuLot9QfOtU0kDc0xJ28EBRJiyW2zFfFHNhc3qLDbFgTRLKf1v6k5IIAw67onTY9pa/Tnxw5gf7CfNZOAguwwg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=CMR6NlXe; arc=none smtp.client-ip=91.218.175.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="CMR6NlXe" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1737307729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2LW50jnrPSPan2lCmyQjpi/v5afkYSe9p/W+xJi8C/8=; b=CMR6NlXe2b/ZlzuZQ/+cdulAUbfu1/68ZtOetdPTFIKg3NE0UigLhk4GJ+kEa9OHeo5cuU kjuwNujakIdrSssrRcBdWatgwuKD6VcxpSSE9xBPHDZnE1lkS7091043aGPGahkYan1AfM LLjxTl0sFQXJPwnPBtaNkt3BsRtNgoQ= From: Zhu Yanjun To: yanjun.zhu@linux.dev, jgg@ziepe.ca, leon@kernel.org, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Subject: [PATCH 3/3] RDMA/rxe: Make rping work with tun device Date: Sun, 19 Jan 2025 18:28:31 +0100 Message-Id: <20250119172831.3123110-4-yanjun.zhu@linux.dev> In-Reply-To: <20250119172831.3123110-1-yanjun.zhu@linux.dev> References: <20250119172831.3123110-1-yanjun.zhu@linux.dev> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Because the type of tun device is ARPHRD_NONE, in cma, ARPHRD_NONE is not recognized. To RXE device, just as SIW does, ARPHRD_NONE is added to support. Signed-off-by: Zhu Yanjun --- drivers/infiniband/core/cma.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 91db10515d74..fedcdb56fb6b 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -739,12 +739,26 @@ cma_validate_port(struct ib_device *device, u32 port, goto out; } - if (dev_type == ARPHRD_ETHER && rdma_protocol_roce(device, port)) { - ndev = dev_get_by_index(dev_addr->net, bound_if_index); - if (!ndev) - goto out; + /* + * For a RXE device, it should work with TUN device and normal ethernet + * devices. Use driver_id to check if a device is a RXE device or not. + * ARPHDR_NONE means a TUN device. + */ + if (device->ops.driver_id == RDMA_DRIVER_RXE) { + if ((dev_type == ARPHRD_NONE || dev_type == ARPHRD_ETHER) + && rdma_protocol_roce(device, port)) { + ndev = dev_get_by_index(dev_addr->net, bound_if_index); + if (!ndev) + goto out; + } } else { - gid_type = IB_GID_TYPE_IB; + if (dev_type == ARPHRD_ETHER && rdma_protocol_roce(device, port)) { + ndev = dev_get_by_index(dev_addr->net, bound_if_index); + if (!ndev) + goto out; + } else { + gid_type = IB_GID_TYPE_IB; + } } sgid_attr = rdma_find_gid_by_port(device, gid, gid_type, port, ndev);