| Message ID | 20190526114156.6827-1-bmt@zurich.ibm.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | SIW: Software iWarp RDMA (siw) driver | expand |
-----linux-rdma-owner@vger.kernel.org wrote: ----- >To: linux-rdma@vger.kernel.org >From: "Bernard Metzler" >Sent by: linux-rdma-owner@vger.kernel.org >Date: 05/26/2019 01:42PM >Cc: "Bernard Metzler" <bmt@zurich.ibm.com> >Subject: [EXTERNAL] [PATCH for-next v1 00/12] SIW: Software iWarp >RDMA (siw) driver > >This patch set contributes the SoftiWarp driver rebased for >Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA >protocol over kernel TCP sockets. The driver integrates with >the linux-rdma framework. > >With this new driver version, the following things where >changed, compared to the v8 RFC of siw: > >o Rebased to 5.2-rc1 > >o All IDR code got removed. > >o Both MR and QP deallocation verbs now synchronously > free the resources referenced by the RDMA mid-layer. > >o IPv6 support was added. > >o For compatibility with Chelsio iWarp hardware, the RX > path was slightly reworked. It now allows packet intersection > between tagged and untagged RDMAP operations. While not > a defined behavior as of IETF RFC 5040/5041, some RDMA hardware > may intersect an ongoing outbound (large) tagged message, such > as an multisegment RDMA Read Response with sending an untagged > message, such as an RDMA Send frame. This behavior was only > detected in an NVMeF setup, where siw was used at target side, > and RDMA hardware at client side (during file write). siw now > implements two input paths for tagged and untagged messages each, > and allows the intersected placement of both messages. > >o The siw kernel abi file got renamed from siw_user.h to siw-abi.h. > >Many thanks for reviewing and testing the driver, especially to >Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to >significantly improve the siw driver over the last year. It is >very much appreciated. > >Many thanks! >Bernard. > >Bernard Metzler (12): > iWarp wire packet format > SIW main include file > SIW network and RDMA core interface > SIW connection management > SIW application interface > SIW application buffer management > SIW queue pair methods > SIW transmit path > SIW receive path > SIW completion queue methods > SIW debugging > SIW addition to kernel build environment > > MAINTAINERS | 7 + > drivers/infiniband/Kconfig | 1 + > drivers/infiniband/sw/Makefile | 1 + > drivers/infiniband/sw/siw/Kconfig | 17 + > drivers/infiniband/sw/siw/Makefile | 12 + > drivers/infiniband/sw/siw/iwarp.h | 380 ++++ > drivers/infiniband/sw/siw/siw.h | 720 ++++++++ > drivers/infiniband/sw/siw/siw_cm.c | 2109 >++++++++++++++++++++++ > drivers/infiniband/sw/siw/siw_cm.h | 133 ++ > drivers/infiniband/sw/siw/siw_cq.c | 109 ++ > drivers/infiniband/sw/siw/siw_debug.c | 102 ++ > drivers/infiniband/sw/siw/siw_debug.h | 35 + > drivers/infiniband/sw/siw/siw_main.c | 701 +++++++ > drivers/infiniband/sw/siw/siw_mem.c | 462 +++++ > drivers/infiniband/sw/siw/siw_mem.h | 74 + > drivers/infiniband/sw/siw/siw_qp.c | 1345 ++++++++++++++ > drivers/infiniband/sw/siw/siw_qp_rx.c | 1537 ++++++++++++++++ > drivers/infiniband/sw/siw/siw_qp_tx.c | 1276 +++++++++++++ > drivers/infiniband/sw/siw/siw_verbs.c | 1778 ++++++++++++++++++ > drivers/infiniband/sw/siw/siw_verbs.h | 102 ++ > include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 + > include/uapi/rdma/siw-abi.h | 186 ++ > 22 files changed, 11088 insertions(+) > create mode 100644 drivers/infiniband/sw/siw/Kconfig > create mode 100644 drivers/infiniband/sw/siw/Makefile > create mode 100644 drivers/infiniband/sw/siw/iwarp.h > create mode 100644 drivers/infiniband/sw/siw/siw.h > create mode 100644 drivers/infiniband/sw/siw/siw_cm.c > create mode 100644 drivers/infiniband/sw/siw/siw_cm.h > create mode 100644 drivers/infiniband/sw/siw/siw_cq.c > create mode 100644 drivers/infiniband/sw/siw/siw_debug.c > create mode 100644 drivers/infiniband/sw/siw/siw_debug.h > create mode 100644 drivers/infiniband/sw/siw/siw_main.c > create mode 100644 drivers/infiniband/sw/siw/siw_mem.c > create mode 100644 drivers/infiniband/sw/siw/siw_mem.h > create mode 100644 drivers/infiniband/sw/siw/siw_qp.c > create mode 100644 drivers/infiniband/sw/siw/siw_qp_rx.c > create mode 100644 drivers/infiniband/sw/siw/siw_qp_tx.c > create mode 100644 drivers/infiniband/sw/siw/siw_verbs.c > create mode 100644 drivers/infiniband/sw/siw/siw_verbs.h > create mode 100644 include/uapi/rdma/siw-abi.h > >-- >2.17.2 > > Hi Jason, Leon, Steve, @all, What's next for getting siw merged? Please help me to keep the ball rolling. I am currently running out of issues I shall fix (which is not a bad feeling though ;)). I see lots of other demanding stuff is going on these days... Thanks very much! Bernard
On Wed, May 29, 2019 at 03:56:57PM +0000, Bernard Metzler wrote: > -----linux-rdma-owner@vger.kernel.org wrote: ----- > > >To: linux-rdma@vger.kernel.org > >From: "Bernard Metzler" > >Sent by: linux-rdma-owner@vger.kernel.org > >Date: 05/26/2019 01:42PM > >Cc: "Bernard Metzler" <bmt@zurich.ibm.com> > >Subject: [EXTERNAL] [PATCH for-next v1 00/12] SIW: Software iWarp > >RDMA (siw) driver > > > >This patch set contributes the SoftiWarp driver rebased for > >Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA > >protocol over kernel TCP sockets. The driver integrates with > >the linux-rdma framework. > > > >With this new driver version, the following things where > >changed, compared to the v8 RFC of siw: > > > >o Rebased to 5.2-rc1 > > > >o All IDR code got removed. > > > >o Both MR and QP deallocation verbs now synchronously > > free the resources referenced by the RDMA mid-layer. > > > >o IPv6 support was added. > > > >o For compatibility with Chelsio iWarp hardware, the RX > > path was slightly reworked. It now allows packet intersection > > between tagged and untagged RDMAP operations. While not > > a defined behavior as of IETF RFC 5040/5041, some RDMA hardware > > may intersect an ongoing outbound (large) tagged message, such > > as an multisegment RDMA Read Response with sending an untagged > > message, such as an RDMA Send frame. This behavior was only > > detected in an NVMeF setup, where siw was used at target side, > > and RDMA hardware at client side (during file write). siw now > > implements two input paths for tagged and untagged messages each, > > and allows the intersected placement of both messages. > > > >o The siw kernel abi file got renamed from siw_user.h to siw-abi.h. > > > >Many thanks for reviewing and testing the driver, especially to > >Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to > >significantly improve the siw driver over the last year. It is > >very much appreciated. > > > >Many thanks! > >Bernard. > > > >Bernard Metzler (12): > > iWarp wire packet format > > SIW main include file > > SIW network and RDMA core interface > > SIW connection management > > SIW application interface > > SIW application buffer management > > SIW queue pair methods > > SIW transmit path > > SIW receive path > > SIW completion queue methods > > SIW debugging > > SIW addition to kernel build environment > > > > MAINTAINERS | 7 + > > drivers/infiniband/Kconfig | 1 + > > drivers/infiniband/sw/Makefile | 1 + > > drivers/infiniband/sw/siw/Kconfig | 17 + > > drivers/infiniband/sw/siw/Makefile | 12 + > > drivers/infiniband/sw/siw/iwarp.h | 380 ++++ > > drivers/infiniband/sw/siw/siw.h | 720 ++++++++ > > drivers/infiniband/sw/siw/siw_cm.c | 2109 > >++++++++++++++++++++++ > > drivers/infiniband/sw/siw/siw_cm.h | 133 ++ > > drivers/infiniband/sw/siw/siw_cq.c | 109 ++ > > drivers/infiniband/sw/siw/siw_debug.c | 102 ++ > > drivers/infiniband/sw/siw/siw_debug.h | 35 + > > drivers/infiniband/sw/siw/siw_main.c | 701 +++++++ > > drivers/infiniband/sw/siw/siw_mem.c | 462 +++++ > > drivers/infiniband/sw/siw/siw_mem.h | 74 + > > drivers/infiniband/sw/siw/siw_qp.c | 1345 ++++++++++++++ > > drivers/infiniband/sw/siw/siw_qp_rx.c | 1537 ++++++++++++++++ > > drivers/infiniband/sw/siw/siw_qp_tx.c | 1276 +++++++++++++ > > drivers/infiniband/sw/siw/siw_verbs.c | 1778 ++++++++++++++++++ > > drivers/infiniband/sw/siw/siw_verbs.h | 102 ++ > > include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 + > > include/uapi/rdma/siw-abi.h | 186 ++ > > 22 files changed, 11088 insertions(+) > > create mode 100644 drivers/infiniband/sw/siw/Kconfig > > create mode 100644 drivers/infiniband/sw/siw/Makefile > > create mode 100644 drivers/infiniband/sw/siw/iwarp.h > > create mode 100644 drivers/infiniband/sw/siw/siw.h > > create mode 100644 drivers/infiniband/sw/siw/siw_cm.c > > create mode 100644 drivers/infiniband/sw/siw/siw_cm.h > > create mode 100644 drivers/infiniband/sw/siw/siw_cq.c > > create mode 100644 drivers/infiniband/sw/siw/siw_debug.c > > create mode 100644 drivers/infiniband/sw/siw/siw_debug.h > > create mode 100644 drivers/infiniband/sw/siw/siw_main.c > > create mode 100644 drivers/infiniband/sw/siw/siw_mem.c > > create mode 100644 drivers/infiniband/sw/siw/siw_mem.h > > create mode 100644 drivers/infiniband/sw/siw/siw_qp.c > > create mode 100644 drivers/infiniband/sw/siw/siw_qp_rx.c > > create mode 100644 drivers/infiniband/sw/siw/siw_qp_tx.c > > create mode 100644 drivers/infiniband/sw/siw/siw_verbs.c > > create mode 100644 drivers/infiniband/sw/siw/siw_verbs.h > > create mode 100644 include/uapi/rdma/siw-abi.h > > > >-- > >2.17.2 > > > > > > Hi Jason, Leon, Steve, @all, > > What's next for getting siw merged? Please help me to > keep the ball rolling. I am currently running out of > issues I shall fix (which is not a bad feeling though ;)). > I see lots of other demanding stuff is going on > these days... Generally speaking, I think that it is ready to be merged. If Jason/Doug doesn't merge this merge before next week, I'll take an extra look and add my ROBs next week, but it is definitely not a blocker for acceptance. Thanks > > Thanks very much! > Bernard >
-----"Leon Romanovsky" <leon@kernel.org> wrote: ----- >To: "Bernard Metzler" <BMT@zurich.ibm.com> >From: "Leon Romanovsky" <leon@kernel.org> >Date: 05/30/2019 03:03PM >Cc: linux-rdma@vger.kernel.org, "Jason Gunthorpe" <jgg@ziepe.ca>, >"Steve Wise" <larrystevenwise@gmail.com> >Subject: [EXTERNAL] Re: [PATCH for-next v1 00/12] SIW: Software iWarp >RDMA (siw) driver > >On Wed, May 29, 2019 at 03:56:57PM +0000, Bernard Metzler wrote: >> -----linux-rdma-owner@vger.kernel.org wrote: ----- >> >> >To: linux-rdma@vger.kernel.org >> >From: "Bernard Metzler" >> >Sent by: linux-rdma-owner@vger.kernel.org >> >Date: 05/26/2019 01:42PM >> >Cc: "Bernard Metzler" <bmt@zurich.ibm.com> >> >Subject: [EXTERNAL] [PATCH for-next v1 00/12] SIW: Software iWarp >> >RDMA (siw) driver >> > >> >This patch set contributes the SoftiWarp driver rebased for >> >Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA >> >protocol over kernel TCP sockets. The driver integrates with >> >the linux-rdma framework. >> > >> >With this new driver version, the following things where >> >changed, compared to the v8 RFC of siw: >> > >> >o Rebased to 5.2-rc1 >> > >> >o All IDR code got removed. >> > >> >o Both MR and QP deallocation verbs now synchronously >> > free the resources referenced by the RDMA mid-layer. >> > >> >o IPv6 support was added. >> > >> >o For compatibility with Chelsio iWarp hardware, the RX >> > path was slightly reworked. It now allows packet intersection >> > between tagged and untagged RDMAP operations. While not >> > a defined behavior as of IETF RFC 5040/5041, some RDMA hardware >> > may intersect an ongoing outbound (large) tagged message, such >> > as an multisegment RDMA Read Response with sending an untagged >> > message, such as an RDMA Send frame. This behavior was only >> > detected in an NVMeF setup, where siw was used at target side, >> > and RDMA hardware at client side (during file write). siw now >> > implements two input paths for tagged and untagged messages >each, >> > and allows the intersected placement of both messages. >> > >> >o The siw kernel abi file got renamed from siw_user.h to >siw-abi.h. >> > >> >Many thanks for reviewing and testing the driver, especially to >> >Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to >> >significantly improve the siw driver over the last year. It is >> >very much appreciated. >> > >> >Many thanks! >> >Bernard. >> > >> >Bernard Metzler (12): >> > iWarp wire packet format >> > SIW main include file >> > SIW network and RDMA core interface >> > SIW connection management >> > SIW application interface >> > SIW application buffer management >> > SIW queue pair methods >> > SIW transmit path >> > SIW receive path >> > SIW completion queue methods >> > SIW debugging >> > SIW addition to kernel build environment >> > >> > MAINTAINERS | 7 + >> > drivers/infiniband/Kconfig | 1 + >> > drivers/infiniband/sw/Makefile | 1 + >> > drivers/infiniband/sw/siw/Kconfig | 17 + >> > drivers/infiniband/sw/siw/Makefile | 12 + >> > drivers/infiniband/sw/siw/iwarp.h | 380 ++++ >> > drivers/infiniband/sw/siw/siw.h | 720 ++++++++ >> > drivers/infiniband/sw/siw/siw_cm.c | 2109 >> >++++++++++++++++++++++ >> > drivers/infiniband/sw/siw/siw_cm.h | 133 ++ >> > drivers/infiniband/sw/siw/siw_cq.c | 109 ++ >> > drivers/infiniband/sw/siw/siw_debug.c | 102 ++ >> > drivers/infiniband/sw/siw/siw_debug.h | 35 + >> > drivers/infiniband/sw/siw/siw_main.c | 701 +++++++ >> > drivers/infiniband/sw/siw/siw_mem.c | 462 +++++ >> > drivers/infiniband/sw/siw/siw_mem.h | 74 + >> > drivers/infiniband/sw/siw/siw_qp.c | 1345 ++++++++++++++ >> > drivers/infiniband/sw/siw/siw_qp_rx.c | 1537 ++++++++++++++++ >> > drivers/infiniband/sw/siw/siw_qp_tx.c | 1276 +++++++++++++ >> > drivers/infiniband/sw/siw/siw_verbs.c | 1778 >++++++++++++++++++ >> > drivers/infiniband/sw/siw/siw_verbs.h | 102 ++ >> > include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 + >> > include/uapi/rdma/siw-abi.h | 186 ++ >> > 22 files changed, 11088 insertions(+) >> > create mode 100644 drivers/infiniband/sw/siw/Kconfig >> > create mode 100644 drivers/infiniband/sw/siw/Makefile >> > create mode 100644 drivers/infiniband/sw/siw/iwarp.h >> > create mode 100644 drivers/infiniband/sw/siw/siw.h >> > create mode 100644 drivers/infiniband/sw/siw/siw_cm.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_cm.h >> > create mode 100644 drivers/infiniband/sw/siw/siw_cq.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_debug.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_debug.h >> > create mode 100644 drivers/infiniband/sw/siw/siw_main.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_mem.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_mem.h >> > create mode 100644 drivers/infiniband/sw/siw/siw_qp.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_qp_rx.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_qp_tx.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_verbs.c >> > create mode 100644 drivers/infiniband/sw/siw/siw_verbs.h >> > create mode 100644 include/uapi/rdma/siw-abi.h >> > >> >-- >> >2.17.2 >> > >> > >> >> Hi Jason, Leon, Steve, @all, >> >> What's next for getting siw merged? Please help me to >> keep the ball rolling. I am currently running out of >> issues I shall fix (which is not a bad feeling though ;)). >> I see lots of other demanding stuff is going on >> these days... > >Generally speaking, I think that it is ready to be merged. > >If Jason/Doug doesn't merge this merge before next week, >I'll take an extra look and add my ROBs next week, but it is >definitely not a blocker for acceptance. > > Leon, Many thanks for the heads-up! Bernard.
On 5/30/2019 9:03 AM, Leon Romanovsky wrote: > On Wed, May 29, 2019 at 03:56:57PM +0000, Bernard Metzler wrote: >> -----linux-rdma-owner@vger.kernel.org wrote: ----- >> >>> To: linux-rdma@vger.kernel.org >>> From: "Bernard Metzler" >>> Sent by: linux-rdma-owner@vger.kernel.org >>> Date: 05/26/2019 01:42PM >>> Cc: "Bernard Metzler" <bmt@zurich.ibm.com> >>> Subject: [EXTERNAL] [PATCH for-next v1 00/12] SIW: Software iWarp >>> RDMA (siw) driver >>> >>> This patch set contributes the SoftiWarp driver rebased for >>> Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA >>> protocol over kernel TCP sockets. The driver integrates with >>> the linux-rdma framework. >>> >>> With this new driver version, the following things where >>> changed, compared to the v8 RFC of siw: >>> >>> o Rebased to 5.2-rc1 >>> >>> o All IDR code got removed. >>> >>> o Both MR and QP deallocation verbs now synchronously >>> free the resources referenced by the RDMA mid-layer. >>> >>> o IPv6 support was added. >>> >>> o For compatibility with Chelsio iWarp hardware, the RX >>> path was slightly reworked. It now allows packet intersection >>> between tagged and untagged RDMAP operations. While not >>> a defined behavior as of IETF RFC 5040/5041, some RDMA hardware >>> may intersect an ongoing outbound (large) tagged message, such >>> as an multisegment RDMA Read Response with sending an untagged >>> message, such as an RDMA Send frame. This behavior was only >>> detected in an NVMeF setup, where siw was used at target side, >>> and RDMA hardware at client side (during file write). siw now >>> implements two input paths for tagged and untagged messages each, >>> and allows the intersected placement of both messages. >>> >>> o The siw kernel abi file got renamed from siw_user.h to siw-abi.h. >>> >>> Many thanks for reviewing and testing the driver, especially to >>> Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to >>> significantly improve the siw driver over the last year. It is >>> very much appreciated. >>> >>> Many thanks! >>> Bernard. >>> >>> Bernard Metzler (12): >>> iWarp wire packet format >>> SIW main include file >>> SIW network and RDMA core interface >>> SIW connection management >>> SIW application interface >>> SIW application buffer management >>> SIW queue pair methods >>> SIW transmit path >>> SIW receive path >>> SIW completion queue methods >>> SIW debugging >>> SIW addition to kernel build environment >>> >>> MAINTAINERS | 7 + >>> drivers/infiniband/Kconfig | 1 + >>> drivers/infiniband/sw/Makefile | 1 + >>> drivers/infiniband/sw/siw/Kconfig | 17 + >>> drivers/infiniband/sw/siw/Makefile | 12 + >>> drivers/infiniband/sw/siw/iwarp.h | 380 ++++ >>> drivers/infiniband/sw/siw/siw.h | 720 ++++++++ >>> drivers/infiniband/sw/siw/siw_cm.c | 2109 >>> ++++++++++++++++++++++ >>> drivers/infiniband/sw/siw/siw_cm.h | 133 ++ >>> drivers/infiniband/sw/siw/siw_cq.c | 109 ++ >>> drivers/infiniband/sw/siw/siw_debug.c | 102 ++ >>> drivers/infiniband/sw/siw/siw_debug.h | 35 + >>> drivers/infiniband/sw/siw/siw_main.c | 701 +++++++ >>> drivers/infiniband/sw/siw/siw_mem.c | 462 +++++ >>> drivers/infiniband/sw/siw/siw_mem.h | 74 + >>> drivers/infiniband/sw/siw/siw_qp.c | 1345 ++++++++++++++ >>> drivers/infiniband/sw/siw/siw_qp_rx.c | 1537 ++++++++++++++++ >>> drivers/infiniband/sw/siw/siw_qp_tx.c | 1276 +++++++++++++ >>> drivers/infiniband/sw/siw/siw_verbs.c | 1778 ++++++++++++++++++ >>> drivers/infiniband/sw/siw/siw_verbs.h | 102 ++ >>> include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 + >>> include/uapi/rdma/siw-abi.h | 186 ++ >>> 22 files changed, 11088 insertions(+) >>> create mode 100644 drivers/infiniband/sw/siw/Kconfig >>> create mode 100644 drivers/infiniband/sw/siw/Makefile >>> create mode 100644 drivers/infiniband/sw/siw/iwarp.h >>> create mode 100644 drivers/infiniband/sw/siw/siw.h >>> create mode 100644 drivers/infiniband/sw/siw/siw_cm.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_cm.h >>> create mode 100644 drivers/infiniband/sw/siw/siw_cq.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_debug.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_debug.h >>> create mode 100644 drivers/infiniband/sw/siw/siw_main.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_mem.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_mem.h >>> create mode 100644 drivers/infiniband/sw/siw/siw_qp.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_qp_rx.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_qp_tx.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_verbs.c >>> create mode 100644 drivers/infiniband/sw/siw/siw_verbs.h >>> create mode 100644 include/uapi/rdma/siw-abi.h >>> >>> -- >>> 2.17.2 >>> >>> >> >> Hi Jason, Leon, Steve, @all, >> >> What's next for getting siw merged? Please help me to >> keep the ball rolling. I am currently running out of >> issues I shall fix (which is not a bad feeling though ;)). >> I see lots of other demanding stuff is going on >> these days... > > Generally speaking, I think that it is ready to be merged. > > If Jason/Doug doesn't merge this merge before next week, > I'll take an extra look and add my ROBs next week, but it is > definitely not a blocker for acceptance. > Agree. I think this looks pretty good. -Denny
On Thu, May 30, 2019 at 8:53 AM Dennis Dalessandro <dennis.dalessandro@intel.com> wrote: > > On 5/30/2019 9:03 AM, Leon Romanovsky wrote: > > On Wed, May 29, 2019 at 03:56:57PM +0000, Bernard Metzler wrote: > >> -----linux-rdma-owner@vger.kernel.org wrote: ----- > >> > >>> To: linux-rdma@vger.kernel.org > >>> From: "Bernard Metzler" > >>> Sent by: linux-rdma-owner@vger.kernel.org > >>> Date: 05/26/2019 01:42PM > >>> Cc: "Bernard Metzler" <bmt@zurich.ibm.com> > >>> Subject: [EXTERNAL] [PATCH for-next v1 00/12] SIW: Software iWarp > >>> RDMA (siw) driver > >>> > >>> This patch set contributes the SoftiWarp driver rebased for > >>> Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA > >>> protocol over kernel TCP sockets. The driver integrates with > >>> the linux-rdma framework. > >>> > >>> With this new driver version, the following things where > >>> changed, compared to the v8 RFC of siw: > >>> > >>> o Rebased to 5.2-rc1 > >>> > >>> o All IDR code got removed. > >>> > >>> o Both MR and QP deallocation verbs now synchronously > >>> free the resources referenced by the RDMA mid-layer. > >>> > >>> o IPv6 support was added. > >>> > >>> o For compatibility with Chelsio iWarp hardware, the RX > >>> path was slightly reworked. It now allows packet intersection > >>> between tagged and untagged RDMAP operations. While not > >>> a defined behavior as of IETF RFC 5040/5041, some RDMA hardware > >>> may intersect an ongoing outbound (large) tagged message, such > >>> as an multisegment RDMA Read Response with sending an untagged > >>> message, such as an RDMA Send frame. This behavior was only > >>> detected in an NVMeF setup, where siw was used at target side, > >>> and RDMA hardware at client side (during file write). siw now > >>> implements two input paths for tagged and untagged messages each, > >>> and allows the intersected placement of both messages. > >>> > >>> o The siw kernel abi file got renamed from siw_user.h to siw-abi.h. > >>> > >>> Many thanks for reviewing and testing the driver, especially to > >>> Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to > >>> significantly improve the siw driver over the last year. It is > >>> very much appreciated. > >>> > >>> Many thanks! > >>> Bernard. > >>> > >>> Bernard Metzler (12): > >>> iWarp wire packet format > >>> SIW main include file > >>> SIW network and RDMA core interface > >>> SIW connection management > >>> SIW application interface > >>> SIW application buffer management > >>> SIW queue pair methods > >>> SIW transmit path > >>> SIW receive path > >>> SIW completion queue methods > >>> SIW debugging > >>> SIW addition to kernel build environment > >>> > >>> MAINTAINERS | 7 + > >>> drivers/infiniband/Kconfig | 1 + > >>> drivers/infiniband/sw/Makefile | 1 + > >>> drivers/infiniband/sw/siw/Kconfig | 17 + > >>> drivers/infiniband/sw/siw/Makefile | 12 + > >>> drivers/infiniband/sw/siw/iwarp.h | 380 ++++ > >>> drivers/infiniband/sw/siw/siw.h | 720 ++++++++ > >>> drivers/infiniband/sw/siw/siw_cm.c | 2109 > >>> ++++++++++++++++++++++ > >>> drivers/infiniband/sw/siw/siw_cm.h | 133 ++ > >>> drivers/infiniband/sw/siw/siw_cq.c | 109 ++ > >>> drivers/infiniband/sw/siw/siw_debug.c | 102 ++ > >>> drivers/infiniband/sw/siw/siw_debug.h | 35 + > >>> drivers/infiniband/sw/siw/siw_main.c | 701 +++++++ > >>> drivers/infiniband/sw/siw/siw_mem.c | 462 +++++ > >>> drivers/infiniband/sw/siw/siw_mem.h | 74 + > >>> drivers/infiniband/sw/siw/siw_qp.c | 1345 ++++++++++++++ > >>> drivers/infiniband/sw/siw/siw_qp_rx.c | 1537 ++++++++++++++++ > >>> drivers/infiniband/sw/siw/siw_qp_tx.c | 1276 +++++++++++++ > >>> drivers/infiniband/sw/siw/siw_verbs.c | 1778 ++++++++++++++++++ > >>> drivers/infiniband/sw/siw/siw_verbs.h | 102 ++ > >>> include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 + > >>> include/uapi/rdma/siw-abi.h | 186 ++ > >>> 22 files changed, 11088 insertions(+) > >>> create mode 100644 drivers/infiniband/sw/siw/Kconfig > >>> create mode 100644 drivers/infiniband/sw/siw/Makefile > >>> create mode 100644 drivers/infiniband/sw/siw/iwarp.h > >>> create mode 100644 drivers/infiniband/sw/siw/siw.h > >>> create mode 100644 drivers/infiniband/sw/siw/siw_cm.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_cm.h > >>> create mode 100644 drivers/infiniband/sw/siw/siw_cq.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_debug.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_debug.h > >>> create mode 100644 drivers/infiniband/sw/siw/siw_main.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_mem.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_mem.h > >>> create mode 100644 drivers/infiniband/sw/siw/siw_qp.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_qp_rx.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_qp_tx.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_verbs.c > >>> create mode 100644 drivers/infiniband/sw/siw/siw_verbs.h > >>> create mode 100644 include/uapi/rdma/siw-abi.h > >>> > >>> -- > >>> 2.17.2 > >>> > >>> > >> > >> Hi Jason, Leon, Steve, @all, > >> > >> What's next for getting siw merged? Please help me to > >> keep the ball rolling. I am currently running out of > >> issues I shall fix (which is not a bad feeling though ;)). > >> I see lots of other demanding stuff is going on > >> these days... > > > > Generally speaking, I think that it is ready to be merged. > > > > If Jason/Doug doesn't merge this merge before next week, > > I'll take an extra look and add my ROBs next week, but it is > > definitely not a blocker for acceptance. > > > > Agree. I think this looks pretty good. > > -Denny Ditto. Steve.
On Sun, May 26, 2019 at 01:41:44PM +0200, Bernard Metzler wrote: > This patch set contributes the SoftiWarp driver rebased for > Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA > protocol over kernel TCP sockets. The driver integrates with > the linux-rdma framework. > > With this new driver version, the following things where > changed, compared to the v8 RFC of siw: > > o Rebased to 5.2-rc1 > > o All IDR code got removed. > > o Both MR and QP deallocation verbs now synchronously > free the resources referenced by the RDMA mid-layer. > > o IPv6 support was added. > > o For compatibility with Chelsio iWarp hardware, the RX > path was slightly reworked. It now allows packet intersection > between tagged and untagged RDMAP operations. While not > a defined behavior as of IETF RFC 5040/5041, some RDMA hardware > may intersect an ongoing outbound (large) tagged message, such > as an multisegment RDMA Read Response with sending an untagged > message, such as an RDMA Send frame. This behavior was only > detected in an NVMeF setup, where siw was used at target side, > and RDMA hardware at client side (during file write). siw now > implements two input paths for tagged and untagged messages each, > and allows the intersected placement of both messages. > > o The siw kernel abi file got renamed from siw_user.h to siw-abi.h. > > Many thanks for reviewing and testing the driver, especially to > Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to > significantly improve the siw driver over the last year. It is > very much appreciated. You need to open a PR for rdma-core before this can be merged with the userspace part. Jason
-----"Jason Gunthorpe" <jgg@ziepe.ca> wrote: ----- >To: "Bernard Metzler" <bmt@zurich.ibm.com> >From: "Jason Gunthorpe" <jgg@ziepe.ca> >Date: 06/03/2019 07:50PM >Cc: linux-rdma@vger.kernel.org >Subject: [EXTERNAL] Re: [PATCH for-next v1 00/12] SIW: Software iWarp >RDMA (siw) driver > >On Sun, May 26, 2019 at 01:41:44PM +0200, Bernard Metzler wrote: >> This patch set contributes the SoftiWarp driver rebased for >> Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA >> protocol over kernel TCP sockets. The driver integrates with >> the linux-rdma framework. >> >> With this new driver version, the following things where >> changed, compared to the v8 RFC of siw: >> >> o Rebased to 5.2-rc1 >> >> o All IDR code got removed. >> >> o Both MR and QP deallocation verbs now synchronously >> free the resources referenced by the RDMA mid-layer. >> >> o IPv6 support was added. >> >> o For compatibility with Chelsio iWarp hardware, the RX >> path was slightly reworked. It now allows packet intersection >> between tagged and untagged RDMAP operations. While not >> a defined behavior as of IETF RFC 5040/5041, some RDMA hardware >> may intersect an ongoing outbound (large) tagged message, such >> as an multisegment RDMA Read Response with sending an untagged >> message, such as an RDMA Send frame. This behavior was only >> detected in an NVMeF setup, where siw was used at target side, >> and RDMA hardware at client side (during file write). siw now >> implements two input paths for tagged and untagged messages each, >> and allows the intersected placement of both messages. >> >> o The siw kernel abi file got renamed from siw_user.h to siw-abi.h. >> >> Many thanks for reviewing and testing the driver, especially to >> Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to >> significantly improve the siw driver over the last year. It is >> very much appreciated. > >You need to open a PR for rdma-core before this can be merged with >the >userspace part. > >Jason > > OK I created PR #536, which adds the siw user library to rdma-core. Unfortunately, when uploading, travis brought up many issues with atomics etc. Is there a good way to have the very same strict checking locally, since local build was always successful... In any case, sorry for abusing the PR procedure for code cleanup (amending commits and force push cycles)! Many thanks, Bernard.
On Tue, Jun 04, 2019 at 05:37:15PM +0000, Bernard Metzler wrote: > > >To: "Bernard Metzler" <bmt@zurich.ibm.com> > >From: "Jason Gunthorpe" <jgg@ziepe.ca> > >Date: 06/03/2019 07:50PM > >Cc: linux-rdma@vger.kernel.org > >Subject: [EXTERNAL] Re: [PATCH for-next v1 00/12] SIW: Software iWarp > >RDMA (siw) driver > > > >On Sun, May 26, 2019 at 01:41:44PM +0200, Bernard Metzler wrote: > >> This patch set contributes the SoftiWarp driver rebased for > >> Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA > >> protocol over kernel TCP sockets. The driver integrates with > >> the linux-rdma framework. > >> > >> With this new driver version, the following things where > >> changed, compared to the v8 RFC of siw: > >> > >> o Rebased to 5.2-rc1 > >> > >> o All IDR code got removed. > >> > >> o Both MR and QP deallocation verbs now synchronously > >> free the resources referenced by the RDMA mid-layer. > >> > >> o IPv6 support was added. > >> > >> o For compatibility with Chelsio iWarp hardware, the RX > >> path was slightly reworked. It now allows packet intersection > >> between tagged and untagged RDMAP operations. While not > >> a defined behavior as of IETF RFC 5040/5041, some RDMA hardware > >> may intersect an ongoing outbound (large) tagged message, such > >> as an multisegment RDMA Read Response with sending an untagged > >> message, such as an RDMA Send frame. This behavior was only > >> detected in an NVMeF setup, where siw was used at target side, > >> and RDMA hardware at client side (during file write). siw now > >> implements two input paths for tagged and untagged messages each, > >> and allows the intersected placement of both messages. > >> > >> o The siw kernel abi file got renamed from siw_user.h to siw-abi.h. > >> > >> Many thanks for reviewing and testing the driver, especially to > >> Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to > >> significantly improve the siw driver over the last year. It is > >> very much appreciated. > > > >You need to open a PR for rdma-core before this can be merged with > >the > >userspace part. > > > >Jason > > > > > > OK I created PR #536, which adds the siw user library to > rdma-core. Unfortunately, when uploading, travis brought > up many issues with atomics etc. Is there a good way to > have the very same strict checking locally, since local build > was always successful... $ buildlib/cbuild build-images travis # once $ buildlib/cbuild pkg travis You'll need docker installed > In any case, sorry for abusing the PR procedure for code cleanup > (amending commits and force push cycles)! It is fine, this is what travis is for.. Jason
-----"Jason Gunthorpe" <jgg@ziepe.ca> wrote: ----- >To: "Bernard Metzler" <BMT@zurich.ibm.com> >From: "Jason Gunthorpe" <jgg@ziepe.ca> >Date: 06/04/2019 08:26PM >Cc: linux-rdma@vger.kernel.org >Subject: [EXTERNAL] Re: [PATCH for-next v1 00/12] SIW: Software iWarp >RDMA (siw) driver > >On Tue, Jun 04, 2019 at 05:37:15PM +0000, Bernard Metzler wrote: >> >> >To: "Bernard Metzler" <bmt@zurich.ibm.com> >> >From: "Jason Gunthorpe" <jgg@ziepe.ca> >> >Date: 06/03/2019 07:50PM >> >Cc: linux-rdma@vger.kernel.org >> >Subject: [EXTERNAL] Re: [PATCH for-next v1 00/12] SIW: Software >iWarp >> >RDMA (siw) driver >> > >> >On Sun, May 26, 2019 at 01:41:44PM +0200, Bernard Metzler wrote: >> >> This patch set contributes the SoftiWarp driver rebased for >> >> Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA >> >> protocol over kernel TCP sockets. The driver integrates with >> >> the linux-rdma framework. >> >> >> >> With this new driver version, the following things where >> >> changed, compared to the v8 RFC of siw: >> >> >> >> o Rebased to 5.2-rc1 >> >> >> >> o All IDR code got removed. >> >> >> >> o Both MR and QP deallocation verbs now synchronously >> >> free the resources referenced by the RDMA mid-layer. >> >> >> >> o IPv6 support was added. >> >> >> >> o For compatibility with Chelsio iWarp hardware, the RX >> >> path was slightly reworked. It now allows packet intersection >> >> between tagged and untagged RDMAP operations. While not >> >> a defined behavior as of IETF RFC 5040/5041, some RDMA >hardware >> >> may intersect an ongoing outbound (large) tagged message, such >> >> as an multisegment RDMA Read Response with sending an untagged >> >> message, such as an RDMA Send frame. This behavior was only >> >> detected in an NVMeF setup, where siw was used at target side, >> >> and RDMA hardware at client side (during file write). siw now >> >> implements two input paths for tagged and untagged messages >each, >> >> and allows the intersected placement of both messages. >> >> >> >> o The siw kernel abi file got renamed from siw_user.h to >siw-abi.h. >> >> >> >> Many thanks for reviewing and testing the driver, especially to >> >> Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to >> >> significantly improve the siw driver over the last year. It is >> >> very much appreciated. >> > >> >You need to open a PR for rdma-core before this can be merged with >> >the >> >userspace part. >> > >> >Jason >> > >> > >> >> OK I created PR #536, which adds the siw user library to >> rdma-core. Unfortunately, when uploading, travis brought >> up many issues with atomics etc. Is there a good way to >> have the very same strict checking locally, since local build >> was always successful... > >$ buildlib/cbuild build-images travis # once >$ buildlib/cbuild pkg travis > >You'll need docker installed > >> In any case, sorry for abusing the PR procedure for code cleanup >> (amending commits and force push cycles)! > >It is fine, this is what travis is for.. > Jason, many thanks for the very instant review at github, very much appreciated! I changed things accordingly with a new commit. Please let me know if you would prefer amend and force push for those changes. Thanks very much! Bernard.
On Wed, Jun 05, 2019 at 01:41:12PM +0000, Bernard Metzler wrote: > > >To: "Bernard Metzler" <BMT@zurich.ibm.com> > >From: "Jason Gunthorpe" <jgg@ziepe.ca> > >Date: 06/04/2019 08:26PM > >Cc: linux-rdma@vger.kernel.org > >Subject: [EXTERNAL] Re: [PATCH for-next v1 00/12] SIW: Software iWarp > >RDMA (siw) driver > > > >On Tue, Jun 04, 2019 at 05:37:15PM +0000, Bernard Metzler wrote: > >> > >> >To: "Bernard Metzler" <bmt@zurich.ibm.com> > >> >From: "Jason Gunthorpe" <jgg@ziepe.ca> > >> >Date: 06/03/2019 07:50PM > >> >Cc: linux-rdma@vger.kernel.org > >> >Subject: [EXTERNAL] Re: [PATCH for-next v1 00/12] SIW: Software > >iWarp > >> >RDMA (siw) driver > >> > > >> >On Sun, May 26, 2019 at 01:41:44PM +0200, Bernard Metzler wrote: > >> >> This patch set contributes the SoftiWarp driver rebased for > >> >> Kernel 5.2-rc1. SoftiWarp (siw) implements the iWarp RDMA > >> >> protocol over kernel TCP sockets. The driver integrates with > >> >> the linux-rdma framework. > >> >> > >> >> With this new driver version, the following things where > >> >> changed, compared to the v8 RFC of siw: > >> >> > >> >> o Rebased to 5.2-rc1 > >> >> > >> >> o All IDR code got removed. > >> >> > >> >> o Both MR and QP deallocation verbs now synchronously > >> >> free the resources referenced by the RDMA mid-layer. > >> >> > >> >> o IPv6 support was added. > >> >> > >> >> o For compatibility with Chelsio iWarp hardware, the RX > >> >> path was slightly reworked. It now allows packet intersection > >> >> between tagged and untagged RDMAP operations. While not > >> >> a defined behavior as of IETF RFC 5040/5041, some RDMA > >hardware > >> >> may intersect an ongoing outbound (large) tagged message, such > >> >> as an multisegment RDMA Read Response with sending an untagged > >> >> message, such as an RDMA Send frame. This behavior was only > >> >> detected in an NVMeF setup, where siw was used at target side, > >> >> and RDMA hardware at client side (during file write). siw now > >> >> implements two input paths for tagged and untagged messages > >each, > >> >> and allows the intersected placement of both messages. > >> >> > >> >> o The siw kernel abi file got renamed from siw_user.h to > >siw-abi.h. > >> >> > >> >> Many thanks for reviewing and testing the driver, especially to > >> >> Steve, Leon, Jason, Doug, Olga, Dennis, Gal. You all helped to > >> >> significantly improve the siw driver over the last year. It is > >> >> very much appreciated. > >> > > >> >You need to open a PR for rdma-core before this can be merged with > >> >the > >> >userspace part. > >> > > >> >Jason > >> > > >> > > >> > >> OK I created PR #536, which adds the siw user library to > >> rdma-core. Unfortunately, when uploading, travis brought > >> up many issues with atomics etc. Is there a good way to > >> have the very same strict checking locally, since local build > >> was always successful... > > > >$ buildlib/cbuild build-images travis # once > >$ buildlib/cbuild pkg travis > > > >You'll need docker installed > > > >> In any case, sorry for abusing the PR procedure for code cleanup > >> (amending commits and force push cycles)! > > > >It is fine, this is what travis is for.. > > > Jason, many thanks for the very instant review at github, very much > appreciated! I changed things accordingly with a new commit. Please let > me know if you would prefer amend and force push for those changes. Force push but it is often nice if you include the 'diff' output between the old and forced commits in the comments, if it is small. Jason