From patchwork Tue Feb 19 10:08:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bernard Metzler X-Patchwork-Id: 10819607 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A68F6CB for ; Tue, 19 Feb 2019 10:09:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14C9E2B307 for ; Tue, 19 Feb 2019 10:09:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 089952B317; Tue, 19 Feb 2019 10:09:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F0D42B307 for ; Tue, 19 Feb 2019 10:09:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727601AbfBSKJh (ORCPT ); Tue, 19 Feb 2019 05:09:37 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:58926 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727739AbfBSKJh (ORCPT ); Tue, 19 Feb 2019 05:09:37 -0500 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x1JA3jxU067489 for ; Tue, 19 Feb 2019 05:09:36 -0500 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0a-001b2d01.pphosted.com with ESMTP id 2qrcvmtu4p-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 19 Feb 2019 05:09:35 -0500 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 19 Feb 2019 10:09:33 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 19 Feb 2019 10:09:30 -0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x1JA9TPl60489918 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 19 Feb 2019 10:09:29 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5AB024203F; Tue, 19 Feb 2019 10:09:29 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3664B42041; Tue, 19 Feb 2019 10:09:29 +0000 (GMT) Received: from spoke.zurich.ibm.com (unknown [9.4.69.152]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 19 Feb 2019 10:09:29 +0000 (GMT) From: Bernard Metzler To: linux-rdma@vger.kernel.org Cc: Bernard Metzler Subject: [PATCH v5 00/13] SIW: Request for Comments Date: Tue, 19 Feb 2019 11:08:50 +0100 X-Mailer: git-send-email 2.17.2 X-TM-AS-GCONF: 00 x-cbid: 19021910-0020-0000-0000-0000031886E6 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19021910-0021-0000-0000-00002169CB63 Message-Id: <20190219100903.15408-1-bmt@zurich.ibm.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-02-19_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902190075 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch set contributes version 5 of the SoftiWarp driver, as originally introduced to the list Oct 6th, 2017. SoftiWarp (siw) implements the iWarp RDMA protocol over kernel TCP sockets. The driver integrates with the linux-rdma framework. In response to the various helpful feedback, we fixed (besides other small fixes) the following issues: 1. The debugfs logic got completely removed. There is no debugfs entry for siw anymore. We will interate with rdmatool in the future. 2. The driver adheres to the device and object management as recently proposed by Jason Gunthorpe. This includes the new protection domain management. 3. An entry to linux MAINTAINERS for siw was added. 4. The packet length at MPA protocol level got restricted to physical MTU size: to interoperate with hardware iWarp devices such as Chelsio T5/T6, GSO usage got unselected. It can be selcted with a global parameter 'gso_seg_limit'. Setting 'gso_seg_limit = 0' makes full use of GSO if advertised by the TCP socket. This almost doubles bandwidth for bulk data transfers in a pure siw <-> siw setting, but does not interoperate wth some hardware RNIC's. 5. An 'SPDX-License-Identifier' got added to all files 6. The user interface as defined in include/uapi/rdma/siwa_user.h got cleaned up. 7. Useless redefinition of functions for kernel reference count reading got removed. The driver continues to make use of rdma-netdev extensions to add and delete links, as proposed by Steve Wise. The driver continues to rely on the iWarp port mapper extensions for software RDMA devices, as proposed by Steve Wise. We maintain a snapshot of the current code at https://github.com/zrlio/softiwarp-for-linux-rdma.git within branch 'siw-for-rdma-next-v5'. This branch includes the latest netlink and portmapper patches from Steve as well as the latest device and object management from Jason. The matching siw user library is maintained at https://github.com/zrlio/softiwarp-user-for-linux-rdma.git. It is based on rdma-core, and extended with Steve's patches to both rdma netlink and portmapper. The relevant branch name is 'siw-for-rdma-next'. Since siw may generate work completion notifications from a kthread context, performance of kernel siw clients like NVMeF depends on the following pending kernel patch (included in https://github.com/zrlio/softiwarp-for-linux-rdma.git): https://www.spinics.net/lists/linux-rdma/msg75081.html As always, we'd highly appreciate your code review. Thanks very much for your time! Bernard Bernard Metzler (13): iWarp wire packet format SIW main include file SIW network and RDMA core interface SIW object management SIW connection management SIW application interface SIW application buffer management SIW queue pair methods SIW transmit path SIW receive path SIW completion queue methods SIW debugging SIW addition to kernel build environment MAINTAINERS | 7 + drivers/infiniband/Kconfig | 1 + drivers/infiniband/sw/Makefile | 1 + drivers/infiniband/sw/siw/Kconfig | 17 + drivers/infiniband/sw/siw/Makefile | 15 + drivers/infiniband/sw/siw/iwarp.h | 414 ++++ drivers/infiniband/sw/siw/siw.h | 790 ++++++++ drivers/infiniband/sw/siw/siw_ae.c | 121 ++ drivers/infiniband/sw/siw/siw_cm.c | 2186 ++++++++++++++++++++++ drivers/infiniband/sw/siw/siw_cm.h | 157 ++ drivers/infiniband/sw/siw/siw_cq.c | 149 ++ drivers/infiniband/sw/siw/siw_debug.c | 124 ++ drivers/infiniband/sw/siw/siw_debug.h | 69 + drivers/infiniband/sw/siw/siw_main.c | 813 ++++++++ drivers/infiniband/sw/siw/siw_mem.c | 217 +++ drivers/infiniband/sw/siw/siw_obj.c | 341 ++++ drivers/infiniband/sw/siw/siw_obj.h | 226 +++ drivers/infiniband/sw/siw/siw_qp.c | 1478 +++++++++++++++ drivers/infiniband/sw/siw/siw_qp_rx.c | 1535 +++++++++++++++ drivers/infiniband/sw/siw/siw_qp_tx.c | 1330 +++++++++++++ drivers/infiniband/sw/siw/siw_verbs.c | 1851 ++++++++++++++++++ drivers/infiniband/sw/siw/siw_verbs.h | 114 ++ include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 + include/uapi/rdma/siw_user.h | 223 +++ 24 files changed, 12180 insertions(+) create mode 100644 drivers/infiniband/sw/siw/Kconfig create mode 100644 drivers/infiniband/sw/siw/Makefile create mode 100644 drivers/infiniband/sw/siw/iwarp.h create mode 100644 drivers/infiniband/sw/siw/siw.h create mode 100644 drivers/infiniband/sw/siw/siw_ae.c create mode 100644 drivers/infiniband/sw/siw/siw_cm.c create mode 100644 drivers/infiniband/sw/siw/siw_cm.h create mode 100644 drivers/infiniband/sw/siw/siw_cq.c create mode 100644 drivers/infiniband/sw/siw/siw_debug.c create mode 100644 drivers/infiniband/sw/siw/siw_debug.h create mode 100644 drivers/infiniband/sw/siw/siw_main.c create mode 100644 drivers/infiniband/sw/siw/siw_mem.c create mode 100644 drivers/infiniband/sw/siw/siw_obj.c create mode 100644 drivers/infiniband/sw/siw/siw_obj.h create mode 100644 drivers/infiniband/sw/siw/siw_qp.c create mode 100644 drivers/infiniband/sw/siw/siw_qp_rx.c create mode 100644 drivers/infiniband/sw/siw/siw_qp_tx.c create mode 100644 drivers/infiniband/sw/siw/siw_verbs.c create mode 100644 drivers/infiniband/sw/siw/siw_verbs.h create mode 100644 include/uapi/rdma/siw_user.h