From patchwork Wed Jan 30 17:21:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bernard Metzler X-Patchwork-Id: 10789045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7869E14E1 for ; Wed, 30 Jan 2019 17:22:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61CB02FD6C for ; Wed, 30 Jan 2019 17:22:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 519CF2FE71; Wed, 30 Jan 2019 17:22:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3472E2FD58 for ; Wed, 30 Jan 2019 17:21:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732381AbfA3RV7 (ORCPT ); Wed, 30 Jan 2019 12:21:59 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:56258 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726230AbfA3RV6 (ORCPT ); Wed, 30 Jan 2019 12:21:58 -0500 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x0UHLDvg139639 for ; Wed, 30 Jan 2019 12:21:57 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2qbg8dr10u-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 30 Jan 2019 12:21:57 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 30 Jan 2019 17:21:55 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 30 Jan 2019 17:21:53 -0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x0UHLqGN4522406 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 30 Jan 2019 17:21:52 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0834CAE04D; Wed, 30 Jan 2019 17:21:52 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D87C7AE045; Wed, 30 Jan 2019 17:21:51 +0000 (GMT) Received: from spoke.zurich.ibm.com (unknown [9.4.69.152]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 30 Jan 2019 17:21:51 +0000 (GMT) From: bmt@zurich.ibm.com To: linux-rdma@vger.kernel.org Cc: Bernard Metzler Subject: [PATCH v4 00/13] SIW: Request for Comments Date: Wed, 30 Jan 2019 18:21:23 +0100 X-Mailer: git-send-email 2.17.2 X-TM-AS-GCONF: 00 x-cbid: 19013017-4275-0000-0000-00000307EF21 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19013017-4276-0000-0000-00003815F5DD Message-Id: <20190130172136.23625-1-bmt@zurich.ibm.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-01-30_13:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1901300133 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Bernard Metzler This patch set contributes a new version of the SoftiWarp driver, as originally introduced to the list Oct 6th, 2017. SoftiWarp (siw) implements the iWarp RDMA protocol over kernel TCP sockets. The driver integrates with the linux-rdma framework. For this patch series, we aimed at fixing the main obstacles, which prevented siw acceptance in the past: 1. siw now uses the recently extended rdma netlink protocol for adding and removing siw devices. It became the only way of managing siw devices. 2. The driver integrates with the currently reviewed RDMA/IWPM patch series on introducing no port mapping requirements. These patches as provided by Steve Wise are a pre-requisit to run siw in an environment with active iwpmd. The code has the following known limitations: 1. Only IPv4 addresses are supported / no IPv6 support. 2. All previously flexible module parameters are translated into const values as defined in siw_main.c. We propose another extension of the netlink protocol to make those driver parameters dynamically settable. Ideally, we would distinguish between link specific and connection specific parameters. Currently, we would like to see the following parameters settable: o MPA peer-to-peer mode (boolean on/off) o MPA CRC (boolean on/off) o MPA CRC negotiation mode: accept different CRC setting from peer (boolean on/off) o TCP_NODELAY to control Nagle settings of TCP socket (boolean on/off) o MPA version (0, 1 or 2) o Zerocopy to let TCP transmit out of application buffers wr/o copying data (boolean on/off) o GSO to select Generic Segementation Offload for larger frames (one frame may span 1 .. n Ethernet frames, if advertised by TCP socket) In principle, all of those parameters could be controlled per connection. At least, dynamically setting those per device is highly desirable. Please let's agree on the most clean solution for that. We maintain a snapshot of the current code at https://github.com/zrlio/softiwarp-for-linux-rdma.git within branch 'siw-for-rdma-next-nvme-5.0'. This branch is based on the rdma-next tree and additionally includes the latest netlink and portmapper patches from Steve Wise as well as the latest nvme-5.0 code changes from git://git.infradead.org/nvme.git. We tested siw with NVMeF host and target applications and therefor merged with latest nvme development. The matching siw user library is maintained at https://github.com/zrlio/softiwarp-user-for-linux-rdma.git. It is based on rdma-core, and extended with Steve's patches to both rdma netlink and portmapper. The relevant branch name is 'siw-for-rdma-next'. As always, we'd highly appreciate your code review. Thanks very much for your time. Bernard Bernard Metzler (13): iWarp wire packet format SIW main include file SIW network and RDMA core interface SIW object management SIW connection management SIW application interface SIW application buffer management SIW queue pair methods SIW transmit path SIW receive path SIW completion queue methods SIW debugging SIW addition to kernel build environment drivers/infiniband/Kconfig | 1 + drivers/infiniband/sw/Makefile | 1 + drivers/infiniband/sw/siw/Kconfig | 17 + drivers/infiniband/sw/siw/Makefile | 15 + drivers/infiniband/sw/siw/iwarp.h | 415 ++++ drivers/infiniband/sw/siw/siw.h | 805 ++++++++ drivers/infiniband/sw/siw/siw_ae.c | 120 ++ drivers/infiniband/sw/siw/siw_cm.c | 2185 ++++++++++++++++++++++ drivers/infiniband/sw/siw/siw_cm.h | 156 ++ drivers/infiniband/sw/siw/siw_cq.c | 150 ++ drivers/infiniband/sw/siw/siw_debug.c | 467 +++++ drivers/infiniband/sw/siw/siw_debug.h | 87 + drivers/infiniband/sw/siw/siw_main.c | 846 +++++++++ drivers/infiniband/sw/siw/siw_mem.c | 243 +++ drivers/infiniband/sw/siw/siw_obj.c | 338 ++++ drivers/infiniband/sw/siw/siw_obj.h | 200 ++ drivers/infiniband/sw/siw/siw_qp.c | 1473 +++++++++++++++ drivers/infiniband/sw/siw/siw_qp_rx.c | 1533 +++++++++++++++ drivers/infiniband/sw/siw/siw_qp_tx.c | 1340 +++++++++++++ drivers/infiniband/sw/siw/siw_verbs.c | 1888 +++++++++++++++++++ drivers/infiniband/sw/siw/siw_verbs.h | 119 ++ include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 + include/uapi/rdma/siw_user.h | 216 +++ 23 files changed, 12616 insertions(+) create mode 100644 drivers/infiniband/sw/siw/Kconfig create mode 100644 drivers/infiniband/sw/siw/Makefile create mode 100644 drivers/infiniband/sw/siw/iwarp.h create mode 100644 drivers/infiniband/sw/siw/siw.h create mode 100644 drivers/infiniband/sw/siw/siw_ae.c create mode 100644 drivers/infiniband/sw/siw/siw_cm.c create mode 100644 drivers/infiniband/sw/siw/siw_cm.h create mode 100644 drivers/infiniband/sw/siw/siw_cq.c create mode 100644 drivers/infiniband/sw/siw/siw_debug.c create mode 100644 drivers/infiniband/sw/siw/siw_debug.h create mode 100644 drivers/infiniband/sw/siw/siw_main.c create mode 100644 drivers/infiniband/sw/siw/siw_mem.c create mode 100644 drivers/infiniband/sw/siw/siw_obj.c create mode 100644 drivers/infiniband/sw/siw/siw_obj.h create mode 100644 drivers/infiniband/sw/siw/siw_qp.c create mode 100644 drivers/infiniband/sw/siw/siw_qp_rx.c create mode 100644 drivers/infiniband/sw/siw/siw_qp_tx.c create mode 100644 drivers/infiniband/sw/siw/siw_verbs.c create mode 100644 drivers/infiniband/sw/siw/siw_verbs.h create mode 100644 include/uapi/rdma/siw_user.h