From patchwork Sun Dec 20 10:16:10 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 7891731 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id B4342BEEE5 for ; Sun, 20 Dec 2015 11:23:56 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 97D2320481 for ; Sun, 20 Dec 2015 11:23:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AA1662047B for ; Sun, 20 Dec 2015 11:23:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754304AbbLTLXy (ORCPT ); Sun, 20 Dec 2015 06:23:54 -0500 Received: from mail-wm0-f44.google.com ([74.125.82.44]:36621 "EHLO mail-wm0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754272AbbLTLXx (ORCPT ); Sun, 20 Dec 2015 06:23:53 -0500 Received: by mail-wm0-f44.google.com with SMTP id p187so36660504wmp.1 for ; Sun, 20 Dec 2015 03:23:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leon-nu.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=Az9Yp5R3ZYetnaVeF4b1CCLYd3/4y816Ou29AShDG2A=; b=kiMtzGOp1Ntrv1r+wYf/QqAdx8HmQ1WJXEVTZZP7F3nPw2EnWrKh2CJ06w9ssWYrrZ 8w8IpTQPjzKfenbAw4VG2/xw0f5yQFGIbMY2al/fFKYpMbTIzMur5+lVwoFwdEk/QN3e 7C8Yzna9ORo2VemDZ/zQzAGvrt1oCcW6Yl/nQLlJo1meUNcJCOCRiDlIIl3m9n+IojTs XSDZOdQLAMFdl7g+mgrCMIlSLFNGADkGY1DhwXI/89PxQMC+1TzhwnJceWUr+xxCnYiW Nw1S9W9xGlvNEUKtg2USzWxBD45N4Io3eM0IRuamY9SFnGnDSjey0ueyMQ7e6cqx0XQs eNiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=Az9Yp5R3ZYetnaVeF4b1CCLYd3/4y816Ou29AShDG2A=; b=jKnYmchDvA+kvksr5qrWMqHtpEPgJXzXk2Q56cXfIvivNy3/sR0DYWZSE0rO9ZIKlP /Qs6T1IalWgNsdzSlI1AZG4FB5LcvzZfQtxheIjZdhRCLKlBT5F09gRtsnMSzmQx4sNE s02tonb9AssbZ2oe2V4EJu+/jR/kehcl7SAB5tbLwdak2fJMkGU6hYFzMKYlA4yrOoCn Lb2X7D9ipwuLpmjOhpzAUHNuMKxt29mUQ5Pf4pA4M+5OdunfA8VjGEvconZEND8jyAt1 5tuxUzZsj+8YuW4AjdZp1hAeHj850JeaUURsrX3VLPi3mj3kbrcFSgkyRbF3QV9yYkV0 PxbA== X-Gm-Message-State: ALoCoQmJaVWJ1H0uvjUJkKJUeYkVZkJOVI0iuDOkhtHa/s6ZwmYS1iy+6p1Nqg9i5uSdlXpZ2eRw+5a872+oi9uZ8x2ej6x27A== X-Received: by 10.194.59.237 with SMTP id c13mr14146103wjr.117.1450610632009; Sun, 20 Dec 2015 03:23:52 -0800 (PST) Received: from localhost ([213.57.247.249]) by smtp.gmail.com with ESMTPSA id ke10sm23066679wjb.3.2015.12.20.03.23.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 20 Dec 2015 03:23:51 -0800 (PST) From: Leon Romanovsky To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Leon Romanovsky Subject: [PATCH V1 2/3] IB/core: Add cross-channel support Date: Sun, 20 Dec 2015 12:16:10 +0200 Message-Id: <1450606571-15877-3-git-send-email-leon@leon.nu> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1450606571-15877-1-git-send-email-leon@leon.nu> References: <1450606571-15877-1-git-send-email-leon@leon.nu> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Leon Romanovsky The cross-channel feature allows to execute WQEs that involve synchronization of I/O operations’ on different QPs. This capability enables to program complex flows with a single function call, hereby significantly reducing overhead associated with I/O processing. Cross-channel operations support is indicated by HCA capability information. The queue pairs can be configured to work as a “sync master queue” or “sync slave queues”. The added flags are: 1. Device capability flag IB_DEVICE_CROSS_CHANNEL for the devices that can perform cross-channel operations. 2. CQ property flag IB_CQ_FLAGS_IGNORE_OVERRUN to disable CQ overrun check. This check is useless in cross-channel scenario. 3. QP property flags to indicate if queues are slave or master: * IB_QP_CREATE_MANAGED_SEND indicates that posted send work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_MANAGED_RECV indicates that posted receive work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_CROSS_CHANNEL declares the QP to work in cross-channel mode. If IB_QP_CREATE_MANAGED_SEND and IB_QP_CREATE_MANAGED_RECV are not provided, this QP will be sync master queue, else it will be sync slave. Reviewed-by: Sagi Grimberg Signed-off-by: Leon Romanovsky --- drivers/infiniband/core/uverbs_cmd.c | 5 ++++- include/rdma/ib_verbs.h | 11 +++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 94816ae..0e9711f 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1843,7 +1843,10 @@ static int create_qp(struct ib_uverbs_file *file, sizeof(cmd->create_flags)) attr.create_flags = cmd->create_flags; - if (attr.create_flags & ~IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) { + if (attr.create_flags & ~(IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK | + IB_QP_CREATE_CROSS_CHANNEL | + IB_QP_CREATE_MANAGED_SEND | + IB_QP_CREATE_MANAGED_RECV)) { ret = -EINVAL; goto err_put; } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index bcf40ad..370fbdf 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -139,6 +139,13 @@ enum ib_device_cap_flags { IB_DEVICE_MEM_WINDOW_TYPE_2B = (1 << 24), IB_DEVICE_RC_IP_CSUM = (1 << 25), IB_DEVICE_RAW_IP_CSUM = (1 << 26), + /* + * Devices should set IB_DEVICE_CROSS_CHANNEL if they + * support execution of WQEs that involve synchronization + * of I/O operations with single completion queue managed + * by hardware. + */ + IB_DEVICE_CROSS_CHANNEL = (1 << 27), IB_DEVICE_MANAGED_FLOW_STEERING = (1 << 29), IB_DEVICE_SIGNATURE_HANDOVER = (1 << 30), IB_DEVICE_ON_DEMAND_PAGING = (1 << 31), @@ -184,6 +191,7 @@ struct ib_odp_caps { enum ib_cq_creation_flags { IB_CQ_FLAGS_TIMESTAMP_COMPLETION = 1 << 0, + IB_CQ_FLAGS_IGNORE_OVERRUN = 1 << 1, }; struct ib_cq_init_attr { @@ -866,6 +874,9 @@ enum ib_qp_type { enum ib_qp_create_flags { IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0, IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1, + IB_QP_CREATE_CROSS_CHANNEL = 1 << 2, + IB_QP_CREATE_MANAGED_SEND = 1 << 3, + IB_QP_CREATE_MANAGED_RECV = 1 << 4, IB_QP_CREATE_NETIF_QP = 1 << 5, IB_QP_CREATE_SIGNATURE_EN = 1 << 6, IB_QP_CREATE_USE_GFP_NOIO = 1 << 7,