From patchwork Thu Dec 24 08:51:11 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 7915771 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 18D79BEEE5 for ; Thu, 24 Dec 2015 08:52:16 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 23B43205C1 for ; Thu, 24 Dec 2015 08:52:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CC760205B5 for ; Thu, 24 Dec 2015 08:52:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751410AbbLXIv5 (ORCPT ); Thu, 24 Dec 2015 03:51:57 -0500 Received: from mail-wm0-f43.google.com ([74.125.82.43]:33715 "EHLO mail-wm0-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751023AbbLXIv4 (ORCPT ); Thu, 24 Dec 2015 03:51:56 -0500 Received: by mail-wm0-f43.google.com with SMTP id p187so171291767wmp.0 for ; Thu, 24 Dec 2015 00:51:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leon-nu.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=CE2aML3tmYneFwM76bQyFgVFBYRggV5tNFfx26edDTo=; b=Jrkh+MwGIZCbNnSemQB3aN0/f4uhHzUhWI/TiNdEof1gnFQ+CkzSC90bzLiM2/IpIc f+P+apllCCZgMuSdZa3MgmHEhhBtnJaXI/rsRjSlSh8J8Zcdlvtjeft6/MAdP/sDMhy/ mWzwS4GFnOiAHQR4dB+SUA+At6JtYlksa4mui+75L0EkNRVxmBytsJW5LIrlLuucpwr+ u4HEDxO/vj6Iy4lP1tCWgmlyRJIV3OAyG5/R5Y1+42A+nVUX/SfIA/J51U+VYNmUJJRI DDyyGfiPNTMCSD1DSMeGT3QGpKv0jl7twKGHNklRZmu39k6NLGuBCDsQ+1QbUzMkdR5t taeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=CE2aML3tmYneFwM76bQyFgVFBYRggV5tNFfx26edDTo=; b=OMN8jXvT1kggSMtGlpr4ukEBy3P2ZHqujEJoikp1L6d1nNJaBc3ZK+IGXMNFUrU5pe RrN/+byyonndTBtJeC/+qLIacBDIgM1eW7yyI+Fpoa8AMSHA2SiLQq2mG1ioEpdOOHpJ sQzJXWLRVxsXechs21ktsFu2mDVabW7wjaFvpDLKqYbvuiLEUN1dhTDSIcVRftxE1TAF nimSI/nXK4HE/8qkw4/4aKnGS4+3q4Cq+20hjlvp0Je1hmaP8lciGOh5Y3uBevLXjD3o 7Mi4f6W8+Wffq8Ko1K/CzYIVJHWfQJZzg/t6IOzSIw76Uqmd/EVNoIlqtOWE80InExvi kBPg== X-Gm-Message-State: ALoCoQkhmkL7oMTxNN330OYiPfYNi+LsEIO78VCytCj8N0oZwZ5KWu4YnUQ7hGHLUj0wXElDSlgv8to+i5okyz1bYRM8RHg0nA== X-Received: by 10.194.24.97 with SMTP id t1mr38560246wjf.127.1450947115152; Thu, 24 Dec 2015 00:51:55 -0800 (PST) Received: from localhost ([213.57.247.249]) by smtp.gmail.com with ESMTPSA id k193sm25713397wmd.16.2015.12.24.00.51.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Dec 2015 00:51:54 -0800 (PST) From: Leon Romanovsky To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Leon Romanovsky Subject: [PATCH V2 1/2] IB/core: Add cross-channel support Date: Thu, 24 Dec 2015 10:51:11 +0200 Message-Id: <1450947072-9295-2-git-send-email-leon@leon.nu> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1450947072-9295-1-git-send-email-leon@leon.nu> References: <1450947072-9295-1-git-send-email-leon@leon.nu> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Leon Romanovsky The cross-channel feature allows to execute WQEs that involve synchronization of IO operations’ on different QPs. This capability enables to program complex flows with a single function call, hereby significantly reducing overhead associated with IO processing. The queue pairs can be configured to work as a “sync master queue” or “sync slave queues”. The added flags are: 1. Device capability flag IB_DEVICE_CROSS_CHANNEL for the devices that can perform cross-channel operations. 2. CQ property flag IB_CQ_FLAGS_IGNORE_OVERRUN to disable CQ overrun check. In cross-channel mode, the send/receive queues will forward their completions to managing QP. 3. QP property flags to indicate if queues are slave or master: * IB_QP_CREATE_MANAGED_SEND indicates that posted send work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_MANAGED_RECV indicates that posted receive work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_CROSS_CHANNEL declares the QP to work in cross-channel mode. If IB_QP_CREATE_MANAGED_SEND and IB_QP_CREATE_MANAGED_RECV are not provided, this QP will be sync master queue, else it will be sync slave. Signed-off-by: Leon Romanovsky Reviewed-by: Sagi Grimberg --- drivers/infiniband/core/uverbs_cmd.c | 5 ++++- include/rdma/ib_verbs.h | 11 +++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 1add53653825..63ddd6a122b0 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1829,7 +1829,10 @@ static int create_qp(struct ib_uverbs_file *file, sizeof(cmd->create_flags)) attr.create_flags = cmd->create_flags; - if (attr.create_flags & ~IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) { + if (attr.create_flags & ~(IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK | + IB_QP_CREATE_CROSS_CHANNEL | + IB_QP_CREATE_MANAGED_SEND | + IB_QP_CREATE_MANAGED_RECV)) { ret = -EINVAL; goto err_put; } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index a7dbbfc9d202..d00f3e2e39ac 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -141,6 +141,13 @@ enum ib_device_cap_flags { IB_DEVICE_MEM_WINDOW_TYPE_2B = (1<<24), IB_DEVICE_RC_IP_CSUM = (1<<25), IB_DEVICE_RAW_IP_CSUM = (1<<26), + /* + * Devices should set IB_DEVICE_CROSS_CHANNEL if they + * support execution of WQEs that involve synchronization + * of I/O operations with single completion queue managed + * by hardware. + */ + IB_DEVICE_CROSS_CHANNEL = (1<<27), IB_DEVICE_MANAGED_FLOW_STEERING = (1<<29), IB_DEVICE_SIGNATURE_HANDOVER = (1<<30), IB_DEVICE_ON_DEMAND_PAGING = (1<<31), @@ -186,6 +193,7 @@ struct ib_odp_caps { enum ib_cq_creation_flags { IB_CQ_FLAGS_TIMESTAMP_COMPLETION = 1 << 0, + IB_CQ_FLAGS_IGNORE_OVERRUN = 1 << 1, }; struct ib_cq_init_attr { @@ -823,6 +831,9 @@ enum ib_qp_type { enum ib_qp_create_flags { IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0, IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1, + IB_QP_CREATE_CROSS_CHANNEL = 1 << 2, + IB_QP_CREATE_MANAGED_SEND = 1 << 3, + IB_QP_CREATE_MANAGED_RECV = 1 << 4, IB_QP_CREATE_NETIF_QP = 1 << 5, IB_QP_CREATE_SIGNATURE_EN = 1 << 6, IB_QP_CREATE_USE_GFP_NOIO = 1 << 7,