From patchwork Sun Dec 13 06:18:35 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Leon Romanovsky X-Patchwork-Id: 7838601 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id D28139F350 for ; Sun, 13 Dec 2015 06:21:48 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id BF94F2038D for ; Sun, 13 Dec 2015 06:21:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BB51520379 for ; Sun, 13 Dec 2015 06:21:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751057AbbLMGVq (ORCPT ); Sun, 13 Dec 2015 01:21:46 -0500 Received: from mail-wm0-f47.google.com ([74.125.82.47]:35156 "EHLO mail-wm0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751007AbbLMGVp (ORCPT ); Sun, 13 Dec 2015 01:21:45 -0500 Received: by mail-wm0-f47.google.com with SMTP id p66so5493003wmp.0 for ; Sat, 12 Dec 2015 22:21:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leon-nu.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=Az9Yp5R3ZYetnaVeF4b1CCLYd3/4y816Ou29AShDG2A=; b=PIBd1g6FNDyOqSxPEajX0S3uu4djoQNE6gPOCaa/2lclotym5rPyzttwSdaed4LO8c +hcdfqiTPJOezY48a9HXFZj52Myik2/69DiLY8q/6tOfFWsPOFvWBOGP7Jq97dBLO8U9 o1sjXrLg2mTaz63bAUw9aQ7sh/FRKlCfpFr7osT9ihcxi9Xq850HQfz54ybjLjYXZHb0 lcHVtkgrIUf91cRJdBtCrq2oR71fARZqS+mTgHRr4Qi3GNx6jY91Odg2CYNMMjf72Gg5 BoX7XlnX9JBjAWoMO7WYf49+Fhsqd1Gi5o9DoyfZsJ8W5mZizThlTQIg3GxE9keA1Er+ sM1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=Az9Yp5R3ZYetnaVeF4b1CCLYd3/4y816Ou29AShDG2A=; b=hu2uoYASnS1/kv+xVI6iO35NqttRjKPKs+hxqB7f91sLnDOpn4euNrZpNAeJBZX5+T 28YiQBR6E4kvwakC9hi4v8GuDjSUqA7ijzwLWTWPNp+6EV2zmluaZNwHarYzMAM34vI0 5GfRpvaiJLD/0C9ppoE6YzHl4lFwvlHOMSjLkMEMk/tVZmWjFhT4QjqjsAXQPi0pQPdO p1TcX0nex6h3ZLIOdB2VUvzzOpN5raJOZ0osRsF38iFl3mQIUA/2EW/ulDPU5pOm2hOl XZjhGu3tOy/wrAhGCvFS+uP1DEAuAPryhkSvPOgIn0TzlI/0IDu5QeHhoMnGGJUIyIwL epGA== X-Gm-Message-State: ALoCoQk3O3eNcwft4cx1M0+KG28bjphNGZKeM7qMy8TRNz9ZrGR0kQziIKffilwT43rQJzXvE8Fj6dY3l8kGuh9brFHs4Rm1fg== X-Received: by 10.28.57.65 with SMTP id g62mr15450897wma.65.1449987704124; Sat, 12 Dec 2015 22:21:44 -0800 (PST) Received: from localhost ([213.57.247.249]) by smtp.gmail.com with ESMTPSA id h5sm23966713wjz.21.2015.12.12.22.21.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 12 Dec 2015 22:21:43 -0800 (PST) From: Leon Romanovsky To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Leon Romanovsky Subject: [PATCH 2/3] IB/core: Add cross-channel support Date: Sun, 13 Dec 2015 08:18:35 +0200 Message-Id: <1449987516-31608-3-git-send-email-leon@leon.nu> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1449987516-31608-1-git-send-email-leon@leon.nu> References: <1449987516-31608-1-git-send-email-leon@leon.nu> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Leon Romanovsky The cross-channel feature allows to execute WQEs that involve synchronization of I/O operations’ on different QPs. This capability enables to program complex flows with a single function call, hereby significantly reducing overhead associated with I/O processing. Cross-channel operations support is indicated by HCA capability information. The queue pairs can be configured to work as a “sync master queue” or “sync slave queues”. The added flags are: 1. Device capability flag IB_DEVICE_CROSS_CHANNEL for the devices that can perform cross-channel operations. 2. CQ property flag IB_CQ_FLAGS_IGNORE_OVERRUN to disable CQ overrun check. This check is useless in cross-channel scenario. 3. QP property flags to indicate if queues are slave or master: * IB_QP_CREATE_MANAGED_SEND indicates that posted send work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_MANAGED_RECV indicates that posted receive work requests will not be executed immediately and requires enabling. * IB_QP_CREATE_CROSS_CHANNEL declares the QP to work in cross-channel mode. If IB_QP_CREATE_MANAGED_SEND and IB_QP_CREATE_MANAGED_RECV are not provided, this QP will be sync master queue, else it will be sync slave. Reviewed-by: Sagi Grimberg Signed-off-by: Leon Romanovsky --- drivers/infiniband/core/uverbs_cmd.c | 5 ++++- include/rdma/ib_verbs.h | 11 +++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 94816ae..0e9711f 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -1843,7 +1843,10 @@ static int create_qp(struct ib_uverbs_file *file, sizeof(cmd->create_flags)) attr.create_flags = cmd->create_flags; - if (attr.create_flags & ~IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) { + if (attr.create_flags & ~(IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK | + IB_QP_CREATE_CROSS_CHANNEL | + IB_QP_CREATE_MANAGED_SEND | + IB_QP_CREATE_MANAGED_RECV)) { ret = -EINVAL; goto err_put; } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index bcf40ad..370fbdf 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -139,6 +139,13 @@ enum ib_device_cap_flags { IB_DEVICE_MEM_WINDOW_TYPE_2B = (1 << 24), IB_DEVICE_RC_IP_CSUM = (1 << 25), IB_DEVICE_RAW_IP_CSUM = (1 << 26), + /* + * Devices should set IB_DEVICE_CROSS_CHANNEL if they + * support execution of WQEs that involve synchronization + * of I/O operations with single completion queue managed + * by hardware. + */ + IB_DEVICE_CROSS_CHANNEL = (1 << 27), IB_DEVICE_MANAGED_FLOW_STEERING = (1 << 29), IB_DEVICE_SIGNATURE_HANDOVER = (1 << 30), IB_DEVICE_ON_DEMAND_PAGING = (1 << 31), @@ -184,6 +191,7 @@ struct ib_odp_caps { enum ib_cq_creation_flags { IB_CQ_FLAGS_TIMESTAMP_COMPLETION = 1 << 0, + IB_CQ_FLAGS_IGNORE_OVERRUN = 1 << 1, }; struct ib_cq_init_attr { @@ -866,6 +874,9 @@ enum ib_qp_type { enum ib_qp_create_flags { IB_QP_CREATE_IPOIB_UD_LSO = 1 << 0, IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1, + IB_QP_CREATE_CROSS_CHANNEL = 1 << 2, + IB_QP_CREATE_MANAGED_SEND = 1 << 3, + IB_QP_CREATE_MANAGED_RECV = 1 << 4, IB_QP_CREATE_NETIF_QP = 1 << 5, IB_QP_CREATE_SIGNATURE_EN = 1 << 6, IB_QP_CREATE_USE_GFP_NOIO = 1 << 7,