From patchwork Thu Feb 5 11:10:37 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Majd Dibbiny X-Patchwork-Id: 5783951 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id E40CC9F30C for ; Thu, 5 Feb 2015 11:32:42 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 112EF2010B for ; Thu, 5 Feb 2015 11:32:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 13F21201EC for ; Thu, 5 Feb 2015 11:32:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756648AbbBELci (ORCPT ); Thu, 5 Feb 2015 06:32:38 -0500 Received: from mailp.voltaire.com ([193.47.165.129]:53896 "EHLO mellanox.co.il" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1757217AbbBELch (ORCPT ); Thu, 5 Feb 2015 06:32:37 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from majd@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Feb 2015 13:10:39 +0200 Received: from dev-l-vrt-196.mtl.labs.mlnx (dev-l-vrt-196.mtl.labs.mlnx [10.134.196.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id t15BAdT8018989; Thu, 5 Feb 2015 13:10:39 +0200 From: majd@mellanox.com To: roland@kernel.org Cc: linux-rdma@vger.kernel.org, haggaie@mellanox.com, talal@mellanox.com, Majd Dibbiny Subject: [PATCH libibverbs 3/3] Add Send with invalidate support Date: Thu, 5 Feb 2015 13:10:37 +0200 Message-Id: <1423134637-14438-4-git-send-email-majd@mellanox.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1423134637-14438-1-git-send-email-majd@mellanox.com> References: <1423134637-14438-1-git-send-email-majd@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Majd Dibbiny Send with invalidate can be used to invalidate a remote memory region or memory window. The complete details are specified in the InfiniBand Architecture Specifications, section 9.4.1.1. According to the specs, the following objects can be invalidated: 1. Memory Window type 2 2. Physical MR 3. Fast register MR The invalidation is done by posting a send work request, where the opcode is IBV_WR_SEND_WITH_INV and the immediate data contain's the R_key. Upon success, the responder's work completion will contain the invalidated R_key in it's immediate data. Signed-off-by: Majd Dibbiny --- include/infiniband/verbs.h | 11 ++++++++--- man/ibv_post_send.3 | 1 + 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/include/infiniband/verbs.h b/include/infiniband/verbs.h index dcee050..b9a930f 100644 --- a/include/infiniband/verbs.h +++ b/include/infiniband/verbs.h @@ -294,7 +294,8 @@ enum ibv_wc_opcode { enum ibv_wc_flags { IBV_WC_GRH = 1 << 0, - IBV_WC_WITH_IMM = 1 << 1 + IBV_WC_WITH_IMM = 1 << 1, + IBV_WC_WITH_INV = 1 << 2 }; struct ibv_wc { @@ -303,7 +304,10 @@ struct ibv_wc { enum ibv_wc_opcode opcode; uint32_t vendor_err; uint32_t byte_len; - uint32_t imm_data; /* in network byte order */ + /* When (wc_flags & IBV_WC_WITH_IMM): Immediate data (network byte order). + * When (wc_flags & IBV_WC_WITH_INV): Stores the invalidated R_Key. + */ + uint32_t imm_data; uint32_t qp_num; uint32_t src_qp; int wc_flags; @@ -636,7 +640,8 @@ enum ibv_wr_opcode { IBV_WR_ATOMIC_CMP_AND_SWP, IBV_WR_ATOMIC_FETCH_AND_ADD, IBV_WR_LOCAL_INV, - IBV_WR_BIND_MW + IBV_WR_BIND_MW, + IBV_WR_SEND_WITH_INV }; enum ibv_send_flags { diff --git a/man/ibv_post_send.3 b/man/ibv_post_send.3 index 0d599ad..891cf4a 100644 --- a/man/ibv_post_send.3 +++ b/man/ibv_post_send.3 @@ -115,6 +115,7 @@ IBV_WR_ATOMIC_CMP_AND_SWP | | | X IBV_WR_ATOMIC_FETCH_AND_ADD | | | X IBV_WR_LOCAL_INV | | X | X IBV_WR_BIND_MW | | X | X +IBV_WR_SEND_WITH_INV | | X | X .fi .PP The attribute send_flags describes the properties of the \s-1WR\s0. It is either 0 or the bitwise \s-1OR\s0 of one or more of the following flags: