From patchwork Mon Nov 3 08:02:44 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eli Cohen X-Patchwork-Id: 5214351 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 79F839FB45 for ; Mon, 3 Nov 2014 08:03:07 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 8C28C2012D for ; Mon, 3 Nov 2014 08:03:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6955520173 for ; Mon, 3 Nov 2014 08:03:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751020AbaKCIDD (ORCPT ); Mon, 3 Nov 2014 03:03:03 -0500 Received: from mail-la0-f45.google.com ([209.85.215.45]:64606 "EHLO mail-la0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751130AbaKCIDD (ORCPT ); Mon, 3 Nov 2014 03:03:03 -0500 Received: by mail-la0-f45.google.com with SMTP id pn19so3682841lab.32 for ; Mon, 03 Nov 2014 00:03:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1lTcO/jL1T+qReq6DBU8VgwIFE1WMXiQRUf2ShLYfwA=; b=nAGMEKZt1tvXK1orGfoU8uNXcMBXLG4NuTcpopKZw5jX0ROWYXrG56yTm93Q0KZ7dp IplYlzVWbheg9qgXErBFnXfuZJQBxj0bjyE1t+N9RzS0J88LfoH6EwaCvN2L2rtjV5oG y6m1ry7gTyYSOIuIIX7wGfsO/H3kqb4emw6cHDZyBfWQMluecHUpVDN5M9WTc15HzoS5 rW65TQKytC+vj+Q4CPt/uTKcGABRl10nBSOlOQ0M8TdpaSWMdeqCP5uC8crvuxIoODda 1eXIID3kdm/KAxTTSiYSksLG5wHozq2TOvJEzg+fLNI1Bf/+Bfk5sbOx2h96Cv/XKoQJ n7Ig== X-Gm-Message-State: ALoCoQnMyk65Da4tcchOCfVFw2nimRtZvwOxLVqe1XrQqciHrkFvCi4YuDQJRxrs8JXxEvEu5G88 X-Received: by 10.152.28.101 with SMTP id a5mr49747294lah.52.1415001781626; Mon, 03 Nov 2014 00:03:01 -0800 (PST) Received: from localhost (out.voltaire.com. [193.47.165.251]) by mx.google.com with ESMTPSA id f2sm4617378lag.29.2014.11.03.00.03.00 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 03 Nov 2014 00:03:01 -0800 (PST) From: Eli Cohen X-Google-Original-From: Eli Cohen To: roland@kernel.org Cc: linux-rdma@vger.kernel.org, ogerlitz@mellanox.com, yevgenyp@mellanox.com, Eli Cohen Subject: [PATCH for-next 3/5] IB/core: Extend atomic operations Date: Mon, 3 Nov 2014 10:02:44 +0200 Message-Id: <1415001766-8366-4-git-send-email-eli@mellanox.com> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1415001766-8366-1-git-send-email-eli@mellanox.com> References: <1415001766-8366-1-git-send-email-eli@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Further enhance the extended atomic operations support as was introduced in commit 5e80ba8ff0bd "IB/core: Add support for masked atomic operations". 1. Allow arbitrary argument sizes. The original extended atomics commit defined 64 bits arguments. This patch allows arbitrary arguments which are power of 2 bytes in size. 2. Add the option to define response for atomic operations in network order. enum ib_atomic_cap is extended to have big endian variants. The device attributes struct defines three new fields: log_atomic_arg_sizes - is a bit mask which encodes which argument sizes are supported. A set bit at location n (zero based) means an argument of size 2 ^ n is supported. max_fa_bit_boundary - Max fetch and add bit boundary. Multi field fetch and add operations use a bit mask that defines bit locations where carry bit is not passed to the next higher order bit. So, if this field has the value 64, it means that the max value subject to fetch and add is 64 bits which means no carry from bit 63 to 64 or from bit 127 to 128 etc. log_max_atomic_inline - atomic arguments can be inline in the WQE or be referenced through a memory key. This value defines the max inline argument size possible. Signed-off-by: Eli Cohen --- drivers/infiniband/core/uverbs_cmd.c | 16 +++++++++++++++- include/rdma/ib_verbs.h | 7 ++++++- include/uapi/rdma/ib_user_verbs.h | 14 ++++++++++++++ 3 files changed, 35 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index da3fd8cdd86e..f66686e40ae6 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -445,6 +445,8 @@ ssize_t ib_uverbs_query_device(struct ib_uverbs_file *file, memset(&resp, 0, sizeof resp); copy_query_dev_fields(file, &resp, &attr); + if (resp.atomic_cap > IB_ATOMIC_GLOB) + resp.atomic_cap = IB_ATOMIC_NONE; if (copy_to_user((void __user *) (unsigned long) cmd.response, &resp, sizeof resp)) @@ -3281,7 +3283,7 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file, if (err) return err; - if (cmd.comp_mask) + if (cmd.comp_mask & ~IB_UVERBS_EX_QUERY_DEV_MAX_MASK) return -EINVAL; err = device->query_device(device, &attr); @@ -3292,6 +3294,18 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file, copy_query_dev_fields(file, &resp.base, &attr); resp.comp_mask = 0; + if (cmd.comp_mask & IB_UVERBS_EX_QUERY_DEV_MASKED_ATOMIC) { + resp.atomics.masked_atomic_cap = attr.masked_atomic_cap; + resp.atomics.log_atomic_arg_sizes = attr.log_atomic_arg_sizes; + resp.atomics.max_fa_bit_boundary = attr.max_fa_bit_boundary; + resp.atomics.log_max_atomic_inline = attr.log_max_atomic_inline; + resp.comp_mask |= IB_UVERBS_EX_QUERY_DEV_MASKED_ATOMIC; + } else { + resp.atomics.masked_atomic_cap = IB_ATOMIC_NONE; + resp.atomics.log_atomic_arg_sizes = 0; + resp.atomics.max_fa_bit_boundary = 0; + resp.atomics.log_max_atomic_inline = 0; + } err = ib_copy_to_udata(ucore, &resp, sizeof(resp)); if (err) return err; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 470a011d6fa4..7f1d20a3f4c7 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -140,7 +140,9 @@ enum ib_signature_guard_cap { enum ib_atomic_cap { IB_ATOMIC_NONE, IB_ATOMIC_HCA, - IB_ATOMIC_GLOB + IB_ATOMIC_GLOB, + IB_ATOMIC_HCA_REPLY_BE, + IB_ATOMIC_GLOB_REPLY_BE, }; struct ib_device_attr { @@ -186,6 +188,9 @@ struct ib_device_attr { u8 local_ca_ack_delay; int sig_prot_cap; int sig_guard_cap; + u32 log_atomic_arg_sizes; /* bit-mask of supported sizes */ + u32 max_fa_bit_boundary; + u32 log_max_atomic_inline; }; enum ib_mtu { diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index ed8c3d9da42c..ec98fe636f2b 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -202,13 +202,27 @@ struct ib_uverbs_query_device_resp { __u8 reserved[4]; }; +enum { + IB_UVERBS_EX_QUERY_DEV_MASKED_ATOMIC = 1 << 0, + IB_UVERBS_EX_QUERY_DEV_LAST = 1 << 1, + IB_UVERBS_EX_QUERY_DEV_MAX_MASK = IB_UVERBS_EX_QUERY_DEV_LAST - 1, +}; + struct ib_uverbs_ex_query_device { __u32 comp_mask; }; +struct ib_uverbs_ex_atomic_caps { + __u32 masked_atomic_cap; + __u32 log_atomic_arg_sizes; /* bit-mask of supported sizes */ + __u32 max_fa_bit_boundary; + __u32 log_max_atomic_inline; +}; + struct ib_uverbs_ex_query_device_resp { struct ib_uverbs_query_device_resp base; __u32 comp_mask; + struct ib_uverbs_ex_atomic_caps atomics; }; struct ib_uverbs_query_port {