From patchwork Sun Jan 27 12:01:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10782871 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1F8D1823 for ; Sun, 27 Jan 2019 12:40:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E19C52B296 for ; Sun, 27 Jan 2019 12:40:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D30462B299; Sun, 27 Jan 2019 12:40:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 76E742B2B4 for ; Sun, 27 Jan 2019 12:40:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726523AbfA0MkV (ORCPT ); Sun, 27 Jan 2019 07:40:21 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:54139 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726458AbfA0MkV (ORCPT ); Sun, 27 Jan 2019 07:40:21 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 27 Jan 2019 14:40:13 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x0RCeDxg017527; Sun, 27 Jan 2019 14:40:13 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id x0RC1koB016760; Sun, 27 Jan 2019 14:01:46 +0200 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id x0RC1kIV016759; Sun, 27 Jan 2019 14:01:46 +0200 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, michaelgur@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH rdma-core 2/3] verbs: Expose PCI atomic operations capabilities in query_device_ex Date: Sun, 27 Jan 2019 14:01:33 +0200 Message-Id: <1548590494-16684-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1548590494-16684-1-git-send-email-yishaih@mellanox.com> References: <1548590494-16684-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Michael Guralnik PCI atomic operations were first introduced in PCIe Base Specification 2.1. The supported operations are swap, fetch & add and compare & swap. Each operation can be supported in a few different operation sizes therefore we expose the capabilities in a bitmask of supported operation sizes per operation type. Unlike other atomic operation modes, pci atomic operations enable an application to perform atomic operations on local memory without involving verbs API and without compromising the operation atomicity. Signed-off-by: Michael Guralnik Signed-off-by: Yishai Hadas --- libibverbs/man/ibv_query_device_ex.3 | 17 +++++++++++++++++ libibverbs/verbs.h | 17 +++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/libibverbs/man/ibv_query_device_ex.3 b/libibverbs/man/ibv_query_device_ex.3 index 2012c63..15a430d 100644 --- a/libibverbs/man/ibv_query_device_ex.3 +++ b/libibverbs/man/ibv_query_device_ex.3 @@ -36,6 +36,7 @@ uint32_t raw_packet_caps; /* Raw packet capabilities, u struct ibv_tm_caps tm_caps; /* Tag matching capabilities */ struct ibv_cq_moderation_caps cq_mod_caps; /* CQ moderation max capabilities */ uint64_t max_dm_size; /* Max Device Memory size (in bytes) available for allocation */ +struct ibv_pci_atomic_caps atomic_caps; /* PCI atomic operations capabilities, use enum ibv_pci_atomic_op_size */ .in -8 }; @@ -107,6 +108,22 @@ struct ibv_cq_moderation_caps { uint16_t max_cq_count; uint16_t max_cq_period; }; + +enum ibv_pci_atomic_op_size { +.in +8 +IBV_PCI_ATOMIC_OPERATION_4_BYTE_SIZE_SUP = 1 << 0, +IBV_PCI_ATOMIC_OPERATION_8_BYTE_SIZE_SUP = 1 << 1, +IBV_PCI_ATOMIC_OPERATION_16_BYTE_SIZE_SUP = 1 << 2, +.in -8 +}; + +struct ibv_pci_atomic_caps { +.in +8 +uint16_t fetch_add; /* Supported sizes for an atomic fetch and add operation, use enum ibv_pci_atomic_op_size */ +uint16_t swap; /* Supported sizes for an atomic unconditional swap operation, use enum ibv_pci_atomic_op_size */ +uint16_t compare_swap; /* Supported sizes for an atomic compare and swap operation, use enum ibv_pci_atomic_op_size */ +.in -8 +}; .fi Extended device capability flags (device_cap_flags_ex): diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h index c9491a7..4cc8720 100644 --- a/libibverbs/verbs.h +++ b/libibverbs/verbs.h @@ -293,6 +293,22 @@ struct ibv_cq_moderation_caps { uint16_t max_cq_period; /* in micro seconds */ }; +enum ibv_pci_atomic_op_size { + IBV_PCI_ATOMIC_OPERATION_4_BYTE_SIZE_SUP = 1 << 0, + IBV_PCI_ATOMIC_OPERATION_8_BYTE_SIZE_SUP = 1 << 1, + IBV_PCI_ATOMIC_OPERATION_16_BYTE_SIZE_SUP = 1 << 2, +}; + +/* + * Bitmask for supported operation sizes + * Use enum ibv_pci_atomic_op_size + */ +struct ibv_pci_atomic_caps { + uint16_t fetch_add; + uint16_t swap; + uint16_t compare_swap; +}; + struct ibv_device_attr_ex { struct ibv_device_attr orig_attr; uint32_t comp_mask; @@ -308,6 +324,7 @@ struct ibv_device_attr_ex { struct ibv_tm_caps tm_caps; struct ibv_cq_moderation_caps cq_mod_caps; uint64_t max_dm_size; + struct ibv_pci_atomic_caps pci_atomic_caps; }; enum ibv_mtu {