From patchwork Thu Apr 29 18:48:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 081DBC433B4 for ; Thu, 29 Apr 2021 18:49:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D226A61408 for ; Thu, 29 Apr 2021 18:49:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241005AbhD2St7 (ORCPT ); Thu, 29 Apr 2021 14:49:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233565AbhD2St6 (ORCPT ); Thu, 29 Apr 2021 14:49:58 -0400 Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A573BC06138B for ; Thu, 29 Apr 2021 11:49:11 -0700 (PDT) Received: by mail-ot1-x331.google.com with SMTP id d3-20020a9d29030000b029027e8019067fso60393488otb.13 for ; Thu, 29 Apr 2021 11:49:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bJeUsGrNnZQzzCLtjN2Dy6J69IfylXkxR5VCG0J4XL4=; b=ZplUMl5DPUl/hHvdNlVUhMWHNjbwuwfVWAHXiPzlVTnmv1fbuKUFIjIq+6IyHA6VTx atHuMwApMOvCaY93bt59dev+ZfylLz2rJ4bO4vX0nRcY/AMFKLcChRNzXidFJWw4fXch 4cXq07f9wtcQNWdJ1gkbbRp7UELSmMJfTWWKnAFcwxQbgWIOml6ViwbHO0ekVzWcc0mu 59zPPeY48QcQEQTGTB06Hf39+p3YnwgW+/C8uUpsq6TgfewzXsxHbs4nAWRiKO8rvpTx OMH5SbPEBYjpGocrFx0HnjQwzBHSUxsX0ateFjdvH7jSmvf8fhfmfsSytgsHKROUDR1D 9Brw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bJeUsGrNnZQzzCLtjN2Dy6J69IfylXkxR5VCG0J4XL4=; b=aQD6frP4PADSrLu/u++oxEBWLK+TKtTCdRLFf+ZgtznMXTszzEFOa/wZQfyF0WutgR NfP1V+kyDFfdyPl7U5gkmeqBPUIn6D7sXOsVKRiSn4JzsgVCXri9gh+S8w7/RZZjWwoQ wR+3IDuVu+Yq9qAYxvEIdCkjcrizT1HtOFxg+t743ZoGFBxfan3d5EZkBWVaaMVd4XN0 juShCCDkARj8yU1hWFnYOmIntqPzZw4v7wyyFA9SsErciXxApzwW3lY8xeZMaD8lag77 icQsAiD+WeP3Oqcu89EozMp311zzULlIZWw2D2GgBJMcQz/p/89oX7sJrFWy6vU3yCES 49sw== X-Gm-Message-State: AOAM53101z/UKUyjxDDyHxziuCujPdFEL4cUFH3sAK3aZDTksz2Q65CY JVBVBMzULNFwykY5AO1vh8Y= X-Google-Smtp-Source: ABdhPJwEpnYCn8dgGMx1sclkCvF497Mnx0nzjyrYcK1e6ADppBP3h8Uw8DOTpzeeQzr9sWLMAUY64A== X-Received: by 2002:a9d:6a88:: with SMTP id l8mr663234otq.236.1619722151175; Thu, 29 Apr 2021 11:49:11 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id c1sm126795oto.20.2021.04.29.11.49.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:10 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 01/10] RDMA/rxe: Add bind MW fields to rxe_send_wr Date: Thu, 29 Apr 2021 13:48:46 -0500 Message-Id: <20210429184855.54939-2-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add fields to struct rxe_send_wr in rdma_user_rxe.h to support bind MW work requests Signed-off-by: Bob Pearson --- include/uapi/rdma/rdma_user_rxe.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index 068433e2229d..b8f408ceb1a8 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -99,7 +99,17 @@ struct rxe_send_wr { __u32 remote_qkey; __u16 pkey_index; } ud; + struct { + __aligned_u64 addr; + __aligned_u64 length; + __u32 mr_lkey; + __u32 mw_rkey; + __u32 rkey; + __u32 access; + __u32 flags; + } mw; /* reg is only used by the kernel and is not part of the uapi */ +#ifdef __KERNEL__ struct { union { struct ib_mr *mr; @@ -108,6 +118,7 @@ struct rxe_send_wr { __u32 key; __u32 access; } reg; +#endif } wr; }; From patchwork Thu Apr 29 18:48:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F3BFC43461 for ; Thu, 29 Apr 2021 18:49:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 38A41613B3 for ; Thu, 29 Apr 2021 18:49:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241025AbhD2SuA (ORCPT ); Thu, 29 Apr 2021 14:50:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235685AbhD2St7 (ORCPT ); Thu, 29 Apr 2021 14:49:59 -0400 Received: from mail-oo1-xc2b.google.com (mail-oo1-xc2b.google.com [IPv6:2607:f8b0:4864:20::c2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77C8AC06138B for ; Thu, 29 Apr 2021 11:49:12 -0700 (PDT) Received: by mail-oo1-xc2b.google.com with SMTP id p6-20020a4adc060000b02901f9a8fc324fso1266856oov.10 for ; Thu, 29 Apr 2021 11:49:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+ot6DW/4UHvRQa+b/PnhlC+ylAGNIjr8rvlsEuhnTXg=; b=JmwsyanG9sk9c00UrRBv+Wb2W1J3HkaQzagi2nLAx+66Bc/L9tMsyyNRZNGBl+tLeH u8diMw/fKoImWBEgsGpYyeagP7vI2c+M+zTV4I+kqHFJxDzy9lMFvAHpk5boB2pEdQs4 icoP3u6ljGyocTBenStorU4wSJhPIReGAmKExu1bN3e4knQug1g88ZwLGiPd/o4PqZhm 0HJeSjf9hna7Odkccc7FfyrH78nfufCcRoEY80ntmierjwxFcnDPa3wwFLxgZkI0JRlF JRnRrs60ZrrY2I5yF4PYa6OoGpu0c13bdOjEKMYiblIRDNfkMBvGjcNGHW+5/eOdFHQQ LDUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+ot6DW/4UHvRQa+b/PnhlC+ylAGNIjr8rvlsEuhnTXg=; b=ofzvtgnxMo7qhjuLhRE/qZ5me0g0U7GaZw3nWeJV52X22pm/LMafBBVeJI9IzqvInm 4nswgfB9VGYImliHN8E6JJJEATeFNc00O/ZE0k8RCHPjDjIRT+Va3RcuToe7HWokmP3M 8v7xqaXYJGIr8RRzHXva86roewyXImeMTqevbrBEICCwoOWLfPXLTzIKOfsrboYSQ661 PjsAs4grQDHnFvbYoGjztjnBHDjgDQMobP5WBx/Yqpuz1LEk9zeiPHpggkNhcGOOdYvZ IqsAqcMbRuxeRnBA3zf7oYk8mWDuyyLRJZzmwrHWL560X3H6FYiXT5vZx46S2+8ZcaoK lbrQ== X-Gm-Message-State: AOAM533AHopNWQg5o5UbityfVkWjYRBu2/hACpNdUEdUGDHXr9btbKT1 DdBTXcZOOjDVTrXH3qjCX0w= X-Google-Smtp-Source: ABdhPJz9RQbq9ebP/M4ZrcujTSLaz5Icvj/HoqgUIIp2dcz5HeC8UeH6GJCP7Bggs9bKxaJqXXUF7w== X-Received: by 2002:a4a:4f4d:: with SMTP id c74mr1132278oob.92.1619722151895; Thu, 29 Apr 2021 11:49:11 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id n37sm155319otn.9.2021.04.29.11.49.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:11 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 02/10] RDMA/rxe: Return errors for add index and key Date: Thu, 29 Apr 2021 13:48:47 -0500 Message-Id: <20210429184855.54939-3-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Modify rxe_add_index() and rxe_add_key() to return an error if the index or key is aleady present in the pool. Currently they print a warning and silently fail with bad consequences to the caller. Signed-off-by: Bob Pearson --- v6: Added rxe_ to static subroutine names in changed lines. --- drivers/infiniband/sw/rxe/rxe_pool.c | 44 ++++++++++++++++++---------- drivers/infiniband/sw/rxe/rxe_pool.h | 8 ++--- 2 files changed, 32 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index d24901f2af3f..df0bec719341 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -183,7 +183,7 @@ static u32 alloc_index(struct rxe_pool *pool) return index + pool->index.min_index; } -static void insert_index(struct rxe_pool *pool, struct rxe_pool_entry *new) +static int rxe_insert_index(struct rxe_pool *pool, struct rxe_pool_entry *new) { struct rb_node **link = &pool->index.tree.rb_node; struct rb_node *parent = NULL; @@ -195,7 +195,7 @@ static void insert_index(struct rxe_pool *pool, struct rxe_pool_entry *new) if (elem->index == new->index) { pr_warn("element already exists!\n"); - goto out; + return -EINVAL; } if (elem->index > new->index) @@ -206,11 +206,11 @@ static void insert_index(struct rxe_pool *pool, struct rxe_pool_entry *new) rb_link_node(&new->index_node, parent, link); rb_insert_color(&new->index_node, &pool->index.tree); -out: - return; + + return 0; } -static void insert_key(struct rxe_pool *pool, struct rxe_pool_entry *new) +static int rxe_insert_key(struct rxe_pool *pool, struct rxe_pool_entry *new) { struct rb_node **link = &pool->key.tree.rb_node; struct rb_node *parent = NULL; @@ -226,7 +226,7 @@ static void insert_key(struct rxe_pool *pool, struct rxe_pool_entry *new) if (cmp == 0) { pr_warn("key already exists!\n"); - goto out; + return -EINVAL; } if (cmp > 0) @@ -237,26 +237,32 @@ static void insert_key(struct rxe_pool *pool, struct rxe_pool_entry *new) rb_link_node(&new->key_node, parent, link); rb_insert_color(&new->key_node, &pool->key.tree); -out: - return; + + return 0; } -void __rxe_add_key_locked(struct rxe_pool_entry *elem, void *key) +int __rxe_add_key_locked(struct rxe_pool_entry *elem, void *key) { struct rxe_pool *pool = elem->pool; + int err; memcpy((u8 *)elem + pool->key.key_offset, key, pool->key.key_size); - insert_key(pool, elem); + err = rxe_insert_key(pool, elem); + + return err; } -void __rxe_add_key(struct rxe_pool_entry *elem, void *key) +int __rxe_add_key(struct rxe_pool_entry *elem, void *key) { struct rxe_pool *pool = elem->pool; unsigned long flags; + int err; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_add_key_locked(elem, key); + err = __rxe_add_key_locked(elem, key); write_unlock_irqrestore(&pool->pool_lock, flags); + + return err; } void __rxe_drop_key_locked(struct rxe_pool_entry *elem) @@ -276,22 +282,28 @@ void __rxe_drop_key(struct rxe_pool_entry *elem) write_unlock_irqrestore(&pool->pool_lock, flags); } -void __rxe_add_index_locked(struct rxe_pool_entry *elem) +int __rxe_add_index_locked(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; + int err; elem->index = alloc_index(pool); - insert_index(pool, elem); + err = rxe_insert_index(pool, elem); + + return err; } -void __rxe_add_index(struct rxe_pool_entry *elem) +int __rxe_add_index(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; unsigned long flags; + int err; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_add_index_locked(elem); + err = __rxe_add_index_locked(elem); write_unlock_irqrestore(&pool->pool_lock, flags); + + return err; } void __rxe_drop_index_locked(struct rxe_pool_entry *elem) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 61210b300a78..1feca1bffced 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -111,11 +111,11 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem); /* assign an index to an indexed object and insert object into * pool's rb tree holding and not holding the pool_lock */ -void __rxe_add_index_locked(struct rxe_pool_entry *elem); +int __rxe_add_index_locked(struct rxe_pool_entry *elem); #define rxe_add_index_locked(obj) __rxe_add_index_locked(&(obj)->pelem) -void __rxe_add_index(struct rxe_pool_entry *elem); +int __rxe_add_index(struct rxe_pool_entry *elem); #define rxe_add_index(obj) __rxe_add_index(&(obj)->pelem) @@ -133,11 +133,11 @@ void __rxe_drop_index(struct rxe_pool_entry *elem); /* assign a key to a keyed object and insert object into * pool's rb tree holding and not holding pool_lock */ -void __rxe_add_key_locked(struct rxe_pool_entry *elem, void *key); +int __rxe_add_key_locked(struct rxe_pool_entry *elem, void *key); #define rxe_add_key_locked(obj, key) __rxe_add_key_locked(&(obj)->pelem, key) -void __rxe_add_key(struct rxe_pool_entry *elem, void *key); +int __rxe_add_key(struct rxe_pool_entry *elem, void *key); #define rxe_add_key(obj, key) __rxe_add_key(&(obj)->pelem, key) From patchwork Thu Apr 29 18:48:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1B20C433ED for ; Thu, 29 Apr 2021 18:49:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2BF261408 for ; Thu, 29 Apr 2021 18:49:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235685AbhD2SuA (ORCPT ); Thu, 29 Apr 2021 14:50:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233565AbhD2SuA (ORCPT ); Thu, 29 Apr 2021 14:50:00 -0400 Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39EBAC06138B for ; Thu, 29 Apr 2021 11:49:13 -0700 (PDT) Received: by mail-ot1-x331.google.com with SMTP id g4-20020a9d6b040000b029029debbbb3ecso29303603otp.7 for ; Thu, 29 Apr 2021 11:49:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0oK6GdjgEoldENJJRDmDumo1Lxa+DQ2oSoWDS5c4v9I=; b=dzkkKSYsZ32V9R8D0SBD8X5d9Nu5ZthmUosFH8AzjOYDjpf1NaXGeYmtiED97Dr9d2 FumxLu2b2MgioGcxdDdNiRIE6l/dz2evZ5nbOdomAEpeBfVwEmVvYxyM78eDcwRwrZGQ FlfNiLVY4koxaGh3nqw6dqODlzuvVV668rpglUCTkg9VeZVplg+hsrdQFpQYx2cD40U6 2PvCUety4J6actF1EApyr6Afr5itdko4BZDfVFFuFCnBPYEogK3Tke5DV7gdugkNwajz /s6OXA9kVSRA0xj4023kF9WfUo5vBOyGMLwbeaA5sz4XFY2+k+kVU6TgatdbUIvUPMQ1 lZog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0oK6GdjgEoldENJJRDmDumo1Lxa+DQ2oSoWDS5c4v9I=; b=sHZET/hDzQgiUGW6MtB4ZB4p9cNhRk19V3lZYvsaXGnwjyCZ7j0GN8gs6UoZxozM9o oQAzDL2YT3ZThaNgqpZj9HXzFIBSETc4PoDcYf6keR9Vk4+KAN4y9U17HrU5Qyl1lKo+ /ZmGqpzydVbtpsFqPnn7ybOstcTE2K2BXe4rIgoJBjtchdJWNtXz0RJOB9YSk3nYYgFp 6hsb4TpeVI0cwD7gv1Q/PnWlvVNACl3UTQloSHW6xp1LqQYVUzNOi4icrSWTW07MUPve 2Qf7mJAu549rPpiNRZWPaJ7K5gFpEPd3IQsRPgJMi+V+0gqTYS+2jwB8Oam9JKLv0fZM cYKA== X-Gm-Message-State: AOAM531pX9diZlX5O57Wv1nrY/KZYYMdFmqZXw23GPYnwqeEw1coNz3v AhKVdNkoT1gvb+IqYYBT4vY= X-Google-Smtp-Source: ABdhPJxmAyNsbQy0m+QE5io6xn9AJ6rjES9cqgJmvnBNZA6O4RSlZ/y36H2ngEez8kLTytziHCFHiQ== X-Received: by 2002:a9d:d05:: with SMTP id 5mr649132oti.312.1619722152711; Thu, 29 Apr 2021 11:49:12 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id r12sm140657otq.28.2021.04.29.11.49.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:12 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 03/10] RDMA/rxe: Enable MW object pool Date: Thu, 29 Apr 2021 13:48:48 -0500 Message-Id: <20210429184855.54939-4-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Currently the rxe driver has a rxe_mw struct object but nothing about memory windows is enabled. This patch turns on memory windows and some minor cleanup. Set device attribute in rxe.c so max_mw = MAX_MW. Change parameters in rxe_param.h so that MAX_MW is the same as MAX_MR. Reduce the number of MRs and MWs to 4K from 256K. Add device capability bits for 2a and 2b memory windows. Removed RXE_MR_TYPE_MW from the rxe_mr_type enum. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 1 + drivers/infiniband/sw/rxe/rxe_param.h | 19 ++++++++++++------- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 - 3 files changed, 13 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 95f0de0c8b49..8e0f9c489cab 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -54,6 +54,7 @@ static void rxe_init_device_param(struct rxe_dev *rxe) rxe->attr.max_cq = RXE_MAX_CQ; rxe->attr.max_cqe = (1 << RXE_MAX_LOG_CQE) - 1; rxe->attr.max_mr = RXE_MAX_MR; + rxe->attr.max_mw = RXE_MAX_MW; rxe->attr.max_pd = RXE_MAX_PD; rxe->attr.max_qp_rd_atom = RXE_MAX_QP_RD_ATOM; rxe->attr.max_res_rd_atom = RXE_MAX_RES_RD_ATOM; diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index 25ab50d9b7c2..742e6ec93686 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -37,7 +37,6 @@ static inline enum ib_mtu eth_mtu_int_to_enum(int mtu) enum rxe_device_param { RXE_MAX_MR_SIZE = -1ull, RXE_PAGE_SIZE_CAP = 0xfffff000, - RXE_MAX_QP = 0x10000, RXE_MAX_QP_WR = 0x4000, RXE_DEVICE_CAP_FLAGS = IB_DEVICE_BAD_PKEY_CNTR | IB_DEVICE_BAD_QKEY_CNTR @@ -49,7 +48,10 @@ enum rxe_device_param { | IB_DEVICE_RC_RNR_NAK_GEN | IB_DEVICE_SRQ_RESIZE | IB_DEVICE_MEM_MGT_EXTENSIONS - | IB_DEVICE_ALLOW_USER_UNREG, + | IB_DEVICE_ALLOW_USER_UNREG + | IB_DEVICE_MEM_WINDOW + | IB_DEVICE_MEM_WINDOW_TYPE_2A + | IB_DEVICE_MEM_WINDOW_TYPE_2B, RXE_MAX_SGE = 32, RXE_MAX_WQE_SIZE = sizeof(struct rxe_send_wqe) + sizeof(struct ib_sge) * RXE_MAX_SGE, @@ -58,7 +60,6 @@ enum rxe_device_param { RXE_MAX_SGE_RD = 32, RXE_MAX_CQ = 16384, RXE_MAX_LOG_CQE = 15, - RXE_MAX_MR = 256 * 1024, RXE_MAX_PD = 0x7ffc, RXE_MAX_QP_RD_ATOM = 128, RXE_MAX_RES_RD_ATOM = 0x3f000, @@ -67,7 +68,6 @@ enum rxe_device_param { RXE_MAX_MCAST_QP_ATTACH = 56, RXE_MAX_TOT_MCAST_QP_ATTACH = 0x70000, RXE_MAX_AH = 100, - RXE_MAX_SRQ = 960, RXE_MAX_SRQ_WR = 0x4000, RXE_MIN_SRQ_WR = 1, RXE_MAX_SRQ_SGE = 27, @@ -80,16 +80,21 @@ enum rxe_device_param { RXE_NUM_PORT = 1, + RXE_MAX_QP = 0x10000, RXE_MIN_QP_INDEX = 16, RXE_MAX_QP_INDEX = 0x00020000, + RXE_MAX_SRQ = 0x00001000, RXE_MIN_SRQ_INDEX = 0x00020001, RXE_MAX_SRQ_INDEX = 0x00040000, + RXE_MAX_MR = 0x00001000, + RXE_MAX_MW = 0x00001000, RXE_MIN_MR_INDEX = 0x00000001, - RXE_MAX_MR_INDEX = 0x00040000, - RXE_MIN_MW_INDEX = 0x00040001, - RXE_MAX_MW_INDEX = 0x00060000, + RXE_MAX_MR_INDEX = 0x00010000, + RXE_MIN_MW_INDEX = 0x00010001, + RXE_MAX_MW_INDEX = 0x00020000, + RXE_MAX_PKT_PER_ACK = 64, RXE_MAX_UNACKED_PSNS = 128, diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 11eba7a3ba8f..8d32e3f50813 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -273,7 +273,6 @@ enum rxe_mr_type { RXE_MR_TYPE_NONE, RXE_MR_TYPE_DMA, RXE_MR_TYPE_MR, - RXE_MR_TYPE_MW, }; #define RXE_BUF_PER_MAP (PAGE_SIZE / sizeof(struct rxe_phys_buf)) From patchwork Thu Apr 29 18:48:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E57DCC43461 for ; Thu, 29 Apr 2021 18:49:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A3555613B3 for ; Thu, 29 Apr 2021 18:49:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241152AbhD2SuC (ORCPT ); Thu, 29 Apr 2021 14:50:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41628 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241128AbhD2SuC (ORCPT ); Thu, 29 Apr 2021 14:50:02 -0400 Received: from mail-ot1-x329.google.com (mail-ot1-x329.google.com [IPv6:2607:f8b0:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 049EFC06138B for ; Thu, 29 Apr 2021 11:49:14 -0700 (PDT) Received: by mail-ot1-x329.google.com with SMTP id y14-20020a056830208eb02902a1c9fa4c64so25778034otq.9 for ; Thu, 29 Apr 2021 11:49:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=m+XESc/Y8IOiSfKZVJSC2pBceUokW24N79phNhgNjTc=; b=kWTmwZlGBxQQyjAjcc2x6h5OvRVY5cFS9+bd9J9pQb9sAF/zzUvMsX6oPihUeZ2cQD KP+NhPz+axqfkycz9UXFxX+k1S/IveSG3OrjkAmcHfoGlhTr3uVpwN36JJqm81XhtEmU Dy/D021NkFVEi8xFKp3T1fMtATG3gCRVd8ORdMx075iU2R/pbTsr5yrw5NdDP7NPKgk4 2Xux90O4xawKSONPANexdi1UyhYBoEZz9SKRG1GHQjyrwZjLXmnp97u/8qSpaMzmOh+j cSlZz0aWVcYsZIfYTO7qros7yjyfgl4XpW8X6oY5xzyiLhfqSlcwkCRLydE/Fb0ybaIE 3Fpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=m+XESc/Y8IOiSfKZVJSC2pBceUokW24N79phNhgNjTc=; b=Bu62oFqnbJTLMxXyqLIOXiGvpZxzKs2ZAMLO4/qX7+/lazgYD05yRidur2DKqDY5nN XnUZamAbhrBEHQLItSQ18c9zCh0p1MX6o0gx8fDB8cOpNjIX9DOw2EstjxMFPyDI+qGm HR+zOC5n/0M7/1hT9jQcjDpzrhTPBauS9IXyzJMUlaWr3VIFXfXbGtH3UJh9KcHf555v inIqE/AqyCyxnxxLhUIKIBpwfEz4Q+Bp2QYDn+WnoB438azRMcY0+QUOfMk4ofYeZimi m+q9HWA9CozBh2PWsC65rLYaTOXjW+tW6gMDJ9GPxdRQc8tPOaXcYoCoG55fT/UF/nHb uV3A== X-Gm-Message-State: AOAM533I2t15t648erMP5HNDYsmAEAEFmbLpPsgZ/iRuYrA6hwa+FEnV +fd4MZBn57DxQlK8w/7juMY= X-Google-Smtp-Source: ABdhPJxBlBa1uv0b+FZ5gqYJaQG4yWS+I3XjEBWDHDEiUAaER/z1LTUcv9bzLTNgbuBO8kg8hoWcHw== X-Received: by 2002:a9d:37b6:: with SMTP id x51mr606000otb.159.1619722153446; Thu, 29 Apr 2021 11:49:13 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id 3sm139249otw.58.2021.04.29.11.49.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:13 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 04/10] RDMA/rxe: Add ib_alloc_mw and ib_dealloc_mw verbs Date: Thu, 29 Apr 2021 13:48:49 -0500 Message-Id: <20210429184855.54939-5-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add ib_alloc_mw and ib_dealloc_mw verbs APIs. Added new file rxe_mw.c focused on MWs. Changed the 8 bit random key generator. Added a cleanup routine for MWs. Added verbs routines to ib_device_ops. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/Makefile | 1 + drivers/infiniband/sw/rxe/rxe_loc.h | 6 +++ drivers/infiniband/sw/rxe/rxe_mr.c | 20 +++++----- drivers/infiniband/sw/rxe/rxe_mw.c | 53 +++++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_pool.c | 1 + drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++ drivers/infiniband/sw/rxe/rxe_verbs.h | 2 + 7 files changed, 75 insertions(+), 11 deletions(-) create mode 100644 drivers/infiniband/sw/rxe/rxe_mw.c diff --git a/drivers/infiniband/sw/rxe/Makefile b/drivers/infiniband/sw/rxe/Makefile index 66af72dca759..1e24673e9318 100644 --- a/drivers/infiniband/sw/rxe/Makefile +++ b/drivers/infiniband/sw/rxe/Makefile @@ -15,6 +15,7 @@ rdma_rxe-y := \ rxe_qp.o \ rxe_cq.o \ rxe_mr.o \ + rxe_mw.o \ rxe_opcode.o \ rxe_mmap.o \ rxe_icrc.o \ diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index ef8061d2fbe0..edf575930a98 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -76,6 +76,7 @@ enum copy_direction { from_mr_obj, }; +u8 rxe_get_next_key(u32 last_key); void rxe_mr_init_dma(struct rxe_pd *pd, int access, struct rxe_mr *mr); int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, @@ -106,6 +107,11 @@ void rxe_mr_cleanup(struct rxe_pool_entry *arg); int advance_dma_data(struct rxe_dma_info *dma, unsigned int length); +/* rxe_mw.c */ +int rxe_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata); +int rxe_dealloc_mw(struct ib_mw *ibmw); +void rxe_mw_cleanup(struct rxe_pool_entry *arg); + /* rxe_net.c */ void rxe_loopback(struct sk_buff *skb); int rxe_send(struct rxe_pkt_info *pkt, struct sk_buff *skb); diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 9f63947bab12..7f2cfc1ce659 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -7,19 +7,17 @@ #include "rxe.h" #include "rxe_loc.h" -/* - * lfsr (linear feedback shift register) with period 255 +/* Return a random 8 bit key value that is + * different than the last_key. Set last_key to -1 + * if this is the first key for an MR or MW */ -static u8 rxe_get_key(void) +u8 rxe_get_next_key(u32 last_key) { - static u32 key = 1; - - key = key << 1; - - key |= (0 != (key & 0x100)) ^ (0 != (key & 0x10)) - ^ (0 != (key & 0x80)) ^ (0 != (key & 0x40)); + u8 key; - key &= 0xff; + do { + get_random_bytes(&key, 1); + } while (key == last_key); return key; } @@ -47,7 +45,7 @@ int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length) static void rxe_mr_init(int access, struct rxe_mr *mr) { - u32 lkey = mr->pelem.index << 8 | rxe_get_key(); + u32 lkey = mr->pelem.index << 8 | rxe_get_next_key(-1); u32 rkey = (access & IB_ACCESS_REMOTE) ? lkey : 0; mr->ibmr.lkey = lkey; diff --git a/drivers/infiniband/sw/rxe/rxe_mw.c b/drivers/infiniband/sw/rxe/rxe_mw.c new file mode 100644 index 000000000000..69128e298d44 --- /dev/null +++ b/drivers/infiniband/sw/rxe/rxe_mw.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * Copyright (c) 2020 Hewlett Packard Enterprise, Inc. All rights reserved. + */ + +#include "rxe.h" + +int rxe_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata) +{ + struct rxe_mw *mw = to_rmw(ibmw); + struct rxe_pd *pd = to_rpd(ibmw->pd); + struct rxe_dev *rxe = to_rdev(ibmw->device); + int ret; + + rxe_add_ref(pd); + + ret = rxe_add_to_pool(&rxe->mw_pool, mw); + if (ret) { + rxe_drop_ref(pd); + return ret; + } + + rxe_add_index(mw); + ibmw->rkey = (mw->pelem.index << 8) | rxe_get_next_key(-1); + mw->state = (mw->ibmw.type == IB_MW_TYPE_2) ? + RXE_MW_STATE_FREE : RXE_MW_STATE_VALID; + spin_lock_init(&mw->lock); + + return 0; +} + +int rxe_dealloc_mw(struct ib_mw *ibmw) +{ + struct rxe_mw *mw = to_rmw(ibmw); + struct rxe_pd *pd = to_rpd(ibmw->pd); + unsigned long flags; + + spin_lock_irqsave(&mw->lock, flags); + mw->state = RXE_MW_STATE_INVALID; + spin_unlock_irqrestore(&mw->lock, flags); + + rxe_drop_ref(mw); + rxe_drop_ref(pd); + + return 0; +} + +void rxe_mw_cleanup(struct rxe_pool_entry *elem) +{ + struct rxe_mw *mw = container_of(elem, typeof(*mw), pelem); + + rxe_drop_index(mw); +} diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index df0bec719341..0b8e7c6255a2 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -65,6 +65,7 @@ struct rxe_type_info rxe_type_info[RXE_NUM_TYPES] = { .name = "rxe-mw", .size = sizeof(struct rxe_mw), .elem_offset = offsetof(struct rxe_mw, pelem), + .cleanup = rxe_mw_cleanup, .flags = RXE_POOL_INDEX | RXE_POOL_NO_ALLOC, .max_index = RXE_MAX_MW_INDEX, .min_index = RXE_MIN_MW_INDEX, diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index aeb5e232c195..fff81bf78a86 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -1060,6 +1060,7 @@ static const struct ib_device_ops rxe_dev_ops = { .alloc_hw_stats = rxe_ib_alloc_hw_stats, .alloc_mr = rxe_alloc_mr, + .alloc_mw = rxe_alloc_mw, .alloc_pd = rxe_alloc_pd, .alloc_ucontext = rxe_alloc_ucontext, .attach_mcast = rxe_attach_mcast, @@ -1069,6 +1070,7 @@ static const struct ib_device_ops rxe_dev_ops = { .create_srq = rxe_create_srq, .create_user_ah = rxe_create_ah, .dealloc_driver = rxe_dealloc, + .dealloc_mw = rxe_dealloc_mw, .dealloc_pd = rxe_dealloc_pd, .dealloc_ucontext = rxe_dealloc_ucontext, .dereg_mr = rxe_dereg_mr, @@ -1106,6 +1108,7 @@ static const struct ib_device_ops rxe_dev_ops = { INIT_RDMA_OBJ_SIZE(ib_ah, rxe_ah, ibah), INIT_RDMA_OBJ_SIZE(ib_cq, rxe_cq, ibcq), + INIT_RDMA_OBJ_SIZE(ib_mw, rxe_mw, ibmw), INIT_RDMA_OBJ_SIZE(ib_pd, rxe_pd, ibpd), INIT_RDMA_OBJ_SIZE(ib_srq, rxe_srq, ibsrq), INIT_RDMA_OBJ_SIZE(ib_ucontext, rxe_ucontext, ibuc), diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 8d32e3f50813..c8597ae8c833 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -323,6 +323,8 @@ enum rxe_mw_state { struct rxe_mw { struct ib_mw ibmw; struct rxe_pool_entry pelem; + spinlock_t lock; + enum rxe_mw_state state; }; struct rxe_mc_grp { From patchwork Thu Apr 29 18:48:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41A1FC433B4 for ; Thu, 29 Apr 2021 18:49:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 05D42613B3 for ; Thu, 29 Apr 2021 18:49:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233565AbhD2SuB (ORCPT ); Thu, 29 Apr 2021 14:50:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41630 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241054AbhD2SuB (ORCPT ); Thu, 29 Apr 2021 14:50:01 -0400 Received: from mail-oi1-x22c.google.com (mail-oi1-x22c.google.com [IPv6:2607:f8b0:4864:20::22c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF545C06138C for ; Thu, 29 Apr 2021 11:49:14 -0700 (PDT) Received: by mail-oi1-x22c.google.com with SMTP id u80so33887632oia.0 for ; Thu, 29 Apr 2021 11:49:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K3HvFw+RtgfAZ5giUI6FTWhZ270iUlqbGN0Kkgaz328=; b=O8H+nlwMbD7C5knyXSOJ9vPZ3fm4Ga84kH7bgH8AbkMCVIpEv+PIKyaTUWSWG72Sff 2zKhsb9kpRyiJOyn9Eh5/ZKLNN3jt1eQu6XCFHTbLdGhaE6rFaiJlV9ri7rJ2dEJ6hI8 nfI4Ykguwmlt6N6uZ7HfCBaHnVfNIThi+xgswhhQiShPqpSGbyXROO22pVcgkvBHR9/6 RutKmreq1FlWAH6d4NQPEz1NCeX/xmyEFelIHdBu6arv1YAEhhdd4BWyCsiprM+WU6Ka aOOzbwcUs2Zt9j+9vJ/lpiKgGi+GbQL4OUM8JaB6Y+eUBeJB+tZ7ehczLm4yjLlGC1KF VZfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K3HvFw+RtgfAZ5giUI6FTWhZ270iUlqbGN0Kkgaz328=; b=W5EQR8/NvTl9LCDk2tM1mk5SKSaZgxsnQ3Dsz8me3fBE7mcMgxkV6AehaexPdzf3tX D8IL5PD0MazlUp1c2d25j7Zk9ZrmxyZ6mUCGN6dTbe1ojfjIbrW3ie7A94wYlxry7hAw 14tSV3Lw1h++cz/d8Q7jOIpddq+uQNML+18gghkhuQj+t5IYC5LNZ3WeAbhyUKMCRHez S76P6rX3UzvQyhPMibRyuNqNDLBvlV82/T67f7sXQaqmzLbY9aRYVyEzWoaQ2JZen2vn BGEvGgimobVHVnVeT7n/3D1QOwqvBVqsHjlz27c7mGRy1gco0syLFgNo5c0iW3CC6gQK H5cQ== X-Gm-Message-State: AOAM5327wqorbvQISuT7LmABkf/XxAE9o7xB2pz7rcLXnRCDZccQKaE7 L7jdNAoLKIgLRkR3VT/z5sE= X-Google-Smtp-Source: ABdhPJx8HCt9y8vATF5K3xH3L1s7sv610Gg6bCWPrSMTLePjCqwPuzomQXbh4OUB1KL2Q8guInvbjA== X-Received: by 2002:a05:6808:1cc:: with SMTP id x12mr8140919oic.114.1619722154227; Thu, 29 Apr 2021 11:49:14 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id w26sm143182otm.27.2021.04.29.11.49.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:13 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 05/10] RDMA/rxe: Replace WR_REG_MASK by WR_LOCAL_OP_MASK Date: Thu, 29 Apr 2021 13:48:50 -0500 Message-Id: <20210429184855.54939-6-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Rxe has two mask bits WR_LOCAL_MASK and WR_REG_MASK with WR_REG_MASK used to indicate any local operation and WR_LOCAL_MASK unused. This patch replaces both of these with one mask bit WR_LOCAL_OP_MASK which is clearer. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_opcode.c | 4 ++-- drivers/infiniband/sw/rxe/rxe_opcode.h | 3 +-- drivers/infiniband/sw/rxe/rxe_req.c | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 2 +- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.c b/drivers/infiniband/sw/rxe/rxe_opcode.c index 0cb4b01fd910..1e4b67b048f3 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.c +++ b/drivers/infiniband/sw/rxe/rxe_opcode.c @@ -87,13 +87,13 @@ struct rxe_wr_opcode_info rxe_wr_opcode_info[] = { [IB_WR_LOCAL_INV] = { .name = "IB_WR_LOCAL_INV", .mask = { - [IB_QPT_RC] = WR_REG_MASK, + [IB_QPT_RC] = WR_LOCAL_OP_MASK, }, }, [IB_WR_REG_MR] = { .name = "IB_WR_REG_MR", .mask = { - [IB_QPT_RC] = WR_REG_MASK, + [IB_QPT_RC] = WR_LOCAL_OP_MASK, }, }, }; diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.h b/drivers/infiniband/sw/rxe/rxe_opcode.h index 1041ac9a9233..e02f039b8c44 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.h +++ b/drivers/infiniband/sw/rxe/rxe_opcode.h @@ -19,8 +19,7 @@ enum rxe_wr_mask { WR_SEND_MASK = BIT(2), WR_READ_MASK = BIT(3), WR_WRITE_MASK = BIT(4), - WR_LOCAL_MASK = BIT(5), - WR_REG_MASK = BIT(6), + WR_LOCAL_OP_MASK = BIT(5), WR_READ_OR_WRITE_MASK = WR_READ_MASK | WR_WRITE_MASK, WR_READ_WRITE_OR_SEND_MASK = WR_READ_OR_WRITE_MASK | WR_SEND_MASK, diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 3664cdae7e1f..0d4dcd514c55 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -593,7 +593,7 @@ int rxe_requester(void *arg) if (unlikely(!wqe)) goto exit; - if (wqe->mask & WR_REG_MASK) { + if (wqe->mask & WR_LOCAL_OP_MASK) { if (wqe->wr.opcode == IB_WR_LOCAL_INV) { struct rxe_dev *rxe = to_rdev(qp->ibqp.device); struct rxe_mr *rmr; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index fff81bf78a86..d22f011a20f3 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -577,7 +577,7 @@ static void init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, init_send_wr(qp, &wqe->wr, ibwr); /* local operation */ - if (unlikely(mask & WR_REG_MASK)) { + if (unlikely(mask & WR_LOCAL_OP_MASK)) { wqe->mask = mask; wqe->state = wqe_state_posted; return; From patchwork Thu Apr 29 18:48:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54112C433ED for ; Thu, 29 Apr 2021 18:49:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1F647613B3 for ; Thu, 29 Apr 2021 18:49:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241128AbhD2SuC (ORCPT ); Thu, 29 Apr 2021 14:50:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241146AbhD2SuC (ORCPT ); Thu, 29 Apr 2021 14:50:02 -0400 Received: from mail-ot1-x329.google.com (mail-ot1-x329.google.com [IPv6:2607:f8b0:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7007CC06138C for ; Thu, 29 Apr 2021 11:49:15 -0700 (PDT) Received: by mail-ot1-x329.google.com with SMTP id 35-20020a9d05260000b029029c82502d7bso30852295otw.2 for ; Thu, 29 Apr 2021 11:49:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hsqY5ElojN7QyWG9J/VNC2zRxRSymbF4BnhNplKOyUE=; b=apyfECOWl3zM+aFHksFf6KThLy79T9a3NOeVmEXnRCc9KG2uxL0WYeuCX0a7W+WLnF 7Iz6CMpjX9w3sPT5fAiexDderl8pUIa82x9Z+LHLYg309B07PHdp8SKFCiVDEnamtrrb CXKUN3iBkbN7RDItYPs2bdQyN60uMulT6gdANq2EZ/4Eb2JYEqrpufZVJ+naJVoy1I+s pUaFC2ChMreZVGJeujLgooRKzlV0yZWbFL+GnoWnjjwkEvm061fwa4j8CpurdRilel6M d0M7wzdruGPB6rPX0HtSr3Ql6399iF46d6cldYavz/JqpqWzDDsQH+Seyuegv4JAN6LF UH7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hsqY5ElojN7QyWG9J/VNC2zRxRSymbF4BnhNplKOyUE=; b=cQ0K8TtnWUdg/I3BIgHpDslWijajUHlgWE7cLVa3gsnHZpnb0ihAbpUH/l8GKR0UHA 5eC2YVbVBTeHjmuuczakyGpgUcWkjWI//eFaK1Mv0PWsMEwGvYxhalscOZyMLQCy1djf t4VU33NdQOcyEarkXTcvFnBx9+bwhgJsLHws4jtDyxVuOeJBjI75kBvXGHYEtZAiLa9z hRVF4GO74HsgRMzDhYRQediz4cvR+6wR5mapbPq+0eb+1Seyv+1S8SafnnBYlcz6pJ4v IZHTfyUrj1ifOXDVgYkodhmoyGzUzRA4h/3AySB6nd2gKpxEl5VpSKgfd2z/XqbXNL8p ADrA== X-Gm-Message-State: AOAM530rZMPXmVc1Z0Vr4jbweL10gb/MXqHd9DoBNUm8PwF2fbTTh2bd XEN2s/NXutlGq0oBJfOUVqQ= X-Google-Smtp-Source: ABdhPJyn+m73YB8e8SVeAMBpEu/Ck+xXqkzobH8UC8USTMlnroWukLbI5s5EE8C8cXmq17UhA1m01Q== X-Received: by 2002:a9d:4e05:: with SMTP id p5mr645989otf.264.1619722154913; Thu, 29 Apr 2021 11:49:14 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id r189sm196060oif.8.2021.04.29.11.49.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:14 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 06/10] RDMA/rxe: Move local ops to subroutine Date: Thu, 29 Apr 2021 13:48:51 -0500 Message-Id: <20210429184855.54939-7-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Simplify rxe_requester() by moving the local operations to a subroutine. Add an error return for illegal send WR opcode. Moved next_index ahead of rxe_run_task which fixed a small bug where work completions were delayed until after the next wqe which was not the intended behavior. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_req.c | 89 +++++++++++++++++------------ 1 file changed, 54 insertions(+), 35 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 0d4dcd514c55..9163aac41848 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -555,6 +555,56 @@ static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe, jiffies + qp->qp_timeout_jiffies); } +static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe) +{ + u8 opcode = wqe->wr.opcode; + struct rxe_dev *rxe; + struct rxe_mr *mr; + u32 rkey; + + switch (opcode) { + case IB_WR_LOCAL_INV: + rxe = to_rdev(qp->ibqp.device); + rkey = wqe->wr.ex.invalidate_rkey; + mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8); + if (!mr) { + pr_err("No MR for rkey %#x\n", rkey); + wqe->state = wqe_state_error; + wqe->status = IB_WC_LOC_QP_OP_ERR; + return -EINVAL; + } + mr->state = RXE_MR_STATE_FREE; + rxe_drop_ref(mr); + break; + case IB_WR_REG_MR: + mr = to_rmr(wqe->wr.wr.reg.mr); + + rxe_add_ref(mr); + mr->state = RXE_MR_STATE_VALID; + mr->access = wqe->wr.wr.reg.access; + mr->ibmr.lkey = wqe->wr.wr.reg.key; + mr->ibmr.rkey = wqe->wr.wr.reg.key; + mr->iova = wqe->wr.wr.reg.mr->iova; + rxe_drop_ref(mr); + break; + default: + pr_err("Unexpected send wqe opcode %d\n", opcode); + wqe->state = wqe_state_error; + wqe->status = IB_WC_LOC_QP_OP_ERR; + return -EINVAL; + } + + wqe->state = wqe_state_done; + wqe->status = IB_WC_SUCCESS; + qp->req.wqe_index = next_index(qp->sq.queue, qp->req.wqe_index); + + if ((wqe->wr.send_flags & IB_SEND_SIGNALED) || + qp->sq_sig_type == IB_SIGNAL_ALL_WR) + rxe_run_task(&qp->comp.task, 1); + + return 0; +} + int rxe_requester(void *arg) { struct rxe_qp *qp = (struct rxe_qp *)arg; @@ -594,42 +644,11 @@ int rxe_requester(void *arg) goto exit; if (wqe->mask & WR_LOCAL_OP_MASK) { - if (wqe->wr.opcode == IB_WR_LOCAL_INV) { - struct rxe_dev *rxe = to_rdev(qp->ibqp.device); - struct rxe_mr *rmr; - - rmr = rxe_pool_get_index(&rxe->mr_pool, - wqe->wr.ex.invalidate_rkey >> 8); - if (!rmr) { - pr_err("No mr for key %#x\n", - wqe->wr.ex.invalidate_rkey); - wqe->state = wqe_state_error; - wqe->status = IB_WC_MW_BIND_ERR; - goto exit; - } - rmr->state = RXE_MR_STATE_FREE; - rxe_drop_ref(rmr); - wqe->state = wqe_state_done; - wqe->status = IB_WC_SUCCESS; - } else if (wqe->wr.opcode == IB_WR_REG_MR) { - struct rxe_mr *rmr = to_rmr(wqe->wr.wr.reg.mr); - - rmr->state = RXE_MR_STATE_VALID; - rmr->access = wqe->wr.wr.reg.access; - rmr->ibmr.lkey = wqe->wr.wr.reg.key; - rmr->ibmr.rkey = wqe->wr.wr.reg.key; - rmr->iova = wqe->wr.wr.reg.mr->iova; - wqe->state = wqe_state_done; - wqe->status = IB_WC_SUCCESS; - } else { + ret = rxe_do_local_ops(qp, wqe); + if (unlikely(ret)) goto exit; - } - if ((wqe->wr.send_flags & IB_SEND_SIGNALED) || - qp->sq_sig_type == IB_SIGNAL_ALL_WR) - rxe_run_task(&qp->comp.task, 1); - qp->req.wqe_index = next_index(qp->sq.queue, - qp->req.wqe_index); - goto next_wqe; + else + goto next_wqe; } if (unlikely(qp_type(qp) == IB_QPT_RC && From patchwork Thu Apr 29 18:48:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFFB7C43460 for ; Thu, 29 Apr 2021 18:49:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8378E613B3 for ; Thu, 29 Apr 2021 18:49:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241139AbhD2SuD (ORCPT ); Thu, 29 Apr 2021 14:50:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241054AbhD2SuD (ORCPT ); Thu, 29 Apr 2021 14:50:03 -0400 Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com [IPv6:2607:f8b0:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47538C06138D for ; Thu, 29 Apr 2021 11:49:16 -0700 (PDT) Received: by mail-ot1-x330.google.com with SMTP id z20-20020a0568301294b02902a52ecbaf18so9925271otp.8 for ; Thu, 29 Apr 2021 11:49:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bBEyPDwSf6NHjJgGiGzNH4t4+7yQEN1Elxt8HYlMpQQ=; b=paWil8G4NdbkON4JMaN3s3EYfh2TfCDu2+nX3w+ZWO8aGcnlt+ohs6omaUyWx1gGGY zso9iLngEzSPsWArVME3v0QQzgQvgCTefOp/NJnF3eD+qqB3oJbAUkCHwvGjMjCHCorL 9Z50J22CqJit7xaQfAgh37UzBbLFQflkgdxbTRBDgazxJyAC/BjlNGWYh/MtErSasx1F iy5kNH7Y+4Hb2Y0/l18dTc/OibkN5JtY8GWQEgbp7e3auC/N+aKnJ5MGm1D0urdG++1D domijHZejgUXmLisJxo7drufVWN9jTxXwS6pgnLrYXx7VAa4LBrPuVvytjl29ZGCQaWL a7Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bBEyPDwSf6NHjJgGiGzNH4t4+7yQEN1Elxt8HYlMpQQ=; b=fJR67CS/E3K7lotIhnqkjzL3h+ifKs4QSdaDmX8KiHnCY1JV+WCr6JfxHWm6M+uBN9 ul6hjN06Ty5oTK824XezhqpyER71s8pLLHJXJRS0aWIWSYFJm5X/SSZoEpRs7OGAuoA1 Txc+csN0iNwNAe1kf7KbYrXpUEwsYqjHzcYxt0QKA6VgBwyIri1UdTYRJs2nyG6uuIIU q3q0NFcQOid9EzsFZzRoNz3zt2hqpZAjsbhAli4WvdpRBp2C482C/OyM0uAFHWIo+xxa rvsBVodhHIgvZiCeNz3NuL8AHiLfGamGRnZWwwKz27fGwI2ECJb+hHpjF4N9WlTTCAgK KNIg== X-Gm-Message-State: AOAM530reOal0D26bLvmhHMO+LNqRnPh87COqBCRu8s4bcl9R2WcTlx0 2jFhGs39+GVm3gxkpT91FzS7K3Pm7zlk8w== X-Google-Smtp-Source: ABdhPJw398Kgi11+Rc8W1fJNlnpn9iYfR6WaoNLGwc5YzqnEvayahQ1al3ByHJJOVZp1cpEbesietw== X-Received: by 2002:a9d:30b:: with SMTP id 11mr608569otv.298.1619722155660; Thu, 29 Apr 2021 11:49:15 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id v15sm147210ots.76.2021.04.29.11.49.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:15 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 07/10] RDMA/rxe: Add support for bind MW work requests Date: Thu, 29 Apr 2021 13:48:52 -0500 Message-Id: <20210429184855.54939-8-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add support for bind MW work requests from user space. Since rdma/core does not support bind mw in ib_send_wr there is no way to support bind mw in kernel space. Added bind_mw local operation in rxe_req.c Added bind_mw WR operation in rxe_opcode.c Added bind_mw WC in rxe_comp.c Added additional fields to rxe_mw in rxe_verbs.h Added rxe_do_dealloc_mw() subroutine to cleanup an mw when rxe_dealloc_mw is called. Added code to implement bind_mw operation in rxe_mw.c Signed-off-by: Bob Pearson --- v6: Added rxe_ to subroutine names in lines that were changed. v3: do_bind_mw() in rxe_mw.c is changed to be a void instead of returning an int. v2: Dropped kernel support for bind_mw in rxe_mw.c Replaced umw with mw in struct rxe_send_wr --- drivers/infiniband/sw/rxe/rxe_comp.c | 1 + drivers/infiniband/sw/rxe/rxe_loc.h | 1 + drivers/infiniband/sw/rxe/rxe_mw.c | 202 ++++++++++++++++++++++++- drivers/infiniband/sw/rxe/rxe_opcode.c | 7 + drivers/infiniband/sw/rxe/rxe_req.c | 9 ++ drivers/infiniband/sw/rxe/rxe_verbs.h | 15 +- 6 files changed, 230 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 2af26737d32d..bc5488af5f55 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -103,6 +103,7 @@ static enum ib_wc_opcode wr_to_wc_opcode(enum ib_wr_opcode opcode) case IB_WR_RDMA_READ_WITH_INV: return IB_WC_RDMA_READ; case IB_WR_LOCAL_INV: return IB_WC_LOCAL_INV; case IB_WR_REG_MR: return IB_WC_REG_MR; + case IB_WR_BIND_MW: return IB_WC_BIND_MW; default: return 0xff; diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index edf575930a98..e6f574973298 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -110,6 +110,7 @@ int advance_dma_data(struct rxe_dma_info *dma, unsigned int length); /* rxe_mw.c */ int rxe_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata); int rxe_dealloc_mw(struct ib_mw *ibmw); +int rxe_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe); void rxe_mw_cleanup(struct rxe_pool_entry *arg); /* rxe_net.c */ diff --git a/drivers/infiniband/sw/rxe/rxe_mw.c b/drivers/infiniband/sw/rxe/rxe_mw.c index 69128e298d44..65215dde9974 100644 --- a/drivers/infiniband/sw/rxe/rxe_mw.c +++ b/drivers/infiniband/sw/rxe/rxe_mw.c @@ -29,6 +29,29 @@ int rxe_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata) return 0; } +static void rxe_do_dealloc_mw(struct rxe_mw *mw) +{ + if (mw->mr) { + struct rxe_mr *mr = mw->mr; + + mw->mr = NULL; + atomic_dec(&mr->num_mw); + rxe_drop_ref(mr); + } + + if (mw->qp) { + struct rxe_qp *qp = mw->qp; + + mw->qp = NULL; + rxe_drop_ref(qp); + } + + mw->access = 0; + mw->addr = 0; + mw->length = 0; + mw->state = RXE_MW_STATE_INVALID; +} + int rxe_dealloc_mw(struct ib_mw *ibmw) { struct rxe_mw *mw = to_rmw(ibmw); @@ -36,7 +59,7 @@ int rxe_dealloc_mw(struct ib_mw *ibmw) unsigned long flags; spin_lock_irqsave(&mw->lock, flags); - mw->state = RXE_MW_STATE_INVALID; + rxe_do_dealloc_mw(mw); spin_unlock_irqrestore(&mw->lock, flags); rxe_drop_ref(mw); @@ -45,6 +68,183 @@ int rxe_dealloc_mw(struct ib_mw *ibmw) return 0; } +static int rxe_check_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe, + struct rxe_mw *mw, struct rxe_mr *mr) +{ + if (mw->ibmw.type == IB_MW_TYPE_1) { + if (unlikely(mw->state != RXE_MW_STATE_VALID)) { + pr_err_once( + "attempt to bind a type 1 MW not in the valid state\n"); + return -EINVAL; + } + + /* o10-36.2.2 */ + if (unlikely((mw->access & IB_ZERO_BASED))) { + pr_err_once("attempt to bind a zero based type 1 MW\n"); + return -EINVAL; + } + } + + if (mw->ibmw.type == IB_MW_TYPE_2) { + /* o10-37.2.30 */ + if (unlikely(mw->state != RXE_MW_STATE_FREE)) { + pr_err_once( + "attempt to bind a type 2 MW not in the free state\n"); + return -EINVAL; + } + + /* C10-72 */ + if (unlikely(qp->pd != to_rpd(mw->ibmw.pd))) { + pr_err_once( + "attempt to bind type 2 MW with qp with different PD\n"); + return -EINVAL; + } + + /* o10-37.2.40 */ + if (unlikely(!mr || wqe->wr.wr.mw.length == 0)) { + pr_err_once( + "attempt to invalidate type 2 MW by binding with NULL or zero length MR\n"); + return -EINVAL; + } + } + + if (unlikely((wqe->wr.wr.mw.rkey & 0xff) == (mw->ibmw.rkey & 0xff))) { + pr_err_once("attempt to bind MW with same key\n"); + return -EINVAL; + } + + /* remaining checks only apply to a nonzero MR */ + if (!mr) + return 0; + + if (unlikely(mr->access & IB_ZERO_BASED)) { + pr_err_once("attempt to bind MW to zero based MR\n"); + return -EINVAL; + } + + /* C10-73 */ + if (unlikely(!(mr->access & IB_ACCESS_MW_BIND))) { + pr_err_once( + "attempt to bind an MW to an MR without bind access\n"); + return -EINVAL; + } + + /* C10-74 */ + if (unlikely((mw->access & + (IB_ACCESS_REMOTE_WRITE | IB_ACCESS_REMOTE_ATOMIC)) && + !(mr->access & IB_ACCESS_LOCAL_WRITE))) { + pr_err_once( + "attempt to bind an writeable MW to an MR without local write access\n"); + return -EINVAL; + } + + /* C10-75 */ + if (mw->access & IB_ZERO_BASED) { + if (unlikely(wqe->wr.wr.mw.length > mr->length)) { + pr_err_once( + "attempt to bind a ZB MW outside of the MR\n"); + return -EINVAL; + } + } else { + if (unlikely((wqe->wr.wr.mw.addr < mr->iova) || + ((wqe->wr.wr.mw.addr + wqe->wr.wr.mw.length) > + (mr->iova + mr->length)))) { + pr_err_once( + "attempt to bind a VA MW outside of the MR\n"); + return -EINVAL; + } + } + + return 0; +} + +static void rxe_do_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe, + struct rxe_mw *mw, struct rxe_mr *mr) +{ + u32 rkey; + u32 new_rkey; + + rkey = mw->ibmw.rkey; + new_rkey = (rkey & 0xffffff00) | (wqe->wr.wr.mw.rkey & 0x000000ff); + + mw->ibmw.rkey = new_rkey; + mw->access = wqe->wr.wr.mw.access; + mw->state = RXE_MW_STATE_VALID; + mw->addr = wqe->wr.wr.mw.addr; + mw->length = wqe->wr.wr.mw.length; + + if (mw->mr) { + rxe_drop_ref(mw->mr); + atomic_dec(&mw->mr->num_mw); + mw->mr = NULL; + } + + if (mw->length) { + mw->mr = mr; + atomic_inc(&mr->num_mw); + rxe_add_ref(mr); + } + + if (mw->ibmw.type == IB_MW_TYPE_2) { + rxe_add_ref(qp); + mw->qp = qp; + } +} + +int rxe_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe) +{ + int ret; + struct rxe_mw *mw; + struct rxe_mr *mr; + struct rxe_dev *rxe = to_rdev(qp->ibqp.device); + unsigned long flags; + + mw = rxe_pool_get_index(&rxe->mw_pool, + wqe->wr.wr.mw.mw_rkey >> 8); + if (unlikely(!mw)) { + ret = -EINVAL; + goto err; + } + + if (unlikely(mw->ibmw.rkey != wqe->wr.wr.mw.mw_rkey)) { + ret = -EINVAL; + goto err_drop_mw; + } + + if (likely(wqe->wr.wr.mw.length)) { + mr = rxe_pool_get_index(&rxe->mr_pool, + wqe->wr.wr.mw.mr_lkey >> 8); + if (unlikely(!mr)) { + ret = -EINVAL; + goto err_drop_mw; + } + + if (unlikely(mr->ibmr.lkey != wqe->wr.wr.mw.mr_lkey)) { + ret = -EINVAL; + goto err_drop_mr; + } + } else { + mr = NULL; + } + + spin_lock_irqsave(&mw->lock, flags); + + ret = rxe_check_bind_mw(qp, wqe, mw, mr); + if (ret) + goto err_unlock; + + rxe_do_bind_mw(qp, wqe, mw, mr); +err_unlock: + spin_unlock_irqrestore(&mw->lock, flags); +err_drop_mr: + if (mr) + rxe_drop_ref(mr); +err_drop_mw: + rxe_drop_ref(mw); +err: + return ret; +} + void rxe_mw_cleanup(struct rxe_pool_entry *elem) { struct rxe_mw *mw = container_of(elem, typeof(*mw), pelem); diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.c b/drivers/infiniband/sw/rxe/rxe_opcode.c index 1e4b67b048f3..3ef5a10a6efd 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.c +++ b/drivers/infiniband/sw/rxe/rxe_opcode.c @@ -96,6 +96,13 @@ struct rxe_wr_opcode_info rxe_wr_opcode_info[] = { [IB_QPT_RC] = WR_LOCAL_OP_MASK, }, }, + [IB_WR_BIND_MW] = { + .name = "IB_WR_BIND_MW", + .mask = { + [IB_QPT_RC] = WR_LOCAL_OP_MASK, + [IB_QPT_UC] = WR_LOCAL_OP_MASK, + }, + }, }; struct rxe_opcode_info rxe_opcode[RXE_NUM_OPCODE] = { diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 9163aac41848..29fa41b4d2ac 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -561,6 +561,7 @@ static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe) struct rxe_dev *rxe; struct rxe_mr *mr; u32 rkey; + int ret; switch (opcode) { case IB_WR_LOCAL_INV: @@ -587,6 +588,14 @@ static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe) mr->iova = wqe->wr.wr.reg.mr->iova; rxe_drop_ref(mr); break; + case IB_WR_BIND_MW: + ret = rxe_bind_mw(qp, wqe); + if (ret) { + wqe->state = wqe_state_error; + wqe->status = IB_WC_MW_BIND_ERR; + return -EINVAL; + } + break; default: pr_err("Unexpected send wqe opcode %d\n", opcode); wqe->state = wqe_state_error; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index c8597ae8c833..7da47b8c707b 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -312,6 +312,8 @@ struct rxe_mr { u32 num_map; struct rxe_map **map; + + atomic_t num_mw; }; enum rxe_mw_state { @@ -321,10 +323,15 @@ enum rxe_mw_state { }; struct rxe_mw { - struct ib_mw ibmw; - struct rxe_pool_entry pelem; - spinlock_t lock; - enum rxe_mw_state state; + struct ib_mw ibmw; + struct rxe_pool_entry pelem; + spinlock_t lock; + enum rxe_mw_state state; + struct rxe_qp *qp; /* Type 2 only */ + struct rxe_mr *mr; + int access; + u64 addr; + u64 length; }; struct rxe_mc_grp { From patchwork Thu Apr 29 18:48:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2945DC433B4 for ; Thu, 29 Apr 2021 18:49:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ECDA861408 for ; Thu, 29 Apr 2021 18:49:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241054AbhD2SuE (ORCPT ); Thu, 29 Apr 2021 14:50:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241166AbhD2SuE (ORCPT ); Thu, 29 Apr 2021 14:50:04 -0400 Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 112F0C06138B for ; Thu, 29 Apr 2021 11:49:17 -0700 (PDT) Received: by mail-ot1-x32f.google.com with SMTP id f75-20020a9d03d10000b0290280def9ab76so57166132otf.12 for ; Thu, 29 Apr 2021 11:49:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eWFfhhGwJCobLzYiya7M0plIMiIfHsOQP3K1CJZ2k5M=; b=ktet2Qm8zLvb6WOBZS1ZGv2500vdKIVA80SwWu8L7iPL0JXm4VdcRZ+IsK/it3U1dO Fet8OHiliDgxGCS+R9SD1kaZLegDEI1nXlhpyfrMPgT3khtUflErhrMsf5AZXH6FvsHY YuDRCffJS2QvE9+pCnq7lJofUwj68fKWG3ocwNGbuuxMtu+3E/L9qNf0bByg5irGTZwS 1Y4aUYfXOK/Dn6sjn+GGO8/A7Di4XiXJiZK/wJjWhoNKvMAvXL/uAW5o8fi5w1q/F4aD TfsQyJQss1nG2o5siSKMm3pxSMtNtaR7xnhPoA+5O08Cz4dPFc9ujdK4KnckLbQyjKlU u7lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eWFfhhGwJCobLzYiya7M0plIMiIfHsOQP3K1CJZ2k5M=; b=oAhfXQph5u7EQw1KEbKjXR3ksEsL/4CiGE1GWctkipZuhWkIXlkXstZueC1FsTtQ+7 ib5v1Ae5xK2IM38UaFmnu4JABYzfdqDgg7eb405g2fho3cXDbkPVyeQPBE97RpOTKiXa EW2fGye7RatbpKhjfdBDJA/c59UN2SepzESbJUY3DjMPE1dm3S4jvb6xBfad3ixHmPkC MEiAXzsvf+ci/mnKvFoblvPbQ5xIWdjzM+Ko86/YH4MMQ+8Gy87YWyTtVLD013qJzM2r uACnw3REJpQTOxmeKJRuk516JcufvQfOGYt3xU7o5b4j04o4Je928UVctic/Qm8wz/P+ HuyQ== X-Gm-Message-State: AOAM532AyqO/ICZSGuuImBk/atf3q1veIUNIhlnuAiRZwoOiWVuhEzTl l5MTURMJPWkJLH+tk1LKmWg= X-Google-Smtp-Source: ABdhPJy/CN/xY4WRsUPSeDsb6GeMmmL1iB5coiiCo5JdZ9Kcs2DyT8e+MCfm/XogDRuMngiCimN1Zg== X-Received: by 2002:a9d:6a0c:: with SMTP id g12mr640562otn.48.1619722156416; Thu, 29 Apr 2021 11:49:16 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id x187sm186785oig.27.2021.04.29.11.49.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:16 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 08/10] RDMA/rxe: Implement invalidate MW operations Date: Thu, 29 Apr 2021 13:48:53 -0500 Message-Id: <20210429184855.54939-9-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Implement invalidate MW and cleaned up invalidate MR operations. Added code to perform remote invalidate for send with invalidate. Added code to perform local invalidation. Deleted some blank lines in rxe_loc.h. Signed-off-by: Bob Pearson --- v6: Added rxe_ to subroutine names in lines that changed. v3: Replaced enums in lower case with upper case and moved to rxe_verbs.h which is where enums live. --- drivers/infiniband/sw/rxe/rxe_comp.c | 4 +- drivers/infiniband/sw/rxe/rxe_loc.h | 29 ++-------- drivers/infiniband/sw/rxe/rxe_mr.c | 81 ++++++++++++++++++--------- drivers/infiniband/sw/rxe/rxe_mw.c | 67 ++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_req.c | 24 ++++---- drivers/infiniband/sw/rxe/rxe_resp.c | 60 ++++++++++++-------- drivers/infiniband/sw/rxe/rxe_verbs.h | 33 ++++++++--- 7 files changed, 204 insertions(+), 94 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index bc5488af5f55..207aa7ef52c4 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -346,7 +346,7 @@ static inline enum comp_state do_read(struct rxe_qp *qp, ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE, &wqe->dma, payload_addr(pkt), - payload_size(pkt), to_mr_obj, NULL); + payload_size(pkt), RXE_TO_MR_OBJ, NULL); if (ret) return COMPST_ERROR; @@ -366,7 +366,7 @@ static inline enum comp_state do_atomic(struct rxe_qp *qp, ret = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE, &wqe->dma, &atomic_orig, - sizeof(u64), to_mr_obj, NULL); + sizeof(u64), RXE_TO_MR_OBJ, NULL); if (ret) return COMPST_ERROR; else diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index e6f574973298..bc0e484f8cde 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -71,46 +71,29 @@ struct rxe_mmap_info *rxe_create_mmap_info(struct rxe_dev *dev, u32 size, int rxe_mmap(struct ib_ucontext *context, struct vm_area_struct *vma); /* rxe_mr.c */ -enum copy_direction { - to_mr_obj, - from_mr_obj, -}; - u8 rxe_get_next_key(u32 last_key); void rxe_mr_init_dma(struct rxe_pd *pd, int access, struct rxe_mr *mr); - int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, int access, struct ib_udata *udata, struct rxe_mr *mr); - int rxe_mr_init_fast(struct rxe_pd *pd, int max_pages, struct rxe_mr *mr); - int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, int length, - enum copy_direction dir, u32 *crcp); - + enum rxe_mr_copy_dir dir, u32 *crcp); int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma, void *addr, int length, - enum copy_direction dir, u32 *crcp); - + enum rxe_mr_copy_dir dir, u32 *crcp); void *iova_to_vaddr(struct rxe_mr *mr, u64 iova, int length); - -enum lookup_type { - lookup_local, - lookup_remote, -}; - struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key, - enum lookup_type type); - + enum rxe_mr_lookup_type type); int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length); - -void rxe_mr_cleanup(struct rxe_pool_entry *arg); - int advance_dma_data(struct rxe_dma_info *dma, unsigned int length); +int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey); +void rxe_mr_cleanup(struct rxe_pool_entry *arg); /* rxe_mw.c */ int rxe_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata); int rxe_dealloc_mw(struct ib_mw *ibmw); int rxe_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe); +int rxe_invalidate_mw(struct rxe_qp *qp, u32 rkey); void rxe_mw_cleanup(struct rxe_pool_entry *arg); /* rxe_net.c */ diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 7f2cfc1ce659..f871879e5f80 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -55,21 +55,6 @@ static void rxe_mr_init(int access, struct rxe_mr *mr) mr->map_shift = ilog2(RXE_BUF_PER_MAP); } -void rxe_mr_cleanup(struct rxe_pool_entry *arg) -{ - struct rxe_mr *mr = container_of(arg, typeof(*mr), pelem); - int i; - - ib_umem_release(mr->umem); - - if (mr->map) { - for (i = 0; i < mr->num_map; i++) - kfree(mr->map[i]); - - kfree(mr->map); - } -} - static int rxe_mr_alloc(struct rxe_mr *mr, int num_buf) { int i; @@ -298,7 +283,7 @@ void *iova_to_vaddr(struct rxe_mr *mr, u64 iova, int length) * crc32 if crcp is not zero. caller must hold a reference to mr */ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, int length, - enum copy_direction dir, u32 *crcp) + enum rxe_mr_copy_dir dir, u32 *crcp) { int err; int bytes; @@ -316,9 +301,9 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, int length, if (mr->type == RXE_MR_TYPE_DMA) { u8 *src, *dest; - src = (dir == to_mr_obj) ? addr : ((void *)(uintptr_t)iova); + src = (dir == RXE_TO_MR_OBJ) ? addr : ((void *)(uintptr_t)iova); - dest = (dir == to_mr_obj) ? ((void *)(uintptr_t)iova) : addr; + dest = (dir == RXE_TO_MR_OBJ) ? ((void *)(uintptr_t)iova) : addr; memcpy(dest, src, length); @@ -346,8 +331,8 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, int length, u8 *src, *dest; va = (u8 *)(uintptr_t)buf->addr + offset; - src = (dir == to_mr_obj) ? addr : va; - dest = (dir == to_mr_obj) ? va : addr; + src = (dir == RXE_TO_MR_OBJ) ? addr : va; + dest = (dir == RXE_TO_MR_OBJ) ? va : addr; bytes = buf->size - offset; @@ -392,7 +377,7 @@ int copy_data( struct rxe_dma_info *dma, void *addr, int length, - enum copy_direction dir, + enum rxe_mr_copy_dir dir, u32 *crcp) { int bytes; @@ -412,7 +397,7 @@ int copy_data( } if (sge->length && (offset < sge->length)) { - mr = lookup_mr(pd, access, sge->lkey, lookup_local); + mr = lookup_mr(pd, access, sge->lkey, RXE_LOOKUP_LOCAL); if (!mr) { err = -EINVAL; goto err1; @@ -438,7 +423,7 @@ int copy_data( if (sge->length) { mr = lookup_mr(pd, access, sge->lkey, - lookup_local); + RXE_LOOKUP_LOCAL); if (!mr) { err = -EINVAL; goto err1; @@ -520,7 +505,7 @@ int advance_dma_data(struct rxe_dma_info *dma, unsigned int length) * (4) verify that mr state is valid */ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key, - enum lookup_type type) + enum rxe_mr_lookup_type type) { struct rxe_mr *mr; struct rxe_dev *rxe = to_rdev(pd->ibpd.device); @@ -530,8 +515,8 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key, if (!mr) return NULL; - if (unlikely((type == lookup_local && mr_lkey(mr) != key) || - (type == lookup_remote && mr_rkey(mr) != key) || + if (unlikely((type == RXE_LOOKUP_LOCAL && mr_lkey(mr) != key) || + (type == RXE_LOOKUP_REMOTE && mr_rkey(mr) != key) || mr_pd(mr) != pd || (access && !(access & mr->access)) || mr->state != RXE_MR_STATE_VALID)) { rxe_drop_ref(mr); @@ -540,3 +525,47 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key, return mr; } + +int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey) +{ + struct rxe_dev *rxe = to_rdev(qp->ibqp.device); + struct rxe_mr *mr; + int ret; + + mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8); + if (!mr) { + pr_err("%s: No MR for rkey %#x\n", __func__, rkey); + ret = -EINVAL; + goto err; + } + + if (rkey != mr->ibmr.rkey) { + pr_err("%s: rkey (%#x) doesn't match mr->ibmr.rkey (%#x)\n", + __func__, rkey, mr->ibmr.rkey); + ret = -EINVAL; + goto err_drop_ref; + } + + mr->state = RXE_MR_STATE_FREE; + ret = 0; + +err_drop_ref: + rxe_drop_ref(mr); +err: + return ret; +} + +void rxe_mr_cleanup(struct rxe_pool_entry *arg) +{ + struct rxe_mr *mr = container_of(arg, typeof(*mr), pelem); + int i; + + ib_umem_release(mr->umem); + + if (mr->map) { + for (i = 0; i < mr->num_map; i++) + kfree(mr->map[i]); + + kfree(mr->map); + } +} diff --git a/drivers/infiniband/sw/rxe/rxe_mw.c b/drivers/infiniband/sw/rxe/rxe_mw.c index 65215dde9974..594f8cef0a08 100644 --- a/drivers/infiniband/sw/rxe/rxe_mw.c +++ b/drivers/infiniband/sw/rxe/rxe_mw.c @@ -245,6 +245,73 @@ int rxe_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe) return ret; } +static int rxe_check_invalidate_mw(struct rxe_qp *qp, struct rxe_mw *mw) +{ + if (unlikely(mw->state == RXE_MW_STATE_INVALID)) + return -EINVAL; + + /* o10-37.2.26 */ + if (unlikely(mw->ibmw.type == IB_MW_TYPE_1)) + return -EINVAL; + + return 0; +} + +static void rxe_do_invalidate_mw(struct rxe_mw *mw) +{ + struct rxe_qp *qp; + struct rxe_mr *mr; + + /* valid type 2 MW will always have a QP pointer */ + qp = mw->qp; + mw->qp = NULL; + rxe_drop_ref(qp); + + /* valid type 2 MW will always have an MR pointer */ + mr = mw->mr; + mw->mr = NULL; + atomic_dec(&mr->num_mw); + rxe_drop_ref(mr); + + mw->access = 0; + mw->addr = 0; + mw->length = 0; + mw->state = RXE_MW_STATE_FREE; +} + +int rxe_invalidate_mw(struct rxe_qp *qp, u32 rkey) +{ + struct rxe_dev *rxe = to_rdev(qp->ibqp.device); + unsigned long flags; + struct rxe_mw *mw; + int ret; + + mw = rxe_pool_get_index(&rxe->mw_pool, rkey >> 8); + if (!mw) { + ret = -EINVAL; + goto err; + } + + if (rkey != mw->ibmw.rkey) { + ret = -EINVAL; + goto err_drop_ref; + } + + spin_lock_irqsave(&mw->lock, flags); + + ret = rxe_check_invalidate_mw(qp, mw); + if (ret) + goto err_unlock; + + rxe_do_invalidate_mw(mw); +err_unlock: + spin_unlock_irqrestore(&mw->lock, flags); +err_drop_ref: + rxe_drop_ref(mw); +err: + return ret; +} + void rxe_mw_cleanup(struct rxe_pool_entry *elem) { struct rxe_mw *mw = container_of(elem, typeof(*mw), pelem); diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 29fa41b4d2ac..52ddf2d1fe61 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -464,7 +464,7 @@ static int fill_packet(struct rxe_qp *qp, struct rxe_send_wqe *wqe, } else { err = copy_data(qp->pd, 0, &wqe->dma, payload_addr(pkt), paylen, - from_mr_obj, + RXE_FROM_MR_OBJ, &crc); if (err) return err; @@ -558,25 +558,25 @@ static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe, static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe) { u8 opcode = wqe->wr.opcode; - struct rxe_dev *rxe; struct rxe_mr *mr; - u32 rkey; int ret; + u32 rkey; switch (opcode) { case IB_WR_LOCAL_INV: - rxe = to_rdev(qp->ibqp.device); rkey = wqe->wr.ex.invalidate_rkey; - mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8); - if (!mr) { - pr_err("No MR for rkey %#x\n", rkey); + if (rkey_is_mw(rkey)) + ret = rxe_invalidate_mw(qp, rkey); + else + ret = rxe_invalidate_mr(qp, rkey); + + if (ret) { wqe->state = wqe_state_error; wqe->status = IB_WC_LOC_QP_OP_ERR; - return -EINVAL; + return ret; } - mr->state = RXE_MR_STATE_FREE; - rxe_drop_ref(mr); break; + case IB_WR_REG_MR: mr = to_rmr(wqe->wr.wr.reg.mr); @@ -588,14 +588,16 @@ static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe) mr->iova = wqe->wr.wr.reg.mr->iova; rxe_drop_ref(mr); break; + case IB_WR_BIND_MW: ret = rxe_bind_mw(qp, wqe); if (ret) { wqe->state = wqe_state_error; wqe->status = IB_WC_MW_BIND_ERR; - return -EINVAL; + return ret; } break; + default: pr_err("Unexpected send wqe opcode %d\n", opcode); wqe->state = wqe_state_error; diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 2b220659bddb..759e9789cd4d 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -35,6 +35,7 @@ enum resp_states { RESPST_ERR_TOO_MANY_RDMA_ATM_REQ, RESPST_ERR_RNR, RESPST_ERR_RKEY_VIOLATION, + RESPST_ERR_INVALIDATE_RKEY, RESPST_ERR_LENGTH, RESPST_ERR_CQ_OVERFLOW, RESPST_ERROR, @@ -68,6 +69,7 @@ static char *resp_state_name[] = { [RESPST_ERR_TOO_MANY_RDMA_ATM_REQ] = "ERR_TOO_MANY_RDMA_ATM_REQ", [RESPST_ERR_RNR] = "ERR_RNR", [RESPST_ERR_RKEY_VIOLATION] = "ERR_RKEY_VIOLATION", + [RESPST_ERR_INVALIDATE_RKEY] = "ERR_INVALIDATE_RKEY_VIOLATION", [RESPST_ERR_LENGTH] = "ERR_LENGTH", [RESPST_ERR_CQ_OVERFLOW] = "ERR_CQ_OVERFLOW", [RESPST_ERROR] = "ERROR", @@ -430,7 +432,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, resid = qp->resp.resid; pktlen = payload_size(pkt); - mr = lookup_mr(qp->pd, access, rkey, lookup_remote); + mr = lookup_mr(qp->pd, access, rkey, RXE_LOOKUP_REMOTE); if (!mr) { state = RESPST_ERR_RKEY_VIOLATION; goto err; @@ -484,7 +486,7 @@ static enum resp_states send_data_in(struct rxe_qp *qp, void *data_addr, int err; err = copy_data(qp->pd, IB_ACCESS_LOCAL_WRITE, &qp->resp.wqe->dma, - data_addr, data_len, to_mr_obj, NULL); + data_addr, data_len, RXE_TO_MR_OBJ, NULL); if (unlikely(err)) return (err == -ENOSPC) ? RESPST_ERR_LENGTH : RESPST_ERR_MALFORMED_WQE; @@ -500,7 +502,7 @@ static enum resp_states write_data_in(struct rxe_qp *qp, int data_len = payload_size(pkt); err = rxe_mr_copy(qp->resp.mr, qp->resp.va, payload_addr(pkt), data_len, - to_mr_obj, NULL); + RXE_TO_MR_OBJ, NULL); if (err) { rc = RESPST_ERR_RKEY_VIOLATION; goto out; @@ -701,7 +703,7 @@ static enum resp_states read_reply(struct rxe_qp *qp, return RESPST_ERR_RNR; err = rxe_mr_copy(res->read.mr, res->read.va, payload_addr(&ack_pkt), - payload, from_mr_obj, &icrc); + payload, RXE_FROM_MR_OBJ, &icrc); if (err) pr_err("Failed copying memory\n"); @@ -751,6 +753,14 @@ static void build_rdma_network_hdr(union rdma_network_hdr *hdr, memcpy(&hdr->ibgrh, ipv6_hdr(skb), sizeof(hdr->ibgrh)); } +static int invalidate_rkey(struct rxe_qp *qp, u32 rkey) +{ + if (rkey_is_mw(rkey)) + return rxe_invalidate_mw(qp, rkey); + else + return rxe_invalidate_mr(qp, rkey); +} + /* Executes a new request. A retried request never reach that function (send * and writes are discarded, and reads and atomics are retried elsewhere. */ @@ -790,6 +800,14 @@ static enum resp_states execute(struct rxe_qp *qp, struct rxe_pkt_info *pkt) WARN_ON_ONCE(1); } + if (pkt->mask & RXE_IETH_MASK) { + u32 rkey = ieth_rkey(pkt); + + err = invalidate_rkey(qp, rkey); + if (err) + return RESPST_ERR_INVALIDATE_RKEY; + } + /* next expected psn, read handles this separately */ qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK; qp->resp.ack_psn = qp->resp.psn; @@ -822,13 +840,13 @@ static enum resp_states do_complete(struct rxe_qp *qp, memset(&cqe, 0, sizeof(cqe)); if (qp->rcq->is_user) { - uwc->status = qp->resp.status; - uwc->qp_num = qp->ibqp.qp_num; - uwc->wr_id = wqe->wr_id; + uwc->status = qp->resp.status; + uwc->qp_num = qp->ibqp.qp_num; + uwc->wr_id = wqe->wr_id; } else { - wc->status = qp->resp.status; - wc->qp = &qp->ibqp; - wc->wr_id = wqe->wr_id; + wc->status = qp->resp.status; + wc->qp = &qp->ibqp; + wc->wr_id = wqe->wr_id; } if (wc->status == IB_WC_SUCCESS) { @@ -883,27 +901,14 @@ static enum resp_states do_complete(struct rxe_qp *qp, } if (pkt->mask & RXE_IETH_MASK) { - struct rxe_mr *rmr; - wc->wc_flags |= IB_WC_WITH_INVALIDATE; wc->ex.invalidate_rkey = ieth_rkey(pkt); - - rmr = rxe_pool_get_index(&rxe->mr_pool, - wc->ex.invalidate_rkey >> 8); - if (unlikely(!rmr)) { - pr_err("Bad rkey %#x invalidation\n", - wc->ex.invalidate_rkey); - return RESPST_ERROR; - } - rmr->state = RXE_MR_STATE_FREE; - rxe_drop_ref(rmr); } - wc->qp = &qp->ibqp; - if (pkt->mask & RXE_DETH_MASK) wc->src_qp = deth_sqp(pkt); + wc->qp = &qp->ibqp; wc->port_num = qp->attr.port_num; } } @@ -1314,6 +1319,13 @@ int rxe_responder(void *arg) } break; + case RESPST_ERR_INVALIDATE_RKEY: + /* RC - Class J. */ + qp->resp.goto_error = 1; + qp->resp.status = IB_WC_REM_INV_REQ_ERR; + state = RESPST_COMPLETE; + break; + case RESPST_ERR_LENGTH: if (qp_type(qp) == IB_QPT_RC) { /* Class C */ diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 7da47b8c707b..74fcd871757d 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -275,6 +275,16 @@ enum rxe_mr_type { RXE_MR_TYPE_MR, }; +enum rxe_mr_copy_dir { + RXE_TO_MR_OBJ, + RXE_FROM_MR_OBJ, +}; + +enum rxe_mr_lookup_type { + RXE_LOOKUP_LOCAL, + RXE_LOOKUP_REMOTE, +}; + #define RXE_BUF_PER_MAP (PAGE_SIZE / sizeof(struct rxe_phys_buf)) struct rxe_phys_buf { @@ -286,6 +296,13 @@ struct rxe_map { struct rxe_phys_buf buf[RXE_BUF_PER_MAP]; }; +static inline int rkey_is_mw(u32 rkey) +{ + u32 index = rkey >> 8; + + return (index >= RXE_MIN_MW_INDEX) && (index <= RXE_MAX_MW_INDEX); +} + struct rxe_mr { struct rxe_pool_entry pelem; struct ib_mr ibmr; @@ -311,23 +328,23 @@ struct rxe_mr { u32 max_buf; u32 num_map; - struct rxe_map **map; - atomic_t num_mw; + + struct rxe_map **map; }; enum rxe_mw_state { - RXE_MW_STATE_INVALID = RXE_MR_STATE_INVALID, - RXE_MW_STATE_FREE = RXE_MR_STATE_FREE, - RXE_MW_STATE_VALID = RXE_MR_STATE_VALID, + RXE_MW_STATE_INVALID = RXE_MR_STATE_INVALID, + RXE_MW_STATE_FREE = RXE_MR_STATE_FREE, + RXE_MW_STATE_VALID = RXE_MR_STATE_VALID, }; struct rxe_mw { - struct ib_mw ibmw; - struct rxe_pool_entry pelem; + struct ib_mw ibmw; + struct rxe_pool_entry pelem; spinlock_t lock; enum rxe_mw_state state; - struct rxe_qp *qp; /* Type 2 only */ + struct rxe_qp *qp; /* Type 2 only */ struct rxe_mr *mr; int access; u64 addr; From patchwork Thu Apr 29 18:48:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56B3DC43462 for ; Thu, 29 Apr 2021 18:49:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 202E56143B for ; Thu, 29 Apr 2021 18:49:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241146AbhD2SuF (ORCPT ); Thu, 29 Apr 2021 14:50:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234028AbhD2SuE (ORCPT ); Thu, 29 Apr 2021 14:50:04 -0400 Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3FB5C06138C for ; Thu, 29 Apr 2021 11:49:17 -0700 (PDT) Received: by mail-ot1-x32c.google.com with SMTP id u21-20020a0568301195b02902a2119f7613so15378064otq.10 for ; Thu, 29 Apr 2021 11:49:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EoNhKdS8t3YwA0z/UiAHN+MCs/4fgfFtTF2IF3qm+QQ=; b=PnqTfg6YEC9o8PNDYLG/1NvkxJoTgVy9GvI/aqA0Mc8czy2ZkQH27RY8MZNcStG2Dg 4F0MzDe6Qp0l7eY8KCnzqlU/i6defcexFOKfiO9Dt5eROBPCCAopiPpDX/ExKIqb6FD0 7/3OzbWwGs0/CTkeJWdl8C/bhXXq8aQUfffuEPAqQxoXfD9SYcP5RcwGokclgI3Y6yCv bBQJtluhGB9B4Nt/ss3VInDGMC/8FDwSFEw4/GOg7Zr8a36deASyC14If9LFZNPR9Gct U5V05egR91b8YmgXojN2+4WXpdQvZEmR79laf4xDRukTpFj9Z3tsfSafiaUc7XAOdGJR 2Akw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EoNhKdS8t3YwA0z/UiAHN+MCs/4fgfFtTF2IF3qm+QQ=; b=arBzNp3wVGBGanc/HyLA3by07/PEPOCQV6xs6AFmIhbLynUVjxCyz/OrigSTefsMVN yPAcfdqwU5tzqzexDiO5mN3mXtCPQEAsijl8iWxRL/sVfwzTpmP1Lu2oThqKoGc4y9rh znLOfc1tI2r8j/mhgf5OL+LpTPemi1yp/qSUc1WCRxFOeLxyuE/vIcW0joLov3o87hOo kjrGp3M681Wwx3s5ZmfbJTJmqGOa+5bKgu4YxanIEgR+hRO2PfRZRJGUE2MmDUQSKFS+ /rpxzCW/AEWWC6Tu0Fwct/RN6i3bIq7ZKbWgocxpDLtk+M/gsD/kLPdyMl4RGBhFPPcc iu3A== X-Gm-Message-State: AOAM531NJjZbYGc5wTm4QIrpJXUjszehLdjmsqZRqyB4nMBZrVnGiQK7 PZ0tCkjE+eJrixXuunSu0X4= X-Google-Smtp-Source: ABdhPJylvwKXOK44aE2kntX2tK5yV4yW/kBfnBxY8YzL7CAayNfmVZdgzjAl3vYec8Toyg/VESmS6Q== X-Received: by 2002:a05:6830:1416:: with SMTP id v22mr648913otp.120.1619722157421; Thu, 29 Apr 2021 11:49:17 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id i9sm133261oog.17.2021.04.29.11.49.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:16 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 09/10] RDMA/rxe: Implement memory access through MWs Date: Thu, 29 Apr 2021 13:48:54 -0500 Message-Id: <20210429184855.54939-10-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add code to implement memory access through memory windows. Signed-off-by: Bob Pearson --- v6: Added rxe_ to subroutine names in lines that changed. v3: Adjusted patch for collisions with v3 of "RDMA/rxe: Implement invalidate MW operations" patch. v2: Removed a copy of changes in ea4922518940 "Fix missing acks from responder" that was submitted separately. --- drivers/infiniband/sw/rxe/rxe_loc.h | 1 + drivers/infiniband/sw/rxe/rxe_mw.c | 23 +++++++++++ drivers/infiniband/sw/rxe/rxe_resp.c | 55 +++++++++++++++++++-------- drivers/infiniband/sw/rxe/rxe_verbs.h | 11 ++++++ 4 files changed, 75 insertions(+), 15 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index bc0e484f8cde..4624033d8dce 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -94,6 +94,7 @@ int rxe_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata); int rxe_dealloc_mw(struct ib_mw *ibmw); int rxe_bind_mw(struct rxe_qp *qp, struct rxe_send_wqe *wqe); int rxe_invalidate_mw(struct rxe_qp *qp, u32 rkey); +struct rxe_mw *rxe_lookup_mw(struct rxe_qp *qp, int access, u32 rkey); void rxe_mw_cleanup(struct rxe_pool_entry *arg); /* rxe_net.c */ diff --git a/drivers/infiniband/sw/rxe/rxe_mw.c b/drivers/infiniband/sw/rxe/rxe_mw.c index 594f8cef0a08..5ba77df7598e 100644 --- a/drivers/infiniband/sw/rxe/rxe_mw.c +++ b/drivers/infiniband/sw/rxe/rxe_mw.c @@ -312,6 +312,29 @@ int rxe_invalidate_mw(struct rxe_qp *qp, u32 rkey) return ret; } +struct rxe_mw *rxe_lookup_mw(struct rxe_qp *qp, int access, u32 rkey) +{ + struct rxe_dev *rxe = to_rdev(qp->ibqp.device); + struct rxe_pd *pd = to_rpd(qp->ibqp.pd); + struct rxe_mw *mw; + int index = rkey >> 8; + + mw = rxe_pool_get_index(&rxe->mw_pool, index); + if (!mw) + return NULL; + + if (unlikely((rxe_mw_rkey(mw) != rkey) || rxe_mw_pd(mw) != pd || + (mw->ibmw.type == IB_MW_TYPE_2 && mw->qp != qp) || + (mw->length == 0) || + (access && !(access & mw->access)) || + mw->state != RXE_MW_STATE_VALID)) { + rxe_drop_ref(mw); + return NULL; + } + + return mw; +} + void rxe_mw_cleanup(struct rxe_pool_entry *elem) { struct rxe_mw *mw = container_of(elem, typeof(*mw), pelem); diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 759e9789cd4d..1ea576d42882 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -394,6 +394,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { struct rxe_mr *mr = NULL; + struct rxe_mw *mw = NULL; u64 va; u32 rkey; u32 resid; @@ -405,6 +406,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, if (pkt->mask & (RXE_READ_MASK | RXE_WRITE_MASK)) { if (pkt->mask & RXE_RETH_MASK) { qp->resp.va = reth_va(pkt); + qp->resp.offset = 0; qp->resp.rkey = reth_rkey(pkt); qp->resp.resid = reth_len(pkt); qp->resp.length = reth_len(pkt); @@ -413,6 +415,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, : IB_ACCESS_REMOTE_WRITE; } else if (pkt->mask & RXE_ATOMIC_MASK) { qp->resp.va = atmeth_va(pkt); + qp->resp.offset = 0; qp->resp.rkey = atmeth_rkey(pkt); qp->resp.resid = sizeof(u64); access = IB_ACCESS_REMOTE_ATOMIC; @@ -432,18 +435,36 @@ static enum resp_states check_rkey(struct rxe_qp *qp, resid = qp->resp.resid; pktlen = payload_size(pkt); - mr = lookup_mr(qp->pd, access, rkey, RXE_LOOKUP_REMOTE); - if (!mr) { - state = RESPST_ERR_RKEY_VIOLATION; - goto err; - } + if (rkey_is_mw(rkey)) { + mw = rxe_lookup_mw(qp, access, rkey); + if (!mw) { + pr_err("%s: no MW matches rkey %#x\n", __func__, rkey); + state = RESPST_ERR_RKEY_VIOLATION; + goto err; + } - if (unlikely(mr->state == RXE_MR_STATE_FREE)) { - state = RESPST_ERR_RKEY_VIOLATION; - goto err; + mr = mw->mr; + if (!mr) { + pr_err("%s: MW doesn't have an MR\n", __func__); + state = RESPST_ERR_RKEY_VIOLATION; + goto err; + } + + if (mw->access & IB_ZERO_BASED) + qp->resp.offset = mw->addr; + + rxe_drop_ref(mw); + rxe_add_ref(mr); + } else { + mr = lookup_mr(qp->pd, access, rkey, RXE_LOOKUP_REMOTE); + if (!mr) { + pr_err("%s: no MR matches rkey %#x\n", __func__, rkey); + state = RESPST_ERR_RKEY_VIOLATION; + goto err; + } } - if (mr_check_range(mr, va, resid)) { + if (mr_check_range(mr, va + qp->resp.offset, resid)) { state = RESPST_ERR_RKEY_VIOLATION; goto err; } @@ -477,6 +498,9 @@ static enum resp_states check_rkey(struct rxe_qp *qp, err: if (mr) rxe_drop_ref(mr); + if (mw) + rxe_drop_ref(mw); + return state; } @@ -501,8 +525,8 @@ static enum resp_states write_data_in(struct rxe_qp *qp, int err; int data_len = payload_size(pkt); - err = rxe_mr_copy(qp->resp.mr, qp->resp.va, payload_addr(pkt), data_len, - RXE_TO_MR_OBJ, NULL); + err = rxe_mr_copy(qp->resp.mr, qp->resp.va + qp->resp.offset, + payload_addr(pkt), data_len, RXE_TO_MR_OBJ, NULL); if (err) { rc = RESPST_ERR_RKEY_VIOLATION; goto out; @@ -521,7 +545,6 @@ static DEFINE_SPINLOCK(atomic_ops_lock); static enum resp_states process_atomic(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { - u64 iova = atmeth_va(pkt); u64 *vaddr; enum resp_states ret; struct rxe_mr *mr = qp->resp.mr; @@ -531,7 +554,7 @@ static enum resp_states process_atomic(struct rxe_qp *qp, goto out; } - vaddr = iova_to_vaddr(mr, iova, sizeof(u64)); + vaddr = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, sizeof(u64)); /* check vaddr is 8 bytes aligned. */ if (!vaddr || (uintptr_t)vaddr & 7) { @@ -655,8 +678,10 @@ static enum resp_states read_reply(struct rxe_qp *qp, res->type = RXE_READ_MASK; res->replay = 0; - res->read.va = qp->resp.va; - res->read.va_org = qp->resp.va; + res->read.va = qp->resp.va + + qp->resp.offset; + res->read.va_org = qp->resp.va + + qp->resp.offset; res->first_psn = req_pkt->psn; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 74fcd871757d..cf8cae64f7df 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -183,6 +183,7 @@ struct rxe_resp_info { /* RDMA read / atomic only */ u64 va; + u64 offset; struct rxe_mr *mr; u32 resid; u32 rkey; @@ -480,6 +481,16 @@ static inline u32 mr_rkey(struct rxe_mr *mr) return mr->ibmr.rkey; } +static inline struct rxe_pd *rxe_mw_pd(struct rxe_mw *mw) +{ + return to_rpd(mw->ibmw.pd); +} + +static inline u32 rxe_mw_rkey(struct rxe_mw *mw) +{ + return mw->ibmw.rkey; +} + int rxe_register_device(struct rxe_dev *rxe, const char *ibdev_name); void rxe_mc_cleanup(struct rxe_pool_entry *arg); From patchwork Thu Apr 29 18:48:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12231687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 457DFC43461 for ; Thu, 29 Apr 2021 18:49:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 00306613B3 for ; Thu, 29 Apr 2021 18:49:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234028AbhD2SuF (ORCPT ); Thu, 29 Apr 2021 14:50:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241184AbhD2SuF (ORCPT ); Thu, 29 Apr 2021 14:50:05 -0400 Received: from mail-ot1-x332.google.com (mail-ot1-x332.google.com [IPv6:2607:f8b0:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B59CAC06138B for ; Thu, 29 Apr 2021 11:49:18 -0700 (PDT) Received: by mail-ot1-x332.google.com with SMTP id n32-20020a9d1ea30000b02902a53d6ad4bdso7278079otn.3 for ; Thu, 29 Apr 2021 11:49:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rcDczA81N59JOUA4+nyxBXcfhrqAYLYY+OocJlGyOXg=; b=UeoS7VmX5laEBquYqG9mNjSkLoj++WHh/c040VA/Qn8s9iLUmP2B6ggZHBgCCJL3LU gSXQ0eyxk61L9Uw9Qq+AL2T0fFrvG6wMDY5ZCwkiiHAEiiC1iJOFDVgjr7WfcMw0gSdF PL5wJQsDIsBLSXfJ+WXDW3BgO+2OakndVVjDSwaGUwKdP4x809qNroQc5OGxq+LXG1On Th67xX/5MOG3EZuEbUr/sNokFtQWVraLnYD0dliPvEWIhBNsdsAUU21qM7kM/5VAb3/3 ei8jKQLvQx0LqUUzP/P3p9+AXtpkcd3TDMXlAZ7cvPJVZrZPRm8+gv0MgA5CwCXQi40U QhqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rcDczA81N59JOUA4+nyxBXcfhrqAYLYY+OocJlGyOXg=; b=bc89uadrvRkWlvHPnGwDshZYOD3XXSL51cIVq+xG6lJG8ne/nnhsE0vixgjGsZMIHy 8t8M+5QDpqfdjMpvOgZd7ogmcDbOvWAx9mNj+UCMRTzVGvX9Elhm6hI8+bMjIA+F/dCi +KFzqR/WPTK1O34fApYyiaSVzL8a1QCt9A8GyMhr7xSBXZz04EA6TvJ5XwXBOLn1nlSK +OhviMouUCzVM7KJmM9+fbFUIFsK3NP6eN6GPR8dRTJUPRFHSe3PsHqT6m1xif4irrQh 1bUHIX13bhGVZx60HWeHR86jP5pBfVJztzqHneHkJbIuyJfR1CF0iIKvrvfea52P0aWz HytQ== X-Gm-Message-State: AOAM5311XXmvGL4egiNopTBE9mRe4gtgKbzRclV5M5yEqpc9IvQHEy9V WuAkJ/OYQn/t2d4Go44GtT/OtmWnreogSg== X-Google-Smtp-Source: ABdhPJxur8+AeCPuIADxVxbo/b4iRXel89koNYNpuhAo7snXmQfjK+ZPU244vS8PBJCBjpM5qDC3ww== X-Received: by 2002:a9d:2787:: with SMTP id c7mr628594otb.105.1619722158211; Thu, 29 Apr 2021 11:49:18 -0700 (PDT) Received: from localhost (2603-8081-140c-1a00-a63d-9643-fc29-df2a.res6.spectrum.com. [2603:8081:140c:1a00:a63d:9643:fc29:df2a]) by smtp.gmail.com with ESMTPSA id q130sm172409oif.40.2021.04.29.11.49.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 11:49:17 -0700 (PDT) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v6 10/10] RDMA/rxe: Disallow MR dereg and invalidate when bound Date: Thu, 29 Apr 2021 13:48:55 -0500 Message-Id: <20210429184855.54939-11-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210429184855.54939-1-rpearson@hpe.com> References: <20210429184855.54939-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Check that an MR has no bound MWs before allowing a dereg or invalidate operation. Signed-off-by: Bob Pearson --- v5: Fixed a typo in v4. v4: Added this patch to check mr->num_mw to disallow dereg and invalidate operations when MR has MW's bound. --- drivers/infiniband/sw/rxe/rxe_loc.h | 1 + drivers/infiniband/sw/rxe/rxe_mr.c | 25 +++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_verbs.c | 11 ----------- 3 files changed, 26 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index 4624033d8dce..a167ae3c3514 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -87,6 +87,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key, int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length); int advance_dma_data(struct rxe_dma_info *dma, unsigned int length); int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey); +int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata); void rxe_mr_cleanup(struct rxe_pool_entry *arg); /* rxe_mw.c */ diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index f871879e5f80..6a2377030f52 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -546,6 +546,13 @@ int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey) goto err_drop_ref; } + if (atomic_read(&mr->num_mw) > 0) { + pr_warn("%s: Attempt to invalidate an MR while bound to MWs\n", + __func__); + ret = -EINVAL; + goto err_drop_ref; + } + mr->state = RXE_MR_STATE_FREE; ret = 0; @@ -555,6 +562,24 @@ int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey) return ret; } +int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata) +{ + struct rxe_mr *mr = to_rmr(ibmr); + + if (atomic_read(&mr->num_mw) > 0) { + pr_warn("%s: Attempt to deregister an MR while bound to MWs\n", + __func__); + return -EINVAL; + } + + mr->state = RXE_MR_STATE_ZOMBIE; + rxe_drop_ref(mr_pd(mr)); + rxe_drop_index(mr); + rxe_drop_ref(mr); + + return 0; +} + void rxe_mr_cleanup(struct rxe_pool_entry *arg) { struct rxe_mr *mr = container_of(arg, typeof(*mr), pelem); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index d22f011a20f3..89f8f00215d6 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -913,17 +913,6 @@ static struct ib_mr *rxe_reg_user_mr(struct ib_pd *ibpd, return ERR_PTR(err); } -static int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata) -{ - struct rxe_mr *mr = to_rmr(ibmr); - - mr->state = RXE_MR_STATE_ZOMBIE; - rxe_drop_ref(mr_pd(mr)); - rxe_drop_index(mr); - rxe_drop_ref(mr); - return 0; -} - static struct ib_mr *rxe_alloc_mr(struct ib_pd *ibpd, enum ib_mr_type mr_type, u32 max_num_sg) {