From patchwork Thu Feb 7 05:41:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800381 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62917922 for ; Thu, 7 Feb 2019 05:42:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3EB6C2CBD8 for ; Thu, 7 Feb 2019 05:42:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 32E312CBF1; Thu, 7 Feb 2019 05:42:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D73392CBD8 for ; Thu, 7 Feb 2019 05:42:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726579AbfBGFmA (ORCPT ); Thu, 7 Feb 2019 00:42:00 -0500 Received: from mail-pf1-f194.google.com ([209.85.210.194]:42196 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726128AbfBGFmA (ORCPT ); Thu, 7 Feb 2019 00:42:00 -0500 Received: by mail-pf1-f194.google.com with SMTP id m6so850947pfa.9 for ; Wed, 06 Feb 2019 21:41:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7zg5ifM6mbabG6G9v879XFtu8Izdob5CTRyjl7tY/iA=; b=dIoopeK1HBDsR+tUioiH4SurCO4GgrNDSZ/wM6A07ilD0yEVFEwa2CqCCbT99VVNUX ROaRJyOocffVPpyznj8A6pKpDQd+JAfEWf0uKZDouL0ogLBAL387iFe4OMFcbZaOsqbe jary6V12RaN/zuBWun2l6k/+K1yVCytqn7BKQtebxmp1fFMej+w0dzCwnHTbXbctGfOb Fz7cX+1POve+ENuOgNSSnto38rYkPHcK810VO0tuKZA1YKY4Be3y4LrPztGwDPUHuEr5 NV/AZkYlGFPlo88TEQKets3L40cIAU1MhcM0rW/5GsoBjYI4aVP+YDxU2NBDimyzeexD LPuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7zg5ifM6mbabG6G9v879XFtu8Izdob5CTRyjl7tY/iA=; b=szAsVXDQP+gtQymt+Xx1bQUXX5UgSvafpPwxErFOCChJYZS/z9cb7eO8lVWIboIJgp UyR4hDbaiMRreiIY1kEeSJUgakJNVZk0D/kxA4ZMeFKkAjtLQttZbJXn4Ad/yPecphbc TD+Rskr6MapvjaW+4E+a8F4mqq4PMDV1Yg14sxEWlBxyvuXsa2j9Cley+anz3DUi15kT qTHOzBhMYRL9UC50vfTmqZpXG7BJp/Ek6qgkTUwhzQbJec7NVROuaieUupD3eWWa6vwx GtmAyT7F4LUFeq+FAbSy2rVOE0diVAqM7J7psd5c2tU52MqaDbfGZ89FILeG1iN9sl5K N9BQ== X-Gm-Message-State: AHQUAuaWouPzDclfrPAwq6qoj/tMEVlkfIA6/b29cLgPj40CgzKcC6jD ZGRvg821VSOVf6O3Xz2UmZsLtKrtcF8= X-Google-Smtp-Source: AHgI3IZ65L94onNGMN/mbkXQMendJlbM8QoHnwzAT/0d8mxSRXbqtD6TwwJ4KDppa3IBCxdirduTlQ== X-Received: by 2002:a63:8742:: with SMTP id i63mr13151081pge.298.1549518119199; Wed, 06 Feb 2019 21:41:59 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id b7sm10602586pfa.52.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:41:58 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRh-0006DI-UB; Wed, 06 Feb 2019 22:41:57 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Parav Pandit Subject: [PATCH v2 1/8] RDMA/device: Check that the rename is nop under the lock Date: Wed, 6 Feb 2019 22:41:47 -0700 Message-Id: <20190207054154.23806-2-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Since another rename could be running in parallel it is safer to check that the name is not changing inside the lock, where we already know the device name will not change. Fixes: d21943dd19b5 ("RDMA/core: Implement IB device rename function") Signed-off-by: Jason Gunthorpe Reviewed-by: Parav Pandit --- drivers/infiniband/core/device.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 55221990d946b2..a7c766cdf28ca4 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -189,12 +189,14 @@ static struct ib_device *__ib_device_get_by_name(const char *name) int ib_device_rename(struct ib_device *ibdev, const char *name) { - int ret = 0; - - if (!strcmp(name, dev_name(&ibdev->dev))) - return ret; + int ret; mutex_lock(&device_mutex); + if (!strcmp(name, dev_name(&ibdev->dev))) { + ret = 0; + goto out; + } + if (__ib_device_get_by_name(name)) { ret = -EEXIST; goto out; From patchwork Thu Feb 7 05:41:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800383 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECF251823 for ; Thu, 7 Feb 2019 05:42:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD1D52CBD8 for ; Thu, 7 Feb 2019 05:42:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CE9632CBF1; Thu, 7 Feb 2019 05:42:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 633D42CBDD for ; Thu, 7 Feb 2019 05:42:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726615AbfBGFmA (ORCPT ); Thu, 7 Feb 2019 00:42:00 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:37245 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726549AbfBGFmA (ORCPT ); Thu, 7 Feb 2019 00:42:00 -0500 Received: by mail-pl1-f194.google.com with SMTP id b5so4288529plr.4 for ; Wed, 06 Feb 2019 21:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e2OduUzYSyKZI+h7Uborb2ptowX1KXnGt7BYZZxqsH8=; b=mX5l4OYvIkPrSxTYn2c4KwwysciBsqbYPRBRf3ET64sfEF2ya9kMqjCVyWDr1PKKod SS92P8gU8UdO7OTVh5JRTCXeI4IezxcbEApp/dsbmlVqA1yrjEbXPRO0xuPynlkafoax cdnNuGQ7z+K2ym51ncSIOMXirCsVxAutoXdY72lus4v4Y79D6uO6w0Qj0rTans6n8okH L+C+FgT081G4BmoKyQS0HhA32ja63WTr94XyCOOHgd+q30+sqri1sDJZqRVXoDDUwWKB T0GaKLBSZffsHbTdV5503Gw2h/L07Dl6KsX4l0uLKMDMzcicsUtnFQ4oMiWKRD3T7PNl mANg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e2OduUzYSyKZI+h7Uborb2ptowX1KXnGt7BYZZxqsH8=; b=Ip3PPIRZbG+qTLWlz0w2MaXlxrWyrTdACTwR1vq+8we08yFArkcIvnTK9TIJukURIR YhGFg3/+u3FYZCGwGHnxEjRw2bng7n4KgpTMQhV0dEWhiiqppZvt6txYktD2yEXQEnyc aOrFX98PLil2U3gGMF30Ku49m/cSNrZhO/mQh624w38mV8KoDj2M9b0H1KF1utn/w7Io i9r+AhN+/yHC/3uYZ7CH9AdI31tZNVHj/mFs37YGtZpHB5tDPgXz3wqEaopypsmWQ6Yx NX3Ip7nOpWHmrY2Am+J6slS5tvL+wT9BZiAupIXQ5E7q5gJ1yopdcvdjiknMjV+biLbC lC0g== X-Gm-Message-State: AHQUAuY7WzICcUg9TdHGmK4TY0RUBmrc4TX2Z6/dBy7nvHn2CgyTLTSV +VgBuhzyPh+AyHSC/AfgAdkPVGqvM2E= X-Google-Smtp-Source: AHgI3IZ0WZHB+ock9PqZ3YIivHCtj2sof/kQxrV5f4+UZsOQBmGCpNFYaXTM+p/ObCac/EXGw05F8A== X-Received: by 2002:a17:902:7243:: with SMTP id c3mr14611287pll.53.1549518119531; Wed, 06 Feb 2019 21:41:59 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id 15sm12382157pfr.55.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:41:58 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRi-0006DP-0E; Wed, 06 Feb 2019 22:41:58 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH v2 2/8] RDMA/device: Ensure that security memory is always freed Date: Wed, 6 Feb 2019 22:41:48 -0700 Message-Id: <20190207054154.23806-3-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Since this only frees memory it should be done during the release callback. Otherwise there are possible error flows where it might not get called if registration aborts. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/core_priv.h | 4 ++-- drivers/infiniband/core/device.c | 10 +++------- drivers/infiniband/core/security.c | 4 +--- 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h index bcb3e3029a9be8..52d17e7eddc35f 100644 --- a/drivers/infiniband/core/core_priv.h +++ b/drivers/infiniband/core/core_priv.h @@ -181,7 +181,7 @@ int ib_get_cached_subnet_prefix(struct ib_device *device, u64 *sn_pfx); #ifdef CONFIG_SECURITY_INFINIBAND -void ib_security_destroy_port_pkey_list(struct ib_device *device); +void ib_security_release_port_pkey_list(struct ib_device *device); void ib_security_cache_change(struct ib_device *device, u8 port_num, @@ -203,7 +203,7 @@ int ib_mad_agent_security_setup(struct ib_mad_agent *agent, void ib_mad_agent_security_cleanup(struct ib_mad_agent *agent); int ib_mad_enforce_security(struct ib_mad_agent_private *map, u16 pkey_index); #else -static inline void ib_security_destroy_port_pkey_list(struct ib_device *device) +static inline void ib_security_release_port_pkey_list(struct ib_device *device) { } diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index a7c766cdf28ca4..3455a3b5d0d1a8 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -253,6 +253,8 @@ static void ib_device_release(struct device *device) ib_cache_release_one(dev); kfree(dev->port_immutable); } + ib_security_release_port_pkey_list(dev); + kfree(dev->port_pkey_list); kfree(dev); } @@ -521,7 +523,6 @@ static void cleanup_device(struct ib_device *device) { ib_cache_cleanup_one(device); ib_cache_release_one(device); - kfree(device->port_pkey_list); kfree(device->port_immutable); } @@ -559,12 +560,10 @@ static int setup_device(struct ib_device *device) if (ret) { dev_warn(&device->dev, "Couldn't set up InfiniBand P_Key/GID cache\n"); - goto pkey_cleanup; + return ret; } return 0; -pkey_cleanup: - kfree(device->port_pkey_list); port_cleanup: kfree(device->port_immutable); return ret; @@ -681,9 +680,6 @@ void ib_unregister_device(struct ib_device *device) ib_cache_cleanup_one(device); - ib_security_destroy_port_pkey_list(device); - kfree(device->port_pkey_list); - down_write(&lists_rwsem); write_lock_irqsave(&device->client_data_lock, flags); list_for_each_entry_safe(context, tmp, &device->client_data_list, diff --git a/drivers/infiniband/core/security.c b/drivers/infiniband/core/security.c index 1efadbccf394df..df2602a91350e6 100644 --- a/drivers/infiniband/core/security.c +++ b/drivers/infiniband/core/security.c @@ -554,13 +554,12 @@ void ib_security_cache_change(struct ib_device *device, } } -void ib_security_destroy_port_pkey_list(struct ib_device *device) +void ib_security_release_port_pkey_list(struct ib_device *device) { struct pkey_index_qp_list *pkey, *tmp_pkey; int i; for (i = rdma_start_port(device); i <= rdma_end_port(device); i++) { - spin_lock(&device->port_pkey_list[i].list_lock); list_for_each_entry_safe(pkey, tmp_pkey, &device->port_pkey_list[i].pkey_list, @@ -568,7 +567,6 @@ void ib_security_destroy_port_pkey_list(struct ib_device *device) list_del(&pkey->pkey_index_list); kfree(pkey); } - spin_unlock(&device->port_pkey_list[i].list_lock); } } From patchwork Thu Feb 7 05:41:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800387 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 49FA71823 for ; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3A8F72CBD8 for ; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2E28C2CBF1; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB6A2CBD8 for ; Thu, 7 Feb 2019 05:42:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726128AbfBGFmB (ORCPT ); Thu, 7 Feb 2019 00:42:01 -0500 Received: from mail-pf1-f195.google.com ([209.85.210.195]:45964 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726549AbfBGFmB (ORCPT ); Thu, 7 Feb 2019 00:42:01 -0500 Received: by mail-pf1-f195.google.com with SMTP id j3so2148419pfi.12 for ; Wed, 06 Feb 2019 21:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WzzTr1zUThAK6Cz4kUZkolUdF8RaZkX/Zb+Ix8lX6S4=; b=lYreKs42v7cj1ATL7Ei2W8X9CWInLNu3mzrRM7PnXLlSES2q1dVR9bNV8pIr0ZQTcn XFbtb/8iKzC/zSNSnp68aL11sNQ+cvIbm9QqI8Lq0F2uH3zrCZxduO3mexKRc492GYZx +x8x6Xnw7SqcZzqYHk5Vgyvw6rlKklR9HiYVDnrnN2dyi3lEwhTq5ib0oblLoY4iW6kU QS6buEHWjcNImqiXkEwTo8dddicgNDlt8nRgipAyzNpAZ0ZO7ME9GW8ZlWJMJ1o3a3e1 cFNJUBFKUdUcVzI2cWWh699n+D2YbYHkSkwmBwEcvW3gPVX5JIxvtrIgYfJO0ETK3R38 fEdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WzzTr1zUThAK6Cz4kUZkolUdF8RaZkX/Zb+Ix8lX6S4=; b=c6uKmwkvYP7p/92JVl22IXObouPVdsMdo4JuwcWYihlqcD73KjiPBU5gMCK1PJLBws U+1DDhFyRjDNmL7dA0dIDuwZAXDsjul7qfigGNnuF5RdM/cxRhYY3NgJ6jpVhdsEaVGI 0DSkks2gqEhMyDrKvoiudNRyh2rh99ADvpr/F+2NJBw+wC33EgmPCYJ4BESW+QqiQbw1 yI+I1WX7AfH8c6K+iBpqCVT3T0dOkZdyacx9x6EwgtDHi3Bo1Jzoax6azAfkFAUYZtxN mtFWVnTnNi8aD5Gaa4wus7nuejID9TBOml8u9OvMJwnQRtAukPpba/poZDuKlMeTeswD MX3A== X-Gm-Message-State: AHQUAuZM11D5Nn1l23FiBn95a0j5XUCq/OKbGiUgBrA8iIf3amkRoBs0 5ZgZmufSUWW9QDK1WtwkMpJVvoSXoas= X-Google-Smtp-Source: AHgI3IZp3OGAhEr6y3+xFwPo26DgAlEtqVWrz9K+YNQqr50m5uYag3hIDhUUezC6MNzXApGGrGdPcg== X-Received: by 2002:a63:e20a:: with SMTP id q10mr12809768pgh.206.1549518120274; Wed, 06 Feb 2019 21:42:00 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id b85sm1333377pfm.0.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:41:58 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRi-0006DV-1f; Wed, 06 Feb 2019 22:41:58 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH v2 3/8] RDMA/device: Call ib_cache_release_one() only from ib_device_release() Date: Wed, 6 Feb 2019 22:41:49 -0700 Message-Id: <20190207054154.23806-4-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Instead of complicated logic about when this memory is freed, always free it during device release(). All the cache pointers start out as NULL, so it is safe to call this before the cache is initialized. This makes for a simpler error unwind flow, and a simpler understanding of the lifetime of the memory allocations inside the struct ib_device. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/cache.c | 3 +++ drivers/infiniband/core/device.c | 41 ++++++++++---------------------- 2 files changed, 15 insertions(+), 29 deletions(-) diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c index 7b04590f307f52..2338d0b3a0cab8 100644 --- a/drivers/infiniband/core/cache.c +++ b/drivers/infiniband/core/cache.c @@ -1460,6 +1460,9 @@ void ib_cache_release_one(struct ib_device *device) { int p; + if (!device->cache.ports) + return; + /* * The release function frees all the cache elements. * This function should be called as part of freeing diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 3455a3b5d0d1a8..803df192b7750e 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -244,17 +244,10 @@ static void ib_device_release(struct device *device) struct ib_device *dev = container_of(device, struct ib_device, dev); WARN_ON(dev->reg_state == IB_DEV_REGISTERED); - if (dev->reg_state == IB_DEV_UNREGISTERED) { - /* - * In IB_DEV_UNINITIALIZED state, cache or port table - * is not even created. Free cache and port table only when - * device reaches UNREGISTERED state. - */ - ib_cache_release_one(dev); - kfree(dev->port_immutable); - } + ib_cache_release_one(dev); ib_security_release_port_pkey_list(dev); kfree(dev->port_pkey_list); + kfree(dev->port_immutable); kfree(dev); } @@ -519,13 +512,6 @@ static void setup_dma_device(struct ib_device *device) } } -static void cleanup_device(struct ib_device *device) -{ - ib_cache_cleanup_one(device); - ib_cache_release_one(device); - kfree(device->port_immutable); -} - static int setup_device(struct ib_device *device) { struct ib_udata uhw = {.outlen = 0, .inlen = 0}; @@ -547,26 +533,16 @@ static int setup_device(struct ib_device *device) if (ret) { dev_warn(&device->dev, "Couldn't query the device attributes\n"); - goto port_cleanup; + return ret; } ret = setup_port_pkey_list(device); if (ret) { dev_warn(&device->dev, "Couldn't create per port_pkey_list\n"); - goto port_cleanup; - } - - ret = ib_cache_setup_one(device); - if (ret) { - dev_warn(&device->dev, - "Couldn't set up InfiniBand P_Key/GID cache\n"); return ret; } - return 0; -port_cleanup: - kfree(device->port_immutable); - return ret; + return 0; } /** @@ -606,6 +582,13 @@ int ib_register_device(struct ib_device *device, const char *name) if (ret) goto out; + ret = ib_cache_setup_one(device); + if (ret) { + dev_warn(&device->dev, + "Couldn't set up InfiniBand P_Key/GID cache\n"); + goto out; + } + device->index = __dev_new_index(); ib_device_register_rdmacg(device); @@ -632,7 +615,7 @@ int ib_register_device(struct ib_device *device, const char *name) cg_cleanup: ib_device_unregister_rdmacg(device); - cleanup_device(device); + ib_cache_cleanup_one(device); out: mutex_unlock(&device_mutex); return ret; From patchwork Thu Feb 7 05:41:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800389 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E6272922 for ; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D653A2CBD8 for ; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CB1752CBF6; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 713B32CBD8 for ; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726549AbfBGFmC (ORCPT ); Thu, 7 Feb 2019 00:42:02 -0500 Received: from mail-pf1-f196.google.com ([209.85.210.196]:42201 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726666AbfBGFmB (ORCPT ); Thu, 7 Feb 2019 00:42:01 -0500 Received: by mail-pf1-f196.google.com with SMTP id m6so850997pfa.9 for ; Wed, 06 Feb 2019 21:42:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NjKBCNcA7YJyWZCtFD3QA/1cCY64AQZMq8LNnDA1zls=; b=OMDcDJnDZhAeWoB7bloljNVZz4uyj6elK5wWvAgaDg3Uc7hVmVouU4cgnbTHV/3J7z 047v63dVkQUv32ydyUIxq3o8FMVp8CfY/vvei9PJHzROPi7EB5PEUKP1MCQN2PFOSMyo BfGDeJZkuxkZwwIbEEitT5mljzCrY2pnHQmX7e2IqYs6EWi4JHHzPOVe/Cw6EknbL6zY MCTzYF4ppC226dpqnkzFwfkmChSr82HZsfyWExhHt6ZUG+BRNVvctEhufLbPO82iq66b ctJYeLu2IcXTXupfWpt/c4u5bzXIkg0SDZVvFwrQDiib+NPkkzQu9e3Q5aGvaX2idNDW oSCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NjKBCNcA7YJyWZCtFD3QA/1cCY64AQZMq8LNnDA1zls=; b=iECK5ws1jx+BiKXc2wej6tX0FT81yC32IHZuEDZBGwOwayMg4vrCR/Ra6C5PzrnAaJ rBd/nCdcNaqDsrfVUP8uaXi3NUwcW8iI3aL3aERwOiZlsBliiFEH2eEl7Fsb1pqqBV5q wH1Yk3LE/SEDWFoI1odZuiD3kN+Mw/Es9qdyMH2egI3SrNbJbdO8s0LP1YrV6l26/pBO 7x/b/Uw5Dr8xzphlyRRtboQ3SdG3lHzeagmJa/IYcoRvUtQiYhO+kBZALpwXdyxUqknG 1OYAmda5ThfQ90a0LbPyUaoFdHJLnImLKbIVpOnQgPBrrQtpwjQNR46Ci/ZlpAMcIkne OPEw== X-Gm-Message-State: AHQUAubJBXjZBe8PSqXdBigvGi5J4Phs+A0mZgUeGIqRrTCLp2ajhqlu pRGOYMwQoVhgY3ByLMfaEbrS83sf3OY= X-Google-Smtp-Source: AHgI3IZwFEqYshJEgr3rlBW5J+qmTbuwy8sWtxql8/k6AcYrpZzxGUq12kqEEPF9AKaEjMdIWp2ZAA== X-Received: by 2002:a63:cf4b:: with SMTP id b11mr13431068pgj.405.1549518120788; Wed, 06 Feb 2019 21:42:00 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id l82sm22014128pfg.127.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:41:58 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRi-0006Db-3K; Wed, 06 Feb 2019 22:41:58 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Parav Pandit Subject: [PATCH v2 4/8] RDMA/device: Get rid of reg_state Date: Wed, 6 Feb 2019 22:41:50 -0700 Message-Id: <20190207054154.23806-5-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This really has no purpose anymore, refcount can be used to tell if the device is still registered. Keeping it around just invites mis-use. Signed-off-by: Jason Gunthorpe Reviewed-by: Parav Pandit --- drivers/infiniband/core/device.c | 8 ++------ include/rdma/ib_verbs.h | 6 ------ 2 files changed, 2 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 803df192b7750e..905d7b3780beca 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -243,7 +243,7 @@ static void ib_device_release(struct device *device) { struct ib_device *dev = container_of(device, struct ib_device, dev); - WARN_ON(dev->reg_state == IB_DEV_REGISTERED); + WARN_ON(refcount_read(&dev->refcount)); ib_cache_release_one(dev); ib_security_release_port_pkey_list(dev); kfree(dev->port_pkey_list); @@ -316,8 +316,7 @@ EXPORT_SYMBOL(_ib_alloc_device); void ib_dealloc_device(struct ib_device *device) { WARN_ON(!list_empty(&device->client_data_list)); - WARN_ON(device->reg_state != IB_DEV_UNREGISTERED && - device->reg_state != IB_DEV_UNINITIALIZED); + WARN_ON(refcount_read(&device->refcount)); rdma_restrack_clean(device); put_device(&device->dev); } @@ -601,7 +600,6 @@ int ib_register_device(struct ib_device *device, const char *name) } refcount_set(&device->refcount, 1); - device->reg_state = IB_DEV_REGISTERED; list_for_each_entry(client, &client_list, list) if (!add_client_context(device, client) && client->add) @@ -672,8 +670,6 @@ void ib_unregister_device(struct ib_device *device) } write_unlock_irqrestore(&device->client_data_lock, flags); up_write(&lists_rwsem); - - device->reg_state = IB_DEV_UNREGISTERED; } EXPORT_SYMBOL(ib_unregister_device); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 2e1f1e885ee502..76138ef4f839ea 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2558,12 +2558,6 @@ struct ib_device { struct kobject *ports_kobj; struct list_head port_list; - enum { - IB_DEV_UNINITIALIZED, - IB_DEV_REGISTERED, - IB_DEV_UNREGISTERED - } reg_state; - int uverbs_abi_ver; u64 uverbs_cmd_mask; u64 uverbs_ex_cmd_mask; From patchwork Thu Feb 7 05:41:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800391 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15DAD13B4 for ; Thu, 7 Feb 2019 05:42:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 05E302CBD8 for ; Thu, 7 Feb 2019 05:42:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EE8C92CBF1; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 922512CBDD for ; Thu, 7 Feb 2019 05:42:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726666AbfBGFmC (ORCPT ); Thu, 7 Feb 2019 00:42:02 -0500 Received: from mail-pf1-f196.google.com ([209.85.210.196]:44400 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726715AbfBGFmC (ORCPT ); Thu, 7 Feb 2019 00:42:02 -0500 Received: by mail-pf1-f196.google.com with SMTP id u6so4275981pfh.11 for ; Wed, 06 Feb 2019 21:42:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YVocA36PkEdry6rLhEdM8Qa+GHo5bScrJiHASI19axc=; b=LYgbEUNEhj3Xx/ElCLI/cB1RKT518O8HvhhewfTR07C/Xy3d4CZigCw2s5u6b/JpQq kAI0xNN9ijLfGErD5UpQ9shaQY9aj+vPlQY5vSuassWGkJvdjd9byIfCyk90ivSlyAWA oZnJXpmFv/OL6LqX7G43umqWUAQsEeI8ziATEqOGaHZ0C88jo1mk5A7Sy/Tp3ROxV+JX gFvYSmLF5s6cZjPUlsNjHEm5lc0Mh9DZGbJZcN6Ry0SjLtQHibCv+IAI1/EhmSCKVVkV qcE6/9zAcQuzPdnOoH0wH62jO0NU1nIZMvdQC0G9UY/bNmEfQfFv9G+5V3QynAfKwEXu H86g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YVocA36PkEdry6rLhEdM8Qa+GHo5bScrJiHASI19axc=; b=ss+spGEb4obzjGI9d4NI3DO8gstjrn08/yQydVKEZsoF4nHvHiHUwGs9MdN5l4gf38 vapvs0b+2q7GiIfq3neiUg9oituAomzXS4IXr6irIKLuBbGZ0wbidAQf1VuNDJqIrYSS hGhr7PAtabsJ/G9V8JkZ3DNbmPQt3J4UZUvYS18k4Lt3+LvMgaQ5w30LTnRb4Z8Ef7+X OOqvbPnz5gwAQx9QKhra9kDqBWafcGrk/X/NoS9E23PIP5Q0lcErUiQ81p1CzhgZjaIp OY9Lu4vmwZyAb/sB+FyrQ8tHUiJA5zhT7rDPD5QCZVkR7l6Vd12btutdey+GYKlx8qvz fGzw== X-Gm-Message-State: AHQUAubpEM5mOpc+RX8xsL7fFUXBjLE8cKsBx3VWk7JyMp0bnYe0h0QR kk9AGwv947Hkdgussc8lYow26/gomzs= X-Google-Smtp-Source: AHgI3IaRiSlb/Us9fzrqeF4Gs/dMw1i+qBt0aUb1rS+33SrNUjyFESHokgeayFkpYTfJh16HTVv4gw== X-Received: by 2002:a62:6f49:: with SMTP id k70mr14342421pfc.7.1549518121332; Wed, 06 Feb 2019 21:42:01 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id t24sm10385660pfh.21.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:41:59 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRi-0006Dh-5t; Wed, 06 Feb 2019 22:41:58 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH v2 5/8] RDMA/device: Use an ida instead of a free page in alloc_name Date: Wed, 6 Feb 2019 22:41:51 -0700 Message-Id: <20190207054154.23806-6-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe ida is the proper data structure to hold list of clustered small integers and then allocate an unused integer. Get rid of the convoluted and limited open-coded bitmap. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/device.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 905d7b3780beca..c16bff8cfba548 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -213,30 +213,36 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) static int alloc_name(struct ib_device *ibdev, const char *name) { - unsigned long *inuse; struct ib_device *device; + struct ida inuse; + int rc; int i; - inuse = (unsigned long *) get_zeroed_page(GFP_KERNEL); - if (!inuse) - return -ENOMEM; - + ida_init(&inuse); list_for_each_entry(device, &device_list, core_list) { char buf[IB_DEVICE_NAME_MAX]; if (sscanf(dev_name(&device->dev), name, &i) != 1) continue; - if (i < 0 || i >= PAGE_SIZE * 8) + if (i < 0 || i >= INT_MAX) continue; snprintf(buf, sizeof buf, name, i); - if (!strcmp(buf, dev_name(&device->dev))) - set_bit(i, inuse); + if (strcmp(buf, dev_name(&device->dev)) != 0) + continue; + + rc = ida_alloc_range(&inuse, i, i, GFP_KERNEL); + if (rc < 0) + goto out; } - i = find_first_zero_bit(inuse, PAGE_SIZE * 8); - free_page((unsigned long) inuse); + rc = ida_alloc(&inuse, GFP_KERNEL); + if (rc < 0) + goto out; - return dev_set_name(&ibdev->dev, name, i); + rc = dev_set_name(&ibdev->dev, name, rc); +out: + ida_destroy(&inuse); + return rc; } static void ib_device_release(struct device *device) From patchwork Thu Feb 7 05:41:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800393 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E4BB13B4 for ; Thu, 7 Feb 2019 05:42:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5DBEE2CBD8 for ; Thu, 7 Feb 2019 05:42:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 521722CBF6; Thu, 7 Feb 2019 05:42:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C69A32CBD8 for ; Thu, 7 Feb 2019 05:42:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726758AbfBGFmD (ORCPT ); Thu, 7 Feb 2019 00:42:03 -0500 Received: from mail-pf1-f196.google.com ([209.85.210.196]:38891 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726732AbfBGFmD (ORCPT ); Thu, 7 Feb 2019 00:42:03 -0500 Received: by mail-pf1-f196.google.com with SMTP id q1so4291886pfi.5 for ; Wed, 06 Feb 2019 21:42:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QAPQEbxAAfspU7zcGP0dJa8lOIfY9qZ5YGkg4+iDDvY=; b=ERr/w3ylsG6p7YMeq75nsVxji+FKrBRSL7d1djvn32SR7YK7Kg/c2/qtySfgypdy/o DIMm4Gv4JBscKn39b6rr08kdVYr5Fic42O7fnOBvHwD8wQ1AN71wPAsuGPH9o3mfGHwK UfrXV8wcU3dVmgHXTvRjnOEdroxeU3/bGhzmSoq7P0bN+D3UlSIpEIWgbmgnZpqBQmiL OyxaJm2dJ5rqLXC8jRwwXlzrPA7JAyEmPMRv3UIO+kkUsByRrU/fWBO5707PGPosBKs+ prUKGcGIMNWyvnKFyJNpdKlCxYtfaBa67q6MDLEl2gRQKtWuMlmmjMszVFmd6wfGrgqc wbag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QAPQEbxAAfspU7zcGP0dJa8lOIfY9qZ5YGkg4+iDDvY=; b=d3c3wyR8Hj0mJBDY8vmzXAHvl1GplO2sPBQ97BGdMtz++4e2sn/05tx1ms/t0SjA3N W37rjeMP5f74jUY9g7uQrxTL4jkmMwV72q5TGAwhYYXco6uUcLKd+Wn7VqldOSm2g2gY vhMOy1/ewjv2rCvFGCrPyrtdc4DguPkp/IXPXR/C5GibCzGA+2KceKUkwV96vJYRnTQI 9Cv13XpwbCZGUl66oCqFU6WhnXJs/SpGxqSZm62RGTVKF1+TI5hmqgG1GSILxo0liemM EFEHAkO1YkvUNFEIo8ZhoDzNlCo8aVOm5Y1b1pmyZu6euB8DcGcXztVSxnx38iH5B/zp H+Pg== X-Gm-Message-State: AHQUAubICbFx8cECcBYxShHIWnpHN9w+iA/qMQz136+H7dVXE895z8b6 NpfcZzJFWueB/9tLVlvwUhubWaw/U/I= X-Google-Smtp-Source: AHgI3IbiuikBD3sJXVL+PprtJqVoCvJpZOHIKZjtYjvUNj8f/OAnQT3+kTaiiA3mDNxMUqcxiX34wA== X-Received: by 2002:aa7:81d3:: with SMTP id c19mr946451pfn.230.1549518121831; Wed, 06 Feb 2019 21:42:01 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id n73sm11537819pfj.148.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:42:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRi-0006Dn-7R; Wed, 06 Feb 2019 22:41:58 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Parav Pandit Subject: [PATCH v2 6/8] RDMA/devices: Use xarray to store the clients Date: Wed, 6 Feb 2019 22:41:52 -0700 Message-Id: <20190207054154.23806-7-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This gives each client a unique ID and will let us move client_data to use xarray, and revise the locking scheme. clients have to be add/removed in strict FIFO/LIFO order as they interdepend. To support this the client_ids are assigned to increase in FIFO order. The existing linked list is kept to support reverse iteration until xarray can get a reverse iteration API. Signed-off-by: Jason Gunthorpe Reviewed-by: Parav Pandit --- drivers/infiniband/core/device.c | 50 ++++++++++++++++++++++++++++---- include/rdma/ib_verbs.h | 3 +- 2 files changed, 47 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index c16bff8cfba548..4b3e0d127cbdc7 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -65,15 +65,17 @@ struct workqueue_struct *ib_comp_unbound_wq; struct workqueue_struct *ib_wq; EXPORT_SYMBOL_GPL(ib_wq); -/* The device_list and client_list contain devices and clients after their +/* The device_list and clients contain devices and clients after their * registration has completed, and the devices and clients are removed * during unregistration. */ static LIST_HEAD(device_list); static LIST_HEAD(client_list); +#define CLIENT_REGISTERED XA_MARK_1 +static DEFINE_XARRAY_FLAGS(clients, XA_FLAGS_ALLOC); /* * device_mutex and lists_rwsem protect access to both device_list and - * client_list. device_mutex protects writer access by device and client + * clients. device_mutex protects writer access by device and client * registration / de-registration. lists_rwsem protects reader access to * these lists. Iterators of these lists must lock it for read, while updates * to the lists must be done with a write lock. A special case is when the @@ -563,6 +565,7 @@ int ib_register_device(struct ib_device *device, const char *name) { int ret; struct ib_client *client; + unsigned long index; setup_dma_device(device); @@ -607,7 +610,7 @@ int ib_register_device(struct ib_device *device, const char *name) refcount_set(&device->refcount, 1); - list_for_each_entry(client, &client_list, list) + xa_for_each_marked (&clients, index, client, CLIENT_REGISTERED) if (!add_client_context(device, client) && client->add) client->add(device); @@ -679,6 +682,32 @@ void ib_unregister_device(struct ib_device *device) } EXPORT_SYMBOL(ib_unregister_device); +static int assign_client_id(struct ib_client *client) +{ + int ret; + + /* + * The add/remove callbacks must be called in FIFO/LIFO order. To + * achieve this we assign client_ids so they are sorted in + * registration order, and retain a linked list we can reverse iterate + * to get the LIFO order. The extra linked list can go away if xarray + * learns to reverse iterate. + */ + if (list_empty(&client_list)) + client->client_id = 0; + else + client->client_id = + list_last_entry(&client_list, struct ib_client, list) + ->client_id; + ret = xa_alloc(&clients, &client->client_id, INT_MAX, client, + GFP_KERNEL); + if (ret) + goto out; + +out: + return ret; +} + /** * ib_register_client - Register an IB client * @client:Client to register @@ -695,15 +724,21 @@ EXPORT_SYMBOL(ib_unregister_device); int ib_register_client(struct ib_client *client) { struct ib_device *device; + int ret; mutex_lock(&device_mutex); + ret = assign_client_id(client); + if (ret) { + mutex_unlock(&device_mutex); + return ret; + } list_for_each_entry(device, &device_list, core_list) if (!add_client_context(device, client) && client->add) client->add(device); down_write(&lists_rwsem); - list_add_tail(&client->list, &client_list); + xa_set_mark(&clients, client->client_id, CLIENT_REGISTERED); up_write(&lists_rwsem); mutex_unlock(&device_mutex); @@ -728,7 +763,7 @@ void ib_unregister_client(struct ib_client *client) mutex_lock(&device_mutex); down_write(&lists_rwsem); - list_del(&client->list); + xa_clear_mark(&clients, client->client_id, CLIENT_REGISTERED); up_write(&lists_rwsem); list_for_each_entry(device, &device_list, core_list) { @@ -764,6 +799,10 @@ void ib_unregister_client(struct ib_client *client) kfree(found_context); } + down_write(&lists_rwsem); + list_del(&client->list); + xa_erase(&clients, client->client_id); + up_write(&lists_rwsem); mutex_unlock(&device_mutex); } EXPORT_SYMBOL(ib_unregister_client); @@ -1417,6 +1456,7 @@ static void __exit ib_core_cleanup(void) destroy_workqueue(ib_comp_wq); /* Make sure that any pending umem accounting work is done. */ destroy_workqueue(ib_wq); + WARN_ON(!xa_empty(&clients)); } MODULE_ALIAS_RDMA_NETLINK(RDMA_NL_LS, 4); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 76138ef4f839ea..84d7523984ee77 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2596,7 +2596,7 @@ struct ib_device { }; struct ib_client { - char *name; + const char *name; void (*add) (struct ib_device *); void (*remove)(struct ib_device *, void *client_data); @@ -2623,6 +2623,7 @@ struct ib_client { const struct sockaddr *addr, void *client_data); struct list_head list; + u32 client_id; /* kverbs are not required by the client */ u8 no_kverbs_req:1; From patchwork Thu Feb 7 05:41:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800397 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED38C1823 for ; Thu, 7 Feb 2019 05:42:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D9CF62CBD8 for ; Thu, 7 Feb 2019 05:42:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDF242CBF1; Thu, 7 Feb 2019 05:42:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9358C2CBD8 for ; Thu, 7 Feb 2019 05:42:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726715AbfBGFmE (ORCPT ); Thu, 7 Feb 2019 00:42:04 -0500 Received: from mail-pl1-f196.google.com ([209.85.214.196]:40246 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726755AbfBGFmD (ORCPT ); Thu, 7 Feb 2019 00:42:03 -0500 Received: by mail-pl1-f196.google.com with SMTP id u18so4286664plq.7 for ; Wed, 06 Feb 2019 21:42:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/t9Poz3D5wYl1qHn9YPbKe22pFPEYVdCAGEgM134WTk=; b=jTmmVg4NqAJLz8cEB7IyhqFjnsHPSxokRF+DZyCklFPza2+1L4mtXWIMubHLusttQn Pw6by3d0PMWF7Dy5qGapJe/vFctrgsu9KW9MqYuw3GuQSnptLNUaBX16vPH9Wgtx/kTL DEDigZblt0hCtNLGY3DIXfdo73s2abLOIuoD3fnDGDM81gs9YKMDDyZStBr4XfMd5J/2 A8LZ7PNTINEXH9po9gA4D2+mdSxkZLnbvf+tj6eXzIHiT6D44kkKGCqIMKe1/VqrEOWs u2S1seN0Q2+0EVveEqz5pepuJn9KCzV6OvqsFxAuubBizYjQ65fvhpzoU9duJgKYridF qOVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/t9Poz3D5wYl1qHn9YPbKe22pFPEYVdCAGEgM134WTk=; b=k4Ausw3iHZHD2r9zwdVvVhwqtkCNynr8FmBCLfFBTcMP9OGAXnxCcwAs0rUnnEWjND qX8KJEGjixTIbalW5pEnVoGlZczP2dcPuszW0+9REapn69HDwN5oINoUZRP93h8RB34i C7PHd0FrPAbLL6Egmk5q7ZF1f7L6gHCNuubPDkNJdm5YptLhRAKiuLHW1mDN8ESqpBDA lJWGlVPgzcBgut21DRsVqCpbw4APOCBGp+WS2wLoeoBQl2P9tc7c6ZeKU1ijeB7HXcon z25tKops8j+o6Fc9r1dNasvXA5PK3pNWddYEpLDBTLwQL1mlLOMkfUjpR6z9L+Wi+Jel D2bA== X-Gm-Message-State: AHQUAuYevDwt1uIBHp55cKeWlBtyv4W/lK9bEJrfje7pB8wkDWp3B1LQ MQpTd3Wa4F2kVebnakammHF1hdeGWV0= X-Google-Smtp-Source: AHgI3IZKla1lexgYckzkwk5Jpx7f/PUCr+bno00B/1sKK2UcbJpWHFTzkF2Fhp59W2Ak79zhBLLcmg== X-Received: by 2002:a17:902:930b:: with SMTP id bc11mr1684379plb.101.1549518122219; Wed, 06 Feb 2019 21:42:02 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id v13sm10404121pff.20.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:42:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRi-0006Dt-95; Wed, 06 Feb 2019 22:41:58 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH v2 7/8] RDMA/devices: Use xarray to store the client_data Date: Wed, 6 Feb 2019 22:41:53 -0700 Message-Id: <20190207054154.23806-8-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Now that we have a small ID for each client we can use xarray instead of linearly searching linked lists for client data. This will give much faster and scalable client data lookup, and will lets us revise the locking scheme. Since xarray can store 'going_down' using a mark just entirely eliminate the struct ib_client_data and directly store the client_data value in the xarray. However this does require a special iterator as we must still iterate over any NULL client_data values. Also eliminate the client_data_lock in favour of internal xarray locking. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/device.c | 348 +++++++++++++++---------------- include/rdma/ib_verbs.h | 23 +- 2 files changed, 186 insertions(+), 185 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 4b3e0d127cbdc7..f80e556d86bf01 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -51,30 +51,72 @@ MODULE_AUTHOR("Roland Dreier"); MODULE_DESCRIPTION("core kernel InfiniBand API"); MODULE_LICENSE("Dual BSD/GPL"); -struct ib_client_data { - struct list_head list; - struct ib_client *client; - void * data; - /* The device or client is going down. Do not call client or device - * callbacks other than remove(). */ - bool going_down; -}; - struct workqueue_struct *ib_comp_wq; struct workqueue_struct *ib_comp_unbound_wq; struct workqueue_struct *ib_wq; EXPORT_SYMBOL_GPL(ib_wq); -/* The device_list and clients contain devices and clients after their - * registration has completed, and the devices and clients are removed - * during unregistration. */ -static LIST_HEAD(device_list); +/* + * devices contains devices that have had their names assigned. The + * devices may not be registered. Users that care about the registration + * status need to call ib_device_try_get() on the device to ensure it is + * registered, and keep it registered, for the required duration. + * + */ +static DEFINE_XARRAY_FLAGS(devices, XA_FLAGS_ALLOC); + +/* + * Note that if the *rwsem is held and the *_REGISTERED mark is seen then the + * object is guaranteed to be and remain registered for the duration of the + * lock. + */ +#define DEVICE_REGISTERED XA_MARK_1 + static LIST_HEAD(client_list); #define CLIENT_REGISTERED XA_MARK_1 static DEFINE_XARRAY_FLAGS(clients, XA_FLAGS_ALLOC); /* - * device_mutex and lists_rwsem protect access to both device_list and + * If client_data is registered then the corresponding client must also still + * be registered. + */ +#define CLIENT_DATA_REGISTERED XA_MARK_1 +/* + * xarray has this behavior where it won't iterate over NULL values stored in + * allocated arrays. So we need our own iterator to see all values stored in + * the array. This does the same thing as xa_for_each except that it also + * returns NULL valued entries if the array is allocating. Simplified to only + * work on simple xarrays. + */ +static void *xan_find_marked(struct xarray *xa, unsigned long *indexp, + xa_mark_t filter) +{ + XA_STATE(xas, xa, *indexp); + void *entry; + + rcu_read_lock(); + do { + entry = xas_find_marked(&xas, ULONG_MAX, filter); + if (xa_is_zero(entry)) + break; + } while (xas_retry(&xas, entry)); + rcu_read_unlock(); + + if (entry) { + *indexp = xas.xa_index; + if (xa_is_zero(entry)) + return NULL; + return entry; + } + return XA_ERROR(-ENOENT); +} +#define xan_for_each_marked(xa, index, entry, filter) \ + for (index = 0, entry = xan_find_marked(xa, &(index), filter); \ + !xa_is_err(entry); \ + (index)++, entry = xan_find_marked(xa, &(index), filter)) + +/* + * device_mutex and lists_rwsem protect access to both devices and * clients. device_mutex protects writer access by device and client * registration / de-registration. lists_rwsem protects reader access to * these lists. Iterators of these lists must lock it for read, while updates @@ -135,17 +177,6 @@ static int ib_device_check_mandatory(struct ib_device *device) return 0; } -static struct ib_device *__ib_device_get_by_index(u32 index) -{ - struct ib_device *device; - - list_for_each_entry(device, &device_list, core_list) - if (device->index == index) - return device; - - return NULL; -} - /* * Caller must perform ib_device_put() to return the device reference count * when ib_device_get_by_index() returns valid device pointer. @@ -155,7 +186,7 @@ struct ib_device *ib_device_get_by_index(u32 index) struct ib_device *device; down_read(&lists_rwsem); - device = __ib_device_get_by_index(index); + device = xa_load(&devices, index); if (device) { if (!ib_device_try_get(device)) device = NULL; @@ -181,8 +212,9 @@ EXPORT_SYMBOL(ib_device_put); static struct ib_device *__ib_device_get_by_name(const char *name) { struct ib_device *device; + unsigned long index; - list_for_each_entry(device, &device_list, core_list) + xa_for_each (&devices, index, device) if (!strcmp(name, dev_name(&device->dev))) return device; @@ -216,12 +248,13 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) static int alloc_name(struct ib_device *ibdev, const char *name) { struct ib_device *device; + unsigned long index; struct ida inuse; int rc; int i; ida_init(&inuse); - list_for_each_entry(device, &device_list, core_list) { + xa_for_each (&devices, index, device) { char buf[IB_DEVICE_NAME_MAX]; if (sscanf(dev_name(&device->dev), name, &i) != 1) @@ -256,6 +289,7 @@ static void ib_device_release(struct device *device) ib_security_release_port_pkey_list(dev); kfree(dev->port_pkey_list); kfree(dev->port_immutable); + xa_destroy(&dev->client_data); kfree(dev); } @@ -306,8 +340,11 @@ struct ib_device *_ib_alloc_device(size_t size) INIT_LIST_HEAD(&device->event_handler_list); spin_lock_init(&device->event_handler_lock); - rwlock_init(&device->client_data_lock); - INIT_LIST_HEAD(&device->client_data_list); + /* + * client_data needs to be alloc because we don't want our mark to be + * destroyed if the user stores NULL in the client data. + */ + xa_init_flags(&device->client_data, XA_FLAGS_ALLOC); INIT_LIST_HEAD(&device->port_list); init_completion(&device->unreg_completion); @@ -323,7 +360,7 @@ EXPORT_SYMBOL(_ib_alloc_device); */ void ib_dealloc_device(struct ib_device *device) { - WARN_ON(!list_empty(&device->client_data_list)); + WARN_ON(!xa_empty(&device->client_data)); WARN_ON(refcount_read(&device->refcount)); rdma_restrack_clean(device); put_device(&device->dev); @@ -332,26 +369,20 @@ EXPORT_SYMBOL(ib_dealloc_device); static int add_client_context(struct ib_device *device, struct ib_client *client) { - struct ib_client_data *context; + void *entry; if (!device->kverbs_provider && !client->no_kverbs_req) return -EOPNOTSUPP; - context = kmalloc(sizeof(*context), GFP_KERNEL); - if (!context) - return -ENOMEM; - - context->client = client; - context->data = NULL; - context->going_down = false; - down_write(&lists_rwsem); - write_lock_irq(&device->client_data_lock); - list_add(&context->list, &device->client_data_list); - write_unlock_irq(&device->client_data_lock); + entry = xa_store(&device->client_data, client->client_id, NULL, + GFP_KERNEL); + if (!xa_is_err(entry)) + xa_set_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED); up_write(&lists_rwsem); - return 0; + return xa_err(entry); } static int verify_immutable(const struct ib_device *dev, u8 port) @@ -428,9 +459,10 @@ static int setup_port_pkey_list(struct ib_device *device) static void ib_policy_change_task(struct work_struct *work) { struct ib_device *dev; + unsigned long index; down_read(&lists_rwsem); - list_for_each_entry(dev, &device_list, core_list) { + xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { int i; for (i = rdma_start_port(dev); i <= rdma_end_port(dev); i++) { @@ -460,28 +492,48 @@ static int ib_security_change(struct notifier_block *nb, unsigned long event, return NOTIFY_OK; } -/** - * __dev_new_index - allocate an device index - * - * Returns a suitable unique value for a new device interface - * number. It assumes that there are less than 2^32-1 ib devices - * will be present in the system. +/* + * Assign the unique string device name and the unique device index. */ -static u32 __dev_new_index(void) +static int assign_name(struct ib_device *device, const char *name) { - /* - * The device index to allow stable naming. - * Similar to struct net -> ifindex. - */ - static u32 index; + static u32 last_id; + int ret; - for (;;) { - if (!(++index)) - index = 1; + /* Assign a unique name to the device */ + if (strchr(name, '%')) + ret = alloc_name(device, name); + else + ret = dev_set_name(&device->dev, name); + if (ret) + goto out; + + if (__ib_device_get_by_name(dev_name(&device->dev))) { + ret = -ENFILE; + goto out; + } + strlcpy(device->name, dev_name(&device->dev), IB_DEVICE_NAME_MAX); - if (!__ib_device_get_by_index(index)) - return index; + /* Cyclically allocate a user visible ID for the device */ + device->index = last_id; + ret = xa_alloc(&devices, &device->index, INT_MAX, device, GFP_KERNEL); + if (ret == -ENOSPC) { + device->index = 0; + ret = xa_alloc(&devices, &device->index, INT_MAX, device, + GFP_KERNEL); } + if (ret) + goto out; + last_id = device->index + 1; + + ret = 0; +out: + return ret; +} + +static void release_name(struct ib_device *device) +{ + xa_erase(&devices, device->index); } static void setup_dma_device(struct ib_device *device) @@ -571,34 +623,21 @@ int ib_register_device(struct ib_device *device, const char *name) mutex_lock(&device_mutex); - if (strchr(name, '%')) { - ret = alloc_name(device, name); - if (ret) - goto out; - } else { - ret = dev_set_name(&device->dev, name); - if (ret) - goto out; - } - if (__ib_device_get_by_name(dev_name(&device->dev))) { - ret = -ENFILE; + ret = assign_name(device, name); + if (ret) goto out; - } - strlcpy(device->name, dev_name(&device->dev), IB_DEVICE_NAME_MAX); ret = setup_device(device); if (ret) - goto out; + goto out_name; ret = ib_cache_setup_one(device); if (ret) { dev_warn(&device->dev, "Couldn't set up InfiniBand P_Key/GID cache\n"); - goto out; + goto out_name; } - device->index = __dev_new_index(); - ib_device_register_rdmacg(device); ret = ib_device_register_sysfs(device); @@ -615,7 +654,7 @@ int ib_register_device(struct ib_device *device, const char *name) client->add(device); down_write(&lists_rwsem); - list_add_tail(&device->core_list, &device_list); + xa_set_mark(&devices, device->index, DEVICE_REGISTERED); up_write(&lists_rwsem); mutex_unlock(&device_mutex); return 0; @@ -623,6 +662,8 @@ int ib_register_device(struct ib_device *device, const char *name) cg_cleanup: ib_device_unregister_rdmacg(device); ib_cache_cleanup_one(device); +out_name: + release_name(device); out: mutex_unlock(&device_mutex); return ret; @@ -637,8 +678,8 @@ EXPORT_SYMBOL(ib_register_device); */ void ib_unregister_device(struct ib_device *device) { - struct ib_client_data *context, *tmp; - unsigned long flags; + struct ib_client *client; + unsigned long index; /* * Wait for all netlink command callers to finish working on the @@ -650,34 +691,31 @@ void ib_unregister_device(struct ib_device *device) mutex_lock(&device_mutex); down_write(&lists_rwsem); - list_del(&device->core_list); - write_lock_irq(&device->client_data_lock); - list_for_each_entry(context, &device->client_data_list, list) - context->going_down = true; - write_unlock_irq(&device->client_data_lock); + xa_clear_mark(&devices, device->index, DEVICE_REGISTERED); + xa_for_each (&clients, index, client) + xa_clear_mark(&device->client_data, index, + CLIENT_DATA_REGISTERED); downgrade_write(&lists_rwsem); - list_for_each_entry(context, &device->client_data_list, list) { - if (context->client->remove) - context->client->remove(device, context->data); - } + list_for_each_entry_reverse(client, &client_list, list) + if (xa_get_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED) && + client->remove) + client->remove(device, xa_load(&device->client_data, + client->client_id)); up_read(&lists_rwsem); ib_device_unregister_sysfs(device); ib_device_unregister_rdmacg(device); + release_name(device); + mutex_unlock(&device_mutex); ib_cache_cleanup_one(device); down_write(&lists_rwsem); - write_lock_irqsave(&device->client_data_lock, flags); - list_for_each_entry_safe(context, tmp, &device->client_data_list, - list) { - list_del(&context->list); - kfree(context); - } - write_unlock_irqrestore(&device->client_data_lock, flags); + xa_destroy(&device->client_data); up_write(&lists_rwsem); } EXPORT_SYMBOL(ib_unregister_device); @@ -724,6 +762,7 @@ static int assign_client_id(struct ib_client *client) int ib_register_client(struct ib_client *client) { struct ib_device *device; + unsigned long index; int ret; mutex_lock(&device_mutex); @@ -733,7 +772,7 @@ int ib_register_client(struct ib_client *client) return ret; } - list_for_each_entry(device, &device_list, core_list) + xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) if (!add_client_context(device, client) && client->add) client->add(device); @@ -757,8 +796,8 @@ EXPORT_SYMBOL(ib_register_client); */ void ib_unregister_client(struct ib_client *client) { - struct ib_client_data *context; struct ib_device *device; + unsigned long index; mutex_lock(&device_mutex); @@ -766,37 +805,19 @@ void ib_unregister_client(struct ib_client *client) xa_clear_mark(&clients, client->client_id, CLIENT_REGISTERED); up_write(&lists_rwsem); - list_for_each_entry(device, &device_list, core_list) { - struct ib_client_data *found_context = NULL; - + xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) { down_write(&lists_rwsem); - write_lock_irq(&device->client_data_lock); - list_for_each_entry(context, &device->client_data_list, list) - if (context->client == client) { - context->going_down = true; - found_context = context; - break; - } - write_unlock_irq(&device->client_data_lock); + xa_clear_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED); up_write(&lists_rwsem); if (client->remove) - client->remove(device, found_context ? - found_context->data : NULL); - - if (!found_context) { - dev_warn(&device->dev, - "No client context found for %s\n", - client->name); - continue; - } + client->remove(device, xa_load(&device->client_data, + client->client_id)); down_write(&lists_rwsem); - write_lock_irq(&device->client_data_lock); - list_del(&found_context->list); - write_unlock_irq(&device->client_data_lock); + xa_erase(&device->client_data, client->client_id); up_write(&lists_rwsem); - kfree(found_context); } down_write(&lists_rwsem); @@ -807,59 +828,28 @@ void ib_unregister_client(struct ib_client *client) } EXPORT_SYMBOL(ib_unregister_client); -/** - * ib_get_client_data - Get IB client context - * @device:Device to get context for - * @client:Client to get context for - * - * ib_get_client_data() returns client context set with - * ib_set_client_data(). - */ -void *ib_get_client_data(struct ib_device *device, struct ib_client *client) -{ - struct ib_client_data *context; - void *ret = NULL; - unsigned long flags; - - read_lock_irqsave(&device->client_data_lock, flags); - list_for_each_entry(context, &device->client_data_list, list) - if (context->client == client) { - ret = context->data; - break; - } - read_unlock_irqrestore(&device->client_data_lock, flags); - - return ret; -} -EXPORT_SYMBOL(ib_get_client_data); - /** * ib_set_client_data - Set IB client context * @device:Device to set context for * @client:Client to set context for * @data:Context to set * - * ib_set_client_data() sets client context that can be retrieved with - * ib_get_client_data(). + * ib_set_client_data() sets client context data that can be retrieved with + * ib_get_client_data(). This can only be called while the client is + * registered to the device, once the ib_client remove() callback returns this + * cannot be called. */ void ib_set_client_data(struct ib_device *device, struct ib_client *client, void *data) { - struct ib_client_data *context; - unsigned long flags; - - write_lock_irqsave(&device->client_data_lock, flags); - list_for_each_entry(context, &device->client_data_list, list) - if (context->client == client) { - context->data = data; - goto out; - } + void *rc; - dev_warn(&device->dev, "No client context found for %s\n", - client->name); + if (WARN_ON(IS_ERR(data))) + data = NULL; -out: - write_unlock_irqrestore(&device->client_data_lock, flags); + rc = xa_store(&device->client_data, client->client_id, data, + GFP_KERNEL); + WARN_ON(xa_is_err(rc)); } EXPORT_SYMBOL(ib_set_client_data); @@ -1017,9 +1007,10 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, void *cookie) { struct ib_device *dev; + unsigned long index; down_read(&lists_rwsem); - list_for_each_entry(dev, &device_list, core_list) + xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) ib_enum_roce_netdev(dev, filter, filter_cookie, cb, cookie); up_read(&lists_rwsem); } @@ -1033,12 +1024,13 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, int ib_enum_all_devs(nldev_callback nldev_cb, struct sk_buff *skb, struct netlink_callback *cb) { + unsigned long index; struct ib_device *dev; unsigned int idx = 0; int ret = 0; down_read(&lists_rwsem); - list_for_each_entry(dev, &device_list, core_list) { + xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { ret = nldev_cb(dev, skb, cb, idx); if (ret) break; @@ -1211,26 +1203,25 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, const struct sockaddr *addr) { struct net_device *net_dev = NULL; - struct ib_client_data *context; + unsigned long index; + void *client_data; if (!rdma_protocol_ib(dev, port)) return NULL; down_read(&lists_rwsem); - list_for_each_entry(context, &dev->client_data_list, list) { - struct ib_client *client = context->client; + xan_for_each_marked (&dev->client_data, index, client_data, + CLIENT_DATA_REGISTERED) { + struct ib_client *client = xa_load(&clients, index); - if (context->going_down) + if (!client || !client->get_net_dev_by_params) continue; - if (client->get_net_dev_by_params) { - net_dev = client->get_net_dev_by_params(dev, port, pkey, - gid, addr, - context->data); - if (net_dev) - break; - } + net_dev = client->get_net_dev_by_params(dev, port, pkey, gid, + addr, client_data); + if (net_dev) + break; } up_read(&lists_rwsem); @@ -1457,6 +1448,7 @@ static void __exit ib_core_cleanup(void) /* Make sure that any pending umem accounting work is done. */ destroy_workqueue(ib_wq); WARN_ON(!xa_empty(&clients)); + WARN_ON(!xa_empty(&devices)); } MODULE_ALIAS_RDMA_NETLINK(RDMA_NL_LS, 4); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 84d7523984ee77..7781eefd8b84eb 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2528,12 +2528,7 @@ struct ib_device { struct list_head event_handler_list; spinlock_t event_handler_lock; - rwlock_t client_data_lock; - struct list_head core_list; - /* Access to the client_data_list is protected by the client_data_lock - * rwlock and the lists_rwsem read-write semaphore - */ - struct list_head client_data_list; + struct xarray client_data; struct ib_cache cache; /** @@ -2646,7 +2641,21 @@ void ib_unregister_device(struct ib_device *device); int ib_register_client (struct ib_client *client); void ib_unregister_client(struct ib_client *client); -void *ib_get_client_data(struct ib_device *device, struct ib_client *client); +/** + * ib_get_client_data - Get IB client context + * @device:Device to get context for + * @client:Client to get context for + * + * ib_get_client_data() returns the client context data set with + * ib_set_client_data(). This can only be called while the client is + * registered to the device, once the ib_client remove() callback returns this + * cannot be called. + */ +static inline void *ib_get_client_data(struct ib_device *device, + struct ib_client *client) +{ + return xa_load(&device->client_data, client->client_id); +} void ib_set_client_data(struct ib_device *device, struct ib_client *client, void *data); void ib_set_device_ops(struct ib_device *device, From patchwork Thu Feb 7 05:41:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800395 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A24E922 for ; Thu, 7 Feb 2019 05:42:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 887332CBDD for ; Thu, 7 Feb 2019 05:42:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D0572CBF6; Thu, 7 Feb 2019 05:42:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2E43E2CBDD for ; Thu, 7 Feb 2019 05:42:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726732AbfBGFmE (ORCPT ); Thu, 7 Feb 2019 00:42:04 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:39452 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726715AbfBGFmE (ORCPT ); Thu, 7 Feb 2019 00:42:04 -0500 Received: by mail-pl1-f194.google.com with SMTP id 101so4292613pld.6 for ; Wed, 06 Feb 2019 21:42:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4dwG3o/HMpizIBQ/Ajwc9PxT+BBD0r4m39/jqq05umw=; b=l0iy3C7wLavVuoYogvXjQbPB7egBcGW+qtUKQIqKoLUsHq9YLvrhipXvIW0HBQyqpK bYYEG/21KvBT+WkEDjMKXzsHy45sVRj/FRZ/utdIElObDkevYFsZPpFmpt6xJ/CGdTkx WheqxohPNg+jahpszM7ck1t9bb/pPLIrCqlPDwBzgnSYapYLtqrDNVBpZxGzKDoOFGnF N8Y1IoDS/j16dDou7XnNFm3k1115LgSTaG4vYpGVlP0Juwn3o6SGgigrV5ZjTA7+cdWE CmKc3LwgBr/F3rLK8JO3tcgtFq5K0NxLCPdECWcPqB3ztio2mGr4x2ctyzKD0gFgPEG5 h7Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4dwG3o/HMpizIBQ/Ajwc9PxT+BBD0r4m39/jqq05umw=; b=OaWWGlV0eSho4T4tsYD/BbqDmSEnyq3VubZFlqgBEIywrkjBx7StC1Hb0pR0kiXkjQ 9A2XIQqNXFmgLzLU7Oqy0S+e0WdcZkJeTSKSml9aTiFCWaqWwAt6QyKWp8D2GZFUGpQ6 XTgkZUvuVbpdSoMcQYmmhts/PHYj3sOZlef4CRiwvFwBz3BUs4MfpUBOI++DGRy3zWX9 1M/dCFmPTbYBnm7qgG3jRjee+rxVY8cCowq5pyqxCVSt21/4od3Bwe2mCfXK6z9roBR7 Kn2ihPIBM9Mv5b8iC5MRZNvzdhTUPW2U0iQxkp6F9sCfi2NTZCn8jShSEpltHnV9VvzW IpiQ== X-Gm-Message-State: AHQUAuY3tvnwxazSd2sz6GScFuA/7B8c2ZLTnAU6yqjTmQIAPOmMGB2j gAO/PJmRFGpgNGmertIeS6KEav7CEAw= X-Google-Smtp-Source: AHgI3Iba0YMpk9V9/P5/xL3GfFEM0XYfG0IA3lNH7LpP9AjeuAhlLl44nD79514gSNKx/f473RQegw== X-Received: by 2002:a17:902:c85:: with SMTP id 5mr14872070plt.339.1549518122724; Wed, 06 Feb 2019 21:42:02 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id j21sm12020660pfn.175.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:42:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRi-0006Dz-AM; Wed, 06 Feb 2019 22:41:58 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH v2 8/8] RDMA/devices: Re-organize device.c locking Date: Wed, 6 Feb 2019 22:41:54 -0700 Message-Id: <20190207054154.23806-9-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190207054154.23806-1-jgg@ziepe.ca> References: <20190207054154.23806-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe The locking here started out with a single lock that covered everything and then has lately veered into crazy town. The fundamental problem is that several places need to iterate over a linked list, but also need to drop their locks to avoid deadlock during client callbacks. xarray's restartable iteration offers a simple solution to the problem. Once all the lists are xarrays we can drop locks in the places that need that and rely on xarray to provide consistency and locking for the data structure. The resulting simplification is that each of the three lists has a dedicated rwsem that must be held when working with the list it covers. One data structure is no longer covered by multiple locks. The sleeping semaphore is selected because the read side generally needs to be held over something sleeping, and using RCU reader locking in those cases is overkill. In the process this simplifies the entire registration/unregistration flow to be the expected list of setups and the reversed list of matching teardowns, and the registration lock 'refcount' can now be revised to be released after the ULPs are removed, providing a very sane semantic for this feature. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/device.c | 361 +++++++++++++++++++------------ include/rdma/ib_verbs.h | 1 + 2 files changed, 222 insertions(+), 140 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index f80e556d86bf01..928bb9c2f6b88c 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -37,7 +37,6 @@ #include #include #include -#include #include #include #include @@ -56,6 +55,29 @@ struct workqueue_struct *ib_comp_unbound_wq; struct workqueue_struct *ib_wq; EXPORT_SYMBOL_GPL(ib_wq); +/* + * Each of the three rwsem locks (devices, clients, client_data) protects the + * xarray of the same name. Specifically it allows the caller to assert that + * the MARK will/will not be changing under the lock, and for devices and + * clients, that the value in the xarray is still a valid pointer. Change of + * the MARK is linked to the object state, so holding the lock and testing the + * MARK also asserts that the contained object is in a certain state. + * + * This is used to build a two stage register/unregister flow where objects + * can continue to be in the xarray even though they are still in progress to + * register/unregister. + * + * The xarray itself provides additional locking, and restartable iteration, + * which is also relied on. + * + * Locks should not be nested, with the exception of client_data, which is + * allowed to nest under the read side of the other two locks. + * + * The devices_rwsem also protects the device name list, any change or + * assignment of device name must also hold the write side to guarantee unique + * names. + */ + /* * devices contains devices that have had their names assigned. The * devices may not be registered. Users that care about the registration @@ -64,17 +86,13 @@ EXPORT_SYMBOL_GPL(ib_wq); * */ static DEFINE_XARRAY_FLAGS(devices, XA_FLAGS_ALLOC); - -/* - * Note that if the *rwsem is held and the *_REGISTERED mark is seen then the - * object is guaranteed to be and remain registered for the duration of the - * lock. - */ +static DECLARE_RWSEM(devices_rwsem); #define DEVICE_REGISTERED XA_MARK_1 static LIST_HEAD(client_list); #define CLIENT_REGISTERED XA_MARK_1 static DEFINE_XARRAY_FLAGS(clients, XA_FLAGS_ALLOC); +static DECLARE_RWSEM(clients_rwsem); /* * If client_data is registered then the corresponding client must also still @@ -115,20 +133,6 @@ static void *xan_find_marked(struct xarray *xa, unsigned long *indexp, !xa_is_err(entry); \ (index)++, entry = xan_find_marked(xa, &(index), filter)) -/* - * device_mutex and lists_rwsem protect access to both devices and - * clients. device_mutex protects writer access by device and client - * registration / de-registration. lists_rwsem protects reader access to - * these lists. Iterators of these lists must lock it for read, while updates - * to the lists must be done with a write lock. A special case is when the - * device_mutex is locked. In this case locking the lists for read access is - * not necessary as the device_mutex implies it. - * - * lists_rwsem also protects access to the client data list. - */ -static DEFINE_MUTEX(device_mutex); -static DECLARE_RWSEM(lists_rwsem); - static int ib_security_change(struct notifier_block *nb, unsigned long event, void *lsm_data); static void ib_policy_change_task(struct work_struct *work); @@ -185,13 +189,13 @@ struct ib_device *ib_device_get_by_index(u32 index) { struct ib_device *device; - down_read(&lists_rwsem); + down_read(&devices_rwsem); device = xa_load(&devices, index); if (device) { if (!ib_device_try_get(device)) device = NULL; } - up_read(&lists_rwsem); + up_read(&devices_rwsem); return device; } @@ -225,7 +229,7 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) { int ret; - mutex_lock(&device_mutex); + down_write(&devices_rwsem); if (!strcmp(name, dev_name(&ibdev->dev))) { ret = 0; goto out; @@ -241,7 +245,7 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) goto out; strlcpy(ibdev->name, name, IB_DEVICE_NAME_MAX); out: - mutex_unlock(&device_mutex); + up_write(&devices_rwsem); return ret; } @@ -253,6 +257,7 @@ static int alloc_name(struct ib_device *ibdev, const char *name) int rc; int i; + lockdep_assert_held_exclusive(&devices_rwsem); ida_init(&inuse); xa_for_each (&devices, index, device) { char buf[IB_DEVICE_NAME_MAX]; @@ -345,6 +350,7 @@ struct ib_device *_ib_alloc_device(size_t size) * destroyed if the user stores NULL in the client data. */ xa_init_flags(&device->client_data, XA_FLAGS_ALLOC); + init_rwsem(&device->client_data_rwsem); INIT_LIST_HEAD(&device->port_list); init_completion(&device->unreg_completion); @@ -367,22 +373,86 @@ void ib_dealloc_device(struct ib_device *device) } EXPORT_SYMBOL(ib_dealloc_device); -static int add_client_context(struct ib_device *device, struct ib_client *client) +/* + * add_client_context() and remove_client_context() must be safe against + * parallel calls on the same device - registration/unregistration of both the + * device and client can be occurring in parallel. + * + * The routines need to be a fence, any caller must not return until the add + * or remove is fully completed. + */ +static int add_client_context(struct ib_device *device, + struct ib_client *client) { - void *entry; + int ret = 0; if (!device->kverbs_provider && !client->no_kverbs_req) - return -EOPNOTSUPP; + return 0; + + down_write(&device->client_data_rwsem); + /* + * Another caller to add_client_context got here first and has already + * completely initialized context. + */ + if (xa_get_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED)) + goto out; + + ret = xa_err(xa_store(&device->client_data, client->client_id, NULL, + GFP_KERNEL)); + if (ret) + goto out; + downgrade_write(&device->client_data_rwsem); + if (client->add) + client->add(device); + + /* Readers shall not see a client until add has been completed */ + xa_set_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED); + up_read(&device->client_data_rwsem); + return 0; + +out: + up_write(&device->client_data_rwsem); + return ret; +} + +static void remove_client_context(struct ib_device *device, + unsigned int client_id) +{ + struct ib_client *client; + void *client_data; - down_write(&lists_rwsem); - entry = xa_store(&device->client_data, client->client_id, NULL, - GFP_KERNEL); - if (!xa_is_err(entry)) - xa_set_mark(&device->client_data, client->client_id, - CLIENT_DATA_REGISTERED); - up_write(&lists_rwsem); + down_write(&device->client_data_rwsem); + if (!xa_get_mark(&device->client_data, client_id, + CLIENT_DATA_REGISTERED)) { + up_write(&device->client_data_rwsem); + return; + } + client_data = xa_load(&device->client_data, client_id); + xa_clear_mark(&device->client_data, client_id, CLIENT_DATA_REGISTERED); + client = xa_load(&clients, client_id); + downgrade_write(&device->client_data_rwsem); - return xa_err(entry); + /* + * Notice we cannot be holding any exclusive locks when calling the + * remove callback as the remove callback can recurse back into any + * public functions in this module and thus try for any locks those + * functions take. + * + * For this reason clients and drivers should not call the + * unregistration functions will holdling any locks. + * + * It tempting to drop the client_data_rwsem too, but this is required + * to ensure that unregister_client does not return until all clients + * are completely unregistered, which is required to avoid module + * unloading races. + */ + if (client->remove) + client->remove(device, client_data); + + xa_erase(&device->client_data, client_id); + up_read(&device->client_data_rwsem); } static int verify_immutable(const struct ib_device *dev, u8 port) @@ -461,7 +531,7 @@ static void ib_policy_change_task(struct work_struct *work) struct ib_device *dev; unsigned long index; - down_read(&lists_rwsem); + down_read(&devices_rwsem); xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { int i; @@ -478,7 +548,7 @@ static void ib_policy_change_task(struct work_struct *work) ib_security_cache_change(dev, i, sp); } } - up_read(&lists_rwsem); + up_read(&devices_rwsem); } static int ib_security_change(struct notifier_block *nb, unsigned long event, @@ -500,6 +570,7 @@ static int assign_name(struct ib_device *device, const char *name) static u32 last_id; int ret; + down_write(&devices_rwsem); /* Assign a unique name to the device */ if (strchr(name, '%')) ret = alloc_name(device, name); @@ -527,13 +598,17 @@ static int assign_name(struct ib_device *device, const char *name) last_id = device->index + 1; ret = 0; + out: + up_write(&devices_rwsem); return ret; } static void release_name(struct ib_device *device) { + down_write(&devices_rwsem); xa_erase(&devices, device->index); + up_write(&devices_rwsem); } static void setup_dma_device(struct ib_device *device) @@ -571,11 +646,18 @@ static void setup_dma_device(struct ib_device *device) } } +/* + * setup_device() allocates memory and sets up data that requires calling the + * device ops, this is the only reason these actions are not done during + * ib_alloc_device. It is undone by ib_dealloc_device(). + */ static int setup_device(struct ib_device *device) { struct ib_udata uhw = {.outlen = 0, .inlen = 0}; int ret; + setup_dma_device(device); + ret = ib_device_check_mandatory(device); if (ret) return ret; @@ -604,6 +686,54 @@ static int setup_device(struct ib_device *device) return 0; } +static void disable_device(struct ib_device *device) +{ + struct ib_client *client; + + WARN_ON(!refcount_read(&device->refcount)); + + down_write(&devices_rwsem); + xa_clear_mark(&devices, device->index, DEVICE_REGISTERED); + up_write(&devices_rwsem); + + down_read(&clients_rwsem); + list_for_each_entry_reverse(client, &client_list, list) + remove_client_context(device, client->client_id); + up_read(&clients_rwsem); + + /* Pairs with refcount_set in enable_device */ + ib_device_put(device); + wait_for_completion(&device->unreg_completion); +} + +/* + * An enabled device is visible to all clients and to all the public facing + * APIs that return a device pointer. + */ +static int enable_device(struct ib_device *device) +{ + struct ib_client *client; + unsigned long index; + int ret; + + refcount_set(&device->refcount, 1); + down_write(&devices_rwsem); + xa_set_mark(&devices, device->index, DEVICE_REGISTERED); + up_write(&devices_rwsem); + + down_read(&clients_rwsem); + xa_for_each_marked (&clients, index, client, CLIENT_REGISTERED) { + ret = add_client_context(device, client); + if (ret) { + up_read(&clients_rwsem); + disable_device(device); + return ret; + } + } + up_read(&clients_rwsem); + return 0; +} + /** * ib_register_device - Register an IB device with IB core * @device:Device to register @@ -616,26 +746,20 @@ static int setup_device(struct ib_device *device) int ib_register_device(struct ib_device *device, const char *name) { int ret; - struct ib_client *client; - unsigned long index; - - setup_dma_device(device); - - mutex_lock(&device_mutex); ret = assign_name(device, name); if (ret) - goto out; + return ret; ret = setup_device(device); if (ret) - goto out_name; + goto out; ret = ib_cache_setup_one(device); if (ret) { dev_warn(&device->dev, "Couldn't set up InfiniBand P_Key/GID cache\n"); - goto out_name; + goto out; } ib_device_register_rdmacg(device); @@ -647,25 +771,19 @@ int ib_register_device(struct ib_device *device, const char *name) goto cg_cleanup; } - refcount_set(&device->refcount, 1); - - xa_for_each_marked (&clients, index, client, CLIENT_REGISTERED) - if (!add_client_context(device, client) && client->add) - client->add(device); + ret = enable_device(device); + if (ret) + goto sysfs_cleanup; - down_write(&lists_rwsem); - xa_set_mark(&devices, device->index, DEVICE_REGISTERED); - up_write(&lists_rwsem); - mutex_unlock(&device_mutex); return 0; +sysfs_cleanup: + ib_device_unregister_sysfs(device); cg_cleanup: ib_device_unregister_rdmacg(device); ib_cache_cleanup_one(device); -out_name: - release_name(device); out: - mutex_unlock(&device_mutex); + release_name(device); return ret; } EXPORT_SYMBOL(ib_register_device); @@ -678,45 +796,11 @@ EXPORT_SYMBOL(ib_register_device); */ void ib_unregister_device(struct ib_device *device) { - struct ib_client *client; - unsigned long index; - - /* - * Wait for all netlink command callers to finish working on the - * device. - */ - ib_device_put(device); - wait_for_completion(&device->unreg_completion); - - mutex_lock(&device_mutex); - - down_write(&lists_rwsem); - xa_clear_mark(&devices, device->index, DEVICE_REGISTERED); - xa_for_each (&clients, index, client) - xa_clear_mark(&device->client_data, index, - CLIENT_DATA_REGISTERED); - downgrade_write(&lists_rwsem); - - list_for_each_entry_reverse(client, &client_list, list) - if (xa_get_mark(&device->client_data, client->client_id, - CLIENT_DATA_REGISTERED) && - client->remove) - client->remove(device, xa_load(&device->client_data, - client->client_id)); - up_read(&lists_rwsem); - + disable_device(device); ib_device_unregister_sysfs(device); ib_device_unregister_rdmacg(device); - - release_name(device); - - mutex_unlock(&device_mutex); - ib_cache_cleanup_one(device); - - down_write(&lists_rwsem); - xa_destroy(&device->client_data); - up_write(&lists_rwsem); + release_name(device); } EXPORT_SYMBOL(ib_unregister_device); @@ -724,6 +808,7 @@ static int assign_client_id(struct ib_client *client) { int ret; + down_write(&clients_rwsem); /* * The add/remove callbacks must be called in FIFO/LIFO order. To * achieve this we assign client_ids so they are sorted in @@ -742,7 +827,11 @@ static int assign_client_id(struct ib_client *client) if (ret) goto out; + xa_set_mark(&clients, client->client_id, CLIENT_REGISTERED); + list_add_tail(&client->list, &client_list); + out: + up_write(&clients_rwsem); return ret; } @@ -765,23 +854,20 @@ int ib_register_client(struct ib_client *client) unsigned long index; int ret; - mutex_lock(&device_mutex); ret = assign_client_id(client); - if (ret) { - mutex_unlock(&device_mutex); + if (ret) return ret; - } - - xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) - if (!add_client_context(device, client) && client->add) - client->add(device); - - down_write(&lists_rwsem); - xa_set_mark(&clients, client->client_id, CLIENT_REGISTERED); - up_write(&lists_rwsem); - - mutex_unlock(&device_mutex); + down_read(&devices_rwsem); + xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) { + ret = add_client_context(device, client); + if (ret) { + up_read(&devices_rwsem); + ib_unregister_client(client); + return ret; + } + } + up_read(&devices_rwsem); return 0; } EXPORT_SYMBOL(ib_register_client); @@ -793,38 +879,31 @@ EXPORT_SYMBOL(ib_register_client); * Upper level users use ib_unregister_client() to remove their client * registration. When ib_unregister_client() is called, the client * will receive a remove callback for each IB device still registered. + * + * This is a full fence, once it returns no client callbacks will be called, + * or are running in another thread. */ void ib_unregister_client(struct ib_client *client) { struct ib_device *device; unsigned long index; - mutex_lock(&device_mutex); - - down_write(&lists_rwsem); + down_write(&clients_rwsem); xa_clear_mark(&clients, client->client_id, CLIENT_REGISTERED); - up_write(&lists_rwsem); - - xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) { - down_write(&lists_rwsem); - xa_clear_mark(&device->client_data, client->client_id, - CLIENT_DATA_REGISTERED); - up_write(&lists_rwsem); - - if (client->remove) - client->remove(device, xa_load(&device->client_data, - client->client_id)); - - down_write(&lists_rwsem); - xa_erase(&device->client_data, client->client_id); - up_write(&lists_rwsem); - } + up_write(&clients_rwsem); + /* + * Every device still known must be serialized to make sure we are + * done with the client callbacks before we return. + */ + down_read(&devices_rwsem); + xa_for_each (&devices, index, device) + remove_client_context(device, client->client_id); + up_read(&devices_rwsem); - down_write(&lists_rwsem); + down_write(&clients_rwsem); list_del(&client->list); xa_erase(&clients, client->client_id); - up_write(&lists_rwsem); - mutex_unlock(&device_mutex); + up_write(&clients_rwsem); } EXPORT_SYMBOL(ib_unregister_client); @@ -1009,10 +1088,10 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, struct ib_device *dev; unsigned long index; - down_read(&lists_rwsem); + down_read(&devices_rwsem); xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) ib_enum_roce_netdev(dev, filter, filter_cookie, cb, cookie); - up_read(&lists_rwsem); + up_read(&devices_rwsem); } /** @@ -1029,15 +1108,14 @@ int ib_enum_all_devs(nldev_callback nldev_cb, struct sk_buff *skb, unsigned int idx = 0; int ret = 0; - down_read(&lists_rwsem); + down_read(&devices_rwsem); xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { ret = nldev_cb(dev, skb, cb, idx); if (ret) break; idx++; } - - up_read(&lists_rwsem); + up_read(&devices_rwsem); return ret; } @@ -1195,6 +1273,7 @@ EXPORT_SYMBOL(ib_find_pkey); * @gid: A GID that the net_dev uses to communicate. * @addr: Contains the IP address that the request specified as its * destination. + * */ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, u8 port, @@ -1209,8 +1288,11 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, if (!rdma_protocol_ib(dev, port)) return NULL; - down_read(&lists_rwsem); - + /* + * Holding the read side guarantees that the client will not become + * unregistered while we are calling get_net_dev_by_params() + */ + down_read(&dev->client_data_rwsem); xan_for_each_marked (&dev->client_data, index, client_data, CLIENT_DATA_REGISTERED) { struct ib_client *client = xa_load(&clients, index); @@ -1223,8 +1305,7 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, if (net_dev) break; } - - up_read(&lists_rwsem); + up_read(&dev->client_data_rwsem); return net_dev; } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 7781eefd8b84eb..bb85045765029c 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2528,6 +2528,7 @@ struct ib_device { struct list_head event_handler_list; spinlock_t event_handler_lock; + struct rw_semaphore client_data_rwsem; struct xarray client_data; struct ib_cache cache;