From patchwork Mon Feb 4 21:17:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796643 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDA0713B4 for ; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD6882B340 for ; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C042C2B5AE; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9AB0D2B340 for ; Mon, 4 Feb 2019 21:18:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729315AbfBDVSB (ORCPT ); Mon, 4 Feb 2019 16:18:01 -0500 Received: from mail-pf1-f195.google.com ([209.85.210.195]:33948 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729224AbfBDVSB (ORCPT ); Mon, 4 Feb 2019 16:18:01 -0500 Received: by mail-pf1-f195.google.com with SMTP id h3so522942pfg.1 for ; Mon, 04 Feb 2019 13:18:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sbIbpXc5wgxbKoNhEP8faxcehxBYMwXEwY7QhLOqGQY=; b=MpoI0pkauEqd4yUsH82PPVceAzHmbus0aROwHn+3wmV7hwy/LEtj4kcGmcv5XEtm9o OElxUnPkAYD4d0CSdXXl/IFFZ+OIUpEigXshZMiPCDIguODX5T0crBeA4k0xvNzIbIuP /d5687dRjtMG0ZMrFlmfXgRNr0lscDwHfpz05ASc2I/4oIYvP1s+fhq3AWR3vC1S8FEo eX0AWZLWajh5OkIBi/yWUAgorGQEolNSq9GaU8vwiB4+AQO0+eaWFzP5OoqvPiCphpTa 0id195ZtWPQ0Eb5JhuPlx+l+QozjN4KH+RWces8b6wD8TdDm/choigTb6JaKT8c34MNG NRKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sbIbpXc5wgxbKoNhEP8faxcehxBYMwXEwY7QhLOqGQY=; b=GGjM/9rUko5kycfdq3FAekW2exJSSwIFqkbml09sqDadmlMfR25AaUzGB+UYXXVk8s tIUj2dUmHWDwIClMBKtWL+Ro2kHYy1n7YGlAF89GBR0MQBw6EBGgUTOr2mDf8Qg9Khdu VgKGt2WUUOFcjbb9kBQse0/np4hJgA6r97KH65m1iB14pxsNxNOJD9ijjPRLv0Bfigjc e3PTLJ94ju9CcwxaTAzVKIksOkKQG1CVBXfK96rm1Xj+rOUTFB7dFnvBtlsKLmQUqctW IplYHIzb6cKwd9gH1Jd9zIRvDQUPqExirsR/nkEVCh8v9z2uUSgk1ejCneowpUA8l0F4 vssg== X-Gm-Message-State: AHQUAubjH4aF+j3kE4xqQNlAB5Aiy0vogtRdcKog3OwJmJ6qJ3YGivyQ 58g64MGlj4e45r0RYpcmK6fWnD73z5Y= X-Google-Smtp-Source: AHgI3IZ+QHkpXjYnbSvG/yIBGyIjtcsYdVf6tJUHryX4WPhW6Y9e6nikcfV742c1hVlMA6EtVF4Jsw== X-Received: by 2002:a63:4c07:: with SMTP id z7mr1289321pga.319.1549315080746; Mon, 04 Feb 2019 13:18:00 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id o5sm1150165pgm.68.2019.02.04.13.17.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005KH-AZ; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Parav Pandit Subject: [PATCH 1/8] RDMA/device: Check that the rename is nop under the lock Date: Mon, 4 Feb 2019 14:17:44 -0700 Message-Id: <20190204211751.19001-2-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Since another rename could be running in parallel it is safer to check that the name is not changing inside the lock, where we already know the device name will not change. Fixes: d21943dd19b5 ("RDMA/core: Implement IB device rename function") Signed-off-by: Jason Gunthorpe Reviewed-by: Parav Pandit --- drivers/infiniband/core/device.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 55221990d946b2..f9f33e842c16b6 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -189,12 +189,14 @@ static struct ib_device *__ib_device_get_by_name(const char *name) int ib_device_rename(struct ib_device *ibdev, const char *name) { - int ret = 0; - - if (!strcmp(name, dev_name(&ibdev->dev))) - return ret; + int ret; mutex_lock(&device_mutex); + if (!strcmp(name, dev_name(&ibdev->dev))) { + ret = -EEXIST; + goto out; + } + if (__ib_device_get_by_name(name)) { ret = -EEXIST; goto out; From patchwork Mon Feb 4 21:17:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796645 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4731913B5 for ; Mon, 4 Feb 2019 21:18:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37B4A2B1EB for ; Mon, 4 Feb 2019 21:18:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2C22B2B500; Mon, 4 Feb 2019 21:18:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5E082B1EB for ; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729486AbfBDVSD (ORCPT ); Mon, 4 Feb 2019 16:18:03 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:33772 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729280AbfBDVSC (ORCPT ); Mon, 4 Feb 2019 16:18:02 -0500 Received: by mail-pg1-f195.google.com with SMTP id z11so495973pgu.0 for ; Mon, 04 Feb 2019 13:18:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wFKNk0zIu62HpNp5Qv2kIqP9VYmjfZM5PmuNx4HVzEY=; b=nRR7yR8x7r7KogJdYxCRuvaC8v6trh15JVzGSqpP8cXrX22REuZhV4OjgCKN+d8WC6 u5Gl9jUhHRZg8JONK/mkpspUfDojvrUh4CtR70Ck87PIRHO3eBEbHVXcX3yxPEHHYUeS JY1yk0YbAQSLEoL88TM8NYbSIbuVFP0pI16D0n12bEKFc02aC2lCkqbV0xwP1F3lWPlF 9gka1Fk3mP4myTzl0XDxvi8Cj/wcdNhDqmeOe5nyh+OlgjXRrQ82szveEGC3r1PchF8T 7TSkwp1gOVvE4zykgfp+75rBvZjZ03D9rJZAAJ7eFeTKKUEQ9zD+RudxyGEx6B3o4bfs KDaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wFKNk0zIu62HpNp5Qv2kIqP9VYmjfZM5PmuNx4HVzEY=; b=bdjNZkYnuLRlYrTv67dEJZd9Aglv8xfRuktwUQQpHYaCSTvXrYmnBJxFK87lHOrOSw g69s6C29NqDD6WqiKQQB028YHHSRWr1bWvoLv/uu/HifM9z00na5zRx6BDdp+sbX/zHl Kd+pt9Mii6rJ3OOqyi3b3nBnLMqxTFxa6198v24fbvDjK+9p3K4R+C/YOXcjmOXAWcrP GUIoGPnD0+P8GI+OtfHCVSaQApk2NE1tcX2XfhHCwDGZY3C324wglUKhP9zMUNZUX7sf Gl2wZQBtf7MkGzy7KIuBISKJ+M20rjuDjcYlrwWNxfePxHHft1CrOWBHhUo4zcvlrS7T WOsw== X-Gm-Message-State: AHQUAuYUrHcM5u6N3gQoSfHbf5H2a4/wSU01zLmJQg1zaTPI3mq930V0 0hvNKE3ZWLQkgD4QYIZY07bzT42qmPg= X-Google-Smtp-Source: AHgI3IaoFR4qhVVYNajWFabimUBOa4bOxSw9hZjtLT/DA5/JsheVsg6g9WN2qVOePIhjTyclOpCu1Q== X-Received: by 2002:a65:6094:: with SMTP id t20mr1252093pgu.285.1549315081030; Mon, 04 Feb 2019 13:18:01 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id p2sm1474794pfb.28.2019.02.04.13.17.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005KN-Cn; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH 2/8] RDMA/device: Ensure that security memory is always freed Date: Mon, 4 Feb 2019 14:17:45 -0700 Message-Id: <20190204211751.19001-3-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Since this only frees memory it should be done during the release callback. Otherwise there are possible error flows where it might not get called if registration aborts. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/core_priv.h | 4 ++-- drivers/infiniband/core/device.c | 10 +++------- drivers/infiniband/core/security.c | 4 +--- 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/core/core_priv.h b/drivers/infiniband/core/core_priv.h index bcb3e3029a9be8..52d17e7eddc35f 100644 --- a/drivers/infiniband/core/core_priv.h +++ b/drivers/infiniband/core/core_priv.h @@ -181,7 +181,7 @@ int ib_get_cached_subnet_prefix(struct ib_device *device, u64 *sn_pfx); #ifdef CONFIG_SECURITY_INFINIBAND -void ib_security_destroy_port_pkey_list(struct ib_device *device); +void ib_security_release_port_pkey_list(struct ib_device *device); void ib_security_cache_change(struct ib_device *device, u8 port_num, @@ -203,7 +203,7 @@ int ib_mad_agent_security_setup(struct ib_mad_agent *agent, void ib_mad_agent_security_cleanup(struct ib_mad_agent *agent); int ib_mad_enforce_security(struct ib_mad_agent_private *map, u16 pkey_index); #else -static inline void ib_security_destroy_port_pkey_list(struct ib_device *device) +static inline void ib_security_release_port_pkey_list(struct ib_device *device) { } diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index f9f33e842c16b6..b21ea9154cf24d 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -253,6 +253,8 @@ static void ib_device_release(struct device *device) ib_cache_release_one(dev); kfree(dev->port_immutable); } + ib_security_release_port_pkey_list(dev); + kfree(dev->port_pkey_list); kfree(dev); } @@ -521,7 +523,6 @@ static void cleanup_device(struct ib_device *device) { ib_cache_cleanup_one(device); ib_cache_release_one(device); - kfree(device->port_pkey_list); kfree(device->port_immutable); } @@ -559,12 +560,10 @@ static int setup_device(struct ib_device *device) if (ret) { dev_warn(&device->dev, "Couldn't set up InfiniBand P_Key/GID cache\n"); - goto pkey_cleanup; + return ret; } return 0; -pkey_cleanup: - kfree(device->port_pkey_list); port_cleanup: kfree(device->port_immutable); return ret; @@ -681,9 +680,6 @@ void ib_unregister_device(struct ib_device *device) ib_cache_cleanup_one(device); - ib_security_destroy_port_pkey_list(device); - kfree(device->port_pkey_list); - down_write(&lists_rwsem); write_lock_irqsave(&device->client_data_lock, flags); list_for_each_entry_safe(context, tmp, &device->client_data_list, diff --git a/drivers/infiniband/core/security.c b/drivers/infiniband/core/security.c index 1efadbccf394df..df2602a91350e6 100644 --- a/drivers/infiniband/core/security.c +++ b/drivers/infiniband/core/security.c @@ -554,13 +554,12 @@ void ib_security_cache_change(struct ib_device *device, } } -void ib_security_destroy_port_pkey_list(struct ib_device *device) +void ib_security_release_port_pkey_list(struct ib_device *device) { struct pkey_index_qp_list *pkey, *tmp_pkey; int i; for (i = rdma_start_port(device); i <= rdma_end_port(device); i++) { - spin_lock(&device->port_pkey_list[i].list_lock); list_for_each_entry_safe(pkey, tmp_pkey, &device->port_pkey_list[i].pkey_list, @@ -568,7 +567,6 @@ void ib_security_destroy_port_pkey_list(struct ib_device *device) list_del(&pkey->pkey_index_list); kfree(pkey); } - spin_unlock(&device->port_pkey_list[i].list_lock); } } From patchwork Mon Feb 4 21:17:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796641 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B12806C2 for ; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB8C2B1EB for ; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 935B92B5B8; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D1402B1EB for ; Mon, 4 Feb 2019 21:18:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729330AbfBDVSC (ORCPT ); Mon, 4 Feb 2019 16:18:02 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:34788 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729224AbfBDVSC (ORCPT ); Mon, 4 Feb 2019 16:18:02 -0500 Received: by mail-pl1-f195.google.com with SMTP id w4so529936plz.1 for ; Mon, 04 Feb 2019 13:18:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/b3usJTzo5aKzuVcaCRq2eC9IV9zhxKKMyiEdMJY9qk=; b=aJbIeZKg8gn8QQls44KitN7xu210Nsn8ilRoZlUdzpdwWXpsUAZihfLZCKNMoWt0+9 AfDQcxGPtTVbeJvitFzQWywkfJ6H/ERP9LL74hPX0JVGy34gBW7Z7s4GBddKjqKb976J VaoKTUGc8pResHPxK9g9XuE+6cVPBTeuatDQ1QUGJ2X7BdJmVq/sERkornVYh9hwYOJQ l1nk9691iTpM81jfxxSNSP5RirPpl35Ng3caql+kH3Lz0gucR+r8fL9VnZuVaaA0qnd5 9o71MiNf5rTVvrRac+UAZdfcNxrYIIV97oCqY3UUbXazw16/9iTiBHmhjvnKDcPdHOvM cTcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/b3usJTzo5aKzuVcaCRq2eC9IV9zhxKKMyiEdMJY9qk=; b=gXCO/VdoKEjHCVJyNwQlZvFPH7NG4GAk+jFBdvoo82dQSwPUq0SNbdF9mRh4fKbtAK 9gIXinUlfICHYhUJAQsqDhMjYCeSTlVWe1Ld8l5yXM2SVEk0cwA4Ef2VvbXtJq3wTbfB n77MQ3mCf756Qp0/MRDLj4hpMkHBqh93OPsBvucIRSQr3b79uTsqGucIAWeD7svFu87P wPBU50qNjd3o0ER7Lqw84m455wbQlP0UMg1tEWS+ElxzHHolqvZydgKai14HFkfTawDO BfcRzy+k1qj0z2kSm5ky1956cp125yioTd4gAJ9Kv5ERMfwvbA8mLfqbfQqd/sqkhM7x xtNw== X-Gm-Message-State: AHQUAub2C+lBKN0m/PDD0YnIWq2tJO3lHMTOL6RB/wCHn4wrUN99IuDG jScpmiM56A/qBWVs4i91nrHGLwmRa+I= X-Google-Smtp-Source: AHgI3Ib5PCU9vavetpYA46FZLiI7MGGeRldXEA08ah6l3AKy23Hm6kIkDb3eyUeGKk75p8Rdr0qrig== X-Received: by 2002:a17:902:7892:: with SMTP id q18mr1433967pll.217.1549315081555; Mon, 04 Feb 2019 13:18:01 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id o189sm1496673pfg.117.2019.02.04.13.17.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005KT-E0; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH 3/8] RDMA/device: Call ib_cache_release_one() only from ib_device_release() Date: Mon, 4 Feb 2019 14:17:46 -0700 Message-Id: <20190204211751.19001-4-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Instead of complicated logic about when this memory is freed, always free it during device release(). All the cache pointers start out as NULL, so it is safe to call this before the cache is initialized. This makes for a simpler error unwind flow, and a simpler understanding of the lifetime of the memory allocations inside the struct ib_device. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/cache.c | 3 +++ drivers/infiniband/core/device.c | 41 ++++++++++---------------------- 2 files changed, 15 insertions(+), 29 deletions(-) diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c index 7b04590f307f52..2338d0b3a0cab8 100644 --- a/drivers/infiniband/core/cache.c +++ b/drivers/infiniband/core/cache.c @@ -1460,6 +1460,9 @@ void ib_cache_release_one(struct ib_device *device) { int p; + if (!device->cache.ports) + return; + /* * The release function frees all the cache elements. * This function should be called as part of freeing diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index b21ea9154cf24d..286a41cd4cda78 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -244,17 +244,10 @@ static void ib_device_release(struct device *device) struct ib_device *dev = container_of(device, struct ib_device, dev); WARN_ON(dev->reg_state == IB_DEV_REGISTERED); - if (dev->reg_state == IB_DEV_UNREGISTERED) { - /* - * In IB_DEV_UNINITIALIZED state, cache or port table - * is not even created. Free cache and port table only when - * device reaches UNREGISTERED state. - */ - ib_cache_release_one(dev); - kfree(dev->port_immutable); - } + ib_cache_release_one(dev); ib_security_release_port_pkey_list(dev); kfree(dev->port_pkey_list); + kfree(dev->port_immutable); kfree(dev); } @@ -519,13 +512,6 @@ static void setup_dma_device(struct ib_device *device) } } -static void cleanup_device(struct ib_device *device) -{ - ib_cache_cleanup_one(device); - ib_cache_release_one(device); - kfree(device->port_immutable); -} - static int setup_device(struct ib_device *device) { struct ib_udata uhw = {.outlen = 0, .inlen = 0}; @@ -547,26 +533,16 @@ static int setup_device(struct ib_device *device) if (ret) { dev_warn(&device->dev, "Couldn't query the device attributes\n"); - goto port_cleanup; + return ret; } ret = setup_port_pkey_list(device); if (ret) { dev_warn(&device->dev, "Couldn't create per port_pkey_list\n"); - goto port_cleanup; - } - - ret = ib_cache_setup_one(device); - if (ret) { - dev_warn(&device->dev, - "Couldn't set up InfiniBand P_Key/GID cache\n"); return ret; } - return 0; -port_cleanup: - kfree(device->port_immutable); - return ret; + return 0; } /** @@ -606,6 +582,13 @@ int ib_register_device(struct ib_device *device, const char *name) if (ret) goto out; + ret = ib_cache_setup_one(device); + if (ret) { + dev_warn(&device->dev, + "Couldn't set up InfiniBand P_Key/GID cache\n"); + goto out; + } + device->index = __dev_new_index(); ib_device_register_rdmacg(device); @@ -632,7 +615,7 @@ int ib_register_device(struct ib_device *device, const char *name) cg_cleanup: ib_device_unregister_rdmacg(device); - cleanup_device(device); + ib_cache_cleanup_one(device); out: mutex_unlock(&device_mutex); return ret; From patchwork Mon Feb 4 21:17:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AFD0E13B5 for ; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB462B1EB for ; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 93DB62B340; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3928D2B500 for ; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729615AbfBDVSE (ORCPT ); Mon, 4 Feb 2019 16:18:04 -0500 Received: from mail-pf1-f194.google.com ([209.85.210.194]:42920 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729224AbfBDVSD (ORCPT ); Mon, 4 Feb 2019 16:18:03 -0500 Received: by mail-pf1-f194.google.com with SMTP id 64so503617pfr.9 for ; Mon, 04 Feb 2019 13:18:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sRRk1/NqVGloHjW6gXCGaQJmxB0HmoNqVSOjkft3r3o=; b=DnXeFUwpKM0a36jWO53m2UE6pNooI8urEZ+Xox7OvHEaR/jm7eXz7isx6YIvp2fUpO +ZSofAl+u2YE5DXzxYre8CKCJ9hH+O5Krbjs1exRwBAAsaAc4fB630Vbc7RnX0BHE5CX JLdHlv96+qKsIeD/qfRDzgpdbgGhXACyEV2/xI7hthTFWxSPMOsWv66EIk7sHu9Fl++x tF6ZZm46IKT4Bug3k3MDN9+1yJH0YMcM/XqAeSH8XdLNu9ckD8u+i+n6hx2GTAdzJ0NP /LfnV2g20YILe0CPzI2s21QKkMb2SR2L+/rRDxbg8J0RxkXqX8F1KE/928NZhWebol4+ 3FHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sRRk1/NqVGloHjW6gXCGaQJmxB0HmoNqVSOjkft3r3o=; b=TicUoSmFZhVG00982sUDGhp4S5lrPDltk55FZdjCiXf6NxVRjePmZLkh04WKFJXNKy kTw3Wkuc2/aOmaUUhhpOV/rzU+dn4TyoOezvhPkwuoGHN0aF33BOGHOP9r+emyPc56mN f1MwEAdNK/7bYJdvMYF6EhqloGvmGiemHquSPNmY0+k9Qg7SitkTvCS1Y3yhkuRCG8iC 4B4lK2uAs3O/fbZc47bGwZHrvL2F1UpMm/OWUI44HtJTuuNbjkFnU3Uy20Int6Rst5cT oe5w0Epwthf7AADqZfoe0yK+XEXx2wyL/EVM4tggr0T1ot9nOqaxYe8VwZh7Xm3d8f0m YYkQ== X-Gm-Message-State: AHQUAuY5mBt+cb/9CZqN8KjxGs7YLgvVlcMuoJdCHIPZ0h7E6Z1Q6ZbA l19PxYHZJA0gtitYr2GrporgRP0IFK0= X-Google-Smtp-Source: AHgI3Iayy2ZIg9qB91GmHr739fuQhSZxtxE+3LZOcsvV+c9p6JtPQLUNxRKleSGyMIYSlpqaRZDM4g== X-Received: by 2002:a63:94:: with SMTP id 142mr1309223pga.74.1549315081985; Mon, 04 Feb 2019 13:18:01 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id z62sm1812092pfl.33.2019.02.04.13.17.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005KZ-FJ; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Parav Pandit Subject: [PATCH 4/8] RDMA/device: Get rid of reg_state Date: Mon, 4 Feb 2019 14:17:47 -0700 Message-Id: <20190204211751.19001-5-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This really has no purpose anymore, refcount can be used to tell if the device is still registered. Keeping it around just invites mis-use. Signed-off-by: Jason Gunthorpe Reviewed-by: Parav Pandit --- drivers/infiniband/core/device.c | 8 ++------ include/rdma/ib_verbs.h | 6 ------ 2 files changed, 2 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 286a41cd4cda78..c846b5f7abb9d8 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -243,7 +243,7 @@ static void ib_device_release(struct device *device) { struct ib_device *dev = container_of(device, struct ib_device, dev); - WARN_ON(dev->reg_state == IB_DEV_REGISTERED); + WARN_ON(refcount_read(&dev->refcount)); ib_cache_release_one(dev); ib_security_release_port_pkey_list(dev); kfree(dev->port_pkey_list); @@ -316,8 +316,7 @@ EXPORT_SYMBOL(_ib_alloc_device); void ib_dealloc_device(struct ib_device *device) { WARN_ON(!list_empty(&device->client_data_list)); - WARN_ON(device->reg_state != IB_DEV_UNREGISTERED && - device->reg_state != IB_DEV_UNINITIALIZED); + WARN_ON(refcount_read(&device->refcount)); rdma_restrack_clean(device); put_device(&device->dev); } @@ -601,7 +600,6 @@ int ib_register_device(struct ib_device *device, const char *name) } refcount_set(&device->refcount, 1); - device->reg_state = IB_DEV_REGISTERED; list_for_each_entry(client, &client_list, list) if (!add_client_context(device, client) && client->add) @@ -672,8 +670,6 @@ void ib_unregister_device(struct ib_device *device) } write_unlock_irqrestore(&device->client_data_lock, flags); up_write(&lists_rwsem); - - device->reg_state = IB_DEV_UNREGISTERED; } EXPORT_SYMBOL(ib_unregister_device); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 2e1f1e885ee502..76138ef4f839ea 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2558,12 +2558,6 @@ struct ib_device { struct kobject *ports_kobj; struct list_head port_list; - enum { - IB_DEV_UNINITIALIZED, - IB_DEV_REGISTERED, - IB_DEV_UNREGISTERED - } reg_state; - int uverbs_abi_ver; u64 uverbs_cmd_mask; u64 uverbs_ex_cmd_mask; From patchwork Mon Feb 4 21:17:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796647 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5457913B4 for ; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 451C92B1EB for ; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 396A32B5AE; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D7D302B1EB for ; Mon, 4 Feb 2019 21:18:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729501AbfBDVSE (ORCPT ); Mon, 4 Feb 2019 16:18:04 -0500 Received: from mail-pl1-f196.google.com ([209.85.214.196]:44254 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729343AbfBDVSD (ORCPT ); Mon, 4 Feb 2019 16:18:03 -0500 Received: by mail-pl1-f196.google.com with SMTP id p4so503471plq.11 for ; Mon, 04 Feb 2019 13:18:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iyT4w/yRYWxwv+HPRN3tEUx7ear6GsieNYA5NP+ZN+Q=; b=dpd2ftUET9PKDMYtzOxU3wGjGGuQ1ped0QMUs7dmVF6ArXxxV+YYEwFEE9uj5AAll9 /hUqmRoz/WBq4AXw1GRDnl8WiNsPiK3gnjwsQ3Z34gsjIocRT305g27kJcA7Rb1bXnj1 P3mvl4z1cyWSgfB3QQVrRnzN/lzfbw3OpzCI7QtGuNXWoR3OkMLJm4fdFxTBetsdEoVe N2QFJnLKvJw52uUlrbu+66jRQUwyAPmwisjpAs5leBy8P+sQs3C+p0AlESC0IbbE99vv k/6sOgE08VDVlmLH+ZFAPO5dcT76ZTBJ5QrcthLVPvayMKBiybLnME/TPi1u2UX0PuW6 kkXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iyT4w/yRYWxwv+HPRN3tEUx7ear6GsieNYA5NP+ZN+Q=; b=mNupOGspcLesKtbTiaW4gP1+hMJc6zY54tUD1o/Zs0mL94OsCa6lyBsymEuAGqJlln Jywuje0To5c0MPRfNYMcIfLVSAy8MY0+cBGhuYQ0oe+G9XwuCrMDdnWFpzdcqThijMzF MeCuc+JYOtml37bx2aBXGisDAsdo8jkihwQGMuk/U3vWpg3zxsJqg2UR7VE5I0Knf4YQ TI8+FZ1/LVQCoewqKHQrVaafDYASSGS6pv3LOVnB8ihNk0zhak8GhBpBKfI4Kl0O7ySZ uCLJR0jE0l+KazSoDdW64qFeHgC3wwW34zcOT6FkFMzNrQvamqIQU7vaORZHsQ8YbrqT FeFA== X-Gm-Message-State: AHQUAubI2ZBnVw2DL00w3Zz9miaOmfV70XhUfUokSB9MeiQebAYa63oM GzkP5D6lK5ZXxXBn8/tpgSeA4TponJQ= X-Google-Smtp-Source: AHgI3Iax3YnEEIavQo0jZztSONl9T60qYWIa5sqr579g/dWs0Q5zHh5pfbhyO9yYmXvNQTObtwp4ww== X-Received: by 2002:a17:902:7044:: with SMTP id h4mr1440797plt.35.1549315082313; Mon, 04 Feb 2019 13:18:02 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id q75sm1284517pfa.38.2019.02.04.13.18.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005Kf-HT; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH 5/8] RDMA/device: Use an ida instead of a free page in alloc_name Date: Mon, 4 Feb 2019 14:17:48 -0700 Message-Id: <20190204211751.19001-6-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe ida is the proper data structure to hold list of clustered small integers and then allocate an unused integer. Get rid of the convoluted and limited open-coded bitmap. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/device.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index c846b5f7abb9d8..300a36beaa300c 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -213,30 +213,36 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) static int alloc_name(struct ib_device *ibdev, const char *name) { - unsigned long *inuse; struct ib_device *device; + struct ida inuse; + int rc; int i; - inuse = (unsigned long *) get_zeroed_page(GFP_KERNEL); - if (!inuse) - return -ENOMEM; - + ida_init(&inuse); list_for_each_entry(device, &device_list, core_list) { char buf[IB_DEVICE_NAME_MAX]; if (sscanf(dev_name(&device->dev), name, &i) != 1) continue; - if (i < 0 || i >= PAGE_SIZE * 8) + if (i < 0 || i >= INT_MAX) continue; snprintf(buf, sizeof buf, name, i); - if (!strcmp(buf, dev_name(&device->dev))) - set_bit(i, inuse); + if (strcmp(buf, dev_name(&device->dev)) != 0) + continue; + + rc = ida_alloc_range(&inuse, i, i, GFP_KERNEL); + if (rc < 0) + goto out; } - i = find_first_zero_bit(inuse, PAGE_SIZE * 8); - free_page((unsigned long) inuse); + rc = ida_alloc(&inuse, GFP_KERNEL); + if (rc < 0) + goto out; - return dev_set_name(&ibdev->dev, name, i); + rc = dev_set_name(&ibdev->dev, name, rc); +out: + ida_destroy(&inuse); + return rc; } static void ib_device_release(struct device *device) From patchwork Mon Feb 4 21:17:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 94C801823 for ; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 842CD2B500 for ; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 784F22B5CC; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD6D22B500 for ; Mon, 4 Feb 2019 21:18:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729616AbfBDVSF (ORCPT ); Mon, 4 Feb 2019 16:18:05 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:40511 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729280AbfBDVSD (ORCPT ); Mon, 4 Feb 2019 16:18:03 -0500 Received: by mail-pl1-f195.google.com with SMTP id u18so514844plq.7 for ; Mon, 04 Feb 2019 13:18:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bKuQbi+FWksfdR9Byh3oXBx/j7YzSi1Vv/T28zYqiiA=; b=gD+yhSNGKeiZDsJ6wN4ckSPQyvOjPihPq5BQ5Keq2Uj3JPV7isDyW/p2ZNfJ60oJou ojBGYF5tGeUMjJxTHjei1DE24m7p8B+5Rk7JXBwxFkmw9Yg6oKRzdzpzBfYHlOVM5CS4 tXwYRqJVhssy2Fp71Te3+eOZ6sRFrXIb2peBVH3wWAn92yckz2PHc5mga+zttuQDMjWW ZPFEds+nmULSjF784GpEmVjvOaf0VCh6AxK251nKXgEVPdZbPmYWw4yDNkRWm/5XJb1j ybqd8fpwHiZMRu8gLvFzscDJBEhmJq5LJKSeqLGO5v176Y4reWXsO+yygnopMsasL0D4 Cyhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bKuQbi+FWksfdR9Byh3oXBx/j7YzSi1Vv/T28zYqiiA=; b=doap9RI4mIbraiwxsNDzNF5Wu+C7Tye7J5A82c5FBzFJRSLFQcveAGtnovHFM0wgsZ xExYv40CUvcN03yCLHlGxMdok8tjIz9O1GvQdjmy90Sg0/9N2Gcob/mHX+iv0Fyc2cRb upQ/OUKiX89ojxePICP82go2P2AlOkYBh+WaygvpgDsVWp5yGusmp6mPX2S2SCzWzbPW EIYIJdqyw985NbB+PK2DfSLZg8+Ij+6KPbGJNftM0uaLUjZR5dqF50usYsvWdUKVP9s/ 2yYkwu4v/xY6Q3umdYuCIBwWLKAjmkYKX5Gjm9dgrUlbKyrGEKSbCr5pkvefRqJPEo6z OHZA== X-Gm-Message-State: AHQUAuYodxCrveCd7/fwMKBGAgi4HWtsDm4KTU3W8H68Q0cbHd3i6zC1 nozS3WsY+f36Uuv5Pa5+B+x+tbT3+t4= X-Google-Smtp-Source: AHgI3Ia5YFTDTFRHND0ckIGMOph10xsS5XDirjA09EEXhXSDVwKxUSHI3emD6oQ2NC/OMcAL2pXHCA== X-Received: by 2002:a17:902:b01:: with SMTP id 1mr1432943plq.331.1549315082863; Mon, 04 Feb 2019 13:18:02 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id t5sm1186574pgo.70.2019.02.04.13.18.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:02 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005Kl-IT; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Matthew Wilcox , Parav Pandit Subject: [PATCH 6/8] RDMA/devices: Use xarray to store the clients Date: Mon, 4 Feb 2019 14:17:49 -0700 Message-Id: <20190204211751.19001-7-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This gives each client a unique ID and will let us move client_data to use xarray, and revise the locking scheme. clients have to be add/removed in strict FIFO/LIFO order as they interdepend. To support this the client_ids are assigned to increase in FIFO order. The existing linked list is kept to support reverse iteration until xarray can get a reverse iteration API. Signed-off-by: Jason Gunthorpe Reviewed-by: Parav Pandit --- drivers/infiniband/core/device.c | 50 ++++++++++++++++++++++++++++---- include/rdma/ib_verbs.h | 3 +- 2 files changed, 47 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 300a36beaa300c..f485e77223d429 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -65,15 +65,17 @@ struct workqueue_struct *ib_comp_unbound_wq; struct workqueue_struct *ib_wq; EXPORT_SYMBOL_GPL(ib_wq); -/* The device_list and client_list contain devices and clients after their +/* The device_list and clients contain devices and clients after their * registration has completed, and the devices and clients are removed * during unregistration. */ static LIST_HEAD(device_list); static LIST_HEAD(client_list); +#define CLIENT_REGISTERED XA_MARK_1 +static DEFINE_XARRAY_FLAGS(clients, XA_FLAGS_ALLOC); /* * device_mutex and lists_rwsem protect access to both device_list and - * client_list. device_mutex protects writer access by device and client + * clients. device_mutex protects writer access by device and client * registration / de-registration. lists_rwsem protects reader access to * these lists. Iterators of these lists must lock it for read, while updates * to the lists must be done with a write lock. A special case is when the @@ -563,6 +565,7 @@ int ib_register_device(struct ib_device *device, const char *name) { int ret; struct ib_client *client; + unsigned long index; setup_dma_device(device); @@ -607,7 +610,7 @@ int ib_register_device(struct ib_device *device, const char *name) refcount_set(&device->refcount, 1); - list_for_each_entry(client, &client_list, list) + xa_for_each_marked (&clients, index, client, CLIENT_REGISTERED) if (!add_client_context(device, client) && client->add) client->add(device); @@ -679,6 +682,32 @@ void ib_unregister_device(struct ib_device *device) } EXPORT_SYMBOL(ib_unregister_device); +static int assign_client_id(struct ib_client *client) +{ + int ret; + + /* + * The add/remove callbacks must be called in FIFO/LIFO order. To + * achieve this we assign client_ids so they are sorted in + * registration order, and retain a linked list we can reverse iterate + * to get the LIFO order. The extra linked list can go away if xarray + * learns to reverse iterate. + */ + if (list_empty(&client_list)) + client->client_id = 0; + else + client->client_id = + list_last_entry(&client_list, struct ib_client, list) + ->client_id; + ret = xa_alloc(&clients, &client->client_id, INT_MAX, client, + GFP_KERNEL); + if (ret) + goto out; + +out: + return ret; +} + /** * ib_register_client - Register an IB client * @client:Client to register @@ -695,15 +724,21 @@ EXPORT_SYMBOL(ib_unregister_device); int ib_register_client(struct ib_client *client) { struct ib_device *device; + int ret; mutex_lock(&device_mutex); + ret = assign_client_id(client); + if (ret) { + mutex_unlock(&device_mutex); + return ret; + } list_for_each_entry(device, &device_list, core_list) if (!add_client_context(device, client) && client->add) client->add(device); down_write(&lists_rwsem); - list_add_tail(&client->list, &client_list); + xa_set_mark(&clients, client->client_id, CLIENT_REGISTERED); up_write(&lists_rwsem); mutex_unlock(&device_mutex); @@ -728,7 +763,7 @@ void ib_unregister_client(struct ib_client *client) mutex_lock(&device_mutex); down_write(&lists_rwsem); - list_del(&client->list); + xa_clear_mark(&clients, client->client_id, CLIENT_REGISTERED); up_write(&lists_rwsem); list_for_each_entry(device, &device_list, core_list) { @@ -764,6 +799,10 @@ void ib_unregister_client(struct ib_client *client) kfree(found_context); } + down_write(&lists_rwsem); + list_del(&client->list); + xa_erase(&clients, client->client_id); + up_write(&lists_rwsem); mutex_unlock(&device_mutex); } EXPORT_SYMBOL(ib_unregister_client); @@ -1417,6 +1456,7 @@ static void __exit ib_core_cleanup(void) destroy_workqueue(ib_comp_wq); /* Make sure that any pending umem accounting work is done. */ destroy_workqueue(ib_wq); + WARN_ON(!xa_empty(&clients)); } MODULE_ALIAS_RDMA_NETLINK(RDMA_NL_LS, 4); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 76138ef4f839ea..84d7523984ee77 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2596,7 +2596,7 @@ struct ib_device { }; struct ib_client { - char *name; + const char *name; void (*add) (struct ib_device *); void (*remove)(struct ib_device *, void *client_data); @@ -2623,6 +2623,7 @@ struct ib_client { const struct sockaddr *addr, void *client_data); struct list_head list; + u32 client_id; /* kverbs are not required by the client */ u8 no_kverbs_req:1; From patchwork Mon Feb 4 21:17:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8499F13B5 for ; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F8A22B1EB for ; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 63BE62B5B8; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2AE262B340 for ; Mon, 4 Feb 2019 21:18:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729738AbfBDVSF (ORCPT ); Mon, 4 Feb 2019 16:18:05 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:38768 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729528AbfBDVSF (ORCPT ); Mon, 4 Feb 2019 16:18:05 -0500 Received: by mail-pl1-f195.google.com with SMTP id e5so520920plb.5 for ; Mon, 04 Feb 2019 13:18:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=na0AVmF6sCXtruXHUk+SFpIEnI7brZ1LgNFV6ITGdcc=; b=E3XiWtc8/IjMPD/5qFu0Rdjeukl4ZReWWn81pM0Kj8obc4ALwzxWpqdbbLw1tG2T5q b2uNj3cUEsn9fjNj/vof64+e6V/azKLc7dDebkyyKb9k6Do5IRQhwNzXDA2kAfJmK4ZP GiqTnKNBBGQCqTtol+2vaAkWhx2rMZjhY3hibgmHDBr4WlQFVyU54IsPl/9lNSh8YrcN eHTXaJHqSSw1ksC4m5nXGR+gzjmxBKRQCltojTLgO65baZ26lL9ph5u4XqFYNmO7GRnb HrkgTyRia4XO0FXavKG51TZMeVxxxUkjRgz5YDMqL55VE21ahxu87Coa8mcdqppXbsH/ l4vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=na0AVmF6sCXtruXHUk+SFpIEnI7brZ1LgNFV6ITGdcc=; b=M/HHQZwey3SazB82fR/sHIXfXMWkQSy8I9c61rlUfs6AK4edo+Qzw1wwyvU37LVsnm jgkinIXw6OhuM8+yf+D6BB3WYR3W6CRpRla/L5lLgUKvhSB9H/HWdrMNFu2NbqxPQfX3 WazSlSSv+GVLc+U/B+xeSk8p9RjsWflGihmMuQAHLNwu/KNjcBB2Fa3qQQNA7oXbYeh9 RiJM0JeqRRN/7c7FHmJKnODV2JmySfKhqhe1OmiAEXulpUzecZTOL8P3uJ7YnYVg+MGE YNk1fz2CCSYhxTotChaMxCRaLXjKUEVp7Si4D1iHLSFr3ABMzXvYurLmBCE6zBQG2C2j W1Hw== X-Gm-Message-State: AHQUAuawklQB4hr384onAToLk+5HtU5ti/Hr2jHsnCM30NyyYr4OzGM/ YrDvW3ymzUpj1xiEUzXdrME/ZbRg+kw= X-Google-Smtp-Source: AHgI3Ib3iWaOvMW9L78kk62dPmaUho5EyiHDXkc4MoLFcp3l1xDXsEUJa+Da90Uj3Y2VxPIVoZICbw== X-Received: by 2002:a17:902:1e8:: with SMTP id b95mr1394219plb.325.1549315083727; Mon, 04 Feb 2019 13:18:03 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id d21sm1106094pgv.37.2019.02.04.13.18.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:02 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005Kr-JW; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe , Matthew Wilcox Subject: [PATCH 7/8] RDMA/devices: Use xarray to store the client_data Date: Mon, 4 Feb 2019 14:17:50 -0700 Message-Id: <20190204211751.19001-8-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Now that we have a small ID for each client we can use xarray instead of linearly searching linked lists for client data. This will give much faster and scalable client data lookup, and will lets us revise the locking scheme. Since xarray can store 'going_down' using a mark just entirely eliminate the struct ib_client_data and directly store the client_data value in the xarray. However this does require a special iterator as we must still iterate over any NULL client_data values. Also eliminate the client_data_lock in favour of internal xarray locking. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/device.c | 351 +++++++++++++++---------------- include/rdma/ib_verbs.h | 23 +- 2 files changed, 189 insertions(+), 185 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index f485e77223d429..7eca7c4bd4071f 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -51,30 +51,75 @@ MODULE_AUTHOR("Roland Dreier"); MODULE_DESCRIPTION("core kernel InfiniBand API"); MODULE_LICENSE("Dual BSD/GPL"); -struct ib_client_data { - struct list_head list; - struct ib_client *client; - void * data; - /* The device or client is going down. Do not call client or device - * callbacks other than remove(). */ - bool going_down; -}; - struct workqueue_struct *ib_comp_wq; struct workqueue_struct *ib_comp_unbound_wq; struct workqueue_struct *ib_wq; EXPORT_SYMBOL_GPL(ib_wq); -/* The device_list and clients contain devices and clients after their - * registration has completed, and the devices and clients are removed - * during unregistration. */ -static LIST_HEAD(device_list); +/* + * devices contains devices that have had their names assigned. The + * devices may not be registered. Users that care about the registration + * status need to call ib_device_try_get() on the device to ensure it is + * registered, and keep it registered, for the required duration. + * + */ +static DEFINE_XARRAY_FLAGS(devices, XA_FLAGS_ALLOC); + +/* + * Note that if the *rwsem is held and the *_REGISTERED mark is seen then the + * object is guaranteed to be and remain registered for the duration of the + * lock. + */ +#define DEVICE_REGISTERED XA_MARK_1 + static LIST_HEAD(client_list); #define CLIENT_REGISTERED XA_MARK_1 static DEFINE_XARRAY_FLAGS(clients, XA_FLAGS_ALLOC); /* - * device_mutex and lists_rwsem protect access to both device_list and + * If client_data is registered then the corresponding client must also still + * be registered. + */ +#define CLIENT_DATA_REGISTERED XA_MARK_1 +/* + * xarray has this annoying behavior where it won't iterate over NULL values + * stored in allocated arrays. So we need our own iterator to see all values + * stored in the array. This does the same thing as xa_for_each except that it + * also returns NULL valued entries if the array is allocating. Simplified to + * only work on simple xarrays. + */ +static void *xan_find(struct xarray *xa, unsigned long *indexp, + unsigned long max, xa_mark_t filter) +{ + XA_STATE(xas, xa, *indexp); + void *entry; + + rcu_read_lock(); + do { + if ((__force unsigned int)filter < XA_MAX_MARKS) + entry = xas_find_marked(&xas, max, filter); + else + entry = xas_find(&xas, max); + if (xa_is_zero(entry) && !xas_get_mark(&xas, XA_FREE_MARK)) + break; + } while (xas_retry(&xas, entry)); + rcu_read_unlock(); + + if (entry) { + *indexp = xas.xa_index; + if (xa_is_zero(entry)) + return NULL; + return entry; + } + return ERR_PTR(-ENOENT); +} +#define xan_for_each(xa, index, entry, filter) \ + for (index = 0, entry = xan_find(xa, &(index), ULONG_MAX, filter); \ + !IS_ERR(entry); \ + (index)++, entry = xan_find(xa, &(index), ULONG_MAX, filter)) + +/* + * device_mutex and lists_rwsem protect access to both devices and * clients. device_mutex protects writer access by device and client * registration / de-registration. lists_rwsem protects reader access to * these lists. Iterators of these lists must lock it for read, while updates @@ -135,17 +180,6 @@ static int ib_device_check_mandatory(struct ib_device *device) return 0; } -static struct ib_device *__ib_device_get_by_index(u32 index) -{ - struct ib_device *device; - - list_for_each_entry(device, &device_list, core_list) - if (device->index == index) - return device; - - return NULL; -} - /* * Caller must perform ib_device_put() to return the device reference count * when ib_device_get_by_index() returns valid device pointer. @@ -155,7 +189,7 @@ struct ib_device *ib_device_get_by_index(u32 index) struct ib_device *device; down_read(&lists_rwsem); - device = __ib_device_get_by_index(index); + device = xa_load(&devices, index); if (device) { if (!ib_device_try_get(device)) device = NULL; @@ -181,8 +215,9 @@ EXPORT_SYMBOL(ib_device_put); static struct ib_device *__ib_device_get_by_name(const char *name) { struct ib_device *device; + unsigned long index; - list_for_each_entry(device, &device_list, core_list) + xa_for_each (&devices, index, device) if (!strcmp(name, dev_name(&device->dev))) return device; @@ -216,12 +251,13 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) static int alloc_name(struct ib_device *ibdev, const char *name) { struct ib_device *device; + unsigned long index; struct ida inuse; int rc; int i; ida_init(&inuse); - list_for_each_entry(device, &device_list, core_list) { + xa_for_each (&devices, index, device) { char buf[IB_DEVICE_NAME_MAX]; if (sscanf(dev_name(&device->dev), name, &i) != 1) @@ -256,6 +292,7 @@ static void ib_device_release(struct device *device) ib_security_release_port_pkey_list(dev); kfree(dev->port_pkey_list); kfree(dev->port_immutable); + xa_destroy(&dev->client_data); kfree(dev); } @@ -306,8 +343,11 @@ struct ib_device *_ib_alloc_device(size_t size) INIT_LIST_HEAD(&device->event_handler_list); spin_lock_init(&device->event_handler_lock); - rwlock_init(&device->client_data_lock); - INIT_LIST_HEAD(&device->client_data_list); + /* + * client_data needs to be alloc because we don't want our mark to be + * destroyed if the user stores NULL in the client data. + */ + xa_init_flags(&device->client_data, XA_FLAGS_ALLOC); INIT_LIST_HEAD(&device->port_list); init_completion(&device->unreg_completion); @@ -323,7 +363,7 @@ EXPORT_SYMBOL(_ib_alloc_device); */ void ib_dealloc_device(struct ib_device *device) { - WARN_ON(!list_empty(&device->client_data_list)); + WARN_ON(!xa_empty(&device->client_data)); WARN_ON(refcount_read(&device->refcount)); rdma_restrack_clean(device); put_device(&device->dev); @@ -332,26 +372,20 @@ EXPORT_SYMBOL(ib_dealloc_device); static int add_client_context(struct ib_device *device, struct ib_client *client) { - struct ib_client_data *context; + void *entry; if (!device->kverbs_provider && !client->no_kverbs_req) return -EOPNOTSUPP; - context = kmalloc(sizeof(*context), GFP_KERNEL); - if (!context) - return -ENOMEM; - - context->client = client; - context->data = NULL; - context->going_down = false; - down_write(&lists_rwsem); - write_lock_irq(&device->client_data_lock); - list_add(&context->list, &device->client_data_list); - write_unlock_irq(&device->client_data_lock); + entry = xa_store(&device->client_data, client->client_id, NULL, + GFP_KERNEL); + if (!xa_is_err(entry)) + xa_set_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED); up_write(&lists_rwsem); - return 0; + return xa_err(entry); } static int verify_immutable(const struct ib_device *dev, u8 port) @@ -428,9 +462,10 @@ static int setup_port_pkey_list(struct ib_device *device) static void ib_policy_change_task(struct work_struct *work) { struct ib_device *dev; + unsigned long index; down_read(&lists_rwsem); - list_for_each_entry(dev, &device_list, core_list) { + xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { int i; for (i = rdma_start_port(dev); i <= rdma_end_port(dev); i++) { @@ -460,28 +495,48 @@ static int ib_security_change(struct notifier_block *nb, unsigned long event, return NOTIFY_OK; } -/** - * __dev_new_index - allocate an device index - * - * Returns a suitable unique value for a new device interface - * number. It assumes that there are less than 2^32-1 ib devices - * will be present in the system. +/* + * Assign the unique string device name and the unique device index. */ -static u32 __dev_new_index(void) +static int assign_name(struct ib_device *device, const char *name) { - /* - * The device index to allow stable naming. - * Similar to struct net -> ifindex. - */ - static u32 index; + static u32 last_id; + int ret; - for (;;) { - if (!(++index)) - index = 1; + /* Assign a unique name to the device */ + if (strchr(name, '%')) + ret = alloc_name(device, name); + else + ret = dev_set_name(&device->dev, name); + if (ret) + goto out; + + if (__ib_device_get_by_name(dev_name(&device->dev))) { + ret = -ENFILE; + goto out; + } + strlcpy(device->name, dev_name(&device->dev), IB_DEVICE_NAME_MAX); - if (!__ib_device_get_by_index(index)) - return index; + /* Cyclically allocate a user visible ID for the device */ + device->index = last_id; + ret = xa_alloc(&devices, &device->index, INT_MAX, device, GFP_KERNEL); + if (ret == -ENOSPC) { + device->index = 0; + ret = xa_alloc(&devices, &device->index, INT_MAX, device, + GFP_KERNEL); } + if (ret) + goto out; + last_id = device->index + 1; + + ret = 0; +out: + return ret; +} + +static void release_name(struct ib_device *device) +{ + xa_erase(&devices, device->index); } static void setup_dma_device(struct ib_device *device) @@ -571,34 +626,21 @@ int ib_register_device(struct ib_device *device, const char *name) mutex_lock(&device_mutex); - if (strchr(name, '%')) { - ret = alloc_name(device, name); - if (ret) - goto out; - } else { - ret = dev_set_name(&device->dev, name); - if (ret) - goto out; - } - if (__ib_device_get_by_name(dev_name(&device->dev))) { - ret = -ENFILE; + ret = assign_name(device, name); + if (ret) goto out; - } - strlcpy(device->name, dev_name(&device->dev), IB_DEVICE_NAME_MAX); ret = setup_device(device); if (ret) - goto out; + goto out_name; ret = ib_cache_setup_one(device); if (ret) { dev_warn(&device->dev, "Couldn't set up InfiniBand P_Key/GID cache\n"); - goto out; + goto out_name; } - device->index = __dev_new_index(); - ib_device_register_rdmacg(device); ret = ib_device_register_sysfs(device); @@ -615,7 +657,7 @@ int ib_register_device(struct ib_device *device, const char *name) client->add(device); down_write(&lists_rwsem); - list_add_tail(&device->core_list, &device_list); + xa_set_mark(&devices, device->index, DEVICE_REGISTERED); up_write(&lists_rwsem); mutex_unlock(&device_mutex); return 0; @@ -623,6 +665,8 @@ int ib_register_device(struct ib_device *device, const char *name) cg_cleanup: ib_device_unregister_rdmacg(device); ib_cache_cleanup_one(device); +out_name: + release_name(device); out: mutex_unlock(&device_mutex); return ret; @@ -637,8 +681,8 @@ EXPORT_SYMBOL(ib_register_device); */ void ib_unregister_device(struct ib_device *device) { - struct ib_client_data *context, *tmp; - unsigned long flags; + struct ib_client *client; + unsigned long index; /* * Wait for all netlink command callers to finish working on the @@ -650,34 +694,31 @@ void ib_unregister_device(struct ib_device *device) mutex_lock(&device_mutex); down_write(&lists_rwsem); - list_del(&device->core_list); - write_lock_irq(&device->client_data_lock); - list_for_each_entry(context, &device->client_data_list, list) - context->going_down = true; - write_unlock_irq(&device->client_data_lock); + xa_clear_mark(&devices, device->index, DEVICE_REGISTERED); + xa_for_each (&clients, index, client) + xa_clear_mark(&device->client_data, index, + CLIENT_DATA_REGISTERED); downgrade_write(&lists_rwsem); - list_for_each_entry(context, &device->client_data_list, list) { - if (context->client->remove) - context->client->remove(device, context->data); - } + list_for_each_entry_reverse(client, &client_list, list) + if (xa_get_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED) && + client->remove) + client->remove(device, xa_load(&device->client_data, + client->client_id)); up_read(&lists_rwsem); ib_device_unregister_sysfs(device); ib_device_unregister_rdmacg(device); + release_name(device); + mutex_unlock(&device_mutex); ib_cache_cleanup_one(device); down_write(&lists_rwsem); - write_lock_irqsave(&device->client_data_lock, flags); - list_for_each_entry_safe(context, tmp, &device->client_data_list, - list) { - list_del(&context->list); - kfree(context); - } - write_unlock_irqrestore(&device->client_data_lock, flags); + xa_destroy(&device->client_data); up_write(&lists_rwsem); } EXPORT_SYMBOL(ib_unregister_device); @@ -724,6 +765,7 @@ static int assign_client_id(struct ib_client *client) int ib_register_client(struct ib_client *client) { struct ib_device *device; + unsigned long index; int ret; mutex_lock(&device_mutex); @@ -733,7 +775,7 @@ int ib_register_client(struct ib_client *client) return ret; } - list_for_each_entry(device, &device_list, core_list) + xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) if (!add_client_context(device, client) && client->add) client->add(device); @@ -757,8 +799,8 @@ EXPORT_SYMBOL(ib_register_client); */ void ib_unregister_client(struct ib_client *client) { - struct ib_client_data *context; struct ib_device *device; + unsigned long index; mutex_lock(&device_mutex); @@ -766,37 +808,19 @@ void ib_unregister_client(struct ib_client *client) xa_clear_mark(&clients, client->client_id, CLIENT_REGISTERED); up_write(&lists_rwsem); - list_for_each_entry(device, &device_list, core_list) { - struct ib_client_data *found_context = NULL; - + xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) { down_write(&lists_rwsem); - write_lock_irq(&device->client_data_lock); - list_for_each_entry(context, &device->client_data_list, list) - if (context->client == client) { - context->going_down = true; - found_context = context; - break; - } - write_unlock_irq(&device->client_data_lock); + xa_clear_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED); up_write(&lists_rwsem); if (client->remove) - client->remove(device, found_context ? - found_context->data : NULL); - - if (!found_context) { - dev_warn(&device->dev, - "No client context found for %s\n", - client->name); - continue; - } + client->remove(device, xa_load(&device->client_data, + client->client_id)); down_write(&lists_rwsem); - write_lock_irq(&device->client_data_lock); - list_del(&found_context->list); - write_unlock_irq(&device->client_data_lock); + xa_erase(&device->client_data, client->client_id); up_write(&lists_rwsem); - kfree(found_context); } down_write(&lists_rwsem); @@ -807,59 +831,28 @@ void ib_unregister_client(struct ib_client *client) } EXPORT_SYMBOL(ib_unregister_client); -/** - * ib_get_client_data - Get IB client context - * @device:Device to get context for - * @client:Client to get context for - * - * ib_get_client_data() returns client context set with - * ib_set_client_data(). - */ -void *ib_get_client_data(struct ib_device *device, struct ib_client *client) -{ - struct ib_client_data *context; - void *ret = NULL; - unsigned long flags; - - read_lock_irqsave(&device->client_data_lock, flags); - list_for_each_entry(context, &device->client_data_list, list) - if (context->client == client) { - ret = context->data; - break; - } - read_unlock_irqrestore(&device->client_data_lock, flags); - - return ret; -} -EXPORT_SYMBOL(ib_get_client_data); - /** * ib_set_client_data - Set IB client context * @device:Device to set context for * @client:Client to set context for * @data:Context to set * - * ib_set_client_data() sets client context that can be retrieved with - * ib_get_client_data(). + * ib_set_client_data() sets client context data that can be retrieved with + * ib_get_client_data(). This can only be called while the client is + * registered to the device, once the ib_client remove() callback returns this + * cannot be called. */ void ib_set_client_data(struct ib_device *device, struct ib_client *client, void *data) { - struct ib_client_data *context; - unsigned long flags; - - write_lock_irqsave(&device->client_data_lock, flags); - list_for_each_entry(context, &device->client_data_list, list) - if (context->client == client) { - context->data = data; - goto out; - } + void *rc; - dev_warn(&device->dev, "No client context found for %s\n", - client->name); + if (WARN_ON(IS_ERR(data))) + data = NULL; -out: - write_unlock_irqrestore(&device->client_data_lock, flags); + rc = xa_store(&device->client_data, client->client_id, data, + GFP_KERNEL); + WARN_ON(xa_is_err(rc)); } EXPORT_SYMBOL(ib_set_client_data); @@ -1017,9 +1010,10 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, void *cookie) { struct ib_device *dev; + unsigned long index; down_read(&lists_rwsem); - list_for_each_entry(dev, &device_list, core_list) + xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) ib_enum_roce_netdev(dev, filter, filter_cookie, cb, cookie); up_read(&lists_rwsem); } @@ -1033,12 +1027,13 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, int ib_enum_all_devs(nldev_callback nldev_cb, struct sk_buff *skb, struct netlink_callback *cb) { + unsigned long index; struct ib_device *dev; unsigned int idx = 0; int ret = 0; down_read(&lists_rwsem); - list_for_each_entry(dev, &device_list, core_list) { + xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { ret = nldev_cb(dev, skb, cb, idx); if (ret) break; @@ -1211,26 +1206,25 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, const struct sockaddr *addr) { struct net_device *net_dev = NULL; - struct ib_client_data *context; + unsigned long index; + void *client_data; if (!rdma_protocol_ib(dev, port)) return NULL; down_read(&lists_rwsem); - list_for_each_entry(context, &dev->client_data_list, list) { - struct ib_client *client = context->client; + xan_for_each (&dev->client_data, index, client_data, + CLIENT_DATA_REGISTERED) { + struct ib_client *client = xa_load(&clients, index); - if (context->going_down) + if (!client || !client->get_net_dev_by_params) continue; - if (client->get_net_dev_by_params) { - net_dev = client->get_net_dev_by_params(dev, port, pkey, - gid, addr, - context->data); - if (net_dev) - break; - } + net_dev = client->get_net_dev_by_params(dev, port, pkey, gid, + addr, client_data); + if (net_dev) + break; } up_read(&lists_rwsem); @@ -1457,6 +1451,7 @@ static void __exit ib_core_cleanup(void) /* Make sure that any pending umem accounting work is done. */ destroy_workqueue(ib_wq); WARN_ON(!xa_empty(&clients)); + WARN_ON(!xa_empty(&devices)); } MODULE_ALIAS_RDMA_NETLINK(RDMA_NL_LS, 4); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 84d7523984ee77..7781eefd8b84eb 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2528,12 +2528,7 @@ struct ib_device { struct list_head event_handler_list; spinlock_t event_handler_lock; - rwlock_t client_data_lock; - struct list_head core_list; - /* Access to the client_data_list is protected by the client_data_lock - * rwlock and the lists_rwsem read-write semaphore - */ - struct list_head client_data_list; + struct xarray client_data; struct ib_cache cache; /** @@ -2646,7 +2641,21 @@ void ib_unregister_device(struct ib_device *device); int ib_register_client (struct ib_client *client); void ib_unregister_client(struct ib_client *client); -void *ib_get_client_data(struct ib_device *device, struct ib_client *client); +/** + * ib_get_client_data - Get IB client context + * @device:Device to get context for + * @client:Client to get context for + * + * ib_get_client_data() returns the client context data set with + * ib_set_client_data(). This can only be called while the client is + * registered to the device, once the ib_client remove() callback returns this + * cannot be called. + */ +static inline void *ib_get_client_data(struct ib_device *device, + struct ib_client *client) +{ + return xa_load(&device->client_data, client->client_id); +} void ib_set_client_data(struct ib_device *device, struct ib_client *client, void *data); void ib_set_device_ops(struct ib_device *device, From patchwork Mon Feb 4 21:17:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10796651 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C9976C2 for ; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B3962B1EB for ; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0FC792B5B8; Mon, 4 Feb 2019 21:18:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B7C212B1EB for ; Mon, 4 Feb 2019 21:18:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729637AbfBDVSF (ORCPT ); Mon, 4 Feb 2019 16:18:05 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:42831 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729514AbfBDVSE (ORCPT ); Mon, 4 Feb 2019 16:18:04 -0500 Received: by mail-pl1-f195.google.com with SMTP id s1so511765plp.9 for ; Mon, 04 Feb 2019 13:18:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=28M4/10gtyyrzzW2MMYryWVBzjvVOOFC/W5yKIj1ePU=; b=hy46J7SVUlHuSWLAXrv6i3WUgQu+pUNPlRDdq/dg0ailyIQpFNLeGdcTaTRQXcatfP MbW4i3p+w026IcjOy21tf8cibmrKYIIU//3XAr6kx0P8tZ41OHkf7EOd49Vtcc+soLDi reI76oZyDPnDyxgg6OohBIRLjNukqBXbVh+Nr/C6nFEpSaEE4+P+KVxZVRPyUJLahu36 +Sp6gg3t18OrRvzYnAf3YIWa9E8Mb+h19HTo1Ha+MCi4CrCxzR1puei5fYyxgONdmA0X Gvi7fkMr1Jlt/vwtzd7haXpa1453YRftoDFciUR+9Jy4iMt/Foh0rvN6EDJlGv1Z35Fv 5/lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=28M4/10gtyyrzzW2MMYryWVBzjvVOOFC/W5yKIj1ePU=; b=BPr9ZbC/u0C1boRiipoPPpDPKOv/WilrhSEuLrmSOtaROMbp9WDGZJgSuHNHAyawJA UL+/TzYM5n2R2s8mNYrOqdsgru9zA5ToriyjMQEAdOXYdeBTzKKLc7jU84Ky8O7DnaZS xNa1oXXb0oM2WjXZleYnOsNRFUhJFUVf5xq73qDOJza5g71LNH+kaWwsp4SaVwVjDxZ3 y9sWkF0DTmQ0U8F89V/bI/MFIPAykIOXi92jCOEt9zHcJkscd+hup+PGMesFXttmw2sI ymiiTQbPuqdVv+rFcSLZFQJzSJmqmxLGERf3JPZApcDP2U7SzZkZBCaucgvPsu64/szz I4pg== X-Gm-Message-State: AHQUAuZuGsJACmRvaPhGJuTIfdicFXCESdeGjdMmFSIKaZvnvCx0dTBD I2GD68HSwflAfKVmI3aNd7e8XoTmalA= X-Google-Smtp-Source: AHgI3IY6RAwQwsvJYWktjsk/tJgPnVKLHuENFfsfc6NMz/+ljVCOeZoUmwZPP4FKaUU6la55P8O8sw== X-Received: by 2002:a17:902:4503:: with SMTP id m3mr1465068pld.23.1549315083235; Mon, 04 Feb 2019 13:18:03 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id t12sm1327756pfi.45.2019.02.04.13.18.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 04 Feb 2019 13:18:02 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gqlct-0005Kx-Kd; Mon, 04 Feb 2019 14:17:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH 8/8] RDMA/devices: Re-organize device.c locking Date: Mon, 4 Feb 2019 14:17:51 -0700 Message-Id: <20190204211751.19001-9-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204211751.19001-1-jgg@ziepe.ca> References: <20190204211751.19001-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe The locking here started out with a single lock that covered everything and then has lately veered into crazy town. The fundamental problem is that several places need to iterate over a linked list, but also need to drop their locks to avoid deadlock during client callbacks. xarray's restartable iteration offers a simple solution to the problem. Once all the lists are xarrays we can drop locks in the places that need that and rely on xarray to provide consistency and locking for the data structure. The resulting simplification is that each of the three lists has a dedicated rwsem that must be held when working with the list it covers. One data structure is no longer covered by multiple locks. The sleeping semaphore is selected because the read side generally needs to be held over something sleeping, and using RCU reader locking in those cases is overkill. In the process this simplifies the entire registration/unregistration flow to be the expected list of setups and the reversed list of matching teardowns, and the registration lock 'refcount' can now be revised to be released after the ULPs are removed, providing a very sane semantic for this feature. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/device.c | 361 +++++++++++++++++++------------ include/rdma/ib_verbs.h | 1 + 2 files changed, 222 insertions(+), 140 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 7eca7c4bd4071f..a19243c514e5e6 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -37,7 +37,6 @@ #include #include #include -#include #include #include #include @@ -56,6 +55,29 @@ struct workqueue_struct *ib_comp_unbound_wq; struct workqueue_struct *ib_wq; EXPORT_SYMBOL_GPL(ib_wq); +/* + * Each of the three rwsem locks (devices, clients, client_data) protects the + * xarray of the same name. Specifically it allows the caller to assert that + * the MARK will/will not be changing under the lock, and for devices and + * clients, that the value in the xarray is still a valid pointer. Change of + * the MARK is linked to the object state, so holding the lock and testing the + * MARK also asserts that the contained object is in a certain state. + * + * This is used to build a two stage register/unregister flow where objects + * can continue to be in the xarray even though they are still in progress to + * register/unregister. + * + * The xarray itself provides additional locking, and restartable iteration, + * which is also relied on. + * + * Locks should not be nested, with the exception of client_data, which is + * allowed to nest under the read side of the other two locks. + * + * The devices_rwsem also protects the device name list, any change or + * assignment of device name must also hold the write side to guarantee unique + * names. + */ + /* * devices contains devices that have had their names assigned. The * devices may not be registered. Users that care about the registration @@ -64,17 +86,13 @@ EXPORT_SYMBOL_GPL(ib_wq); * */ static DEFINE_XARRAY_FLAGS(devices, XA_FLAGS_ALLOC); - -/* - * Note that if the *rwsem is held and the *_REGISTERED mark is seen then the - * object is guaranteed to be and remain registered for the duration of the - * lock. - */ +static DECLARE_RWSEM(devices_rwsem); #define DEVICE_REGISTERED XA_MARK_1 static LIST_HEAD(client_list); #define CLIENT_REGISTERED XA_MARK_1 static DEFINE_XARRAY_FLAGS(clients, XA_FLAGS_ALLOC); +static DECLARE_RWSEM(clients_rwsem); /* * If client_data is registered then the corresponding client must also still @@ -118,20 +136,6 @@ static void *xan_find(struct xarray *xa, unsigned long *indexp, !IS_ERR(entry); \ (index)++, entry = xan_find(xa, &(index), ULONG_MAX, filter)) -/* - * device_mutex and lists_rwsem protect access to both devices and - * clients. device_mutex protects writer access by device and client - * registration / de-registration. lists_rwsem protects reader access to - * these lists. Iterators of these lists must lock it for read, while updates - * to the lists must be done with a write lock. A special case is when the - * device_mutex is locked. In this case locking the lists for read access is - * not necessary as the device_mutex implies it. - * - * lists_rwsem also protects access to the client data list. - */ -static DEFINE_MUTEX(device_mutex); -static DECLARE_RWSEM(lists_rwsem); - static int ib_security_change(struct notifier_block *nb, unsigned long event, void *lsm_data); static void ib_policy_change_task(struct work_struct *work); @@ -188,13 +192,13 @@ struct ib_device *ib_device_get_by_index(u32 index) { struct ib_device *device; - down_read(&lists_rwsem); + down_read(&devices_rwsem); device = xa_load(&devices, index); if (device) { if (!ib_device_try_get(device)) device = NULL; } - up_read(&lists_rwsem); + up_read(&devices_rwsem); return device; } @@ -228,7 +232,7 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) { int ret; - mutex_lock(&device_mutex); + down_write(&devices_rwsem); if (!strcmp(name, dev_name(&ibdev->dev))) { ret = -EEXIST; goto out; @@ -244,7 +248,7 @@ int ib_device_rename(struct ib_device *ibdev, const char *name) goto out; strlcpy(ibdev->name, name, IB_DEVICE_NAME_MAX); out: - mutex_unlock(&device_mutex); + up_write(&devices_rwsem); return ret; } @@ -256,6 +260,7 @@ static int alloc_name(struct ib_device *ibdev, const char *name) int rc; int i; + lockdep_assert_held_exclusive(&devices_rwsem); ida_init(&inuse); xa_for_each (&devices, index, device) { char buf[IB_DEVICE_NAME_MAX]; @@ -348,6 +353,7 @@ struct ib_device *_ib_alloc_device(size_t size) * destroyed if the user stores NULL in the client data. */ xa_init_flags(&device->client_data, XA_FLAGS_ALLOC); + init_rwsem(&device->client_data_rwsem); INIT_LIST_HEAD(&device->port_list); init_completion(&device->unreg_completion); @@ -370,22 +376,86 @@ void ib_dealloc_device(struct ib_device *device) } EXPORT_SYMBOL(ib_dealloc_device); -static int add_client_context(struct ib_device *device, struct ib_client *client) +/* + * add_client_context() and remove_client_context() must be safe against + * parallel calls on the same device - registration/unregistration of both the + * device and client can be occurring in parallel. + * + * The routines need to be a fence, any caller must not return until the add + * or remove is fully completed. + */ +static int add_client_context(struct ib_device *device, + struct ib_client *client) { - void *entry; + int ret = 0; if (!device->kverbs_provider && !client->no_kverbs_req) - return -EOPNOTSUPP; + return 0; + + down_write(&device->client_data_rwsem); + /* + * Another caller to add_client_context got here first and has already + * completely initialized context. + */ + if (xa_get_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED)) + goto out; + + ret = xa_err(xa_store(&device->client_data, client->client_id, NULL, + GFP_KERNEL)); + if (ret) + goto out; + downgrade_write(&device->client_data_rwsem); + if (client->add) + client->add(device); + + /* Readers shall not see a client until add has been completed */ + xa_set_mark(&device->client_data, client->client_id, + CLIENT_DATA_REGISTERED); + up_read(&device->client_data_rwsem); + return 0; + +out: + up_write(&device->client_data_rwsem); + return ret; +} + +static void remove_client_context(struct ib_device *device, + unsigned int client_id) +{ + struct ib_client *client; + void *client_data; - down_write(&lists_rwsem); - entry = xa_store(&device->client_data, client->client_id, NULL, - GFP_KERNEL); - if (!xa_is_err(entry)) - xa_set_mark(&device->client_data, client->client_id, - CLIENT_DATA_REGISTERED); - up_write(&lists_rwsem); + down_write(&device->client_data_rwsem); + if (!xa_get_mark(&device->client_data, client_id, + CLIENT_DATA_REGISTERED)) { + up_write(&device->client_data_rwsem); + return; + } + client_data = xa_load(&device->client_data, client_id); + xa_clear_mark(&device->client_data, client_id, CLIENT_DATA_REGISTERED); + client = xa_load(&clients, client_id); + downgrade_write(&device->client_data_rwsem); - return xa_err(entry); + /* + * Notice we cannot be holding any exclusive locks when calling the + * remove callback as the remove callback can recurse back into any + * public functions in this module and thus try for any locks those + * functions take. + * + * For this reason clients and drivers should not call the + * unregistration functions will holdling any locks. + * + * It tempting to drop the client_data_rwsem too, but this is required + * to ensure that unregister_client does not return until all clients + * are completely unregistered, which is required to avoid module + * unloading races. + */ + if (client->remove) + client->remove(device, client_data); + + xa_erase(&device->client_data, client_id); + up_read(&device->client_data_rwsem); } static int verify_immutable(const struct ib_device *dev, u8 port) @@ -464,7 +534,7 @@ static void ib_policy_change_task(struct work_struct *work) struct ib_device *dev; unsigned long index; - down_read(&lists_rwsem); + down_read(&devices_rwsem); xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { int i; @@ -481,7 +551,7 @@ static void ib_policy_change_task(struct work_struct *work) ib_security_cache_change(dev, i, sp); } } - up_read(&lists_rwsem); + up_read(&devices_rwsem); } static int ib_security_change(struct notifier_block *nb, unsigned long event, @@ -503,6 +573,7 @@ static int assign_name(struct ib_device *device, const char *name) static u32 last_id; int ret; + down_write(&devices_rwsem); /* Assign a unique name to the device */ if (strchr(name, '%')) ret = alloc_name(device, name); @@ -530,13 +601,17 @@ static int assign_name(struct ib_device *device, const char *name) last_id = device->index + 1; ret = 0; + out: + up_write(&devices_rwsem); return ret; } static void release_name(struct ib_device *device) { + down_write(&devices_rwsem); xa_erase(&devices, device->index); + up_write(&devices_rwsem); } static void setup_dma_device(struct ib_device *device) @@ -574,11 +649,18 @@ static void setup_dma_device(struct ib_device *device) } } +/* + * setup_device() allocates memory and sets up data that requires calling the + * device ops, this is the only reason these actions are not done during + * ib_alloc_device. It is undone by ib_dealloc_device(). + */ static int setup_device(struct ib_device *device) { struct ib_udata uhw = {.outlen = 0, .inlen = 0}; int ret; + setup_dma_device(device); + ret = ib_device_check_mandatory(device); if (ret) return ret; @@ -607,6 +689,54 @@ static int setup_device(struct ib_device *device) return 0; } +static void disable_device(struct ib_device *device) +{ + struct ib_client *client; + + WARN_ON(!refcount_read(&device->refcount)); + + down_write(&devices_rwsem); + xa_clear_mark(&devices, device->index, DEVICE_REGISTERED); + up_write(&devices_rwsem); + + down_read(&clients_rwsem); + list_for_each_entry_reverse(client, &client_list, list) + remove_client_context(device, client->client_id); + up_read(&clients_rwsem); + + /* Pairs with refcount_set in enable_device */ + ib_device_put(device); + wait_for_completion(&device->unreg_completion); +} + +/* + * An enabled device is visible to all clients and to all the public facing + * APIs that return a device pointer. + */ +static int enable_device(struct ib_device *device) +{ + struct ib_client *client; + unsigned long index; + int ret; + + refcount_set(&device->refcount, 1); + down_write(&devices_rwsem); + xa_set_mark(&devices, device->index, DEVICE_REGISTERED); + up_write(&devices_rwsem); + + down_read(&clients_rwsem); + xa_for_each_marked (&clients, index, client, CLIENT_REGISTERED) { + ret = add_client_context(device, client); + if (ret) { + up_read(&clients_rwsem); + disable_device(device); + return ret; + } + } + up_read(&clients_rwsem); + return 0; +} + /** * ib_register_device - Register an IB device with IB core * @device:Device to register @@ -619,26 +749,20 @@ static int setup_device(struct ib_device *device) int ib_register_device(struct ib_device *device, const char *name) { int ret; - struct ib_client *client; - unsigned long index; - - setup_dma_device(device); - - mutex_lock(&device_mutex); ret = assign_name(device, name); if (ret) - goto out; + return ret; ret = setup_device(device); if (ret) - goto out_name; + goto out; ret = ib_cache_setup_one(device); if (ret) { dev_warn(&device->dev, "Couldn't set up InfiniBand P_Key/GID cache\n"); - goto out_name; + goto out; } ib_device_register_rdmacg(device); @@ -650,25 +774,19 @@ int ib_register_device(struct ib_device *device, const char *name) goto cg_cleanup; } - refcount_set(&device->refcount, 1); - - xa_for_each_marked (&clients, index, client, CLIENT_REGISTERED) - if (!add_client_context(device, client) && client->add) - client->add(device); + ret = enable_device(device); + if (ret) + goto sysfs_cleanup; - down_write(&lists_rwsem); - xa_set_mark(&devices, device->index, DEVICE_REGISTERED); - up_write(&lists_rwsem); - mutex_unlock(&device_mutex); return 0; +sysfs_cleanup: + ib_device_unregister_sysfs(device); cg_cleanup: ib_device_unregister_rdmacg(device); ib_cache_cleanup_one(device); -out_name: - release_name(device); out: - mutex_unlock(&device_mutex); + release_name(device); return ret; } EXPORT_SYMBOL(ib_register_device); @@ -681,45 +799,11 @@ EXPORT_SYMBOL(ib_register_device); */ void ib_unregister_device(struct ib_device *device) { - struct ib_client *client; - unsigned long index; - - /* - * Wait for all netlink command callers to finish working on the - * device. - */ - ib_device_put(device); - wait_for_completion(&device->unreg_completion); - - mutex_lock(&device_mutex); - - down_write(&lists_rwsem); - xa_clear_mark(&devices, device->index, DEVICE_REGISTERED); - xa_for_each (&clients, index, client) - xa_clear_mark(&device->client_data, index, - CLIENT_DATA_REGISTERED); - downgrade_write(&lists_rwsem); - - list_for_each_entry_reverse(client, &client_list, list) - if (xa_get_mark(&device->client_data, client->client_id, - CLIENT_DATA_REGISTERED) && - client->remove) - client->remove(device, xa_load(&device->client_data, - client->client_id)); - up_read(&lists_rwsem); - + disable_device(device); ib_device_unregister_sysfs(device); ib_device_unregister_rdmacg(device); - - release_name(device); - - mutex_unlock(&device_mutex); - ib_cache_cleanup_one(device); - - down_write(&lists_rwsem); - xa_destroy(&device->client_data); - up_write(&lists_rwsem); + release_name(device); } EXPORT_SYMBOL(ib_unregister_device); @@ -727,6 +811,7 @@ static int assign_client_id(struct ib_client *client) { int ret; + down_write(&clients_rwsem); /* * The add/remove callbacks must be called in FIFO/LIFO order. To * achieve this we assign client_ids so they are sorted in @@ -745,7 +830,11 @@ static int assign_client_id(struct ib_client *client) if (ret) goto out; + xa_set_mark(&clients, client->client_id, CLIENT_REGISTERED); + list_add_tail(&client->list, &client_list); + out: + up_write(&clients_rwsem); return ret; } @@ -768,23 +857,20 @@ int ib_register_client(struct ib_client *client) unsigned long index; int ret; - mutex_lock(&device_mutex); ret = assign_client_id(client); - if (ret) { - mutex_unlock(&device_mutex); + if (ret) return ret; - } - - xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) - if (!add_client_context(device, client) && client->add) - client->add(device); - - down_write(&lists_rwsem); - xa_set_mark(&clients, client->client_id, CLIENT_REGISTERED); - up_write(&lists_rwsem); - - mutex_unlock(&device_mutex); + down_read(&devices_rwsem); + xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) { + ret = add_client_context(device, client); + if (ret) { + up_read(&devices_rwsem); + ib_unregister_client(client); + return ret; + } + } + up_read(&devices_rwsem); return 0; } EXPORT_SYMBOL(ib_register_client); @@ -796,38 +882,31 @@ EXPORT_SYMBOL(ib_register_client); * Upper level users use ib_unregister_client() to remove their client * registration. When ib_unregister_client() is called, the client * will receive a remove callback for each IB device still registered. + * + * This is a full fence, once it returns no client callbacks will be called, + * or are running in another thread. */ void ib_unregister_client(struct ib_client *client) { struct ib_device *device; unsigned long index; - mutex_lock(&device_mutex); - - down_write(&lists_rwsem); + down_write(&clients_rwsem); xa_clear_mark(&clients, client->client_id, CLIENT_REGISTERED); - up_write(&lists_rwsem); - - xa_for_each_marked (&devices, index, device, DEVICE_REGISTERED) { - down_write(&lists_rwsem); - xa_clear_mark(&device->client_data, client->client_id, - CLIENT_DATA_REGISTERED); - up_write(&lists_rwsem); - - if (client->remove) - client->remove(device, xa_load(&device->client_data, - client->client_id)); - - down_write(&lists_rwsem); - xa_erase(&device->client_data, client->client_id); - up_write(&lists_rwsem); - } + up_write(&clients_rwsem); + /* + * Every device still known must be serialized to make sure we are + * done with the client callbacks before we return. + */ + down_read(&devices_rwsem); + xa_for_each (&devices, index, device) + remove_client_context(device, client->client_id); + up_read(&devices_rwsem); - down_write(&lists_rwsem); + down_write(&clients_rwsem); list_del(&client->list); xa_erase(&clients, client->client_id); - up_write(&lists_rwsem); - mutex_unlock(&device_mutex); + up_write(&clients_rwsem); } EXPORT_SYMBOL(ib_unregister_client); @@ -1012,10 +1091,10 @@ void ib_enum_all_roce_netdevs(roce_netdev_filter filter, struct ib_device *dev; unsigned long index; - down_read(&lists_rwsem); + down_read(&devices_rwsem); xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) ib_enum_roce_netdev(dev, filter, filter_cookie, cb, cookie); - up_read(&lists_rwsem); + up_read(&devices_rwsem); } /** @@ -1032,15 +1111,14 @@ int ib_enum_all_devs(nldev_callback nldev_cb, struct sk_buff *skb, unsigned int idx = 0; int ret = 0; - down_read(&lists_rwsem); + down_read(&devices_rwsem); xa_for_each_marked (&devices, index, dev, DEVICE_REGISTERED) { ret = nldev_cb(dev, skb, cb, idx); if (ret) break; idx++; } - - up_read(&lists_rwsem); + up_read(&devices_rwsem); return ret; } @@ -1198,6 +1276,7 @@ EXPORT_SYMBOL(ib_find_pkey); * @gid: A GID that the net_dev uses to communicate. * @addr: Contains the IP address that the request specified as its * destination. + * */ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, u8 port, @@ -1212,8 +1291,11 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, if (!rdma_protocol_ib(dev, port)) return NULL; - down_read(&lists_rwsem); - + /* + * Holding the read side guarantees that the client will not become + * unregistered while we are calling get_net_dev_by_params() + */ + down_read(&dev->client_data_rwsem); xan_for_each (&dev->client_data, index, client_data, CLIENT_DATA_REGISTERED) { struct ib_client *client = xa_load(&clients, index); @@ -1226,8 +1308,7 @@ struct net_device *ib_get_net_dev_by_params(struct ib_device *dev, if (net_dev) break; } - - up_read(&lists_rwsem); + up_read(&dev->client_data_rwsem); return net_dev; } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 7781eefd8b84eb..bb85045765029c 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2528,6 +2528,7 @@ struct ib_device { struct list_head event_handler_list; spinlock_t event_handler_lock; + struct rw_semaphore client_data_rwsem; struct xarray client_data; struct ib_cache cache;