From patchwork Thu Feb 7 05:41:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10800385 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC612922 for ; Thu, 7 Feb 2019 05:42:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8F522CBDD for ; Thu, 7 Feb 2019 05:42:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9767F2CBF1; Thu, 7 Feb 2019 05:42:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3CFDF2CBD8 for ; Thu, 7 Feb 2019 05:42:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726699AbfBGFmB (ORCPT ); Thu, 7 Feb 2019 00:42:01 -0500 Received: from mail-pf1-f195.google.com ([209.85.210.195]:33327 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726128AbfBGFmB (ORCPT ); Thu, 7 Feb 2019 00:42:01 -0500 Received: by mail-pf1-f195.google.com with SMTP id c123so4300572pfb.0 for ; Wed, 06 Feb 2019 21:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=jKQwiXGkk6PEd14TJIkdKffMq3F3yUEfn2h5QVdvuH0=; b=A3eRSiiiJxX3/uhJOKtkCwROa2WTJF7XtfEB9WLbHY6ya/TEnvPiDgebqyLi0Tk8xd QeiOqkjhTJ28Cd8Ayq2HwFBz752TtJn4vGSUWSI/WH2wa3D/5JKIU3KS+peDzeHwMtcI HGMSaFqcaSRdbRSeA+yziyWi3hxvpoC9TYkYVceUKR5sVB71MRVEuXFA2q6k9kYIiP9Q hSgt78lXKCtWh0Py4y2K1y1jhlFolbDeKZBdGSMLIw7XIDh0dDfD4Nm9QO+Tlk/zi743 xGjLJyY57u7Rzyd0bUY2HKlGGfKc3jtqtLR4sI68j6gLFl4zm6pUuvktZiB4CCO1rvW+ /UxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=jKQwiXGkk6PEd14TJIkdKffMq3F3yUEfn2h5QVdvuH0=; b=VvY+3O3vob9xlQ8+1fsb8MaS4hc4RRB+bse1Fi81eswIf5+zbFY9+9TadooQYYSS3J lZhID3HSqx/hoE5Zi/WHutrUJ1YlRHYHbSK+C2Q7LhDnXQhDDWxOrqZamjoUxuTTHheF 0iR+Rs6PpMHJd4YU/fKUNu+qnWGLdEWVv5n4dJYtctIe7Hkxznl5Gd8Y3dX0EWcFCj5O PsyA4oDRiTxowfMfYPr0g9CJvHnNWnuchl0DdLLbRWO/RrgmP6GHgwxqeQubzCfYk8LB mtlDugVY5QdiJ+B+b1ZnMduVMDubhKktQ6XtHS+RI4lXNLzbYWUC7t7Kd39weYzs3wkY wrTA== X-Gm-Message-State: AHQUAuZJEpdWJwNKwN+4juPrJimzZHr/S5YEm5KHOavwioILVwqVmiHr kTFsTHF05Yr8cGmGPDSFGlIik6gchS4= X-Google-Smtp-Source: AHgI3IZs+6iufTRO48BUp6U2BOLCrRaBbnUT3sP7yVE5/Z+fNFpmPdnqPMt36yyPNR9ZwY4XOH0ptw== X-Received: by 2002:a62:670f:: with SMTP id b15mr14376047pfc.212.1549518119924; Wed, 06 Feb 2019 21:41:59 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id b2sm11444234pfm.3.2019.02.06.21.41.58 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 06 Feb 2019 21:41:58 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1grcRh-0006DD-Sk; Wed, 06 Feb 2019 22:41:57 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH v2 0/8] Rework device, client and client data locking Date: Wed, 6 Feb 2019 22:41:46 -0700 Message-Id: <20190207054154.23806-1-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe The convoluted locking in this file has become a problem now that we want to do more complicated things with the device registration life cycle. This replaces the scheme with simple approaching using xarray instead of linked lists. Each of the three xarray's has its own lock. Which lock to hold becomes very clear as they only protect one xarray, no more multiple locks protecting the same data. xarray makes this possible because it has restartable iteration which trivially solves the sticky locking problem between client data and client registration. The xarray is a natural fit for these datastructures as we now have an ID number direct lookup for the struct device and we have a direct lookup access pattern for client data. The client list is converted as well as that is a straightforward way to get unique per-client ID numbers for the client data array and keeps the locking scheme symmetrical. This code could be improved with two functional improvement to xarray - iteration over all allocated values (including NULL) and reverse find. For now some work arounds are used and I can explore xarray improvements later. v2: - Change the xarray as discussed (Matthew) - Don't fail rename (Gal) Jason Gunthorpe (8): RDMA/device: Check that the rename is nop under the lock RDMA/device: Ensure that security memory is always freed RDMA/device: Call ib_cache_release_one() only from ib_device_release() RDMA/device: Get rid of reg_state RDMA/device: Use an ida instead of a free page in alloc_name RDMA/devices: Use xarray to store the clients RDMA/devices: Use xarray to store the client_data RDMA/devices: Re-organize device.c locking drivers/infiniband/core/cache.c | 3 + drivers/infiniband/core/core_priv.h | 4 +- drivers/infiniband/core/device.c | 726 ++++++++++++++++------------ drivers/infiniband/core/security.c | 4 +- include/rdma/ib_verbs.h | 33 +- 5 files changed, 436 insertions(+), 334 deletions(-)