From patchwork Wed Feb 13 04:12:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10809095 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F2701390 for ; Wed, 13 Feb 2019 04:13:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 21FD12BFAF for ; Wed, 13 Feb 2019 04:13:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 13E662C06B; Wed, 13 Feb 2019 04:13:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8B09F2BFAF for ; Wed, 13 Feb 2019 04:13:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728772AbfBMENC (ORCPT ); Tue, 12 Feb 2019 23:13:02 -0500 Received: from mail-pg1-f194.google.com ([209.85.215.194]:35714 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726902AbfBMENC (ORCPT ); Tue, 12 Feb 2019 23:13:02 -0500 Received: by mail-pg1-f194.google.com with SMTP id s198so534896pgs.2 for ; Tue, 12 Feb 2019 20:13:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ynhsqQuZumwhCPqGimoq6FfLou2KLT8k5MaJksPwUD4=; b=Or0GsJBTHoYNukiFWTK+ArVhiNR586ooc3h8n7xhgUybyQvUICnZv5eHLs47y5pZr5 tcmZ9aEaqPZ4BYsMhA7qlk6Yc5oFGBHuSWMnNDNN4HuS04qRjl+J7OPsHE1VPtcCbteR K0TPt2zWA3Lw1ZoFvEAgMSYk1DgJipFeBP8Rr1XbaUq2U1S8vVApPBzjsTA8wZnSnm6X G6puasC1UBx6rKn48r6aq1A9H2quMW3je4ycWI+QnqWleyVoD/GDL1SL8jM7UiqvKViH ekB8h91dGvIBmBv7ziHFS13MM3jJAZvGgcHJgXoU0v+iuHxKxv1XcTpaPxMi+2prb5FN LKWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=ynhsqQuZumwhCPqGimoq6FfLou2KLT8k5MaJksPwUD4=; b=uPVs32YIhAIxwFEoerADihIrsNcnuYxOhcnALiGJQ4nYrb3z+UZqEyjYsl32Oml9nz hV06VwkLbXKYVhEFXiyiDg4TT8niMNmeAMvScc4ZXJGcijcpYzkF2vty6Bybr5mN+Ei+ /VehHwa9X3o0TYFC4U6nBXfrj3VRyr3BOhj/HzBcsY+t5hQaQcM9g2Ug9uaJO6pBc3Fi eFcth24FMOGKYftgB/C27ASbl5H1Z5ed8wz7a/GYX8Ht5jN9Dv48wUa/5zbjv1cw0mYy g6Lrcpv3H0GQf9UXWPgfwlV6XBh5AS4zXXUxNGLb8gFbgfdA/k/C6+0nLELqPzg0NPJb KOkQ== X-Gm-Message-State: AHQUAuYwkxbkoxFHcCnH4cNpLowoNS3LVhKdXOmeIigxD8PsLURx336A neLdFbw75B8jgKRoSXf6KusUBekpL4Q= X-Google-Smtp-Source: AHgI3IaYJn8AuG5f+jcJ6sgKjB+fEyiUHIBhccP7EOSIR8EAxZ1p8OQ57ZquPtV2J4XMn+BzR7DTAA== X-Received: by 2002:a63:235f:: with SMTP id u31mr6973391pgm.122.1550031181399; Tue, 12 Feb 2019 20:13:01 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id k63sm30456124pfc.76.2019.02.12.20.13.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Feb 2019 20:13:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gtlut-0005rt-LM; Tue, 12 Feb 2019 21:12:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH 00/10] Revise device handling in rxe Date: Tue, 12 Feb 2019 21:12:46 -0700 Message-Id: <20190213041256.22437-1-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This creates a series of additional core entry points that meet the needs of a driver like rxe, and other drivers, as it relates to their register/unregister and netdev notifier paths. The existing driver code is generally full of races in this area, while these new core routines sit on the new locking schemes and provide a complete race free solution. rxe is used as an example conversion, but other drivers should be converted as well. The approach uses a combination of the registration refcount (combined with core level tracking and lookup) and a series of new lifecycle callbacks that the driver can implement to reliably get the correct destruction sequence. These work together with the new unregistration entry points to allow the driver to have a robust lifetime model. To protect the typical asynchronous access via netdev notifiers from racing with unregistration this adds ib_device_get_by_netdev(). It does the lookup and holds the registration refcount under a proper core driven locking scheme. usnic has already posted a patch to use this api, siw will need it, rxe is converted here and other drivers should be adjusted as well. Jason Gunthorpe (10): RDMA: Add and use rdma_for_each_port RDMA/device: Consolidate ib_device per_port data into one place RDMA/cache: Move the cache per-port data into the main ib_port_data RDMA/device: Add ib_device_set_netdev() as an alternative to get_netdev RDMA/device: Add ib_device_get_by_netdev() RDMA/rxe: Use ib_device_get_by_netdev() instead of open coding RDMA/device: Provide APIs from the core code to help unregistration RDMA/rxe: Use driver_unregister and new unregistration API RDMA/rxe: Add ib_device_get_by_name() and use it in rxe RDMA/rxe: Close a race after ib_register_device .clang-format | 1 + drivers/infiniband/core/cache.c | 121 ++--- drivers/infiniband/core/cma.c | 7 +- drivers/infiniband/core/core_priv.h | 3 + drivers/infiniband/core/device.c | 620 ++++++++++++++++++---- drivers/infiniband/core/mad.c | 4 +- drivers/infiniband/core/nldev.c | 8 +- drivers/infiniband/core/security.c | 35 +- drivers/infiniband/core/sysfs.c | 12 +- drivers/infiniband/core/user_mad.c | 9 +- drivers/infiniband/core/verbs.c | 5 +- drivers/infiniband/sw/rxe/rxe.c | 41 +- drivers/infiniband/sw/rxe/rxe.h | 14 +- drivers/infiniband/sw/rxe/rxe_loc.h | 2 +- drivers/infiniband/sw/rxe/rxe_net.c | 84 +-- drivers/infiniband/sw/rxe/rxe_net.h | 2 +- drivers/infiniband/sw/rxe/rxe_pool.c | 14 +- drivers/infiniband/sw/rxe/rxe_sysfs.c | 36 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 48 +- drivers/infiniband/sw/rxe/rxe_verbs.h | 3 - drivers/infiniband/ulp/ipoib/ipoib_main.c | 4 +- drivers/infiniband/ulp/srp/ib_srp.c | 5 +- include/rdma/ib_verbs.h | 118 ++-- 23 files changed, 765 insertions(+), 431 deletions(-)