From patchwork Wed Feb 13 04:12:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10809097 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C18701399 for ; Wed, 13 Feb 2019 04:13:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ADD692BFE1 for ; Wed, 13 Feb 2019 04:13:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A1D512C06B; Wed, 13 Feb 2019 04:13:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 087022BFED for ; Wed, 13 Feb 2019 04:13:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729015AbfBMEND (ORCPT ); Tue, 12 Feb 2019 23:13:03 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:40911 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727915AbfBMEND (ORCPT ); Tue, 12 Feb 2019 23:13:03 -0500 Received: by mail-pl1-f194.google.com with SMTP id bj4so536921plb.7 for ; Tue, 12 Feb 2019 20:13:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E8Lxa/NkIuAHp4LzaEi4XW7GDT/KE7+a89lAfPknEwQ=; b=SCXF/PTM/q9KJ/IhSlQvoT0Ku3/l50C/g4eMouRrgG+uQWwb+JhnqFMP+smOZ3Yrcf lTOPsE1rqV5ZI/J0KVr7DpVQqmnwn2xK79TzXSBHVNha0BA1bnVkbpPsPN9vA34FlWQr pFX/+FXkK81GEwk2WiiwmDWevOa5wNQ6DlXHSiWILhoUz6W+wZ6mQVAHEMb46eF1e+KL GrjsrXbATxzDkgleUNozPImTba9PTJRGlpRUjcc7E9wR2LLrMfJ8FKDRNFYNS+knc8V+ mU1gaAGbUid+/uWQ05LjfiC6nnxKw4cfQXwggGWpe4QTl9teN/TRXVuTExlOYPxY1Bn5 2Gaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E8Lxa/NkIuAHp4LzaEi4XW7GDT/KE7+a89lAfPknEwQ=; b=cDMVQMyuDN6txOHo/q0PnjP9aciuRyloSKyJzseFCZNwc39A0ov/JTvpRn9gpv6Oiv qFv5OVSzSVGI91LMqRAP/BtAe5PhKcwsvOUOdZfJHibeJ0Cq+88kor/5kCCBEU+J0slT 6T5so+pOutpPbEZXqU1SQccUHSJWCc4FbZMcN8+mRnORiASuDoV9CCSCWW0eMTHMEjFs snse+fpDQoGq+aUrnSRcbV6SuWgOJUXyW1GU/R56b47g0WOqmIcx2LFBZoadAegDrSUa CbVX02KMAClXflHuzaoQDOWlrKfO+ZtHAXG/xqn24FjxSFmK6+MYxjc5BTbKYl+iqZhK bdIQ== X-Gm-Message-State: AHQUAuYj8AmjcM7o+cBKqWpIxdmIc7c94zreTfBs8WSCEhzgMdSnuTI4 OW8XRWtLiSTeak7OZAC8QIa2hyUUIDc= X-Google-Smtp-Source: AHgI3IbYULYCJ4VH8QIqgS2amyctjDIlIlMCxuI28oP4vkfg0dJjXUViAnGCrxo61CXtMRu+JA3Z+Q== X-Received: by 2002:a17:902:12f:: with SMTP id 44mr7768605plb.74.1550031181956; Tue, 12 Feb 2019 20:13:01 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id l28sm12390572pfi.178.2019.02.12.20.13.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Feb 2019 20:13:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gtlut-0005sU-WE; Tue, 12 Feb 2019 21:13:00 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH 06/10] RDMA/rxe: Use ib_device_get_by_netdev() instead of open coding Date: Tue, 12 Feb 2019 21:12:52 -0700 Message-Id: <20190213041256.22437-7-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190213041256.22437-1-jgg@ziepe.ca> References: <20190213041256.22437-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe The core API handles the locking correctly and is faster if there are multiple devices. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/sw/rxe/rxe.h | 12 ++++++++- drivers/infiniband/sw/rxe/rxe_net.c | 39 +++++++++++---------------- drivers/infiniband/sw/rxe/rxe_sysfs.c | 17 ++++++------ drivers/infiniband/sw/rxe/rxe_verbs.c | 17 +++--------- 4 files changed, 38 insertions(+), 47 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.h b/drivers/infiniband/sw/rxe/rxe.h index 5bde2ad964d277..2b875875962f6e 100644 --- a/drivers/infiniband/sw/rxe/rxe.h +++ b/drivers/infiniband/sw/rxe/rxe.h @@ -105,9 +105,19 @@ static inline void rxe_dev_put(struct rxe_dev *rxe) { kref_put(&rxe->ref_cnt, rxe_release); } -struct rxe_dev *net_to_rxe(struct net_device *ndev); struct rxe_dev *get_rxe_by_name(const char *name); +/* The caller must do a matching ib_device_put(&dev->ib_dev) */ +static inline struct rxe_dev *rxe_get_dev_from_net(struct net_device *ndev) +{ + struct ib_device *ibdev = + ib_device_get_by_netdev(ndev, RDMA_DRIVER_RXE); + + if (!ibdev) + return NULL; + return container_of(ibdev, struct rxe_dev, ib_dev); +} + void rxe_port_up(struct rxe_dev *rxe); void rxe_port_down(struct rxe_dev *rxe); void rxe_set_port_state(struct rxe_dev *rxe); diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 3b162e92e8e8d7..56878453f1aed8 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -48,23 +48,6 @@ static LIST_HEAD(rxe_dev_list); static DEFINE_SPINLOCK(dev_list_lock); /* spinlock for device list */ -struct rxe_dev *net_to_rxe(struct net_device *ndev) -{ - struct rxe_dev *rxe; - struct rxe_dev *found = NULL; - - spin_lock_bh(&dev_list_lock); - list_for_each_entry(rxe, &rxe_dev_list, list) { - if (rxe->ndev == ndev) { - found = rxe; - break; - } - } - spin_unlock_bh(&dev_list_lock); - - return found; -} - struct rxe_dev *get_rxe_by_name(const char *name) { struct rxe_dev *rxe; @@ -81,7 +64,6 @@ struct rxe_dev *get_rxe_by_name(const char *name) return found; } - static struct rxe_recv_sockets recv_sockets; struct device *rxe_dma_device(struct rxe_dev *rxe) @@ -229,18 +211,19 @@ static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) struct udphdr *udph; struct net_device *ndev = skb->dev; struct net_device *rdev = ndev; - struct rxe_dev *rxe = net_to_rxe(ndev); + struct rxe_dev *rxe = rxe_get_dev_from_net(ndev); struct rxe_pkt_info *pkt = SKB_TO_PKT(skb); if (!rxe && is_vlan_dev(rdev)) { rdev = vlan_dev_real_dev(ndev); - rxe = net_to_rxe(rdev); + rxe = rxe_get_dev_from_net(rdev); } if (!rxe) goto drop; if (skb_linearize(skb)) { pr_err("skb_linearize failed\n"); + ib_device_put(&rxe->ib_dev); goto drop; } @@ -253,6 +236,12 @@ static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) rxe_rcv(skb); + /* + * FIXME: this is in the wrong place, it needs to be done when pkt is + * destroyed + */ + ib_device_put(&rxe->ib_dev); + return 0; drop: kfree_skb(skb); @@ -635,16 +624,17 @@ static int rxe_notify(struct notifier_block *not_blk, void *arg) { struct net_device *ndev = netdev_notifier_info_to_dev(arg); - struct rxe_dev *rxe = net_to_rxe(ndev); + struct rxe_dev *rxe = rxe_get_dev_from_net(ndev); if (!rxe) - goto out; + return NOTIFY_OK; switch (event) { case NETDEV_UNREGISTER: list_del(&rxe->list); + ib_device_put(&rxe->ib_dev); rxe_remove(rxe); - break; + return NOTIFY_OK; case NETDEV_UP: rxe_port_up(rxe); break; @@ -668,7 +658,8 @@ static int rxe_notify(struct notifier_block *not_blk, event, ndev->name); break; } -out: + + ib_device_put(&rxe->ib_dev); return NOTIFY_OK; } diff --git a/drivers/infiniband/sw/rxe/rxe_sysfs.c b/drivers/infiniband/sw/rxe/rxe_sysfs.c index 95a15892f7e659..6802be71bf9b20 100644 --- a/drivers/infiniband/sw/rxe/rxe_sysfs.c +++ b/drivers/infiniband/sw/rxe/rxe_sysfs.c @@ -58,24 +58,25 @@ static int rxe_param_set_add(const char *val, const struct kernel_param *kp) int len; int err = 0; char intf[32]; - struct net_device *ndev = NULL; + struct net_device *ndev; + struct rxe_dev *exists; struct rxe_dev *rxe; len = sanitize_arg(val, intf, sizeof(intf)); if (!len) { pr_err("add: invalid interface name\n"); - err = -EINVAL; - goto err; + return -EINVAL; } ndev = dev_get_by_name(&init_net, intf); if (!ndev) { pr_err("interface %s not found\n", intf); - err = -EINVAL; - goto err; + return -EINVAL; } - if (net_to_rxe(ndev)) { + exists = rxe_get_dev_from_net(ndev); + if (exists) { + ib_device_put(&exists->ib_dev); pr_err("already configured on %s\n", intf); err = -EINVAL; goto err; @@ -90,9 +91,9 @@ static int rxe_param_set_add(const char *val, const struct kernel_param *kp) rxe_set_port_state(rxe); dev_info(&rxe->ib_dev.dev, "added %s\n", intf); + err: - if (ndev) - dev_put(ndev); + dev_put(ndev); return err; } diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 051c3930e808ed..de96c18f09968b 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -79,19 +79,6 @@ static int rxe_query_port(struct ib_device *dev, return rc; } -static struct net_device *rxe_get_netdev(struct ib_device *device, - u8 port_num) -{ - struct rxe_dev *rxe = to_rdev(device); - - if (rxe->ndev) { - dev_hold(rxe->ndev); - return rxe->ndev; - } - - return NULL; -} - static int rxe_query_pkey(struct ib_device *device, u8 port_num, u16 index, u16 *pkey) { @@ -1157,7 +1144,6 @@ static const struct ib_device_ops rxe_dev_ops = { .get_dma_mr = rxe_get_dma_mr, .get_hw_stats = rxe_ib_get_hw_stats, .get_link_layer = rxe_get_link_layer, - .get_netdev = rxe_get_netdev, .get_port_immutable = rxe_port_immutable, .map_mr_sg = rxe_map_mr_sg, .mmap = rxe_mmap, @@ -1238,6 +1224,9 @@ int rxe_register_device(struct rxe_dev *rxe) ; ib_set_device_ops(dev, &rxe_dev_ops); + err = ib_device_set_netdev(&rxe->ib_dev, rxe->ndev, 1); + if (err) + return err; tfm = crypto_alloc_shash("crc32", 0, 0); if (IS_ERR(tfm)) {