From patchwork Wed Feb 13 04:12:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 10809103 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D3921390 for ; Wed, 13 Feb 2019 04:13:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B4182BFE1 for ; Wed, 13 Feb 2019 04:13:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 501BA2C06B; Wed, 13 Feb 2019 04:13:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3127C2BFE1 for ; Wed, 13 Feb 2019 04:13:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729623AbfBMENE (ORCPT ); Tue, 12 Feb 2019 23:13:04 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:35714 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726902AbfBMEND (ORCPT ); Tue, 12 Feb 2019 23:13:03 -0500 Received: by mail-pg1-f193.google.com with SMTP id s198so534917pgs.2 for ; Tue, 12 Feb 2019 20:13:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=of8MsjYkr+7U45wiFWVundbJoy7Pm4aaQIZBujd48pk=; b=i53tuM8jFuFq4TtfIAY0jCZwAMeiEAe4Lds+UOhip10ImhLmABD7stIGTNf1a358XS 2AAs1QO2oGN4S17p9KDnpb1VAbK0IFPiiBXMBXzIkIgZMPsRnI3r5o+2J5sgPctqZOB0 fDCGRxA37BmL1KuFm2eIRSJbqmGK+p/k9btBQ7vnN3l41d6XQUJYipKKe1EO0yI4rmgG CKqF8N7q1ILjEOVvOoHqPZdkpmax2hBVMHuSGNjmgQ1gGsoWITUgvLgCDiwyWjq1crXq eoqWXQ7iHDlbpSxr/ChzlvoMIu92xz6cxGq9mPNjPk+QW/PdlqWup99DSnv8AZLdLdHr JKuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=of8MsjYkr+7U45wiFWVundbJoy7Pm4aaQIZBujd48pk=; b=m8mwI8+4kUdmviJeTKiy4n24Iy6H9qfQB3QpNh1w/9CB5uqZrT/sXK7KQYLUP/H1Bl jA3qZBwZVgKfCuNIZuxrN1usKs8GVqOsHol1ea9/HxzCwgKp8ecur9+T34Y0mmsTlLed fLIplqEMbgdQ/EcK9DGXBQ2ZWBe+69LnjjAn3JiDieE6rkAUwred1cXsEsIczZPAfe9F PVt0fYyE0rxe1mqV20d83DNIKXveqK8wS0uLXt2hnWS7FSu/MDZQr6UMsWLmmMb4OL+B 3GKw5Bofy8zpflsH8RBJ0kLCh+xh5SxZM5xEBzROTn0b2pHbqSxjuZC3PKQf60LrE0QH QPsQ== X-Gm-Message-State: AHQUAuZUM1oq99Q3n3AZ9aQp7mwUdRQfc8t2xljVkUHBHVTxLs6ki6CC iWX6kFK5JUunu/Uf6yn6fIxApV1SO7E= X-Google-Smtp-Source: AHgI3IZCZnClw+df2rE2yNZZMAQ1rweJjmD2e2C+6Lz6DoEDiyLtrIdi1DTwIPGlFQUW+15EyJnqsQ== X-Received: by 2002:a63:ce0e:: with SMTP id y14mr7028430pgf.145.1550031182402; Tue, 12 Feb 2019 20:13:02 -0800 (PST) Received: from ziepe.ca (S010614cc2056d97f.ed.shawcable.net. [174.3.196.123]) by smtp.gmail.com with ESMTPSA id y21sm7175528pfe.57.2019.02.12.20.13.00 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Feb 2019 20:13:00 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1gtlut-0005sB-RO; Tue, 12 Feb 2019 21:12:59 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Jason Gunthorpe Subject: [PATCH 03/10] RDMA/cache: Move the cache per-port data into the main ib_port_data Date: Tue, 12 Feb 2019 21:12:49 -0700 Message-Id: <20190213041256.22437-4-jgg@ziepe.ca> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190213041256.22437-1-jgg@ziepe.ca> References: <20190213041256.22437-1-jgg@ziepe.ca> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe Like the other cases there no real reason to have another array just for the cache. This larger conversion gets its own patch. Signed-off-by: Jason Gunthorpe --- drivers/infiniband/core/cache.c | 89 ++++++++++++--------------------- include/rdma/ib_verbs.h | 3 +- 2 files changed, 33 insertions(+), 59 deletions(-) diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c index 9d0e8aca741a6d..a28dc1901c8000 100644 --- a/drivers/infiniband/core/cache.c +++ b/drivers/infiniband/core/cache.c @@ -185,7 +185,7 @@ EXPORT_SYMBOL(ib_cache_gid_parse_type_str); static struct ib_gid_table *rdma_gid_table(struct ib_device *device, u8 port) { - return device->cache.ports[port - rdma_start_port(device)].gid; + return device->port_data[port].cache.gid; } static bool is_gid_entry_free(const struct ib_gid_table_entry *entry) @@ -765,7 +765,7 @@ static struct ib_gid_table *alloc_gid_table(int sz) return NULL; } -static void release_gid_table(struct ib_device *device, u8 port, +static void release_gid_table(struct ib_device *device, struct ib_gid_table *table) { bool leak = false; @@ -863,31 +863,27 @@ static void gid_table_reserve_default(struct ib_device *ib_dev, u8 port, static void gid_table_release_one(struct ib_device *ib_dev) { - struct ib_gid_table *table; - u8 port; + unsigned int p; - for (port = 0; port < ib_dev->phys_port_cnt; port++) { - table = ib_dev->cache.ports[port].gid; - release_gid_table(ib_dev, port, table); - ib_dev->cache.ports[port].gid = NULL; + rdma_for_each_port (ib_dev, p) { + release_gid_table(ib_dev, ib_dev->port_data[p].cache.gid); + ib_dev->port_data[p].cache.gid = NULL; } } static int _gid_table_setup_one(struct ib_device *ib_dev) { - u8 port; struct ib_gid_table *table; + unsigned int rdma_port; - for (port = 0; port < ib_dev->phys_port_cnt; port++) { - u8 rdma_port = port + rdma_start_port(ib_dev); - + rdma_for_each_port (ib_dev, rdma_port) { table = alloc_gid_table( ib_dev->port_data[rdma_port].immutable.gid_tbl_len); if (!table) goto rollback_table_setup; gid_table_reserve_default(ib_dev, rdma_port, table); - ib_dev->cache.ports[port].gid = table; + ib_dev->port_data[rdma_port].cache.gid = table; } return 0; @@ -898,14 +894,11 @@ static int _gid_table_setup_one(struct ib_device *ib_dev) static void gid_table_cleanup_one(struct ib_device *ib_dev) { - struct ib_gid_table *table; - u8 port; + unsigned int p; - for (port = 0; port < ib_dev->phys_port_cnt; port++) { - table = ib_dev->cache.ports[port].gid; - cleanup_gid_table_port(ib_dev, port + rdma_start_port(ib_dev), - table); - } + rdma_for_each_port (ib_dev, p) + cleanup_gid_table_port(ib_dev, p, + ib_dev->port_data[p].cache.gid); } static int gid_table_setup_one(struct ib_device *ib_dev) @@ -983,17 +976,17 @@ const struct ib_gid_attr *rdma_find_gid(struct ib_device *device, unsigned long mask = GID_ATTR_FIND_MASK_GID | GID_ATTR_FIND_MASK_GID_TYPE; struct ib_gid_attr gid_attr_val = {.ndev = ndev, .gid_type = gid_type}; - u8 p; + unsigned int p; if (ndev) mask |= GID_ATTR_FIND_MASK_NETDEV; - for (p = 0; p < device->phys_port_cnt; p++) { + rdma_for_each_port(device, p) { struct ib_gid_table *table; unsigned long flags; int index; - table = device->cache.ports[p].gid; + table = device->port_data[p].cache.gid; read_lock_irqsave(&table->rwlock, flags); index = find_gid(table, gid, &gid_attr_val, false, mask, NULL); if (index >= 0) { @@ -1025,7 +1018,7 @@ int ib_get_cached_pkey(struct ib_device *device, read_lock_irqsave(&device->cache.lock, flags); - cache = device->cache.ports[port_num - rdma_start_port(device)].pkey; + cache = device->port_data[port_num].cache.pkey; if (index < 0 || index >= cache->table_len) ret = -EINVAL; @@ -1043,14 +1036,12 @@ int ib_get_cached_subnet_prefix(struct ib_device *device, u64 *sn_pfx) { unsigned long flags; - int p; if (!rdma_is_port_valid(device, port_num)) return -EINVAL; - p = port_num - rdma_start_port(device); read_lock_irqsave(&device->cache.lock, flags); - *sn_pfx = device->cache.ports[p].subnet_prefix; + *sn_pfx = device->port_data[port_num].cache.subnet_prefix; read_unlock_irqrestore(&device->cache.lock, flags); return 0; @@ -1073,7 +1064,7 @@ int ib_find_cached_pkey(struct ib_device *device, read_lock_irqsave(&device->cache.lock, flags); - cache = device->cache.ports[port_num - rdma_start_port(device)].pkey; + cache = device->port_data[port_num].cache.pkey; *index = -1; @@ -1113,7 +1104,7 @@ int ib_find_exact_cached_pkey(struct ib_device *device, read_lock_irqsave(&device->cache.lock, flags); - cache = device->cache.ports[port_num - rdma_start_port(device)].pkey; + cache = device->port_data[port_num].cache.pkey; *index = -1; @@ -1141,7 +1132,7 @@ int ib_get_cached_lmc(struct ib_device *device, return -EINVAL; read_lock_irqsave(&device->cache.lock, flags); - *lmc = device->cache.ports[port_num - rdma_start_port(device)].lmc; + *lmc = device->port_data[port_num].cache.lmc; read_unlock_irqrestore(&device->cache.lock, flags); return ret; @@ -1159,8 +1150,7 @@ int ib_get_cached_port_state(struct ib_device *device, return -EINVAL; read_lock_irqsave(&device->cache.lock, flags); - *port_state = device->cache.ports[port_num - - rdma_start_port(device)].port_state; + *port_state = device->port_data[port_num].cache.port_state; read_unlock_irqrestore(&device->cache.lock, flags); return ret; @@ -1361,16 +1351,13 @@ static void ib_cache_update(struct ib_device *device, write_lock_irq(&device->cache.lock); - old_pkey_cache = device->cache.ports[port - - rdma_start_port(device)].pkey; + old_pkey_cache = device->port_data[port].cache.pkey; - device->cache.ports[port - rdma_start_port(device)].pkey = pkey_cache; - device->cache.ports[port - rdma_start_port(device)].lmc = tprops->lmc; - device->cache.ports[port - rdma_start_port(device)].port_state = - tprops->state; + device->port_data[port].cache.pkey = pkey_cache; + device->port_data[port].cache.lmc = tprops->lmc; + device->port_data[port].cache.port_state = tprops->state; - device->cache.ports[port - rdma_start_port(device)].subnet_prefix = - tprops->subnet_prefix; + device->port_data[port].cache.subnet_prefix = tprops->subnet_prefix; write_unlock_irq(&device->cache.lock); if (enforce_security) @@ -1433,19 +1420,9 @@ int ib_cache_setup_one(struct ib_device *device) rwlock_init(&device->cache.lock); - device->cache.ports = - kcalloc(rdma_end_port(device) - rdma_start_port(device) + 1, - sizeof(*device->cache.ports), - GFP_KERNEL); - if (!device->cache.ports) - return -ENOMEM; - err = gid_table_setup_one(device); - if (err) { - kfree(device->cache.ports); - device->cache.ports = NULL; + if (err) return err; - } rdma_for_each_port (device, p) ib_cache_update(device, p, true); @@ -1458,10 +1435,7 @@ int ib_cache_setup_one(struct ib_device *device) void ib_cache_release_one(struct ib_device *device) { - int p; - - if (!device->cache.ports) - return; + unsigned int p; /* * The release function frees all the cache elements. @@ -1469,11 +1443,10 @@ void ib_cache_release_one(struct ib_device *device) * all the device's resources when the cache could no * longer be accessed. */ - for (p = 0; p <= rdma_end_port(device) - rdma_start_port(device); ++p) - kfree(device->cache.ports[p].pkey); + rdma_for_each_port (device, p) + kfree(device->port_data[p].cache.pkey); gid_table_release_one(device); - kfree(device->cache.ports); } void ib_cache_cleanup_one(struct ib_device *device) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 4225cd9eb6f840..e5eea5a50e27f2 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2186,7 +2186,6 @@ struct ib_port_cache { struct ib_cache { rwlock_t lock; struct ib_event_handler event_handler; - struct ib_port_cache *ports; }; struct iw_cm_verbs; @@ -2203,6 +2202,8 @@ struct ib_port_data { spinlock_t pkey_list_lock; struct list_head pkey_list; + + struct ib_port_cache cache; }; /* rdma netdev type - specifies protocol type */