| Message ID | 1343983258-6268-4-git-send-email-jackm@dev.mellanox.co.il (mailing list archive) |
|---|---|
| State | Accepted, archived |
| Delegated to: | Roland Dreier |
| Headers | show |
On 8/3/2012 4:40 AM, Jack Morgenstein wrote: > When port pkey table potentially contains both full and partial > membership copies for the same pkey, we need a function to find > the exact (16-bit) pkey index. The code on this patch is fine, just see my previous email about the function naming... > This is particularly necessary when the master forwards QP1 MADS > sent by guests. If the guest has sent the MAD with a limited > membership pkey, we wish to forward the MAD using the same limited > membership pkey. Since master may have both the limited and > the full member pkeys in its table, we must make sure to retrieve > the limited membership pkey in this case. > > This requires the 16-bit pkey lookup function (which includes the > membership bit). > > Signed-off-by: Jack Morgenstein <jackm@dev.mellanox.co.il> > Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> > --- > drivers/infiniband/core/cache.c | 32 ++++++++++++++++++++++++++++++++ > include/rdma/ib_cache.h | 16 ++++++++++++++++ > 2 files changed, 48 insertions(+), 0 deletions(-) > > diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c > index 0f2f2b7..d8a8c83 100644 > --- a/drivers/infiniband/core/cache.c > +++ b/drivers/infiniband/core/cache.c > @@ -198,6 +198,38 @@ int ib_find_cached_pkey(struct ib_device *device, > } > EXPORT_SYMBOL(ib_find_cached_pkey); > > +int ib_find_exact_cached_pkey(struct ib_device *device, > + u8 port_num, > + u16 pkey, > + u16 *index) > +{ > + struct ib_pkey_cache *cache; > + unsigned long flags; > + int i; > + int ret = -ENOENT; > + > + if (port_num < start_port(device) || port_num > end_port(device)) > + return -EINVAL; > + > + read_lock_irqsave(&device->cache.lock, flags); > + > + cache = device->cache.pkey_cache[port_num - start_port(device)]; > + > + *index = -1; > + > + for (i = 0; i < cache->table_len; ++i) > + if (cache->table[i] == pkey) { > + *index = i; > + ret = 0; > + break; > + } > + > + read_unlock_irqrestore(&device->cache.lock, flags); > + > + return ret; > +} > +EXPORT_SYMBOL(ib_find_exact_cached_pkey); > + > int ib_get_cached_lmc(struct ib_device *device, > u8 port_num, > u8 *lmc) > diff --git a/include/rdma/ib_cache.h b/include/rdma/ib_cache.h > index 00a2b8e..ad9a3c2 100644 > --- a/include/rdma/ib_cache.h > +++ b/include/rdma/ib_cache.h > @@ -101,6 +101,22 @@ int ib_find_cached_pkey(struct ib_device *device, > u16 *index); > > /** > + * ib_find_exact_cached_pkey - Returns the PKey table index where a specified > + * PKey value occurs. Comparison uses the FULL 16 bits (incl membership bit) > + * @device: The device to query. > + * @port_num: The port number of the device to search for the PKey. > + * @pkey: The PKey value to search for. > + * @index: The index into the cached PKey table where the PKey was found. > + * > + * ib_find_exact_cached_pkey() searches the specified PKey table in > + * the local software cache. > + */ > +int ib_find_exact_cached_pkey(struct ib_device *device, > + u8 port_num, > + u16 pkey, > + u16 *index); > + > +/** > * ib_get_cached_lmc - Returns a cached lmc table entry > * @device: The device to query. > * @port_num: The port number of the device to query. >
On 8/3/2012 4:40 AM, Jack Morgenstein wrote: > When port pkey table potentially contains both full and partial > membership copies for the same pkey, we need a function to find > the exact (16-bit) pkey index. > > This is particularly necessary when the master forwards QP1 MADS > sent by guests. If the guest has sent the MAD with a limited > membership pkey, we wish to forward the MAD using the same limited > membership pkey. Since master may have both the limited and > the full member pkeys in its table, we must make sure to retrieve > the limited membership pkey in this case. > > This requires the 16-bit pkey lookup function (which includes the > membership bit). As a second note, I would like to know why Intel (previously QLogic) does not use these functions in their driver and what it would take to get all drivers to use the functions. Do we need to add more to them? In my opinion these should be generally useful and used by all drivers. Mike?
On Tue, Sep 11, 2012 at 10:12 AM, Doug Ledford <dledford@redhat.com> wrote: > As a second note, I would like to know why Intel (previously QLogic) > does not use these functions in their driver and what it would take to > get all drivers to use the functions. Do we need to add more to them? > In my opinion these should be generally useful and used by all drivers. Use which functions? The P_Key lookup functions? What would a low-level driver use them for? I thought these are for use by upper-level protocols. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 9/11/2012 3:07 PM, Roland Dreier wrote: > On Tue, Sep 11, 2012 at 10:12 AM, Doug Ledford <dledford@redhat.com> wrote: >> As a second note, I would like to know why Intel (previously QLogic) >> does not use these functions in their driver and what it would take to >> get all drivers to use the functions. Do we need to add more to them? >> In my opinion these should be generally useful and used by all drivers. > > Use which functions? The P_Key lookup functions? > > What would a low-level driver use them for? I thought these are for > use by upper-level protocols. Well, at this point, the mlx4 driver uses them, the rdmacm kernel driver uses them, and both QLogic/Intel drivers have their own internal pkey table implementation. So, it isn't so much upper layer as it is drivers.
On Tue, Sep 11, 2012 at 1:34 PM, Doug Ledford <dledford@redhat.com> wrote: > Well, at this point, the mlx4 driver uses them, the rdmacm kernel driver > uses them, and both QLogic/Intel drivers have their own internal pkey > table implementation. So, it isn't so much upper layer as it is drivers. rdmacm is an upper-level protocol (it's above the midlayer / hardware abstraction). mlx4 and mthca look up P_Keys because of internal details of how they send MADs, and really they should move to maintaining their own P_Key table too. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 9/11/2012 4:43 PM, Roland Dreier wrote: > On Tue, Sep 11, 2012 at 1:34 PM, Doug Ledford <dledford@redhat.com> wrote: >> Well, at this point, the mlx4 driver uses them, the rdmacm kernel driver >> uses them, and both QLogic/Intel drivers have their own internal pkey >> table implementation. So, it isn't so much upper layer as it is drivers. > > rdmacm is an upper-level protocol (it's above the midlayer / hardware > abstraction). Yeah, I know. My point wasn't that it was a low level item, just that it's the only upper layer consumer that I saw. > mlx4 and mthca look up P_Keys because of internal details of how they > send MADs, and really they should move to maintaining their own P_Key > table too. Why not make the routines useful for all users instead of multiple implementations of the same thing?
diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c index 0f2f2b7..d8a8c83 100644 --- a/drivers/infiniband/core/cache.c +++ b/drivers/infiniband/core/cache.c @@ -198,6 +198,38 @@ int ib_find_cached_pkey(struct ib_device *device, } EXPORT_SYMBOL(ib_find_cached_pkey); +int ib_find_exact_cached_pkey(struct ib_device *device, + u8 port_num, + u16 pkey, + u16 *index) +{ + struct ib_pkey_cache *cache; + unsigned long flags; + int i; + int ret = -ENOENT; + + if (port_num < start_port(device) || port_num > end_port(device)) + return -EINVAL; + + read_lock_irqsave(&device->cache.lock, flags); + + cache = device->cache.pkey_cache[port_num - start_port(device)]; + + *index = -1; + + for (i = 0; i < cache->table_len; ++i) + if (cache->table[i] == pkey) { + *index = i; + ret = 0; + break; + } + + read_unlock_irqrestore(&device->cache.lock, flags); + + return ret; +} +EXPORT_SYMBOL(ib_find_exact_cached_pkey); + int ib_get_cached_lmc(struct ib_device *device, u8 port_num, u8 *lmc) diff --git a/include/rdma/ib_cache.h b/include/rdma/ib_cache.h index 00a2b8e..ad9a3c2 100644 --- a/include/rdma/ib_cache.h +++ b/include/rdma/ib_cache.h @@ -101,6 +101,22 @@ int ib_find_cached_pkey(struct ib_device *device, u16 *index); /** + * ib_find_exact_cached_pkey - Returns the PKey table index where a specified + * PKey value occurs. Comparison uses the FULL 16 bits (incl membership bit) + * @device: The device to query. + * @port_num: The port number of the device to search for the PKey. + * @pkey: The PKey value to search for. + * @index: The index into the cached PKey table where the PKey was found. + * + * ib_find_exact_cached_pkey() searches the specified PKey table in + * the local software cache. + */ +int ib_find_exact_cached_pkey(struct ib_device *device, + u8 port_num, + u16 pkey, + u16 *index); + +/** * ib_get_cached_lmc - Returns a cached lmc table entry * @device: The device to query. * @port_num: The port number of the device to query.