From patchwork Fri Apr 29 13:27:01 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hal Rosenstock X-Patchwork-Id: 740951 X-Patchwork-Delegate: alexne@voltaire.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id p3TDQl4O010898 for ; Fri, 29 Apr 2011 13:27:11 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753664Ab1D2N1K (ORCPT ); Fri, 29 Apr 2011 09:27:10 -0400 Received: from mail-wy0-f174.google.com ([74.125.82.174]:62837 "EHLO mail-wy0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751310Ab1D2N1I (ORCPT ); Fri, 29 Apr 2011 09:27:08 -0400 Received: by wya21 with SMTP id 21so2840843wya.19 for ; Fri, 29 Apr 2011 06:27:07 -0700 (PDT) Received: by 10.227.197.83 with SMTP id ej19mr881189wbb.105.1304083626160; Fri, 29 Apr 2011 06:27:06 -0700 (PDT) Received: from [192.168.1.100] (c-71-192-10-85.hsd1.ma.comcast.net [71.192.10.85]) by mx.google.com with ESMTPS id y12sm534708wby.25.2011.04.29.06.27.03 (version=SSLv3 cipher=OTHER); Fri, 29 Apr 2011 06:27:05 -0700 (PDT) Message-ID: <4DBABCA5.8040000@dev.mellanox.co.il> Date: Fri, 29 Apr 2011 09:27:01 -0400 From: Hal Rosenstock User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9 MIME-Version: 1.0 To: Alex Netes CC: "linux-rdma@vger.kernel.org" Subject: [PATCH 1/13] opensm: Add support for alias GUIDs Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.6 (demeter1.kernel.org [140.211.167.41]); Fri, 29 Apr 2011 13:27:11 +0000 (UTC) From 4e40ac7961b0530abfec00db05b8f54e5c098e22 Mon Sep 17 00:00:00 2001 From: Hal Rosenstock Date: Thu, 28 Apr 2011 22:43:07 +0300 Subject: [PATCH] opensm: Add support for alias GUIDs In order to support virtualized environments, alias GUID support is added. Each virtualized environment can have an alias GUID used for mapping to that environment. This patch adds support for alias GUIDs by adding an alias guid port table and populates it based on sets to SM GUIDInfo attribute based on SA sets/deletes of GUIDInfoRecord. Base port as well as alternate (alias) guids are included in this table. An SM assigned GUID byte is added as a configuration option where an alias GUID is formed from OpenFabrics OUI followed by 40 bits xy 00 ab cd ef where xy is the SM assigned guid byte and ab cd ef is an SM autogenerated 24 bits. The SM assigned GUID byte should be configured as subnet unique. Signed-off-by: Hal Rosenstock --- include/opensm/osm_base.h | 11 + include/opensm/osm_madw.h | 48 +++++- include/opensm/osm_msgdef.h | 3 +- include/opensm/osm_sm.h | 3 +- include/opensm/osm_subnet.h | 1 + opensm/Makefile.am | 2 +- opensm/osm_guid_info_rcv.c | 115 ++++++++++ opensm/osm_helper.c | 1 + opensm/osm_sa_class_port_info.c | 3 +- opensm/osm_sa_guidinfo_record.c | 438 +++++++++++++++++++++++++++++++++++--- opensm/osm_sm.c | 9 +- opensm/osm_sm_mad_ctrl.c | 6 +- opensm/osm_subnet.c | 12 +- 13 files changed, 607 insertions(+), 45 deletions(-) create mode 100644 opensm/osm_guid_info_rcv.c diff --git a/include/opensm/osm_base.h b/include/opensm/osm_base.h index 07e971d..17de12d 100644 --- a/include/opensm/osm_base.h +++ b/include/opensm/osm_base.h @@ -820,6 +820,17 @@ typedef enum _osm_thread_state { #define OSM_CAP2_IS_HIERARCHY_SUPPORTED (1 << 4) /***********/ +/****d* OpenSM: Base/OSM_CAP2_IS_ALIAS_GUIDS_SUPPORTED +* Name +* +* DESCRIPTION +* Alias GUIDs supported +* +* SYNOPSIS +*/ +#define OSM_CAP2_IS_ALIAS_GUIDS_SUPPORTED (1 << 5) +/***********/ + /****d* OpenSM: Base/osm_signal_t * NAME * osm_signal_t diff --git a/include/opensm/osm_madw.h b/include/opensm/osm_madw.h index 9c63151..817de82 100644 --- a/include/opensm/osm_madw.h +++ b/include/opensm/osm_madw.h @@ -1,6 +1,6 @@ /* * Copyright (c) 2004-2009 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 2002-2010 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * Copyright (c) 2009 HNR Consulting. All rights reserved. * @@ -185,6 +185,23 @@ typedef struct osm_pi_context { } osm_pi_context_t; /*********/ +/****s* OpenSM: MAD Wrapper/osm_gi_context_t +* NAME +* osm_gi_context_t +* +* DESCRIPTION +* Context needed by recipient of GUIDInfo attribute. +* +* SYNOPSIS +*/ +typedef struct osm_gi_context { + ib_net64_t node_guid; + ib_net64_t port_guid; + boolean_t set_method; + uint8_t port_num; +} osm_gi_context_t; +/*********/ + /****s* OpenSM: MAD Wrapper/osm_nd_context_t * NAME * osm_nd_context_t @@ -352,6 +369,7 @@ typedef struct osm_arbitrary_context { typedef union _osm_madw_context { osm_ni_context_t ni_context; osm_pi_context_t pi_context; + osm_gi_context_t gi_context; osm_nd_context_t nd_context; osm_si_context_t si_context; osm_lft_context_t lft_context; @@ -650,6 +668,34 @@ static inline osm_pi_context_t *osm_madw_get_pi_context_ptr(IN const osm_madw_t * SEE ALSO *********/ +/****f* OpenSM: MAD Wrapper/osm_madw_get_gi_context_ptr +* NAME +* osm_madw_get_gi_context_ptr +* +* DESCRIPTION +* Gets a pointer to the GUIDInfo context in this MAD. +* +* SYNOPSIS +*/ +static inline osm_gi_context_t *osm_madw_get_gi_context_ptr(IN const osm_madw_t + * p_madw) +{ + return ((osm_gi_context_t *) & p_madw->context); +} + +/* +* PARAMETERS +* p_madw +* [in] Pointer to an osm_madw_t object. +* +* RETURN VALUES +* Pointer to the start of the context structure. +* +* NOTES +* +* SEE ALSO +*********/ + /****f* OpenSM: MAD Wrapper/osm_madw_get_nd_context_ptr * NAME * osm_madw_get_nd_context_ptr diff --git a/include/opensm/osm_msgdef.h b/include/opensm/osm_msgdef.h index dbf3e53..cae8d93 100644 --- a/include/opensm/osm_msgdef.h +++ b/include/opensm/osm_msgdef.h @@ -1,6 +1,6 @@ /* * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 2002-2010 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two @@ -130,6 +130,7 @@ enum { OSM_MSG_MAD_NODE_INFO, OSM_MSG_MAD_PORT_INFO, OSM_MSG_MAD_SWITCH_INFO, + OSM_MSG_MAD_GUID_INFO, OSM_MSG_MAD_NODE_DESC, OSM_MSG_MAD_NODE_RECORD, OSM_MSG_MAD_PORTINFO_RECORD, diff --git a/include/opensm/osm_sm.h b/include/opensm/osm_sm.h index a3b770f..dae68da 100644 --- a/include/opensm/osm_sm.h +++ b/include/opensm/osm_sm.h @@ -1,6 +1,6 @@ /* * Copyright (c) 2004-2009 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005,2009 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 2002-2010 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two @@ -135,6 +135,7 @@ typedef struct osm_sm { cl_disp_reg_handle_t sweep_fail_disp_h; cl_disp_reg_handle_t ni_disp_h; cl_disp_reg_handle_t pi_disp_h; + cl_disp_reg_handle_t gi_disp_h; cl_disp_reg_handle_t nd_disp_h; cl_disp_reg_handle_t si_disp_h; cl_disp_reg_handle_t lft_disp_h; diff --git a/include/opensm/osm_subnet.h b/include/opensm/osm_subnet.h index 83ef77e..914b9d4 100644 --- a/include/opensm/osm_subnet.h +++ b/include/opensm/osm_subnet.h @@ -181,6 +181,7 @@ typedef struct osm_subn_opt { unsigned long log_max_size; char *partition_config_file; boolean_t no_partition_enforcement; + uint8_t sm_assigned_guid; boolean_t qos; char *qos_policy_file; boolean_t accum_log_file; diff --git a/opensm/Makefile.am b/opensm/Makefile.am index 69ff593..ec626bb 100644 --- a/opensm/Makefile.am +++ b/opensm/Makefile.am @@ -28,7 +28,7 @@ sbin_PROGRAMS = opensm opensm_LDFLAGS = -rdynamic opensm_DEPENDENCIES = libopensm.la opensm_SOURCES = main.c osm_console_io.c osm_console.c osm_db_files.c \ - osm_db_pack.c osm_drop_mgr.c \ + osm_db_pack.c osm_drop_mgr.c osm_guid_info_rcv.c \ osm_inform.c osm_lid_mgr.c osm_lin_fwd_rcv.c \ osm_link_mgr.c osm_mcast_fwd_rcv.c \ osm_mcast_mgr.c osm_mcast_tbl.c \ diff --git a/opensm/osm_guid_info_rcv.c b/opensm/osm_guid_info_rcv.c new file mode 100644 index 0000000..26baf71 --- /dev/null +++ b/opensm/osm_guid_info_rcv.c @@ -0,0 +1,115 @@ +/* + * Copyright (c) 2004-2009 Voltaire, Inc. All rights reserved. + * Copyright (c) 2002-2010 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +/* + * Abstract: + * Implementation of osm_gi_rcv_t. + * This object represents the GUIDInfo Receiver object. + * This object is part of the opensm family of objects. + */ + +#if HAVE_CONFIG_H +# include +#endif /* HAVE_CONFIG_H */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +void osm_gi_rcv_process(IN void *context, IN void *data) +{ + osm_sm_t *sm = context; + osm_madw_t *p_madw = data; + ib_guid_info_t *p_gi; + ib_smp_t *p_smp; + osm_port_t *p_port; + osm_gi_context_t *p_context; + ib_net64_t port_guid, node_guid; + uint8_t block_num, port_num; + + OSM_LOG_ENTER(sm->p_log); + + CL_ASSERT(sm); + CL_ASSERT(p_madw); + + p_smp = osm_madw_get_smp_ptr(p_madw); + p_context = osm_madw_get_gi_context_ptr(p_madw); + p_gi = ib_smp_get_payload_ptr(p_smp); + + CL_ASSERT(p_smp->attr_id == IB_MAD_ATTR_GUID_INFO); + + block_num = (uint8_t) cl_ntoh32(p_smp->attr_mod); + + port_guid = p_context->port_guid; + node_guid = p_context->node_guid; + port_num = p_context->port_num; + + osm_dump_guid_info(sm->p_log, node_guid, port_guid, block_num, p_gi, + OSM_LOG_DEBUG); + + CL_PLOCK_EXCL_ACQUIRE(sm->p_lock); + p_port = osm_get_port_by_guid(sm->p_subn, port_guid); + if (!p_port) { + CL_PLOCK_RELEASE(sm->p_lock); + OSM_LOG(sm->p_log, OSM_LOG_ERROR, "ERR 4701: " + "No port object for port with GUID 0x%" PRIx64 + "\n\t\t\t\tfor parent node GUID 0x%" PRIx64 + ", TID 0x%" PRIx64 "\n", + cl_ntoh64(port_guid), cl_ntoh64(node_guid), + cl_ntoh64(p_smp->trans_id)); + goto Exit; + } + + CL_PLOCK_RELEASE(sm->p_lock); + +Exit: + /* + Release the lock before jumping here!! + */ + OSM_LOG_EXIT(sm->p_log); +} diff --git a/opensm/osm_helper.c b/opensm/osm_helper.c index 0b0580a..f7e80ea 100644 --- a/opensm/osm_helper.c +++ b/opensm/osm_helper.c @@ -2030,6 +2030,7 @@ static const char *disp_msg_str[] = { "OSM_MSG_MAD_NODE_INFO", "OSM_MSG_MAD_PORT_INFO", "OSM_MSG_MAD_SWITCH_INFO", + "OSM_MSG_MAD_GUID_INFO", "OSM_MSG_MAD_NODE_DESC", "OSM_MSG_MAD_NODE_RECORD", "OSM_MSG_MAD_PORTINFO_RECORD", diff --git a/opensm/osm_sa_class_port_info.c b/opensm/osm_sa_class_port_info.c index 02dab82..8108a9a 100644 --- a/opensm/osm_sa_class_port_info.c +++ b/opensm/osm_sa_class_port_info.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004-2008 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2007 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 2002-2010 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two @@ -156,6 +156,7 @@ static void cpi_rcv_respond(IN osm_sa_t * sa, IN const osm_madw_t * p_madw) p_resp_cpi->cap_mask = OSM_CAP_IS_SUBN_GET_SET_NOTICE_SUP | OSM_CAP_IS_PORT_INFO_CAPMASK_MATCH_SUPPORTED; #endif + ib_class_set_cap_mask2(p_resp_cpi, OSM_CAP2_IS_ALIAS_GUIDS_SUPPORTED); if (sa->p_subn->opt.qos) ib_class_set_cap_mask2(p_resp_cpi, OSM_CAP2_IS_QOS_SUPPORTED | OSM_CAP2_IS_MCAST_TOP_SUPPORTED); diff --git a/opensm/osm_sa_guidinfo_record.c b/opensm/osm_sa_guidinfo_record.c index 3559a39..484633f 100644 --- a/opensm/osm_sa_guidinfo_record.c +++ b/opensm/osm_sa_guidinfo_record.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2006-2009 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2006 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 2002-2011 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two @@ -57,6 +57,9 @@ #include #include +#define IB_OPENIB_OUI 0x001405 +#define MOD_GIR_COMP_MASK (IB_GIR_COMPMASK_LID | IB_GIR_COMPMASK_BLOCKNUM) + typedef struct osm_gir_item { cl_list_item_t list_item; ib_guidinfo_record_t rec; @@ -75,7 +78,7 @@ static ib_api_status_t gir_rcv_new_gir(IN osm_sa_t * sa, IN cl_qlist_t * p_list, IN ib_net64_t const match_port_guid, IN ib_net16_t const match_lid, - IN const osm_physp_t * p_req_physp, + IN const osm_physp_t * p_physp, IN uint8_t const block_num) { osm_gir_item_t *p_rec_item; @@ -99,9 +102,12 @@ static ib_api_status_t gir_rcv_new_gir(IN osm_sa_t * sa, p_rec_item->rec.lid = match_lid; p_rec_item->rec.block_num = block_num; - if (!block_num) - p_rec_item->rec.guid_info.guid[0] = - osm_physp_get_port_guid(p_req_physp); + if (p_physp->p_guids) + memcpy(&p_rec_item->rec.guid_info, + *p_physp->p_guids + block_num * GUID_TABLE_MAX_ENTRIES, + sizeof(ib_guid_info_t)); + else if (!block_num) + p_rec_item->rec.guid_info.guid[0] = osm_physp_get_port_guid(p_physp); cl_qlist_insert_tail(p_list, &p_rec_item->list_item); @@ -279,50 +285,342 @@ Exit: OSM_LOG_EXIT(p_ctxt->sa->p_log); } -void osm_gir_rcv_process(IN void *ctx, IN void *data) +static inline boolean_t check_mod_comp_mask(ib_net64_t comp_mask) { - osm_sa_t *sa = ctx; - osm_madw_t *p_madw = data; - const ib_sa_mad_t *p_rcvd_mad; - const ib_guidinfo_record_t *p_rcvd_rec; - cl_qlist_t rec_list; - osm_gir_search_ctxt_t context; - osm_physp_t *p_req_physp; + return ((comp_mask & MOD_GIR_COMP_MASK) == MOD_GIR_COMP_MASK); +} - CL_ASSERT(sa); +static uint8_t coalesce_comp_mask(IN osm_madw_t *p_madw) +{ + uint8_t comp_mask = 0; + ib_sa_mad_t *p_sa_mad; + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID0) + comp_mask |= 1<<0; + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID1) + comp_mask |= 1<<1; + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID2) + comp_mask |= 1<<2; + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID3) + comp_mask |= 1<<3; + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID4) + comp_mask |= 1<<4; + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID5) + comp_mask |= 1<<5; + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID6) + comp_mask |= 1<<6; + if (p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID7) + comp_mask |= 1<<7; + return comp_mask; +} + +static void guidinfo_respond(IN osm_sa_t *sa, IN osm_madw_t *p_madw, + IN ib_guidinfo_record_t * p_guidinfo_rec) +{ + cl_qlist_t rec_list; + osm_gir_item_t *item; OSM_LOG_ENTER(sa->p_log); - CL_ASSERT(p_madw); + item = malloc(sizeof(*item)); + if (!item) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5101: " + "rec_item alloc failed\n"); + goto Exit; + } - p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); - p_rcvd_rec = - (ib_guidinfo_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + item->rec = *p_guidinfo_rec; - CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_GUIDINFO_RECORD); + cl_qlist_init(&rec_list); + cl_qlist_insert_tail(&rec_list, &item->list_item); - /* we only support SubnAdmGet and SubnAdmGetTable methods */ - if (p_rcvd_mad->method != IB_MAD_METHOD_GET && - p_rcvd_mad->method != IB_MAD_METHOD_GETTABLE) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5105: " - "Unsupported Method (%s)\n", - ib_get_sa_method_str(p_rcvd_mad->method)); - osm_sa_send_error(sa, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR); - goto Exit; + osm_sa_respond(sa, p_madw, sizeof(ib_guidinfo_record_t), &rec_list); + +Exit: + OSM_LOG_EXIT(sa->p_log); +} + +static void gir_respond(IN osm_sa_t *sa, IN osm_madw_t *p_madw) +{ + ib_sa_mad_t *p_sa_mad; + ib_guidinfo_record_t *p_rcvd_rec; + ib_guidinfo_record_t guidinfo_rec; + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = (ib_guidinfo_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) + osm_dump_guidinfo_record(sa->p_log, p_rcvd_rec, OSM_LOG_DEBUG); + + guidinfo_rec = *p_rcvd_rec; + guidinfo_respond(sa, p_madw, &guidinfo_rec); +} + +static ib_net64_t sm_assigned_guid(uint8_t assigned_byte) +{ + static uint32_t uniq_count; + + if (++uniq_count == 0) { + uniq_count--; + return 0; } + return cl_hton64(((uint64_t) uniq_count) | + (((uint64_t) assigned_byte) << 32) | + (((uint64_t) IB_OPENIB_OUI) << 40)); +} - /* update the requester physical port. */ - p_req_physp = osm_get_physp_by_mad_addr(sa->p_log, sa->p_subn, - osm_madw_get_mad_addr_ptr - (p_madw)); - if (p_req_physp == NULL) { - OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5104: " - "Cannot find requester physical port\n"); +static void guidinfo_set(IN osm_sa_t *sa, IN osm_port_t *p_port, + IN uint8_t block_num) +{ + uint8_t payload[IB_SMP_DATA_SIZE]; + osm_madw_context_t context; + ib_api_status_t status; + + memcpy(payload, + &((*p_port->p_physp->p_guids)[block_num * GUID_TABLE_MAX_ENTRIES]), + sizeof(ib_guid_info_t)); + + context.gi_context.node_guid = osm_node_get_node_guid(p_port->p_node); + context.gi_context.port_guid = osm_physp_get_port_guid(p_port->p_physp); + context.gi_context.set_method = TRUE; + context.gi_context.port_num = osm_physp_get_port_num(p_port->p_physp); + + status = osm_req_set(sa->sm, osm_physp_get_dr_path_ptr(p_port->p_physp), + payload, sizeof(payload), IB_MAD_ATTR_GUID_INFO, + cl_hton32((uint32_t)block_num), + CL_DISP_MSGID_NONE, &context); + if (status != IB_SUCCESS) + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5109: " + "Failure initiating GUIDInfo request (%s)\n", + ib_get_err_str(status)); +} + +static void del_guidinfo(IN osm_sa_t *sa, IN osm_madw_t *p_madw, + IN osm_port_t *p_port, IN uint8_t block_num) +{ + int i; + ib_sa_mad_t *p_sa_mad; + ib_guidinfo_record_t *p_rcvd_rec; + ib_net64_t del_alias_guid; + osm_alias_guid_t *p_alias_guid; + uint8_t del_mask; + int dirty = 0; + + if (!p_port->p_physp->p_guids) goto Exit; + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_guidinfo_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + del_mask = coalesce_comp_mask(p_madw); + + for (i = block_num * GUID_TABLE_MAX_ENTRIES; + (block_num + 1) * GUID_TABLE_MAX_ENTRIES < p_port->p_physp->port_info.guid_cap ? i < (block_num + 1) * GUID_TABLE_MAX_ENTRIES : i < p_port->p_physp->port_info.guid_cap; + i++) { + /* can't delete block 0 index 0 (base guid is RO) for alias guid table */ + if (i == 0 && p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID0) { + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Not allowed to delete RO GID 0\n"); + osm_sa_send_error(sa, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + return; + } + if (!(del_mask & 1<<(i % 8))) + continue; + + del_alias_guid = (*p_port->p_physp->p_guids)[i]; + if (del_alias_guid) { + /* remove original from alias guid table */ + p_alias_guid = (osm_alias_guid_t *) + cl_qmap_remove(&sa->p_subn->alias_port_guid_tbl, + del_alias_guid); + if (p_alias_guid != (osm_alias_guid_t *) + cl_qmap_end(&sa->p_subn->alias_port_guid_tbl)) + osm_alias_guid_delete(&p_alias_guid); + else + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 510B: " + "Original alias GUID 0x%" PRIx64 + " at index %u not found\n", + cl_ntoh64(del_alias_guid), i); + /* clear guid at index */ + (*p_port->p_physp->p_guids)[i] = 0; + dirty = 1; + } } - if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) + if (dirty) + guidinfo_set(sa, p_port, block_num); + + memcpy(&p_rcvd_rec->guid_info, + &((*p_port->p_physp->p_guids)[block_num * GUID_TABLE_MAX_ENTRIES]), + sizeof(ib_guid_info_t)); + +Exit: + gir_respond(sa, p_madw); +} + +static void set_guidinfo(IN osm_sa_t *sa, IN osm_madw_t *p_madw, + IN osm_port_t *p_port, IN uint8_t block_num) +{ + uint32_t max_block; + int i, j, dirty = 0; + ib_sa_mad_t *p_sa_mad; + ib_guidinfo_record_t *p_rcvd_rec; + osm_alias_guid_t *p_alias_guid, *p_alias_guid_check; + cl_map_item_t *p_item; + ib_net64_t set_alias_guid, del_alias_guid, assigned_guid; + uint8_t set_mask; + + if (!p_port->p_physp->p_guids) { + max_block = (p_port->p_physp->port_info.guid_cap + GUID_TABLE_MAX_ENTRIES - 1) / + GUID_TABLE_MAX_ENTRIES; + + p_port->p_physp->p_guids = calloc(max_block * GUID_TABLE_MAX_ENTRIES, + sizeof(ib_net64_t)); + if (!p_port->p_physp->p_guids) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5103: " + "GUID table memory allocation failed for port " + "GUID 0x%" PRIx64 "\n", + cl_ntoh64(p_port->p_physp->port_guid)); + osm_sa_send_error(sa, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + return; + } + /* setup base port guid in index 0 */ + (*p_port->p_physp->p_guids)[0] = p_port->p_physp->port_guid; + } + + p_sa_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = (ib_guidinfo_record_t *) ib_sa_mad_get_payload_ptr(p_sa_mad); + + if (osm_log_is_active(sa->p_log, OSM_LOG_DEBUG)) { + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, "Dump of incoming record\n"); osm_dump_guidinfo_record(sa->p_log, p_rcvd_rec, OSM_LOG_DEBUG); + } + + set_mask = coalesce_comp_mask(p_madw); + + for (i = block_num * GUID_TABLE_MAX_ENTRIES; + (block_num + 1) * GUID_TABLE_MAX_ENTRIES < p_port->p_physp->port_info.guid_cap ? i < (block_num + 1) * GUID_TABLE_MAX_ENTRIES : i < p_port->p_physp->port_info.guid_cap; + i++) { + /* can't set block 0 index 0 (base guid is RO) for alias guid table */ + if (i == 0 && p_sa_mad->comp_mask & IB_GIR_COMPMASK_GID0) { + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Not allowed to set RO GID 0\n"); + osm_sa_send_error(sa, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + return; + } + + if (!(set_mask & 1<<(i % 8))) + continue; + + set_alias_guid = p_rcvd_rec->guid_info.guid[i % 8]; + if (!set_alias_guid) { + /* was a GUID previously assigned for this index ? */ + set_alias_guid = (*p_port->p_physp->p_guids)[i]; + if (set_alias_guid) { + p_rcvd_rec->guid_info.guid[i % 8] = set_alias_guid; + continue; + } + } + if (!set_alias_guid) { + for (j = 0; j < 1000; j++) { + assigned_guid = sm_assigned_guid(sa->p_subn->opt.sm_assigned_guid); + if (!assigned_guid) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, + "ERR 510E: No more assigned guids available\n"); + osm_sa_send_error(sa, p_madw, + IB_SA_MAD_STATUS_NO_RESOURCES); + return; + } + p_item = cl_qmap_get(&sa->sm->p_subn->alias_port_guid_tbl, + assigned_guid); + if (p_item == cl_qmap_end(&sa->sm->p_subn->alias_port_guid_tbl)) { + set_alias_guid = assigned_guid; + p_rcvd_rec->guid_info.guid[i % 8] = assigned_guid; + break; + } + } + if (!set_alias_guid) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 510A: " + "SA assigned GUID %d failed for " + "port GUID 0x%" PRIx64 "\n", i, + cl_ntoh64(p_port->p_physp->port_guid)); + continue; + } + } + + /* allocate alias guid and add to alias guid table */ + p_alias_guid = osm_alias_guid_new(set_alias_guid, p_port); + if (!p_alias_guid) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5107: " + "Alias guid %d memory allocation failed" + " for port GUID 0x%" PRIx64 "\n", + i, cl_ntoh64(p_port->p_physp->port_guid)); + return; + } + + p_alias_guid_check = + (osm_alias_guid_t *) cl_qmap_insert(&sa->sm->p_subn->alias_port_guid_tbl, + p_alias_guid->alias_guid, + &p_alias_guid->map_item); + if (p_alias_guid_check != p_alias_guid) { + /* alias GUID is a duplicate */ + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5108: " + "Duplicate alias port GUID 0x%" PRIx64 + " index %d base port GUID 0x%" PRIx64 "\n", + cl_ntoh64(p_alias_guid->alias_guid), i, + cl_ntoh64(p_alias_guid->p_base_port->guid)); + osm_alias_guid_delete(&p_alias_guid); + /* clear response guid at index to indicate duplicate */ + p_rcvd_rec->guid_info.guid[i % 8] = 0; + } else { + del_alias_guid = (*p_port->p_physp->p_guids)[i]; + if (del_alias_guid) { + /* remove original from alias guid table */ + p_alias_guid_check = (osm_alias_guid_t *) + cl_qmap_remove(&sa->p_subn->alias_port_guid_tbl, + del_alias_guid); + if (p_alias_guid_check) + osm_alias_guid_delete(&p_alias_guid_check); + else + OSM_LOG(sa->p_log, OSM_LOG_ERROR, + "ERR 510C: Original alias GUID " + "0x%" PRIx64 "at index %u " + "not found\n", + cl_ntoh64(del_alias_guid), + i); + } + + /* insert or replace guid at index */ + (*p_port->p_physp->p_guids)[i] = set_alias_guid; + dirty = 1; + } + } + + if (dirty) + guidinfo_set(sa, p_port, block_num); + + memcpy(&p_rcvd_rec->guid_info, + &((*p_port->p_physp->p_guids)[block_num * GUID_TABLE_MAX_ENTRIES]), + sizeof(ib_guid_info_t)); + + gir_respond(sa, p_madw); +} + +static void get_guidinfo(IN osm_sa_t *sa, IN osm_madw_t *p_madw, + IN osm_physp_t *p_req_physp) +{ + const ib_sa_mad_t *p_rcvd_mad; + const ib_guidinfo_record_t *p_rcvd_rec; + cl_qlist_t rec_list; + osm_gir_search_ctxt_t context; + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + p_rcvd_rec = + (ib_guidinfo_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); cl_qlist_init(&rec_list); @@ -340,6 +638,76 @@ void osm_gir_rcv_process(IN void *ctx, IN void *data) cl_plock_release(sa->p_lock); osm_sa_respond(sa, p_madw, sizeof(ib_guidinfo_record_t), &rec_list); +} + +void osm_gir_rcv_process(IN void *ctx, IN void *data) +{ + osm_sa_t *sa = ctx; + osm_madw_t *p_madw = data; + const ib_sa_mad_t *p_rcvd_mad; + osm_physp_t *p_req_physp; + osm_port_t *p_port; + const ib_guidinfo_record_t *p_rcvd_rec; + + CL_ASSERT(sa); + + OSM_LOG_ENTER(sa->p_log); + + CL_ASSERT(p_madw); + + p_rcvd_mad = osm_madw_get_sa_mad_ptr(p_madw); + + CL_ASSERT(p_rcvd_mad->attr_id == IB_MAD_ATTR_GUIDINFO_RECORD); + + /* update the requester physical port */ + p_req_physp = osm_get_physp_by_mad_addr(sa->p_log, sa->p_subn, + osm_madw_get_mad_addr_ptr(p_madw)); + if (p_req_physp == NULL) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5104: " + "Cannot find requester physical port\n"); + goto Exit; + } + + switch(p_rcvd_mad->method) { + case IB_MAD_METHOD_GET: + case IB_MAD_METHOD_GETTABLE: + get_guidinfo(sa, p_madw, p_req_physp); + break; + case IB_MAD_METHOD_SET: + case IB_MAD_METHOD_DELETE: + if (!check_mod_comp_mask(p_rcvd_mad->comp_mask)) { + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5106: " + "component mask = 0x%016" PRIx64 ", " + "expected comp mask = 0x%016" PRIx64 "\n", + cl_ntoh64(p_rcvd_mad->comp_mask), + CL_NTOH64(MOD_GIR_COMP_MASK)); + osm_sa_send_error(sa, p_madw, + IB_SA_MAD_STATUS_REQ_INVALID); + goto Exit; + } + p_rcvd_rec = (ib_guidinfo_record_t *) ib_sa_mad_get_payload_ptr(p_rcvd_mad); + p_port = osm_get_port_by_lid(sa->p_subn, p_rcvd_rec->lid); + if (!p_port) { + OSM_LOG(sa->p_log, OSM_LOG_DEBUG, + "Port with LID %u not found\n", + cl_ntoh16(p_rcvd_rec->lid)); + goto Exit; + } + if (!osm_physp_share_pkey(sa->p_log, p_req_physp, + p_port->p_physp)) + goto Exit; + if (p_rcvd_mad->method == IB_MAD_METHOD_SET) + set_guidinfo(sa, p_madw, p_port, p_rcvd_rec->block_num); + else + del_guidinfo(sa, p_madw, p_port, p_rcvd_rec->block_num); + break; + default: + OSM_LOG(sa->p_log, OSM_LOG_ERROR, "ERR 5105: " + "Unsupported Method (%s)\n", + ib_get_sa_method_str(p_rcvd_mad->method)); + osm_sa_send_error(sa, p_madw, IB_MAD_STATUS_UNSUP_METHOD_ATTR); + break; + } Exit: OSM_LOG_EXIT(sa->p_log); diff --git a/opensm/osm_sm.c b/opensm/osm_sm.c index b8d7c55..e155576 100644 --- a/opensm/osm_sm.c +++ b/opensm/osm_sm.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004-2009 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 2002-2010 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * Copyright (c) 2008 Xsigo Systems Inc. All rights reserved. * @@ -68,6 +68,7 @@ extern void osm_nd_rcv_process(IN void *context, IN void *data); extern void osm_ni_rcv_process(IN void *context, IN void *data); extern void osm_pkey_rcv_process(IN void *context, IN void *data); extern void osm_pi_rcv_process(IN void *context, IN void *data); +extern void osm_gi_rcv_process(IN void *context, IN void *data); extern void osm_slvl_rcv_process(IN void *context, IN void *p_data); extern void osm_sminfo_rcv_process(IN void *context, IN void *data); extern void osm_si_rcv_process(IN void *context, IN void *data); @@ -199,6 +200,7 @@ void osm_sm_shutdown(IN osm_sm_t * p_sm) osm_sm_mad_ctrl_destroy(&p_sm->mad_ctrl); cl_disp_unregister(p_sm->ni_disp_h); cl_disp_unregister(p_sm->pi_disp_h); + cl_disp_unregister(p_sm->gi_disp_h); cl_disp_unregister(p_sm->si_disp_h); cl_disp_unregister(p_sm->nd_disp_h); cl_disp_unregister(p_sm->lft_disp_h); @@ -321,6 +323,11 @@ ib_api_status_t osm_sm_init(IN osm_sm_t * p_sm, IN osm_subn_t * p_subn, if (p_sm->pi_disp_h == CL_DISP_INVALID_HANDLE) goto Exit; + p_sm->gi_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_GUID_INFO, + osm_gi_rcv_process, p_sm); + if (p_sm->gi_disp_h == CL_DISP_INVALID_HANDLE) + goto Exit; + p_sm->si_disp_h = cl_disp_register(p_disp, OSM_MSG_MAD_SWITCH_INFO, osm_si_rcv_process, p_sm); if (p_sm->si_disp_h == CL_DISP_INVALID_HANDLE) diff --git a/opensm/osm_sm_mad_ctrl.c b/opensm/osm_sm_mad_ctrl.c index 086f71c..47fc32f 100644 --- a/opensm/osm_sm_mad_ctrl.c +++ b/opensm/osm_sm_mad_ctrl.c @@ -1,6 +1,6 @@ /* * Copyright (c) 2004-2009 Voltaire, Inc. All rights reserved. - * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved. + * Copyright (c) 2002-2010 Mellanox Technologies LTD. All rights reserved. * Copyright (c) 1996-2003 Intel Corporation. All rights reserved. * Copyright (c) 2009 HNR Consulting. All rights reserved. * Copyright (c) 2009 Sun Microsystems, Inc. All rights reserved. @@ -224,6 +224,9 @@ static void sm_mad_ctrl_process_get_resp(IN osm_sm_mad_ctrl_t * p_ctrl, case IB_MAD_ATTR_NODE_INFO: msg_id = OSM_MSG_MAD_NODE_INFO; break; + case IB_MAD_ATTR_GUID_INFO: + msg_id = OSM_MSG_MAD_GUID_INFO; + break; case IB_MAD_ATTR_SWITCH_INFO: msg_id = OSM_MSG_MAD_SWITCH_INFO; break; @@ -248,7 +251,6 @@ static void sm_mad_ctrl_process_get_resp(IN osm_sm_mad_ctrl_t * p_ctrl, case IB_MAD_ATTR_P_KEY_TABLE: msg_id = OSM_MSG_MAD_PKEY; break; - case IB_MAD_ATTR_GUID_INFO: case IB_MAD_ATTR_CLASS_PORT_INFO: case IB_MAD_ATTR_NOTICE: case IB_MAD_ATTR_INFORM_INFO: diff --git a/opensm/osm_subnet.c b/opensm/osm_subnet.c index 2e94aa1..34eaf29 100644 --- a/opensm/osm_subnet.c +++ b/opensm/osm_subnet.c @@ -340,6 +340,7 @@ static const opt_rec_t opt_tbl[] = { { "accum_log_file", OPT_OFFSET(accum_log_file), opts_parse_boolean, opts_setup_accum_log_file, 1 }, { "partition_config_file", OPT_OFFSET(partition_config_file), opts_parse_charp, NULL, 0 }, { "no_partition_enforcement", OPT_OFFSET(no_partition_enforcement), opts_parse_boolean, NULL, 1 }, + { "sm_assigned_guid", OPT_OFFSET(sm_assigned_guid), opts_parse_uint8, NULL, 1 }, { "qos", OPT_OFFSET(qos), opts_parse_boolean, NULL, 1 }, { "qos_policy_file", OPT_OFFSET(qos_policy_file), opts_parse_charp, NULL, 0 }, { "dump_files_dir", OPT_OFFSET(dump_files_dir), opts_parse_charp, NULL, 0 }, @@ -755,6 +756,7 @@ void osm_subn_set_default_opt(IN osm_subn_opt_t * p_opt) p_opt->log_max_size = 0; p_opt->partition_config_file = strdup(OSM_DEFAULT_PARTITION_CONFIG_FILE); p_opt->no_partition_enforcement = FALSE; + p_opt->sm_assigned_guid = 0; p_opt->qos = FALSE; p_opt->qos_policy_file = strdup(OSM_DEFAULT_QOS_POLICY_FILE); p_opt->accum_log_file = TRUE; @@ -1360,9 +1362,15 @@ int osm_subn_output_conf(FILE *out, IN osm_subn_opt_t * p_opts) "# Partition configuration file to be used\n" "partition_config_file %s\n\n" "# Disable partition enforcement by switches\n" - "no_partition_enforcement %s\n\n", + "no_partition_enforcement %s\n\n" + "# SM assigned GUID byte where GUID is formed from OpenFabrics OUI\n" + "# followed by 40 bits xy 00 ab cd ef where xy is the SM assigned GUID byte\n" + "# and ab cd ef is an SM autogenerated 24 bits\n" + "# SM assigned GUID byte should be configured as subnet unique\n" + "sm_assigned_guid 0x%02x\n\n", p_opts->partition_config_file, - p_opts->no_partition_enforcement ? "TRUE" : "FALSE"); + p_opts->no_partition_enforcement ? "TRUE" : "FALSE", + p_opts->sm_assigned_guid); fprintf(out, "#\n# SWEEP OPTIONS\n#\n"