From patchwork Wed Jun 3 14:59:38 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hal Rosenstock X-Patchwork-Id: 6539061 X-Patchwork-Delegate: hal@mellanox.com Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 3F0EEC0020 for ; Wed, 3 Jun 2015 14:59:52 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id D364C2066C for ; Wed, 3 Jun 2015 14:59:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 439592066B for ; Wed, 3 Jun 2015 14:59:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932478AbbFCO7s (ORCPT ); Wed, 3 Jun 2015 10:59:48 -0400 Received: from mail-wi0-f177.google.com ([209.85.212.177]:37631 "EHLO mail-wi0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932454AbbFCO7q (ORCPT ); Wed, 3 Jun 2015 10:59:46 -0400 Received: by wifw1 with SMTP id w1so25553240wif.0 for ; Wed, 03 Jun 2015 07:59:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:content-type:content-transfer-encoding; bh=XuUGa5Dv46q+Dwx5Gmt32JOAVyKrcPE7TuaRvTVYoRA=; b=cINoyFeS10MCiBh9lKdAfP3IQ/VNcxrYBzc7gMAvfwYkczw9LkP5XcRhXTekvr+fuN EqbqAO6Nb82wgHocoPed2iu0V43jX6BDFHkoTRXKqrGDiEqBRNyCKhfZrkIHEpaYHqV1 HEiYffnAUl1j808JobOoHtS9UIUvImyWaQTJZgiMRX00BsVJh8WwjVyIUsiiP8e9IXsQ vPMSZjZIZuqhYIY3MB+qx4yA+D5G54IrctLX/TDryN3WYG7m1c6Z8JcSVlT9EysJo5xZ I+VtuaCmaXcCUzJKHruZyOUlcVCsLWFCTXuCdQNtYVpFGdZyQupL8BXnEQAFf3DhPIZX K5QQ== X-Gm-Message-State: ALoCoQl5q1oVxb+9k3X5yDo5se5Zn1Vv6k75EdwwAFT+m2t5YuAJ6m01+S702kHfDMVShzHiOyxO X-Received: by 10.180.79.5 with SMTP id f5mr42445739wix.46.1433343585477; Wed, 03 Jun 2015 07:59:45 -0700 (PDT) Received: from [192.168.1.102] (c-71-234-225-223.hsd1.ct.comcast.net. [71.234.225.223]) by mx.google.com with ESMTPSA id o5sm27263248wia.0.2015.06.03.07.59.43 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 03 Jun 2015 07:59:44 -0700 (PDT) Message-ID: <556F165A.7000805@dev.mellanox.co.il> Date: Wed, 03 Jun 2015 10:59:38 -0400 From: Hal Rosenstock User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: "linux-rdma (linux-rdma@vger.kernel.org)" CC: Alex Netes , Ilya Nelkenbaum , "rafiw@mellanox.com" Subject: [PATCH opensm] Add cleanup of SA cache after handover Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Alex Netes Date: Sun, 19 Aug 2012 18:39:44 +0300 Previously, when SM becomes STANDBY after being MASTER, it preserved the SA cache. When the SM will become MASTER again, it's SA cache might be inconsistent. The solution is to clean the SA cache each time that the SM becomes STANDBY after a handover. Also, enhance drop_event_subscriptions option for ServiceRecords drop_event_subscriptions is supposed to be just drop SA event subscriptions (InformInfos). AGUID and MC subscriptions are always dropped as client reregister mechanism works there. As we are aware of potential issues with ServiceRecords (not being reregistered), dropping of those is also controlled by this option. Signed-off-by: Alex Netes Signed-off-by: Ilya Nelkenbaum Signed-off-by: Hal Rosenstock --- -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/opensm/osm_port.h b/include/opensm/osm_port.h index 206c4a9..da06d31 100644 --- a/include/opensm/osm_port.h +++ b/include/opensm/osm_port.h @@ -1611,5 +1611,61 @@ void osm_alias_guid_delete(IN OUT osm_alias_guid_t ** pp_alias_guid); * Port *********/ +/****f* OpenSM: Port/osm_alias_guid_get_alias_guid +* NAME +* osm_alias_guid_get_alias_guid +* +* DESCRIPTION +* This function retrieves alias guid from alias guid object. +* +* SYNOPSIS +*/ +static inline ib_net64_t osm_alias_guid_get_alias_guid(IN osm_alias_guid_t *p_alias_guid) +{ + CL_ASSERT(p_alias_guid); + return p_alias_guid->alias_guid; +} +/* +* PARAMETERS +* p_alias_guid +* [in] Pointer to a pointer to an alias guid object. +* +* RETURN VALUE +* This function returns the alias guid or NULL if fails. +* +* NOTES +* +* SEE ALSO +* Port +*********/ + +/****f* OpenSM: Port/osm_alias_guid_get_base_guid +* NAME +* osm_alias_guid_get_base_guid +* +* DESCRIPTION +* This function retrieves base guid from alias guid object. +* +* SYNOPSIS +*/ +static inline ib_net64_t osm_alias_guid_get_base_guid(IN osm_alias_guid_t *p_alias_guid) +{ + CL_ASSERT(p_alias_guid); + return osm_port_get_guid(p_alias_guid->p_base_port); +} +/* +* PARAMETERS +* p_alias_guid +* [in] Pointer to a pointer to an alias guid object. +* +* RETURN VALUE +* This function returns the base guid or NULL if fails. +* +* NOTES +* +* SEE ALSO +* Port +*********/ + END_C_DECLS #endif /* _OSM_PORT_H_ */ diff --git a/opensm/osm_state_mgr.c b/opensm/osm_state_mgr.c index 976a691..9094db6 100644 --- a/opensm/osm_state_mgr.c +++ b/opensm/osm_state_mgr.c @@ -69,6 +69,8 @@ #include #include #include +#include +#include extern void osm_drop_mgr_process(IN osm_sm_t * sm); extern int osm_qos_setup(IN osm_opensm_t * p_osm); @@ -291,6 +293,105 @@ static ib_api_status_t state_mgr_clean_known_lids(IN osm_sm_t * sm) } /********************************************************************** + Clear SA cache +**********************************************************************/ +static ib_api_status_t state_mgr_sa_clean(IN osm_sm_t * sm) +{ + ib_api_status_t status = IB_SUCCESS; + cl_qmap_t *p_port_guid_tbl; + osm_assigned_guids_t *p_assigned_guids, *p_next_assigned_guids; + osm_alias_guid_t *p_alias_guid, *p_next_alias_guid; + osm_mcm_port_t *mcm_port; + osm_subn_t * p_subn; + osm_port_t *p_port; + osm_infr_t *p_infr; + osm_svcr_t *p_svcr; + + OSM_LOG_ENTER(sm->p_log); + + p_subn = sm->p_subn; + + /* we need a lock here! */ + CL_PLOCK_EXCL_ACQUIRE(sm->p_lock); + + if (p_subn->opt.drop_event_subscriptions) { + /* Clean InformInfo records */ + p_infr = (osm_infr_t *) cl_qlist_remove_head(&p_subn->sa_infr_list); + while (p_infr != + (osm_infr_t *) cl_qlist_end(&p_subn->sa_infr_list)) { + osm_infr_delete(p_infr); + p_infr = (osm_infr_t *) cl_qlist_remove_head(&p_subn->sa_infr_list); + } + + /* For now, treat Service Records in same category as InformInfos */ + /* Clean Service records */ + p_svcr = (osm_svcr_t *) cl_qlist_remove_head(&p_subn->sa_sr_list); + while (p_svcr != + (osm_svcr_t *) cl_qlist_end(&p_subn->sa_sr_list)) { + osm_svcr_delete(p_svcr); + p_svcr = (osm_svcr_t *) cl_qlist_remove_head(&p_subn->sa_sr_list); + } + } + + /* Clean Multicast member list on each port */ + p_port_guid_tbl = &p_subn->port_guid_tbl; + for (p_port = (osm_port_t *) cl_qmap_head(p_port_guid_tbl); + p_port != (osm_port_t *) cl_qmap_end(p_port_guid_tbl); + p_port = (osm_port_t *) cl_qmap_next(&p_port->map_item)) { + while (!cl_is_qlist_empty(&p_port->mcm_list)) { + mcm_port = cl_item_obj(cl_qlist_head(&p_port->mcm_list), + mcm_port, list_item); + osm_mgrp_delete_port(p_subn, sm->p_log, mcm_port->mgrp, + p_port); + } + /* Hack - clean alias guid table from physp */ + free(p_port->p_physp->p_guids); + p_port->p_physp->p_guids = NULL; + } + + /* Clean Alias Guid work objects */ + while (cl_qlist_count(&p_subn->alias_guid_list)) + osm_guid_work_obj_delete((osm_guidinfo_work_obj_t *) + cl_qlist_remove_head(&p_subn->alias_guid_list)); + + /* Clean Assigned GUIDs table */ + p_next_assigned_guids = (osm_assigned_guids_t *) + cl_qmap_head(&p_subn->assigned_guids_tbl); + while (p_next_assigned_guids != + (osm_assigned_guids_t *) cl_qmap_end(&p_subn->assigned_guids_tbl)) { + p_assigned_guids = p_next_assigned_guids; + p_next_assigned_guids = (osm_assigned_guids_t *) + cl_qmap_next(&p_assigned_guids->map_item); + cl_qmap_remove_item(&p_subn->assigned_guids_tbl, + &p_assigned_guids->map_item); + osm_assigned_guids_delete(&p_assigned_guids); + } + + /* Clean Alias GUIDs table */ + p_next_alias_guid = (osm_alias_guid_t *) + cl_qmap_head(&p_subn->alias_port_guid_tbl); + while (p_next_alias_guid != + (osm_alias_guid_t *) cl_qmap_end(&p_subn->alias_port_guid_tbl)) { + p_alias_guid = p_next_alias_guid; + p_next_alias_guid = (osm_alias_guid_t *) + cl_qmap_next(&p_alias_guid->map_item); + if (osm_alias_guid_get_alias_guid(p_alias_guid) != + osm_alias_guid_get_base_guid(p_alias_guid)) { + /* Clean if it's not base port GUID */ + cl_qmap_remove_item(&p_subn->alias_port_guid_tbl, + &p_alias_guid->map_item); + osm_alias_guid_delete(&p_alias_guid); + } + } + + p_subn->p_osm->sa.dirty = TRUE; + + CL_PLOCK_RELEASE(sm->p_lock); + OSM_LOG_EXIT(sm->p_log); + return status; +} + +/********************************************************************** Notifies the transport layer that the local LID has changed, which give it a chance to update address vectors, etc.. **********************************************************************/ @@ -1197,6 +1298,12 @@ static void do_sweep(osm_sm_t * sm) state_mgr_clean_known_lids(sm); /* + * Need to clean SA cache when state changes to STANDBY + * after handover. + */ + state_mgr_sa_clean(sm); + + /* * Need to reconfigure LFTs, PKEYs, and QoS on all switches * when coming out of STANDBY */ @@ -1323,6 +1430,7 @@ repeat_discovery: if (state_mgr_is_sm_port_down(sm) == TRUE) { if (sm->p_subn->last_sm_port_state) { sm->p_subn->last_sm_port_state = 0; + state_mgr_sa_clean(sm); osm_log_v2(sm->p_log, OSM_LOG_SYS, FILE_ID, "SM port is down\n"); OSM_LOG_MSG_BOX(sm->p_log, OSM_LOG_VERBOSE, diff --git a/opensm/osm_subnet.c b/opensm/osm_subnet.c index 26038c3..457772f 100644 --- a/opensm/osm_subnet.c +++ b/opensm/osm_subnet.c @@ -2613,7 +2613,7 @@ void osm_subn_output_conf(FILE *out, IN osm_subn_opt_t * p_opts) "sm_inactive %s\n\n" "# Babbling Port Policy\n" "babbling_port_policy %s\n\n" - "# Drop event subscriptions (InformInfo) if the port goes away\n" + "# Drop event subscriptions (InformInfo and ServiceRecord) on port removal and SM coming out of STANDBY\n" "drop_event_subscriptions %s\n\n" "# Use Optimized SLtoVLMapping programming if supported by device\n" "use_optimized_slvl %s\n\n"