From patchwork Wed Nov 28 19:03:19 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hal Rosenstock X-Patchwork-Id: 1817901 X-Patchwork-Delegate: alexne@voltaire.com Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 852F9DF26F for ; Wed, 28 Nov 2012 19:03:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754757Ab2K1TDY (ORCPT ); Wed, 28 Nov 2012 14:03:24 -0500 Received: from mail-bk0-f46.google.com ([209.85.214.46]:61131 "EHLO mail-bk0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754421Ab2K1TDX (ORCPT ); Wed, 28 Nov 2012 14:03:23 -0500 Received: by mail-bk0-f46.google.com with SMTP id q16so5900875bkw.19 for ; Wed, 28 Nov 2012 11:03:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :content-type:content-transfer-encoding:x-gm-message-state; bh=VtEEci90nBd0U3q6lTW71x9hMU1qmqDzEqG0Ge1YNds=; b=OpzDy5PWiT0MO4VmCaYQKfdoylvbsD85l3RGBZvm/CELjAD6GnVhA0e/paMSixgi4Y uzrJQ8zbSJHOnVWqnzEJCVqlJjf0zjK8feuzvq7yQS+5pi7iJtr84zwW1QjDw3XUw53D XoHcM0ka69x8qu8yJfq+OU1ZxuT9/p4babcc2456c2PZvTnOlEvyCdiev6gga2bIZmnE U1E5xMMiW0R227pmJJLcXRNTZzEr1m5tpnEAfuPbhohxNBPZQ30LhyGs5b2Z3lkBhnB1 8LBMcfKJMG151qIW9fwK6ME/rej+xycU3NkE8uJJF/n3Jqj4xD3BQMG7gOsnohyfZzVZ fOdw== Received: by 10.204.128.197 with SMTP id l5mr6211143bks.59.1354129402295; Wed, 28 Nov 2012 11:03:22 -0800 (PST) Received: from [192.168.1.102] (c-71-192-10-85.hsd1.ma.comcast.net. [71.192.10.85]) by mx.google.com with ESMTPS id 18sm4297225bkv.0.2012.11.28.11.03.20 (version=SSLv3 cipher=OTHER); Wed, 28 Nov 2012 11:03:21 -0800 (PST) Message-ID: <50B65FF7.3000307@dev.mellanox.co.il> Date: Wed, 28 Nov 2012 14:03:19 -0500 From: Hal Rosenstock User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: Alex Netes CC: "linux-rdma (linux-rdma@vger.kernel.org)" , Jim Schutt Subject: [PATCH 4/8] opensm: Update doc for changes to torus routing for CA, support X-Gm-Message-State: ALoCoQmW7ZjcKV5/QN1H83eWr/zvWZy9x6lWXKMIMRr4KbAnlrPJZIE6lnARI43DPb0Use/idF3H Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Signed-off-by: Hal Rosenstock Signed-off-by: Jim Schutt --- doc/current-routing.txt | 25 +++++++++++++++++-------- man/torus-2QoS.8.in | 30 +++++++++++++++++++----------- 2 files changed, 36 insertions(+), 19 deletions(-) diff --git a/doc/current-routing.txt b/doc/current-routing.txt index 495a0d2..c984801 100644 --- a/doc/current-routing.txt +++ b/doc/current-routing.txt @@ -1,5 +1,5 @@ Current OpenSM Routing -7/4/11 +11/14/12 OpenSM offers nine routing engines: @@ -746,14 +746,23 @@ Since SL to VL map configuration must be under the complete control of torus-2QoS, any configuration via qos_sl2vl, qos_swe_sl2vl, etc., must and will be ignored, and a warning will be generated. -Torus-2QoS uses VL values 0-3 to implement one of its supported QoS levels, -and VL values 4-7 to implement the other. Hard-to-diagnose application +For inter-switch links, Torus-2QoS uses VL values 0-3 to implement one of +its supported QoS levels, and VL values 4-7 to implement the other. For +CA links, Torus-2QoS uses VL value 0 for one of its supported QoS +levels and VL value 1 to implement the other. Hard-to-diagnose application issues may arise if traffic is not delivered fairly across each of these -two VL ranges. Torus-2QoS will detect and warn if VL arbitration is -configured unfairly across VLs in the range 0-3, and also in the range -4-7. Note that the default OpenSM VL arbitration configuration does not -meet this constraint, so all torus-2QoS users should configure VL -arbitration via qos_vlarb_high, qos_vlarb_low, etc. +two VL ranges. For inter-switch links, Torus-2QoS will detect and warn if +VL arbitration is configured unfairly across VLs in the range 0-3, and also +in the range 4-7. Note that the default OpenSM VL arbitration configuration +does not meet this constraint, so all torus-2QoS users should configure VL +arbitration via qos_ca_vlarb_high, qos_swe_vlarb_high, qos_ca_vlarb_low, +qos_swe_vlarb_low, etc. + +Note that torus-2QoS maps SL values to VL values differently +for inter-switch and CA links. This is why qos_vlarb_high and +qos_vlarb_low should not be used, as using them may result in +VL arbitration for a QoS level being different across inter-switch +links vs. across CA links. Operational Considerations: diff --git a/man/torus-2QoS.8.in b/man/torus-2QoS.8.in index be74f26..d62beba 100644 --- a/man/torus-2QoS.8.in +++ b/man/torus-2QoS.8.in @@ -19,8 +19,8 @@ Routing that is free of credit loops. .in \[en] 'in +2m -Two levels of Quality of Service (QoS), assuming switches and channel -adapters support eight data VLs. +Two levels of Quality of Service (QoS), assuming switches support eight +data VLs and channel adapters support two data VLs. .in \[en] 'in +2m @@ -404,15 +404,23 @@ Since SL to VL map configuration must be under the complete control of torus-2QoS, any configuration via qos_sl2vl, qos_swe_sl2vl, \fIetc.\fR, must and will be ignored, and a warning will be generated. .P -Torus-2QoS uses VL values 0-3 to implement one of its supported QoS -levels, and VL values 4-7 to implement the other. Hard-to-diagnose -application issues may arise if traffic is not delivered fairly -across each of these two VL ranges. -Torus-2QoS will detect and warn if VL arbitration is configured -unfairly across VLs in the range 0-3, and also in the range 4-7. -Note that the default OpenSM VL arbitration configuration -does not meet this constraint, so all torus-2QoS users should -configure VL arbitration via qos_vlarb_high, qos_vlarb_low, \fIetc.\fR +For inter-switch links, Torus-2QoS uses VL values 0-3 to implement one of +its supported QoS levels, and VL values 4-7 to implement the other. For +CA links, Torus-2QoS uses VL value 0 for one of its supported QoS +levels and VL value 1 to implement the other. Hard-to-diagnose application +issues may arise if traffic is not delivered fairly across each of these +two VL ranges. For inter-switch links, Torus-2QoS will detect and warn if +VL arbitration is configured unfairly across VLs in the range 0-3, and also +in the range 4-7. Note that the default OpenSM VL arbitration configuration +does not meet this constraint, so all torus-2QoS users should configure VL +arbitration via qos_ca_vlarb_high, qos_swe_vlarb_high, qos_ca_vlarb_low, +qos_swe_vlarb_low, \fIetc.\fR +.P +Note that torus-2QoS maps SL values to VL values differently +for inter-switch and CA links. This is why qos_vlarb_high and +qos_vlarb_low should not be used, as using them may result in +VL arbitration for a QoS level being different across inter-switch +links vs. across CA links. . .SH OPERATIONAL CONSIDERATIONS .