From patchwork Wed May 22 16:06:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Proshkin X-Patchwork-Id: 13670970 X-Patchwork-Delegate: bhelgaas@google.com Received: from mta-04.yadro.com (mta-04.yadro.com [89.207.88.248]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E5CA3716D for ; Wed, 22 May 2024 16:14:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.207.88.248 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394478; cv=none; b=NlYOiABjFGEyLRkcpoKtIwz2FNu6Zjt/KCSKrpxDC6HNeyYhA2ozWRfaECG6xu/DRS208yKEewWnY4DxKwzGdpwcmf8/FF7OPAPVbFbV6x7rqydrlmCKt/p/XAQhKkEeeH0cjW3yFB29ClWstWvEQKQlMsry76XD6lLv1jkd1pw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394478; c=relaxed/simple; bh=WArgKRqIZJHfBQWJC/Txb5NijV0EbdIZY4dLVULGMWw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=laTDjpJmQRqx0MrWWM7Jouh4Sf0R5Xx/6EdhIhveJgtiCNjEO6DXXQj7cVV1umond614o2YJaHwKX7kchRD6JEIsW1dC42JT+AUnQF8igF/TkzU9iJevRLyIXX1sPogfET0m6+PdvJYkLV+l76Topr7PLtJ+SOiUlRYjk82pUK8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com; spf=pass smtp.mailfrom=yadro.com; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=GEEPRggY; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=HRsuarvn; arc=none smtp.client-ip=89.207.88.248 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=yadro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="GEEPRggY"; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="HRsuarvn" DKIM-Filter: OpenDKIM Filter v2.11.0 mta-04.yadro.com D5401C0002 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-04; t=1716394046; bh=6LnXonmUDYG/FVfq6GHJUUzs67aSqZMtmH3nQ3nxI0w=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=GEEPRggYgPprUjCUDxnc3+0wcnXPLO8HzTrbm+/++Zwh+sVCixcPsO9R15pk91+PW c09r1K1+eHWlOzNZ9p5X1ycwxp2IOneBd4GLxww/VM9DC3BFBn8jukMhDZBuZGgB/O qx70pXTHLrYgsbSZa0xXwp2TlHEvXNkdhLKlZ0LQFb65fzaqNWRZ9giLcN+f+VWHTM 8kWmwO4QnzOZlxgRm1BC3YM36PSzS9Wg/jNmpsuo1SXLeG9R4kpznnUr2INK9g/uto d+4I/TM4QGePGtL3NVr8QFWbdPjd9/7cxbp5Nx73z6NshKb2i0rG9mA5OC3ti5cjiO 3NvXoP/AdR0Vw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-03; t=1716394046; bh=6LnXonmUDYG/FVfq6GHJUUzs67aSqZMtmH3nQ3nxI0w=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=HRsuarvnYlrqxI6Uw9Z934Nt1hqZSAG30ZPMDz/E89HEoMnmbF7TdOfRBdtTX74DQ ss9+PXzQT8k7KPqMNHo1MrHEQmHBUtZ12P6BZS9o80iTi7UUnnaNOcWT3Oe+vk2X1Q gKg3l71AiR46ujOULc0YEqCQHV9dbXPyjkQtjYD3Bk0MrHFpS+0XYGpOpXxiEtmDv+ t52kzm6N23FZPYjkAuajJZxdVcSLLlL11geXElpF4Y7jJp0lQiWowDlORDzEf0R6eX 6QacvKQHA9fQh8ghYkA8nfwXCAIhnXHtHAAD92F8bxRLkUog0XRWinChLxhoGg3dzE TkEg5umd8BBhg== From: Nikita Proshkin To: , Martin Mares CC: , Sergei Miroshnichenko , Nikita Proshkin Subject: [PATCH pciutils 1/6] pcilmr: Ensure that utility can accept either Downstream or Upstream link port Date: Wed, 22 May 2024 19:06:29 +0300 Message-ID: <20240522160634.29831-2-n.proshkin@yadro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240522160634.29831-1-n.proshkin@yadro.com> References: <20240522160634.29831-1-n.proshkin@yadro.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: T-EXCH-08.corp.yadro.com (172.17.11.58) To S-Exch-02.corp.yadro.com (10.78.5.239) Previously, the utility expected only the Upstream Port to be input and, in fact, passing the Downstream Port led to strange and buggy error messages. Improve arguments parsing logic to accept any side of the link. It seems that the only use case that will not be available now is margining the internal links of the switch, but this scenario looks as strange as possible. Signed-off-by: Nikita Proshkin --- lmr/lmr.h | 14 +++++++++---- lmr/margin.c | 27 ++----------------------- lmr/margin_hw.c | 54 ++++++++++++++++++++++++++++++++++++++++++------- pcilmr.c | 40 +++++++++++++----------------------- 4 files changed, 73 insertions(+), 62 deletions(-) diff --git a/lmr/lmr.h b/lmr/lmr.h index 7375c33..6cac3d4 100644 --- a/lmr/lmr.h +++ b/lmr/lmr.h @@ -1,7 +1,7 @@ /* * The PCI Utilities -- Margining utility main header * - * Copyright (c) 2023 KNS Group LLC (YADRO) + * Copyright (c) 2023-2024 KNS Group LLC (YADRO) * * Can be freely distributed and used under the terms of the GNU GPL v2+. * @@ -95,7 +95,7 @@ enum margin_test_status { /* All lanes Receiver results */ struct margin_results { - u8 recvn; // Receiver Number + u8 recvn; // Receiver Number; from 1 to 6 struct margin_params params; bool lane_reversal; u8 link_speed; @@ -104,7 +104,7 @@ struct margin_results { /* Used to convert steps to physical quantity. Calculated from MaxOffset and NumSteps */ - double tim_coef; + double tim_coef; // from steps to % UI double volt_coef; bool tim_off_reported; @@ -134,7 +134,7 @@ struct margin_args { /* Receiver structure */ struct margin_recv { struct margin_dev *dev; - u8 recvn; // Receiver Number + u8 recvn; // Receiver Number; from 1 to 6 bool lane_reversal; struct margin_params *params; @@ -161,6 +161,12 @@ struct margin_lanes_data { /* margin_hw */ +bool margin_port_is_down(struct pci_dev *dev); + +/* Results through down/up ports */ +bool margin_find_pair(struct pci_access *pacc, struct pci_dev *dev, struct pci_dev **down_port, + struct pci_dev **up_port); + /* Verify that devices form the link with 16 GT/s or 32 GT/s data rate */ bool margin_verify_link(struct pci_dev *down_port, struct pci_dev *up_port); diff --git a/lmr/margin.c b/lmr/margin.c index a8c6571..e3758df 100644 --- a/lmr/margin.c +++ b/lmr/margin.c @@ -426,13 +426,8 @@ margin_read_params(struct pci_access *pacc, struct pci_dev *dev, u8 recvn, struct pci_cap *cap = pci_find_cap(dev, PCI_CAP_ID_EXP, PCI_CAP_NORMAL); if (!cap) return false; - u8 dev_dir = GET_REG_MASK(pci_read_word(dev, cap->addr + PCI_EXP_FLAGS), PCI_EXP_FLAGS_TYPE); - bool dev_down; - if (dev_dir == PCI_EXP_TYPE_ROOT_PORT || dev_dir == PCI_EXP_TYPE_DOWNSTREAM) - dev_down = true; - else - dev_down = false; + bool dev_down = margin_port_is_down(dev); if (recvn == 0) { @@ -453,25 +448,7 @@ margin_read_params(struct pci_access *pacc, struct pci_dev *dev, u8 recvn, struct pci_dev *up = NULL; struct margin_link link; - for (struct pci_dev *p = pacc->devices; p; p = p->next) - { - if (dev_down && pci_read_byte(dev, PCI_SECONDARY_BUS) == p->bus && dev->domain == p->domain - && p->func == 0) - { - down = dev; - up = p; - break; - } - else if (!dev_down && pci_read_byte(p, PCI_SECONDARY_BUS) == dev->bus - && dev->domain == p->domain) - { - down = p; - up = dev; - break; - } - } - - if (!down) + if (!margin_find_pair(pacc, dev, &down, &up)) return false; if (!margin_fill_link(down, up, &link)) diff --git a/lmr/margin_hw.c b/lmr/margin_hw.c index fc427c8..43fa1bd 100644 --- a/lmr/margin_hw.c +++ b/lmr/margin_hw.c @@ -1,7 +1,7 @@ /* * The PCI Utilities -- Verify and prepare devices before margining * - * Copyright (c) 2023 KNS Group LLC (YADRO) + * Copyright (c) 2023-2024 KNS Group LLC (YADRO) * * Can be freely distributed and used under the terms of the GNU GPL v2+. * @@ -31,6 +31,51 @@ detect_unique_hw(struct pci_dev *dev) return MARGIN_HW_DEFAULT; } +bool +margin_port_is_down(struct pci_dev *dev) +{ + struct pci_cap *cap = pci_find_cap(dev, PCI_CAP_ID_EXP, PCI_CAP_NORMAL); + if (!cap) + return false; + u8 type = pci_read_byte(dev, PCI_HEADER_TYPE) & 0x7F; + u8 dir = GET_REG_MASK(pci_read_word(dev, cap->addr + PCI_EXP_FLAGS), PCI_EXP_FLAGS_TYPE); + + if (type == PCI_HEADER_TYPE_BRIDGE + && (dir == PCI_EXP_TYPE_ROOT_PORT || dir == PCI_EXP_TYPE_DOWNSTREAM)) + return true; + else + return false; +} + +bool +margin_find_pair(struct pci_access *pacc, struct pci_dev *dev, struct pci_dev **down_port, + struct pci_dev **up_port) +{ + struct pci_cap *cap = pci_find_cap(dev, PCI_CAP_ID_EXP, PCI_CAP_NORMAL); + if (!cap) + return false; + bool given_down = margin_port_is_down(dev); + + for (struct pci_dev *p = pacc->devices; p; p = p->next) + { + if (given_down && pci_read_byte(dev, PCI_SECONDARY_BUS) == p->bus && dev->domain == p->domain + && p->func == 0) + { + *down_port = dev; + *up_port = p; + return true; + } + else if (!given_down && pci_read_byte(p, PCI_SECONDARY_BUS) == dev->bus + && dev->domain == p->domain) + { + *down_port = p; + *up_port = dev; + return true; + } + } + return false; +} + bool margin_verify_link(struct pci_dev *down_port, struct pci_dev *up_port) { @@ -42,16 +87,11 @@ margin_verify_link(struct pci_dev *down_port, struct pci_dev *up_port) if ((pci_read_word(down_port, cap->addr + PCI_EXP_LNKSTA) & PCI_EXP_LNKSTA_SPEED) > 5) return false; - u8 down_type = pci_read_byte(down_port, PCI_HEADER_TYPE) & 0x7F; u8 down_sec = pci_read_byte(down_port, PCI_SECONDARY_BUS); - u8 down_dir - = GET_REG_MASK(pci_read_word(down_port, cap->addr + PCI_EXP_FLAGS), PCI_EXP_FLAGS_TYPE); // Verify that devices are linked, down_port is Root Port or Downstream Port of Switch, // up_port is Function 0 of a Device - if (!(down_sec == up_port->bus && down_type == PCI_HEADER_TYPE_BRIDGE - && (down_dir == PCI_EXP_TYPE_ROOT_PORT || down_dir == PCI_EXP_TYPE_DOWNSTREAM) - && up_port->func == 0)) + if (!(down_sec == up_port->bus && margin_port_is_down(down_port) && up_port->func == 0)) return false; struct pci_cap *pm = pci_find_cap(up_port, PCI_CAP_ID_PM, PCI_CAP_NORMAL); diff --git a/pcilmr.c b/pcilmr.c index cb8bd77..345ce7a 100644 --- a/pcilmr.c +++ b/pcilmr.c @@ -1,7 +1,7 @@ /* * The PCI Utilities -- Margining utility main function * - * Copyright (c) 2023 KNS Group LLC (YADRO) + * Copyright (c) 2023-2024 KNS Group LLC (YADRO) * * Can be freely distributed and used under the terms of the GNU GPL v2+. * @@ -83,21 +83,6 @@ dev_for_filter(struct pci_access *pacc, char *filter) die("No such PCI device: %s or you don't have enough privileges.\n", filter); } -static struct pci_dev * -find_down_port_for_up(struct pci_access *pacc, struct pci_dev *up) -{ - struct pci_dev *down = NULL; - for (struct pci_dev *p = pacc->devices; p; p = p->next) - { - if (pci_read_byte(p, PCI_SECONDARY_BUS) == up->bus && up->domain == p->domain) - { - down = p; - break; - } - } - return down; -} - static u8 parse_csv_arg(char *arg, u8 *vals) { @@ -120,11 +105,13 @@ scan_links(struct pci_access *pacc, bool only_ready) else printf("Links with Lane Margining at the Receiver capabilities:\n"); bool flag = true; - for (struct pci_dev *up = pacc->devices; up; up = up->next) + for (struct pci_dev *p = pacc->devices; p; p = p->next) { - if (pci_find_cap(up, PCI_EXT_CAP_ID_LMR, PCI_CAP_EXTENDED)) + if (pci_find_cap(p, PCI_EXT_CAP_ID_LMR, PCI_CAP_EXTENDED) && margin_port_is_down(p)) { - struct pci_dev *down = find_down_port_for_up(pacc, up); + struct pci_dev *down = NULL; + struct pci_dev *up = NULL; + margin_find_pair(pacc, p, &down, &up); if (down && margin_verify_link(down, up)) { @@ -147,11 +134,13 @@ find_ready_links(struct pci_access *pacc, struct pci_dev **down_ports, struct pc bool cnt_only) { u8 cnt = 0; - for (struct pci_dev *up = pacc->devices; up; up = up->next) + for (struct pci_dev *p = pacc->devices; p; p = p->next) { - if (pci_find_cap(up, PCI_EXT_CAP_ID_LMR, PCI_CAP_EXTENDED)) + if (pci_find_cap(p, PCI_EXT_CAP_ID_LMR, PCI_CAP_EXTENDED) && margin_port_is_down(p)) { - struct pci_dev *down = find_down_port_for_up(pacc, up); + struct pci_dev *down = NULL; + struct pci_dev *up = NULL; + margin_find_pair(pacc, p, &down, &up); if (down && margin_verify_link(down, up) && (margin_check_ready_bit(down) || margin_check_ready_bit(up))) @@ -328,10 +317,9 @@ main(int argc, char **argv) u8 cnt = 0; while (optind != argc) { - up_ports[cnt] = dev_for_filter(pacc, argv[optind]); - down_ports[cnt] = find_down_port_for_up(pacc, up_ports[cnt]); - if (!down_ports[cnt]) - die("Cannot find Upstream Component for the specified device: %s\n", argv[optind]); + struct pci_dev *dev = dev_for_filter(pacc, argv[optind]); + if (!margin_find_pair(pacc, dev, &(down_ports[cnt]), &(up_ports[cnt]))) + die("Cannot find pair for the specified device: %s\n", argv[optind]); cnt++; optind++; } From patchwork Wed May 22 16:06:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Proshkin X-Patchwork-Id: 13670971 X-Patchwork-Delegate: bhelgaas@google.com Received: from mta-04.yadro.com (mta-04.yadro.com [89.207.88.248]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E54F1F17B for ; Wed, 22 May 2024 16:14:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.207.88.248 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394479; cv=none; b=l7WzfpIto48eRpmOK4RWrpYkx9eFoxRuOLtY/W5tOoI+olPnlUZipuHhxwy8/sHIjIECfv2d1hQLojAA0QfgFxmVERaGVI2qLOo0Q8hvrdftzR1/K/EtrXYyxEYErHYnc+uqjL+/9cdb6McQ4l02oyk1PkjydKjnMFe76GfxtF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394479; c=relaxed/simple; bh=60Rw+fe2C49j7CQRYjAe7D73tZELduOGgmas1ZN+/vQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FltrZrmU5CMP2G/lzyOvlr3VkRuldjeWlcw4wLJIu2gqOL6YEymh482L4oN5DuLGHJOk/b1LmyVIU5VK6JK0NImIwNzq6YWrKOojuZOukuz31BAVghU91fBOBLBLOs/rmF90WcIxIKu+C2nFi8xgVJxEmX9znowfu2QQZ9STnZo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com; spf=pass smtp.mailfrom=yadro.com; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=pDCZ+ehd; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=xtoeb8KR; arc=none smtp.client-ip=89.207.88.248 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=yadro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="pDCZ+ehd"; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="xtoeb8KR" DKIM-Filter: OpenDKIM Filter v2.11.0 mta-04.yadro.com 7A865C0002 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-04; t=1716394048; bh=v3eVbveUtchmcH6oeQST2ELhKIr74aeJZvKsCaqUOsA=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=pDCZ+ehd6unegF2K9cgZY6P8YL/QdpQ9bg6OeHDKJJ1SMud7/pOFKXh5iX+sKKwym nYxjdgBcD6JCyEsMEpCyQGttsa4Vjq3tAxw/WQGwVU6yLiWYeoWB7S5xeEpD0UICYV sp5bcw/RyBEOUuExWFVI7yBZj2zN5WDNnt9tJqN1qWyw0jF5FRSWkFj9TzBXjuYug3 OTFuQ4hr2+lEDwra9+yIE6hg4SJgl3mVDcFcBeEiRyLNFrYy2WAbQYXD0ElXEIEcbU PPdaQ+zGvFG8LWD/+IQySD34BEO+gBpgZEL9skL053Xn3TLkvfjAICOEehI2g3FL+A 2qvNJSj9rve6w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-03; t=1716394048; bh=v3eVbveUtchmcH6oeQST2ELhKIr74aeJZvKsCaqUOsA=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=xtoeb8KRjD/1INX3xu0r/QbAQIoWrM7kWiJWivtLWEDUf0OiffwYk5inOdyA7KuQt y2JWWAKtMfvxl6KLYOekBW+jbyfuW0UqKdqPn/YhH4OjEK5DdFzi67F79/q12EIw2x omwodmlRiIEQfH9Njzg+1QZlEWXjHX4XGDnQhWNhEKk+5T77uCeAqNJWaROdQ6WgTL MpGK7UygSW6HVKKBRxZdiqUg31Sx079Eud0iGd/3sS90gAnn/SfPRhqV6LIMHPfzz4 TOEF3+4G+gzazpLrwmAb2k+TBko/NBtz0vZTbYJegCywEb6ozMI6QoLHOUk9HZDEey mwrR1+aHnObCA== From: Nikita Proshkin To: , Martin Mares CC: , Sergei Miroshnichenko , Nikita Proshkin Subject: [PATCH pciutils 2/6] pcilmr: Move most of pcilmr arguments parsing logic to the separate file Date: Wed, 22 May 2024 19:06:30 +0300 Message-ID: <20240522160634.29831-3-n.proshkin@yadro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240522160634.29831-1-n.proshkin@yadro.com> References: <20240522160634.29831-1-n.proshkin@yadro.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: T-EXCH-08.corp.yadro.com (172.17.11.58) To S-Exch-02.corp.yadro.com (10.78.5.239) Also change arguments parsing logic: now link parameters (selected lane numbers, timing or voltage steps, etc) need to be specified after link port and will affect only this link margining (previously, one option was applied to all links). See updated man for syntax and example. Signed-off-by: Nikita Proshkin --- Makefile | 2 +- lmr/lmr.h | 50 ++++--- lmr/margin.c | 32 +++-- lmr/margin_args.c | 263 +++++++++++++++++++++++++++++++++++ lmr/margin_hw.c | 3 + lmr/margin_log.c | 13 +- lmr/margin_results.c | 14 +- pcilmr.c | 323 ++++++------------------------------------- 8 files changed, 379 insertions(+), 321 deletions(-) create mode 100644 lmr/margin_args.c diff --git a/Makefile b/Makefile index be23593..abb5cb4 100644 --- a/Makefile +++ b/Makefile @@ -67,7 +67,7 @@ PCIINC_INS=lib/config.h lib/header.h lib/pci.h lib/types.h UTILINC=pciutils.h bitops.h $(PCIINC) -LMR=margin_hw.o margin.o margin_log.o margin_results.o +LMR=margin_hw.o margin.o margin_log.o margin_results.o margin_args.o LMROBJS=$(addprefix lmr/,$(LMR)) LMRINC=lmr/lmr.h $(UTILINC) diff --git a/lmr/lmr.h b/lmr/lmr.h index 6cac3d4..71480f0 100644 --- a/lmr/lmr.h +++ b/lmr/lmr.h @@ -39,11 +39,6 @@ struct margin_dev { bool hawd; // Hardware Autonomous Width Disable }; -struct margin_link { - struct margin_dev down_port; - struct margin_dev up_port; -}; - /* Specification Revision 5.0 Table 8-11 */ struct margin_params { bool ind_error_sampler; @@ -115,20 +110,33 @@ struct margin_results { }; /* pcilmr arguments */ -struct margin_args { + +// Common args +struct margin_com_args { + u8 error_limit; // [0; 63] + bool run_margin; // Or print params only + u8 verbosity; // 0 - basic; + // 1 - add info about remaining time and lanes in progress during margining + u64 steps_utility; // For ETA logging + bool save_csv; + char *dir_for_csv; +}; + +struct margin_link_args { + struct margin_com_args *common; u8 steps_t; // 0 == use NumTimingSteps u8 steps_v; // 0 == use NumVoltageSteps u8 parallel_lanes; // [1; MaxLanes + 1] - u8 error_limit; // [0; 63] u8 recvs[6]; // Receivers Numbers u8 recvs_n; // 0 == margin all available receivers u8 lanes[32]; // Lanes to Margin u8 lanes_n; // 0 == margin all available lanes - bool run_margin; // Or print params only - u8 verbosity; // 0 - basic; - // 1 - add info about remaining time and lanes in progress during margining +}; - u64 *steps_utility; // For ETA logging +struct margin_link { + struct margin_dev down_port; + struct margin_dev up_port; + struct margin_link_args args; }; /* Receiver structure */ @@ -159,6 +167,15 @@ struct margin_lanes_data { u8 verbosity; }; +/* margin_args */ + +enum margin_mode { MARGIN, FULL, SCAN }; + +extern const char *usage; + +struct margin_link *margin_parse_util_args(struct pci_access *pacc, int argc, char **argv, + enum margin_mode mode, u8 *links_n); + /* margin_hw */ bool margin_port_is_down(struct pci_dev *dev); @@ -189,12 +206,11 @@ void margin_restore_link(struct margin_link *link); bool margin_read_params(struct pci_access *pacc, struct pci_dev *dev, u8 recvn, struct margin_params *params); -enum margin_test_status margin_process_args(struct margin_dev *dev, struct margin_args *args); +enum margin_test_status margin_process_args(struct margin_link *link); -/* Awaits that args are prepared through process_args. +/* Awaits that links are prepared through process_args. Returns number of margined Receivers through recvs_n */ -struct margin_results *margin_test_link(struct margin_link *link, struct margin_args *args, - u8 *recvs_n); +struct margin_results *margin_test_link(struct margin_link *link, u8 *recvs_n); void margin_free_results(struct margin_results *results, u8 results_n); @@ -207,6 +223,7 @@ void margin_log(char *format, ...); /* b:d.f -> b:d.f */ void margin_log_bdfs(struct pci_dev *down_port, struct pci_dev *up_port); +void margin_gen_bdfs(struct pci_dev *down_port, struct pci_dev *up_port, char *dest, size_t maxlen); /* Print Link header (bdfs, width, speed) */ void margin_log_link(struct margin_link *link); @@ -228,7 +245,6 @@ void margin_log_hw_quirks(struct margin_recv *recv); void margin_results_print_brief(struct margin_results *results, u8 recvs_n); -void margin_results_save_csv(struct margin_results *results, u8 recvs_n, char *dir, - struct pci_dev *up_port); +void margin_results_save_csv(struct margin_results *results, u8 recvs_n, struct margin_link *link); #endif diff --git a/lmr/margin.c b/lmr/margin.c index e3758df..e2ea300 100644 --- a/lmr/margin.c +++ b/lmr/margin.c @@ -1,7 +1,7 @@ /* * The PCI Utilities -- Obtain the margin information of the Link * - * Copyright (c) 2023 KNS Group LLC (YADRO) + * Copyright (c) 2023-2024 KNS Group LLC (YADRO) * * Can be freely distributed and used under the terms of the GNU GPL v2+. * @@ -300,7 +300,7 @@ margin_test_lanes(struct margin_lanes_data arg) /* Awaits that Receiver is prepared through prep_dev function */ static bool -margin_test_receiver(struct margin_dev *dev, u8 recvn, struct margin_args *args, +margin_test_receiver(struct margin_dev *dev, u8 recvn, struct margin_link_args *args, struct margin_results *results) { u8 *lanes_to_margin = args->lanes; @@ -312,7 +312,7 @@ margin_test_receiver(struct margin_dev *dev, u8 recvn, struct margin_args *args, .lane_reversal = false, .params = ¶ms, .parallel_lanes = args->parallel_lanes ? args->parallel_lanes : 1, - .error_limit = args->error_limit }; + .error_limit = args->common->error_limit }; results->recvn = recvn; results->lanes_n = lanes_n; @@ -364,12 +364,13 @@ margin_test_receiver(struct margin_dev *dev, u8 recvn, struct margin_args *args, = recv.lane_reversal ? dev->width - lanes_to_margin[i] - 1 : lanes_to_margin[i]; } - if (args->run_margin) + if (args->common->run_margin) { - if (args->verbosity > 0) + if (args->common->verbosity > 0) margin_log("\n"); - struct margin_lanes_data lanes_data - = { .recv = &recv, .verbosity = args->verbosity, .steps_utility = args->steps_utility }; + struct margin_lanes_data lanes_data = { .recv = &recv, + .verbosity = args->common->verbosity, + .steps_utility = &args->common->steps_utility }; enum margin_dir dir[] = { TIM_LEFT, TIM_RIGHT, VOLT_UP, VOLT_DOWN }; @@ -399,15 +400,15 @@ margin_test_receiver(struct margin_dev *dev, u8 recvn, struct margin_args *args, lanes_data.ind = timing ? params.ind_left_right_tim : params.ind_up_down_volt; lanes_data.dir = dir[i]; lanes_data.steps_lane_total = timing ? steps_t : steps_v; - if (*args->steps_utility >= lanes_data.steps_lane_total) - *args->steps_utility -= lanes_data.steps_lane_total; + if (args->common->steps_utility >= lanes_data.steps_lane_total) + args->common->steps_utility -= lanes_data.steps_lane_total; else - *args->steps_utility = 0; + args->common->steps_utility = 0; margin_test_lanes(lanes_data); } lanes_done += use_lanes; } - if (args->verbosity > 0) + if (args->common->verbosity > 0) margin_log("\n"); if (recv.lane_reversal) { @@ -476,8 +477,11 @@ margin_read_params(struct pci_access *pacc, struct pci_dev *dev, u8 recvn, } enum margin_test_status -margin_process_args(struct margin_dev *dev, struct margin_args *args) +margin_process_args(struct margin_link *link) { + struct margin_dev *dev = &link->down_port; + struct margin_link_args *args = &link->args; + u8 receivers_n = 2 + 2 * dev->retimers_n; if (!args->recvs_n) @@ -520,8 +524,10 @@ margin_process_args(struct margin_dev *dev, struct margin_args *args) } struct margin_results * -margin_test_link(struct margin_link *link, struct margin_args *args, u8 *recvs_n) +margin_test_link(struct margin_link *link, u8 *recvs_n) { + struct margin_link_args *args = &link->args; + bool status = margin_prep_link(link); u8 receivers_n = status ? args->recvs_n : 1; diff --git a/lmr/margin_args.c b/lmr/margin_args.c new file mode 100644 index 0000000..c2d2e79 --- /dev/null +++ b/lmr/margin_args.c @@ -0,0 +1,263 @@ +/* + * The PCI Utilities -- Parse pcilmr utility arguments + * + * Copyright (c) 2024 KNS Group LLC (YADRO) + * + * Can be freely distributed and used under the terms of the GNU GPL v2+. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include +#include +#include + +#include "lmr.h" + +const char* usage + = "! Utility requires preliminary preparation of the system. Refer to the pcilmr man page !\n\n" + "Usage:\n" + "pcilmr [--margin] [] ...\n" + "pcilmr --full []\n" + "pcilmr --scan\n\n" + "Device Specifier:\n" + ":\t[:]:.\n\n" + "Modes:\n" + "--margin\t\tMargin selected Links\n" + "--full\t\t\tMargin all ready for testing Links in the system (one by one)\n" + "--scan\t\t\tScan for Links available for margining\n\n" + "Margining options:\n\n" + "Margining Test settings:\n" + "-c\t\t\tPrint Device Lane Margining Capabilities only. Do not run margining.\n" + "-l [,...]\tSpecify lanes for margining. Default: all link lanes.\n" + "\t\t\tRemember that Device may use Lane Reversal for Lane numbering.\n" + "\t\t\tHowever, utility uses logical lane numbers in arguments and for logging.\n" + "\t\t\tUtility will automatically determine Lane Reversal and tune its calls.\n" + "-e \t\tSpecify Error Count Limit for margining. Default: 4.\n" + "-r [,...]\tSpecify Receivers to select margining targets.\n" + "\t\t\tDefault: all available Receivers (including Retimers).\n" + "-p \tSpecify number of lanes to margin simultaneously.\n" + "\t\t\tDefault: 1.\n" + "\t\t\tAccording to spec it's possible for Receiver to margin up\n" + "\t\t\tto MaxLanes + 1 lanes simultaneously, but usually this works\n" + "\t\t\tbad, so this option is for experiments mostly.\n" + "-T\t\t\tTime Margining will continue until the Error Count is no more\n" + "\t\t\tthan an Error Count Limit. Use this option to find Link limit.\n" + "-V\t\t\tSame as -T option, but for Voltage.\n" + "-t \t\tSpecify maximum number of steps for Time Margining.\n" + "-v \t\tSpecify maximum number of steps for Voltage Margining.\n" + "Use only one of -T/-t options at the same time (same for -V/-v).\n" + "Without these options utility will use MaxSteps from Device\n" + "capabilities as test limit.\n\n" + "Margining Log settings:\n" + "-o \t\tSave margining results in csv form into the\n" + "\t\t\tspecified directory. Utility will generate file with the\n" + "\t\t\tname in form of 'lmr__Rx#_.csv'\n" + "\t\t\tfor each successfully tested receiver.\n"; + +static struct pci_dev * +dev_for_filter(struct pci_access *pacc, char *filter) +{ + struct pci_filter pci_filter; + pci_filter_init(pacc, &pci_filter); + if (pci_filter_parse_slot(&pci_filter, filter)) + die("Invalid device ID: %s\n", filter); + + if (pci_filter.bus == -1 || pci_filter.slot == -1 || pci_filter.func == -1) + die("Invalid device ID: %s\n", filter); + + if (pci_filter.domain == -1) + pci_filter.domain = 0; + + for (struct pci_dev *p = pacc->devices; p; p = p->next) + { + if (pci_filter_match(&pci_filter, p)) + return p; + } + + die("No such PCI device: %s or you don't have enough privileges.\n", filter); +} + +static u8 +parse_csv_arg(char *arg, u8 *vals) +{ + u8 cnt = 0; + char *token = strtok(arg, ","); + while (token) + { + vals[cnt] = atoi(token); + cnt++; + token = strtok(NULL, ","); + } + return cnt; +} + +static u8 +find_ready_links(struct pci_access *pacc, struct margin_link *links, bool cnt_only) +{ + u8 cnt = 0; + for (struct pci_dev *p = pacc->devices; p; p = p->next) + { + if (pci_find_cap(p, PCI_EXT_CAP_ID_LMR, PCI_CAP_EXTENDED) && margin_port_is_down(p)) + { + struct pci_dev *down = NULL; + struct pci_dev *up = NULL; + margin_find_pair(pacc, p, &down, &up); + + if (down && margin_verify_link(down, up) + && (margin_check_ready_bit(down) || margin_check_ready_bit(up))) + { + if (!cnt_only) + margin_fill_link(down, up, &(links[cnt])); + cnt++; + } + } + } + return cnt; +} + +static void +init_link_args(struct margin_link_args *link_args, struct margin_com_args *com_args) +{ + memset(link_args, 0, sizeof(*link_args)); + link_args->common = com_args; + link_args->parallel_lanes = 1; +} + +static void +parse_dev_args(int argc, char **argv, struct margin_link_args *args, u8 link_speed) +{ + if (argc == optind) + return; + int c; + while ((c = getopt(argc, argv, "+r:l:p:t:v:VT")) != -1) + { + switch (c) + { + case 't': + args->steps_t = atoi(optarg); + break; + case 'T': + args->steps_t = 63; + break; + case 'v': + args->steps_v = atoi(optarg); + break; + case 'V': + args->steps_v = 127; + break; + case 'p': + args->parallel_lanes = atoi(optarg); + break; + case 'l': + args->lanes_n = parse_csv_arg(optarg, args->lanes); + break; + case 'r': + args->recvs_n = parse_csv_arg(optarg, args->recvs); + break; + case '?': + die("Invalid arguments\n\n%s", usage); + break; + default: + return; + } + } +} + +struct margin_link * +margin_parse_util_args(struct pci_access *pacc, int argc, char **argv, enum margin_mode mode, + u8 *links_n) +{ + struct margin_com_args *com_args = xmalloc(sizeof(*com_args)); + com_args->error_limit = 4; + com_args->run_margin = true; + com_args->verbosity = 1; + com_args->steps_utility = 0; + com_args->dir_for_csv = NULL; + com_args->save_csv = false; + + int c; + while ((c = getopt(argc, argv, "+e:co:")) != -1) + { + switch (c) + { + case 'c': + com_args->run_margin = false; + break; + case 'e': + com_args->error_limit = atoi(optarg); + break; + case 'o': + com_args->dir_for_csv = optarg; + com_args->save_csv = true; + break; + default: + die("Invalid arguments\n\n%s", usage); + } + } + + bool status = true; + if (mode == FULL && optind != argc) + status = false; + if (mode == MARGIN && optind == argc) + status = false; + if (!status && argc > 1) + die("Invalid arguments\n\n%s", usage); + if (!status) + { + printf("%s", usage); + exit(0); + } + + u8 ports_n = 0; + struct margin_link *links = NULL; + char err[128]; + + if (mode == FULL) + { + ports_n = find_ready_links(pacc, NULL, true); + if (ports_n == 0) + die("Links not found or you don't have enough privileges.\n"); + else + { + links = xmalloc(ports_n * sizeof(*links)); + find_ready_links(pacc, links, false); + for (int i = 0; i < ports_n; i++) + init_link_args(&(links[i].args), com_args); + } + } + else if (mode == MARGIN) + { + while (optind != argc) + { + struct pci_dev *dev = dev_for_filter(pacc, argv[optind]); + optind++; + links = xrealloc(links, (ports_n + 1) * sizeof(*links)); + struct pci_dev *down; + struct pci_dev *up; + if (!margin_find_pair(pacc, dev, &down, &up)) + die("Cannot find pair for the specified device: %s\n", argv[optind]); + struct pci_cap *cap = pci_find_cap(down, PCI_CAP_ID_EXP, PCI_CAP_NORMAL); + if (!cap) + die("Looks like you don't have enough privileges to access " + "Device Configuration Space.\nTry to run utility as root.\n"); + if (!margin_fill_link(down, up, &(links[ports_n]))) + { + margin_gen_bdfs(down, up, err, sizeof(err)); + die("Link %s is not ready for margining.\n" + "Link data rate must be 16 GT/s or 32 GT/s.\n" + "Downstream Component must be at D0 PM state.\n", + err); + } + init_link_args(&(links[ports_n].args), com_args); + parse_dev_args(argc, argv, &(links[ports_n].args), + links[ports_n].down_port.link_speed - 4); + ports_n++; + } + } + else + die("Bug in the args parsing!\n"); + + *links_n = ports_n; + return links; +} diff --git a/lmr/margin_hw.c b/lmr/margin_hw.c index 43fa1bd..2585ca1 100644 --- a/lmr/margin_hw.c +++ b/lmr/margin_hw.c @@ -8,6 +8,8 @@ * SPDX-License-Identifier: GPL-2.0-or-later */ +#include + #include "lmr.h" static u16 special_hw[][4] = @@ -125,6 +127,7 @@ fill_dev_wrapper(struct pci_dev *dev) bool margin_fill_link(struct pci_dev *down_port, struct pci_dev *up_port, struct margin_link *wrappers) { + memset(wrappers, 0, sizeof(*wrappers)); if (!margin_verify_link(down_port, up_port)) return false; wrappers->down_port = fill_dev_wrapper(down_port); diff --git a/lmr/margin_log.c b/lmr/margin_log.c index b3c4bd5..6fa4f09 100644 --- a/lmr/margin_log.c +++ b/lmr/margin_log.c @@ -1,7 +1,7 @@ /* * The PCI Utilities -- Log margining process * - * Copyright (c) 2023 KNS Group LLC (YADRO) + * Copyright (c) 2023-2024 KNS Group LLC (YADRO) * * Can be freely distributed and used under the terms of the GNU GPL v2+. * @@ -37,6 +37,17 @@ margin_log_bdfs(struct pci_dev *down, struct pci_dev *up) up->func); } +void +margin_gen_bdfs(struct pci_dev *down, struct pci_dev *up, char *dest, size_t maxlen) +{ + if (margin_print_domain) + snprintf(dest, maxlen, "%x:%x:%x.%x -> %x:%x:%x.%x", down->domain, down->bus, down->dev, + down->func, up->domain, up->bus, up->dev, up->func); + else + snprintf(dest, maxlen, "%x:%x.%x -> %x:%x.%x", down->bus, down->dev, down->func, up->bus, + up->dev, up->func); +} + void margin_log_link(struct margin_link *link) { diff --git a/lmr/margin_results.c b/lmr/margin_results.c index 4d28f04..b320e90 100644 --- a/lmr/margin_results.c +++ b/lmr/margin_results.c @@ -1,7 +1,7 @@ /* * The PCI Utilities -- Display/save margining results * - * Copyright (c) 2023 KNS Group LLC (YADRO) + * Copyright (c) 2023-2024 KNS Group LLC (YADRO) * * Can be freely distributed and used under the terms of the GNU GPL v2+. * @@ -172,13 +172,13 @@ margin_results_print_brief(struct margin_results *results, u8 recvs_n) } void -margin_results_save_csv(struct margin_results *results, u8 recvs_n, char *dir, - struct pci_dev *up_port) +margin_results_save_csv(struct margin_results *results, u8 recvs_n, struct margin_link *link) { char timestamp[64]; time_t tim = time(NULL); strftime(timestamp, sizeof(timestamp), "%Y-%m-%dT%H:%M:%S", gmtime(&tim)); + char *dir = link->args.common->dir_for_csv; size_t pathlen = strlen(dir) + 128; char *path = xmalloc(pathlen); FILE *csv; @@ -190,6 +190,8 @@ margin_results_save_csv(struct margin_results *results, u8 recvs_n, char *dir, enum lane_rating lane_rating; u8 link_speed; + struct pci_dev *port; + for (int i = 0; i < recvs_n; i++) { res = &(results[i]); @@ -198,9 +200,11 @@ margin_results_save_csv(struct margin_results *results, u8 recvs_n, char *dir, if (res->test_status != MARGIN_TEST_OK) continue; + + port = res->recvn == 6 ? link->up_port.dev : link->down_port.dev; snprintf(path, pathlen, "%s/lmr_%0*x.%02x.%02x.%x_Rx%X_%s.csv", dir, - up_port->domain_16 == 0xffff ? 8 : 4, up_port->domain, up_port->bus, up_port->dev, - up_port->func, 10 + res->recvn - 1, timestamp); + port->domain_16 == 0xffff ? 8 : 4, port->domain, port->bus, port->dev, + port->func, 10 + res->recvn - 1, timestamp); csv = fopen(path, "w"); if (!csv) die("Error while saving %s\n", path); diff --git a/pcilmr.c b/pcilmr.c index 345ce7a..f1ef140 100644 --- a/pcilmr.c +++ b/pcilmr.c @@ -11,92 +11,11 @@ #include #include #include -#include #include "lmr/lmr.h" const char program_name[] = "pcilmr"; -enum mode { MARGIN, FULL, SCAN }; - -static const char usage_msg[] - = "! Utility requires preliminary preparation of the system. Refer to the pcilmr man page !\n\n" - "Usage:\n" - "pcilmr [--margin] [] ...\n" - "pcilmr --full []\n" - "pcilmr --scan\n\n" - "Device Specifier:\n" - ":\t[:]:.\n\n" - "Modes:\n" - "--margin\t\tMargin selected Links\n" - "--full\t\t\tMargin all ready for testing Links in the system (one by one)\n" - "--scan\t\t\tScan for Links available for margining\n\n" - "Margining options:\n\n" - "Margining Test settings:\n" - "-c\t\t\tPrint Device Lane Margining Capabilities only. Do not run margining.\n" - "-l [,...]\tSpecify lanes for margining. Default: all link lanes.\n" - "\t\t\tRemember that Device may use Lane Reversal for Lane numbering.\n" - "\t\t\tHowever, utility uses logical lane numbers in arguments and for logging.\n" - "\t\t\tUtility will automatically determine Lane Reversal and tune its calls.\n" - "-e \t\tSpecify Error Count Limit for margining. Default: 4.\n" - "-r [,...]\tSpecify Receivers to select margining targets.\n" - "\t\t\tDefault: all available Receivers (including Retimers).\n" - "-p \tSpecify number of lanes to margin simultaneously.\n" - "\t\t\tDefault: 1.\n" - "\t\t\tAccording to spec it's possible for Receiver to margin up\n" - "\t\t\tto MaxLanes + 1 lanes simultaneously, but usually this works\n" - "\t\t\tbad, so this option is for experiments mostly.\n" - "-T\t\t\tTime Margining will continue until the Error Count is no more\n" - "\t\t\tthan an Error Count Limit. Use this option to find Link limit.\n" - "-V\t\t\tSame as -T option, but for Voltage.\n" - "-t \t\tSpecify maximum number of steps for Time Margining.\n" - "-v \t\tSpecify maximum number of steps for Voltage Margining.\n" - "Use only one of -T/-t options at the same time (same for -V/-v).\n" - "Without these options utility will use MaxSteps from Device\n" - "capabilities as test limit.\n\n" - "Margining Log settings:\n" - "-o \t\tSave margining results in csv form into the\n" - "\t\t\tspecified directory. Utility will generate file with the\n" - "\t\t\tname in form of 'lmr__Rx#_.csv'\n" - "\t\t\tfor each successfully tested receiver.\n"; - -static struct pci_dev * -dev_for_filter(struct pci_access *pacc, char *filter) -{ - struct pci_filter pci_filter; - pci_filter_init(pacc, &pci_filter); - if (pci_filter_parse_slot(&pci_filter, filter)) - die("Invalid device ID: %s\n", filter); - - if (pci_filter.bus == -1 || pci_filter.slot == -1 || pci_filter.func == -1) - die("Invalid device ID: %s\n", filter); - - if (pci_filter.domain == -1) - pci_filter.domain = 0; - - for (struct pci_dev *p = pacc->devices; p; p = p->next) - { - if (pci_filter_match(&pci_filter, p)) - return p; - } - - die("No such PCI device: %s or you don't have enough privileges.\n", filter); -} - -static u8 -parse_csv_arg(char *arg, u8 *vals) -{ - u8 cnt = 0; - char *token = strtok(arg, ","); - while (token) - { - vals[cnt] = atoi(token); - cnt++; - token = strtok(NULL, ","); - } - return cnt; -} - static void scan_links(struct pci_access *pacc, bool only_ready) { @@ -129,72 +48,21 @@ scan_links(struct pci_access *pacc, bool only_ready) exit(0); } -static u8 -find_ready_links(struct pci_access *pacc, struct pci_dev **down_ports, struct pci_dev **up_ports, - bool cnt_only) -{ - u8 cnt = 0; - for (struct pci_dev *p = pacc->devices; p; p = p->next) - { - if (pci_find_cap(p, PCI_EXT_CAP_ID_LMR, PCI_CAP_EXTENDED) && margin_port_is_down(p)) - { - struct pci_dev *down = NULL; - struct pci_dev *up = NULL; - margin_find_pair(pacc, p, &down, &up); - - if (down && margin_verify_link(down, up) - && (margin_check_ready_bit(down) || margin_check_ready_bit(up))) - { - if (!cnt_only) - { - up_ports[cnt] = up; - down_ports[cnt] = down; - } - cnt++; - } - } - } - return cnt; -} - int main(int argc, char **argv) { struct pci_access *pacc; - struct pci_dev **up_ports; - struct pci_dev **down_ports; - u8 ports_n = 0; - + u8 links_n = 0; struct margin_link *links; bool *checks_status_ports; - bool status = true; - enum mode mode; + enum margin_mode mode; /* each link has several receivers -> several results */ struct margin_results **results; u8 *results_n; - struct margin_args *args; - - u8 steps_t_arg = 0; - u8 steps_v_arg = 0; - u8 parallel_lanes_arg = 1; - u8 error_limit = 4; - u8 lanes_arg[32]; - u8 recvs_arg[6]; - - u8 lanes_n = 0; - u8 recvs_n = 0; - - bool run_margin = true; - - char *dir_for_csv = NULL; - bool save_csv = false; - - u64 total_steps = 0; - pacc = pci_alloc(); pci_init(pacc); pci_scan_bus(pacc); @@ -217,8 +85,9 @@ main(int argc, char **argv) { .name = "full", .has_arg = no_argument, .flag = NULL, .val = 2 }, { 0, 0, 0, 0 } }; + opterr = 0; int c; - c = getopt_long(argc, argv, ":", long_options, NULL); + c = getopt_long(argc, argv, "+", long_options, NULL); switch (c) { @@ -231,7 +100,8 @@ main(int argc, char **argv) mode = SCAN; if (optind == argc) scan_links(pacc, false); - optind--; + else + die("Invalid arguments\n\n%s", usage); break; case 2: mode = FULL; @@ -242,128 +112,20 @@ main(int argc, char **argv) break; } - while ((c = getopt(argc, argv, ":r:e:l:cp:t:v:VTo:")) != -1) - { - switch (c) - { - case 't': - steps_t_arg = atoi(optarg); - break; - case 'T': - steps_t_arg = 63; - break; - case 'v': - steps_v_arg = atoi(optarg); - break; - case 'V': - steps_v_arg = 127; - break; - case 'p': - parallel_lanes_arg = atoi(optarg); - break; - case 'c': - run_margin = false; - break; - case 'l': - lanes_n = parse_csv_arg(optarg, lanes_arg); - break; - case 'e': - error_limit = atoi(optarg); - break; - case 'r': - recvs_n = parse_csv_arg(optarg, recvs_arg); - break; - case 'o': - dir_for_csv = optarg; - save_csv = true; - break; - default: - die("Invalid arguments\n\n%s", usage_msg); - } - } - - if (mode == FULL && optind != argc) - status = false; - if (mode == MARGIN && optind == argc) - status = false; - if (!status && argc > 1) - die("Invalid arguments\n\n%s", usage_msg); - if (!status) - { - printf("%s", usage_msg); - exit(0); - } - - if (mode == FULL) - { - ports_n = find_ready_links(pacc, NULL, NULL, true); - if (ports_n == 0) - { - die("Links not found or you don't have enough privileges.\n"); - } - else - { - up_ports = xmalloc(ports_n * sizeof(*up_ports)); - down_ports = xmalloc(ports_n * sizeof(*down_ports)); - find_ready_links(pacc, down_ports, up_ports, false); - } - } - else if (mode == MARGIN) - { - ports_n = argc - optind; - up_ports = xmalloc(ports_n * sizeof(*up_ports)); - down_ports = xmalloc(ports_n * sizeof(*down_ports)); - - u8 cnt = 0; - while (optind != argc) - { - struct pci_dev *dev = dev_for_filter(pacc, argv[optind]); - if (!margin_find_pair(pacc, dev, &(down_ports[cnt]), &(up_ports[cnt]))) - die("Cannot find pair for the specified device: %s\n", argv[optind]); - cnt++; - optind++; - } - } - else - die("Bug in the args parsing!\n"); + opterr = 1; - if (!pci_find_cap(up_ports[0], PCI_CAP_ID_EXP, PCI_CAP_NORMAL)) - die("Looks like you don't have enough privileges to access " - "Device Configuration Space.\nTry to run utility as root.\n"); + links = margin_parse_util_args(pacc, argc, argv, mode, &links_n); + struct margin_com_args *com_args = links[0].args.common; - results = xmalloc(ports_n * sizeof(*results)); - results_n = xmalloc(ports_n * sizeof(*results_n)); - links = xmalloc(ports_n * sizeof(*links)); - checks_status_ports = xmalloc(ports_n * sizeof(*checks_status_ports)); - args = xmalloc(ports_n * sizeof(*args)); + results = xmalloc(links_n * sizeof(*results)); + results_n = xmalloc(links_n * sizeof(*results_n)); + checks_status_ports = xmalloc(links_n * sizeof(*checks_status_ports)); - for (int i = 0; i < ports_n; i++) + for (int i = 0; i < links_n; i++) { - args[i].error_limit = error_limit; - args[i].parallel_lanes = parallel_lanes_arg; - args[i].run_margin = run_margin; - args[i].verbosity = 1; - args[i].steps_t = steps_t_arg; - args[i].steps_v = steps_v_arg; - for (int j = 0; j < recvs_n; j++) - args[i].recvs[j] = recvs_arg[j]; - args[i].recvs_n = recvs_n; - for (int j = 0; j < lanes_n; j++) - args[i].lanes[j] = lanes_arg[j]; - args[i].lanes_n = lanes_n; - args[i].steps_utility = &total_steps; - enum margin_test_status args_status; - if (!margin_fill_link(down_ports[i], up_ports[i], &links[i])) - { - checks_status_ports[i] = false; - results[i] = xmalloc(sizeof(*results[i])); - results[i]->test_status = MARGIN_TEST_PREREQS; - continue; - } - - if ((args_status = margin_process_args(&links[i].down_port, &args[i])) != MARGIN_TEST_OK) + if ((args_status = margin_process_args(&links[i])) != MARGIN_TEST_OK) { checks_status_ports[i] = false; results[i] = xmalloc(sizeof(*results[i])); @@ -373,49 +135,44 @@ main(int argc, char **argv) checks_status_ports[i] = true; struct margin_params params; + struct margin_link_args *link_args = &links[i].args; - for (int j = 0; j < args[i].recvs_n; j++) + for (int j = 0; j < link_args->recvs_n; j++) { - if (margin_read_params(pacc, args[i].recvs[j] == 6 ? up_ports[i] : down_ports[i], - args[i].recvs[j], ¶ms)) + if (margin_read_params( + pacc, link_args->recvs[j] == 6 ? links[i].up_port.dev : links[i].down_port.dev, + link_args->recvs[j], ¶ms)) { - u8 steps_t = steps_t_arg ? steps_t_arg : params.timing_steps; - u8 steps_v = steps_v_arg ? steps_v_arg : params.volt_steps; - u8 parallel_recv = parallel_lanes_arg > params.max_lanes + 1 ? params.max_lanes + 1 : - parallel_lanes_arg; + u8 steps_t = link_args->steps_t ? link_args->steps_t : params.timing_steps; + u8 steps_v = link_args->steps_v ? link_args->steps_v : params.volt_steps; + u8 parallel_recv = link_args->parallel_lanes > params.max_lanes + 1 ? + params.max_lanes + 1 : + link_args->parallel_lanes; u8 step_multiplier - = args[i].lanes_n / parallel_recv + ((args[i].lanes_n % parallel_recv) > 0); + = link_args->lanes_n / parallel_recv + ((link_args->lanes_n % parallel_recv) > 0); - total_steps += steps_t * step_multiplier; + com_args->steps_utility += steps_t * step_multiplier; if (params.ind_left_right_tim) - total_steps += steps_t * step_multiplier; + com_args->steps_utility += steps_t * step_multiplier; if (params.volt_support) { - total_steps += steps_v * step_multiplier; + com_args->steps_utility += steps_v * step_multiplier; if (params.ind_up_down_volt) - total_steps += steps_v * step_multiplier; + com_args->steps_utility += steps_v * step_multiplier; } } } } - for (int i = 0; i < ports_n; i++) + for (int i = 0; i < links_n; i++) { if (checks_status_ports[i]) - results[i] = margin_test_link(&links[i], &args[i], &results_n[i]); + results[i] = margin_test_link(&links[i], &results_n[i]); else { results_n[i] = 1; - if (results[i]->test_status == MARGIN_TEST_PREREQS) - { - printf("Link "); - margin_log_bdfs(down_ports[i], up_ports[i]); - printf(" is not ready for margining.\n" - "Link data rate must be 16 GT/s or 32 GT/s.\n" - "Downstream Component must be at D0 PM state.\n"); - } - else if (results[i]->test_status == MARGIN_TEST_ARGS_RECVS) + if (results[i]->test_status == MARGIN_TEST_ARGS_RECVS) { margin_log_link(&links[i]); printf("\nInvalid RecNums specified.\n"); @@ -429,7 +186,7 @@ main(int argc, char **argv) printf("\n----\n\n"); } - if (run_margin) + if (com_args->run_margin) { printf("Results:\n"); printf("\nPass/fail criteria:\nTiming:\n"); @@ -442,27 +199,25 @@ main(int argc, char **argv) printf("THR -\tThe set (using the utility options) \n\tstep threshold has been reached\n\n"); printf("Notations:\nst - steps\n\n"); - for (int i = 0; i < ports_n; i++) + for (int i = 0; i < links_n; i++) { printf("Link "); - margin_log_bdfs(down_ports[i], up_ports[i]); + margin_log_bdfs(links[i].down_port.dev, links[i].up_port.dev); printf(":\n\n"); margin_results_print_brief(results[i], results_n[i]); - if (save_csv) - margin_results_save_csv(results[i], results_n[i], dir_for_csv, up_ports[i]); + if (com_args->save_csv) + margin_results_save_csv(results[i], results_n[i], &links[i]); printf("\n"); } } - for (int i = 0; i < ports_n; i++) + for (int i = 0; i < links_n; i++) margin_free_results(results[i], results_n[i]); free(results_n); free(results); - free(up_ports); - free(down_ports); + free(com_args); free(links); free(checks_status_ports); - free(args); pci_cleanup(pacc); return 0; From patchwork Wed May 22 16:06:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Proshkin X-Patchwork-Id: 13670964 X-Patchwork-Delegate: bhelgaas@google.com Received: from mta-04.yadro.com (mta-04.yadro.com [89.207.88.248]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D43A1139B for ; Wed, 22 May 2024 16:07:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.207.88.248 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394065; cv=none; b=hbBiFA5V2UP42NjFDrtfbRxqlg9g7ISMvlhDpDs76z37JALH4Ju9c4Wdk16466owkkCVqnd0PugdLGd8tGq6LCOeckALzbw1DisJoo2k9TA1CqOpvahmyRsqI18rrGaGVs2yAS2JaZEeyeE1OUW/cRhnPPYDhTylDxT1k8NIYoY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394065; c=relaxed/simple; bh=zRqEYzsq7wwyUBXmBzdQU3HgcWQgErIpIWR3sDLINCg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jx3Hc2ItoULYPq4mpcSvIrv+yhmZyIREqIMmW7+BraoTULKPG1pyMdxsw1KLzctybVZdhwyiLI63JZbl7fLUzfrVT2/pxJ92si8RWb4HWfw0DWJQcB5yYDxdKwMX5VjOLTw9JNNnG6uYXv6F3iWORD4taGtABHqHbA5sWm0OpZA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com; spf=pass smtp.mailfrom=yadro.com; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=wX+aRzLB; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=BvgoTXcu; arc=none smtp.client-ip=89.207.88.248 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=yadro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="wX+aRzLB"; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="BvgoTXcu" DKIM-Filter: OpenDKIM Filter v2.11.0 mta-04.yadro.com 45E2CC0002 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-04; t=1716394057; bh=+AyhGU3zrpsZkij0GMgpKz2Yen84USPT0xHPLckaPwU=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=wX+aRzLBxj5ipQY4Jhs3bwmfcMPjLAdpY82qjR1tLinLnu92Syd4xdo1ufPH+n7UG PrJ58sIB1KSjM/Pzp5WLvVHuaSqoGwWTjwabSV2EDEJeezCJc1dRy7r0H3det53PPX x9F/s+f7nvPh+IfxabumIrbhugwjUBFA5HinKlSKYFIihUWk4RJKdQAxoKFxQY+p6i 5uNdzrxiVH/D4ssT67VDADI1aVK/aoLRsiwpbauhg32NH0UEcJtzMKICitmElvNMUm tyhDFN26tTxZX5cYfwzUTaML6RzwXwp3Nlbu6wS9NlrFPLir2v+NWF0rKo9/wqtLMh 42O9dCdQRHeJg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-03; t=1716394057; bh=+AyhGU3zrpsZkij0GMgpKz2Yen84USPT0xHPLckaPwU=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=BvgoTXcuHle2w+rfYqXQxD21hoUEahV75Zr2zednuqhaj0ninRLix1Hsmv2hJPyZ2 5Xjur4wGeHakjJfMfNm2DSc1TRZiYE1dLO2r1rMVLCQAb/dxTt4x+hRUYYfW+0y1pi 4kcZ5Iv18kcUbhpyrmGY8QySWe4xpzWjhD+PmZdwkHQpJup9M16X7p9jK5x5hEbsTB oPSWHP8eZWuYiUXENCn+PNGrQnslbkoEzkIU2PKkUSE6pCL2Pn/uDLfExtDgXOdOnt re3+jA0jqBe8gXcSCuYx+89mlZCRP9aSej6h025XelQ75TY8xcA64tW+HJiH4c2d2Z zQBz9IJwFe0YA== From: Nikita Proshkin To: , Martin Mares CC: , Sergei Miroshnichenko , Nikita Proshkin Subject: [PATCH pciutils 3/6] pcilmr: Add new grading option Date: Wed, 22 May 2024 19:06:31 +0300 Message-ID: <20240522160634.29831-4-n.proshkin@yadro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240522160634.29831-1-n.proshkin@yadro.com> References: <20240522160634.29831-1-n.proshkin@yadro.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: T-EXCH-08.corp.yadro.com (172.17.11.58) To S-Exch-02.corp.yadro.com (10.78.5.239) Original version of the utility used values from the Table 8-11 of the PCIe Base Spec Rev 5.0 to evaluate lanes. But it seems that these values relate only to the margining equipment and are not relevant to evaluating the quality of connections. The PCIe Base Spec Rev 5.0 sets the minimum values for the eye in the section 8.4.2. Change default grading values in the utility according to this section. The specification uses the values of the full width and height of the eye, so add these values to the output of the utility. In addition, manufacturers can provide criteria for their devices that differ from the standard ones. Usually this information falls under the NDA, so add an option to the utility that will allow the user to set necessary criteria for evaluating the quality of lanes. Implement the following syntax for the -g(rading) option: -g 1t=15ps,f | -g 6v=20 Use passed per link receiver criteria for the eye width (timing - t) or height (voltage - v) in the utility results. Additional flag f is for situations when port doesn't support two side independent margining. In such cases by default calculate EW or EH as a double one side result. User can add f flag for -g option to tell the utility that the result in one direction is actually the measurement of the full eye (for example, Ice Lake RC ports work in this way) and it does not need to be multiplied. Signed-off-by: Nikita Proshkin --- lmr/lmr.h | 35 ++++- lmr/margin_args.c | 60 ++++++++- lmr/margin_results.c | 312 ++++++++++++++++++++++++++++--------------- pcilmr.c | 6 +- 4 files changed, 290 insertions(+), 123 deletions(-) diff --git a/lmr/lmr.h b/lmr/lmr.h index 71480f0..da40bfe 100644 --- a/lmr/lmr.h +++ b/lmr/lmr.h @@ -17,12 +17,11 @@ #define MARGIN_STEP_MS 1000 -#define MARGIN_TIM_MIN 20 -#define MARGIN_TIM_RECOMMEND 30 -#define MARGIN_VOLT_MIN 50 - enum margin_hw { MARGIN_HW_DEFAULT, MARGIN_ICE_LAKE_RC }; +// in ps +static const double margin_ui[] = { 62.5, 31.25 }; + /* PCI Device wrapper for margining functions */ struct margin_dev { struct pci_dev *dev; @@ -122,6 +121,15 @@ struct margin_com_args { char *dir_for_csv; }; +struct margin_recv_args { + // Grading options + struct { + bool valid; + double criteria; // in ps/mV + bool one_side_is_whole; + } t, v; +}; + struct margin_link_args { struct margin_com_args *common; u8 steps_t; // 0 == use NumTimingSteps @@ -129,8 +137,9 @@ struct margin_link_args { u8 parallel_lanes; // [1; MaxLanes + 1] u8 recvs[6]; // Receivers Numbers u8 recvs_n; // 0 == margin all available receivers - u8 lanes[32]; // Lanes to Margin - u8 lanes_n; // 0 == margin all available lanes + struct margin_recv_args recv_args[6]; + u8 lanes[32]; // Lanes to Margin + u8 lanes_n; // 0 == margin all available lanes }; struct margin_link { @@ -243,7 +252,19 @@ void margin_log_hw_quirks(struct margin_recv *recv); /* margin_results */ -void margin_results_print_brief(struct margin_results *results, u8 recvs_n); +// Min values are taken from PCIe Base Spec Rev. 5.0 Section 8.4.2. +// Rec values are based on PCIe Arch PHY Test Spec Rev 5.0 +// (Transmitter Electrical Compliance) + +// values in ps +static const double margin_ew_min[] = { 18.75, 9.375 }; +static const double margin_ew_rec[] = { 23.75, 10.1565 }; + +static const double margin_eh_min[] = { 15, 15 }; +static const double margin_eh_rec[] = { 21, 19.75 }; + +void margin_results_print_brief(struct margin_results *results, u8 recvs_n, + struct margin_link_args *args); void margin_results_save_csv(struct margin_results *results, u8 recvs_n, struct margin_link *link); diff --git a/lmr/margin_args.c b/lmr/margin_args.c index c2d2e79..484c58f 100644 --- a/lmr/margin_args.c +++ b/lmr/margin_args.c @@ -14,7 +14,7 @@ #include "lmr.h" -const char* usage +const char *usage = "! Utility requires preliminary preparation of the system. Refer to the pcilmr man page !\n\n" "Usage:\n" "pcilmr [--margin] [] ...\n" @@ -130,7 +130,7 @@ parse_dev_args(int argc, char **argv, struct margin_link_args *args, u8 link_spe if (argc == optind) return; int c; - while ((c = getopt(argc, argv, "+r:l:p:t:v:VT")) != -1) + while ((c = getopt(argc, argv, "+r:l:p:t:v:VTg:")) != -1) { switch (c) { @@ -155,6 +155,60 @@ parse_dev_args(int argc, char **argv, struct margin_link_args *args, u8 link_spe case 'r': args->recvs_n = parse_csv_arg(optarg, args->recvs); break; + case 'g': { + char recv[2] = { 0 }; + char dir[2] = { 0 }; + char unit[4] = { 0 }; + float criteria = 0.0; + char eye[2] = { 0 }; + int cons[3] = { 0 }; + + int ret = sscanf(optarg, "%1[1-6]%1[tv]=%f%n%3[%,ps]%n%1[f]%n", recv, dir, &criteria, + &cons[0], unit, &cons[1], eye, &cons[2]); + if (ret < 3) + { + ret = sscanf(optarg, "%1[1-6]%1[tv]=%1[f]%n,%f%n%2[ps%]%n", recv, dir, eye, + &cons[0], &criteria, &cons[1], unit, &cons[2]); + if (ret < 3) + die("Invalid arguments\n\n%s", usage); + } + + int consumed = 0; + for (int i = 0; i < 3; i++) + if (cons[i] > consumed) + consumed = cons[i]; + if ((size_t)consumed != strlen(optarg)) + die("Invalid arguments\n\n%s", usage); + if (criteria < 0) + die("Invalid arguments\n\n%s", usage); + if (strstr(unit, ",") && eye[0] == 0) + die("Invalid arguments\n\n%s", usage); + + u8 recv_n = recv[0] - '0' - 1; + if (dir[0] == 'v') + { + if (unit[0] != ',' && unit[0] != 0) + die("Invalid arguments\n\n%s", usage); + args->recv_args[recv_n].v.valid = true; + args->recv_args[recv_n].v.criteria = criteria; + if (eye[0] != 0) + args->recv_args[recv_n].v.one_side_is_whole = true; + } + else + { + if (unit[0] == '%') + criteria = criteria / 100.0 * margin_ui[link_speed]; + else if (unit[0] != 0 && (unit[0] != 'p' || unit[1] != 's')) + die("Invalid arguments\n\n%s", usage); + else if (unit[0] == 0 && criteria != 0) + die("Invalid arguments\n\n%s", usage); + args->recv_args[recv_n].t.valid = true; + args->recv_args[recv_n].t.criteria = criteria; + if (eye[0] != 0) + args->recv_args[recv_n].t.one_side_is_whole = true; + } + break; + } case '?': die("Invalid arguments\n\n%s", usage); break; @@ -236,7 +290,7 @@ margin_parse_util_args(struct pci_access *pacc, int argc, char **argv, enum marg struct pci_dev *down; struct pci_dev *up; if (!margin_find_pair(pacc, dev, &down, &up)) - die("Cannot find pair for the specified device: %s\n", argv[optind]); + die("Cannot find pair for the specified device: %s\n", argv[optind - 1]); struct pci_cap *cap = pci_find_cap(down, PCI_CAP_ID_EXP, PCI_CAP_NORMAL); if (!cap) die("Looks like you don't have enough privileges to access " diff --git a/lmr/margin_results.c b/lmr/margin_results.c index b320e90..b0c5c26 100644 --- a/lmr/margin_results.c +++ b/lmr/margin_results.c @@ -16,25 +16,23 @@ #include "lmr.h" enum lane_rating { - BAD = 0, - OKAY, + FAIL = 0, + PASS, PERFECT, - WEIRD, INIT, }; -static char *const grades[] = { "Bad", "Okay", "Perfect", "Weird" }; +static char *const grades[] = { "Fail", "Pass", "Perfect" }; static char *const sts_strings[] = { "NAK", "LIM", "THR" }; -static const double ui[] = { 62.5 / 100, 31.25 / 100 }; static enum lane_rating rate_lane(double value, double min, double recommended, enum lane_rating cur_rate) { enum lane_rating res = PERFECT; if (value < recommended) - res = OKAY; + res = PASS; if (value < min) - res = BAD; + res = FAIL; if (cur_rate == INIT) return res; if (res < cur_rate) @@ -43,34 +41,9 @@ rate_lane(double value, double min, double recommended, enum lane_rating cur_rat return cur_rate; } -static bool -check_recv_weird(struct margin_results *results, double tim_min, double volt_min) -{ - bool result = true; - - struct margin_res_lane *lane; - for (int i = 0; i < results->lanes_n && result; i++) - { - lane = &(results->lanes[i]); - if (lane->steps[TIM_LEFT] * results->tim_coef != tim_min) - result = false; - if (results->params.ind_left_right_tim - && lane->steps[TIM_RIGHT] * results->tim_coef != tim_min) - result = false; - if (results->params.volt_support) - { - if (lane->steps[VOLT_UP] * results->volt_coef != volt_min) - result = false; - if (results->params.ind_up_down_volt - && lane->steps[VOLT_DOWN] * results->volt_coef != volt_min) - result = false; - } - } - return result; -} - void -margin_results_print_brief(struct margin_results *results, u8 recvs_n) +margin_results_print_brief(struct margin_results *results, u8 recvs_n, + struct margin_link_args *args) { struct margin_res_lane *lane; struct margin_results *res; @@ -80,6 +53,14 @@ margin_results_print_brief(struct margin_results *results, u8 recvs_n) u8 link_speed; + struct margin_recv_args grade_args; + bool spec_ref_only; + + double ew_min; + double ew_rec; + double eh_min; + double eh_rec; + char *no_test_msgs[] = { "", "Margining Ready bit is Clear", "Error during caps reading", @@ -102,6 +83,67 @@ margin_results_print_brief(struct margin_results *results, u8 recvs_n) continue; } + spec_ref_only = true; + grade_args = args->recv_args[res->recvn - 1]; + if (grade_args.t.criteria != 0) + { + spec_ref_only = false; + ew_min = grade_args.t.criteria; + ew_rec = grade_args.t.criteria; + } + else + { + ew_min = margin_ew_min[link_speed]; + ew_rec = margin_ew_rec[link_speed]; + } + + if (grade_args.v.criteria != 0) + { + spec_ref_only = false; + eh_min = grade_args.v.criteria; + eh_rec = grade_args.v.criteria; + } + else + { + eh_min = margin_eh_min[link_speed]; + eh_rec = margin_eh_rec[link_speed]; + } + + printf("Rx(%X) - Grading criteria:\n", 10 + res->recvn - 1); + if (spec_ref_only) + { + printf("\tUsing spec only:\n"); + printf("\tEW: minimum - %.2f ps; recommended - %.2f ps\n", ew_min, ew_rec); + printf("\tEH: minimum - %.2f mV; recommended - %.2f mV\n\n", eh_min, eh_rec); + } + else + { + printf("\tEW: pass - %.2f ps\n", ew_min); + printf("\tEH: pass - %.2f mV\n\n", eh_min); + } + + if (!params.ind_left_right_tim) + { + printf("Rx(%X) - EW: independent left/right timing margin is not supported:\n", + 10 + res->recvn - 1); + if (grade_args.t.one_side_is_whole) + printf("\tmanual setting - the entire margin across the eye " + "is what is reported by one side margining\n\n"); + else + printf("\tdefault - calculating EW as double one side result\n\n"); + } + + if (params.volt_support && !params.ind_up_down_volt) + { + printf("Rx(%X) - EH: independent up and down voltage margining is not supported:\n", + 10 + res->recvn - 1); + if (grade_args.v.one_side_is_whole) + printf("\tmanual setting - the entire margin across the eye " + "is what is reported by one side margining\n\n"); + else + printf("\tdefault - calculating EH as double one side result\n\n"); + } + if (res->lane_reversal) printf("Rx(%X) - Lane Reversal\n", 10 + res->recvn - 1); @@ -118,51 +160,60 @@ margin_results_print_brief(struct margin_results *results, u8 recvs_n) "reliable.\n\n", 10 + res->recvn - 1); - if (check_recv_weird(res, MARGIN_TIM_MIN, MARGIN_VOLT_MIN)) - lane_rating = WEIRD; - else - lane_rating = INIT; - - for (u8 j = 0; j < res->lanes_n; j++) + for (int j = 0; j < res->lanes_n; j++) { + if (spec_ref_only) + lane_rating = INIT; + else + lane_rating = PASS; + lane = &(res->lanes[j]); - double left_ui = lane->steps[TIM_LEFT] * res->tim_coef; - double right_ui = lane->steps[TIM_RIGHT] * res->tim_coef; + double left_ps = lane->steps[TIM_LEFT] * res->tim_coef / 100.0 * margin_ui[link_speed]; + double right_ps = lane->steps[TIM_RIGHT] * res->tim_coef / 100.0 * margin_ui[link_speed]; double up_volt = lane->steps[VOLT_UP] * res->volt_coef; double down_volt = lane->steps[VOLT_DOWN] * res->volt_coef; - if (lane_rating != WEIRD) + double ew = left_ps; + if (params.ind_left_right_tim) + ew += right_ps; + else if (!grade_args.t.one_side_is_whole) + ew *= 2.0; + + double eh = 0.0; + if (params.volt_support) { - lane_rating = rate_lane(left_ui, MARGIN_TIM_MIN, MARGIN_TIM_RECOMMEND, INIT); - if (params.ind_left_right_tim) - lane_rating - = rate_lane(right_ui, MARGIN_TIM_MIN, MARGIN_TIM_RECOMMEND, lane_rating); - if (params.volt_support) - { - lane_rating = rate_lane(up_volt, MARGIN_VOLT_MIN, MARGIN_VOLT_MIN, lane_rating); - if (params.ind_up_down_volt) - lane_rating - = rate_lane(down_volt, MARGIN_VOLT_MIN, MARGIN_VOLT_MIN, lane_rating); - } + eh += up_volt; + if (params.ind_up_down_volt) + eh += down_volt; + else if (!grade_args.v.one_side_is_whole) + eh *= 2.0; } - printf("Rx(%X) Lane %2d - %s\t", 10 + res->recvn - 1, lane->lane, grades[lane_rating]); + lane_rating = rate_lane(ew, ew_min, ew_rec, lane_rating); + if (params.volt_support) + lane_rating = rate_lane(eh, eh_min, eh_rec, lane_rating); + + printf("Rx(%X) Lane %2d: %s\t (W %4.1f%% UI - %5.2fps", 10 + res->recvn - 1, lane->lane, + grades[lane_rating], ew / margin_ui[link_speed] * 100.0, ew); + if (params.volt_support) + printf(", H %5.1f mV", eh); if (params.ind_left_right_tim) - printf("L %4.1f%% UI - %5.2fps - %2dst %s, R %4.1f%% UI - %5.2fps - %2dst %s", left_ui, - left_ui * ui[link_speed], lane->steps[TIM_LEFT], - sts_strings[lane->statuses[TIM_LEFT]], right_ui, right_ui * ui[link_speed], - lane->steps[TIM_RIGHT], sts_strings[lane->statuses[TIM_RIGHT]]); + printf(") (L %4.1f%% UI - %5.2fps - %2dst %s) (R %4.1f%% UI - %5.2fps - %2dst %s)", + left_ps / margin_ui[link_speed] * 100.0, left_ps, lane->steps[TIM_LEFT], + sts_strings[lane->statuses[TIM_LEFT]], right_ps / margin_ui[link_speed] * 100.0, + right_ps, lane->steps[TIM_RIGHT], sts_strings[lane->statuses[TIM_RIGHT]]); else - printf("T %4.1f%% UI - %5.2fps - %2dst %s", left_ui, left_ui * ui[link_speed], - lane->steps[TIM_LEFT], sts_strings[lane->statuses[TIM_LEFT]]); + printf(") (T %4.1f%% UI - %5.2fps - %2dst %s)", + left_ps / margin_ui[link_speed] * 100.0, left_ps, lane->steps[TIM_LEFT], + sts_strings[lane->statuses[TIM_LEFT]]); if (params.volt_support) { if (params.ind_up_down_volt) - printf(", U %5.1f mV - %3dst %s, D %5.1f mV - %3dst %s", up_volt, + printf(" (U %5.1f mV - %3dst %s) (D %5.1f mV - %3dst %s)", up_volt, lane->steps[VOLT_UP], sts_strings[lane->statuses[VOLT_UP]], down_volt, lane->steps[VOLT_DOWN], sts_strings[lane->statuses[VOLT_DOWN]]); else - printf(", V %5.1f mV - %3dst %s", up_volt, lane->steps[VOLT_UP], + printf(" (V %5.1f mV - %3dst %s)", up_volt, lane->steps[VOLT_UP], sts_strings[lane->statuses[VOLT_UP]]); } printf("\n"); @@ -190,6 +241,14 @@ margin_results_save_csv(struct margin_results *results, u8 recvs_n, struct margi enum lane_rating lane_rating; u8 link_speed; + struct margin_recv_args grade_args; + bool spec_ref_only; + + double ew_min; + double ew_rec; + double eh_min; + double eh_rec; + struct pci_dev *port; for (int i = 0; i < recvs_n; i++) @@ -203,80 +262,117 @@ margin_results_save_csv(struct margin_results *results, u8 recvs_n, struct margi port = res->recvn == 6 ? link->up_port.dev : link->down_port.dev; snprintf(path, pathlen, "%s/lmr_%0*x.%02x.%02x.%x_Rx%X_%s.csv", dir, - port->domain_16 == 0xffff ? 8 : 4, port->domain, port->bus, port->dev, - port->func, 10 + res->recvn - 1, timestamp); + port->domain_16 == 0xffff ? 8 : 4, port->domain, port->bus, port->dev, port->func, + 10 + res->recvn - 1, timestamp); csv = fopen(path, "w"); if (!csv) die("Error while saving %s\n", path); - fprintf(csv, "Lane,Lane Status,Left %% UI,Left ps,Left Steps,Left Status," - "Right %% UI,Right ps,Right Steps,Right Status," - "Time %% UI,Time ps,Time Steps,Time Status," - "Up mV,Up Steps,Up Status,Down mV,Down Steps,Down Status," - "Voltage mV,Voltage Steps,Voltage Status\n"); + fprintf(csv, "Lane,EW Min,EW Rec,EW,EH Min,EH Rec,EH,Lane Status,Left %% UI,Left " + "ps,Left Steps,Left Status,Right %% UI,Right ps,Right Steps,Right Status,Up " + "mV,Up Steps,Up Status,Down mV,Down Steps,Down Status\n"); - if (check_recv_weird(res, MARGIN_TIM_MIN, MARGIN_VOLT_MIN)) - lane_rating = WEIRD; + spec_ref_only = true; + grade_args = link->args.recv_args[res->recvn - 1]; + if (grade_args.t.criteria != 0) + { + spec_ref_only = false; + ew_min = grade_args.t.criteria; + ew_rec = grade_args.t.criteria; + } + else + { + ew_min = margin_ew_min[link_speed]; + ew_rec = margin_ew_rec[link_speed]; + } + if (grade_args.v.criteria != 0) + { + spec_ref_only = false; + eh_min = grade_args.v.criteria; + eh_rec = grade_args.v.criteria; + } else - lane_rating = INIT; + { + eh_min = margin_eh_min[link_speed]; + eh_rec = margin_eh_rec[link_speed]; + } for (int j = 0; j < res->lanes_n; j++) { + if (spec_ref_only) + lane_rating = INIT; + else + lane_rating = PASS; + lane = &(res->lanes[j]); - double left_ui = lane->steps[TIM_LEFT] * res->tim_coef; - double right_ui = lane->steps[TIM_RIGHT] * res->tim_coef; + double left_ps = lane->steps[TIM_LEFT] * res->tim_coef / 100.0 * margin_ui[link_speed]; + double right_ps = lane->steps[TIM_RIGHT] * res->tim_coef / 100.0 * margin_ui[link_speed]; double up_volt = lane->steps[VOLT_UP] * res->volt_coef; double down_volt = lane->steps[VOLT_DOWN] * res->volt_coef; - if (lane_rating != WEIRD) + double ew = left_ps; + if (params.ind_left_right_tim) + ew += right_ps; + else if (!grade_args.t.one_side_is_whole) + ew *= 2.0; + + double eh = 0.0; + if (params.volt_support) { - lane_rating = rate_lane(left_ui, MARGIN_TIM_MIN, MARGIN_TIM_RECOMMEND, INIT); - if (params.ind_left_right_tim) - lane_rating - = rate_lane(right_ui, MARGIN_TIM_MIN, MARGIN_TIM_RECOMMEND, lane_rating); - if (params.volt_support) - { - lane_rating = rate_lane(up_volt, MARGIN_VOLT_MIN, MARGIN_VOLT_MIN, lane_rating); - if (params.ind_up_down_volt) - lane_rating - = rate_lane(down_volt, MARGIN_VOLT_MIN, MARGIN_VOLT_MIN, lane_rating); - } + eh += up_volt; + if (params.ind_up_down_volt) + eh += down_volt; + else if (!grade_args.v.one_side_is_whole) + eh *= 2.0; } - fprintf(csv, "%d,%s,", lane->lane, grades[lane_rating]); - if (params.ind_left_right_tim) + lane_rating = rate_lane(ew, ew_min, ew_rec, lane_rating); + if (params.volt_support) + lane_rating = rate_lane(eh, eh_min, eh_rec, lane_rating); + + fprintf(csv, "%d,%f,", lane->lane, ew_min); + if (spec_ref_only) + fprintf(csv, "%f,", ew_rec); + else + fprintf(csv, "NA,"); + fprintf(csv, "%f,", ew); + if (params.volt_support) { - fprintf(csv, "%f,%f,%d,%s,%f,%f,%d,%s,NA,NA,NA,NA,", left_ui, - left_ui * ui[link_speed], lane->steps[TIM_LEFT], - sts_strings[lane->statuses[TIM_LEFT]], right_ui, right_ui * ui[link_speed], - lane->steps[TIM_RIGHT], sts_strings[lane->statuses[TIM_RIGHT]]); + fprintf(csv, "%f,", eh_min); + if (spec_ref_only) + fprintf(csv, "%f,", eh_rec); + else + fprintf(csv, "NA,"); + fprintf(csv, "%f,", eh); } + else + fprintf(csv, "NA,NA,NA,"); + fprintf(csv, "%s,", grades[lane_rating]); + + fprintf(csv, "%f,%f,%d,%s,", left_ps * 100.0 / margin_ui[link_speed], left_ps, + lane->steps[TIM_LEFT], sts_strings[lane->statuses[TIM_LEFT]]); + + if (params.ind_left_right_tim) + fprintf(csv, "%f,%f,%d,%s,", right_ps * 100.0 / margin_ui[link_speed], right_ps, + lane->steps[TIM_RIGHT], sts_strings[lane->statuses[TIM_RIGHT]]); else { - for (int k = 0; k < 8; k++) + for (int k = 0; k < 4; k++) fprintf(csv, "NA,"); - fprintf(csv, "%f,%f,%d,%s,", left_ui, left_ui * ui[link_speed], lane->steps[TIM_LEFT], - sts_strings[lane->statuses[TIM_LEFT]]); } if (params.volt_support) { + fprintf(csv, "%f,%d,%s,", up_volt, lane->steps[VOLT_UP], + sts_strings[lane->statuses[VOLT_UP]]); if (params.ind_up_down_volt) - { - fprintf(csv, "%f,%d,%s,%f,%d,%s,NA,NA,NA\n", up_volt, lane->steps[VOLT_UP], - sts_strings[lane->statuses[VOLT_UP]], down_volt, lane->steps[VOLT_DOWN], - sts_strings[lane->statuses[VOLT_DOWN]]); - } + fprintf(csv, "%f,%d,%s\n", down_volt, lane->steps[VOLT_DOWN], + sts_strings[lane->statuses[VOLT_DOWN]]); else - { - for (int k = 0; k < 6; k++) - fprintf(csv, "NA,"); - fprintf(csv, "%f,%d,%s\n", up_volt, lane->steps[VOLT_UP], - sts_strings[lane->statuses[VOLT_UP]]); - } + fprintf(csv, "NA,NA,NA\n"); } else { - for (int k = 0; k < 8; k++) + for (int k = 0; k < 5; k++) fprintf(csv, "NA,"); fprintf(csv, "NA\n"); } diff --git a/pcilmr.c b/pcilmr.c index f1ef140..accee44 100644 --- a/pcilmr.c +++ b/pcilmr.c @@ -189,10 +189,6 @@ main(int argc, char **argv) if (com_args->run_margin) { printf("Results:\n"); - printf("\nPass/fail criteria:\nTiming:\n"); - printf("Minimum Offset (spec): %d %% UI\nRecommended Offset: %d %% UI\n", MARGIN_TIM_MIN, - MARGIN_TIM_RECOMMEND); - printf("\nVoltage:\nMinimum Offset (spec): %d mV\n\n", MARGIN_VOLT_MIN); printf( "Margining statuses:\nLIM -\tErrorCount exceeded Error Count Limit (found device limit)\n"); printf("NAK -\tDevice didn't execute last command, \n\tso result may be less reliable\n"); @@ -204,7 +200,7 @@ main(int argc, char **argv) printf("Link "); margin_log_bdfs(links[i].down_port.dev, links[i].up_port.dev); printf(":\n\n"); - margin_results_print_brief(results[i], results_n[i]); + margin_results_print_brief(results[i], results_n[i], &links[i].args); if (com_args->save_csv) margin_results_save_csv(results[i], results_n[i], &links[i]); printf("\n"); From patchwork Wed May 22 16:06:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Proshkin X-Patchwork-Id: 13670965 X-Patchwork-Delegate: bhelgaas@google.com Received: from mta-04.yadro.com (mta-04.yadro.com [89.207.88.248]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDE0224B2A for ; Wed, 22 May 2024 16:07:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.207.88.248 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394065; cv=none; b=ewzHc6CuuKItdoTq9buA3bI19Kj7ZcLIC2yCx0tmNDEyiVSqy7FoHoIHb65//v0cml0N9/JaDQ2K7+sYS0RRamGRzRuhHvCGkKaRIqdLSjuq3T02pR0DFkBT77qn9vlxTw8iS1FgRWeumXGb2f9UM8ossziKPJS8p6aQQHeargw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394065; c=relaxed/simple; bh=ySpoV1MLUxmI+QYzS6cHiRWO4CpfEvSrKjdbjsqxSn0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=oDMZKF2W/sjERnjQhxIUCyet4at30vsjoea75IKUS5U0tkxbP+JB/wVQRTzVS1sdWtXuZFJqwlWyOMADuMX7jjO0r7XOWCKXKCbcdMfJiG6Yr7gx5O+Xqi+c5Uh/c8rWdz4gh1hNlDr0z5yDrLO2urua3JIDRZQ0ZEPOIntj8Jo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com; spf=pass smtp.mailfrom=yadro.com; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=MdiQlWrr; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=NRqA7QEn; arc=none smtp.client-ip=89.207.88.248 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=yadro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="MdiQlWrr"; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="NRqA7QEn" DKIM-Filter: OpenDKIM Filter v2.11.0 mta-04.yadro.com 076A3C000B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-04; t=1716394060; bh=cQPL5uc2qmIz+bTbHSCmVXNVrh3as7mPmfeYqjnjq+Y=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=MdiQlWrrg+lU36v12/okt6XcjCr3GKi4+hqXoyxUK4vJDoowV+QUM85yDCJpgqmO7 Wu9Y7wd7V+B+n9SRCtd8euhR5JfupKjAV36UK613iMI95gu9FZCIFLvA3WRsZum6Ab b8ZWXryasV0I6tic5xToFWn5JsfIGnrwFeoVCwe9820D3OymH1gAziMXQ9GHjsHon0 MgkGyMUI/dJSWy8rXAAV6/Nz6XPhmuUAlZE+C9lEoYKjuAUk8OndWp7vsUnkM6lton Ne3XbYNoAaQ7ZC7H4+CEdvPhbXmCCz1juYpYjjhQqquD6+TDd8UenoLODC3t+8ZOqW f1LkLyaEw0V5A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-03; t=1716394060; bh=cQPL5uc2qmIz+bTbHSCmVXNVrh3as7mPmfeYqjnjq+Y=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=NRqA7QEnlCotnx9TjhAFuQNNA7srhObMFaamYtjq8XaYSDUgBOo8EzofLyTFeU8Kb UwpCplBStvOudGYZ+5oHXDbumsVfHwwAfZhTZ1buAeyascZ+HbR9GqRfFAd/L2ccPj bs1kHJmkBwp0BgUt1nz7KyrOoT3m8hsvG9M26tCCL9JWQQmyqeJ8l4eJHa4fHwv1lM LrpbcqJDw+8DrjmO07K90C6xUR/yXpIWJsvzLy4qIde6Yo7el7c5kh9h109xuMeOk7 QFbr+W1gFx33m6OxWAciFkudxUicL5uZPWU1NIj6DRDf71VLmvZKFNgymRvCCNp1DN HzN5PfzpiO6fg== From: Nikita Proshkin To: , Martin Mares CC: , Sergei Miroshnichenko , Nikita Proshkin Subject: [PATCH pciutils 4/6] pcilmr: Add option to configure margining dwell time Date: Wed, 22 May 2024 19:06:32 +0300 Message-ID: <20240522160634.29831-5-n.proshkin@yadro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240522160634.29831-1-n.proshkin@yadro.com> References: <20240522160634.29831-1-n.proshkin@yadro.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: T-EXCH-08.corp.yadro.com (172.17.11.58) To S-Exch-02.corp.yadro.com (10.78.5.239) Signed-off-by: Nikita Proshkin --- lmr/lmr.h | 4 ++-- lmr/margin.c | 5 +++-- lmr/margin_args.c | 6 +++++- lmr/margin_log.c | 7 ++++--- 4 files changed, 14 insertions(+), 8 deletions(-) diff --git a/lmr/lmr.h b/lmr/lmr.h index da40bfe..f070309 100644 --- a/lmr/lmr.h +++ b/lmr/lmr.h @@ -15,8 +15,6 @@ #include "pciutils.h" -#define MARGIN_STEP_MS 1000 - enum margin_hw { MARGIN_HW_DEFAULT, MARGIN_ICE_LAKE_RC }; // in ps @@ -119,6 +117,7 @@ struct margin_com_args { u64 steps_utility; // For ETA logging bool save_csv; char *dir_for_csv; + u8 dwell_time; }; struct margin_recv_args { @@ -157,6 +156,7 @@ struct margin_recv { u8 parallel_lanes; u8 error_limit; + u8 dwell_time; }; struct margin_lanes_data { diff --git a/lmr/margin.c b/lmr/margin.c index e2ea300..6ce4fe6 100644 --- a/lmr/margin.c +++ b/lmr/margin.c @@ -260,7 +260,7 @@ margin_test_lanes(struct margin_lanes_data arg) pci_write_word(arg.recv->dev->dev, ctrl_addr, step_cmd); } } - msleep(MARGIN_STEP_MS); + msleep(arg.recv->dwell_time * 1000); for (int i = 0; i < arg.lanes_n; i++) { @@ -312,7 +312,8 @@ margin_test_receiver(struct margin_dev *dev, u8 recvn, struct margin_link_args * .lane_reversal = false, .params = ¶ms, .parallel_lanes = args->parallel_lanes ? args->parallel_lanes : 1, - .error_limit = args->common->error_limit }; + .error_limit = args->common->error_limit, + .dwell_time = args->common->dwell_time }; results->recvn = recvn; results->lanes_n = lanes_n; diff --git a/lmr/margin_args.c b/lmr/margin_args.c index 484c58f..8a6345f 100644 --- a/lmr/margin_args.c +++ b/lmr/margin_args.c @@ -229,9 +229,10 @@ margin_parse_util_args(struct pci_access *pacc, int argc, char **argv, enum marg com_args->steps_utility = 0; com_args->dir_for_csv = NULL; com_args->save_csv = false; + com_args->dwell_time = 1; int c; - while ((c = getopt(argc, argv, "+e:co:")) != -1) + while ((c = getopt(argc, argv, "+e:co:d:")) != -1) { switch (c) { @@ -245,6 +246,9 @@ margin_parse_util_args(struct pci_access *pacc, int argc, char **argv, enum marg com_args->dir_for_csv = optarg; com_args->save_csv = true; break; + case 'd': + com_args->dwell_time = atoi(optarg); + break; default: die("Invalid arguments\n\n%s", usage); } diff --git a/lmr/margin_log.c b/lmr/margin_log.c index 6fa4f09..88e3594 100644 --- a/lmr/margin_log.c +++ b/lmr/margin_log.c @@ -88,7 +88,8 @@ void margin_log_receiver(struct margin_recv *recv) { margin_log("\nError Count Limit = %d\n", recv->error_limit); - margin_log("Parallel Lanes: %d\n\n", recv->parallel_lanes); + margin_log("Parallel Lanes: %d\n", recv->parallel_lanes); + margin_log("Margining dwell time: %d s\n\n", recv->dwell_time); margin_log_params(recv->params); @@ -143,8 +144,8 @@ margin_log_margining(struct margin_lanes_data arg) } margin_log("]"); - u64 lane_eta_s = (arg.steps_lane_total - arg.steps_lane_done) * MARGIN_STEP_MS / 1000; - u64 total_eta_s = *arg.steps_utility * MARGIN_STEP_MS / 1000 + lane_eta_s; + u64 lane_eta_s = (arg.steps_lane_total - arg.steps_lane_done) * arg.recv->dwell_time; + u64 total_eta_s = *arg.steps_utility * arg.recv->dwell_time + lane_eta_s; margin_log(" - ETA: %3ds Steps: %3d Total ETA: %3dm %2ds", lane_eta_s, arg.steps_lane_done, total_eta_s / 60, total_eta_s % 60); From patchwork Wed May 22 16:06:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Proshkin X-Patchwork-Id: 13670966 X-Patchwork-Delegate: bhelgaas@google.com Received: from mta-04.yadro.com (mta-04.yadro.com [89.207.88.248]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D72CD1E517 for ; Wed, 22 May 2024 16:07:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.207.88.248 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394067; cv=none; b=Dq2vIUIGNC0tMpdf/DoQ0D0faihCokx4HWgGg4FX9tz5gyn4IpnLKg+Bv0kI6zQhN5ebRLJgNCibme9pRSGuXntv9kuBHoVy2pD7w+xOPoLmKtauWyCFTXkWypsSyLbpAiaQQXlgGuF+9sXg9cOrmRx1EQwIFKWVXqOLvnXbON4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394067; c=relaxed/simple; bh=YKM5C3gQHacK5KXI471ZcUgV2JuZ+1yMk7r9o+nX24A=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=g4ByfTB3wPIA3JG4T14gy3uOCv1/lABdgbQifoXHzPnCsgqJdOBJ+uy95b9iUuBKGe/be+R3f1+xgFkc8VJ3bx8hoIsrYwa0P31T0ZoMphpK41pg1t7ujN4T2vj4myp9AOi9Ug9egZKNeiDyi4CcJH8Y7Z3Nafal9DYmO93qnW4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com; spf=pass smtp.mailfrom=yadro.com; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=kwkx4hGY; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=u4PZGmy6; arc=none smtp.client-ip=89.207.88.248 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=yadro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="kwkx4hGY"; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="u4PZGmy6" DKIM-Filter: OpenDKIM Filter v2.11.0 mta-04.yadro.com C953EC000B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-04; t=1716394061; bh=PGIQ99wEjMkqv0RbbNWPTiVylEm/fL5ZQZ+EfhP209c=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=kwkx4hGYWT1fbb009cTAqf9g1v6OPWLxhiL+QWAqbcaZssnf+CcMNxvUTubBSTS4d gg0eOYNWbTIVMaP5frLbBc/bENvOsc987+/Pk4VEgsWmiCluTeRPHMkaU0nJtSIDKs v+fvWqDXmswJlHW27wA1NHK2VFOxllueV0OsSr9sEgLPmtv1yGMtd+m4CL/MLGXYdN NKjKCg068WpuhaGRvrRxJmBEfLpoFajS9/r7gjkmA2GJFFnWipky2tXRBGlw3+59Ig symL1Mw1mVJX7TtXEMa5DIusW1EpvvtFri1gm74SBv+wNMh6SR8o6q3qum3U1rB0xY 6RRYJ1n5Y9Ghg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-03; t=1716394061; bh=PGIQ99wEjMkqv0RbbNWPTiVylEm/fL5ZQZ+EfhP209c=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=u4PZGmy6ulmJN71Zspsa3kjmLbaJi18VqqEHZltV9hNlicBaxytYRjWNZE70InnTY J5XaKHyYFlbmO0UCU4CjyLk9OWKsHnvh8Vjr/lIide7Vo5oYZDgtsGITxmIile/jhk GABNc4I3oF7SiDH/OIH3aLJY3mw7r/z96r5wX3zqiowps/4YDUKYUk8R3dEPgTARjZ wDgbMD6WFHIVaBzDHfYhvDN/4kuQoNyWbw0S5ln41h71PZYyHScCeSROBxj/E6hS7d f2cUBgjg/UIw90CTSKuMJjjA8hTT2OrE44tfch8f5RmmNEM9RF+Fk3eFs9FF3mVb56 breWohyVAZg/A== From: Nikita Proshkin To: , Martin Mares CC: , Sergei Miroshnichenko , Nikita Proshkin Subject: [PATCH pciutils 5/6] pcilmr: Apply grading quirk for Ice Lake RC ports Date: Wed, 22 May 2024 19:06:33 +0300 Message-ID: <20240522160634.29831-6-n.proshkin@yadro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240522160634.29831-1-n.proshkin@yadro.com> References: <20240522160634.29831-1-n.proshkin@yadro.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: T-EXCH-08.corp.yadro.com (172.17.11.58) To S-Exch-02.corp.yadro.com (10.78.5.239) Ice Lake RC ports don't support two side independent timing margining, however the entire margin across the eye is what is reported by one side margining. Utility already has quirks for Ice Lake RC, so expand them based on this grading information. Signed-off-by: Nikita Proshkin --- lmr/margin.c | 10 +++++++--- lmr/margin_log.c | 3 ++- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/lmr/margin.c b/lmr/margin.c index 6ce4fe6..d05bb59 100644 --- a/lmr/margin.c +++ b/lmr/margin.c @@ -143,13 +143,17 @@ margin_report_cmd(struct margin_dev *dev, u8 lane, margin_cmd cmd, margin_cmd *r } static void -margin_apply_hw_quirks(struct margin_recv *recv) +margin_apply_hw_quirks(struct margin_recv *recv, struct margin_link_args *args) { switch (recv->dev->hw) { case MARGIN_ICE_LAKE_RC: if (recv->recvn == 1) - recv->params->volt_offset = 12; + { + recv->params->volt_offset = 12; + args->recv_args[recv->recvn - 1].t.one_side_is_whole = true; + args->recv_args[recv->recvn - 1].t.valid = true; + } break; default: break; @@ -341,7 +345,7 @@ margin_test_receiver(struct margin_dev *dev, u8 recvn, struct margin_link_args * if (recv.parallel_lanes > params.max_lanes + 1) recv.parallel_lanes = params.max_lanes + 1; - margin_apply_hw_quirks(&recv); + margin_apply_hw_quirks(&recv, args); margin_log_hw_quirks(&recv); results->tim_off_reported = params.timing_offset != 0; diff --git a/lmr/margin_log.c b/lmr/margin_log.c index 88e3594..60c135d 100644 --- a/lmr/margin_log.c +++ b/lmr/margin_log.c @@ -162,7 +162,8 @@ margin_log_hw_quirks(struct margin_recv *recv) if (recv->recvn == 1) margin_log("\nRx(A) is Intel Ice Lake RC port.\n" "Applying next quirks for margining process:\n" - " - Set MaxVoltageOffset to 12 (120 mV).\n"); + " - Set MaxVoltageOffset to 12 (120 mV);\n" + " - Force the use of 'one side is the whole' grading mode.\n"); break; default: break; From patchwork Wed May 22 16:06:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Proshkin X-Patchwork-Id: 13670967 X-Patchwork-Delegate: bhelgaas@google.com Received: from mta-04.yadro.com (mta-04.yadro.com [89.207.88.248]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BD63139B for ; Wed, 22 May 2024 16:07:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=89.207.88.248 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394069; cv=none; b=ZGsJgDambD7njaNJPkEo9YdP/cglBxDrhbvdPzr/zsQdJdY6tps/5dDzc6RJb0Liv9XuOgjN8X48WRdGfgV/pzuwjL6kPSrGncL8bJM5bRA2xGjhXxtyCYdEQiMChTPj3BV+uwQuBEnKqAO9aYbpkmrvwBwzC/Afbvzwu3N76AY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716394069; c=relaxed/simple; bh=HzY6f3iUJ24wgnbI0vrAbRTn2REESZKtC2FIuUgeknA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=POf/Jn0pA14akp1WR1jrqMQaJnsmokhvZsd6k/5k9U+t7fl/ZVm0Dv59V86GYMyN3lNlDz7a2et0iPIqe8QcaabspRh1qs1MSPwmIc9VqTUvZSkk9rwUcxph3z75WRYpx3P7DooOBzOwN2hDrXBrRbNrTJn7RryZjZbYuFaXYjQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com; spf=pass smtp.mailfrom=yadro.com; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=c7FWYryq; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b=XD0TdhFQ; arc=none smtp.client-ip=89.207.88.248 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=yadro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=yadro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="c7FWYryq"; dkim=pass (2048-bit key) header.d=yadro.com header.i=@yadro.com header.b="XD0TdhFQ" DKIM-Filter: OpenDKIM Filter v2.11.0 mta-04.yadro.com 5BEAFC0002 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-04; t=1716394064; bh=jWWwDnMW62IXvIk8a/CHEuRMWiiYVXoCwUzJF54YzYg=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=c7FWYryqQjHPmQkK8ImNylzr/dcHPGcyZoYjPqBUsbl4kQ/xsyugpb+MKS86+CPQo MFINKIcXkJIZ42Lmn8b1wuLhaOxybB7SEyorlhtF9FAj1QEfrmH+Oxm8QW2r88ev7W M1FE3sl/6HNo89iQ7bRZO078cNWQv6oIYeFGLx6UELOix7ZhiiCplxBcX+HDIsUUrK wtADbTRZcN4AIdL3W5690j8KB8VHBiC6dHPb/MmKmMpoIVX6SNhYWnyuH7R8tn7X/M X3WGUU0fRZFvHIAWTwI+ocwtQW/sP601VzxSMro67USfcn95aR6PDzJG/WWdVvNVIV Rc33EmCR7IW3w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yadro.com; s=mta-03; t=1716394064; bh=jWWwDnMW62IXvIk8a/CHEuRMWiiYVXoCwUzJF54YzYg=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=XD0TdhFQQ5kCotDdpdawB4fFdW4MuVeYdcj44CElzPSmtLcySEHDd9Yt7bEMbN3x1 tx193lfMNwKSuA14RDBN70XmVTZCwFF2t9R7FPjRkNxAOWMuvhzWiWgxLmYJ9Gwf8x YtcOXvDJ1VmIN0xGR3us1yf5gR6c8DAtoRF1uDDWJNE+DvcXZue06xLTFQFS7+csyZ A0iXa5jdL2kRZcriYFbk9sDKnCG7l8MkRvyfSCLtb5J4VnsJTUnjaH6zrCZI84wKL0 s21HB1o06aZiGFv07VgU8CXjXTD10LoxNKZJ007Y6t/2x3OBbq0Al/EMT1IYVzQlH3 nUppAVgsazxGg== From: Nikita Proshkin To: , Martin Mares CC: , Sergei Miroshnichenko , Nikita Proshkin Subject: [PATCH pciutils 6/6] pcilmr: Update usage and man: new arguments format and grading Date: Wed, 22 May 2024 19:06:34 +0300 Message-ID: <20240522160634.29831-7-n.proshkin@yadro.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240522160634.29831-1-n.proshkin@yadro.com> References: <20240522160634.29831-1-n.proshkin@yadro.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: T-EXCH-08.corp.yadro.com (172.17.11.58) To S-Exch-02.corp.yadro.com (10.78.5.239) Signed-off-by: Nikita Proshkin --- lmr/margin_args.c | 39 ++++--------- pcilmr.man | 138 ++++++++++++++++++++++++++++++++++++---------- 2 files changed, 120 insertions(+), 57 deletions(-) diff --git a/lmr/margin_args.c b/lmr/margin_args.c index 8a6345f..57a1d0a 100644 --- a/lmr/margin_args.c +++ b/lmr/margin_args.c @@ -16,44 +16,25 @@ const char *usage = "! Utility requires preliminary preparation of the system. Refer to the pcilmr man page !\n\n" - "Usage:\n" - "pcilmr [--margin] [] ...\n" - "pcilmr --full []\n" + "Brief usage (see man for all options):\n" + "pcilmr [--margin] [] [] [ [] ...]\n" + "pcilmr --full []\n" "pcilmr --scan\n\n" - "Device Specifier:\n" + "You can specify Downstream or Upstream Port of the Link.\nPort Specifier:\n" ":\t[:]:.\n\n" "Modes:\n" "--margin\t\tMargin selected Links\n" "--full\t\t\tMargin all ready for testing Links in the system (one by one)\n" "--scan\t\t\tScan for Links available for margining\n\n" - "Margining options:\n\n" - "Margining Test settings:\n" - "-c\t\t\tPrint Device Lane Margining Capabilities only. Do not run margining.\n" - "-l [,...]\tSpecify lanes for margining. Default: all link lanes.\n" - "\t\t\tRemember that Device may use Lane Reversal for Lane numbering.\n" - "\t\t\tHowever, utility uses logical lane numbers in arguments and for logging.\n" - "\t\t\tUtility will automatically determine Lane Reversal and tune its calls.\n" - "-e \t\tSpecify Error Count Limit for margining. Default: 4.\n" + "Margining options (see man for all options):\n\n" + "Common (for all specified links) options:\n" + "-c\t\t\tPrint Device Lane Margining Capabilities only. Do not run margining.\n\n" + "Link specific options:\n" "-r [,...]\tSpecify Receivers to select margining targets.\n" "\t\t\tDefault: all available Receivers (including Retimers).\n" - "-p \tSpecify number of lanes to margin simultaneously.\n" - "\t\t\tDefault: 1.\n" - "\t\t\tAccording to spec it's possible for Receiver to margin up\n" - "\t\t\tto MaxLanes + 1 lanes simultaneously, but usually this works\n" - "\t\t\tbad, so this option is for experiments mostly.\n" - "-T\t\t\tTime Margining will continue until the Error Count is no more\n" - "\t\t\tthan an Error Count Limit. Use this option to find Link limit.\n" - "-V\t\t\tSame as -T option, but for Voltage.\n" "-t \t\tSpecify maximum number of steps for Time Margining.\n" - "-v \t\tSpecify maximum number of steps for Voltage Margining.\n" - "Use only one of -T/-t options at the same time (same for -V/-v).\n" - "Without these options utility will use MaxSteps from Device\n" - "capabilities as test limit.\n\n" - "Margining Log settings:\n" - "-o \t\tSave margining results in csv form into the\n" - "\t\t\tspecified directory. Utility will generate file with the\n" - "\t\t\tname in form of 'lmr__Rx#_.csv'\n" - "\t\t\tfor each successfully tested receiver.\n"; + "-v \t\tSpecify maximum number of steps for Voltage Margining.\n"; static struct pci_dev * dev_for_filter(struct pci_access *pacc, char *filter) diff --git a/pcilmr.man b/pcilmr.man index 673262f..3f4140c 100644 --- a/pcilmr.man +++ b/pcilmr.man @@ -4,10 +4,10 @@ pcilmr \- margin PCIe Links .SH SYNOPSIS .B pcilmr .RB [ "--margin" ] -.RI [ "" ] " ..." +.RI [ "" ] " " [ "" "] [" " " [ "" ] " ..." ] .br .B pcilmr --full -.RI [ "" ] +.RI [ "" ] .br .B pcilmr --scan .SH CONFIGURATION @@ -64,26 +64,81 @@ Utility allows to get an approximation of the eye margin diagram in the form of links without a hardware debugger and without the need to stop the target system. Utility can be useful to debug link issues due to receiver margins. -However, the utility results may be not particularly accurate and, as it was found out during -testing, specific devices provide rather dubious capability support and the reliability of -the information they provide is questionable. The PCIe specification provides reference values -for the eye diagram, which are also used by the +.B pcilmr +requires root privileges (to access Extended Configuration Space), but during our testing +there were no problems with the devices and they successfully returned to their normal initial +state after the end of testing. + +.SH RESULTS GRADING +The PCIe specification provides reference values for the eye diagram, which are also used by the .B pcilmr to evaluate the results, but it seems that it makes sense to contact the manufacturer of a particular device for references. -The PCIe Base Specification Revision 5.0 sets allowed range for Timing Margin from 20%\~UI to 50%\~UI and -for Voltage Margin from 50\~mV to 500\~mV. Utility uses 30%\~UI as the recommended -value for Timing - taken from NVIDIA presentation ("PCIe 4.0 Mass Electrical Margins Data -Collection"). +The utility uses values set in PCIe Base Spec Rev. 5.0 Section 8.4.2 as the default eye width and height +minimum references. Recommended values were taken from +the PCIe Architecture PHY Test Spec Rev 5.0 (Transmitter Electrical Compliance). + +Reference grading values currently used by the utility are presented in the table below: + +.TS +box tab(:); +C | Cb S | Cb S +C | Cb | Cb | Cb | Cb +Lb | C | C | C | C. +\&:16 GT/s (Gen 4):32 GT/s (Gen 5) +\&:EW:EH:EW:EH +_ +Min:T{ +18.75 ps +.br +30% UI +T}:15 mV:T{ +9.375 ps +.br +30% UI +T}:15 mV +_ +Rec:T{ +23.75 ps +.br +38% UI +T}:21 mV:T{ +10.157 ps +.br +33% UI +T}:19.75 mV +.TE .B pcilmr -requires root privileges (to access Extended Configuration Space), but during our testing -there were no problems with the devices and they successfully returned to their normal initial -state after the end of testing. +uses full eye width and height values to grade lanes. However, it is possible that +device supports only one side margining. In such cases by default utility will +calculate EW or EH as a double one side result. + +If info for specific device is available, you can configure grading criteria +and tweak utility behavior in one-side only cases for your device using +.I -g +link specific option (see below). + +.SH HARDWARE QUIRKS SUPPORT + +Thanks to testing or directly from the manufacturer's documentation, we know that +some devices require special treatment during the margining. +Utility detects such devices based on their Vendor ID - Device ID pair. +Right now the list of special devices is hardcoded in +.I margin_hw +file. For such devices utility can automatically adjust port margining parameters +or grading options. + +For example, for Ice Lake CPUs RC ports +.B pcilmr +will change device MaxVoltageOffset value and will force the use of +.RI ' "one side is the whole" "' grading mode." .SH OPTIONS .SS Device Specifier +.B "You can specify Downstream or Upstream Port of the Link." +.TP .B "" \t .RI [ "" :] : . (see @@ -102,13 +157,29 @@ option) Links in the system (one by one). Scan for Links with negotiated speed 16 GT/s or higher. Mark "Ready" those of them in which at least one of the Link sides have Margining Ready bit set meaning that these Links are ready for testing and you can run utility on them. -.SS Margining Test options -.TP +.SS Margining Common (for all specified links) options .B -c Print Device Lane Margining Capabilities only. Do not run margining. .TP +.BI -e " " +Specify Error Count Limit for margining. +.br +Default: 4. +.TP +.BI -o " " +Save margining results in csv form into the specified directory. Utility +will generate file with the name in form of +.RI "\[dq]lmr_" "" "_Rx" # _ ".csv\[dq]" +for each successfully tested receiver. +.TP +.BI -d "