From patchwork Sat Oct 12 21:28:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Martin Wilck X-Patchwork-Id: 11187113 X-Patchwork-Delegate: christophe.varoqui@free.fr Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 523AC112B for ; Sat, 12 Oct 2019 21:36:21 +0000 (UTC) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25A8920700 for ; Sat, 12 Oct 2019 21:36:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25A8920700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dm-devel-bounces@redhat.com Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6979819D381; Sat, 12 Oct 2019 21:36:20 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 45BD45D71C; Sat, 12 Oct 2019 21:36:20 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0E400180B76F; Sat, 12 Oct 2019 21:36:20 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x9CLUr8a006859 for ; Sat, 12 Oct 2019 17:30:53 -0400 Received: by smtp.corp.redhat.com (Postfix) id 8B5645DA8D; Sat, 12 Oct 2019 21:30:53 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx07.extmail.prod.ext.phx2.redhat.com [10.5.110.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EEAC05DA5B; Sat, 12 Oct 2019 21:30:50 +0000 (UTC) Received: from m9a0001g.houston.softwaregrp.com (m9a0001g.houston.softwaregrp.com [15.124.64.66]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5A0EFC075BD2; Sat, 12 Oct 2019 21:30:48 +0000 (UTC) Received: FROM m9a0001g.houston.softwaregrp.com (15.121.0.190) BY m9a0001g.houston.softwaregrp.com WITH ESMTP; Sat, 12 Oct 2019 21:30:04 +0000 Received: from M4W0334.microfocus.com (2002:f78:1192::f78:1192) by M9W0067.microfocus.com (2002:f79:be::f79:be) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Sat, 12 Oct 2019 21:29:39 +0000 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (15.124.8.11) by M4W0334.microfocus.com (15.120.17.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Sat, 12 Oct 2019 21:29:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RUoTgTCTN/pJEF8M1TQGI9UlKvcNe13TkFa4QFEkgR6v3trP3dgFLbE/93W0d2rX65Uz70qcPRzjf+BpgyVmS4ZRtgXllyLEgz9XTq8LmEkp3lE0latJtYg5o2iaBG8ePtDyUWo2jpMQ4VfWH5g/xTWKS2tvPXIGj/HgHp3FTFPGvrBnSD6hFAfs5Ats7fgIyivxE/5jox4KM/WTQIeNuu1Pi9MIYlk5csUEAlsN43QY7asmG/WwdOOwiCNWO3TwK0Jb7IxgWXwUaXRN/p+Gz7Yng24sr3xzm+2nZjDJ90YJcSeoUTHCB36ZBumojX/gzLkxtflWEnCXENtPdue56g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aMIWr+Iyd2fb8QUYhvqNzslHU/8WIWw/znCJeYbkkIc=; b=mhsXIrBOOxPTzo/q8R5a0nXdzrjQipQUdnUiSXIY/19Mv/TtWu100+Fb2Ojiz2mmav4v1nzpyuqDcqLLb9EWyTmj4iBGOb7o+Bkx/4EyFPz/vGv1LmQlPP6QNLQzd+Jf6tO4Fjd1dtW5NPk6ICAmq+5zrt3NnZd2uxMNlla7jgdPpi/nRdPCEaTzLJHo9eHBZVOa4mwJLSYAs1LIKLkQQfqKYTe9kqq/n/9e72AEK5kwmwHCt73/rvQE1HTkywAvJHE80OjeScRROAkgW3SkonV8f5PQo552yo/PTuFSMNjqPNp8/nyswER0z44OTmP5nT4orc+Jp9UXSs6wXa4XRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Received: from CH2PR18MB3349.namprd18.prod.outlook.com (52.132.245.83) by CH2PR18MB3317.namprd18.prod.outlook.com (52.132.245.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.17; Sat, 12 Oct 2019 21:29:33 +0000 Received: from CH2PR18MB3349.namprd18.prod.outlook.com ([fe80::1075:2453:9278:e985]) by CH2PR18MB3349.namprd18.prod.outlook.com ([fe80::1075:2453:9278:e985%5]) with mapi id 15.20.2347.021; Sat, 12 Oct 2019 21:29:33 +0000 From: Martin Wilck To: Christophe Varoqui , Benjamin Marzinski Thread-Topic: [PATCH 65/72] libmultipath: nvme: update to nvme-cli v1.9 Thread-Index: AQHVgUQOpD5IfFU9uUWVbFCSqwyM1w== Date: Sat, 12 Oct 2019 21:28:58 +0000 Message-ID: <20191012212703.12989-66-martin.wilck@suse.com> References: <20191012212703.12989-1-martin.wilck@suse.com> In-Reply-To: <20191012212703.12989-1-martin.wilck@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: AM0PR02CA0045.eurprd02.prod.outlook.com (2603:10a6:208:d2::22) To CH2PR18MB3349.namprd18.prod.outlook.com (2603:10b6:610:28::19) authentication-results: spf=none (sender IP is ) smtp.mailfrom=Martin.Wilck@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [2.203.223.119] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3a024407-064c-45b9-acd1-08d74f5b313f x-ms-traffictypediagnostic: CH2PR18MB3317: x-ld-processed: 856b813c-16e5-49a5-85ec-6f081e13b527,ExtFwd x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5797; x-forefront-prvs: 0188D66E61 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(136003)(366004)(39860400002)(376002)(346002)(396003)(189003)(199004)(446003)(15650500001)(6512007)(11346002)(4326008)(8676002)(476003)(6666004)(81156014)(71200400001)(71190400001)(99286004)(14454004)(8936002)(66066001)(107886003)(2616005)(486006)(81166006)(54906003)(256004)(14444005)(5024004)(86362001)(110136005)(44832011)(50226002)(25786009)(316002)(6436002)(478600001)(26005)(102836004)(386003)(6506007)(5660300002)(1076003)(52116002)(305945005)(36756003)(2906002)(6116002)(30864003)(76176011)(66446008)(6486002)(7736002)(66946007)(64756008)(66556008)(186003)(66476007)(3846002)(559001)(579004); DIR:OUT; SFP:1102; SCL:1; SRVR:CH2PR18MB3317; H:CH2PR18MB3349.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: XsJSY915p6vSHRSLRQzL1IYOeo9HxqCC2MObOLQ4KohFM/Id7Z7TQwAjqG5prWFNy3+oajZAE423foT/Qb997kQFv9OBZvQtVu8U8o5CHPcXAvHP9iuaPZtcRklEF+ljXWCcGl+Y8smw0dUs2+ugXf8WdhlxLhT8NjTrgMKbA4tnGayW+BQ27PKYrV09PJFvSQk2y5fWa4ysJ4bi7t4sKQz+wwLFUZfkyOgcXfwkI3yw7uD88SiqPmQ0RmE6w6TJN1h6u8hnoQYWVL6FqEx+FN/5gR4F1Hmo90xmlt8AEsf+REBFMPyEo6gIyDqGgoay0KNy8osEnfgm+zKKvzKTRYYl6a1L3VR5CkmSCHFVE5v82HHYfzCRYmk96JARjmBV+AvLU1+Ys4W16d1/WoPbH1KITh136LcbwWzMOALsGfA= Content-ID: <5D06F790F0EDFF4E92130F4983ED33DA@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 3a024407-064c-45b9-acd1-08d74f5b313f X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Oct 2019 21:28:58.6017 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: f9VEl2cC5A1p7upXcO2uRgtG0YWYV2MShquZ5cLMrJ0HpR8FVzXQM3Pp1F7ClFbE2HAoc2VGQw6iFizNHttouA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR18MB3317 X-OriginatorOrg: suse.com X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 238 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Sat, 12 Oct 2019 21:30:49 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Sat, 12 Oct 2019 21:30:49 +0000 (UTC) for IP:'15.124.64.66' DOMAIN:'m9a0001g.houston.softwaregrp.com' HELO:'m9a0001g.houston.softwaregrp.com' FROM:'Martin.Wilck@suse.com' RCPT:'' X-RedHat-Spam-Score: 0.001 (RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY) 15.124.64.66 m9a0001g.houston.softwaregrp.com 15.124.64.66 m9a0001g.houston.softwaregrp.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.31 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MIME-Autoconverted: from base64 to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id x9CLUr8a006859 X-loop: dm-devel@redhat.com Cc: "dm-devel@redhat.com" , Martin Wilck Subject: [dm-devel] [PATCH 65/72] libmultipath: nvme: update to nvme-cli v1.9 X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Sat, 12 Oct 2019 21:36:20 +0000 (UTC) From: Martin Wilck Update nvme code to nvme-cli v1.9 (977e7d4, Thu Aug 15 2019). Signed-off-by: Martin Wilck --- libmultipath/nvme/linux/nvme.h | 136 +++++++++++++++----- libmultipath/nvme/nvme-ioctl.c | 229 +++++++++++++++++++++------------ libmultipath/nvme/nvme-ioctl.h | 31 ++++- libmultipath/nvme/nvme.h | 121 +++++++++++++---- 4 files changed, 375 insertions(+), 142 deletions(-) diff --git a/libmultipath/nvme/linux/nvme.h b/libmultipath/nvme/linux/nvme.h index 68000eb8..a6975549 100644 --- a/libmultipath/nvme/linux/nvme.h +++ b/libmultipath/nvme/linux/nvme.h @@ -124,6 +124,9 @@ enum { NVME_REG_BPINFO = 0x0040, /* Boot Partition Information */ NVME_REG_BPRSEL = 0x0044, /* Boot Partition Read Select */ NVME_REG_BPMBL = 0x0048, /* Boot Partition Memory Buffer Location */ + NVME_REG_PMRCAP = 0x0e00, /* Persistent Memory Capabilities */ + NVME_REG_PMRCTL = 0x0e04, /* Persistent Memory Region Control */ + NVME_REG_PMRSTS = 0x0e08, /* Persistent Memory Region Status */ NVME_REG_DBS = 0x1000, /* SQ 0 Tail Doorbell */ }; @@ -221,7 +224,11 @@ struct nvme_id_ctrl { __le32 oaes; __le32 ctratt; __le16 rrls; - __u8 rsvd102[154]; + __u8 rsvd102[26]; + __le16 crdt1; + __le16 crdt2; + __le16 crdt3; + __u8 rsvd134[122]; __le16 oacs; __u8 acl; __u8 aerl; @@ -302,6 +309,8 @@ enum { NVME_CTRL_CTRATT_READ_RECV_LVLS = 1 << 3, NVME_CTRL_CTRATT_ENDURANCE_GROUPS = 1 << 4, NVME_CTRL_CTRATT_PREDICTABLE_LAT = 1 << 5, + NVME_CTRL_CTRATT_NAMESPACE_GRANULARITY = 1 << 7, + NVME_CTRL_CTRATT_UUID_LIST = 1 << 9, }; struct nvme_lbaf { @@ -332,7 +341,12 @@ struct nvme_id_ns { __le16 nabspf; __le16 noiob; __u8 nvmcap[16]; - __u8 rsvd64[28]; + __le16 npwg; + __le16 npwa; + __le16 npdg; + __le16 npda; + __le16 nows; + __u8 rsvd74[18]; __le32 anagrpid; __u8 rsvd96[3]; __u8 nsattr; @@ -355,6 +369,9 @@ enum { NVME_ID_CNS_NS_PRESENT = 0x11, NVME_ID_CNS_CTRL_NS_LIST = 0x12, NVME_ID_CNS_CTRL_LIST = 0x13, + NVME_ID_CNS_SCNDRY_CTRL_LIST = 0x15, + NVME_ID_CNS_NS_GRANULARITY = 0x16, + NVME_ID_CNS_UUID_LIST = 0x17, }; enum { @@ -425,26 +442,56 @@ struct nvme_id_nvmset { struct nvme_nvmset_attr_entry ent[NVME_MAX_NVMSET]; }; -/* Derived from 1.3a Figure 101: Get Log Page – Telemetry Host - * -Initiated Log (Log Identifier 07h) +struct nvme_id_ns_granularity_list_entry { + __le64 namespace_size_granularity; + __le64 namespace_capacity_granularity; +}; + +struct nvme_id_ns_granularity_list { + __le32 attributes; + __u8 num_descriptors; + __u8 rsvd[27]; + struct nvme_id_ns_granularity_list_entry entry[16]; +}; + +#define NVME_MAX_UUID_ENTRIES 128 +struct nvme_id_uuid_list_entry { + __u8 header; + __u8 rsvd1[15]; + __u8 uuid[16]; +}; + +struct nvme_id_uuid_list { + struct nvme_id_uuid_list_entry entry[NVME_MAX_UUID_ENTRIES]; +}; + +/** + * struct nvme_telemetry_log_page_hdr - structure for telemetry log page + * @lpi: Log page identifier + * @iee_oui: IEEE OUI Identifier + * @dalb1: Data area 1 last block + * @dalb2: Data area 2 last block + * @dalb3: Data area 3 last block + * @ctrlavail: Controller initiated data available + * @ctrldgn: Controller initiated telemetry Data Generation Number + * @rsnident: Reason Identifier + * @telemetry_dataarea: Contains telemetry data block + * + * This structure can be used for both telemetry host-initiated log page + * and controller-initiated log page. */ struct nvme_telemetry_log_page_hdr { - __u8 lpi; /* Log page identifier */ - __u8 rsvd[4]; - __u8 iee_oui[3]; - __u16 dalb1; /* Data area 1 last block */ - __u16 dalb2; /* Data area 2 last block */ - __u16 dalb3; /* Data area 3 last block */ - __u8 rsvd1[368]; /* TODO verify */ - __u8 ctrlavail; /* Controller initiated data avail?*/ - __u8 ctrldgn; /* Controller initiated telemetry Data Gen # */ - __u8 rsnident[128]; - /* We'll have to double fetch so we can get the header, - * parse dalb1->3 determine how much size we need for the - * log then alloc below. Or just do a secondary non-struct - * allocation. - */ - __u8 telemetry_dataarea[0]; + __u8 lpi; + __u8 rsvd[4]; + __u8 iee_oui[3]; + __le16 dalb1; + __le16 dalb2; + __le16 dalb3; + __u8 rsvd1[368]; + __u8 ctrlavail; + __u8 ctrldgn; + __u8 rsnident[128]; + __u8 telemetry_dataarea[0]; }; struct nvme_endurance_group_log { @@ -513,6 +560,21 @@ struct nvme_fw_slot_info_log { __u8 rsvd64[448]; }; +struct nvme_lba_status_desc { + __u64 dslba; + __u32 nlb; + __u8 rsvd_12; + __u8 status; + __u8 rsvd_15_14[2]; +}; + +struct nvme_lba_status { + __u32 nlsd; + __u8 cmpc; + __u8 rsvd_7_5[3]; + struct nvme_lba_status_desc descs[0]; +}; + /* NVMe Namespace Write Protect State */ enum { NVME_NS_NO_WRITE_PROTECT = 0, @@ -534,6 +596,7 @@ enum { NVME_CMD_EFFECTS_NIC = 1 << 3, NVME_CMD_EFFECTS_CCC = 1 << 4, NVME_CMD_EFFECTS_CSE_MASK = 3 << 16, + NVME_CMD_EFFECTS_UUID_SEL = 1 << 19, }; struct nvme_effects_log { @@ -581,9 +644,6 @@ enum { NVME_AER_SMART = 1, NVME_AER_CSS = 6, NVME_AER_VS = 7, - NVME_AER_NOTICE_NS_CHANGED = 0x0002, - NVME_AER_NOTICE_ANA = 0x0003, - NVME_AER_NOTICE_FW_ACT_STARTING = 0x0102, }; struct nvme_lba_range_type { @@ -606,12 +666,13 @@ enum { NVME_LBART_ATTRIB_HIDE = 1 << 1, }; +/* Predictable Latency Mode - Deterministic Threshold Configuration Data */ struct nvme_plm_config { - __u16 enable_event; + __le16 enable_event; __u8 rsvd2[30]; - __u64 dtwin_reads_thresh; - __u64 dtwin_writes_thresh; - __u64 dtwin_time_thresh; + __le64 dtwin_reads_thresh; + __le64 dtwin_writes_thresh; + __le64 dtwin_time_thresh; __u8 rsvd56[456]; }; @@ -665,6 +726,7 @@ enum nvme_opcode { nvme_cmd_compare = 0x05, nvme_cmd_write_zeroes = 0x08, nvme_cmd_dsm = 0x09, + nvme_cmd_verify = 0x0c, nvme_cmd_resv_register = 0x0d, nvme_cmd_resv_report = 0x0e, nvme_cmd_resv_acquire = 0x11, @@ -892,6 +954,7 @@ enum nvme_admin_opcode { nvme_admin_security_send = 0x81, nvme_admin_security_recv = 0x82, nvme_admin_sanitize_nvm = 0x84, + nvme_admin_get_lba_status = 0x86, }; enum { @@ -921,6 +984,8 @@ enum { NVME_FEAT_RRL = 0x12, NVME_FEAT_PLM_CONFIG = 0x13, NVME_FEAT_PLM_WINDOW = 0x14, + NVME_FEAT_HOST_BEHAVIOR = 0x16, + NVME_FEAT_SANITIZE = 0x17, NVME_FEAT_SW_PROGRESS = 0x80, NVME_FEAT_HOST_ID = 0x81, NVME_FEAT_RESV_MASK = 0x82, @@ -972,6 +1037,7 @@ enum { NVME_SANITIZE_LOG_COMPLETED_SUCCESS = 0x0001, NVME_SANITIZE_LOG_IN_PROGESS = 0x0002, NVME_SANITIZE_LOG_COMPLETED_FAILED = 0x0003, + NVME_SANITIZE_LOG_ND_COMPLETED_SUCCESS = 0x0004, }; enum { @@ -1131,6 +1197,9 @@ struct nvme_sanitize_log_page { __le32 est_ovrwrt_time; __le32 est_blk_erase_time; __le32 est_crypto_erase_time; + __le32 est_ovrwrt_time_with_no_deallocate; + __le32 est_blk_erase_time_with_no_deallocate; + __le32 est_crypto_erase_time_with_no_deallocate; }; /* @@ -1314,6 +1383,12 @@ static inline bool nvme_is_write(struct nvme_command *cmd) return cmd->common.opcode & 1; } +enum { + NVME_SCT_GENERIC = 0x0, + NVME_SCT_CMD_SPECIFIC = 0x1, + NVME_SCT_MEDIA = 0x2, +}; + enum { /* * Generic Command Status: @@ -1344,6 +1419,7 @@ enum { NVME_SC_SANITIZE_IN_PROGRESS = 0x1D, NVME_SC_NS_WRITE_PROTECTED = 0x20, + NVME_SC_CMD_INTERRUPTED = 0x21, NVME_SC_LBA_RANGE = 0x80, NVME_SC_CAP_EXCEEDED = 0x81, @@ -1372,9 +1448,9 @@ enum { NVME_SC_FW_NEEDS_SUBSYS_RESET = 0x110, NVME_SC_FW_NEEDS_RESET = 0x111, NVME_SC_FW_NEEDS_MAX_TIME = 0x112, - NVME_SC_FW_ACIVATE_PROHIBITED = 0x113, + NVME_SC_FW_ACTIVATE_PROHIBITED = 0x113, NVME_SC_OVERLAPPING_RANGE = 0x114, - NVME_SC_NS_INSUFFICENT_CAP = 0x115, + NVME_SC_NS_INSUFFICIENT_CAP = 0x115, NVME_SC_NS_ID_UNAVAILABLE = 0x116, NVME_SC_NS_ALREADY_ATTACHED = 0x118, NVME_SC_NS_IS_PRIVATE = 0x119, @@ -1382,6 +1458,7 @@ enum { NVME_SC_THIN_PROV_NOT_SUPP = 0x11b, NVME_SC_CTRL_LIST_INVALID = 0x11c, NVME_SC_BP_WRITE_PROHIBITED = 0x11e, + NVME_SC_PMR_SAN_PROHIBITED = 0x123, /* * I/O Command Set Specific - NVM commands: @@ -1422,6 +1499,7 @@ enum { NVME_SC_ANA_INACCESSIBLE = 0x302, NVME_SC_ANA_TRANSITION = 0x303, + NVME_SC_CRD = 0x1800, NVME_SC_DNR = 0x4000, }; diff --git a/libmultipath/nvme/nvme-ioctl.c b/libmultipath/nvme/nvme-ioctl.c index 70a16ced..69599763 100644 --- a/libmultipath/nvme/nvme-ioctl.c +++ b/libmultipath/nvme/nvme-ioctl.c @@ -1,3 +1,4 @@ +#include #include #include #include @@ -177,6 +178,22 @@ int nvme_compare(int fd, __u64 slba, __u16 nblocks, __u16 control, __u32 dsmgmt, reftag, apptag, appmask, data, metadata); } +int nvme_verify(int fd, __u32 nsid, __u64 slba, __u16 nblocks, + __u16 control, __u32 reftag, __u16 apptag, __u16 appmask) +{ + struct nvme_passthru_cmd cmd = { + .opcode = nvme_cmd_verify, + .nsid = nsid, + .cdw10 = slba & 0xffffffff, + .cdw11 = slba >> 32, + .cdw12 = nblocks | (control << 16), + .cdw14 = reftag, + .cdw15 = apptag | (appmask << 16), + }; + + return nvme_submit_io_passthru(fd, &cmd); +} + int nvme_passthru_io(int fd, __u8 opcode, __u8 flags, __u16 rsvd, __u32 nsid, __u32 cdw2, __u32 cdw3, __u32 cdw10, __u32 cdw11, __u32 cdw12, __u32 cdw13, __u32 cdw14, @@ -370,6 +387,11 @@ int nvme_identify_ctrl_list(int fd, __u32 nsid, __u16 cntid, void *data) return nvme_identify(fd, nsid, (cntid << 16) | cns, data); } +int nvme_identify_secondary_ctrl_list(int fd, __u32 nsid, __u16 cntid, void *data) +{ + return nvme_identify(fd, nsid, (cntid << 16) | NVME_ID_CNS_SCNDRY_CTRL_LIST, data); +} + int nvme_identify_ns_descs(int fd, __u32 nsid, void *data) { @@ -381,8 +403,18 @@ int nvme_identify_nvmset(int fd, __u16 nvmset_id, void *data) return nvme_identify13(fd, 0, NVME_ID_CNS_NVMSET_LIST, nvmset_id, data); } -int nvme_get_log13(int fd, __u32 nsid, __u8 log_id, __u8 lsp, __u64 lpo, - __u16 lsi, bool rae, __u32 data_len, void *data) +int nvme_identify_ns_granularity(int fd, void *data) +{ + return nvme_identify13(fd, 0, NVME_ID_CNS_NS_GRANULARITY, 0, data); +} + +int nvme_identify_uuid(int fd, void *data) +{ + return nvme_identify(fd, 0, NVME_ID_CNS_UUID_LIST, data); +} + +int nvme_get_log14(int fd, __u32 nsid, __u8 log_id, __u8 lsp, __u64 lpo, + __u16 lsi, bool rae, __u8 uuid_ix, __u32 data_len, void *data) { struct nvme_admin_cmd cmd = { .opcode = nvme_admin_get_log_page, @@ -400,6 +432,7 @@ int nvme_get_log13(int fd, __u32 nsid, __u8 log_id, __u8 lsp, __u64 lpo, cmd.cdw11 = numdu | (lsi << 16); cmd.cdw12 = lpo; cmd.cdw13 = (lpo >> 32); + cmd.cdw14 = uuid_ix; return nvme_submit_admin_passthru(fd, &cmd); @@ -498,7 +531,7 @@ int nvme_self_test_log(int fd, struct nvme_self_test_log *self_test_log) int nvme_effects_log(int fd, struct nvme_effects_log_page *effects_log) { - return nvme_get_log(fd, 0, NVME_LOG_CMD_EFFECTS, false, + return nvme_get_log(fd, NVME_NSID_ALL, NVME_LOG_CMD_EFFECTS, false, sizeof(*effects_log), effects_log); } @@ -542,77 +575,61 @@ int nvme_set_feature(int fd, __u32 nsid, __u8 fid, __u32 value, __u32 cdw12, cdw12, data_len, data, result); } -static int nvme_property(int fd, __u8 fctype, __le32 off, __le64 *value, __u8 attrib) -{ - int err; - struct nvme_admin_cmd cmd = { - .opcode = nvme_fabrics_command, - .cdw10 = attrib, - .cdw11 = off, - }; - - if (!value) { - errno = EINVAL; - return -errno; - } - - if (fctype == nvme_fabrics_type_property_get){ - cmd.nsid = nvme_fabrics_type_property_get; - } else if(fctype == nvme_fabrics_type_property_set) { - cmd.nsid = nvme_fabrics_type_property_set; - cmd.cdw12 = *value; - } else { - errno = EINVAL; - return -errno; - } - err = nvme_submit_admin_passthru(fd, &cmd); - if (!err && fctype == nvme_fabrics_type_property_get) - *value = cpu_to_le64(cmd.result); - return err; +/* + * Perform the opposite operation of the byte-swapping code at the start of the + * kernel function nvme_user_cmd(). + */ +static void nvme_to_passthru_cmd(struct nvme_passthru_cmd *pcmd, + const struct nvme_command *ncmd) +{ + assert(sizeof(*ncmd) < sizeof(*pcmd)); + memset(pcmd, 0, sizeof(*pcmd)); + pcmd->opcode = ncmd->common.opcode; + pcmd->flags = ncmd->common.flags; + pcmd->rsvd1 = ncmd->common.command_id; + pcmd->nsid = le32_to_cpu(ncmd->common.nsid); + pcmd->cdw2 = le32_to_cpu(ncmd->common.cdw2[0]); + pcmd->cdw3 = le32_to_cpu(ncmd->common.cdw2[1]); + /* Skip metadata and addr */ + pcmd->cdw10 = le32_to_cpu(ncmd->common.cdw10[0]); + pcmd->cdw11 = le32_to_cpu(ncmd->common.cdw10[1]); + pcmd->cdw12 = le32_to_cpu(ncmd->common.cdw10[2]); + pcmd->cdw13 = le32_to_cpu(ncmd->common.cdw10[3]); + pcmd->cdw14 = le32_to_cpu(ncmd->common.cdw10[4]); + pcmd->cdw15 = le32_to_cpu(ncmd->common.cdw10[5]); } -static int get_property_helper(int fd, int offset, void *value, int *advance) +int nvme_get_property(int fd, int offset, uint64_t *value) { - __le64 value64; - int err = -EINVAL; - - switch (offset) { - case NVME_REG_CAP: - case NVME_REG_ASQ: - case NVME_REG_ACQ: - *advance = 8; - break; - default: - *advance = 4; - } - - if (!value) - return err; - - err = nvme_property(fd, nvme_fabrics_type_property_get, - cpu_to_le32(offset), &value64, (*advance == 8)); + struct nvme_passthru_cmd pcmd; + struct nvmf_property_get_command pg = { + .opcode = nvme_fabrics_command, + .fctype = nvme_fabrics_type_property_get, + .offset = cpu_to_le32(offset), + .attrib = is_64bit_reg(offset), + }; + struct nvme_command gcmd; + int err; + gcmd.prop_get = pg; + nvme_to_passthru_cmd(&pcmd, &gcmd); + err = nvme_submit_admin_passthru(fd, &pcmd); if (!err) { - if (*advance == 8) - *((uint64_t *)value) = le64_to_cpu(value64); - else - *((uint32_t *)value) = le32_to_cpu(value64); + /* + * nvme_submit_admin_passthru() stores the lower 32 bits + * of the property value in pcmd.result using CPU endianness. + */ + *value = pcmd.result; } - return err; } -int nvme_get_property(int fd, int offset, uint64_t *value) -{ - int advance; - return get_property_helper(fd, offset, value, &advance); -} - int nvme_get_properties(int fd, void **pbar) { - int offset, advance; - int err, ret = -EINVAL; + int offset; + uint64_t value; + int err; int size = getpagesize(); *pbar = malloc(size); @@ -622,33 +639,42 @@ int nvme_get_properties(int fd, void **pbar) } memset(*pbar, 0xff, size); - for (offset = NVME_REG_CAP; offset <= NVME_REG_CMBSZ; offset += advance) { - err = get_property_helper(fd, offset, *pbar + offset, &advance); - if (!err) - ret = 0; + for (offset = NVME_REG_CAP; offset <= NVME_REG_CMBSZ;) { + err = nvme_get_property(fd, offset, &value); + if (err > 0 && (err & 0xff) == NVME_SC_INVALID_FIELD) { + err = 0; + value = -1; + } else if (err) { + free(*pbar); + break; + } + if (is_64bit_reg(offset)) { + *(uint64_t *)(*pbar + offset) = value; + offset += 8; + } else { + *(uint32_t *)(*pbar + offset) = value; + offset += 4; + } } - return ret; + return err; } -int nvme_set_property(int fd, int offset, int value) +int nvme_set_property(int fd, int offset, uint64_t value) { - __le64 val = cpu_to_le64(value); - __le32 off = cpu_to_le32(offset); - bool is64bit; - - switch (off) { - case NVME_REG_CAP: - case NVME_REG_ASQ: - case NVME_REG_ACQ: - is64bit = true; - break; - default: - is64bit = false; - } + struct nvmf_property_set_command ps = { + .opcode = nvme_fabrics_command, + .fctype = nvme_fabrics_type_property_set, + .offset = cpu_to_le32(offset), + .value = cpu_to_le64(value), + .attrib = is_64bit_reg(offset), + }; + struct nvme_command scmd; + struct nvme_passthru_cmd pcmd; - return nvme_property(fd, nvme_fabrics_type_property_set, - off, &val, is64bit ? 1: 0); + scmd.prop_set = ps; + nvme_to_passthru_cmd(&pcmd, &scmd); + return nvme_submit_admin_passthru(fd, &pcmd); } int nvme_get_feature(int fd, __u32 nsid, __u8 fid, __u8 sel, __u32 cdw11, @@ -675,7 +701,7 @@ int nvme_format(int fd, __u32 nsid, __u8 lbaf, __u8 ses, __u8 pi, } int nvme_ns_create(int fd, __u64 nsze, __u64 ncap, __u8 flbas, - __u8 dps, __u8 nmic, __u32 *result) + __u8 dps, __u8 nmic, __u32 timeout, __u32 *result) { struct nvme_id_ns ns = { .nsze = cpu_to_le64(nsze), @@ -689,6 +715,7 @@ int nvme_ns_create(int fd, __u64 nsze, __u64 ncap, __u8 flbas, .addr = (__u64)(uintptr_t) ((void *)&ns), .cdw10 = 0, .data_len = 0x1000, + .timeout_ms = timeout, }; int err; @@ -698,12 +725,13 @@ int nvme_ns_create(int fd, __u64 nsze, __u64 ncap, __u8 flbas, return err; } -int nvme_ns_delete(int fd, __u32 nsid) +int nvme_ns_delete(int fd, __u32 nsid, __u32 timeout) { struct nvme_admin_cmd cmd = { .opcode = nvme_admin_ns_mgmt, .nsid = nsid, .cdw10 = 1, + .timeout_ms = timeout, }; return nvme_submit_admin_passthru(fd, &cmd); @@ -803,6 +831,21 @@ int nvme_sec_recv(int fd, __u32 nsid, __u8 nssf, __u16 spsp, return err; } +int nvme_get_lba_status(int fd, __u64 slba, __u32 mndw, __u8 atype, __u16 rl, + void *data) +{ + struct nvme_admin_cmd cmd = { + .opcode = nvme_admin_get_lba_status, + .addr = (__u64)(uintptr_t) data, + .cdw10 = slba & 0xffffffff, + .cdw11 = slba >> 32, + .cdw12 = mndw, + .cdw13 = (atype << 24) | rl, + }; + + return nvme_submit_admin_passthru(fd, &cmd); +} + int nvme_dir_send(int fd, __u32 nsid, __u16 dspec, __u8 dtype, __u8 doper, __u32 data_len, __u32 dw12, void *data, __u32 *result) { @@ -867,3 +910,19 @@ int nvme_self_test_start(int fd, __u32 nsid, __u32 cdw10) return nvme_submit_admin_passthru(fd, &cmd); } + +int nvme_virtual_mgmt(int fd, __u32 cdw10, __u32 cdw11, __u32 *result) +{ + struct nvme_admin_cmd cmd = { + .opcode = nvme_admin_virtual_mgmt, + .cdw10 = cdw10, + .cdw11 = cdw11, + }; + int err; + + err = nvme_submit_admin_passthru(fd, &cmd); + if (!err && result) + *result = cmd.result; + + return err; +} diff --git a/libmultipath/nvme/nvme-ioctl.h b/libmultipath/nvme/nvme-ioctl.h index 3fb740c3..565f7648 100644 --- a/libmultipath/nvme/nvme-ioctl.h +++ b/libmultipath/nvme/nvme-ioctl.h @@ -6,6 +6,8 @@ #include "linux/nvme_ioctl.h" #include "nvme.h" +#define NVME_IOCTL_TIMEOUT 120000 /* in milliseconds */ + int nvme_get_nsid(int fd); /* Generic passthrough */ @@ -36,6 +38,9 @@ int nvme_compare(int fd, __u64 slba, __u16 nblocks, __u16 control, __u32 dsmgmt, __u32 reftag, __u16 apptag, __u16 appmask, void *data, void *metadata); +int nvme_verify(int fd, __u32 nsid, __u64 slba, __u16 nblocks, + __u16 control, __u32 reftag, __u16 apptag, __u16 appmask); + /* NVME_IO_CMD */ int nvme_passthru_io(int fd, __u8 opcode, __u8 flags, __u16 rsvd, __u32 nsid, __u32 cdw2, __u32 cdw3, @@ -73,11 +78,22 @@ int nvme_identify_ns_list(int fd, __u32 nsid, bool all, void *data); int nvme_identify_ctrl_list(int fd, __u32 nsid, __u16 cntid, void *data); int nvme_identify_ns_descs(int fd, __u32 nsid, void *data); int nvme_identify_nvmset(int fd, __u16 nvmset_id, void *data); -int nvme_get_log13(int fd, __u32 nsid, __u8 log_id, __u8 lsp, __u64 lpo, - __u16 group_id, bool rae, __u32 data_len, void *data); +int nvme_identify_uuid(int fd, void *data); +int nvme_identify_secondary_ctrl_list(int fd, __u32 nsid, __u16 cntid, void *data); +int nvme_identify_ns_granularity(int fd, void *data); int nvme_get_log(int fd, __u32 nsid, __u8 log_id, bool rae, __u32 data_len, void *data); - +int nvme_get_log14(int fd, __u32 nsid, __u8 log_id, __u8 lsp, __u64 lpo, + __u16 group_id, bool rae, __u8 uuid_ix, + __u32 data_len, void *data); + +static inline int nvme_get_log13(int fd, __u32 nsid, __u8 log_id, __u8 lsp, + __u64 lpo, __u16 lsi, bool rae, __u32 data_len, + void *data) +{ + return nvme_get_log14(fd, nsid, log_id, lsp, lpo, lsi, rae, 0, + data_len, data); +} int nvme_get_telemetry_log(int fd, void *lp, int generate_report, int ctrl_gen, size_t log_page_size, __u64 offset); @@ -105,8 +121,8 @@ int nvme_format(int fd, __u32 nsid, __u8 lbaf, __u8 ses, __u8 pi, __u8 pil, __u8 ms, __u32 timeout); int nvme_ns_create(int fd, __u64 nsze, __u64 ncap, __u8 flbas, - __u8 dps, __u8 nmic, __u32 *result); -int nvme_ns_delete(int fd, __u32 nsid); + __u8 dps, __u8 nmic, __u32 timeout, __u32 *result); +int nvme_ns_delete(int fd, __u32 nsid, __u32 timeout); int nvme_ns_attachment(int fd, __u32 nsid, __u16 num_ctrls, __u16 *ctrlist, bool attach); @@ -125,15 +141,18 @@ int nvme_subsystem_reset(int fd); int nvme_reset_controller(int fd); int nvme_ns_rescan(int fd); +int nvme_get_lba_status(int fd, __u64 slba, __u32 mndw, __u8 atype, __u16 rl, + void *data); int nvme_dir_send(int fd, __u32 nsid, __u16 dspec, __u8 dtype, __u8 doper, __u32 data_len, __u32 dw12, void *data, __u32 *result); int nvme_dir_recv(int fd, __u32 nsid, __u16 dspec, __u8 dtype, __u8 doper, __u32 data_len, __u32 dw12, void *data, __u32 *result); int nvme_get_properties(int fd, void **pbar); -int nvme_set_property(int fd, int offset, int value); +int nvme_set_property(int fd, int offset, uint64_t value); int nvme_get_property(int fd, int offset, uint64_t *value); int nvme_sanitize(int fd, __u8 sanact, __u8 ause, __u8 owpass, __u8 oipbp, __u8 no_dealloc, __u32 ovrpat); int nvme_self_test_start(int fd, __u32 nsid, __u32 cdw10); int nvme_self_test_log(int fd, struct nvme_self_test_log *self_test_log); +int nvme_virtual_mgmt(int fd, __u32 cdw10, __u32 cdw11, __u32 *result); #endif /* _NVME_LIB_H */ diff --git a/libmultipath/nvme/nvme.h b/libmultipath/nvme/nvme.h index 685d1799..7e0278b5 100644 --- a/libmultipath/nvme/nvme.h +++ b/libmultipath/nvme/nvme.h @@ -40,16 +40,16 @@ struct nvme_effects_log_page { }; struct nvme_error_log_page { - __u64 error_count; - __u16 sqid; - __u16 cmdid; - __u16 status_field; - __u16 parm_error_location; - __u64 lba; - __u32 nsid; + __le64 error_count; + __le16 sqid; + __le16 cmdid; + __le16 status_field; + __le16 parm_error_location; + __le64 lba; + __le32 nsid; __u8 vs; __u8 resv[3]; - __u64 cs; + __le64 cs; __u8 resv2[24]; }; @@ -87,13 +87,30 @@ struct nvme_controller_list { __le16 identifier[]; }; +struct nvme_secondary_controller_entry { + __le16 scid; /* Secondary Controller Identifier */ + __le16 pcid; /* Primary Controller Identifier */ + __u8 scs; /* Secondary Controller State */ + __u8 rsvd5[3]; + __le16 vfn; /* Virtual Function Number */ + __le16 nvq; /* Number of VQ Flexible Resources Assigned */ + __le16 nvi; /* Number of VI Flexible Resources Assigned */ + __u8 rsvd14[18]; +}; + +struct nvme_secondary_controllers_list { + __u8 num; + __u8 rsvd[31]; + struct nvme_secondary_controller_entry sc_entry[127]; +}; + struct nvme_bar_cap { __u16 mqes; __u8 ams_cqr; __u8 to; __u16 bps_css_nssrs_dstrd; __u8 mpsmax_mpsmin; - __u8 reserved; + __u8 rsvd_pmrs; }; #ifdef __CHECKER__ @@ -102,19 +119,31 @@ struct nvme_bar_cap { #define __force #endif -#define cpu_to_le16(x) \ - ((__force __le16)htole16(x)) -#define cpu_to_le32(x) \ - ((__force __le32)htole32(x)) -#define cpu_to_le64(x) \ - ((__force __le64)htole64(x)) - -#define le16_to_cpu(x) \ - le16toh((__force __u16)(x)) -#define le32_to_cpu(x) \ - le32toh((__force __u32)(x)) -#define le64_to_cpu(x) \ - le64toh((__force __u64)(x)) +static inline __le16 cpu_to_le16(uint16_t x) +{ + return (__force __le16)htole16(x); +} +static inline __le32 cpu_to_le32(uint32_t x) +{ + return (__force __le32)htole32(x); +} +static inline __le64 cpu_to_le64(uint64_t x) +{ + return (__force __le64)htole64(x); +} + +static inline uint16_t le16_to_cpu(__le16 x) +{ + return le16toh((__force __u16)x); +} +static inline uint32_t le32_to_cpu(__le32 x) +{ + return le32toh((__force __u32)x); +} +static inline uint64_t le64_to_cpu(__le64 x) +{ + return le64toh((__force __u64)x); +} #define MAX_LIST_ITEMS 256 struct list_item { @@ -131,6 +160,10 @@ struct ctrl_list_item { char *transport; char *state; char *ana_state; + char *subsysnqn; + char *traddr; + char *trsvcid; + char *host_traddr; }; struct subsys_list_item { @@ -146,6 +179,26 @@ enum { BINARY, }; +struct connect_args { + char *subsysnqn; + char *transport; + char *traddr; + char *trsvcid; + char *host_traddr; +}; + +#define SYS_NVME "/sys/class/nvme" + +bool ctrl_matches_connectargs(char *name, struct connect_args *args); +char *find_ctrl_with_connectargs(struct connect_args *args); +char *__parse_connect_arg(char *conargs, const char delim, const char *fieldnm); + +extern const char *conarg_nqn; +extern const char *conarg_transport; +extern const char *conarg_traddr; +extern const char *conarg_trsvcid; +extern const char *conarg_host_traddr; + void register_extension(struct plugin *plugin); #include "argconfig.h" @@ -160,4 +213,28 @@ int validate_output_format(char *format); struct subsys_list_item *get_subsys_list(int *subcnt, char *subsysnqn, __u32 nsid); void free_subsys_list(struct subsys_list_item *slist, int n); char *nvme_char_from_block(char *block); + +/* + * is_64bit_reg - It checks whether given offset of the controller register is + * 64bit or not. + * @offset: offset of controller register field in bytes + * + * It gives true if given offset is 64bit register, otherwise it returns false. + * + * Notes: This function does not care about transport so that the offset is + * not going to be checked inside of this function for the unsupported fields + * in a specific transport. For example, BPMBL(Boot Partition Memory Buffer + * Location) register is not supported by fabrics, but it can be chcked here. + */ +static inline bool is_64bit_reg(__u32 offset) +{ + if (offset == NVME_REG_CAP || + offset == NVME_REG_ASQ || + offset == NVME_REG_ACQ || + offset == NVME_REG_BPMBL) + return true; + + return false; +} + #endif /* _NVME_H */