From patchwork Thu Feb 9 23:32:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13135201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D42DBC6379F for ; Thu, 9 Feb 2023 23:34:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231153AbjBIXe7 (ORCPT ); Thu, 9 Feb 2023 18:34:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231156AbjBIXez (ORCPT ); Thu, 9 Feb 2023 18:34:55 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B17D6E891; Thu, 9 Feb 2023 15:34:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675985670; x=1707521670; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Zuwc1m+vbftupav4vZpK2NNYPN3Qaqlb6C2lE3DNfUY=; b=fChwPiiskdK1q1A3XseUh9/+wJX6ZgMCU5r3Liypl30sRtg1bzD+pMMI 01j5g0zsm11II9epV3Sz9HY66Sq+BB5XLkKXRpc2qeA6/Xc59x3rtgMW5 Y8ROGQ++nTA2/nX4gIi/KS236My506+NzklHaSSZ9UNMTDMhIiyKaPNTS E6IUkOXZ0FJtrkjKbRw4ZKH71wIzST+odAiJe2p+RDxgsWarrIml6CWCZ snELYRHk5zeP/B5WkZ0ALSWuD3OqilFx+yf2zkGmW1UfbYPdPNSziPJmW lpZwf8MeEd7CFA8WkPpdDm2u99eFbXjl6TiWVtLpDNC8fgzmcCVS3+L48 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416508698" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="416508698" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:04 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="756592028" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="756592028" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.13.232]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:03 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Dave Jiang , Ben Widawsky , Steven Rostedt Cc: Alison Schofield , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Jonathan Cameron Subject: [PATCH v6 1/6] cxl/mbox: Add GET_POISON_LIST mailbox command Date: Thu, 9 Feb 2023 15:32:54 -0800 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield CXL devices maintain a list of locations that are poisoned or result in poison if the addresses are accessed by the host. Per the spec (CXL 3.0 8.2.9.8.4.1), the device returns this Poison list as a set of Media Error Records that include the source of the error, the starting device physical address and length. The length is the number of adjacent DPAs in the record and is in units of 64 bytes. Retrieve the poison list. Reviewed-by: Jonathan Cameron Signed-off-by: Alison Schofield --- drivers/cxl/core/mbox.c | 72 +++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 67 ++++++++++++++++++++++++++++++++++++++ drivers/cxl/pci.c | 4 +++ 3 files changed, 143 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 03909b6cef55..a57de8896846 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -5,6 +5,8 @@ #include #include #include +#include +#include #include #include @@ -989,6 +991,7 @@ int cxl_dev_state_identify(struct cxl_dev_state *cxlds) /* See CXL 2.0 Table 175 Identify Memory Device Output Payload */ struct cxl_mbox_identify id; struct cxl_mbox_cmd mbox_cmd; + u32 val; int rc; mbox_cmd = (struct cxl_mbox_cmd) { @@ -1012,6 +1015,11 @@ int cxl_dev_state_identify(struct cxl_dev_state *cxlds) cxlds->lsa_size = le32_to_cpu(id.lsa_size); memcpy(cxlds->firmware_version, id.fw_revision, sizeof(id.fw_revision)); + if (test_bit(CXL_MEM_COMMAND_ID_GET_POISON, cxlds->enabled_cmds)) { + val = get_unaligned_le24(id.poison_list_max_mer); + cxlds->poison.max_errors = min_t(u32, val, CXL_POISON_LIST_MAX); + } + return 0; } EXPORT_SYMBOL_NS_GPL(cxl_dev_state_identify, CXL); @@ -1102,6 +1110,70 @@ int cxl_set_timestamp(struct cxl_dev_state *cxlds) } EXPORT_SYMBOL_NS_GPL(cxl_set_timestamp, CXL); +int cxl_mem_get_poison(struct cxl_memdev *cxlmd, u64 offset, u64 len, + struct cxl_region *cxlr) +{ + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_mbox_poison_payload_out *po; + struct cxl_mbox_poison_payload_in pi; + struct cxl_mbox_cmd mbox_cmd; + int nr_records = 0; + int rc; + + rc = mutex_lock_interruptible(&cxlds->poison.lock); + if (rc) + return rc; + + po = cxlds->poison.payload_out; + pi.offset = cpu_to_le64(offset); + pi.length = cpu_to_le64(len / CXL_POISON_LEN_MULT); + + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_POISON, + .size_in = sizeof(pi), + .payload_in = &pi, + .size_out = cxlds->payload_size, + .payload_out = po, + .min_out = struct_size(po, record, 0), + }; + + do { + rc = cxl_internal_send_cmd(cxlds, &mbox_cmd); + if (rc) + break; + + /* TODO TRACE the media error records */ + + /* Protect against an uncleared _FLAG_MORE */ + nr_records = nr_records + le16_to_cpu(po->count); + if (nr_records >= cxlds->poison.max_errors) { + dev_dbg(&cxlmd->dev, "Max Error Records reached: %d\n", + nr_records); + break; + } + } while (po->flags & CXL_POISON_FLAG_MORE); + + mutex_unlock(&cxlds->poison.lock); + return rc; +} +EXPORT_SYMBOL_NS_GPL(cxl_mem_get_poison, CXL); + +int cxl_poison_state_init(struct cxl_dev_state *cxlds) +{ + if (!test_bit(CXL_MEM_COMMAND_ID_GET_POISON, cxlds->enabled_cmds)) + return 0; + + cxlds->poison.payload_out = devm_kzalloc(cxlds->dev, + cxlds->payload_size, + GFP_KERNEL); + if (!cxlds->poison.payload_out) + return -ENOMEM; + + mutex_init(&cxlds->poison.lock); + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_poison_state_init, CXL); + struct cxl_dev_state *cxl_dev_state_create(struct device *dev) { struct cxl_dev_state *cxlds; diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 802b5b396daf..17ea575f8a06 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -222,6 +222,24 @@ struct cxl_event_state { struct mutex log_lock; }; +/** + * struct cxl_poison_state - Poison list retrieval + * + * @max_errors: Maximum media error records held in device cache + * @payload_out: The poison list payload returned by device + * @lock: Protect reads of the poison list + * + * Reads of the poison list are synchronized to ensure that a reader + * does not get an incomplete list because their request overlapped + * (was interrupted or preceded by) another read request of the same + * DPA range. CXL Spec 3.0 Section 8.2.9.8.4.1 + */ +struct cxl_poison_state { + u32 max_errors; + struct cxl_mbox_poison_payload_out *payload_out; + struct mutex lock; /* Protect reads of poison list */ +}; + /** * struct cxl_dev_state - The driver device state * @@ -257,6 +275,7 @@ struct cxl_event_state { * @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array + * @poison: poison list retrieval info * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for @@ -296,6 +315,7 @@ struct cxl_dev_state { struct xarray doe_mbs; struct cxl_event_state event; + struct cxl_poison_state poison; int (*mbox_send)(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd); }; @@ -544,6 +564,50 @@ struct cxl_mbox_set_timestamp_in { } __packed; +/* Get Poison List CXL 3.0 Spec 8.2.9.8.4.1 */ +struct cxl_mbox_poison_payload_in { + __le64 offset; + __le64 length; +} __packed; + +struct cxl_mbox_poison_payload_out { + u8 flags; + u8 rsvd1; + __le64 overflow_t; + __le16 count; + u8 rsvd2[20]; + struct cxl_poison_record { + __le64 address; + __le32 length; + __le32 rsvd; + } __packed record[]; +} __packed; + +/* + * Get Poison List address field encodes the starting + * address of poison, and the source of the poison. + */ +#define CXL_POISON_START_MASK GENMASK_ULL(63, 6) +#define CXL_POISON_SOURCE_MASK GENMASK(2, 0) + +/* Get Poison List record length is in units of 64 bytes */ +#define CXL_POISON_LEN_MULT 64 + +/* Kernel defined maximum for a list of poison errors */ +#define CXL_POISON_LIST_MAX 1024 + +/* Get Poison List: Payload out flags */ +#define CXL_POISON_FLAG_MORE BIT(0) +#define CXL_POISON_FLAG_OVERFLOW BIT(1) +#define CXL_POISON_FLAG_SCANNING BIT(2) + +/* Get Poison List: Poison Source */ +#define CXL_POISON_SOURCE_UNKNOWN 0 +#define CXL_POISON_SOURCE_EXTERNAL 1 +#define CXL_POISON_SOURCE_INTERNAL 2 +#define CXL_POISON_SOURCE_INJECTED 3 +#define CXL_POISON_SOURCE_VENDOR 7 + /** * struct cxl_mem_command - Driver representation of a memory device command * @info: Command information as it exists for the UAPI @@ -615,6 +679,9 @@ void set_exclusive_cxl_commands(struct cxl_dev_state *cxlds, unsigned long *cmds void clear_exclusive_cxl_commands(struct cxl_dev_state *cxlds, unsigned long *cmds); void cxl_mem_get_event_records(struct cxl_dev_state *cxlds, u32 status); int cxl_set_timestamp(struct cxl_dev_state *cxlds); +int cxl_poison_state_init(struct cxl_dev_state *cxlds); +int cxl_mem_get_poison(struct cxl_memdev *cxlmd, u64 offset, u64 len, + struct cxl_region *cxlr); #ifdef CONFIG_CXL_SUSPEND void cxl_mem_active_inc(void); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 4cf9a2191602..7ca06016dac9 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -713,6 +713,10 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; + rc = cxl_poison_state_init(cxlds); + if (rc) + return rc; + rc = cxl_dev_state_identify(cxlds); if (rc) return rc; From patchwork Thu Feb 9 23:32:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13135202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A6F4C05027 for ; Thu, 9 Feb 2023 23:35:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231145AbjBIXfc (ORCPT ); Thu, 9 Feb 2023 18:35:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231208AbjBIXfW (ORCPT ); Thu, 9 Feb 2023 18:35:22 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59ED36D8DB; Thu, 9 Feb 2023 15:34:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675985695; x=1707521695; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T968P/FAEe6VKnvmjVp5xmKSUqYceyzzdlU0Og/7JsY=; b=n89/G/28vyQnspx5kkDccJVu6kFBstTCwHlxvd8YVezfpae7snxcNKMT bAodr0HJt7zVaikrFy7VE4w8tXGBsPz9XLrpVmfoSBWadgbbWXgehD51d q3RoKzCDFaySIYa1ZuqKGHf0azWg7xM2Kyja/mPYg3zzT7nO6RMl6o2Ie hITY0i9CykjHRnz42aVTLrYhemWlTxvXtnuCS9kP2STFzygZ6DTyDYkgZ DBzQGLoO0FbImwMsfdgS9lLfvdGIAFbwO8gxEwEduaFc9iDm4cUA9qguQ D9UtIgKNNrQ99dQ5n465WUjbKggibbYyKNUdlBiR0v5Nf0CGwsRVSOCuH w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416508716" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="416508716" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:06 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="756592035" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="756592035" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.13.232]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:05 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Dave Jiang , Ben Widawsky , Steven Rostedt Cc: Alison Schofield , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Jonathan Cameron Subject: [PATCH v6 2/6] cxl/trace: Add TRACE support for CXL media-error records Date: Thu, 9 Feb 2023 15:32:55 -0800 Message-Id: <4cf3efce0cfe76930a3b3e427a121acd9f8964e4.1675983077.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield CXL devices may support the retrieval of a device poison list. Add a new trace event that the CXL subsystem may use to log the media-error records returned in the poison list. Log each media-error record as a trace event of type 'cxl_poison'. Reviewed-by: Jonathan Cameron Signed-off-by: Alison Schofield --- drivers/cxl/core/mbox.c | 4 +- drivers/cxl/core/trace.h | 85 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 88 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index a57de8896846..131ee336b69e 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1142,7 +1142,9 @@ int cxl_mem_get_poison(struct cxl_memdev *cxlmd, u64 offset, u64 len, if (rc) break; - /* TODO TRACE the media error records */ + for (int i = 0; i < le16_to_cpu(po->count); i++) + trace_cxl_poison(cxlmd, cxlr, &po->record[i], + po->flags, po->overflow_t); /* Protect against an uncleared _FLAG_MORE */ nr_records = nr_records + le16_to_cpu(po->count); diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index c72ef9321cfe..c493955719ce 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -7,6 +7,7 @@ #define _CXL_EVENTS_H #include +#include #include #include @@ -581,6 +582,90 @@ TRACE_EVENT(cxl_memory_module, ) ); +#define __show_poison_source(source) \ + __print_symbolic(source, \ + { CXL_POISON_SOURCE_UNKNOWN, "Unknown" }, \ + { CXL_POISON_SOURCE_EXTERNAL, "External" }, \ + { CXL_POISON_SOURCE_INTERNAL, "Internal" }, \ + { CXL_POISON_SOURCE_INJECTED, "Injected" }, \ + { CXL_POISON_SOURCE_VENDOR, "Vendor" }) + +#define show_poison_source(source) \ + (((source > CXL_POISON_SOURCE_INJECTED) && \ + (source != CXL_POISON_SOURCE_VENDOR)) ? "Reserved" \ + : __show_poison_source(source)) + +#define show_poison_flags(flags) \ + __print_flags(flags, "|", \ + { CXL_POISON_FLAG_MORE, "More" }, \ + { CXL_POISON_FLAG_OVERFLOW, "Overflow" }, \ + { CXL_POISON_FLAG_SCANNING, "Scanning" }) + +#define __cxl_poison_addr(record) \ + (le64_to_cpu(record->address)) +#define cxl_poison_record_dpa(record) \ + (__cxl_poison_addr(record) & CXL_POISON_START_MASK) +#define cxl_poison_record_source(record) \ + (__cxl_poison_addr(record) & CXL_POISON_SOURCE_MASK) +#define cxl_poison_record_length(record) \ + (le32_to_cpu(record->length) * CXL_POISON_LEN_MULT) +#define cxl_poison_overflow(flags, time) \ + (flags & CXL_POISON_FLAG_OVERFLOW ? le64_to_cpu(time) : 0) +#define cxl_poison_host_name(cxlmd) \ + (dev_name(cxlmd->dev.parent)) + +TRACE_EVENT(cxl_poison, + + TP_PROTO(struct cxl_memdev *cxlmd, struct cxl_region *region, + const struct cxl_poison_record *record, + u8 flags, __le64 overflow_t), + + TP_ARGS(cxlmd, region, record, flags, overflow_t), + + TP_STRUCT__entry( + __string(memdev, dev_name(&cxlmd->dev)) + __string(host, cxl_poison_host_name(cxlmd)) + __field(u64, serial) + __string(region, region) + __field(u64, overflow_t) + __field(u64, dpa) + __field(u32, length) + __array(char, uuid, 16) + __field(u8, source) + __field(u8, flags) + ), + + TP_fast_assign( + __assign_str(memdev, dev_name(&cxlmd->dev)); + __assign_str(host, cxl_poison_host_name(cxlmd)); + __entry->serial = cxlmd->cxlds->serial; + __entry->overflow_t = cxl_poison_overflow(flags, overflow_t); + __entry->dpa = cxl_poison_record_dpa(record); + __entry->length = cxl_poison_record_length(record); + __entry->source = cxl_poison_record_source(record); + __entry->flags = flags; + if (region) { + __assign_str(region, dev_name(®ion->dev)); + memcpy(__entry->uuid, ®ion->params.uuid, 16); + } else { + __assign_str(region, ""); + memset(__entry->uuid, 0, 16); + } + ), + + TP_printk("memdev=%s serial=%llu host=%s region=%s region_uuid=%pU dpa=0x%llx length=0x%x source=%s flags=%s overflow_time=%llu", + __get_str(memdev), + __entry->serial, + __get_str(host), + __get_str(region), + __entry->uuid, + __entry->dpa, + __entry->length, + show_poison_source(__entry->source), + show_poison_flags(__entry->flags), + __entry->overflow_t) +); + #endif /* _CXL_EVENTS_H */ #define TRACE_INCLUDE_FILE trace From patchwork Thu Feb 9 23:32:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13135200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38EA1C61DA4 for ; Thu, 9 Feb 2023 23:34:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231168AbjBIXe6 (ORCPT ); Thu, 9 Feb 2023 18:34:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231148AbjBIXez (ORCPT ); Thu, 9 Feb 2023 18:34:55 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD81567794; Thu, 9 Feb 2023 15:34:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675985670; x=1707521670; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ib0/GcCFRrSl6cREklL0bMUgy5eAnlntcEuONs3IVBQ=; b=PlF2puszAQ3OZMhxAjNeYtzVXE3E2SHmFK6o2+4/e7iaTgfRP4nlIJpu gR7px1U3eKcFeUI3cvx1bmLyJ8N7oByLQ2qsuKZ9DEb61ztcTUmaV5EU5 WC7GUumYf/SeS5SqJHIl635rFlTL6YIe/j8nHRmupON6ccx2HdDs2ysZ5 A75zvmOP1mhHqzCmysTmVOtkrizKddIt7ArddK4o4M16XX3JyMeH5LFDr w6PM/+3yOHKhyzV982VWg/UcV5pPnxl46B5s7fSp8L6129JNkuGR/URBg DVuaIkz5r5YgcFKaqTKq7WeonDZ68f8SLGeB/yrU9Xd2X1D0JqQSOmR1C w==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416508723" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="416508723" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:08 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="756592040" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="756592040" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.13.232]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:06 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Dave Jiang , Ben Widawsky , Steven Rostedt Cc: Alison Schofield , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 3/6] cxl/memdev: Add trigger_poison_list sysfs attribute Date: Thu, 9 Feb 2023 15:32:56 -0800 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield When a boolean 'true' is written to this attribute the memdev driver retrieves the poison list from the device. The list consists of addresses that are poisoned, or would result in poison if accessed, and the source of the poison. This attribute is only visible for devices supporting the capability. The retrieved errors are logged as kernel trace events with the label 'cxl_poison'. Signed-off-by: Alison Schofield --- Documentation/ABI/testing/sysfs-bus-cxl | 14 +++++++ drivers/cxl/core/memdev.c | 56 +++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 2 +- 3 files changed, 71 insertions(+), 1 deletion(-) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 329a7e46c805..035213f47882 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -403,3 +403,17 @@ Description: 1), and checks that the hardware accepts the commit request. Reading this value indicates whether the region is committed or not. + + +What: /sys/bus/cxl/devices/memX/trigger_poison_list +Date: November, 2022 +KernelVersion: v6.2 +Contact: linux-cxl@vger.kernel.org +Description: + (WO) When a boolean 'true' is written to this attribute the + memdev driver retrieves the poison list from the device. The + list consists of addresses that are poisoned, or would result + in poison if accessed, and the source of the poison. This + attribute is only visible for devices supporting the + capability. The retrieved errors are logged as kernel + trace events with the label 'cxl_poison'. diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index a74a93310d26..19b833c9cf35 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -106,12 +106,60 @@ static ssize_t numa_node_show(struct device *dev, struct device_attribute *attr, } static DEVICE_ATTR_RO(numa_node); +static int cxl_get_poison_by_memdev(struct cxl_memdev *cxlmd) +{ + struct cxl_dev_state *cxlds = cxlmd->cxlds; + u64 offset, length; + int rc = 0; + + /* CXL 3.0 Spec 8.2.9.8.4.1 Separate pmem and ram poison requests */ + if (resource_size(&cxlds->pmem_res)) { + offset = cxlds->pmem_res.start; + length = resource_size(&cxlds->pmem_res); + rc = cxl_mem_get_poison(cxlmd, offset, length, NULL); + if (rc) + return rc; + } + if (resource_size(&cxlds->ram_res)) { + offset = cxlds->ram_res.start; + length = resource_size(&cxlds->ram_res); + rc = cxl_mem_get_poison(cxlmd, offset, length, NULL); + /* + * Invalid Physical Address is not an error for + * volatile addresses. Device support is optional. + */ + if (rc == -EFAULT) + rc = 0; + } + return rc; +} + +static ssize_t trigger_poison_list_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + bool trigger; + int rc; + + if (kstrtobool(buf, &trigger) || !trigger) + return -EINVAL; + + down_read(&cxl_dpa_rwsem); + rc = cxl_get_poison_by_memdev(cxlmd); + up_read(&cxl_dpa_rwsem); + + return rc ? rc : len; +} +static DEVICE_ATTR_WO(trigger_poison_list); + static struct attribute *cxl_memdev_attributes[] = { &dev_attr_serial.attr, &dev_attr_firmware_version.attr, &dev_attr_payload_max.attr, &dev_attr_label_storage_size.attr, &dev_attr_numa_node.attr, + &dev_attr_trigger_poison_list.attr, NULL, }; @@ -130,6 +178,14 @@ static umode_t cxl_memdev_visible(struct kobject *kobj, struct attribute *a, { if (!IS_ENABLED(CONFIG_NUMA) && a == &dev_attr_numa_node.attr) return 0; + + if (a == &dev_attr_trigger_poison_list.attr) { + struct device *dev = kobj_to_dev(kobj); + + if (!test_bit(CXL_MEM_COMMAND_ID_GET_POISON, + to_cxl_memdev(dev)->cxlds->enabled_cmds)) + return 0; + } return a->mode; } diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 17ea575f8a06..7c2489dd344c 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -140,7 +140,7 @@ struct cxl_mbox_cmd { C(FWROLLBACK, -ENXIO, "rolled back to the previous active FW"), \ C(FWRESET, -ENXIO, "FW failed to activate, needs cold reset"), \ C(HANDLE, -ENXIO, "one or more Event Record Handles were invalid"), \ - C(PADDR, -ENXIO, "physical address specified is invalid"), \ + C(PADDR, -EFAULT, "physical address specified is invalid"), \ C(POISONLMT, -ENXIO, "poison injection limit has been reached"), \ C(MEDIAFAILURE, -ENXIO, "permanent issue with the media"), \ C(ABORT, -ENXIO, "background cmd was aborted by device"), \ From patchwork Thu Feb 9 23:32:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13135203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 834F3C636D3 for ; Thu, 9 Feb 2023 23:35:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231201AbjBIXfd (ORCPT ); Thu, 9 Feb 2023 18:35:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231210AbjBIXfW (ORCPT ); Thu, 9 Feb 2023 18:35:22 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59F8E6D8DE; Thu, 9 Feb 2023 15:34:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675985695; x=1707521695; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Cv6WybSbF9U1JSwkfFCNpheiIUUtR7LMTKLupRFUKQ8=; b=UxcwgfrZhniLHUgbJOfe6cMWhVd1y0yZnNl1yswMFdIg3YgMe/h30PLw 1rpIJcBd8ai2+nvwCCVkLQGj4LONfDYGus+QES/R46WKw6lMbPi9Mumst TGcO2doeqOeyqKNQjFVFLM7Kt1zCOPuzHWUUNrfGb8nDDDFHT0Ei8Edjx WaCwVRG+6Wou1v7drRSGKp8jgtCds2nLKjV9HLSOcjPHpOfVYrOoZvtuJ pwjL8OfeKc0Mq9zUI4YHVYgpLRMcZHzuzxsFP/i/BBjrMar5bv5XVPRNA jb7YYUcy/Eza31WTP4+zNH4qiKU35GdJBckhhpyxe3jUBP5LdSxG2bNGJ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416508730" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="416508730" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:09 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="756592045" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="756592045" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.13.232]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:08 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Dave Jiang , Ben Widawsky , Steven Rostedt Cc: Alison Schofield , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 4/6] cxl/region: Provide region info to the cxl_poison trace event Date: Thu, 9 Feb 2023 15:32:57 -0800 Message-Id: <7862ef793a3b4fce49b0c8074b014b53b389ce12.1675983077.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield User space may need to know which region, if any, maps the poison address(es) logged in a cxl_poison trace event. Since the mapping of DPAs (device physical addresses) to a region can change, the kernel must provide this information at the time the poison list is read. The event informs user space that at event this mapped to this , which is poisoned. The cxl_poison trace event is already wired up to log the region name and uuid if it receives param 'struct cxl_region'. In order to provide that cxl_region, add another method for gathering poison - by committed endpoint decoder mappings. This method is only available with CONFIG_CXL_REGION and is only used if a region actually maps the memdev where poison is being read. The default method remains: read the poison by memdev resource. Signed-off-by: Alison Schofield --- drivers/cxl/core/core.h | 5 +++ drivers/cxl/core/memdev.c | 14 ++++++- drivers/cxl/core/region.c | 82 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 8c04672dca56..2f9bd8651eb1 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -22,7 +22,12 @@ void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled); #define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type) int cxl_region_init(void); void cxl_region_exit(void); +int cxl_get_poison_by_endpoint(struct device *dev, void *data); #else +static inline int cxl_get_poison_by_endpoint(struct device *dev, void *data) +{ + return 0; +} static inline void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled) { } diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 19b833c9cf35..8696d7b508b6 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -139,14 +139,26 @@ static ssize_t trigger_poison_list_store(struct device *dev, const char *buf, size_t len) { struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_port *port; bool trigger; int rc; if (kstrtobool(buf, &trigger) || !trigger) return -EINVAL; + port = dev_get_drvdata(&cxlmd->dev); + if (!port || !is_cxl_endpoint(port)) + return -EINVAL; + down_read(&cxl_dpa_rwsem); - rc = cxl_get_poison_by_memdev(cxlmd); + if (port->commit_end == -1) + /* No regions mapped to this memdev */ + rc = cxl_get_poison_by_memdev(cxlmd); + else + /* Regions mapped, collect poison by endpoint */ + rc = device_for_each_child(&port->dev, port, + cxl_get_poison_by_endpoint); + up_read(&cxl_dpa_rwsem); return rc ? rc : len; diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 67e83d961670..0ac08e9106af 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1826,6 +1826,88 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) } EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, CXL); +int cxl_get_poison_by_endpoint(struct device *dev, void *data) +{ + struct cxl_endpoint_decoder *cxled; + struct cxl_port *port = data; + struct cxl_dev_state *cxlds; + struct cxl_memdev *cxlmd; + u64 offset, length; + int rc = 0; + + down_read(&cxl_dpa_rwsem); + + if (!is_endpoint_decoder(dev)) + goto out; + + cxled = to_cxl_endpoint_decoder(dev); + if (!cxled->dpa_res || !resource_size(cxled->dpa_res)) + goto out; + + /* + * Get the poison by decoder for mapped resources. This + * separates pmem and ram poison list reads, as the spec + * requires, and provides the region for the trace event. + */ + cxlmd = cxled_to_memdev(cxled); + length = cxled->dpa_res->end - cxled->dpa_res->start + 1; + rc = cxl_mem_get_poison(cxlmd, cxled->dpa_res->start, length, + cxled->cxld.region); + if (rc == -EFAULT && cxled->mode == CXL_DECODER_RAM) + rc = 0; + if (rc) + goto out; + + /* Get poison in a skip range */ + if (cxled->skip) { + rc = cxl_mem_get_poison(cxlmd, 0, cxled->skip, NULL); + if (rc == -EFAULT && cxled->mode == CXL_DECODER_RAM) + rc = 0; + if (rc) + goto out; + } + + /* Iterate until commit_end is reached */ + if (cxled->cxld.id < port->commit_end) + goto out; + + /* + * Reach here with the last committed decoder only. + * Knowing that PMEM must always follow RAM, get poison + * for unmapped ranges based on the last decoder's mode: + * ram: scan remains of ram range, then scan for pmem + * pmem: scan remains of pmem range + */ + cxlds = cxlmd->cxlds; + + if (cxled->mode == CXL_DECODER_RAM) { + offset = cxled->dpa_res->end + 1; + length = resource_size(&cxlds->ram_res) - offset; + rc = cxl_mem_get_poison(cxlmd, offset, length, NULL); + if (rc == -EFAULT) + rc = 0; + if (rc) + goto out; + } + if (cxled->mode == CXL_DECODER_PMEM) { + offset = cxled->dpa_res->end + 1; + length = resource_size(&cxlds->pmem_res) - offset; + } else if (resource_size(&cxlds->pmem_res)) { + offset = cxlds->pmem_res.start; + length = resource_size(&cxlds->pmem_res); + } else { + rc = 1; + goto out; + } + /* Final get poison call. Return rc or 1 to stop iteration. */ + rc = cxl_mem_get_poison(cxlmd, offset, length, NULL); + if (!rc) + rc = 1; +out: + up_read(&cxl_dpa_rwsem); + return rc; +} + static struct lock_class_key cxl_pmem_region_key; static struct cxl_pmem_region *cxl_pmem_region_alloc(struct cxl_region *cxlr) From patchwork Thu Feb 9 23:32:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13135204 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE138C6379F for ; Thu, 9 Feb 2023 23:35:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231134AbjBIXfd (ORCPT ); Thu, 9 Feb 2023 18:35:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231213AbjBIXfW (ORCPT ); Thu, 9 Feb 2023 18:35:22 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97F066E8B; Thu, 9 Feb 2023 15:34:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675985695; x=1707521695; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2Lhw8ZtRWVlhOMwT0fnjXVLH8KzoyLq8BHXrBOWKlG8=; b=YaREexw57Q3SU/Qwkf6cL5JNAmtwYZnC2JufK+LZ7GQNjLfKZ9uDl0FM BS/FsG90ztCFez+jTIPHKZKL4RhJUcv+z9QQ4OLXpeFrCf5sVI1CVyp/f dB3vJTq6X0cga4/aKqTkWtGPpbGG25/0Apr252R+XCpol13ScROnZFoKE BXiW3L4FEnLp9YIiZHKaHwDZrdwt2Ll6166jNsTcCFjHuzKsthBLDSbWK xr7PUDZsZNdjTzRUxAcRtkH7Xhus4X9BowpAPmcP3WciGv4l5AMpwXF22 MFYzjlPFQn/4jc6SctHPuisLNHhGP0upr3Lnf0OiWk6WNXPItRfqSLwdV g==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416508742" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="416508742" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:11 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="756592049" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="756592049" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.13.232]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:10 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Dave Jiang , Ben Widawsky , Steven Rostedt Cc: Alison Schofield , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Jonathan Cameron Subject: [PATCH v6 5/6] cxl/trace: Add an HPA to cxl_poison trace events Date: Thu, 9 Feb 2023 15:32:58 -0800 Message-Id: <38b10009d7d222d871829f127b5651f0d6d9ca81.1675983077.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield When a cxl_poison trace event is reported for a region, the poisoned Device Physical Address (DPA) can be translated to a Host Physical Address (HPA) for consumption by user space. Translate and add the resulting HPA to the cxl_poison trace event. Follow the device decode logic as defined in the CXL Spec 3.0 Section 8.2.4.19.13. If no region currently maps the poison, assign ULLONG_MAX to the cxl_poison hpa event field. Reviewed-by: Jonathan Cameron Signed-off-by: Alison Schofield --- drivers/cxl/core/trace.c | 94 ++++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/trace.h | 9 +++- 2 files changed, 102 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/trace.c b/drivers/cxl/core/trace.c index 29ae7ce81dc5..d0403dc3c8ab 100644 --- a/drivers/cxl/core/trace.c +++ b/drivers/cxl/core/trace.c @@ -1,5 +1,99 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2022 Intel Corporation. All rights reserved. */ +#include +#include "core.h" + #define CREATE_TRACE_POINTS #include "trace.h" + +static bool cxl_is_hpa_in_range(u64 hpa, struct cxl_region *cxlr, int pos) +{ + struct cxl_region_params *p = &cxlr->params; + int gran = p->interleave_granularity; + int ways = p->interleave_ways; + u64 offset; + + /* Is the hpa within this region at all */ + if (hpa < p->res->start || hpa > p->res->end) { + dev_dbg(&cxlr->dev, + "Addr trans fail: hpa 0x%llx not in region\n", hpa); + return false; + } + + /* Is the hpa in an expected chunk for its pos(-ition) */ + offset = hpa - p->res->start; + offset = do_div(offset, gran * ways); + if ((offset >= pos * gran) && (offset < (pos + 1) * gran)) + return true; + + dev_dbg(&cxlr->dev, + "Addr trans fail: hpa 0x%llx not in expected chunk\n", hpa); + + return false; +} + +static u64 cxl_dpa_to_hpa(u64 dpa, struct cxl_region *cxlr, + struct cxl_endpoint_decoder *cxled) +{ + u64 dpa_offset, hpa_offset, bits_upper, mask_upper, hpa; + struct cxl_region_params *p = &cxlr->params; + int pos = cxled->pos; + u16 eig = 0; + u8 eiw = 0; + + ways_to_eiw(p->interleave_ways, &eiw); + granularity_to_eig(p->interleave_granularity, &eig); + + /* + * The device position in the region interleave set was removed + * from the offset at HPA->DPA translation. To reconstruct the + * HPA, place the 'pos' in the offset. + * + * The placement of 'pos' in the HPA is determined by interleave + * ways and granularity and is defined in the CXL Spec 3.0 Section + * 8.2.4.19.13 Implementation Note: Device Decode Logic + */ + + /* Remove the dpa base */ + dpa_offset = dpa - cxl_dpa_resource_start(cxled); + + mask_upper = GENMASK_ULL(51, eig + 8); + + if (eiw < 8) { + hpa_offset = (dpa_offset & mask_upper) << eiw; + hpa_offset |= pos << (eig + 8); + } else { + bits_upper = (dpa_offset & mask_upper) >> (eig + 8); + bits_upper = bits_upper * 3; + hpa_offset = ((bits_upper << (eiw - 8)) + pos) << (eig + 8); + } + + /* The lower bits remain unchanged */ + hpa_offset |= dpa_offset & GENMASK_ULL(eig + 7, 0); + + /* Apply the hpa_offset to the region base address */ + hpa = hpa_offset + p->res->start; + + if (!cxl_is_hpa_in_range(hpa, cxlr, cxled->pos)) + return ULLONG_MAX; + + return hpa; +} + +u64 cxl_trace_hpa(struct cxl_region *cxlr, struct cxl_memdev *cxlmd, + u64 dpa) +{ + struct cxl_region_params *p = &cxlr->params; + struct cxl_endpoint_decoder *cxled = NULL; + + for (int i = 0; i < p->nr_targets; i++) { + cxled = p->targets[i]; + if (cxlmd == cxled_to_memdev(cxled)) + break; + } + if (!cxled || cxlmd != cxled_to_memdev(cxled)) + return ULLONG_MAX; + + return cxl_dpa_to_hpa(dpa, cxlr, cxled); +} diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index c493955719ce..e6114dd0a95c 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -614,6 +614,8 @@ TRACE_EVENT(cxl_memory_module, #define cxl_poison_host_name(cxlmd) \ (dev_name(cxlmd->dev.parent)) +u64 cxl_trace_hpa(struct cxl_region *cxlr, struct cxl_memdev *memdev, u64 dpa); + TRACE_EVENT(cxl_poison, TP_PROTO(struct cxl_memdev *cxlmd, struct cxl_region *region, @@ -628,6 +630,7 @@ TRACE_EVENT(cxl_poison, __field(u64, serial) __string(region, region) __field(u64, overflow_t) + __field(u64, hpa) __field(u64, dpa) __field(u32, length) __array(char, uuid, 16) @@ -647,18 +650,22 @@ TRACE_EVENT(cxl_poison, if (region) { __assign_str(region, dev_name(®ion->dev)); memcpy(__entry->uuid, ®ion->params.uuid, 16); + __entry->hpa = cxl_trace_hpa(region, cxlmd, + __entry->dpa); } else { __assign_str(region, ""); memset(__entry->uuid, 0, 16); + __entry->hpa = ULLONG_MAX; } ), - TP_printk("memdev=%s serial=%llu host=%s region=%s region_uuid=%pU dpa=0x%llx length=0x%x source=%s flags=%s overflow_time=%llu", + TP_printk("memdev=%s serial=%llu host=%s region=%s region_uuid=%pU hpa=0x%llx dpa=0x%llx length=0x%x source=%s flags=%s overflow_time=%llu", __get_str(memdev), __entry->serial, __get_str(host), __get_str(region), __entry->uuid, + __entry->hpa, __entry->dpa, __entry->length, show_poison_source(__entry->source), From patchwork Thu Feb 9 23:32:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13135205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D730C05027 for ; Thu, 9 Feb 2023 23:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231214AbjBIXgG (ORCPT ); Thu, 9 Feb 2023 18:36:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231172AbjBIXfg (ORCPT ); Thu, 9 Feb 2023 18:35:36 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D68176F201; Thu, 9 Feb 2023 15:35:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675985712; x=1707521712; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XCVgqF8shH5rFyH7hk0Emjtujq62iDahDM3kYDntmz4=; b=HvkrHeTa4VBHBXazw+WB9hly1+XkQGAY3hnHhnLC0o4Yjd1ti+qLFq6Q lYsfOfqS0jmhu5lj3PZZreN+439RfPPCMjY9aLaABoXB0niUYWaB+gINV gG2rTnA47I965A81UueXJMP7T7nJ0Hqdp5uqoaOh78WxKIOeA1IjoFclf E03hFiy7cxeCfDPSjXZpAHw3C4XGUhq9kjBvlzc5qEXgBWu4HaUtE6D1Q uuNFXh7d6bhIX+YTY+ZY/RAeYH2FTFyIfMBHd62K25RUAzXzVF6w/fpMY 4hAtDyd8PWS9A/lIXSMQ+9HvjMvIo07frg+2+7gbT8LjyhVeRArEEjbPr A==; X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="416508753" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="416508753" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:12 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10616"; a="756592054" X-IronPort-AV: E=Sophos;i="5.97,285,1669104000"; d="scan'208";a="756592054" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.13.232]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2023 15:33:11 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Dave Jiang , Ben Widawsky , Steven Rostedt Cc: Alison Schofield , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Jonathan Cameron Subject: [PATCH v6 6/6] tools/testing/cxl: Mock support for Get Poison List Date: Thu, 9 Feb 2023 15:32:59 -0800 Message-Id: <26eb1ac9b8d6e7ff969537e29812d9e2d007632a.1675983077.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield Make mock memdevs support the Get Poison List mailbox command. Return a fake poison error record when the get poison list command is issued. This supports testing the kernel tracing and cxl list capabilities for media errors. Reviewed-by: Jonathan Cameron Signed-off-by: Alison Schofield --- tools/testing/cxl/test/mem.c | 42 ++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 9263b04d35f7..2fa9c18d4c2c 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include "trace.h" @@ -40,6 +41,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_HEALTH_INFO), .effect = cpu_to_le16(0), }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_POISON), + .effect = cpu_to_le16(0), + }, }; /* See CXL 2.0 Table 181 Get Health Info Output Payload */ @@ -471,6 +476,8 @@ static int mock_id(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) cpu_to_le64(DEV_SIZE / CXL_CAPACITY_MULTIPLIER), }; + put_unaligned_le24(CXL_POISON_LIST_MAX, id.poison_list_max_mer); + if (cmd->size_out < sizeof(id)) return -EINVAL; @@ -888,6 +895,34 @@ static int mock_health_info(struct cxl_dev_state *cxlds, return 0; } +static int mock_get_poison(struct cxl_dev_state *cxlds, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_poison_payload_in *pi = cmd->payload_in; + + /* Mock one poison record at pi.offset for 64 bytes */ + struct { + struct cxl_mbox_poison_payload_out po; + struct cxl_poison_record record; + } __packed mock_plist = { + .po = { + .count = cpu_to_le16(1), + }, + .record = { + .length = cpu_to_le32(1), + .address = cpu_to_le64(le64_to_cpu(pi->offset) + + CXL_POISON_SOURCE_INJECTED), + }, + }; + + if (cmd->size_out < sizeof(mock_plist)) + return -EINVAL; + + memcpy(cmd->payload_out, &mock_plist, sizeof(mock_plist)); + cmd->size_out = sizeof(mock_plist); + return 0; +} + static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) { struct device *dev = cxlds->dev; @@ -942,6 +977,9 @@ static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd * case CXL_MBOX_OP_PASSPHRASE_SECURE_ERASE: rc = mock_passphrase_secure_erase(cxlds, cmd); break; + case CXL_MBOX_OP_GET_POISON: + rc = mock_get_poison(cxlds, cmd); + break; default: break; } @@ -1010,6 +1048,10 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) if (rc) return rc; + rc = cxl_poison_state_init(cxlds); + if (rc) + return rc; + rc = cxl_dev_state_identify(cxlds); if (rc) return rc;