From patchwork Tue Apr 18 17:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 13216046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAD58C6FD18 for ; Tue, 18 Apr 2023 18:04:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230010AbjDRSES (ORCPT ); Tue, 18 Apr 2023 14:04:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232340AbjDRSER (ORCPT ); Tue, 18 Apr 2023 14:04:17 -0400 Received: from bird.elm.relay.mailchannels.net (bird.elm.relay.mailchannels.net [23.83.212.17]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 709DD61BA for ; Tue, 18 Apr 2023 11:04:15 -0700 (PDT) X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 22C278C0FBE; Tue, 18 Apr 2023 17:55:06 +0000 (UTC) Received: from pdx1-sub0-mail-a226.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 8D57C8C0F40; Tue, 18 Apr 2023 17:55:05 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1681840505; a=rsa-sha256; cv=none; b=JPJIr/Pc/iTngpFqOVgFDkxPgKxE2K5xTzVt9zsm4hx5wt4+Qnh9mz6ku9jFmmTwH3E39o DA0D22EKSpwbKQp4W7FpqtLYledCoVF+i8ddOrXJDVmMCQnjG5+XM329I8tBtnSuFbmaJs IHkq8ntvY/WfkY9m9YGkHO4jqvJw3mfJnrt5F6gLT7IDn9zQS1MJdQVBNaalYT4rFE8FMw yUdwn0gTpScR59resK/Kw1WHaNF0T61q32zoh3b631mCBXAE5NU3JRjk618mRSrjyYNX53 Svlfh6RzQUN80yxH2ufWDvpjQcBLaCOA9Jc7VssosyhfV58i4bzg+A1isB6Z8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1681840505; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XmAzEjoPyyq2SsP40EifHb6h7nFUwyqB1i9NozkHkKI=; b=5vt1zUTSiuZuk6OxQLgRWlyQUCD9iFwjPSq2T1keHJtoKEgT04NXZJjepyrJLUeg7wT86L d0/xsIPez2m12C9Cn0aMhzhS2siUl/mc1AdFnu7iMnE6yG4Ri0gWeuI0lyNWoatA4pQyaA NMg7ldwTHdtDHBW7TYw67NX/Dimjzz/lfs/MCNY3wIB2jw61tHrHM3kEG4Gre7WNRWsryG 1SClsomMzqYGNZXT6Ez0egKYoQXodJLRZS6Om6mzavdkvtp7JxsXF0tZMMsNpZs34dJYQs pSzD4CKU0ZHIkXLrz9GMBv6Ek1KUjlcGOQMoCqlanigEPrV9qCLejQCgGiMBkQ== ARC-Authentication-Results: i=1; rspamd-548d6c8f77-4vvzd; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Abortive-Attack: 6429c9b14ef085de_1681840505963_1479405905 X-MC-Loop-Signature: 1681840505963:1610252168 X-MC-Ingress-Time: 1681840505963 Received: from pdx1-sub0-mail-a226.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.123.193.179 (trex/6.7.2); Tue, 18 Apr 2023 17:55:05 +0000 Received: from localhost.localdomain (ip72-199-50-187.sd.sd.cox.net [72.199.50.187]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a226.dreamhost.com (Postfix) with ESMTPSA id 4Q1BQc4tBJzS6; Tue, 18 Apr 2023 10:55:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1681840505; bh=XmAzEjoPyyq2SsP40EifHb6h7nFUwyqB1i9NozkHkKI=; h=From:To:Cc:Subject:Date:Content-Transfer-Encoding; b=ldxjpA0hknemtnC8qkG9ametbKoxrz7hugKCT22+B4dzyXWN8rxCJEnrZLeeZy/tj 6gBpSEsZ0JjIgdbtXi06G49zRzMsrxo2F7yrh3tlrU3VLyXqBo7g2chzzizvuR8U37 lxuHeB5jSMIFnsG5TyDfjd0L7h1VNvu2nk9WEn3rJ7Y7SDN3doySh+hQQfUgusCvWN iohtedVIMwAjky/MzQqc0md7nigfzaTwNYjhMTFzaSvCcn6hNEFbH+Uzr3E8Akiaym bqrJ8QYax8wgwqPhQHBuyBGx8FeOyRzhSVoTNVZCiFWvQutPfaq3Uhbimxp+WDDgNv FX6ddNHUByF6Q== From: Davidlohr Bueso To: Jonathan.Cameron@huawei.com Cc: dan.j.williams@intel.com, fan.ni@samsung.com, a.manzanares@samsung.com, dave.jiang@intel.com, dave@stgolabs.net, linux-cxl@vger.kernel.org Subject: [PATCH 1/5] cxl/mbox: Add support for background operations Date: Tue, 18 Apr 2023 10:23:33 -0700 Message-Id: <20230418172337.19207-2-dave@stgolabs.net> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230418172337.19207-1-dave@stgolabs.net> References: <20230418172337.19207-1-dave@stgolabs.net> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Support background commands in the mailbox, and while at it update cmd_infostat_bg_op_sts() accordingly. This patch does not implement mbox interrupts upon completion, so the kernel driver must rely on polling to know when the operation is done. Signed-off-by: Davidlohr Bueso --- hw/cxl/cxl-device-utils.c | 5 +- hw/cxl/cxl-mailbox-utils.c | 95 ++++++++++++++++++++++++++++++++++++- hw/mem/cxl_type3.c | 2 + include/hw/cxl/cxl_device.h | 10 ++++ 4 files changed, 107 insertions(+), 5 deletions(-) diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c index 50c76c65e755..4bb4e85dae19 100644 --- a/hw/cxl/cxl-device-utils.c +++ b/hw/cxl/cxl-device-utils.c @@ -101,8 +101,7 @@ static void mailbox_mem_writeq(uint64_t *reg_state, hwaddr offset, case A_CXL_DEV_MAILBOX_CMD: break; case A_CXL_DEV_BG_CMD_STS: - /* BG not supported */ - /* fallthrough */ + break; case A_CXL_DEV_MAILBOX_STS: /* Read only register, will get updated by the state machine */ return; @@ -273,7 +272,7 @@ static void device_reg_init_common(CXLDeviceState *cxl_dstate) static void mailbox_reg_init_common(CXLDeviceState *cxl_dstate) { - /* 2048 payload size, with no interrupt or background support */ + /* 2048 payload size, with no interrupt */ ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CAP, PAYLOAD_SIZE, CXL_MAILBOX_PAYLOAD_SHIFT); cxl_dstate->payload_size = CXL_MAILBOX_MAX_PAYLOAD_SIZE; diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index b02908c4a4ba..1a4480d42908 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -350,7 +350,14 @@ static CXLRetCode cmd_infostat_bg_op_sts(struct cxl_cmd *cmd, bg_op_status = (void *)cmd->payload; memset(bg_op_status, 0, sizeof(*bg_op_status)); - /* No support yet for background operations so status all 0 */ + bg_op_status->status = ARRAY_FIELD_EX64(cxl_dstate->mbox_reg_state64, + CXL_DEV_BG_CMD_STS, PERCENTAGE_COMP) << 1; + if (cxl_dstate->bg.runtime > 0) { + bg_op_status->status |= 1U << 0; + } + bg_op_status->opcode = cxl_dstate->bg.opcode; + bg_op_status->returncode = ARRAY_FIELD_EX64(cxl_dstate->mbox_reg_state64, + CXL_DEV_BG_CMD_STS, RET_CODE); *len = sizeof(*bg_op_status); return CXL_MBOX_SUCCESS; } @@ -808,6 +815,7 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd *cmd, #define IMMEDIATE_DATA_CHANGE (1 << 2) #define IMMEDIATE_POLICY_CHANGE (1 << 3) #define IMMEDIATE_LOG_CHANGE (1 << 4) +#define BACKGROUND_OPERATION (1 << 5) static struct cxl_cmd cxl_cmd_set[256][256] = { [EVENTS][GET_RECORDS] = { "EVENTS_GET_RECORDS", @@ -856,12 +864,20 @@ static struct cxl_cmd cxl_cmd_set_sw[256][256] = { cmd_identify_switch_device, 0, 0x49 }, }; +/* + * While the command is executing in the background, the device should + * update the percentage complete in the Background Command Status Register + * at least once per second. + */ +#define CXL_MBOX_BG_UPDATE_FREQ 1000ULL + void cxl_process_mailbox(CXLDeviceState *cxl_dstate) { uint16_t ret = CXL_MBOX_SUCCESS; struct cxl_cmd *cxl_cmd; uint64_t status_reg; opcode_handler h; + uint8_t bg_started = 0; uint64_t command_reg = cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD]; uint8_t set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET); @@ -873,7 +889,17 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate) if (len == cxl_cmd->in || cxl_cmd->in == ~0) { cxl_cmd->payload = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD; + /* Only one bg command at a time */ + if ((cxl_cmd->effect & BACKGROUND_OPERATION) && + cxl_dstate->bg.runtime > 0) { + ret = CXL_MBOX_BUSY; + goto done; + } ret = (*h)(cxl_cmd, cxl_dstate, &len); + if ((cxl_cmd->effect & BACKGROUND_OPERATION) && + ret == CXL_MBOX_BG_STARTED) { + bg_started = 1; + } assert(len <= cxl_dstate->payload_size); } else { ret = CXL_MBOX_INVALID_PAYLOAD_LENGTH; @@ -884,8 +910,13 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate) ret = CXL_MBOX_UNSUPPORTED; } - /* Set the return code */ +done: + /* Set the return code and background status */ status_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_STS, ERRNO, ret); + if (bg_started) { + status_reg = FIELD_DP64(status_reg, CXL_DEV_MAILBOX_STS, + BG_OP, bg_started); + } /* Set the return length */ command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND_SET, 0); @@ -895,11 +926,66 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate) cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD] = command_reg; cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_STS] = status_reg; + if (bg_started) { + uint64_t bg_status_reg, now; + + cxl_dstate->bg.opcode = (set << 8) | cmd; + + bg_status_reg = FIELD_DP64(0, CXL_DEV_BG_CMD_STS, OP, cxl_dstate->bg.opcode); + bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS, + PERCENTAGE_COMP, 0); + cxl_dstate->mbox_reg_state64[R_CXL_DEV_BG_CMD_STS] = bg_status_reg; + + now = qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); + cxl_dstate->bg.starttime = now; + timer_mod(cxl_dstate->bg.timer, now + CXL_MBOX_BG_UPDATE_FREQ); + } + /* Tell the host we're done */ ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL, DOORBELL, 0); } +static void bg_timercb(void *opaque) +{ + CXLDeviceState *cxl_dstate = opaque; + uint64_t bg_status_reg = 0; + uint64_t now = qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL); + uint64_t total_time = cxl_dstate->bg.starttime + cxl_dstate->bg.runtime; + + assert(cxl_dstate->bg.runtime > 0); + bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS, + OP, cxl_dstate->bg.opcode); + + if (now >= total_time) { /* we are done */ + uint64_t status_reg; + uint16_t ret = CXL_MBOX_SUCCESS; + + cxl_dstate->bg.complete_pct = 100; + /* clear bg */ + status_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_STS, BG_OP, 0); + cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_STS] = status_reg; + + bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS, RET_CODE, ret); + + /* TODO add ad-hoc cmd succesful completion handling */ + } else { + /* estimate only */ + cxl_dstate->bg.complete_pct = 100 * now / total_time; + timer_mod(cxl_dstate->bg.timer, now + CXL_MBOX_BG_UPDATE_FREQ); + } + + bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS, PERCENTAGE_COMP, + cxl_dstate->bg.complete_pct); + cxl_dstate->mbox_reg_state64[R_CXL_DEV_BG_CMD_STS] = bg_status_reg; + + if (cxl_dstate->bg.complete_pct == 100) { + cxl_dstate->bg.starttime = 0; + /* registers are updated, allow new bg-capable cmds */ + cxl_dstate->bg.runtime = 0; + } +} + void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, bool switch_cci) { if (!switch_cci) { @@ -920,4 +1006,9 @@ void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, bool switch_cci) } } } + cxl_dstate->bg.complete_pct = 0; + cxl_dstate->bg.starttime = 0; + cxl_dstate->bg.runtime = 0; + cxl_dstate->bg.timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, + bg_timercb, cxl_dstate); } diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 334ce92f5e0f..63c28a1ed5d2 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -870,12 +870,14 @@ static void ct3_exit(PCIDevice *pci_dev) { CXLType3Dev *ct3d = CXL_TYPE3(pci_dev); CXLComponentState *cxl_cstate = &ct3d->cxl_cstate; + CXLDeviceState *cxl_dstate = &ct3d->cxl_dstate; ComponentRegisters *regs = &cxl_cstate->crb; pcie_aer_exit(pci_dev); cxl_doe_cdat_release(cxl_cstate); spdm_sock_fini(ct3d->doe_spdm.socket); g_free(regs->special_ops); + timer_free(cxl_dstate->bg.timer); if (ct3d->hostpmem) { address_space_destroy(&ct3d->hostpmem_as); } diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 1fa8522d33fa..dbb8a955723b 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -216,6 +216,16 @@ typedef struct cxl_device_state { struct cxl_cmd (*cxl_cmd_set)[256]; CPMUState cpmu[CXL_NUM_CPMU_INSTANCES]; CXLEventLog event_logs[CXL_EVENT_TYPE_MAX]; + + /* background command handling (times in ms) */ + struct { + uint16_t opcode; + uint16_t complete_pct; + uint64_t starttime; + /* set by each bg cmd, cleared by the bg_timer when complete */ + uint64_t runtime; + QEMUTimer *timer; + } bg; } CXLDeviceState; /* Initialize the register block for a device */ From patchwork Tue Apr 18 17:23:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 13216043 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11E28C6FD18 for ; Tue, 18 Apr 2023 18:01:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231262AbjDRSB0 (ORCPT ); Tue, 18 Apr 2023 14:01:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230435AbjDRSBY (ORCPT ); Tue, 18 Apr 2023 14:01:24 -0400 Received: from bird.elm.relay.mailchannels.net (bird.elm.relay.mailchannels.net [23.83.212.17]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57B7E1384B for ; Tue, 18 Apr 2023 11:01:18 -0700 (PDT) X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id D0A84922A40; Tue, 18 Apr 2023 17:55:06 +0000 (UTC) Received: from pdx1-sub0-mail-a226.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 270009227F9; Tue, 18 Apr 2023 17:55:06 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1681840506; a=rsa-sha256; cv=none; b=31UUYueAbC/suiN+l7Vvt3BxcqWqUZJHrceCsy950roByDOKUF04cHioypUPwINw+/4y+w hSspiETliPYBeXtwL7DtMHvchm24oWAKwtr1nHkZYv0cFmUES2DY+WgHJq34YWlt2vlXtt HwzwXTaYwxJxO/9T4+2k7AJrZN6mM9bHagn9kdDLSK5c+fp7C7SDLcX2fRZjUPPZ3K/wmW 4Io7CS6YrpmG+B/+rtb3ra1H55sYXPuM82E9opxHRlwAbcd+hDQREZNhdqULCUAowjPlMa MxaEE4Wlnz5YiocY/5daFohn13Zc5Nl9DAs1T9txbsbyXsHm8q5qo9X94zfU4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1681840506; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VD6bLHmOCXGMv/QgtQaA/I+v5UKYnbaBd/iqcXqEjLo=; b=vGorH7IxCVX7cN17AZvXokGQysnpa2G9ynZ/3Zqk1Fiwbdaulv/T5fOV0tIHo22B6ZAO/5 RI1sEDZH6TinneOqBmp8nAVxDiwWAEN415gkq6q1AIpT5ZIg19XrIuYJEiO5GgDl62QEYP ldZ4BhKDOa8mhm5sJcOBWE6bnAhjXYGLmAOHGvdQB+52s/q/KhpIUfnsZM9uveBo8A/lhY LjpcCu7qgoAatfYs9IPDGnnPyHWPE4BT5L2B9S8UfsVCPKuM7D5gvvmCOQ/zs+MZnZcgF5 j9eiipqA0+n3U3D7yco+AL0/sLmfy92TcLcZ5dgHgjgGQGyZUGaj3PTc/nyy0g== ARC-Authentication-Results: i=1; rspamd-548d6c8f77-lxbh4; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Lyrical-Lonely: 032546760402fc8f_1681840506634_715316162 X-MC-Loop-Signature: 1681840506634:525926386 X-MC-Ingress-Time: 1681840506634 Received: from pdx1-sub0-mail-a226.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.125.42.190 (trex/6.7.2); Tue, 18 Apr 2023 17:55:06 +0000 Received: from localhost.localdomain (ip72-199-50-187.sd.sd.cox.net [72.199.50.187]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a226.dreamhost.com (Postfix) with ESMTPSA id 4Q1BQd3XCYzC2; Tue, 18 Apr 2023 10:55:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1681840505; bh=VD6bLHmOCXGMv/QgtQaA/I+v5UKYnbaBd/iqcXqEjLo=; h=From:To:Cc:Subject:Date:Content-Transfer-Encoding; b=gmn+fmAhfd8wu7artzLTt38KcrTXc8Hv6vop3dSOYMuC/adfXZdqEvOHu51xqh9jP ZRwYRfCaIUtWPY0mzY7e7B6WTBDr/pNwTDK6hpk3EnNLBYk3DRkeccx6ZfKBuSfmSo DRlBtJRFUy6djuy1xvznWwUwh28z7PFSGhjHl4rMq0VKy7tkkb6abWbkwjIVftLloq 4ie0XgAT5vKQlSjto6BeDq/rfGGujjkzb31zBSLB18tc49J8rWuq2/sXoogw4gdNRS szdahxqUxzd7uiexJVd7y88ADGN1EOkPoAF4V+5oW6v0pqeBvmbbqQHrtXtUeFWlnn sk3SdAO3Neprg== From: Davidlohr Bueso To: Jonathan.Cameron@huawei.com Cc: dan.j.williams@intel.com, fan.ni@samsung.com, a.manzanares@samsung.com, dave.jiang@intel.com, dave@stgolabs.net, linux-cxl@vger.kernel.org Subject: [PATCH 2/5] cxl/mbox: Wire up interrupts for background completion Date: Tue, 18 Apr 2023 10:23:34 -0700 Message-Id: <20230418172337.19207-3-dave@stgolabs.net> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230418172337.19207-1-dave@stgolabs.net> References: <20230418172337.19207-1-dave@stgolabs.net> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Notify when the background operation is done. Signed-off-by: Davidlohr Bueso Reviewed-by: Jonathan Cameron --- hw/cxl/cxl-device-utils.c | 10 +++++++++- hw/cxl/cxl-mailbox-utils.c | 11 +++++++++++ hw/mem/cxl_type3.c | 2 +- include/hw/cxl/cxl_device.h | 1 + 4 files changed, 22 insertions(+), 2 deletions(-) diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c index 4bb4e85dae19..b10ab390234c 100644 --- a/hw/cxl/cxl-device-utils.c +++ b/hw/cxl/cxl-device-utils.c @@ -272,10 +272,18 @@ static void device_reg_init_common(CXLDeviceState *cxl_dstate) static void mailbox_reg_init_common(CXLDeviceState *cxl_dstate) { - /* 2048 payload size, with no interrupt */ + const uint8_t msi_n = 11; + + /* 2048 payload size */ ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CAP, PAYLOAD_SIZE, CXL_MAILBOX_PAYLOAD_SHIFT); cxl_dstate->payload_size = CXL_MAILBOX_MAX_PAYLOAD_SIZE; + /* irq support */ + ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CAP, + BG_INT_CAP, 1); + ARRAY_FIELD_DP32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CAP, + MSI_N, msi_n); + cxl_dstate->mbox_msi_n = msi_n; } static void memdev_reg_init_common(CXLDeviceState *cxl_dstate) { } diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 1a4480d42908..224bfdb4bfca 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -8,6 +8,8 @@ */ #include "qemu/osdep.h" +#include "hw/pci/msi.h" +#include "hw/pci/msix.h" #include "hw/cxl/cxl.h" #include "hw/cxl/cxl_events.h" #include "hw/pci/pci.h" @@ -980,9 +982,18 @@ static void bg_timercb(void *opaque) cxl_dstate->mbox_reg_state64[R_CXL_DEV_BG_CMD_STS] = bg_status_reg; if (cxl_dstate->bg.complete_pct == 100) { + CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate); + PCIDevice *pdev = PCI_DEVICE(ct3d); + cxl_dstate->bg.starttime = 0; /* registers are updated, allow new bg-capable cmds */ cxl_dstate->bg.runtime = 0; + + if (msix_enabled(pdev)) { + msix_notify(pdev, cxl_dstate->mbox_msi_n); + } else if (msi_enabled(pdev)) { + msi_notify(pdev, cxl_dstate->mbox_msi_n); + } } } diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 63c28a1ed5d2..801f72c5fcae 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -777,7 +777,7 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp) ComponentRegisters *regs = &cxl_cstate->crb; MemoryRegion *mr = ®s->component_registers; uint8_t *pci_conf = pci_dev->config; - unsigned short msix_num = 10; + unsigned short msix_num = 12; int i, rc; QTAILQ_INIT(&ct3d->error_list); diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index dbb8a955723b..f986651b6ead 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -189,6 +189,7 @@ typedef struct cxl_device_state { struct { MemoryRegion mailbox; uint16_t payload_size; + uint8_t mbox_msi_n; union { uint8_t mbox_reg_state[CXL_MAILBOX_REGISTERS_LENGTH]; uint16_t mbox_reg_state16[CXL_MAILBOX_REGISTERS_LENGTH / 2]; From patchwork Tue Apr 18 17:23:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 13216024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D264C77B78 for ; Tue, 18 Apr 2023 17:55:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231877AbjDRRzN (ORCPT ); Tue, 18 Apr 2023 13:55:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231600AbjDRRzM (ORCPT ); Tue, 18 Apr 2023 13:55:12 -0400 Received: from bird.elm.relay.mailchannels.net (bird.elm.relay.mailchannels.net [23.83.212.17]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42175198E for ; Tue, 18 Apr 2023 10:55:08 -0700 (PDT) X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 4E6C020299B; Tue, 18 Apr 2023 17:55:07 +0000 (UTC) Received: from pdx1-sub0-mail-a226.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id CD937201326; Tue, 18 Apr 2023 17:55:06 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1681840506; a=rsa-sha256; cv=none; b=s45ZVf/zuVh57Pqh8+FmpLUkCteQ9VgBUdOAVmorThICx1L8UWSzbA+neo/7L/wRdp/IiK obok2GBcnSgkCuUo2QHRJyRpjAHsfGAsSpnm0eiqFjwib0k3D+08ye61LTY5sGMrjAV1UC K+3tfNntZedtvpAJcsz7+ltRr1I69WEZW5idOEwrvpw11hqg+w7V5awCbdXaXW9SZ2aaoe ApQY//qI2sfjMn+fRDKCPUZYhIqeXM/iv32+sq0Xe8jdDrt1pNCc3Uq9wfKyJdDZIm8Xjc wMuCsb+cCLBR1MjiNzFR4MMcUasErBFkrx1YXh+gDN1+45uUYNjmVP1jqMgSYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1681840506; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L5+BPO3KJybMlN+XQEIg8dSrDJJ0T5WcTPZcOOyBYug=; b=1vGA7tXP494xj/QzuJ7jxKH7v0KDpjw3dFrBJv1rCcBJcI62doH8XxGtzz3rE891PIivbE wzaM+LRawmqyRVbmUph9HpIKUiaOFFkh8wHTDUDP8GCHfW5C9bKl2qKq5JbPH2w2g66J0i O75p8t9P+zp1qrklhfTVY50T7gaGKBkIfdKMTSHNU7Ia+hZGxG5ZJR/PFWd97nNw1cJFwU J0eR+ghgoer56XubDiTJX8BqTAnoEpSudvmleOalevBbSLFK50uoPl7relpPrUcBQG8l4R wQF3ETrKFRYVHNjGVckHG93PIt8373l05KlG3NJgeVtBe18TaCLFr/L/GUZUTw== ARC-Authentication-Results: i=1; rspamd-548d6c8f77-qlpbm; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Befitting-Callous: 6fba04ed06a60d82_1681840507143_3062619259 X-MC-Loop-Signature: 1681840507143:604960009 X-MC-Ingress-Time: 1681840507143 Received: from pdx1-sub0-mail-a226.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.104.253.222 (trex/6.7.2); Tue, 18 Apr 2023 17:55:07 +0000 Received: from localhost.localdomain (ip72-199-50-187.sd.sd.cox.net [72.199.50.187]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a226.dreamhost.com (Postfix) with ESMTPSA id 4Q1BQf1KRTzFG; Tue, 18 Apr 2023 10:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1681840506; bh=L5+BPO3KJybMlN+XQEIg8dSrDJJ0T5WcTPZcOOyBYug=; h=From:To:Cc:Subject:Date:Content-Transfer-Encoding; b=OzBjm5jXr+eHsrl8qWeM16VLX6BJ0qEz58qAkVNbLaH7DWR6lWLrpqEBHQWn1aKHd eD5c9yYJUxaeaRWuMObJnXNylKbBBj++1Nz66Ym6mZQgJKdUEseIkjamup4kZHljP+ iTOnjdbBdBQdKy8nef919q+4VjS6nRWjIuqGcrdcdtlNEvBPKujEFzTTCBfE75a6Mv 9o7Nn+rPjKR7R6OreA8CLfWjfiXm4xP09p7+6ZFJJ8fTVLqTlAc3Oi56uzwy5kFqyT z3YXIgAQnD9G48vFVs7XxM3srVOMr9dID7vAbMtYqjCIfGldQR+F82ESuZZWI/PCbw gGRoDv9IHJHRQ== From: Davidlohr Bueso To: Jonathan.Cameron@huawei.com Cc: dan.j.williams@intel.com, fan.ni@samsung.com, a.manzanares@samsung.com, dave.jiang@intel.com, dave@stgolabs.net, linux-cxl@vger.kernel.org Subject: [PATCH 3/5] cxl/device: Handle Media Disabled state Date: Tue, 18 Apr 2023 10:23:35 -0700 Message-Id: <20230418172337.19207-4-dave@stgolabs.net> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230418172337.19207-1-dave@stgolabs.net> References: <20230418172337.19207-1-dave@stgolabs.net> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Ensure that both memory device read/writes and mailbox commands can deal with the device media being disabled - per CXL 3.0 specs 8.2.9.8.5.1 - Sanitize. Disabled semantics here are strictly 11b, however noting that if the media is not in the ready state (01b), user data is not accessible. Signed-off-by: Davidlohr Bueso --- hw/cxl/cxl-mailbox-utils.c | 15 +++++++++++++++ hw/mem/cxl_type3.c | 10 ++++++++++ include/hw/cxl/cxl_device.h | 33 +++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 224bfdb4bfca..e0e20bc3a2bb 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -891,6 +891,21 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate) if (len == cxl_cmd->in || cxl_cmd->in == ~0) { cxl_cmd->payload = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD; + + if (cxl_dev_media_disabled(cxl_dstate)) { + if (h == cmd_events_get_records || + h == cmd_logs_get_log || + h == cmd_ccls_get_partition_info || + h == cmd_ccls_get_lsa || + h == cmd_ccls_set_lsa || + h == cmd_media_get_poison_list || + h == cmd_media_inject_poison || + h == cmd_media_clear_poison) { + ret = CXL_MBOX_MEDIA_DISABLED; + goto done; + } + } + /* Only one bg command at a time */ if ((cxl_cmd->effect & BACKGROUND_OPERATION) && cxl_dstate->bg.runtime > 0) { diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 801f72c5fcae..707bdc263f03 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -12,6 +12,7 @@ #include "qemu/pmem.h" #include "qemu/range.h" #include "qemu/rcu.h" +#include "qemu/guest-random.h" #include "sysemu/hostmem.h" #include "sysemu/numa.h" #include "hw/cxl/cxl.h" @@ -989,6 +990,11 @@ MemTxResult cxl_type3_read(PCIDevice *d, hwaddr host_addr, uint64_t *data, if (res) { return MEMTX_ERROR; } + /* all memory reads will return random values */ + if (cxl_dev_media_disabled(&CXL_TYPE3(d)->cxl_dstate)) { + qemu_guest_getrandom_nofail(data, size); + return MEMTX_OK; + } return address_space_read(as, dpa_offset, attrs, data, size); } @@ -1005,6 +1011,10 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data, if (res) { return MEMTX_ERROR; } + /* memory writes to the device will have no effect */ + if (cxl_dev_media_disabled(&CXL_TYPE3(d)->cxl_dstate)) { + return MEMTX_OK; + } return address_space_write(as, dpa_offset, attrs, &data, size); } diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index f986651b6ead..3ef7a7cded95 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -346,6 +346,39 @@ REG64(CXL_MEM_DEV_STS, 0) FIELD(CXL_MEM_DEV_STS, MBOX_READY, 4, 1) FIELD(CXL_MEM_DEV_STS, RESET_NEEDED, 5, 3) +static inline void cxl_dev_media_toggle(CXLDeviceState *cxl_dstate, int val) +{ + uint64_t reg; + + reg = ldq_le_p(cxl_dstate->mbox_reg_state64 + R_CXL_MEM_DEV_STS); + if (FIELD_EX64(reg, CXL_MEM_DEV_STS, MEDIA_STATUS) == val) { + return; + } + + reg = FIELD_DP64(0, CXL_MEM_DEV_STS, MEDIA_STATUS, val); + reg = FIELD_DP64(reg, CXL_MEM_DEV_STS, MBOX_READY, 0x1); + + stq_le_p(&cxl_dstate->mbox_reg_state64 + R_CXL_MEM_DEV_STS, reg); +} + +static inline void cxl_dev_media_enable(CXLDeviceState *cxl_dstate) +{ + cxl_dev_media_toggle(cxl_dstate, 0x1); +} + +static inline void cxl_dev_media_disable(CXLDeviceState *cxl_dstate) +{ + cxl_dev_media_toggle(cxl_dstate, 0x3); +} + +static inline bool cxl_dev_media_disabled(CXLDeviceState *cxl_dstate) +{ + uint64_t reg; + + reg = ldq_le_p(cxl_dstate->mbox_reg_state64 + R_CXL_MEM_DEV_STS); + return unlikely(FIELD_EX64(reg, CXL_MEM_DEV_STS, MEDIA_STATUS) == 0x3); +} + typedef struct CXLError { QTAILQ_ENTRY(CXLError) node; int type; /* Error code as per FE definition */ From patchwork Tue Apr 18 17:23:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 13216026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08134C77B7E for ; Tue, 18 Apr 2023 17:55:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231701AbjDRRzO (ORCPT ); Tue, 18 Apr 2023 13:55:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231637AbjDRRzM (ORCPT ); Tue, 18 Apr 2023 13:55:12 -0400 Received: from bee.birch.relay.mailchannels.net (bee.birch.relay.mailchannels.net [23.83.209.14]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 303C5CC1B for ; Tue, 18 Apr 2023 10:55:09 -0700 (PDT) X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 5AA5B3C0FAD; Tue, 18 Apr 2023 17:55:08 +0000 (UTC) Received: from pdx1-sub0-mail-a226.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id DC3AF3C21F5; Tue, 18 Apr 2023 17:55:07 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1681840507; a=rsa-sha256; cv=none; b=Q/NIgeVVIZ+BRDJ6s+PqhEjfc/7ai+wUj8nA2q91FzhxiY0CaXncGpVM5kO0N7MavzcFZw 78suI329t5JRhqfU6fJd8PjWGTVnw/uh12Dvl/1dxGXJf0NMjsCR5SbIMBJKZ3CVj1eikO N1+PKmzDNxY70ArF3IJVtpTNP9MyHLs8/XxUHE7lHAQqVNyPI/M+NbXXBJZHGob/VCWc0r Z0udfgnPtLpwqe/Orh+1BhcQ9+RWwaxR2griQUPiBy7b+Eszgrxbj452StzdM+JN+aU1G5 rpSvwB1vuvXj5FypxvO1LnSS0O8d6srPohnRJAszXZl4NHq2/T6lujsMVmFZ/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1681840507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=35KmwLYXlr2Dk0FAwKgQFfarggDjni7NtBBnP+Y5sSw=; b=DQEQO4zYnUKPkLirpL4kEl7zEBDPftGq5BNXrb9fyDekT5aBEJkcImZ/k6SOThbMcHcnQp SvOv8o8GKdrDL4dpMjZFxzmwx0WhoRmX7aXKANPgTAy6YdxlbZBFeYX9yJxk3KGNOf7T5h fr9Tx9ObK7f13IkwzN3KBe0/Uv8aelqzpCptdVnT5kzULwZgK2Vn28iW2AkSCO4L2WTm4h VwiD8OiygM7Vt9tmxWg3vNyI7jVEd0KJ0jVDzZCxOQP8RXnCrm5by19L1NeAdhIC8PDs4J W23c/ygcBtrxJicjGFQH6zKcKP5sMU/mgNKW7DVeYcdNLhxi3s/vq+c8nkU2Qg== ARC-Authentication-Results: i=1; rspamd-548d6c8f77-xbcmw; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Skirt-Oafish: 7907daf4732ae619_1681840508179_1184193323 X-MC-Loop-Signature: 1681840508179:1813911708 X-MC-Ingress-Time: 1681840508179 Received: from pdx1-sub0-mail-a226.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.123.193.179 (trex/6.7.2); Tue, 18 Apr 2023 17:55:08 +0000 Received: from localhost.localdomain (ip72-199-50-187.sd.sd.cox.net [72.199.50.187]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a226.dreamhost.com (Postfix) with ESMTPSA id 4Q1BQf6fwZz2x; Tue, 18 Apr 2023 10:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1681840507; bh=35KmwLYXlr2Dk0FAwKgQFfarggDjni7NtBBnP+Y5sSw=; h=From:To:Cc:Subject:Date:Content-Transfer-Encoding; b=JzmSWpClOVxRLsYMQGvnzKYf7XRdC21vT/pZa++r9dIh7BweHnP164rObSxtrj0Kd iAure4Z0D7gxVN+EjXWjmLr0TDVAJIlNerpDRaVJyP6O5A2O7iWtRtbkH/xOsx4UoU 3+fpZWFyC4dTP7/5ovT6pHmoQPqqwn7P2u1zqHpkRwdbMQosiMFekO7JdndYbAM7B0 qj4LqsL+fGehYIpWf/kuSpnJKjhkCh/Aufox1n9U3Tvm3q58HSzoyepFzjpwk2e/pk 2vDlYli6q3uBbggoBnZ6uoLx/qAENBIG/Z7rZtTDy60bSdMDYJTGx1HFyV5iCwoBLh gHW6OkooXlvDQ== From: Davidlohr Bueso To: Jonathan.Cameron@huawei.com Cc: dan.j.williams@intel.com, fan.ni@samsung.com, a.manzanares@samsung.com, dave.jiang@intel.com, dave@stgolabs.net, linux-cxl@vger.kernel.org Subject: [PATCH 4/5] hw/cxl: Add support for device sanitation Date: Tue, 18 Apr 2023 10:23:36 -0700 Message-Id: <20230418172337.19207-5-dave@stgolabs.net> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230418172337.19207-1-dave@stgolabs.net> References: <20230418172337.19207-1-dave@stgolabs.net> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Make use of the background operations through the sanitize command, per CXL 3.0 specs. Traditionally run times can be rather long, depending on the size of the media. Signed-off-by: Davidlohr Bueso --- hw/cxl/cxl-mailbox-utils.c | 125 ++++++++++++++++++++++++++++++++++++- 1 file changed, 124 insertions(+), 1 deletion(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index e0e20bc3a2bb..2808d0161a38 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -18,6 +18,7 @@ #include "qemu/log.h" #include "qemu/units.h" #include "qemu/uuid.h" +#include "sysemu/hostmem.h" #define CXL_CAPACITY_MULTIPLIER (256 * MiB) @@ -71,6 +72,9 @@ enum { #define GET_PARTITION_INFO 0x0 #define GET_LSA 0x2 #define SET_LSA 0x3 + SANITATION = 0x44, + #define SANITIZE 0x0 + #define SECURE_ERASE 0x1 MEDIA_AND_POISON = 0x43, #define GET_POISON_LIST 0x0 #define INJECT_POISON 0x1 @@ -819,6 +823,112 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd *cmd, #define IMMEDIATE_LOG_CHANGE (1 << 4) #define BACKGROUND_OPERATION (1 << 5) +/* re-initialize the address space */ +static void __do_sanitization(CXLDeviceState *cxl_dstate) +{ + CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate); + DeviceState *ds = DEVICE(ct3d); + char *name; + MemoryRegion *mr; + + if (ct3d->hostvmem) { + mr = host_memory_backend_get_memory(ct3d->hostvmem); + if (!mr) { + return; + } + if (ds->id) { + name = g_strdup_printf("cxl-type3-dpa-vmem-space:%s", ds->id); + } else { + name = g_strdup("cxl-type3-dpa-vmem-space"); + } + + address_space_destroy(&ct3d->hostvmem_as); + address_space_init(&ct3d->hostvmem_as, mr, name); + g_free(name); + } + if (ct3d->hostpmem) { + mr = host_memory_backend_get_memory(ct3d->hostpmem); + if (!mr) { + return; + } + if (ds->id) { + name = g_strdup_printf("cxl-type3-dpa-pmem-space:%s", ds->id); + } else { + name = g_strdup("cxl-type3-dpa-pmem-space"); + } + + address_space_destroy(&ct3d->hostpmem_as); + address_space_init(&ct3d->hostpmem_as, mr, name); + g_free(name); + } +} + +/* + * CXL 3.0 spec section 8.2.9.8.5.1 - Sanitize. + * + * Once the Sanitize command has started successfully, the device shall + * shall be placed in the media disabled state. If the command fails or + * is interrupted by a reset or power failure, it shall remain in the + * media disabled state until a successful Sanitize command has been + * completed. + */ +static CXLRetCode cmd_sanitation_sanitize(struct cxl_cmd *cmd, + CXLDeviceState *cxl_dstate, + uint16_t *len) +{ + unsigned int secs; + uint64_t total_mem; /* in Mb */ + struct cxl_cmd *c; + + *len = 0; + + /* Reported in CEL */ + c = &cxl_dstate->cxl_cmd_set[SANITATION][SANITIZE]; + if (!(c->effect & BACKGROUND_OPERATION)) { + __do_sanitization(cxl_dstate); + return CXL_MBOX_SUCCESS; + } + + /* + * Estimated times based on: + * https://pmem.io/documents/NVDIMM_DSM_Interface-V1.8.pdf + */ + total_mem = (cxl_dstate->vmem_size + cxl_dstate->pmem_size) >> 20; + if (total_mem <= 512) { + secs = 4; + } else if (total_mem <= 1024) { + secs = 8; + } else if (total_mem <= 2 * 1024) { + secs = 15; + } else if (total_mem <= 4 * 1024) { + secs = 30; + } else if (total_mem <= 8 * 1024) { + secs = 60; + } else if (total_mem <= 16 * 1024) { + secs = 2 * 60; + } else if (total_mem <= 32 * 1024) { + secs = 4 * 60; + } else if (total_mem <= 64 * 1024) { + secs = 8 * 60; + } else if (total_mem <= 128 * 1024) { + secs = 15 * 60; + } else if (total_mem <= 256 * 1024) { + secs = 30 * 60; + } else if (total_mem <= 512 * 1024) { + secs = 60 * 60; + } else if (total_mem <= 1024 * 1024) { + secs = 120 * 60; + } else { + secs = 240 * 60; /* max 4 hrs */ + } + + /* sanitize when done */ + cxl_dev_media_disable(cxl_dstate); + cxl_dstate->bg.runtime = secs * 1000UL; + + return CXL_MBOX_BG_STARTED; +} + static struct cxl_cmd cxl_cmd_set[256][256] = { [EVENTS][GET_RECORDS] = { "EVENTS_GET_RECORDS", cmd_events_get_records, 1, 0 }, @@ -842,6 +952,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = { [CCLS][GET_LSA] = { "CCLS_GET_LSA", cmd_ccls_get_lsa, 8, 0 }, [CCLS][SET_LSA] = { "CCLS_SET_LSA", cmd_ccls_set_lsa, ~0, IMMEDIATE_CONFIG_CHANGE | IMMEDIATE_DATA_CHANGE }, + [SANITATION][SANITIZE] = { "SANITATION_SANITIZE", cmd_sanitation_sanitize, + 0, IMMEDIATE_DATA_CHANGE | BACKGROUND_OPERATION }, [MEDIA_AND_POISON][GET_POISON_LIST] = { "MEDIA_AND_POISON_GET_POISON_LIST", cmd_media_get_poison_list, 16, 0 }, [MEDIA_AND_POISON][INJECT_POISON] = { "MEDIA_AND_POISON_INJECT_POISON", @@ -985,7 +1097,18 @@ static void bg_timercb(void *opaque) bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS, RET_CODE, ret); - /* TODO add ad-hoc cmd succesful completion handling */ + /* ad-hoc cmd succesful completion handling */ + if (ret == CXL_MBOX_SUCCESS) { + switch (cxl_dstate->bg.opcode) { + case 0x4400: /* sanitize */ + __do_sanitization(cxl_dstate); + cxl_dev_media_enable(cxl_dstate); + break; + default: + __builtin_unreachable(); + break; + } + } } else { /* estimate only */ cxl_dstate->bg.complete_pct = 100 * now / total_time; From patchwork Tue Apr 18 17:23:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 13216025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E0E8C77B75 for ; Tue, 18 Apr 2023 17:55:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231485AbjDRRzP (ORCPT ); Tue, 18 Apr 2023 13:55:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231657AbjDRRzM (ORCPT ); Tue, 18 Apr 2023 13:55:12 -0400 Received: from bird.elm.relay.mailchannels.net (bird.elm.relay.mailchannels.net [23.83.212.17]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C3344688 for ; Tue, 18 Apr 2023 10:55:10 -0700 (PDT) X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id B5BF27E277A; Tue, 18 Apr 2023 17:55:09 +0000 (UTC) Received: from pdx1-sub0-mail-a226.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 371C77E2818; Tue, 18 Apr 2023 17:55:09 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1681840509; a=rsa-sha256; cv=none; b=4/j6v1HiYzyO+fUNRNWl4ez+2FHcKk0w21udiLeBNi1G8LAN8dx7mU4bkwPCoGQ3L/XJAA Ycywx7sDbi8sig9mX4CApfoWUYvFhhBjV3xoKkqLHmLPdRS0syonb+9zAWzOtWzM2sZem5 Vu80Tz/ulthFDA31zya6aRMycUFAfSaxdSU8chYu0Dt1V48WzqDu6sm3cc5zKFqg3IWd7d 64//JGntZSrhwC6FfcrVrM36w5+iCxmSgNkoC5xEpxQ9m1c0xG6YpzQMJ2CZyLaV9M0E8j 5xtQr9WBRLAAzO5YCOVfaBK7/P58lA77+0ivwUjnMMemo8Q5YGaen5hTOzV5Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1681840509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E6s93KH+ZAPOkb9pHYT5RNI2+vfthKtochxbW4rHpuQ=; b=uwIEaM7brlLnlccw4c9ZbRg33tz+IB2Cx0omcWTH69lf2CUmA4aP6VPZ3n6qai+u9xoHWJ Wil/T9MpBmx2WNRaCyXpV1O97yXa8tQs2mmx11cXbl+0FD1b1bQtpYY+VfKxPaS0vC3MCq /ql7OHZcBv9KBvOh2jXTKNJCftUBcvmXXUKKb8KbrOC8TRAEiY42wF88bs51oUQ3WZaZd7 FK17up98K2zplQrUITrBANb20+aTjcEO8UptwEAv8MGJTPHn+Vjy0n1fynt0Uscfu3B6UG +Vnyt4IBkHMv9eoyjQANcM2nhMnaKvZXf2TIu84g9CqxZlmn051YiE1q4Zvayg== ARC-Authentication-Results: i=1; rspamd-548d6c8f77-kj4fj; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Glossy-Juvenile: 597acd4b524b480d_1681840509531_394784509 X-MC-Loop-Signature: 1681840509531:3930250491 X-MC-Ingress-Time: 1681840509531 Received: from pdx1-sub0-mail-a226.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.104.253.222 (trex/6.7.2); Tue, 18 Apr 2023 17:55:09 +0000 Received: from localhost.localdomain (ip72-199-50-187.sd.sd.cox.net [72.199.50.187]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a226.dreamhost.com (Postfix) with ESMTPSA id 4Q1BQh1Dv1zBW; Tue, 18 Apr 2023 10:55:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1681840509; bh=E6s93KH+ZAPOkb9pHYT5RNI2+vfthKtochxbW4rHpuQ=; h=From:To:Cc:Subject:Date:Content-Transfer-Encoding; b=AQ9RQZuJMAoAzh/mZo6ihjpPJti+kNfcRrtWugFDPYoasewkDCIdCiZT4PACExQIM 5t4TGhbkBJTplbOhwQczkwGyDXFuwt5iMKlMO7Xz8jaViQV4awUtMMf/0fKWyl9DBi 1uQeU6GstVDF5Esv1dZAWNN9ZGrR+kPnwXIx6aotqiG6iJ3xjojooM8IN2NcEe+F6+ YwCpgNjtcBq5Dxy0VDAwsUExTxTk+Dh+agF+PnAkF2x7DfN9uleR8CQNOrmsez6UOE t68tedjjBMVW+/qTxNt5Y5/TyJMkjcbDVWmoFdxttV+EvZBoYqjGw6udDPkNfvXytq qy+NZrdoWGu9w== From: Davidlohr Bueso To: Jonathan.Cameron@huawei.com Cc: dan.j.williams@intel.com, fan.ni@samsung.com, a.manzanares@samsung.com, dave.jiang@intel.com, dave@stgolabs.net, linux-cxl@vger.kernel.org Subject: [PATCH 5/5] cxl/type3: Remove superfluous statements Date: Tue, 18 Apr 2023 10:23:37 -0700 Message-Id: <20230418172337.19207-6-dave@stgolabs.net> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230418172337.19207-1-dave@stgolabs.net> References: <20230418172337.19207-1-dave@stgolabs.net> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Simple cleanup while going through the code, remove the bogus return statements from end of void returning functions. Signed-off-by: Davidlohr Bueso Reviewed-by: Jonathan Cameron --- hw/mem/cxl_type3.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 707bdc263f03..415a8c8a3973 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -864,7 +864,6 @@ err_address_space_free: if (ct3d->hostvmem) { address_space_destroy(&ct3d->hostvmem_as); } - return; } static void ct3_exit(PCIDevice *pci_dev) @@ -1290,8 +1289,6 @@ void qmp_cxl_inject_uncorrectable_errors(const char *path, stl_le_p(reg_state + R_CXL_RAS_UNC_ERR_STATUS, unc_err); pcie_aer_inject_error(PCI_DEVICE(obj), &err); - - return; } void qmp_cxl_inject_correctable_error(const char *path, CxlCorErrorType type, @@ -1564,7 +1561,6 @@ void qmp_cxl_inject_dram_event(const char *path, CxlEventLog log, uint8_t flags, if (cxl_event_insert(cxlds, enc_log, (CXLEventRecordRaw *)&dram)) { cxl_event_irq_assert(ct3d); } - return; } void qmp_cxl_inject_memory_module_event(const char *path, CxlEventLog log,