From patchwork Tue Oct 11 21:19:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13004392 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B96C4C4332F for ; Tue, 11 Oct 2022 21:20:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229605AbiJKVU0 (ORCPT ); Tue, 11 Oct 2022 17:20:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229520AbiJKVUZ (ORCPT ); Tue, 11 Oct 2022 17:20:25 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C24274D16D for ; Tue, 11 Oct 2022 14:20:24 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id m6so4263175qkm.4 for ; Tue, 11 Oct 2022 14:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2ozVAThgn2oeccWPcE+jwIIHUv3uIclTtkDlamqTwwk=; b=k4fm6dEkcYzgX5Q7VPbkNgie4EmRxxWgLPHRI3Nax8PSmgCd7AbMlPrjI3hyajdpsD OLh9ezcYAu/z46FVdEr3qpsCfGnR5uASxk3kWBknjPlz9ChU1BKL/3GBfH7a/S2T5HUE ZT3CVG+ctHA27V3xOsWC2jtopjCdiW7VKk3vlGh1/ZdZxnBWfkySpHYKndA0anhZDaLr TzgelghthKQk2+vTvYjHcRKCcMUyHO91rMycqIlPojbBNyxdyBSVQYMHkNSeBi9a99He m9xREuRmkubmc/iijzY2yvkOBPDGnEYfvQ73y4tWCpjyK9hnd9PbUTApFV22FYFfXzvZ Hzfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2ozVAThgn2oeccWPcE+jwIIHUv3uIclTtkDlamqTwwk=; b=hACbNe7EC93i+zYCUjYnBYpVB8gesFMwbd8gipypu2W7vumzhTYr69I/drYBNs4H+H DEdZUssUNSNipn2S3uZ77dt20g8vTn6DmaJ44XpxJgO7/I+V5hNSjQfGvVLakuTfLcfl PltobdMz7qK7UD1FBJODFuEWV5cR0oV0XuhzCQO9m813JwFyAQ+v6G2t7xL5xe+thSSY eJrxIy463dqs+d9G5saPuaMQhtoILBsK9b24GgUbIdlgCRspPnECcFvl3QATvW5eZk0X eYx+0vWrsc74yYKwph8dthXDBeCGC7Y5tJSzvPdkcCFqFlfpKEXAku/3XZ0ajcK7tups l5oA== X-Gm-Message-State: ACrzQf32YMak3zozHcn6OXFRiT+Rp50P/VPX9fknrH9VY2pclhzlkqqP g9FgCFU+XqSwX8W9s6aaHA== X-Google-Smtp-Source: AMsMyM7kFZvP7defG2iJkjuNTo0HSF3VoSHFoaf7ehStKnxdYQZUqMA9fVaJJS+cPNwjYDO4WLwAiw== X-Received: by 2002:a05:620a:22c3:b0:6ec:53bb:d296 with SMTP id o3-20020a05620a22c300b006ec53bbd296mr10773609qki.158.1665523223758; Tue, 11 Oct 2022 14:20:23 -0700 (PDT) Received: from fedora.mshome.net (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id y21-20020a05620a44d500b006b8f4ade2c9sm14493164qkp.19.2022.10.11.14.20.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:20:23 -0700 (PDT) From: Gregory Price X-Google-Original-From: Gregory Price To: jonathan.cameron@huawei.com Cc: qemu-devel@nongnu.org, linux-cxl@vger.kernel.org, alison.schofield@intel.com, dave@stgolabs.net, a.manzanares@samsung.com, bwidawsk@kernel.org, gregory.price@memverge.com, mst@redhat.com, hchkuo@avery-design.com.tw, cbrowy@avery-design.com, ira.weiny@intel.com, Jonathan Cameron Subject: [PATCH 1/5] hw/cxl: set cxl-type3 device type to PCI_CLASS_MEMORY_CXL Date: Tue, 11 Oct 2022 17:19:12 -0400 Message-Id: <20221011211916.117552-2-gregory.price@memverge.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011211916.117552-1-gregory.price@memverge.com> References: <20221007152156.24883-1-Jonathan.Cameron@huawei.com> <20221011211916.117552-1-gregory.price@memverge.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Current code sets to STORAGE_EXPRESS and then overrides it. Signed-off-by: Gregory Price Reviewed-by: Davidlohr Bueso Reviewed-by: Jonathan Cameron --- hw/mem/cxl_type3.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 3e7ca7a455..282f274266 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -535,7 +535,6 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp) } pci_config_set_prog_interface(pci_conf, 0x10); - pci_config_set_class(pci_conf, PCI_CLASS_MEMORY_CXL); pcie_endpoint_cap_init(pci_dev, 0x80); if (ct3d->sn != UI64_NULL) { @@ -763,7 +762,7 @@ static void ct3_class_init(ObjectClass *oc, void *data) pc->config_read = ct3d_config_read; pc->realize = ct3_realize; pc->exit = ct3_exit; - pc->class_id = PCI_CLASS_STORAGE_EXPRESS; + pc->class_id = PCI_CLASS_MEMORY_CXL; pc->vendor_id = PCI_VENDOR_ID_INTEL; pc->device_id = 0xd93; /* LVF for now */ pc->revision = 1; From patchwork Tue Oct 11 21:19:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13004393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54146C43217 for ; Tue, 11 Oct 2022 21:20:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229545AbiJKVU3 (ORCPT ); Tue, 11 Oct 2022 17:20:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229520AbiJKVU2 (ORCPT ); Tue, 11 Oct 2022 17:20:28 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70D324D26E for ; Tue, 11 Oct 2022 14:20:27 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id z18so9793590qvn.6 for ; Tue, 11 Oct 2022 14:20:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tk6+oDslH4pTPbHcXpDk0IpR14fq+cZd9zEm6po2Q3M=; b=URTPnec8rUQ2Pm6MSDK4VRTy9jqNMUprXSPDGota9/3oLYIssbz4Pj1bR14A85IIg8 GNKCDGBnP0G2UKdEMJ1EbvddJFMQS3OYheXyyMu2h4vMedJJ8h1vtftIuVcj8iCm+cGy fjI2ZioQ9I1IombD+cb60hsc1gxmfKglf/897r22WH0KV4JQSUHnnc5RO395EpZBxsPo MB0Xkl4dd0oRcUFoKs0CsOqq0RmTR8+ZjcWX2FICpIjoJnEt/Wm1MkAS4WkqsNe9Ii0H bHT7cVyE0rTC742Jde+ZCfrpOoEcC0hqZeKpNbyZm/nzK8dpv3pHKAkHxaojQ8x69GQy IKtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tk6+oDslH4pTPbHcXpDk0IpR14fq+cZd9zEm6po2Q3M=; b=bBMgmA8k2FD3a5GgfZZeGL9xfIfvxVEpgws9cvfCrVW0GQBZY3exjVHIaNMppo4llo DLXX6HIT8VZniSfMrQr8gQ6cDIXXi38w1kEG+p5JBS5rKVYJ3LfgnyL1FONMKJnMMhfL yFu+pM50jtfXSy04bjtVK5sDDHgT4M4eGrN0fp8LaALBNPpybSuAc64Y1h9e2kYEntxe 5cVxRaRQt+FkIuNYo7h/B2iqcEV4y50llLnU3En61xmJMkPCvjYA4x6bGddtBZJeucfO OYvxWV1+wqTLw568SnG9JGUOCP3K6ldhWKFyi2rKTQkyvCWrXwaL/AXnW4CVWe1D+XgV Z5eA== X-Gm-Message-State: ACrzQf3gjyUtVQDiRuUjfVb/IBXk17sqPkNHHF26np6el8bxFXemkNi9 W7Ufs/ApK5riOipZA8/6/A== X-Google-Smtp-Source: AMsMyM7NpmrToXN1ecncH/qlivf1f9hvWXGrXotwBzOGtP6e2p4xkY5CJElPoUzkp2tJXJY934n/ow== X-Received: by 2002:a0c:9c4d:0:b0:4b1:c126:b1e8 with SMTP id w13-20020a0c9c4d000000b004b1c126b1e8mr20543436qve.21.1665523226605; Tue, 11 Oct 2022 14:20:26 -0700 (PDT) Received: from fedora.mshome.net (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id y21-20020a05620a44d500b006b8f4ade2c9sm14493164qkp.19.2022.10.11.14.20.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:20:26 -0700 (PDT) From: Gregory Price X-Google-Original-From: Gregory Price To: jonathan.cameron@huawei.com Cc: qemu-devel@nongnu.org, linux-cxl@vger.kernel.org, alison.schofield@intel.com, dave@stgolabs.net, a.manzanares@samsung.com, bwidawsk@kernel.org, gregory.price@memverge.com, mst@redhat.com, hchkuo@avery-design.com.tw, cbrowy@avery-design.com, ira.weiny@intel.com Subject: [PATCH 2/5] hw/cxl: Add CXL_CAPACITY_MULTIPLIER definition Date: Tue, 11 Oct 2022 17:19:13 -0400 Message-Id: <20221011211916.117552-3-gregory.price@memverge.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011211916.117552-1-gregory.price@memverge.com> References: <20221007152156.24883-1-Jonathan.Cameron@huawei.com> <20221011211916.117552-1-gregory.price@memverge.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Remove usage of magic numbers when accessing capacity fields and replace with CXL_CAPACITY_MULTIPLIER, matching the kernel definition. Signed-off-by: Gregory Price Reviewed-by: Davidlohr Bueso --- hw/cxl/cxl-mailbox-utils.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index c7e1a88b44..776c8cbadc 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -14,6 +14,8 @@ #include "qemu/log.h" #include "qemu/uuid.h" +#define CXL_CAPACITY_MULTIPLIER 0x10000000 /* SZ_256M */ + /* * How to add a new command, example. The command set FOO, with cmd BAR. * 1. Add the command set and cmd to the enum. @@ -140,7 +142,7 @@ static ret_code cmd_firmware_update_get_info(struct cxl_cmd *cmd, } QEMU_PACKED *fw_info; QEMU_BUILD_BUG_ON(sizeof(*fw_info) != 0x50); - if (cxl_dstate->pmem_size < (256 << 20)) { + if (cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) { return CXL_MBOX_INTERNAL_ERROR; } @@ -285,7 +287,7 @@ static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd, CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d); uint64_t size = cxl_dstate->pmem_size; - if (!QEMU_IS_ALIGNED(size, 256 << 20)) { + if (!QEMU_IS_ALIGNED(size, CXL_CAPACITY_MULTIPLIER)) { return CXL_MBOX_INTERNAL_ERROR; } @@ -295,8 +297,8 @@ static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd, /* PMEM only */ snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0); - id->total_capacity = size / (256 << 20); - id->persistent_capacity = size / (256 << 20); + id->total_capacity = size / CXL_CAPACITY_MULTIPLIER; + id->persistent_capacity = size / CXL_CAPACITY_MULTIPLIER; id->lsa_size = cvc->get_lsa_size(ct3d); id->poison_list_max_mer[1] = 0x1; /* 256 poison records */ @@ -317,14 +319,14 @@ static ret_code cmd_ccls_get_partition_info(struct cxl_cmd *cmd, QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20); uint64_t size = cxl_dstate->pmem_size; - if (!QEMU_IS_ALIGNED(size, 256 << 20)) { + if (!QEMU_IS_ALIGNED(size, CXL_CAPACITY_MULTIPLIER)) { return CXL_MBOX_INTERNAL_ERROR; } /* PMEM only */ part_info->active_vmem = 0; part_info->next_vmem = 0; - part_info->active_pmem = size / (256 << 20); + part_info->active_pmem = size / CXL_CAPACITY_MULTIPLIER; part_info->next_pmem = 0; *len = sizeof(*part_info); From patchwork Tue Oct 11 21:19:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13004394 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78383C4332F for ; Tue, 11 Oct 2022 21:20:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229617AbiJKVUc (ORCPT ); Tue, 11 Oct 2022 17:20:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229520AbiJKVUb (ORCPT ); Tue, 11 Oct 2022 17:20:31 -0400 Received: from mail-qk1-x744.google.com (mail-qk1-x744.google.com [IPv6:2607:f8b0:4864:20::744]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7BA94F3AB for ; Tue, 11 Oct 2022 14:20:29 -0700 (PDT) Received: by mail-qk1-x744.google.com with SMTP id o22so8964320qkl.8 for ; Tue, 11 Oct 2022 14:20:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FEsY8vod6zC9rVPrYYS0T9zyb/vlYbm5XlYZdRBYkDo=; b=h0iddW6AEYNxaJP5LAfQrgN08/tnQWYtABmOjtfpnF/jimzdVYK/kKQl4M8HsGt7U8 TOLw164ivuxr7ComAtxciN9B6+om1pJ+ZghvxngYfgDPcQlyhyBALc7GwNKeYZgSAIOM 7ZFN5QuJSDB3XTAebZC5NMZXrWhf5qF6qUcng+MylSEet8/Q7eLv5SxVIg8FWmv55DV8 UDr0zo0wK5jPeWpMyDLqS5Bv5n0rv40VFW+nw7Ga5uaybog8nSWVcVYzDDvgERzw+WLJ OF6SqNdkBWuxjjej8Pe0HSXEcM2d+Y9Uuk0WBid8+UHlgrOascuf2uaCXuc/au7mRBEt NE+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FEsY8vod6zC9rVPrYYS0T9zyb/vlYbm5XlYZdRBYkDo=; b=gdzYHHW5uDmggi+iDKV6dFjlxHwEqPXSTqA4eaCfNrdDT24rXmSIrpzLL9yRAVM/RR DXWwULzxgf9EoGcqdJ+J0TyU98+249Ki7ZSWXrg/f444hvuksGLYonwluONfN9eyjvA4 CYkv5s7mx3pkAaPf6vhnHiq9hs5pxe5iFKElg8uTOeE8MGP8ZEPKdx2XAumFNqFLmy1m ffTxXzOIzK3bJMFWEd85/spsWi1cjHcEJDOAY1zn/W6fj0FqTpTMW2EFpcYbzWD97OZT OPLHfVmMl5KGokledLz8kR8R9LNgdwPmCsPde+Vz7j6UCM80aw/gIToW0TSVIybzAezT dn4g== X-Gm-Message-State: ACrzQf1eGWA6Ujqhv2bB0xoNLvaXFsEyPuu2MSK1x/1XD8A0+EcJlWI2 g14LAAPiBTutbGIIQsZrRQ== X-Google-Smtp-Source: AMsMyM4GJTv0qKRb4+WrRd4T4Ry+IBGN2HKAGGqClGeeFPX1ffh50YZBkJC7ACLFRb3/qQUABmtFkw== X-Received: by 2002:a05:620a:e:b0:6ee:86e5:66f8 with SMTP id j14-20020a05620a000e00b006ee86e566f8mr2895888qki.163.1665523229084; Tue, 11 Oct 2022 14:20:29 -0700 (PDT) Received: from fedora.mshome.net (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id y21-20020a05620a44d500b006b8f4ade2c9sm14493164qkp.19.2022.10.11.14.20.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:20:28 -0700 (PDT) From: Gregory Price X-Google-Original-From: Gregory Price To: jonathan.cameron@huawei.com Cc: qemu-devel@nongnu.org, linux-cxl@vger.kernel.org, alison.schofield@intel.com, dave@stgolabs.net, a.manzanares@samsung.com, bwidawsk@kernel.org, gregory.price@memverge.com, mst@redhat.com, hchkuo@avery-design.com.tw, cbrowy@avery-design.com, ira.weiny@intel.com Subject: [PATCH 3/5] hw/mem/cxl_type: Generalize CDATDsmas initialization for Memory Regions Date: Tue, 11 Oct 2022 17:19:14 -0400 Message-Id: <20221011211916.117552-4-gregory.price@memverge.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011211916.117552-1-gregory.price@memverge.com> References: <20221007152156.24883-1-Jonathan.Cameron@huawei.com> <20221011211916.117552-1-gregory.price@memverge.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org This is a preparatory commit for enabling multiple memory regions within a single CXL Type-3 device. We will need to initialize multiple CDAT DSMAS regions (and subsequent DSLBIS, and DSEMTS entries), so generalize the intialization into a function. Signed-off-by: Gregory Price --- hw/mem/cxl_type3.c | 275 +++++++++++++++++++++++++-------------------- 1 file changed, 154 insertions(+), 121 deletions(-) diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index 282f274266..dda78704c2 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -24,145 +24,178 @@ #define UI64_NULL ~(0ULL) #define DWORD_BYTE 4 +static int ct3_build_dsmas(CDATDsmas *dsmas, + CDATDslbis *dslbis, + CDATDsemts *dsemts, + MemoryRegion *mr, + int dsmad_handle, + bool is_pmem, + uint64_t dpa_base) +{ + int len = 0; + /* ttl_len should be incremented for every entry */ + + /* Device Scoped Memory Affinity Structure */ + *dsmas = (CDATDsmas) { + .header = { + .type = CDAT_TYPE_DSMAS, + .length = sizeof(*dsmas), + }, + .DSMADhandle = dsmad_handle, + .flags = (is_pmem ? CDAT_DSMAS_FLAG_NV : 0), + .DPA_base = dpa_base, + .DPA_length = int128_get64(mr->size), + }; + len++; + + /* For now, no memory side cache, plausiblish numbers */ + dslbis[0] = (CDATDslbis) { + .header = { + .type = CDAT_TYPE_DSLBIS, + .length = sizeof(*dslbis), + }, + .handle = dsmad_handle, + .flags = HMAT_LB_MEM_MEMORY, + .data_type = HMAT_LB_DATA_READ_LATENCY, + .entry_base_unit = 10000, /* 10ns base */ + .entry[0] = 15, /* 150ns */ + }; + len++; + + dslbis[1] = (CDATDslbis) { + .header = { + .type = CDAT_TYPE_DSLBIS, + .length = sizeof(*dslbis), + }, + .handle = dsmad_handle, + .flags = HMAT_LB_MEM_MEMORY, + .data_type = HMAT_LB_DATA_WRITE_LATENCY, + .entry_base_unit = 10000, + .entry[0] = 25, /* 250ns */ + }; + len++; + + dslbis[2] = (CDATDslbis) { + .header = { + .type = CDAT_TYPE_DSLBIS, + .length = sizeof(*dslbis), + }, + .handle = dsmad_handle, + .flags = HMAT_LB_MEM_MEMORY, + .data_type = HMAT_LB_DATA_READ_BANDWIDTH, + .entry_base_unit = 1000, /* GB/s */ + .entry[0] = 16, + }; + len++; + + dslbis[3] = (CDATDslbis) { + .header = { + .type = CDAT_TYPE_DSLBIS, + .length = sizeof(*dslbis), + }, + .handle = dsmad_handle, + .flags = HMAT_LB_MEM_MEMORY, + .data_type = HMAT_LB_DATA_WRITE_BANDWIDTH, + .entry_base_unit = 1000, /* GB/s */ + .entry[0] = 16, + }; + len++; + + *dsemts = (CDATDsemts) { + .header = { + .type = CDAT_TYPE_DSEMTS, + .length = sizeof(*dsemts), + }, + .DSMAS_handle = dsmad_handle, + /* EFI_MEMORY_NV implies EfiReservedMemoryType */ + .EFI_memory_type_attr = is_pmem ? 2 : 0, + /* Reserved - the non volatile from DSMAS matters */ + .DPA_offset = 0, + .DPA_length = int128_get64(mr->size), + }; + len++; + return len; +} + static int ct3_build_cdat_table(CDATSubHeader ***cdat_table, void *priv) { - g_autofree CDATDsmas *dsmas_nonvolatile = NULL; - g_autofree CDATDslbis *dslbis_nonvolatile = NULL; - g_autofree CDATDsemts *dsemts_nonvolatile = NULL; + g_autofree CDATDsmas *dsmas = NULL; + g_autofree CDATDslbis *dslbis = NULL; + g_autofree CDATDsemts *dsemts = NULL; CXLType3Dev *ct3d = priv; - int len = 0; - int i = 0; - int next_dsmad_handle = 0; - int nonvolatile_dsmad = -1; - int dslbis_nonvolatile_num = 4; + int cdat_len = 0; + int cdat_idx = 0, sub_idx = 0; + int dsmas_num, dslbis_num, dsemts_num; + int dsmad_handle = 0; + uint64_t dpa_base = 0; MemoryRegion *mr; - /* Non volatile aspects */ - if (ct3d->hostmem) { - dsmas_nonvolatile = g_malloc(sizeof(*dsmas_nonvolatile)); - if (!dsmas_nonvolatile) { - return -ENOMEM; - } - nonvolatile_dsmad = next_dsmad_handle++; - mr = host_memory_backend_get_memory(ct3d->hostmem); - if (!mr) { - return -EINVAL; - } - *dsmas_nonvolatile = (CDATDsmas) { - .header = { - .type = CDAT_TYPE_DSMAS, - .length = sizeof(*dsmas_nonvolatile), - }, - .DSMADhandle = nonvolatile_dsmad, - .flags = CDAT_DSMAS_FLAG_NV, - .DPA_base = 0, - .DPA_length = int128_get64(mr->size), - }; - len++; - - /* For now, no memory side cache, plausiblish numbers */ - dslbis_nonvolatile = g_malloc(sizeof(*dslbis_nonvolatile) * dslbis_nonvolatile_num); - if (!dslbis_nonvolatile) - return -ENOMEM; - - dslbis_nonvolatile[0] = (CDATDslbis) { - .header = { - .type = CDAT_TYPE_DSLBIS, - .length = sizeof(*dslbis_nonvolatile), - }, - .handle = nonvolatile_dsmad, - .flags = HMAT_LB_MEM_MEMORY, - .data_type = HMAT_LB_DATA_READ_LATENCY, - .entry_base_unit = 10000, /* 10ns base */ - .entry[0] = 15, /* 150ns */ - }; - len++; - - dslbis_nonvolatile[1] = (CDATDslbis) { - .header = { - .type = CDAT_TYPE_DSLBIS, - .length = sizeof(*dslbis_nonvolatile), - }, - .handle = nonvolatile_dsmad, - .flags = HMAT_LB_MEM_MEMORY, - .data_type = HMAT_LB_DATA_WRITE_LATENCY, - .entry_base_unit = 10000, - .entry[0] = 25, /* 250ns */ - }; - len++; - - dslbis_nonvolatile[2] = (CDATDslbis) { - .header = { - .type = CDAT_TYPE_DSLBIS, - .length = sizeof(*dslbis_nonvolatile), - }, - .handle = nonvolatile_dsmad, - .flags = HMAT_LB_MEM_MEMORY, - .data_type = HMAT_LB_DATA_READ_BANDWIDTH, - .entry_base_unit = 1000, /* GB/s */ - .entry[0] = 16, - }; - len++; - - dslbis_nonvolatile[3] = (CDATDslbis) { - .header = { - .type = CDAT_TYPE_DSLBIS, - .length = sizeof(*dslbis_nonvolatile), - }, - .handle = nonvolatile_dsmad, - .flags = HMAT_LB_MEM_MEMORY, - .data_type = HMAT_LB_DATA_WRITE_BANDWIDTH, - .entry_base_unit = 1000, /* GB/s */ - .entry[0] = 16, - }; - len++; - - mr = host_memory_backend_get_memory(ct3d->hostmem); - if (!mr) { - return -EINVAL; - } - dsemts_nonvolatile = g_malloc(sizeof(*dsemts_nonvolatile)); - *dsemts_nonvolatile = (CDATDsemts) { - .header = { - .type = CDAT_TYPE_DSEMTS, - .length = sizeof(*dsemts_nonvolatile), - }, - .DSMAS_handle = nonvolatile_dsmad, - .EFI_memory_type_attr = 2, /* Reserved - the non volatile from DSMAS matters */ - .DPA_offset = 0, - .DPA_length = int128_get64(mr->size), - }; - len++; + if (!ct3d->hostmem | !host_memory_backend_get_memory(ct3d->hostmem)) { + return -EINVAL; + } + + dsmas_num = 1; + dslbis_num = 4 * dsmas_num; + dsemts_num = dsmas_num; + + dsmas = g_malloc(sizeof(*dsmas) * dsmas_num); + dslbis = g_malloc(sizeof(*dslbis) * dslbis_num); + dsemts = g_malloc(sizeof(*dsemts) * dsemts_num); + + if (!dsmas || !dslbis || !dsemts) { + return -ENOMEM; + } + + mr = host_memory_backend_get_memory(ct3d->hostmem); + cdat_len += ct3_build_dsmas(&dsmas[dsmad_handle], + &dslbis[4 * dsmad_handle], + &dsemts[dsmad_handle], + mr, + dsmad_handle, + false, + dpa_base); + dpa_base += mr->size; + dsmad_handle++; + + /* Allocate and fill in the CDAT table */ + *cdat_table = g_malloc0(cdat_len * sizeof(*cdat_table)); + if (!*cdat_table) { + return -ENOMEM; } - *cdat_table = g_malloc0(len * sizeof(*cdat_table)); /* Header always at start of structure */ - if (dsmas_nonvolatile) { - (*cdat_table)[i++] = g_steal_pointer(&dsmas_nonvolatile); + CDATDsmas *dsmas_ent = g_steal_pointer(&dsmas); + for (sub_idx = 0; sub_idx < dsmas_num; sub_idx++) { + (*cdat_table)[cdat_idx++] = (CDATSubHeader*)&dsmas_ent[sub_idx]; } - if (dslbis_nonvolatile) { - CDATDslbis *dslbis = g_steal_pointer(&dslbis_nonvolatile); - int j; - for (j = 0; j < dslbis_nonvolatile_num; j++) { - (*cdat_table)[i++] = (CDATSubHeader *)&dslbis[j]; - } + CDATDslbis *dslbis_ent = g_steal_pointer(&dslbis); + for (sub_idx = 0; sub_idx < dslbis_num; sub_idx++) { + (*cdat_table)[cdat_idx++] = (CDATSubHeader*)&dslbis_ent[sub_idx]; } - if (dsemts_nonvolatile) { - (*cdat_table)[i++] = g_steal_pointer(&dsemts_nonvolatile); + + CDATDsemts *dsemts_ent = g_steal_pointer(&dsemts); + for (sub_idx = 0; sub_idx < dsemts_num; sub_idx++) { + (*cdat_table)[cdat_idx++] = (CDATSubHeader*)&dsemts_ent[sub_idx]; } - - return len; + + return cdat_len; } static void ct3_free_cdat_table(CDATSubHeader **cdat_table, int num, void *priv) { - int i; + int dsmas_num = 1; + int dslbis_idx = dsmas_num; + int dsemts_idx = dsmas_num + (dsmas_num * 4); + + /* There are only 3 sub-tables to free: dsmas, dslbis, dsemts */ + assert(num == (dsmas_num + (dsmas_num * 4) + (dsmas_num))); + + g_free(cdat_table[0]); + g_free(cdat_table[dslbis_idx]); + g_free(cdat_table[dsemts_idx]); - for (i = 0; i < num; i++) { - g_free(cdat_table[i]); - } g_free(cdat_table); } From patchwork Tue Oct 11 21:19:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13004395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C373C43217 for ; Tue, 11 Oct 2022 21:20:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229462AbiJKVUe (ORCPT ); Tue, 11 Oct 2022 17:20:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229520AbiJKVUd (ORCPT ); Tue, 11 Oct 2022 17:20:33 -0400 Received: from mail-qk1-x741.google.com (mail-qk1-x741.google.com [IPv6:2607:f8b0:4864:20::741]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2398558CB for ; Tue, 11 Oct 2022 14:20:31 -0700 (PDT) Received: by mail-qk1-x741.google.com with SMTP id o22so8964367qkl.8 for ; Tue, 11 Oct 2022 14:20:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HLt8Ha8SRNDCr78HCvP6MyBvZdCHkmHdkfxlYMSesP4=; b=QJHcL19tRcMTItrnje33Kg+UOabwQRfpUhgDgSu9BIf1LRMk3QhzCcQdHVtrFcbgS4 Rc12CARydhAGqGjc8t/HcDX+1MVXIujYURAxYWYuHktg8+TkUD096z+XHlTpv9Uthu9z D9iOpO6P00qbI20v/QCt/rb6Qd7pzcNXjN9VF/PlK+Ucw+aIRpRNn1VZ69WcfDO8k/YI kiNp30mvQgKrZXILnuyPPczERm4fX9Br9AAg0j91iAXIe7uBl0MsIFGPbAlFlLDygQ9h 16nptakKWKiQDf3Za0m2U22nLKs3JJN7jKO+lOnk+e24V+Q7WK4fAiKnXgLP//1dFVTp Hcnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HLt8Ha8SRNDCr78HCvP6MyBvZdCHkmHdkfxlYMSesP4=; b=YZXs7H4p7DSxz/sVbHJzXM+monTs9V+NBy+PtwHVnIWwAOIJ3xA9BDB9BHfDKBbbrf 7VmhEXdQBOIxivf/TbsF6OGlpzDbBhOmmsAD9DFUWkxprBWnevO4z46UXhJtozenn2OT tTJ2DkcghG1hkwl4eGcq9j/jloM4rdT0m1MzncRSww1YZfIKnLIn9xHGNPMNwD8ZBm7K PHnrtwkXPA40jyxW/VjfDtS6/FPlPhbbOkn2Ocq9ZHKsp77afQTsBMobpX+AzqCB4v5Z drUAYQY6RJ6XIHzhfLcGiOHRa5w+QhoygxcCGXphhtMouinGUP3UR9uJyNDBHIDr2FFS G8Zw== X-Gm-Message-State: ACrzQf1NoY7AeVCGquHuF0VNdq9+Mu6R6H2PAuQQlV+5a0y0fFuIxq7M aQRxIYfKOICwTjProqMFow== X-Google-Smtp-Source: AMsMyM7p+vzWevLZOg9St6xTSc/0Bd8qtN9jd8pNav9dhFGL82jmi5CNgnb4UY4CaEgdGL9CVEiwpg== X-Received: by 2002:a05:620a:318b:b0:6ce:d5d3:f914 with SMTP id bi11-20020a05620a318b00b006ced5d3f914mr17760582qkb.248.1665523230872; Tue, 11 Oct 2022 14:20:30 -0700 (PDT) Received: from fedora.mshome.net (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id y21-20020a05620a44d500b006b8f4ade2c9sm14493164qkp.19.2022.10.11.14.20.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:20:30 -0700 (PDT) From: Gregory Price X-Google-Original-From: Gregory Price To: jonathan.cameron@huawei.com Cc: qemu-devel@nongnu.org, linux-cxl@vger.kernel.org, alison.schofield@intel.com, dave@stgolabs.net, a.manzanares@samsung.com, bwidawsk@kernel.org, gregory.price@memverge.com, mst@redhat.com, hchkuo@avery-design.com.tw, cbrowy@avery-design.com, ira.weiny@intel.com Subject: [PATCH 4/5] hw/cxl: Multi-Region CXL Type-3 Devices (Volatile and Persistent) Date: Tue, 11 Oct 2022 17:19:15 -0400 Message-Id: <20221011211916.117552-5-gregory.price@memverge.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011211916.117552-1-gregory.price@memverge.com> References: <20221007152156.24883-1-Jonathan.Cameron@huawei.com> <20221011211916.117552-1-gregory.price@memverge.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org This commit enables each CXL Type-3 device to contain one volatile memory region and one persistent region. Two new properties have been added to cxl-type3 device initialization: [volatile-memdev] and [persistent-memdev] The existing [memdev] property has been deprecated and will default the memory region to a persistent memory region (although a user may assign the region to a ram or file backed region). It cannot be used in combination with the new [persistent-memdev] property. Partitioning volatile memory from persistent memory is not yet supported. Volatile memory is mapped at DPA(0x0), while Persistent memory is mapped at DPA(vmem->size), per CXL Spec 8.2.9.8.2.0 - Get Partition Info. Signed-off-by: Gregory Price --- hw/cxl/cxl-mailbox-utils.c | 21 ++-- hw/mem/cxl_type3.c | 197 ++++++++++++++++++++++++++---------- include/hw/cxl/cxl_device.h | 11 +- 3 files changed, 162 insertions(+), 67 deletions(-) diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c index 776c8cbadc..88d33e9a37 100644 --- a/hw/cxl/cxl-mailbox-utils.c +++ b/hw/cxl/cxl-mailbox-utils.c @@ -142,7 +142,7 @@ static ret_code cmd_firmware_update_get_info(struct cxl_cmd *cmd, } QEMU_PACKED *fw_info; QEMU_BUILD_BUG_ON(sizeof(*fw_info) != 0x50); - if (cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) { + if (cxl_dstate->mem_size < CXL_CAPACITY_MULTIPLIER) { return CXL_MBOX_INTERNAL_ERROR; } @@ -285,20 +285,20 @@ static ret_code cmd_identify_memory_device(struct cxl_cmd *cmd, CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate); CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d); - uint64_t size = cxl_dstate->pmem_size; - if (!QEMU_IS_ALIGNED(size, CXL_CAPACITY_MULTIPLIER)) { + if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) || + (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) { return CXL_MBOX_INTERNAL_ERROR; } id = (void *)cmd->payload; memset(id, 0, sizeof(*id)); - /* PMEM only */ snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0); - id->total_capacity = size / CXL_CAPACITY_MULTIPLIER; - id->persistent_capacity = size / CXL_CAPACITY_MULTIPLIER; + id->total_capacity = cxl_dstate->mem_size / CXL_CAPACITY_MULTIPLIER; + id->persistent_capacity = cxl_dstate->pmem_size / CXL_CAPACITY_MULTIPLIER; + id->volatile_capacity = cxl_dstate->vmem_size / CXL_CAPACITY_MULTIPLIER; id->lsa_size = cvc->get_lsa_size(ct3d); id->poison_list_max_mer[1] = 0x1; /* 256 poison records */ @@ -317,16 +317,15 @@ static ret_code cmd_ccls_get_partition_info(struct cxl_cmd *cmd, uint64_t next_pmem; } QEMU_PACKED *part_info = (void *)cmd->payload; QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20); - uint64_t size = cxl_dstate->pmem_size; - if (!QEMU_IS_ALIGNED(size, CXL_CAPACITY_MULTIPLIER)) { + if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) || + (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) { return CXL_MBOX_INTERNAL_ERROR; } - /* PMEM only */ - part_info->active_vmem = 0; + part_info->active_vmem = cxl_dstate->vmem_size / CXL_CAPACITY_MULTIPLIER; part_info->next_vmem = 0; - part_info->active_pmem = size / CXL_CAPACITY_MULTIPLIER; + part_info->active_pmem = cxl_dstate->pmem_size / CXL_CAPACITY_MULTIPLIER; part_info->next_pmem = 0; *len = sizeof(*part_info); diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c index dda78704c2..c371cd06e1 100644 --- a/hw/mem/cxl_type3.c +++ b/hw/mem/cxl_type3.c @@ -131,11 +131,13 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat_table, uint64_t dpa_base = 0; MemoryRegion *mr; - if (!ct3d->hostmem | !host_memory_backend_get_memory(ct3d->hostmem)) { + if ((!ct3d->hostvmem && !ct3d->hostpmem) || + (ct3d->hostvmem && !host_memory_backend_get_memory(ct3d->hostvmem)) || + (ct3d->hostpmem && !host_memory_backend_get_memory(ct3d->hostpmem))) { return -EINVAL; } - dsmas_num = 1; + dsmas_num = (ct3d->hostvmem ? 1 : 0) + (ct3d->hostpmem ? 1 : 0); dslbis_num = 4 * dsmas_num; dsemts_num = dsmas_num; @@ -147,16 +149,30 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat_table, return -ENOMEM; } - mr = host_memory_backend_get_memory(ct3d->hostmem); - cdat_len += ct3_build_dsmas(&dsmas[dsmad_handle], - &dslbis[4 * dsmad_handle], - &dsemts[dsmad_handle], - mr, - dsmad_handle, - false, - dpa_base); - dpa_base += mr->size; - dsmad_handle++; + if (ct3d->hostvmem) { + mr = host_memory_backend_get_memory(ct3d->hostvmem); + cdat_len += ct3_build_dsmas(&dsmas[dsmad_handle], + &dslbis[4 * dsmad_handle], + &dsemts[dsmad_handle], + mr, + dsmad_handle, + false, + dpa_base); + dpa_base += mr->size; + dsmad_handle++; + } + if (ct3d->hostpmem) { + mr = host_memory_backend_get_memory(ct3d->hostpmem); + cdat_len += ct3_build_dsmas(&dsmas[dsmad_handle], + &dslbis[4 * dsmad_handle], + &dsemts[dsmad_handle], + mr, + dsmad_handle, + false, + dpa_base); + dpa_base += mr->size; + dsmad_handle++; + } /* Allocate and fill in the CDAT table */ *cdat_table = g_malloc0(cdat_len * sizeof(*cdat_table)); @@ -185,7 +201,8 @@ static int ct3_build_cdat_table(CDATSubHeader ***cdat_table, static void ct3_free_cdat_table(CDATSubHeader **cdat_table, int num, void *priv) { - int dsmas_num = 1; + CXLType3Dev *ct3d = priv; + int dsmas_num = (ct3d->hostvmem ? 1 : 0) + (ct3d->hostpmem ? 1 : 0); int dslbis_idx = dsmas_num; int dsemts_idx = dsmas_num + (dsmas_num * 4); @@ -386,16 +403,48 @@ static void build_dvsecs(CXLType3Dev *ct3d) CXLDVSECRegisterLocator *regloc_dvsec; uint8_t *dvsec; int i; + uint32_t range1_size_hi = 0, range1_size_lo = 0, + range1_base_hi = 0, range1_base_lo = 0, + range2_size_hi = 0, range2_size_lo = 0, + range2_base_hi = 0, range2_base_lo = 0; + + /* + * Volatile memory is mapped as (0x0) + * Persistent memory is mapped at (volatile->size) + */ + if (ct3d->hostvmem && ct3d->hostpmem) { + range1_size_hi = ct3d->hostvmem->size >> 32; + range1_size_lo = (2 << 5) | (2 << 2) | 0x3 | + (ct3d->hostvmem->size & 0xF0000000); + range1_base_hi = 0; + range1_base_lo = 0; + range2_size_hi = ct3d->hostpmem->size >> 32; + range2_size_lo = (2 << 5) | (2 << 2) | 0x3 | + (ct3d->hostpmem->size & 0xF0000000); + range2_base_hi = ct3d->hostvmem->size >> 32; + range2_base_lo = ct3d->hostvmem->size & 0xF0000000; + } else { + HostMemoryBackend* hmbe = ct3d->hostvmem ? + ct3d->hostvmem : ct3d->hostpmem; + range1_size_hi = hmbe->size >> 32; + range1_size_lo = (2 << 5) | (2 << 2) | 0x3 | + (hmbe->size & 0xF0000000); + range1_base_hi = 0; + range1_base_lo = 0; + } dvsec = (uint8_t *)&(CXLDVSECDevice){ .cap = 0x1e, .ctrl = 0x2, .status2 = 0x2, - .range1_size_hi = ct3d->hostmem->size >> 32, - .range1_size_lo = (2 << 5) | (2 << 2) | 0x3 | - (ct3d->hostmem->size & 0xF0000000), - .range1_base_hi = 0, - .range1_base_lo = 0, + .range1_size_hi = range1_size_hi, + .range1_size_lo = range1_size_lo, + .range1_base_hi = range1_base_hi, + .range1_base_lo = range1_base_lo, + .range2_size_hi = range2_size_hi, + .range2_size_lo = range2_size_lo, + .range2_base_hi = range2_base_hi, + .range2_base_lo = range2_base_lo }; cxl_component_create_dvsec(cxl_cstate, CXL2_TYPE3_DEVICE, PCIE_CXL_DEVICE_DVSEC_LENGTH, @@ -483,35 +532,57 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) MemoryRegion *mr; char *name; - if (!ct3d->hostmem) { - error_setg(errp, "memdev property must be set"); + if (!ct3d->hostmem && !ct3d->hostvmem && !ct3d->hostpmem) { + error_setg(errp, "at least one memdev property must be set"); + return false; + } else if (ct3d->hostmem && ct3d->hostpmem) { + error_setg(errp, "[memdev] cannot be used with new " + "[persistent-memdev] property"); return false; + } else if (ct3d->hostmem) { + /* Use of hostmem property implies pmem */ + ct3d->hostpmem = ct3d->hostmem; + ct3d->hostmem = NULL; } - mr = host_memory_backend_get_memory(ct3d->hostmem); - if (!mr) { - error_setg(errp, "memdev property must be set"); + if (ct3d->hostpmem && !ct3d->lsa) { + error_setg(errp, "lsa property must be set for persistent devices"); return false; } - memory_region_set_nonvolatile(mr, true); - memory_region_set_enabled(mr, true); - host_memory_backend_set_mapped(ct3d->hostmem, true); - if (ds->id) { - name = g_strdup_printf("cxl-type3-dpa-space:%s", ds->id); - } else { - name = g_strdup("cxl-type3-dpa-space"); + if (ct3d->hostvmem) + { + mr = host_memory_backend_get_memory(ct3d->hostvmem); + memory_region_set_nonvolatile(mr, false); + memory_region_set_enabled(mr, true); + host_memory_backend_set_mapped(ct3d->hostvmem, true); + if (ds->id) { + name = g_strdup_printf("cxl-type3-dpa-vmem-space:%s", ds->id); + } else { + name = g_strdup("cxl-type3-dpa-vmem-space"); + } + address_space_init(&ct3d->hostvmem_as, mr, name); + ct3d->cxl_dstate.vmem_size = mr->size; + ct3d->cxl_dstate.mem_size += mr->size; + g_free(name); } - address_space_init(&ct3d->hostmem_as, mr, name); - g_free(name); - ct3d->cxl_dstate.pmem_size = ct3d->hostmem->size; - - if (!ct3d->lsa) { - error_setg(errp, "lsa property must be set"); - return false; + if (ct3d->hostpmem) + { + mr = host_memory_backend_get_memory(ct3d->hostpmem); + memory_region_set_nonvolatile(mr, true); + memory_region_set_enabled(mr, true); + host_memory_backend_set_mapped(ct3d->hostpmem, true); + if (ds->id) { + name = g_strdup_printf("cxl-type3-dpa-pmem-space:%s", ds->id); + } else { + name = g_strdup("cxl-type3-dpa-pmem-space"); + } + address_space_init(&ct3d->hostpmem_as, mr, name); + ct3d->cxl_dstate.pmem_size = mr->size; + ct3d->cxl_dstate.mem_size += mr->size; + g_free(name); } - return true; } @@ -627,7 +698,10 @@ static void ct3_exit(PCIDevice *pci_dev) cxl_doe_cdat_release(cxl_cstate); spdm_sock_fini(ct3d->doe_spdm.socket); g_free(regs->special_ops); - address_space_destroy(&ct3d->hostmem_as); + if (ct3d->hostvmem) + address_space_destroy(&ct3d->hostvmem_as); + if (ct3d->hostpmem) + address_space_destroy(&ct3d->hostpmem_as); } /* TODO: Support multiple HDM decoders and DPA skip */ @@ -667,11 +741,15 @@ MemTxResult cxl_type3_read(PCIDevice *d, hwaddr host_addr, uint64_t *data, { CXLType3Dev *ct3d = CXL_TYPE3(d); uint64_t dpa_offset; - MemoryRegion *mr; + MemoryRegion *vmr = NULL, *pmr = NULL; + AddressSpace* as; - /* TODO support volatile region */ - mr = host_memory_backend_get_memory(ct3d->hostmem); - if (!mr) { + if (ct3d->hostvmem) + vmr = host_memory_backend_get_memory(ct3d->hostvmem); + if (ct3d->hostpmem) + pmr = host_memory_backend_get_memory(ct3d->hostpmem); + + if (!vmr && !pmr) { return MEMTX_ERROR; } @@ -679,11 +757,13 @@ MemTxResult cxl_type3_read(PCIDevice *d, hwaddr host_addr, uint64_t *data, return MEMTX_ERROR; } - if (dpa_offset > int128_get64(mr->size)) { + if (dpa_offset > int128_get64(ct3d->cxl_dstate.mem_size)) { return MEMTX_ERROR; } - return address_space_read(&ct3d->hostmem_as, dpa_offset, attrs, data, size); + as = (vmr && (dpa_offset <= int128_get64(vmr->size))) ? + &ct3d->hostvmem_as : &ct3d->hostpmem_as; + return address_space_read(as, dpa_offset, attrs, data, size); } MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data, @@ -691,10 +771,15 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data, { CXLType3Dev *ct3d = CXL_TYPE3(d); uint64_t dpa_offset; - MemoryRegion *mr; + MemoryRegion *vmr = NULL, *pmr = NULL; + AddressSpace* as; + + if (ct3d->hostvmem) + vmr = host_memory_backend_get_memory(ct3d->hostvmem); + if (ct3d->hostpmem) + pmr = host_memory_backend_get_memory(ct3d->hostpmem); - mr = host_memory_backend_get_memory(ct3d->hostmem); - if (!mr) { + if (!vmr && !pmr) { return MEMTX_OK; } @@ -702,11 +787,13 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data, return MEMTX_OK; } - if (dpa_offset > int128_get64(mr->size)) { + if (dpa_offset > int128_get64(ct3d->cxl_dstate.mem_size)) { return MEMTX_OK; } - return address_space_write(&ct3d->hostmem_as, dpa_offset, attrs, - &data, size); + + as = (vmr && (dpa_offset <= int128_get64(vmr->size))) ? + &ct3d->hostvmem_as : &ct3d->hostpmem_as; + return address_space_write(as, dpa_offset, attrs, &data, size); } static void ct3d_reset(DeviceState *dev) @@ -721,7 +808,11 @@ static void ct3d_reset(DeviceState *dev) static Property ct3_props[] = { DEFINE_PROP_LINK("memdev", CXLType3Dev, hostmem, TYPE_MEMORY_BACKEND, - HostMemoryBackend *), + HostMemoryBackend *), /* for backward compatibility */ + DEFINE_PROP_LINK("persistent-memdev", CXLType3Dev, hostpmem, + TYPE_MEMORY_BACKEND, HostMemoryBackend *), + DEFINE_PROP_LINK("volatile-memdev", CXLType3Dev, hostvmem, + TYPE_MEMORY_BACKEND, HostMemoryBackend *), DEFINE_PROP_LINK("lsa", CXLType3Dev, lsa, TYPE_MEMORY_BACKEND, HostMemoryBackend *), DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL), @@ -804,7 +895,7 @@ static void ct3_class_init(ObjectClass *oc, void *data) pc->config_read = ct3d_config_read; set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); - dc->desc = "CXL PMEM Device (Type 3)"; + dc->desc = "CXL Memory Device (Type 3)"; dc->reset = ct3d_reset; device_class_set_props(dc, ct3_props); diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h index 0f4e29345f..458853b373 100644 --- a/include/hw/cxl/cxl_device.h +++ b/include/hw/cxl/cxl_device.h @@ -141,8 +141,10 @@ typedef struct cxl_device_state { uint64_t host_set; } timestamp; - /* memory region for persistent memory, HDM */ + /* memory region size, HDM */ + uint64_t mem_size; uint64_t pmem_size; + uint64_t vmem_size; /* Move me later */ CPMUState cpmu[CXL_NUM_CPMU_INSTANCES]; @@ -270,12 +272,15 @@ struct CXLType3Dev { PCIDevice parent_obj; /* Properties */ - HostMemoryBackend *hostmem; + HostMemoryBackend *hostmem; /* deprecated */ + HostMemoryBackend *hostvmem; + HostMemoryBackend *hostpmem; HostMemoryBackend *lsa; uint64_t sn; /* State */ - AddressSpace hostmem_as; + AddressSpace hostvmem_as; + AddressSpace hostpmem_as; CXLComponentState cxl_cstate; CXLDeviceState cxl_dstate; From patchwork Tue Oct 11 21:19:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13004396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5D9AC4332F for ; Tue, 11 Oct 2022 21:20:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229520AbiJKVUg (ORCPT ); Tue, 11 Oct 2022 17:20:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229612AbiJKVUf (ORCPT ); Tue, 11 Oct 2022 17:20:35 -0400 Received: from mail-qv1-xf43.google.com (mail-qv1-xf43.google.com [IPv6:2607:f8b0:4864:20::f43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98A694F3AB for ; Tue, 11 Oct 2022 14:20:33 -0700 (PDT) Received: by mail-qv1-xf43.google.com with SMTP id i9so9792731qvu.1 for ; Tue, 11 Oct 2022 14:20:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=J70hlX25E9ElY7Y0OLPVeghzHqgWbU0QsD3KrqAJXcA=; b=XEwLdU7PToSL6K/C6Da+/gEmrPiWVBKDwNnv75ttnzW7AqBPftHO58T8hEmWan8R4N cogE6vLVYvZhiB55FgKqjWoVjFnMgrjEbK1s+U1+hN66y4x0f1DywC6P2AJOTytG/vm4 5/l2ZKrVZBTGMSABORBPoubISv7JXKjxDrZgfbjb/NzdaIwGxgQSOoHLlqDLgvK0bzXF FY2vMYfKRbo62IHK6l7tmMRnKoW4NcxFvLqbfQIYLXBveQB7T8r0Xe9Gi7hY7q6tCAQV BqyOGaEMcSA9gMTqaelogYOABK+AiXTACYzii3Pvx/3Cv3R0z9Vzay7Xb7NXCV6WGbkm pmLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J70hlX25E9ElY7Y0OLPVeghzHqgWbU0QsD3KrqAJXcA=; b=E2Dkilu/Eu33aUTUZn5P9ZD1AMXwHKP2i1Xjc7mJFn0qrQVlpYGn+GWrRWpTnU6x0h 289HDiRWzVJmOW/46W9K1xldNMCwS0mpA2sXX09PraEOj8k6pMcWYPZJME3GrACcw+78 x3gm/xTUsqC7JOUEGCKnTuHmGXSaw5bJt8SbM0zKt3MJoP6Fs3UDDlPw/yVeHSagYWEu ClQvnYzM7ZHoBqyQTHeLw1/m//o5r/rtzyh6rgtXj0ouxAGxmo1Dpmosy0iBsYmUia+j k6gBhC9IFk0DpJi8/MCuMRM5Mb4Ej7rpQg0xEZnasoxM4cfY2rS32h1S9ABvB24o5qw1 lC9Q== X-Gm-Message-State: ACrzQf16mLMO7MR6Uf22aa8YJcwyzGl7PyrizjjokcMcctTiUjKB03Y+ 96TI96zgyugmeap5gAxsKQ== X-Google-Smtp-Source: AMsMyM6e0/uUBi/TSGucpsZ08beHNefxsEt9v/RiauPWEaQTl6XMxfn/f6gsXAboSxfTB2QBWmXSjQ== X-Received: by 2002:a0c:f349:0:b0:4b1:a0f5:4a2f with SMTP id e9-20020a0cf349000000b004b1a0f54a2fmr20034188qvm.112.1665523232686; Tue, 11 Oct 2022 14:20:32 -0700 (PDT) Received: from fedora.mshome.net (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id y21-20020a05620a44d500b006b8f4ade2c9sm14493164qkp.19.2022.10.11.14.20.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Oct 2022 14:20:32 -0700 (PDT) From: Gregory Price X-Google-Original-From: Gregory Price To: jonathan.cameron@huawei.com Cc: qemu-devel@nongnu.org, linux-cxl@vger.kernel.org, alison.schofield@intel.com, dave@stgolabs.net, a.manzanares@samsung.com, bwidawsk@kernel.org, gregory.price@memverge.com, mst@redhat.com, hchkuo@avery-design.com.tw, cbrowy@avery-design.com, ira.weiny@intel.com Subject: [PATCH 5/5] cxl: update tests and documentation for new cxl properties Date: Tue, 11 Oct 2022 17:19:16 -0400 Message-Id: <20221011211916.117552-6-gregory.price@memverge.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20221011211916.117552-1-gregory.price@memverge.com> References: <20221007152156.24883-1-Jonathan.Cameron@huawei.com> <20221011211916.117552-1-gregory.price@memverge.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Adds explicit examples for --persistent-memdev and --volatile-memdev Signed-off-by: Gregory Price --- docs/system/devices/cxl.rst | 53 ++++++++++++++++++------ tests/qtest/cxl-test.c | 81 +++++++++++++++++++++++++++++++------ 2 files changed, 110 insertions(+), 24 deletions(-) diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst index f25783a4ec..9e165064c8 100644 --- a/docs/system/devices/cxl.rst +++ b/docs/system/devices/cxl.rst @@ -300,15 +300,36 @@ Example topology involving a switch:: Example command lines --------------------- -A very simple setup with just one directly attached CXL Type 3 device:: +A very simple setup with just one directly attached CXL Type 3 Persistent Memory device:: qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ ... - -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \ - -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \ + -object memory-backend-file,pmem=true,id=pmem0,share=on,mem-path=/tmp/cxltest.raw,size=256M \ + -object memory-backend-file,pmem=true,id=cxl-lsa0,share=on,mem-path=/tmp/lsa.raw,size=256M \ + -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ + -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ + -device cxl-type3,bus=root_port13,persistent-memdev=pmem0,lsa=cxl-lsa1,id=cxl-pmem0 \ + -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G + +A very simple setup with just one directly attached CXL Type 3 Volatile Memory device:: + + qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ + ... + -object memory-backend-ram,id=vmem0,share=on,size=256M \ -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ - -device cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \ + -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0 \ + -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G + +The same volatile setup may optionally include an LSA region:: + + qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ + ... + -object memory-backend-ram,id=vmem0,share=on,size=256M \ + -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa.raw,size=256M \ + -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ + -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ + -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,lsa=cxl-lsa0,id=cxl-vmem0 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G A setup suitable for 4 way interleave. Only one fixed window provided, to enable 2 way @@ -328,13 +349,13 @@ the CXL Type3 device directly attached (no switches).:: -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ -device pxb-cxl,bus_nr=222,bus=pcie.0,id=cxl.2 \ -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ - -device cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \ + -device cxl-type3,bus=root_port13,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \ -device cxl-rp,port=1,bus=cxl.1,id=root_port14,chassis=0,slot=3 \ - -device cxl-type3,bus=root_port14,memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem1 \ + -device cxl-type3,bus=root_port14,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem1 \ -device cxl-rp,port=0,bus=cxl.2,id=root_port15,chassis=0,slot=5 \ - -device cxl-type3,bus=root_port15,memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem2 \ + -device cxl-type3,bus=root_port15,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem2 \ -device cxl-rp,port=1,bus=cxl.2,id=root_port16,chassis=0,slot=6 \ - -device cxl-type3,bus=root_port16,memdev=cxl-mem4,lsa=cxl-lsa4,id=cxl-pmem3 \ + -device cxl-type3,bus=root_port16,persistent-memdev=cxl-mem4,lsa=cxl-lsa4,id=cxl-pmem3 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.targets.1=cxl.2,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=8k An example of 4 devices below a switch suitable for 1, 2 or 4 way interleave:: @@ -354,15 +375,23 @@ An example of 4 devices below a switch suitable for 1, 2 or 4 way interleave:: -device cxl-rp,port=1,bus=cxl.1,id=root_port1,chassis=0,slot=1 \ -device cxl-upstream,bus=root_port0,id=us0 \ -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \ - -device cxl-type3,bus=swport0,memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0,size=256M \ + -device cxl-type3,bus=swport0,persistent-memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0,size=256M \ -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \ - -device cxl-type3,bus=swport1,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1,size=256M \ + -device cxl-type3,bus=swport1,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1,size=256M \ -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \ - -device cxl-type3,bus=swport2,memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2,size=256M \ + -device cxl-type3,bus=swport2,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2,size=256M \ -device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \ - -device cxl-type3,bus=swport3,memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem3,size=256M \ + -device cxl-type3,bus=swport3,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem3,size=256M \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k +Deprecations +------------ + +The Type 3 device [memdev] attribute has been deprecated in favor +of the [persistent-memdev] and [volatile-memdev] attributes. [memdev] +will default to a persistent memory device for backward compatibility +and is incapable of being used in combination with [persistent-memdev]. + Kernel Configuration Options ---------------------------- diff --git a/tests/qtest/cxl-test.c b/tests/qtest/cxl-test.c index f0a8a4045d..1a7a25dc53 100644 --- a/tests/qtest/cxl-test.c +++ b/tests/qtest/cxl-test.c @@ -34,29 +34,44 @@ "-device cxl-rp,id=rp2,bus=cxl.1,chassis=0,slot=2 " \ "-device cxl-rp,id=rp3,bus=cxl.1,chassis=0,slot=3 " -#define QEMU_T3D "-object memory-backend-file,id=cxl-mem0,mem-path=%s,size=256M " \ - "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M " \ - "-device cxl-type3,bus=rp0,memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " +#define QEMU_T3D_DEPRECATED \ + "-object memory-backend-file,id=cxl-mem0,mem-path=%s,size=256M " \ + "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M " \ + "-device cxl-type3,bus=rp0,memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " + +#define QEMU_T3D_PMEM \ + "-object memory-backend-file,id=m0,mem-path=%s,size=256M " \ + "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M " \ + "-device cxl-type3,bus=rp0,persistent-memdev=cxl-m0,lsa=lsa0,id=pmem0 " + +#define QEMU_T3D_VMEM \ + "-object memory-backend-ram,id=mem0,size=256M " \ + "-device cxl-type3,bus=rp0,volatile-memdev=mem0,id=mem0 " + +#define QEMU_T3D_VMEM_LSA \ + "-object memory-backend-ram,id=mem0,size=256M " \ + "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M " \ + "-device cxl-type3,bus=rp0,volatile-memdev=mem0,lsa=lsa0,id=mem0 " #define QEMU_2T3D "-object memory-backend-file,id=cxl-mem0,mem-path=%s,size=256M " \ "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M " \ - "-device cxl-type3,bus=rp0,memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " \ + "-device cxl-type3,bus=rp0,persistent-memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " \ "-object memory-backend-file,id=cxl-mem1,mem-path=%s,size=256M " \ "-object memory-backend-file,id=lsa1,mem-path=%s,size=256M " \ - "-device cxl-type3,bus=rp1,memdev=cxl-mem1,lsa=lsa1,id=cxl-pmem1 " + "-device cxl-type3,bus=rp1,persistent-memdev=cxl-mem1,lsa=lsa1,id=cxl-pmem1 " #define QEMU_4T3D "-object memory-backend-file,id=cxl-mem0,mem-path=%s,size=256M " \ "-object memory-backend-file,id=lsa0,mem-path=%s,size=256M " \ - "-device cxl-type3,bus=rp0,memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " \ + "-device cxl-type3,bus=rp0,persistent-memdev=cxl-mem0,lsa=lsa0,id=cxl-pmem0 " \ "-object memory-backend-file,id=cxl-mem1,mem-path=%s,size=256M " \ "-object memory-backend-file,id=lsa1,mem-path=%s,size=256M " \ - "-device cxl-type3,bus=rp1,memdev=cxl-mem1,lsa=lsa1,id=cxl-pmem1 " \ + "-device cxl-type3,bus=rp1,persistent-memdev=cxl-mem1,lsa=lsa1,id=cxl-pmem1 " \ "-object memory-backend-file,id=cxl-mem2,mem-path=%s,size=256M " \ "-object memory-backend-file,id=lsa2,mem-path=%s,size=256M " \ - "-device cxl-type3,bus=rp2,memdev=cxl-mem2,lsa=lsa2,id=cxl-pmem2 " \ + "-device cxl-type3,bus=rp2,persistent-memdev=cxl-mem2,lsa=lsa2,id=cxl-pmem2 " \ "-object memory-backend-file,id=cxl-mem3,mem-path=%s,size=256M " \ "-object memory-backend-file,id=lsa3,mem-path=%s,size=256M " \ - "-device cxl-type3,bus=rp3,memdev=cxl-mem3,lsa=lsa3,id=cxl-pmem3 " + "-device cxl-type3,bus=rp3,persistent-memdev=cxl-mem3,lsa=lsa3,id=cxl-pmem3 " static void cxl_basic_hb(void) { @@ -95,14 +110,53 @@ static void cxl_2root_port(void) } #ifdef CONFIG_POSIX -static void cxl_t3d(void) +static void cxl_t3d_deprecated(void) +{ + g_autoptr(GString) cmdline = g_string_new(NULL); + g_autofree const char *tmpfs = NULL; + + tmpfs = g_dir_make_tmp("cxl-test-XXXXXX", NULL); + + g_string_printf(cmdline, QEMU_PXB_CMD QEMU_RP QEMU_T3D_DEPRECATED, + tmpfs, tmpfs); + + qtest_start(cmdline->str); + qtest_end(); +} + +static void cxl_t3d_persistent(void) +{ + g_autoptr(GString) cmdline = g_string_new(NULL); + g_autofree const char *tmpfs = NULL; + + tmpfs = g_dir_make_tmp("cxl-test-XXXXXX", NULL); + + g_string_printf(cmdline, QEMU_PXB_CMD QEMU_RP QEMU_T3D_PMEM, + tmpfs, tmpfs); + + qtest_start(cmdline->str); + qtest_end(); +} + +static void cxl_t3d_volatile(void) +{ + g_autoptr(GString) cmdline = g_string_new(NULL); + + g_string_printf(cmdline, QEMU_PXB_CMD QEMU_RP QEMU_T3D_VMEM); + + qtest_start(cmdline->str); + qtest_end(); +} + +static void cxl_t3d_volatile_lsa(void) { g_autoptr(GString) cmdline = g_string_new(NULL); g_autofree const char *tmpfs = NULL; tmpfs = g_dir_make_tmp("cxl-test-XXXXXX", NULL); - g_string_printf(cmdline, QEMU_PXB_CMD QEMU_RP QEMU_T3D, tmpfs, tmpfs); + g_string_printf(cmdline, QEMU_PXB_CMD QEMU_RP QEMU_T3D_VMEM_LSA, + tmpfs); qtest_start(cmdline->str); qtest_end(); @@ -167,7 +221,10 @@ int main(int argc, char **argv) qtest_add_func("/pci/cxl/rp", cxl_root_port); qtest_add_func("/pci/cxl/rp_x2", cxl_2root_port); #ifdef CONFIG_POSIX - qtest_add_func("/pci/cxl/type3_device", cxl_t3d); + qtest_add_func("/pci/cxl/type3_device", cxl_t3d_deprecated); + qtest_add_func("/pci/cxl/type3_device_pmem", cxl_t3d_persistent); + qtest_add_func("/pci/cxl/type3_device_vmem", cxl_t3d_volatile); + qtest_add_func("/pci/cxl/type3_device_vmem_lsa", cxl_t3d_volatile_lsa); qtest_add_func("/pci/cxl/rp_x2_type3_x2", cxl_1pxb_2rp_2t3d); qtest_add_func("/pci/cxl/pxb_x2_root_port_x4_type3_x4", cxl_2pxb_4rp_4t3d);