From patchwork Fri Sep 21 14:25:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 10610183 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFBE16CB for ; Fri, 21 Sep 2018 14:32:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE3712E194 for ; Fri, 21 Sep 2018 14:32:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B06BA2E1DE; Fri, 21 Sep 2018 14:32:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8ABA62E194 for ; Fri, 21 Sep 2018 14:32:18 +0000 (UTC) Received: from localhost ([::1]:55934 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3MTh-0007Qo-L5 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 21 Sep 2018 10:32:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36481) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3MNP-0001dt-8x for qemu-devel@nongnu.org; Fri, 21 Sep 2018 10:25:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3MNL-0001J4-7W for qemu-devel@nongnu.org; Fri, 21 Sep 2018 10:25:47 -0400 Received: from mail-bn3nam01on0080.outbound.protection.outlook.com ([104.47.33.80]:16765 helo=NAM01-BN3-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3MNL-0001IV-0D for qemu-devel@nongnu.org; Fri, 21 Sep 2018 10:25:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kf/lMWPwm2vfaRD4PDAUISDfi0rvJ6xA3TJ52l+VWWU=; b=F660if6z3EKdcREhr72XsVU0Kzl308NcGgO238aGW6HNBys1mMr3FPSG59UMAggXiZg8aNHbIt6D/WSJYzCpYvwRLAJqHoxU0rFtmytECrGxxC401X6RDVG8pqUhzCfWxJl21A7DxxsnxWD4HyOCLtTr22m8GS1EoEIxejRho9A= Received: from DM6PR12MB2682.namprd12.prod.outlook.com (20.176.116.31) by DM6PR12MB2651.namprd12.prod.outlook.com (20.176.116.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.18; Fri, 21 Sep 2018 14:25:39 +0000 Received: from DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::7c34:f2e8:a8a9:cab8]) by DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::7c34:f2e8:a8a9:cab8%2]) with mapi id 15.20.1143.017; Fri, 21 Sep 2018 14:25:39 +0000 From: "Singh, Brijesh" To: "qemu-devel@nongnu.org" Thread-Topic: [PATCH v3 5/9] x86_iommu/amd: Prepare for interrupt remap support Thread-Index: AQHUUbb47YtFhR0gLkm4ofM+z9CFzA== Date: Fri, 21 Sep 2018 14:25:39 +0000 Message-ID: <1537539920-30662-6-git-send-email-brijesh.singh@amd.com> References: <1537539920-30662-1-git-send-email-brijesh.singh@amd.com> In-Reply-To: <1537539920-30662-1-git-send-email-brijesh.singh@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DM5PR21CA0008.namprd21.prod.outlook.com (2603:10b6:3:ac::18) To DM6PR12MB2682.namprd12.prod.outlook.com (2603:10b6:5:4a::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [165.204.77.1] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DM6PR12MB2651; 20:u+azZ19o4Et6B5Wazy4PcMXdFsf4qMzE2XFLvNoGIFL9aNUpiDh4+jM3nsxRhsY3XNZvOa+moIJnKj0ltGJWV+jwwFCco/zSZM7OEMYKg5vsRzlWDiv9INLYeHaUKFOUChV9QI0eJSCNN0vBeL+aJ+Jy019UJX/VpsP+hX2s/8QJ1OrosAOSJ/Ko+DFetOE2umQ8jHiQLyK6KzWsCez+q0l4iMdIsvA9+OLxGomBlYjFHtsaX1PfjFcJM5kcTQFQ x-ms-office365-filtering-correlation-id: a0805a65-d32e-4e5c-1918-08d61fce1a5a x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534165)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DM6PR12MB2651; x-ms-traffictypediagnostic: DM6PR12MB2651: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(85827821059158)(767451399110); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(823301075)(93006095)(93001095)(3002001)(3231355)(944501410)(52105095)(10201501046)(6055026)(149027)(150027)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123560045)(201708071742011)(7699051); SRVR:DM6PR12MB2651; BCL:0; PCL:0; RULEID:; SRVR:DM6PR12MB2651; x-forefront-prvs: 0802ADD973 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(136003)(39860400002)(366004)(346002)(376002)(199004)(189003)(6512007)(81166006)(5250100002)(2501003)(14444005)(256004)(486006)(11346002)(446003)(3846002)(39060400002)(8676002)(2351001)(186003)(8936002)(6116002)(2616005)(5640700003)(2906002)(53936002)(66066001)(81156014)(25786009)(6436002)(99286004)(68736007)(105586002)(476003)(6486002)(575784001)(86362001)(6916009)(305945005)(2900100001)(5660300001)(316002)(7736002)(106356001)(26005)(478600001)(76176011)(36756003)(386003)(52116002)(6506007)(54906003)(4326008)(102836004)(14454004)(71190400001)(97736004)(71200400001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR12MB2651; H:DM6PR12MB2682.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: amd.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: imhu2xob9Zt4BvDy1a9QtHjqBLNhJXi1AqD95on4OTL9WaWqrCjr6o/TiI41iBjqVGk+Z7G8Es4LRLlFWn/Yl15mgnGU6kijyksXAxkCnawU1kLTw6EfWq+tW0M9lYArzJ4lhFtFjhg1F0J3SuQAAt570aMZ0ydnSXs91yMJn1P7tUQuMrvGLRQuU4OkYnRaRIMbFZacEPwCTy0hpcHv+6iYhjCYhTHKYrThlAtlHrJZJVkOgOIv79g7Gbvg3ox1tjdM6iHwsF3BG49dH5ss9+ATQochrL0sJPLfpWcOJh7T4LTd7cOBDAaGXCpIQjm7qabP9/FyI+HlM/C1P1FDOz/kUPFaDh2/Gb5HlVd9FSk= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: a0805a65-d32e-4e5c-1918-08d61fce1a5a X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Sep 2018 14:25:39.1590 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2651 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 104.47.33.80 Subject: [Qemu-devel] [PATCH v3 5/9] x86_iommu/amd: Prepare for interrupt remap support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Lendacky, Thomas" , "Singh, Brijesh" , Eduardo Habkost , "Michael S. Tsirkin" , Peter Xu , Paolo Bonzini , "Suthikulpanit, Suravee" , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Register the interrupt remapping callback and read/write ops for the amd-iommu-ir memory region. amd-iommu-ir is set to higher priority to ensure that this region won't be masked out by other memory regions. Signed-off-by: Brijesh Singh Cc: Peter Xu Cc: "Michael S. Tsirkin" Cc: Paolo Bonzini Cc: Richard Henderson Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Tom Lendacky Cc: Suravee Suthikulpanit --- hw/i386/amd_iommu.c | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++ hw/i386/amd_iommu.h | 14 ++++++- hw/i386/trace-events | 5 +++ 3 files changed, 123 insertions(+), 2 deletions(-) diff --git a/hw/i386/amd_iommu.c b/hw/i386/amd_iommu.c index 56839f1..dcf0bfb 100644 --- a/hw/i386/amd_iommu.c +++ b/hw/i386/amd_iommu.c @@ -26,6 +26,7 @@ #include "amd_iommu.h" #include "qapi/error.h" #include "qemu/error-report.h" +#include "hw/i386/apic_internal.h" #include "trace.h" /* used AMD-Vi MMIO registers */ @@ -1028,6 +1029,99 @@ static IOMMUTLBEntry amdvi_translate(IOMMUMemoryRegion *iommu, hwaddr addr, return ret; } +/* Interrupt remapping for MSI/MSI-X entry */ +static int amdvi_int_remap_msi(AMDVIState *iommu, + MSIMessage *origin, + MSIMessage *translated, + uint16_t sid) +{ + assert(origin && translated); + + trace_amdvi_ir_remap_msi_req(origin->address, origin->data, sid); + + if (!iommu || !X86_IOMMU_DEVICE(iommu)->intr_supported) { + memcpy(translated, origin, sizeof(*origin)); + goto out; + } + + if (origin->address & AMDVI_MSI_ADDR_HI_MASK) { + trace_amdvi_err("MSI address high 32 bits non-zero when " + "Interrupt Remapping enabled."); + return -AMDVI_IR_ERR; + } + + if ((origin->address & AMDVI_MSI_ADDR_LO_MASK) != APIC_DEFAULT_ADDRESS) { + trace_amdvi_err("MSI is not from IOAPIC."); + return -AMDVI_IR_ERR; + } + +out: + trace_amdvi_ir_remap_msi(origin->address, origin->data, + translated->address, translated->data); + return 0; +} + +static int amdvi_int_remap(X86IOMMUState *iommu, + MSIMessage *origin, + MSIMessage *translated, + uint16_t sid) +{ + return amdvi_int_remap_msi(AMD_IOMMU_DEVICE(iommu), origin, + translated, sid); +} + +static MemTxResult amdvi_mem_ir_write(void *opaque, hwaddr addr, + uint64_t value, unsigned size, + MemTxAttrs attrs) +{ + int ret; + MSIMessage from = { 0, 0 }, to = { 0, 0 }; + uint16_t sid = AMDVI_IOAPIC_SB_DEVID; + + from.address = (uint64_t) addr + AMDVI_INT_ADDR_FIRST; + from.data = (uint32_t) value; + + trace_amdvi_mem_ir_write_req(addr, value, size); + + if (!attrs.unspecified) { + /* We have explicit Source ID */ + sid = attrs.requester_id; + } + + ret = amdvi_int_remap_msi(opaque, &from, &to, sid); + if (ret < 0) { + /* TODO: log the event using IOMMU log event interface */ + error_report("failed to remap interrupt from devid 0x%x", sid); + return MEMTX_ERROR; + } + + apic_get_class()->send_msi(&to); + + trace_amdvi_mem_ir_write(to.address, to.data); + return MEMTX_OK; +} + +static MemTxResult amdvi_mem_ir_read(void *opaque, hwaddr addr, + uint64_t *data, unsigned size, + MemTxAttrs attrs) +{ + return MEMTX_OK; +} + +static const MemoryRegionOps amdvi_ir_ops = { + .read_with_attrs = amdvi_mem_ir_read, + .write_with_attrs = amdvi_mem_ir_write, + .endianness = DEVICE_LITTLE_ENDIAN, + .impl = { + .min_access_size = 4, + .max_access_size = 4, + }, + .valid = { + .min_access_size = 4, + .max_access_size = 4, + } +}; + static AddressSpace *amdvi_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) { char name[128]; @@ -1063,6 +1157,7 @@ static AddressSpace *amdvi_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) * |-----------------+-------------------+----------+ * | amdvi_root | 00000000-ffffffff | 0 | * | amdvi_iommu | 00000000-ffffffff | 1 | + * | amdvi_iommu_ir | fee00000-feefffff | 64 | * |-----------------+-------------------+----------| */ memory_region_init_iommu(&amdvi_dev_as->iommu, @@ -1073,6 +1168,13 @@ static AddressSpace *amdvi_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) memory_region_init(&amdvi_dev_as->root, OBJECT(s), "amdvi_root", UINT64_MAX); address_space_init(&amdvi_dev_as->as, &amdvi_dev_as->root, name); + memory_region_init_io(&amdvi_dev_as->iommu_ir, OBJECT(s), + &amdvi_ir_ops, s, "amd-iommu-ir", + AMDVI_INT_ADDR_SIZE); + memory_region_add_subregion_overlap(&amdvi_dev_as->root, + AMDVI_INT_ADDR_FIRST, + &amdvi_dev_as->iommu_ir, + 64); memory_region_add_subregion_overlap(&amdvi_dev_as->root, 0, MEMORY_REGION(&amdvi_dev_as->iommu), 1); @@ -1193,6 +1295,9 @@ static void amdvi_realize(DeviceState *dev, Error **err) return; } + /* Pseudo address space under root PCI bus. */ + pcms->ioapic_as = amdvi_host_dma_iommu(bus, s, AMDVI_IOAPIC_SB_DEVID); + /* set up MMIO */ memory_region_init_io(&s->mmio, OBJECT(s), &mmio_mem_ops, s, "amdvi-mmio", AMDVI_MMIO_SIZE); @@ -1226,6 +1331,7 @@ static void amdvi_class_init(ObjectClass *klass, void* data) dc->vmsd = &vmstate_amdvi; dc->hotpluggable = false; dc_class->realize = amdvi_realize; + dc_class->int_remap = amdvi_int_remap; /* Supported by the pc-q35-* machine types */ dc->user_creatable = true; } diff --git a/hw/i386/amd_iommu.h b/hw/i386/amd_iommu.h index 8740305..4e7cc27 100644 --- a/hw/i386/amd_iommu.h +++ b/hw/i386/amd_iommu.h @@ -206,8 +206,18 @@ #define AMDVI_COMMAND_SIZE 16 -#define AMDVI_INT_ADDR_FIRST 0xfee00000 -#define AMDVI_INT_ADDR_LAST 0xfeefffff +#define AMDVI_INT_ADDR_FIRST 0xfee00000 +#define AMDVI_INT_ADDR_LAST 0xfeefffff +#define AMDVI_INT_ADDR_SIZE (AMDVI_INT_ADDR_LAST - AMDVI_INT_ADDR_FIRST + 1) +#define AMDVI_MSI_ADDR_HI_MASK (0xffffffff00000000ULL) +#define AMDVI_MSI_ADDR_LO_MASK (0x00000000ffffffffULL) + +/* SB IOAPIC is always on this device in AMD systems */ +#define AMDVI_IOAPIC_SB_DEVID PCI_BUILD_BDF(0, PCI_DEVFN(0x14, 0)) + +/* Interrupt remapping errors */ +#define AMDVI_IR_ERR 0x1 + #define TYPE_AMD_IOMMU_DEVICE "amd-iommu" #define AMD_IOMMU_DEVICE(obj)\ diff --git a/hw/i386/trace-events b/hw/i386/trace-events index 9e6fc4d..41d533c 100644 --- a/hw/i386/trace-events +++ b/hw/i386/trace-events @@ -101,6 +101,11 @@ amdvi_mode_invalid(uint8_t level, uint64_t addr)"error: translation level 0x%"PR amdvi_page_fault(uint64_t addr) "error: page fault accessing guest physical address 0x%"PRIx64 amdvi_iotlb_hit(uint8_t bus, uint8_t slot, uint8_t func, uint64_t addr, uint64_t txaddr) "hit iotlb devid %02x:%02x.%x gpa 0x%"PRIx64" hpa 0x%"PRIx64 amdvi_translation_result(uint8_t bus, uint8_t slot, uint8_t func, uint64_t addr, uint64_t txaddr) "devid: %02x:%02x.%x gpa 0x%"PRIx64" hpa 0x%"PRIx64 +amdvi_mem_ir_write_req(uint64_t addr, uint64_t val, uint32_t size) "addr 0x%"PRIx64" data 0x%"PRIx64" size 0x%"PRIx32 +amdvi_mem_ir_write(uint64_t addr, uint64_t val) "addr 0x%"PRIx64" data 0x%"PRIx64 +amdvi_ir_remap_msi_req(uint64_t addr, uint64_t data, uint8_t devid) "addr 0x%"PRIx64" data 0x%"PRIx64" devid 0x%"PRIx8 +amdvi_ir_remap_msi(uint64_t addr, uint64_t data, uint64_t addr2, uint64_t data2) "(addr 0x%"PRIx64", data 0x%"PRIx64") -> (addr 0x%"PRIx64", data 0x%"PRIx64")" +amdvi_err(const char *str) "%s" # hw/i386/vmport.c vmport_register(unsigned char command, void *func, void *opaque) "command: 0x%02x func: %p opaque: %p"