From patchwork Mon Feb 5 07:22:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi Min Zhao X-Patchwork-Id: 10199925 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 650D1602CA for ; Mon, 5 Feb 2018 07:24:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4EC1828666 for ; Mon, 5 Feb 2018 07:24:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 41C142866F; Mon, 5 Feb 2018 07:24:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 99F4E28666 for ; Mon, 5 Feb 2018 07:24:54 +0000 (UTC) Received: from localhost ([::1]:34558 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eib93-0000Nr-Pg for patchwork-qemu-devel@patchwork.kernel.org; Mon, 05 Feb 2018 02:24:53 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44094) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eib7d-0007Ru-Su for qemu-devel@nongnu.org; Mon, 05 Feb 2018 02:23:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eib7Z-0000i9-Ua for qemu-devel@nongnu.org; Mon, 05 Feb 2018 02:23:25 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:32950 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eib7Z-0000hO-Ns for qemu-devel@nongnu.org; Mon, 05 Feb 2018 02:23:21 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w157JW1c119768 for ; Mon, 5 Feb 2018 02:23:21 -0500 Received: from e16.ny.us.ibm.com (e16.ny.us.ibm.com [129.33.205.206]) by mx0a-001b2d01.pphosted.com with ESMTP id 2fxdbe23h3-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 05 Feb 2018 02:23:20 -0500 Received: from localhost by e16.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 5 Feb 2018 02:23:20 -0500 Received: from b01cxnp22036.gho.pok.ibm.com (9.57.198.26) by e16.ny.us.ibm.com (146.89.104.203) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 5 Feb 2018 02:23:18 -0500 Received: from b01ledav001.gho.pok.ibm.com (b01ledav001.gho.pok.ibm.com [9.57.199.106]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w157NHNZ49217662; Mon, 5 Feb 2018 07:23:17 GMT Received: from b01ledav001.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D9FA028046; Mon, 5 Feb 2018 02:22:41 -0500 (EST) Received: from zyimindembp.cn.ibm.com (unknown [9.115.192.114]) by b01ledav001.gho.pok.ibm.com (Postfix) with ESMTP id A44F12803F; Mon, 5 Feb 2018 02:22:38 -0500 (EST) From: Yi Min Zhao To: qemu-devel@nongnu.org Date: Mon, 5 Feb 2018 15:22:57 +0800 X-Mailer: git-send-email 2.14.3 (Apple Git-98) In-Reply-To: <20180205072258.5968-1-zyimin@linux.vnet.ibm.com> References: <20180205072258.5968-1-zyimin@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18020507-0024-0000-0000-0000031EB7EE X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008474; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000248; SDB=6.00985168; UDB=6.00499851; IPR=6.00764539; BA=6.00005811; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00019368; XFM=3.00000015; UTC=2018-02-05 07:23:19 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18020507-0025-0000-0000-000046E06669 Message-Id: <20180205072258.5968-3-zyimin@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-02-05_01:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1802050094 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v2 2/3] s390x/pci: fixup global refresh X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pasic@linux.vnet.ibm.com, zyimin@linux.vnet.ibm.com, cohuck@redhat.com, pmorel@linux.vnet.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, marcel@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The VFIO common code doesn't provide the possibility to modify a previous mapping entry in another way than unmapping and mapping again with new properties. To avoid -EEXIST DMA mapping error, we introduce a GHashTable to store S390IOTLBEntry instances in order to cache the mapped entries. When intercepting rpcit instruction, ignore the identical mapped entries to avoid doing map operations multiple times and do unmap and re-map operations for the case of updating the valid entries. Acked-by: Pierre Morel Signed-off-by: Yi Min Zhao --- hw/s390x/s390-pci-bus.c | 24 +++++++++++++++------- hw/s390x/s390-pci-bus.h | 1 + hw/s390x/s390-pci-inst.c | 53 ++++++++++++++++++++++++++++++++++++++++-------- 3 files changed, 63 insertions(+), 15 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index e7ef7d28d9..77a50cab36 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -487,7 +487,8 @@ static IOMMUTLBEntry s390_translate_iommu(IOMMUMemoryRegion *mr, hwaddr addr, IOMMUAccessFlags flag) { S390PCIIOMMU *iommu = container_of(mr, S390PCIIOMMU, iommu_mr); - S390IOTLBEntry entry; + S390IOTLBEntry *entry; + uint64_t iova = addr & PAGE_MASK; uint16_t error = 0; IOMMUTLBEntry ret = { .target_as = &address_space_memory, @@ -515,12 +516,17 @@ static IOMMUTLBEntry s390_translate_iommu(IOMMUMemoryRegion *mr, hwaddr addr, goto err; } - error = s390_guest_io_table_walk(iommu->g_iota, addr, &entry); - - ret.iova = entry.iova; - ret.translated_addr = entry.translated_addr; - ret.addr_mask = entry.len - 1; - ret.perm = entry.perm; + entry = g_hash_table_lookup(iommu->iotlb, &iova); + if (entry) { + ret.iova = entry->iova; + ret.translated_addr = entry->translated_addr; + ret.addr_mask = entry->len - 1; + ret.perm = entry->perm; + } else { + ret.iova = iova; + ret.addr_mask = ~PAGE_MASK; + ret.perm = IOMMU_NONE; + } if (flag != IOMMU_NONE && !(flag & ret.perm)) { error = ERR_EVENT_TPROTE; @@ -572,6 +578,8 @@ static S390PCIIOMMU *s390_pci_get_iommu(S390pciState *s, PCIBus *bus, PCI_FUNC(devfn)); memory_region_init(&iommu->mr, OBJECT(iommu), mr_name, UINT64_MAX); address_space_init(&iommu->as, &iommu->mr, as_name); + iommu->iotlb = g_hash_table_new_full(g_int64_hash, g_int64_equal, + NULL, g_free); table->iommu[PCI_SLOT(devfn)] = iommu; g_free(mr_name); @@ -661,6 +669,7 @@ void s390_pci_iommu_enable(S390PCIIOMMU *iommu) void s390_pci_iommu_disable(S390PCIIOMMU *iommu) { iommu->enabled = false; + g_hash_table_remove_all(iommu->iotlb); memory_region_del_subregion(&iommu->mr, MEMORY_REGION(&iommu->iommu_mr)); object_unparent(OBJECT(&iommu->iommu_mr)); } @@ -676,6 +685,7 @@ static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn) } table->iommu[PCI_SLOT(devfn)] = NULL; + g_hash_table_destroy(iommu->iotlb); address_space_destroy(&iommu->as); object_unparent(OBJECT(&iommu->mr)); object_unparent(OBJECT(iommu)); diff --git a/hw/s390x/s390-pci-bus.h b/hw/s390x/s390-pci-bus.h index 7ed577c806..1f7f9b5814 100644 --- a/hw/s390x/s390-pci-bus.h +++ b/hw/s390x/s390-pci-bus.h @@ -278,6 +278,7 @@ typedef struct S390PCIIOMMU { uint64_t g_iota; uint64_t pba; uint64_t pal; + GHashTable *iotlb; } S390PCIIOMMU; typedef struct S390PCIIOMMUTable { diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index 1d33a89351..997a9cc2e9 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -571,6 +571,45 @@ int pcistg_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra) return 0; } +static void s390_pci_update_iotlb(S390PCIIOMMU *iommu, S390IOTLBEntry *entry) +{ + S390IOTLBEntry *cache = g_hash_table_lookup(iommu->iotlb, &entry->iova); + IOMMUTLBEntry notify = { + .target_as = &address_space_memory, + .iova = entry->iova, + .translated_addr = entry->translated_addr, + .perm = entry->perm, + .addr_mask = ~PAGE_MASK, + }; + + if (entry->perm == IOMMU_NONE) { + if (!cache) { + return; + } + g_hash_table_remove(iommu->iotlb, &entry->iova); + } else { + if (cache) { + if (cache->perm == entry->perm && + cache->translated_addr == entry->translated_addr) { + return; + } + + notify.perm = IOMMU_NONE; + memory_region_notify_iommu(&iommu->iommu_mr, notify); + notify.perm = entry->perm; + } + + cache = g_new(S390IOTLBEntry, 1); + cache->iova = entry->iova; + cache->translated_addr = entry->translated_addr; + cache->len = PAGE_SIZE; + cache->perm = entry->perm; + g_hash_table_replace(iommu->iotlb, &cache->iova, cache); + } + + memory_region_notify_iommu(&iommu->iommu_mr, notify); +} + int rpcit_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra) { CPUS390XState *env = &cpu->env; @@ -580,7 +619,6 @@ int rpcit_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra) S390PCIIOMMU *iommu; S390IOTLBEntry entry; hwaddr start, end; - IOMMUTLBEntry notify; cpu_synchronize_state(CPU(cpu)); @@ -636,15 +674,14 @@ int rpcit_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra) if (error) { break; } - notify.target_as = &address_space_memory; - notify.iova = entry.iova; - notify.translated_addr = entry.translated_addr; - notify.addr_mask = entry.len - 1; - notify.perm = entry.perm; - memory_region_notify_iommu(&iommu->iommu_mr, notify); + start += entry.len; + while (entry.iova < start && entry.iova < end) { + s390_pci_update_iotlb(iommu, &entry); + entry.iova += PAGE_SIZE; + entry.translated_addr += PAGE_SIZE; + } } - err: if (error) { pbdev->state = ZPCI_FS_ERROR;