From patchwork Fri Oct 2 20:06:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814593 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D3E9139F for ; Fri, 2 Oct 2020 20:10:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ED56A206E3 for ; Fri, 2 Oct 2020 20:10:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="r5/u56vL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED56A206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:56128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORO6-0003Z5-1i for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:10:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44988) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKK-0006rm-VZ; Fri, 02 Oct 2020 16:06:49 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:61836) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKE-0003oY-FM; Fri, 02 Oct 2020 16:06:48 -0400 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K2IgL045958; Fri, 2 Oct 2020 16:06:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=6ZW/41ftrmBxNwLC6nTaLXjFv/PX4Uzy6FoZPgJfKHM=; b=r5/u56vL99nbO4nlM6wFd31SkxYBj/SIaWgy09D7zwJ+QQVrx8E/eL4/gjK8hhvZfYRE nCAUH6TnJ4hmSW9SYEIoRCmMsqH6MNqGnqG05kqSxRV2DxmBHmo4dI2YJf7gfXLKJKaV sR8POSqiMTRQ6uXktERB1TyvzJ85QCv1jEgsZD74vEc/SG3rMWIhwaseRVX409fqS8pJ 2Atf6R1wHLlEZyQddxR2fqMMXk+xwbh39LvhHv83KoeXwJhUyrYsHbx0EhE5NS514kS5 sE8HW1MeWsTScWW1K/A58nmig+gz+oJgaueyRd/hPDJ22XOD/pQyfUbbbbR52oIyUkxr zA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xaks8hc0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:40 -0400 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K305K048436; Fri, 2 Oct 2020 16:06:39 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xaks8hb7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:39 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092JmDHq007928; Fri, 2 Oct 2020 20:06:38 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma01wdc.us.ibm.com with ESMTP id 33sw9a0996-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:38 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6UC72753254 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:30 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D81B9BE051; Fri, 2 Oct 2020 20:06:36 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 69AC7BE04F; Fri, 2 Oct 2020 20:06:35 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:35 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 1/9] s390x/pci: Move header files to include/hw/s390x Date: Fri, 2 Oct 2020 16:06:23 -0400 Message-Id: <1601669191-6731-2-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 spamscore=0 clxscore=1015 adultscore=0 priorityscore=1501 mlxscore=0 suspectscore=2 impostorscore=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020142 Received-SPF: pass client-ip=148.163.156.1; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:40 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Seems a more appropriate location for them. Signed-off-by: Matthew Rosato --- hw/s390x/s390-pci-bus.c | 4 +- hw/s390x/s390-pci-bus.h | 372 --------------------------------------- hw/s390x/s390-pci-inst.c | 4 +- hw/s390x/s390-pci-inst.h | 312 -------------------------------- hw/s390x/s390-virtio-ccw.c | 2 +- include/hw/s390x/s390-pci-bus.h | 372 +++++++++++++++++++++++++++++++++++++++ include/hw/s390x/s390-pci-inst.h | 312 ++++++++++++++++++++++++++++++++ 7 files changed, 689 insertions(+), 689 deletions(-) delete mode 100644 hw/s390x/s390-pci-bus.h delete mode 100644 hw/s390x/s390-pci-inst.h create mode 100644 include/hw/s390x/s390-pci-bus.h create mode 100644 include/hw/s390x/s390-pci-inst.h diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index fb4cee8..a929340 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -15,8 +15,8 @@ #include "qapi/error.h" #include "qapi/visitor.h" #include "cpu.h" -#include "s390-pci-bus.h" -#include "s390-pci-inst.h" +#include "hw/s390x/s390-pci-bus.h" +#include "hw/s390x/s390-pci-inst.h" #include "hw/pci/pci_bus.h" #include "hw/qdev-properties.h" #include "hw/pci/pci_bridge.h" diff --git a/hw/s390x/s390-pci-bus.h b/hw/s390x/s390-pci-bus.h deleted file mode 100644 index 97464d0..0000000 --- a/hw/s390x/s390-pci-bus.h +++ /dev/null @@ -1,372 +0,0 @@ -/* - * s390 PCI BUS definitions - * - * Copyright 2014 IBM Corp. - * Author(s): Frank Blaschka - * Hong Bo Li - * Yi Min Zhao - * - * This work is licensed under the terms of the GNU GPL, version 2 or (at - * your option) any later version. See the COPYING file in the top-level - * directory. - */ - -#ifndef HW_S390_PCI_BUS_H -#define HW_S390_PCI_BUS_H - -#include "hw/pci/pci.h" -#include "hw/pci/pci_host.h" -#include "hw/s390x/sclp.h" -#include "hw/s390x/s390_flic.h" -#include "hw/s390x/css.h" -#include "qom/object.h" - -#define TYPE_S390_PCI_HOST_BRIDGE "s390-pcihost" -#define TYPE_S390_PCI_BUS "s390-pcibus" -#define TYPE_S390_PCI_DEVICE "zpci" -#define TYPE_S390_PCI_IOMMU "s390-pci-iommu" -#define TYPE_S390_IOMMU_MEMORY_REGION "s390-iommu-memory-region" -#define FH_MASK_ENABLE 0x80000000 -#define FH_MASK_INSTANCE 0x7f000000 -#define FH_MASK_SHM 0x00ff0000 -#define FH_MASK_INDEX 0x0000ffff -#define FH_SHM_VFIO 0x00010000 -#define FH_SHM_EMUL 0x00020000 -#define ZPCI_MAX_FID 0xffffffff -#define ZPCI_MAX_UID 0xffff -#define UID_UNDEFINED 0 -#define UID_CHECKING_ENABLED 0x01 - -OBJECT_DECLARE_SIMPLE_TYPE(S390pciState, S390_PCI_HOST_BRIDGE) -OBJECT_DECLARE_SIMPLE_TYPE(S390PCIBus, S390_PCI_BUS) -OBJECT_DECLARE_SIMPLE_TYPE(S390PCIBusDevice, S390_PCI_DEVICE) -OBJECT_DECLARE_SIMPLE_TYPE(S390PCIIOMMU, S390_PCI_IOMMU) - -#define HP_EVENT_TO_CONFIGURED 0x0301 -#define HP_EVENT_RESERVED_TO_STANDBY 0x0302 -#define HP_EVENT_DECONFIGURE_REQUEST 0x0303 -#define HP_EVENT_CONFIGURED_TO_STBRES 0x0304 -#define HP_EVENT_STANDBY_TO_RESERVED 0x0308 - -#define ERR_EVENT_INVALAS 0x1 -#define ERR_EVENT_OORANGE 0x2 -#define ERR_EVENT_INVALTF 0x3 -#define ERR_EVENT_TPROTE 0x4 -#define ERR_EVENT_APROTE 0x5 -#define ERR_EVENT_KEYE 0x6 -#define ERR_EVENT_INVALTE 0x7 -#define ERR_EVENT_INVALTL 0x8 -#define ERR_EVENT_TT 0x9 -#define ERR_EVENT_INVALMS 0xa -#define ERR_EVENT_SERR 0xb -#define ERR_EVENT_NOMSI 0x10 -#define ERR_EVENT_INVALBV 0x11 -#define ERR_EVENT_AIBV 0x12 -#define ERR_EVENT_AIRERR 0x13 -#define ERR_EVENT_FMBA 0x2a -#define ERR_EVENT_FMBUP 0x2b -#define ERR_EVENT_FMBPRO 0x2c -#define ERR_EVENT_CCONF 0x30 -#define ERR_EVENT_SERVAC 0x3a -#define ERR_EVENT_PERMERR 0x3b - -#define ERR_EVENT_Q_BIT 0x2 -#define ERR_EVENT_MVN_OFFSET 16 - -#define ZPCI_MSI_VEC_BITS 11 -#define ZPCI_MSI_VEC_MASK 0x7ff - -#define ZPCI_MSI_ADDR 0xfe00000000000000ULL -#define ZPCI_SDMA_ADDR 0x100000000ULL -#define ZPCI_EDMA_ADDR 0x1ffffffffffffffULL - -#define PAGE_SHIFT 12 -#define PAGE_SIZE (1 << PAGE_SHIFT) -#define PAGE_MASK (~(PAGE_SIZE-1)) -#define PAGE_DEFAULT_ACC 0 -#define PAGE_DEFAULT_KEY (PAGE_DEFAULT_ACC << 4) - -/* I/O Translation Anchor (IOTA) */ -enum ZpciIoatDtype { - ZPCI_IOTA_STO = 0, - ZPCI_IOTA_RTTO = 1, - ZPCI_IOTA_RSTO = 2, - ZPCI_IOTA_RFTO = 3, - ZPCI_IOTA_PFAA = 4, - ZPCI_IOTA_IOPFAA = 5, - ZPCI_IOTA_IOPTO = 7 -}; - -#define ZPCI_IOTA_IOT_ENABLED 0x800ULL -#define ZPCI_IOTA_DT_ST (ZPCI_IOTA_STO << 2) -#define ZPCI_IOTA_DT_RT (ZPCI_IOTA_RTTO << 2) -#define ZPCI_IOTA_DT_RS (ZPCI_IOTA_RSTO << 2) -#define ZPCI_IOTA_DT_RF (ZPCI_IOTA_RFTO << 2) -#define ZPCI_IOTA_DT_PF (ZPCI_IOTA_PFAA << 2) -#define ZPCI_IOTA_FS_4K 0 -#define ZPCI_IOTA_FS_1M 1 -#define ZPCI_IOTA_FS_2G 2 -#define ZPCI_KEY (PAGE_DEFAULT_KEY << 5) - -#define ZPCI_IOTA_STO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_ST) -#define ZPCI_IOTA_RTTO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_RT) -#define ZPCI_IOTA_RSTO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_RS) -#define ZPCI_IOTA_RFTO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_RF) -#define ZPCI_IOTA_RFAA_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY |\ - ZPCI_IOTA_DT_PF | ZPCI_IOTA_FS_2G) - -/* I/O Region and segment tables */ -#define ZPCI_INDEX_MASK 0x7ffULL - -#define ZPCI_TABLE_TYPE_MASK 0xc -#define ZPCI_TABLE_TYPE_RFX 0xc -#define ZPCI_TABLE_TYPE_RSX 0x8 -#define ZPCI_TABLE_TYPE_RTX 0x4 -#define ZPCI_TABLE_TYPE_SX 0x0 - -#define ZPCI_TABLE_LEN_RFX 0x3 -#define ZPCI_TABLE_LEN_RSX 0x3 -#define ZPCI_TABLE_LEN_RTX 0x3 - -#define ZPCI_TABLE_OFFSET_MASK 0xc0 -#define ZPCI_TABLE_SIZE 0x4000 -#define ZPCI_TABLE_ALIGN ZPCI_TABLE_SIZE -#define ZPCI_TABLE_ENTRY_SIZE (sizeof(unsigned long)) -#define ZPCI_TABLE_ENTRIES (ZPCI_TABLE_SIZE / ZPCI_TABLE_ENTRY_SIZE) - -#define ZPCI_TABLE_BITS 11 -#define ZPCI_PT_BITS 8 -#define ZPCI_ST_SHIFT (ZPCI_PT_BITS + PAGE_SHIFT) -#define ZPCI_RT_SHIFT (ZPCI_ST_SHIFT + ZPCI_TABLE_BITS) - -#define ZPCI_RTE_FLAG_MASK 0x3fffULL -#define ZPCI_RTE_ADDR_MASK (~ZPCI_RTE_FLAG_MASK) -#define ZPCI_STE_FLAG_MASK 0x7ffULL -#define ZPCI_STE_ADDR_MASK (~ZPCI_STE_FLAG_MASK) - -#define ZPCI_SFAA_MASK (~((1ULL << 20) - 1)) - -/* I/O Page tables */ -#define ZPCI_PTE_VALID_MASK 0x400 -#define ZPCI_PTE_INVALID 0x400 -#define ZPCI_PTE_VALID 0x000 -#define ZPCI_PT_SIZE 0x800 -#define ZPCI_PT_ALIGN ZPCI_PT_SIZE -#define ZPCI_PT_ENTRIES (ZPCI_PT_SIZE / ZPCI_TABLE_ENTRY_SIZE) -#define ZPCI_PT_MASK (ZPCI_PT_ENTRIES - 1) - -#define ZPCI_PTE_FLAG_MASK 0xfffULL -#define ZPCI_PTE_ADDR_MASK (~ZPCI_PTE_FLAG_MASK) - -/* Shared bits */ -#define ZPCI_TABLE_VALID 0x00 -#define ZPCI_TABLE_INVALID 0x20 -#define ZPCI_TABLE_PROTECTED 0x200 -#define ZPCI_TABLE_UNPROTECTED 0x000 -#define ZPCI_TABLE_FC 0x400 - -#define ZPCI_TABLE_VALID_MASK 0x20 -#define ZPCI_TABLE_PROT_MASK 0x200 - -#define ZPCI_ETT_RT 1 -#define ZPCI_ETT_ST 0 -#define ZPCI_ETT_PT -1 - -/* PCI Function States - * - * reserved: default; device has just been plugged or is in progress of being - * unplugged - * standby: device is present but not configured; transition from any - * configured state/to this state via sclp configure/deconfigure - * - * The following states make up the "configured" meta-state: - * disabled: device is configured but not enabled; transition between this - * state and enabled via clp enable/disable - * enbaled: device is ready for use; transition to disabled via clp disable; - * may enter an error state - * blocked: ignore all DMA and interrupts; transition back to enabled or from - * error state via mpcifc - * error: an error occurred; transition back to enabled via mpcifc - * permanent error: an unrecoverable error occurred; transition to standby via - * sclp deconfigure - */ -typedef enum { - ZPCI_FS_RESERVED, - ZPCI_FS_STANDBY, - ZPCI_FS_DISABLED, - ZPCI_FS_ENABLED, - ZPCI_FS_BLOCKED, - ZPCI_FS_ERROR, - ZPCI_FS_PERMANENT_ERROR, -} ZpciState; - -typedef struct SeiContainer { - QTAILQ_ENTRY(SeiContainer) link; - uint32_t fid; - uint32_t fh; - uint8_t cc; - uint16_t pec; - uint64_t faddr; - uint32_t e; -} SeiContainer; - -typedef struct PciCcdfErr { - uint32_t reserved1; - uint32_t fh; - uint32_t fid; - uint32_t e; - uint64_t faddr; - uint32_t reserved3; - uint16_t reserved4; - uint16_t pec; -} QEMU_PACKED PciCcdfErr; - -typedef struct PciCcdfAvail { - uint32_t reserved1; - uint32_t fh; - uint32_t fid; - uint32_t reserved2; - uint32_t reserved3; - uint32_t reserved4; - uint32_t reserved5; - uint16_t reserved6; - uint16_t pec; -} QEMU_PACKED PciCcdfAvail; - -typedef struct ChscSeiNt2Res { - uint16_t length; - uint16_t code; - uint16_t reserved1; - uint8_t reserved2; - uint8_t nt; - uint8_t flags; - uint8_t reserved3; - uint8_t reserved4; - uint8_t cc; - uint32_t reserved5[13]; - uint8_t ccdf[4016]; -} QEMU_PACKED ChscSeiNt2Res; - -typedef struct S390MsixInfo { - uint8_t table_bar; - uint8_t pba_bar; - uint16_t entries; - uint32_t table_offset; - uint32_t pba_offset; -} S390MsixInfo; - -typedef struct S390IOTLBEntry { - uint64_t iova; - uint64_t translated_addr; - uint64_t len; - uint64_t perm; -} S390IOTLBEntry; - -struct S390PCIIOMMU { - Object parent_obj; - S390PCIBusDevice *pbdev; - AddressSpace as; - MemoryRegion mr; - IOMMUMemoryRegion iommu_mr; - bool enabled; - uint64_t g_iota; - uint64_t pba; - uint64_t pal; - GHashTable *iotlb; -}; - -typedef struct S390PCIIOMMUTable { - uint64_t key; - S390PCIIOMMU *iommu[PCI_SLOT_MAX]; -} S390PCIIOMMUTable; - -/* Function Measurement Block */ -#define DEFAULT_MUI 4000 -#define UPDATE_U_BIT 0x1ULL -#define FMBK_MASK 0xfULL - -typedef struct ZpciFmbFmt0 { - uint64_t dma_rbytes; - uint64_t dma_wbytes; -} ZpciFmbFmt0; - -#define ZPCI_FMB_CNT_LD 0 -#define ZPCI_FMB_CNT_ST 1 -#define ZPCI_FMB_CNT_STB 2 -#define ZPCI_FMB_CNT_RPCIT 3 -#define ZPCI_FMB_CNT_MAX 4 - -#define ZPCI_FMB_FORMAT 0 - -typedef struct ZpciFmb { - uint32_t format; - uint32_t sample; - uint64_t last_update; - uint64_t counter[ZPCI_FMB_CNT_MAX]; - ZpciFmbFmt0 fmt0; -} ZpciFmb; -QEMU_BUILD_BUG_MSG(offsetof(ZpciFmb, fmt0) != 48, "padding in ZpciFmb"); - -struct S390PCIBusDevice { - DeviceState qdev; - PCIDevice *pdev; - ZpciState state; - char *target; - uint16_t uid; - uint32_t idx; - uint32_t fh; - uint32_t fid; - bool fid_defined; - uint64_t fmb_addr; - ZpciFmb fmb; - QEMUTimer *fmb_timer; - uint8_t isc; - uint16_t noi; - uint16_t maxstbl; - uint8_t sum; - S390MsixInfo msix; - AdapterRoutes routes; - S390PCIIOMMU *iommu; - MemoryRegion msix_notify_mr; - IndAddr *summary_ind; - IndAddr *indicator; - bool pci_unplug_request_processed; - bool unplug_requested; - QTAILQ_ENTRY(S390PCIBusDevice) link; -}; - -struct S390PCIBus { - BusState qbus; -}; - -struct S390pciState { - PCIHostState parent_obj; - uint32_t next_idx; - int bus_no; - S390PCIBus *bus; - GHashTable *iommu_table; - GHashTable *zpci_table; - QTAILQ_HEAD(, SeiContainer) pending_sei; - QTAILQ_HEAD(, S390PCIBusDevice) zpci_devs; -}; - -S390pciState *s390_get_phb(void); -int pci_chsc_sei_nt2_get_event(void *res); -int pci_chsc_sei_nt2_have_event(void); -void s390_pci_sclp_configure(SCCB *sccb); -void s390_pci_sclp_deconfigure(SCCB *sccb); -void s390_pci_iommu_enable(S390PCIIOMMU *iommu); -void s390_pci_iommu_disable(S390PCIIOMMU *iommu); -void s390_pci_generate_error_event(uint16_t pec, uint32_t fh, uint32_t fid, - uint64_t faddr, uint32_t e); -uint16_t s390_guest_io_table_walk(uint64_t g_iota, hwaddr addr, - S390IOTLBEntry *entry); -S390PCIBusDevice *s390_pci_find_dev_by_idx(S390pciState *s, uint32_t idx); -S390PCIBusDevice *s390_pci_find_dev_by_fh(S390pciState *s, uint32_t fh); -S390PCIBusDevice *s390_pci_find_dev_by_fid(S390pciState *s, uint32_t fid); -S390PCIBusDevice *s390_pci_find_dev_by_target(S390pciState *s, - const char *target); -S390PCIBusDevice *s390_pci_find_next_avail_dev(S390pciState *s, - S390PCIBusDevice *pbdev); - -#endif diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index 2f7a7d7..639b13c 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -13,12 +13,12 @@ #include "qemu/osdep.h" #include "cpu.h" -#include "s390-pci-inst.h" -#include "s390-pci-bus.h" #include "exec/memop.h" #include "exec/memory-internal.h" #include "qemu/error-report.h" #include "sysemu/hw_accel.h" +#include "hw/s390x/s390-pci-inst.h" +#include "hw/s390x/s390-pci-bus.h" #include "hw/s390x/tod.h" #ifndef DEBUG_S390PCI_INST diff --git a/hw/s390x/s390-pci-inst.h b/hw/s390x/s390-pci-inst.h deleted file mode 100644 index fa3bf8b..0000000 --- a/hw/s390x/s390-pci-inst.h +++ /dev/null @@ -1,312 +0,0 @@ -/* - * s390 PCI instruction definitions - * - * Copyright 2014 IBM Corp. - * Author(s): Frank Blaschka - * Hong Bo Li - * Yi Min Zhao - * - * This work is licensed under the terms of the GNU GPL, version 2 or (at - * your option) any later version. See the COPYING file in the top-level - * directory. - */ - -#ifndef HW_S390_PCI_INST_H -#define HW_S390_PCI_INST_H - -#include "s390-pci-bus.h" -#include "sysemu/dma.h" - -/* CLP common request & response block size */ -#define CLP_BLK_SIZE 4096 -#define PCI_BAR_COUNT 6 -#define PCI_MAX_FUNCTIONS 4096 - -typedef struct ClpReqHdr { - uint16_t len; - uint16_t cmd; -} QEMU_PACKED ClpReqHdr; - -typedef struct ClpRspHdr { - uint16_t len; - uint16_t rsp; -} QEMU_PACKED ClpRspHdr; - -/* CLP Response Codes */ -#define CLP_RC_OK 0x0010 /* Command request successfully */ -#define CLP_RC_CMD 0x0020 /* Command code not recognized */ -#define CLP_RC_PERM 0x0030 /* Command not authorized */ -#define CLP_RC_FMT 0x0040 /* Invalid command request format */ -#define CLP_RC_LEN 0x0050 /* Invalid command request length */ -#define CLP_RC_8K 0x0060 /* Command requires 8K LPCB */ -#define CLP_RC_RESNOT0 0x0070 /* Reserved field not zero */ -#define CLP_RC_NODATA 0x0080 /* No data available */ -#define CLP_RC_FC_UNKNOWN 0x0100 /* Function code not recognized */ - -/* - * Call Logical Processor - Command Codes - */ -#define CLP_LIST_PCI 0x0002 -#define CLP_QUERY_PCI_FN 0x0003 -#define CLP_QUERY_PCI_FNGRP 0x0004 -#define CLP_SET_PCI_FN 0x0005 - -/* PCI function handle list entry */ -typedef struct ClpFhListEntry { - uint16_t device_id; - uint16_t vendor_id; -#define CLP_FHLIST_MASK_CONFIG 0x80000000 - uint32_t config; - uint32_t fid; - uint32_t fh; -} QEMU_PACKED ClpFhListEntry; - -#define CLP_RC_SETPCIFN_FH 0x0101 /* Invalid PCI fn handle */ -#define CLP_RC_SETPCIFN_FHOP 0x0102 /* Fn handle not valid for op */ -#define CLP_RC_SETPCIFN_DMAAS 0x0103 /* Invalid DMA addr space */ -#define CLP_RC_SETPCIFN_RES 0x0104 /* Insufficient resources */ -#define CLP_RC_SETPCIFN_ALRDY 0x0105 /* Fn already in requested state */ -#define CLP_RC_SETPCIFN_ERR 0x0106 /* Fn in permanent error state */ -#define CLP_RC_SETPCIFN_RECPND 0x0107 /* Error recovery pending */ -#define CLP_RC_SETPCIFN_BUSY 0x0108 /* Fn busy */ -#define CLP_RC_LISTPCI_BADRT 0x010a /* Resume token not recognized */ -#define CLP_RC_QUERYPCIFG_PFGID 0x010b /* Unrecognized PFGID */ - -/* request or response block header length */ -#define LIST_PCI_HDR_LEN 32 - -/* Number of function handles fitting in response block */ -#define CLP_FH_LIST_NR_ENTRIES \ - ((CLP_BLK_SIZE - 2 * LIST_PCI_HDR_LEN) \ - / sizeof(ClpFhListEntry)) - -#define CLP_SET_ENABLE_PCI_FN 0 /* Yes, 0 enables it */ -#define CLP_SET_DISABLE_PCI_FN 1 /* Yes, 1 disables it */ - -#define CLP_UTIL_STR_LEN 64 - -#define CLP_MASK_FMT 0xf0000000 - -/* List PCI functions request */ -typedef struct ClpReqListPci { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint64_t resume_token; - uint64_t reserved2; -} QEMU_PACKED ClpReqListPci; - -/* List PCI functions response */ -typedef struct ClpRspListPci { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint64_t resume_token; - uint32_t mdd; - uint16_t max_fn; - uint8_t flags; - uint8_t entry_size; - ClpFhListEntry fh_list[CLP_FH_LIST_NR_ENTRIES]; -} QEMU_PACKED ClpRspListPci; - -/* Query PCI function request */ -typedef struct ClpReqQueryPci { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint32_t fh; /* function handle */ - uint32_t reserved2; - uint64_t reserved3; -} QEMU_PACKED ClpReqQueryPci; - -/* Query PCI function response */ -typedef struct ClpRspQueryPci { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint16_t vfn; /* virtual fn number */ -#define CLP_RSP_QPCI_MASK_UTIL 0x100 -#define CLP_RSP_QPCI_MASK_PFGID 0xff - uint16_t ug; - uint32_t fid; /* pci function id */ - uint8_t bar_size[PCI_BAR_COUNT]; - uint16_t pchid; - uint32_t bar[PCI_BAR_COUNT]; - uint64_t reserved2; - uint64_t sdma; /* start dma as */ - uint64_t edma; /* end dma as */ - uint32_t reserved3[11]; - uint32_t uid; - uint8_t util_str[CLP_UTIL_STR_LEN]; /* utility string */ -} QEMU_PACKED ClpRspQueryPci; - -/* Query PCI function group request */ -typedef struct ClpReqQueryPciGrp { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; -#define CLP_REQ_QPCIG_MASK_PFGID 0xff - uint32_t g; - uint32_t reserved2; - uint64_t reserved3; -} QEMU_PACKED ClpReqQueryPciGrp; - -/* Query PCI function group response */ -typedef struct ClpRspQueryPciGrp { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; -#define CLP_RSP_QPCIG_MASK_NOI 0xfff - uint16_t i; - uint8_t version; -#define CLP_RSP_QPCIG_MASK_FRAME 0x2 -#define CLP_RSP_QPCIG_MASK_REFRESH 0x1 - uint8_t fr; - uint16_t maxstbl; - uint16_t mui; - uint64_t reserved3; - uint64_t dasm; /* dma address space mask */ - uint64_t msia; /* MSI address */ - uint64_t reserved4; - uint64_t reserved5; -} QEMU_PACKED ClpRspQueryPciGrp; - -/* Set PCI function request */ -typedef struct ClpReqSetPci { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint32_t fh; /* function handle */ - uint16_t reserved2; - uint8_t oc; /* operation controls */ - uint8_t ndas; /* number of dma spaces */ - uint64_t reserved3; -} QEMU_PACKED ClpReqSetPci; - -/* Set PCI function response */ -typedef struct ClpRspSetPci { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint32_t fh; /* function handle */ - uint32_t reserved3; - uint64_t reserved4; -} QEMU_PACKED ClpRspSetPci; - -typedef struct ClpReqRspListPci { - ClpReqListPci request; - ClpRspListPci response; -} QEMU_PACKED ClpReqRspListPci; - -typedef struct ClpReqRspSetPci { - ClpReqSetPci request; - ClpRspSetPci response; -} QEMU_PACKED ClpReqRspSetPci; - -typedef struct ClpReqRspQueryPci { - ClpReqQueryPci request; - ClpRspQueryPci response; -} QEMU_PACKED ClpReqRspQueryPci; - -typedef struct ClpReqRspQueryPciGrp { - ClpReqQueryPciGrp request; - ClpRspQueryPciGrp response; -} QEMU_PACKED ClpReqRspQueryPciGrp; - -/* Load/Store status codes */ -#define ZPCI_PCI_ST_FUNC_NOT_ENABLED 4 -#define ZPCI_PCI_ST_FUNC_IN_ERR 8 -#define ZPCI_PCI_ST_BLOCKED 12 -#define ZPCI_PCI_ST_INSUF_RES 16 -#define ZPCI_PCI_ST_INVAL_AS 20 -#define ZPCI_PCI_ST_FUNC_ALREADY_ENABLED 24 -#define ZPCI_PCI_ST_DMA_AS_NOT_ENABLED 28 -#define ZPCI_PCI_ST_2ND_OP_IN_INV_AS 36 -#define ZPCI_PCI_ST_FUNC_NOT_AVAIL 40 -#define ZPCI_PCI_ST_ALREADY_IN_RQ_STATE 44 - -/* Load/Store return codes */ -#define ZPCI_PCI_LS_OK 0 -#define ZPCI_PCI_LS_ERR 1 -#define ZPCI_PCI_LS_BUSY 2 -#define ZPCI_PCI_LS_INVAL_HANDLE 3 - -/* Modify PCI status codes */ -#define ZPCI_MOD_ST_RES_NOT_AVAIL 4 -#define ZPCI_MOD_ST_INSUF_RES 16 -#define ZPCI_MOD_ST_SEQUENCE 24 -#define ZPCI_MOD_ST_DMAAS_INVAL 28 -#define ZPCI_MOD_ST_FRAME_INVAL 32 -#define ZPCI_MOD_ST_ERROR_RECOVER 40 - -/* Modify PCI Function Controls */ -#define ZPCI_MOD_FC_REG_INT 2 -#define ZPCI_MOD_FC_DEREG_INT 3 -#define ZPCI_MOD_FC_REG_IOAT 4 -#define ZPCI_MOD_FC_DEREG_IOAT 5 -#define ZPCI_MOD_FC_REREG_IOAT 6 -#define ZPCI_MOD_FC_RESET_ERROR 7 -#define ZPCI_MOD_FC_RESET_BLOCK 9 -#define ZPCI_MOD_FC_SET_MEASURE 10 - -/* Store PCI Function Controls status codes */ -#define ZPCI_STPCIFC_ST_PERM_ERROR 8 -#define ZPCI_STPCIFC_ST_INVAL_DMAAS 28 -#define ZPCI_STPCIFC_ST_ERROR_RECOVER 40 - -/* FIB function controls */ -#define ZPCI_FIB_FC_ENABLED 0x80 -#define ZPCI_FIB_FC_ERROR 0x40 -#define ZPCI_FIB_FC_LS_BLOCKED 0x20 -#define ZPCI_FIB_FC_DMAAS_REG 0x10 - -/* FIB function controls */ -#define ZPCI_FIB_FC_ENABLED 0x80 -#define ZPCI_FIB_FC_ERROR 0x40 -#define ZPCI_FIB_FC_LS_BLOCKED 0x20 -#define ZPCI_FIB_FC_DMAAS_REG 0x10 - -/* Function Information Block */ -typedef struct ZpciFib { - uint8_t fmt; /* format */ - uint8_t reserved1[7]; - uint8_t fc; /* function controls */ - uint8_t reserved2; - uint16_t reserved3; - uint32_t reserved4; - uint64_t pba; /* PCI base address */ - uint64_t pal; /* PCI address limit */ - uint64_t iota; /* I/O Translation Anchor */ -#define FIB_DATA_ISC(x) (((x) >> 28) & 0x7) -#define FIB_DATA_NOI(x) (((x) >> 16) & 0xfff) -#define FIB_DATA_AIBVO(x) (((x) >> 8) & 0x3f) -#define FIB_DATA_SUM(x) (((x) >> 7) & 0x1) -#define FIB_DATA_AISBO(x) ((x) & 0x3f) - uint32_t data; - uint32_t reserved5; - uint64_t aibv; /* Adapter int bit vector address */ - uint64_t aisb; /* Adapter int summary bit address */ - uint64_t fmb_addr; /* Function measurement address and key */ - uint32_t reserved6; - uint32_t gd; -} QEMU_PACKED ZpciFib; - -int pci_dereg_irqs(S390PCIBusDevice *pbdev); -void pci_dereg_ioat(S390PCIIOMMU *iommu); -int clp_service_call(S390CPU *cpu, uint8_t r2, uintptr_t ra); -int pcilg_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra); -int pcistg_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra); -int rpcit_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra); -int pcistb_service_call(S390CPU *cpu, uint8_t r1, uint8_t r3, uint64_t gaddr, - uint8_t ar, uintptr_t ra); -int mpcifc_service_call(S390CPU *cpu, uint8_t r1, uint64_t fiba, uint8_t ar, - uintptr_t ra); -int stpcifc_service_call(S390CPU *cpu, uint8_t r1, uint64_t fiba, uint8_t ar, - uintptr_t ra); -void fmb_timer_free(S390PCIBusDevice *pbdev); - -#define ZPCI_IO_BAR_MIN 0 -#define ZPCI_IO_BAR_MAX 5 -#define ZPCI_CONFIG_BAR 15 - -#endif diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 28266a3..f31ae2e 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -28,7 +28,7 @@ #include "qemu/error-report.h" #include "qemu/option.h" #include "qemu/qemu-print.h" -#include "s390-pci-bus.h" +#include "hw/s390x/s390-pci-bus.h" #include "sysemu/reset.h" #include "hw/s390x/storage-keys.h" #include "hw/s390x/storage-attributes.h" diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bus.h new file mode 100644 index 0000000..97464d0 --- /dev/null +++ b/include/hw/s390x/s390-pci-bus.h @@ -0,0 +1,372 @@ +/* + * s390 PCI BUS definitions + * + * Copyright 2014 IBM Corp. + * Author(s): Frank Blaschka + * Hong Bo Li + * Yi Min Zhao + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#ifndef HW_S390_PCI_BUS_H +#define HW_S390_PCI_BUS_H + +#include "hw/pci/pci.h" +#include "hw/pci/pci_host.h" +#include "hw/s390x/sclp.h" +#include "hw/s390x/s390_flic.h" +#include "hw/s390x/css.h" +#include "qom/object.h" + +#define TYPE_S390_PCI_HOST_BRIDGE "s390-pcihost" +#define TYPE_S390_PCI_BUS "s390-pcibus" +#define TYPE_S390_PCI_DEVICE "zpci" +#define TYPE_S390_PCI_IOMMU "s390-pci-iommu" +#define TYPE_S390_IOMMU_MEMORY_REGION "s390-iommu-memory-region" +#define FH_MASK_ENABLE 0x80000000 +#define FH_MASK_INSTANCE 0x7f000000 +#define FH_MASK_SHM 0x00ff0000 +#define FH_MASK_INDEX 0x0000ffff +#define FH_SHM_VFIO 0x00010000 +#define FH_SHM_EMUL 0x00020000 +#define ZPCI_MAX_FID 0xffffffff +#define ZPCI_MAX_UID 0xffff +#define UID_UNDEFINED 0 +#define UID_CHECKING_ENABLED 0x01 + +OBJECT_DECLARE_SIMPLE_TYPE(S390pciState, S390_PCI_HOST_BRIDGE) +OBJECT_DECLARE_SIMPLE_TYPE(S390PCIBus, S390_PCI_BUS) +OBJECT_DECLARE_SIMPLE_TYPE(S390PCIBusDevice, S390_PCI_DEVICE) +OBJECT_DECLARE_SIMPLE_TYPE(S390PCIIOMMU, S390_PCI_IOMMU) + +#define HP_EVENT_TO_CONFIGURED 0x0301 +#define HP_EVENT_RESERVED_TO_STANDBY 0x0302 +#define HP_EVENT_DECONFIGURE_REQUEST 0x0303 +#define HP_EVENT_CONFIGURED_TO_STBRES 0x0304 +#define HP_EVENT_STANDBY_TO_RESERVED 0x0308 + +#define ERR_EVENT_INVALAS 0x1 +#define ERR_EVENT_OORANGE 0x2 +#define ERR_EVENT_INVALTF 0x3 +#define ERR_EVENT_TPROTE 0x4 +#define ERR_EVENT_APROTE 0x5 +#define ERR_EVENT_KEYE 0x6 +#define ERR_EVENT_INVALTE 0x7 +#define ERR_EVENT_INVALTL 0x8 +#define ERR_EVENT_TT 0x9 +#define ERR_EVENT_INVALMS 0xa +#define ERR_EVENT_SERR 0xb +#define ERR_EVENT_NOMSI 0x10 +#define ERR_EVENT_INVALBV 0x11 +#define ERR_EVENT_AIBV 0x12 +#define ERR_EVENT_AIRERR 0x13 +#define ERR_EVENT_FMBA 0x2a +#define ERR_EVENT_FMBUP 0x2b +#define ERR_EVENT_FMBPRO 0x2c +#define ERR_EVENT_CCONF 0x30 +#define ERR_EVENT_SERVAC 0x3a +#define ERR_EVENT_PERMERR 0x3b + +#define ERR_EVENT_Q_BIT 0x2 +#define ERR_EVENT_MVN_OFFSET 16 + +#define ZPCI_MSI_VEC_BITS 11 +#define ZPCI_MSI_VEC_MASK 0x7ff + +#define ZPCI_MSI_ADDR 0xfe00000000000000ULL +#define ZPCI_SDMA_ADDR 0x100000000ULL +#define ZPCI_EDMA_ADDR 0x1ffffffffffffffULL + +#define PAGE_SHIFT 12 +#define PAGE_SIZE (1 << PAGE_SHIFT) +#define PAGE_MASK (~(PAGE_SIZE-1)) +#define PAGE_DEFAULT_ACC 0 +#define PAGE_DEFAULT_KEY (PAGE_DEFAULT_ACC << 4) + +/* I/O Translation Anchor (IOTA) */ +enum ZpciIoatDtype { + ZPCI_IOTA_STO = 0, + ZPCI_IOTA_RTTO = 1, + ZPCI_IOTA_RSTO = 2, + ZPCI_IOTA_RFTO = 3, + ZPCI_IOTA_PFAA = 4, + ZPCI_IOTA_IOPFAA = 5, + ZPCI_IOTA_IOPTO = 7 +}; + +#define ZPCI_IOTA_IOT_ENABLED 0x800ULL +#define ZPCI_IOTA_DT_ST (ZPCI_IOTA_STO << 2) +#define ZPCI_IOTA_DT_RT (ZPCI_IOTA_RTTO << 2) +#define ZPCI_IOTA_DT_RS (ZPCI_IOTA_RSTO << 2) +#define ZPCI_IOTA_DT_RF (ZPCI_IOTA_RFTO << 2) +#define ZPCI_IOTA_DT_PF (ZPCI_IOTA_PFAA << 2) +#define ZPCI_IOTA_FS_4K 0 +#define ZPCI_IOTA_FS_1M 1 +#define ZPCI_IOTA_FS_2G 2 +#define ZPCI_KEY (PAGE_DEFAULT_KEY << 5) + +#define ZPCI_IOTA_STO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_ST) +#define ZPCI_IOTA_RTTO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_RT) +#define ZPCI_IOTA_RSTO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_RS) +#define ZPCI_IOTA_RFTO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_DT_RF) +#define ZPCI_IOTA_RFAA_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY |\ + ZPCI_IOTA_DT_PF | ZPCI_IOTA_FS_2G) + +/* I/O Region and segment tables */ +#define ZPCI_INDEX_MASK 0x7ffULL + +#define ZPCI_TABLE_TYPE_MASK 0xc +#define ZPCI_TABLE_TYPE_RFX 0xc +#define ZPCI_TABLE_TYPE_RSX 0x8 +#define ZPCI_TABLE_TYPE_RTX 0x4 +#define ZPCI_TABLE_TYPE_SX 0x0 + +#define ZPCI_TABLE_LEN_RFX 0x3 +#define ZPCI_TABLE_LEN_RSX 0x3 +#define ZPCI_TABLE_LEN_RTX 0x3 + +#define ZPCI_TABLE_OFFSET_MASK 0xc0 +#define ZPCI_TABLE_SIZE 0x4000 +#define ZPCI_TABLE_ALIGN ZPCI_TABLE_SIZE +#define ZPCI_TABLE_ENTRY_SIZE (sizeof(unsigned long)) +#define ZPCI_TABLE_ENTRIES (ZPCI_TABLE_SIZE / ZPCI_TABLE_ENTRY_SIZE) + +#define ZPCI_TABLE_BITS 11 +#define ZPCI_PT_BITS 8 +#define ZPCI_ST_SHIFT (ZPCI_PT_BITS + PAGE_SHIFT) +#define ZPCI_RT_SHIFT (ZPCI_ST_SHIFT + ZPCI_TABLE_BITS) + +#define ZPCI_RTE_FLAG_MASK 0x3fffULL +#define ZPCI_RTE_ADDR_MASK (~ZPCI_RTE_FLAG_MASK) +#define ZPCI_STE_FLAG_MASK 0x7ffULL +#define ZPCI_STE_ADDR_MASK (~ZPCI_STE_FLAG_MASK) + +#define ZPCI_SFAA_MASK (~((1ULL << 20) - 1)) + +/* I/O Page tables */ +#define ZPCI_PTE_VALID_MASK 0x400 +#define ZPCI_PTE_INVALID 0x400 +#define ZPCI_PTE_VALID 0x000 +#define ZPCI_PT_SIZE 0x800 +#define ZPCI_PT_ALIGN ZPCI_PT_SIZE +#define ZPCI_PT_ENTRIES (ZPCI_PT_SIZE / ZPCI_TABLE_ENTRY_SIZE) +#define ZPCI_PT_MASK (ZPCI_PT_ENTRIES - 1) + +#define ZPCI_PTE_FLAG_MASK 0xfffULL +#define ZPCI_PTE_ADDR_MASK (~ZPCI_PTE_FLAG_MASK) + +/* Shared bits */ +#define ZPCI_TABLE_VALID 0x00 +#define ZPCI_TABLE_INVALID 0x20 +#define ZPCI_TABLE_PROTECTED 0x200 +#define ZPCI_TABLE_UNPROTECTED 0x000 +#define ZPCI_TABLE_FC 0x400 + +#define ZPCI_TABLE_VALID_MASK 0x20 +#define ZPCI_TABLE_PROT_MASK 0x200 + +#define ZPCI_ETT_RT 1 +#define ZPCI_ETT_ST 0 +#define ZPCI_ETT_PT -1 + +/* PCI Function States + * + * reserved: default; device has just been plugged or is in progress of being + * unplugged + * standby: device is present but not configured; transition from any + * configured state/to this state via sclp configure/deconfigure + * + * The following states make up the "configured" meta-state: + * disabled: device is configured but not enabled; transition between this + * state and enabled via clp enable/disable + * enbaled: device is ready for use; transition to disabled via clp disable; + * may enter an error state + * blocked: ignore all DMA and interrupts; transition back to enabled or from + * error state via mpcifc + * error: an error occurred; transition back to enabled via mpcifc + * permanent error: an unrecoverable error occurred; transition to standby via + * sclp deconfigure + */ +typedef enum { + ZPCI_FS_RESERVED, + ZPCI_FS_STANDBY, + ZPCI_FS_DISABLED, + ZPCI_FS_ENABLED, + ZPCI_FS_BLOCKED, + ZPCI_FS_ERROR, + ZPCI_FS_PERMANENT_ERROR, +} ZpciState; + +typedef struct SeiContainer { + QTAILQ_ENTRY(SeiContainer) link; + uint32_t fid; + uint32_t fh; + uint8_t cc; + uint16_t pec; + uint64_t faddr; + uint32_t e; +} SeiContainer; + +typedef struct PciCcdfErr { + uint32_t reserved1; + uint32_t fh; + uint32_t fid; + uint32_t e; + uint64_t faddr; + uint32_t reserved3; + uint16_t reserved4; + uint16_t pec; +} QEMU_PACKED PciCcdfErr; + +typedef struct PciCcdfAvail { + uint32_t reserved1; + uint32_t fh; + uint32_t fid; + uint32_t reserved2; + uint32_t reserved3; + uint32_t reserved4; + uint32_t reserved5; + uint16_t reserved6; + uint16_t pec; +} QEMU_PACKED PciCcdfAvail; + +typedef struct ChscSeiNt2Res { + uint16_t length; + uint16_t code; + uint16_t reserved1; + uint8_t reserved2; + uint8_t nt; + uint8_t flags; + uint8_t reserved3; + uint8_t reserved4; + uint8_t cc; + uint32_t reserved5[13]; + uint8_t ccdf[4016]; +} QEMU_PACKED ChscSeiNt2Res; + +typedef struct S390MsixInfo { + uint8_t table_bar; + uint8_t pba_bar; + uint16_t entries; + uint32_t table_offset; + uint32_t pba_offset; +} S390MsixInfo; + +typedef struct S390IOTLBEntry { + uint64_t iova; + uint64_t translated_addr; + uint64_t len; + uint64_t perm; +} S390IOTLBEntry; + +struct S390PCIIOMMU { + Object parent_obj; + S390PCIBusDevice *pbdev; + AddressSpace as; + MemoryRegion mr; + IOMMUMemoryRegion iommu_mr; + bool enabled; + uint64_t g_iota; + uint64_t pba; + uint64_t pal; + GHashTable *iotlb; +}; + +typedef struct S390PCIIOMMUTable { + uint64_t key; + S390PCIIOMMU *iommu[PCI_SLOT_MAX]; +} S390PCIIOMMUTable; + +/* Function Measurement Block */ +#define DEFAULT_MUI 4000 +#define UPDATE_U_BIT 0x1ULL +#define FMBK_MASK 0xfULL + +typedef struct ZpciFmbFmt0 { + uint64_t dma_rbytes; + uint64_t dma_wbytes; +} ZpciFmbFmt0; + +#define ZPCI_FMB_CNT_LD 0 +#define ZPCI_FMB_CNT_ST 1 +#define ZPCI_FMB_CNT_STB 2 +#define ZPCI_FMB_CNT_RPCIT 3 +#define ZPCI_FMB_CNT_MAX 4 + +#define ZPCI_FMB_FORMAT 0 + +typedef struct ZpciFmb { + uint32_t format; + uint32_t sample; + uint64_t last_update; + uint64_t counter[ZPCI_FMB_CNT_MAX]; + ZpciFmbFmt0 fmt0; +} ZpciFmb; +QEMU_BUILD_BUG_MSG(offsetof(ZpciFmb, fmt0) != 48, "padding in ZpciFmb"); + +struct S390PCIBusDevice { + DeviceState qdev; + PCIDevice *pdev; + ZpciState state; + char *target; + uint16_t uid; + uint32_t idx; + uint32_t fh; + uint32_t fid; + bool fid_defined; + uint64_t fmb_addr; + ZpciFmb fmb; + QEMUTimer *fmb_timer; + uint8_t isc; + uint16_t noi; + uint16_t maxstbl; + uint8_t sum; + S390MsixInfo msix; + AdapterRoutes routes; + S390PCIIOMMU *iommu; + MemoryRegion msix_notify_mr; + IndAddr *summary_ind; + IndAddr *indicator; + bool pci_unplug_request_processed; + bool unplug_requested; + QTAILQ_ENTRY(S390PCIBusDevice) link; +}; + +struct S390PCIBus { + BusState qbus; +}; + +struct S390pciState { + PCIHostState parent_obj; + uint32_t next_idx; + int bus_no; + S390PCIBus *bus; + GHashTable *iommu_table; + GHashTable *zpci_table; + QTAILQ_HEAD(, SeiContainer) pending_sei; + QTAILQ_HEAD(, S390PCIBusDevice) zpci_devs; +}; + +S390pciState *s390_get_phb(void); +int pci_chsc_sei_nt2_get_event(void *res); +int pci_chsc_sei_nt2_have_event(void); +void s390_pci_sclp_configure(SCCB *sccb); +void s390_pci_sclp_deconfigure(SCCB *sccb); +void s390_pci_iommu_enable(S390PCIIOMMU *iommu); +void s390_pci_iommu_disable(S390PCIIOMMU *iommu); +void s390_pci_generate_error_event(uint16_t pec, uint32_t fh, uint32_t fid, + uint64_t faddr, uint32_t e); +uint16_t s390_guest_io_table_walk(uint64_t g_iota, hwaddr addr, + S390IOTLBEntry *entry); +S390PCIBusDevice *s390_pci_find_dev_by_idx(S390pciState *s, uint32_t idx); +S390PCIBusDevice *s390_pci_find_dev_by_fh(S390pciState *s, uint32_t fh); +S390PCIBusDevice *s390_pci_find_dev_by_fid(S390pciState *s, uint32_t fid); +S390PCIBusDevice *s390_pci_find_dev_by_target(S390pciState *s, + const char *target); +S390PCIBusDevice *s390_pci_find_next_avail_dev(S390pciState *s, + S390PCIBusDevice *pbdev); + +#endif diff --git a/include/hw/s390x/s390-pci-inst.h b/include/hw/s390x/s390-pci-inst.h new file mode 100644 index 0000000..fa3bf8b --- /dev/null +++ b/include/hw/s390x/s390-pci-inst.h @@ -0,0 +1,312 @@ +/* + * s390 PCI instruction definitions + * + * Copyright 2014 IBM Corp. + * Author(s): Frank Blaschka + * Hong Bo Li + * Yi Min Zhao + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#ifndef HW_S390_PCI_INST_H +#define HW_S390_PCI_INST_H + +#include "s390-pci-bus.h" +#include "sysemu/dma.h" + +/* CLP common request & response block size */ +#define CLP_BLK_SIZE 4096 +#define PCI_BAR_COUNT 6 +#define PCI_MAX_FUNCTIONS 4096 + +typedef struct ClpReqHdr { + uint16_t len; + uint16_t cmd; +} QEMU_PACKED ClpReqHdr; + +typedef struct ClpRspHdr { + uint16_t len; + uint16_t rsp; +} QEMU_PACKED ClpRspHdr; + +/* CLP Response Codes */ +#define CLP_RC_OK 0x0010 /* Command request successfully */ +#define CLP_RC_CMD 0x0020 /* Command code not recognized */ +#define CLP_RC_PERM 0x0030 /* Command not authorized */ +#define CLP_RC_FMT 0x0040 /* Invalid command request format */ +#define CLP_RC_LEN 0x0050 /* Invalid command request length */ +#define CLP_RC_8K 0x0060 /* Command requires 8K LPCB */ +#define CLP_RC_RESNOT0 0x0070 /* Reserved field not zero */ +#define CLP_RC_NODATA 0x0080 /* No data available */ +#define CLP_RC_FC_UNKNOWN 0x0100 /* Function code not recognized */ + +/* + * Call Logical Processor - Command Codes + */ +#define CLP_LIST_PCI 0x0002 +#define CLP_QUERY_PCI_FN 0x0003 +#define CLP_QUERY_PCI_FNGRP 0x0004 +#define CLP_SET_PCI_FN 0x0005 + +/* PCI function handle list entry */ +typedef struct ClpFhListEntry { + uint16_t device_id; + uint16_t vendor_id; +#define CLP_FHLIST_MASK_CONFIG 0x80000000 + uint32_t config; + uint32_t fid; + uint32_t fh; +} QEMU_PACKED ClpFhListEntry; + +#define CLP_RC_SETPCIFN_FH 0x0101 /* Invalid PCI fn handle */ +#define CLP_RC_SETPCIFN_FHOP 0x0102 /* Fn handle not valid for op */ +#define CLP_RC_SETPCIFN_DMAAS 0x0103 /* Invalid DMA addr space */ +#define CLP_RC_SETPCIFN_RES 0x0104 /* Insufficient resources */ +#define CLP_RC_SETPCIFN_ALRDY 0x0105 /* Fn already in requested state */ +#define CLP_RC_SETPCIFN_ERR 0x0106 /* Fn in permanent error state */ +#define CLP_RC_SETPCIFN_RECPND 0x0107 /* Error recovery pending */ +#define CLP_RC_SETPCIFN_BUSY 0x0108 /* Fn busy */ +#define CLP_RC_LISTPCI_BADRT 0x010a /* Resume token not recognized */ +#define CLP_RC_QUERYPCIFG_PFGID 0x010b /* Unrecognized PFGID */ + +/* request or response block header length */ +#define LIST_PCI_HDR_LEN 32 + +/* Number of function handles fitting in response block */ +#define CLP_FH_LIST_NR_ENTRIES \ + ((CLP_BLK_SIZE - 2 * LIST_PCI_HDR_LEN) \ + / sizeof(ClpFhListEntry)) + +#define CLP_SET_ENABLE_PCI_FN 0 /* Yes, 0 enables it */ +#define CLP_SET_DISABLE_PCI_FN 1 /* Yes, 1 disables it */ + +#define CLP_UTIL_STR_LEN 64 + +#define CLP_MASK_FMT 0xf0000000 + +/* List PCI functions request */ +typedef struct ClpReqListPci { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint64_t resume_token; + uint64_t reserved2; +} QEMU_PACKED ClpReqListPci; + +/* List PCI functions response */ +typedef struct ClpRspListPci { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint64_t resume_token; + uint32_t mdd; + uint16_t max_fn; + uint8_t flags; + uint8_t entry_size; + ClpFhListEntry fh_list[CLP_FH_LIST_NR_ENTRIES]; +} QEMU_PACKED ClpRspListPci; + +/* Query PCI function request */ +typedef struct ClpReqQueryPci { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint32_t fh; /* function handle */ + uint32_t reserved2; + uint64_t reserved3; +} QEMU_PACKED ClpReqQueryPci; + +/* Query PCI function response */ +typedef struct ClpRspQueryPci { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint16_t vfn; /* virtual fn number */ +#define CLP_RSP_QPCI_MASK_UTIL 0x100 +#define CLP_RSP_QPCI_MASK_PFGID 0xff + uint16_t ug; + uint32_t fid; /* pci function id */ + uint8_t bar_size[PCI_BAR_COUNT]; + uint16_t pchid; + uint32_t bar[PCI_BAR_COUNT]; + uint64_t reserved2; + uint64_t sdma; /* start dma as */ + uint64_t edma; /* end dma as */ + uint32_t reserved3[11]; + uint32_t uid; + uint8_t util_str[CLP_UTIL_STR_LEN]; /* utility string */ +} QEMU_PACKED ClpRspQueryPci; + +/* Query PCI function group request */ +typedef struct ClpReqQueryPciGrp { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; +#define CLP_REQ_QPCIG_MASK_PFGID 0xff + uint32_t g; + uint32_t reserved2; + uint64_t reserved3; +} QEMU_PACKED ClpReqQueryPciGrp; + +/* Query PCI function group response */ +typedef struct ClpRspQueryPciGrp { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; +#define CLP_RSP_QPCIG_MASK_NOI 0xfff + uint16_t i; + uint8_t version; +#define CLP_RSP_QPCIG_MASK_FRAME 0x2 +#define CLP_RSP_QPCIG_MASK_REFRESH 0x1 + uint8_t fr; + uint16_t maxstbl; + uint16_t mui; + uint64_t reserved3; + uint64_t dasm; /* dma address space mask */ + uint64_t msia; /* MSI address */ + uint64_t reserved4; + uint64_t reserved5; +} QEMU_PACKED ClpRspQueryPciGrp; + +/* Set PCI function request */ +typedef struct ClpReqSetPci { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint32_t fh; /* function handle */ + uint16_t reserved2; + uint8_t oc; /* operation controls */ + uint8_t ndas; /* number of dma spaces */ + uint64_t reserved3; +} QEMU_PACKED ClpReqSetPci; + +/* Set PCI function response */ +typedef struct ClpRspSetPci { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint32_t fh; /* function handle */ + uint32_t reserved3; + uint64_t reserved4; +} QEMU_PACKED ClpRspSetPci; + +typedef struct ClpReqRspListPci { + ClpReqListPci request; + ClpRspListPci response; +} QEMU_PACKED ClpReqRspListPci; + +typedef struct ClpReqRspSetPci { + ClpReqSetPci request; + ClpRspSetPci response; +} QEMU_PACKED ClpReqRspSetPci; + +typedef struct ClpReqRspQueryPci { + ClpReqQueryPci request; + ClpRspQueryPci response; +} QEMU_PACKED ClpReqRspQueryPci; + +typedef struct ClpReqRspQueryPciGrp { + ClpReqQueryPciGrp request; + ClpRspQueryPciGrp response; +} QEMU_PACKED ClpReqRspQueryPciGrp; + +/* Load/Store status codes */ +#define ZPCI_PCI_ST_FUNC_NOT_ENABLED 4 +#define ZPCI_PCI_ST_FUNC_IN_ERR 8 +#define ZPCI_PCI_ST_BLOCKED 12 +#define ZPCI_PCI_ST_INSUF_RES 16 +#define ZPCI_PCI_ST_INVAL_AS 20 +#define ZPCI_PCI_ST_FUNC_ALREADY_ENABLED 24 +#define ZPCI_PCI_ST_DMA_AS_NOT_ENABLED 28 +#define ZPCI_PCI_ST_2ND_OP_IN_INV_AS 36 +#define ZPCI_PCI_ST_FUNC_NOT_AVAIL 40 +#define ZPCI_PCI_ST_ALREADY_IN_RQ_STATE 44 + +/* Load/Store return codes */ +#define ZPCI_PCI_LS_OK 0 +#define ZPCI_PCI_LS_ERR 1 +#define ZPCI_PCI_LS_BUSY 2 +#define ZPCI_PCI_LS_INVAL_HANDLE 3 + +/* Modify PCI status codes */ +#define ZPCI_MOD_ST_RES_NOT_AVAIL 4 +#define ZPCI_MOD_ST_INSUF_RES 16 +#define ZPCI_MOD_ST_SEQUENCE 24 +#define ZPCI_MOD_ST_DMAAS_INVAL 28 +#define ZPCI_MOD_ST_FRAME_INVAL 32 +#define ZPCI_MOD_ST_ERROR_RECOVER 40 + +/* Modify PCI Function Controls */ +#define ZPCI_MOD_FC_REG_INT 2 +#define ZPCI_MOD_FC_DEREG_INT 3 +#define ZPCI_MOD_FC_REG_IOAT 4 +#define ZPCI_MOD_FC_DEREG_IOAT 5 +#define ZPCI_MOD_FC_REREG_IOAT 6 +#define ZPCI_MOD_FC_RESET_ERROR 7 +#define ZPCI_MOD_FC_RESET_BLOCK 9 +#define ZPCI_MOD_FC_SET_MEASURE 10 + +/* Store PCI Function Controls status codes */ +#define ZPCI_STPCIFC_ST_PERM_ERROR 8 +#define ZPCI_STPCIFC_ST_INVAL_DMAAS 28 +#define ZPCI_STPCIFC_ST_ERROR_RECOVER 40 + +/* FIB function controls */ +#define ZPCI_FIB_FC_ENABLED 0x80 +#define ZPCI_FIB_FC_ERROR 0x40 +#define ZPCI_FIB_FC_LS_BLOCKED 0x20 +#define ZPCI_FIB_FC_DMAAS_REG 0x10 + +/* FIB function controls */ +#define ZPCI_FIB_FC_ENABLED 0x80 +#define ZPCI_FIB_FC_ERROR 0x40 +#define ZPCI_FIB_FC_LS_BLOCKED 0x20 +#define ZPCI_FIB_FC_DMAAS_REG 0x10 + +/* Function Information Block */ +typedef struct ZpciFib { + uint8_t fmt; /* format */ + uint8_t reserved1[7]; + uint8_t fc; /* function controls */ + uint8_t reserved2; + uint16_t reserved3; + uint32_t reserved4; + uint64_t pba; /* PCI base address */ + uint64_t pal; /* PCI address limit */ + uint64_t iota; /* I/O Translation Anchor */ +#define FIB_DATA_ISC(x) (((x) >> 28) & 0x7) +#define FIB_DATA_NOI(x) (((x) >> 16) & 0xfff) +#define FIB_DATA_AIBVO(x) (((x) >> 8) & 0x3f) +#define FIB_DATA_SUM(x) (((x) >> 7) & 0x1) +#define FIB_DATA_AISBO(x) ((x) & 0x3f) + uint32_t data; + uint32_t reserved5; + uint64_t aibv; /* Adapter int bit vector address */ + uint64_t aisb; /* Adapter int summary bit address */ + uint64_t fmb_addr; /* Function measurement address and key */ + uint32_t reserved6; + uint32_t gd; +} QEMU_PACKED ZpciFib; + +int pci_dereg_irqs(S390PCIBusDevice *pbdev); +void pci_dereg_ioat(S390PCIIOMMU *iommu); +int clp_service_call(S390CPU *cpu, uint8_t r2, uintptr_t ra); +int pcilg_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra); +int pcistg_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra); +int rpcit_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra); +int pcistb_service_call(S390CPU *cpu, uint8_t r1, uint8_t r3, uint64_t gaddr, + uint8_t ar, uintptr_t ra); +int mpcifc_service_call(S390CPU *cpu, uint8_t r1, uint64_t fiba, uint8_t ar, + uintptr_t ra); +int stpcifc_service_call(S390CPU *cpu, uint8_t r1, uint64_t fiba, uint8_t ar, + uintptr_t ra); +void fmb_timer_free(S390PCIBusDevice *pbdev); + +#define ZPCI_IO_BAR_MIN 0 +#define ZPCI_IO_BAR_MAX 5 +#define ZPCI_CONFIG_BAR 15 + +#endif From patchwork Fri Oct 2 20:06:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814583 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9089E112E for ; Fri, 2 Oct 2020 20:08:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 290A7206E3 for ; Fri, 2 Oct 2020 20:08:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="D4A4AMqZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 290A7206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:47570 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORLh-0008UT-0W for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:08:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44930) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKH-0006jy-0Z; Fri, 02 Oct 2020 16:06:45 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:61290) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKE-0003on-W8; Fri, 02 Oct 2020 16:06:44 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K1xNc008894; Fri, 2 Oct 2020 16:06:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=fDEXaxz0QKGvR3UiRZHVNYokGxZBSHMdE9mUiKZPVVM=; b=D4A4AMqZK8Yg2d5yjNdAv5cFKTV+WTsD87Krf/2dMCqPoYHw/G7SgNW8vbr0LfrQjnwH PwXTaatyLqlIuuIwg3CunYTv95hrXajCWzGCIs3YDqn9/LGjwyp+KqGy8DBL66dQamrN tOCuxVjsYYxBgv2nEFOFjEVw/Fnu2Z91t9JbYD3oFZYM65evde72KEcQ3IrWyZRKpPGt RGnbTlPTntsZQKr3bgdV1X69u7w7oclJoHWxWKW7r+DxUlSI2y/cdKofBGzL1qjEFLKX 1ArC2pgHaYSKdk6//962p0Gf1+HCIaqVk0Pq4mjPhv2HcArGUsGu+5i0vEpCLMJOJjqs ZQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 33x9uasnw1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:41 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K22IH009102; Fri, 2 Oct 2020 16:06:41 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 33x9uasnvh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:40 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092Jm8YU011491; Fri, 2 Oct 2020 20:06:39 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma03wdc.us.ibm.com with ESMTP id 33sw99raa9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:39 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6YPQ33882496 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:34 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 505D8BE051; Fri, 2 Oct 2020 20:06:38 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 07C58BE04F; Fri, 2 Oct 2020 20:06:37 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:36 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 2/9] MAINTAINERS: Update s390 PCI entry to include headers Date: Fri, 2 Oct 2020 16:06:24 -0400 Message-Id: <1601669191-6731-3-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 impostorscore=0 spamscore=0 malwarescore=0 bulkscore=0 phishscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020142 Received-SPF: pass client-ip=148.163.156.1; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:40 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Accomodate changes to file locations. Signed-off-by: Matthew Rosato --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS index b76fb31..dd4e0ea 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1442,6 +1442,7 @@ S390 PCI M: Matthew Rosato S: Supported F: hw/s390x/s390-pci* +F: include/hw/s390x/s390-pci* L: qemu-s390x@nongnu.org UniCore32 Machines From patchwork Fri Oct 2 20:06:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814587 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A390139F for ; Fri, 2 Oct 2020 20:08:17 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B83DF206E3 for ; Fri, 2 Oct 2020 20:08:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="A6pu9qlz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B83DF206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:47812 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORLj-00008t-ED for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:08:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44956) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKI-0006ml-IN; Fri, 02 Oct 2020 16:06:46 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:17028) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKG-0003pA-Io; Fri, 02 Oct 2020 16:06:46 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K51PT089210; Fri, 2 Oct 2020 16:06:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=WK8gF4N9HS62KlaG4AFwexv7peq6mKBPRGLNCAZO4F8=; b=A6pu9qlzV49WKg//Bai6rwBs6kP78l168GIhLq1ifH/KHHckrF8Y7Udz2pG1UoHimjU+ jEjMe6zDYMqMTKGMDAu5nn0D5j95084HYK9Er5fiYHfOUzCeFTYcJiR4rh23ASSoPsCy liZ1okr/+RjMcOw2N7wuRNNviofTTAYIfccR1cGnlNp+99199hp5bjVhSGSmyM6kynT/ OI5Bppq+SCagSEn3D/lYJurre2y+OqlfAAqkeGivAeQbIUH7xeyAWh+iEes9KbjLgAT4 SFcFq0KIQ1TY5eWRfp7miTdGbhlE92kqjWcUWVnw0li8MULWO1KGXKerwYj7zjsILmLp Vw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xa63gwv9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:43 -0400 Received: from m0098404.ppops.net (m0098404.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K5AG3090616; Fri, 2 Oct 2020 16:06:42 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xa63gwuq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:42 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092JmGPb007960; Fri, 2 Oct 2020 20:06:41 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma01wdc.us.ibm.com with ESMTP id 33sw9a099g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:41 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6dFl43188538 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:39 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B6E48BE051; Fri, 2 Oct 2020 20:06:39 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 734B9BE04F; Fri, 2 Oct 2020 20:06:38 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:38 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 3/9] update-linux-headers: Add vfio_zdev.h Date: Fri, 2 Oct 2020 16:06:25 -0400 Message-Id: <1601669191-6731-4-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0 adultscore=0 mlxscore=0 priorityscore=1501 suspectscore=0 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 spamscore=0 clxscore=1015 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020145 Received-SPF: pass client-ip=148.163.156.1; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:40 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" vfio_zdev.h is used by s390x zPCI support to pass device-specific CLP information between host and userspace. Signed-off-by: Matthew Rosato Acked-by: Cornelia Huck --- scripts/update-linux-headers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh index 29c27f4..9efbaf2 100755 --- a/scripts/update-linux-headers.sh +++ b/scripts/update-linux-headers.sh @@ -141,7 +141,7 @@ done rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" -for header in kvm.h vfio.h vfio_ccw.h vhost.h \ +for header in kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \ psci.h psp-sev.h userfaultfd.h mman.h; do cp "$tmpdir/include/linux/$header" "$output/linux-headers/linux" done From patchwork Fri Oct 2 20:06:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814585 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5FD24139F for ; Fri, 2 Oct 2020 20:08:16 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ECA4E206E3 for ; Fri, 2 Oct 2020 20:08:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Uph0el6i" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ECA4E206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:47782 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORLj-000085-0t for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:08:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44966) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKJ-0006p5-MT; Fri, 02 Oct 2020 16:06:47 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:1280) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKH-0003pO-Lq; Fri, 02 Oct 2020 16:06:47 -0400 Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K3LNe012048; Fri, 2 Oct 2020 16:06:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=e+SGbm7cj2gyvu7uJ/UfzF/L24O5PtO52HK/7KtT950=; b=Uph0el6icI4BrBDcEWOfuzAKeTeLA1vvfKkLdafGMOOf8BTT7aKi+nEdSx8jDeJL2UI1 RZgItUnmWSIsRoKhUdvQN6uf0JrFBevAOBsJ56CtfgH7MCmhkPf0YJDMYFfP0R/Euf4G +w5iuAUR2URwhjZhoMhfvQA77/m7swaKvun8Esp7teivOSgR7t1YFvfSuy2ScbU+JzoZ DXixFOxnTNcK9Mg8ZWU2iRaNZsX3xgdiEF2wzM/7cUGohkzwvI+Ar6FPcjYRhU5mDgEz Q3MefJ9aAcCY67QAN09EyQymlzMV8CEffa/2NR9ACQ9T0sk7osgUG8wtXSpZwZOgOPgQ Ug== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xasvr7uf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:44 -0400 Received: from m0098399.ppops.net (m0098399.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K3LAG012111; Fri, 2 Oct 2020 16:06:43 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xasvr7u0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:43 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092JlCpx011313; Fri, 2 Oct 2020 20:06:42 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma01dal.us.ibm.com with ESMTP id 33sw9aecax-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:42 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6bKR64684296 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:37 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 274FABE054; Fri, 2 Oct 2020 20:06:41 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DB90BBE04F; Fri, 2 Oct 2020 20:06:39 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:39 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 4/9] linux-headers: update against 5.9-rc7 Date: Fri, 2 Oct 2020 16:06:26 -0400 Message-Id: <1601669191-6731-5-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 lowpriorityscore=0 malwarescore=0 spamscore=0 suspectscore=0 phishscore=0 priorityscore=1501 impostorscore=0 adultscore=0 mlxlogscore=999 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020145 Received-SPF: pass client-ip=148.163.156.1; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:40 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" PLACEHOLDER as the kernel patch driving the need for this ("vfio-pci/zdev: define the vfio_zdev header") isn't merged yet. Signed-off-by: Matthew Rosato --- .../drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h | 14 +++++++------- linux-headers/linux/kvm.h | 6 ++++-- linux-headers/linux/vfio.h | 5 +++++ 3 files changed, 16 insertions(+), 9 deletions(-) diff --git a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h b/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h index 7b4062a..acd4c83 100644 --- a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h +++ b/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h @@ -68,7 +68,7 @@ static inline int pvrdma_idx_valid(uint32_t idx, uint32_t max_elems) static inline int32_t pvrdma_idx(int *var, uint32_t max_elems) { - const unsigned int idx = qatomic_read(var); + const unsigned int idx = atomic_read(var); if (pvrdma_idx_valid(idx, max_elems)) return idx & (max_elems - 1); @@ -77,17 +77,17 @@ static inline int32_t pvrdma_idx(int *var, uint32_t max_elems) static inline void pvrdma_idx_ring_inc(int *var, uint32_t max_elems) { - uint32_t idx = qatomic_read(var) + 1; /* Increment. */ + uint32_t idx = atomic_read(var) + 1; /* Increment. */ idx &= (max_elems << 1) - 1; /* Modulo size, flip gen. */ - qatomic_set(var, idx); + atomic_set(var, idx); } static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r, uint32_t max_elems, uint32_t *out_tail) { - const uint32_t tail = qatomic_read(&r->prod_tail); - const uint32_t head = qatomic_read(&r->cons_head); + const uint32_t tail = atomic_read(&r->prod_tail); + const uint32_t head = atomic_read(&r->cons_head); if (pvrdma_idx_valid(tail, max_elems) && pvrdma_idx_valid(head, max_elems)) { @@ -100,8 +100,8 @@ static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r, static inline int32_t pvrdma_idx_ring_has_data(const struct pvrdma_ring *r, uint32_t max_elems, uint32_t *out_head) { - const uint32_t tail = qatomic_read(&r->prod_tail); - const uint32_t head = qatomic_read(&r->cons_head); + const uint32_t tail = atomic_read(&r->prod_tail); + const uint32_t head = atomic_read(&r->cons_head); if (pvrdma_idx_valid(tail, max_elems) && pvrdma_idx_valid(head, max_elems)) { diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index 6683e2e..43580c7 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -790,9 +790,10 @@ struct kvm_ppc_resize_hpt { #define KVM_VM_PPC_HV 1 #define KVM_VM_PPC_PR 2 -/* on MIPS, 0 forces trap & emulate, 1 forces VZ ASE */ -#define KVM_VM_MIPS_TE 0 +/* on MIPS, 0 indicates auto, 1 forces VZ ASE, 2 forces trap & emulate */ +#define KVM_VM_MIPS_AUTO 0 #define KVM_VM_MIPS_VZ 1 +#define KVM_VM_MIPS_TE 2 #define KVM_S390_SIE_PAGE_OFFSET 1 @@ -1035,6 +1036,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_LAST_CPU 184 #define KVM_CAP_SMALLER_MAXPHYADDR 185 #define KVM_CAP_S390_DIAG318 186 +#define KVM_CAP_STEAL_TIME 187 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index a906724..68fd67a 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -326,6 +326,11 @@ struct vfio_region_info_cap_type { * to do TLB invalidation on a GPU. */ #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD (1) +/* + * IBM zPCI specific hardware feature information for a devcie. The contents + * of this region are mapped by struct vfio_region_zpci_info. + */ +#define VFIO_REGION_SUBTYPE_IBM_ZPCI_CLP (2) /* sub-types for VFIO_REGION_TYPE_GFX */ #define VFIO_REGION_SUBTYPE_GFX_EDID (1) From patchwork Fri Oct 2 20:06:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06875112E for ; Fri, 2 Oct 2020 20:10:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9FE49206E3 for ; Fri, 2 Oct 2020 20:10:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="T/ABABOm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9FE49206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:56050 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORO4-0003Wy-Og for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:10:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44990) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKL-0006ry-9U; Fri, 02 Oct 2020 16:06:49 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:11958 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKI-0003pf-OU; Fri, 02 Oct 2020 16:06:49 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K2EO0169900; Fri, 2 Oct 2020 16:06:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=Q24bkZF+WW1d7nBwracJLi6xpU3c/qO9sjaQ1kn/fA0=; b=T/ABABOmIuQaD/vV2bPTsp5qOCbZDwMF4y3ulTkm6itTTFm4o8ytDCwyck3wIbgehZyE jhrFCN20tuSxYXZSIMy699QIgsSTOMRTG4rnv9BSCdIbNC7OGj1cVQExtfI8NJSlRxNA f3VW44QlqXpOwrH/d1QKLgCOuczvRqQ6Dz9jJwF44Uzpfen6fEiSOfbC8+FeJy/IsWVD YnqVjcCTLuGuE8K64woGuB9CZNeEHBX3YEqzUzaE9qKBt1XzC7BV4o9oQ0b8/jMkmlGU YnPyEPeim9TAqXAN3dZOXDiEh2f6897Ep79/itIbgbnNewndkJIZLPT5kRByiKaSiSty Ww== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 33xa0xsbw7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:45 -0400 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K2J50170521; Fri, 2 Oct 2020 16:06:45 -0400 Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0b-001b2d01.pphosted.com with ESMTP id 33xa0xsbvq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:45 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092JmGDg007952; Fri, 2 Oct 2020 20:06:44 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma01wdc.us.ibm.com with ESMTP id 33sw9a099u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:44 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6gqr57540954 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:42 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C1F03BE051; Fri, 2 Oct 2020 20:06:42 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4A43DBE04F; Fri, 2 Oct 2020 20:06:41 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:41 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 5/9] s390x/pci: create a header dedicated to PCI CLP Date: Fri, 2 Oct 2020 16:06:27 -0400 Message-Id: <1601669191-6731-6-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 bulkscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxscore=0 clxscore=1015 phishscore=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020142 Received-SPF: pass client-ip=148.163.158.5; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:37 X-ACL-Warn: Detected OS = Linux 3.x [generic] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Pierre Morel To have a clean separation between s390-pci-bus.h and s390-pci-inst.h headers we export the PCI CLP instructions in a dedicated header. Signed-off-by: Pierre Morel Signed-off-by: Matthew Rosato Reviewed-by: Cornelia Huck --- include/hw/s390x/s390-pci-bus.h | 1 + include/hw/s390x/s390-pci-clp.h | 211 +++++++++++++++++++++++++++++++++++++++ include/hw/s390x/s390-pci-inst.h | 196 ------------------------------------ 3 files changed, 212 insertions(+), 196 deletions(-) create mode 100644 include/hw/s390x/s390-pci-clp.h diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bus.h index 97464d0..04e7cb6 100644 --- a/include/hw/s390x/s390-pci-bus.h +++ b/include/hw/s390x/s390-pci-bus.h @@ -19,6 +19,7 @@ #include "hw/s390x/sclp.h" #include "hw/s390x/s390_flic.h" #include "hw/s390x/css.h" +#include "hw/s390x/s390-pci-clp.h" #include "qom/object.h" #define TYPE_S390_PCI_HOST_BRIDGE "s390-pcihost" diff --git a/include/hw/s390x/s390-pci-clp.h b/include/hw/s390x/s390-pci-clp.h new file mode 100644 index 0000000..e442307 --- /dev/null +++ b/include/hw/s390x/s390-pci-clp.h @@ -0,0 +1,211 @@ +/* + * s390 CLPinstruction definitions + * + * Copyright 2019 IBM Corp. + * Author(s): Pierre Morel + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#ifndef HW_S390_PCI_CLP +#define HW_S390_PCI_CLP + +/* CLP common request & response block size */ +#define CLP_BLK_SIZE 4096 +#define PCI_BAR_COUNT 6 +#define PCI_MAX_FUNCTIONS 4096 + +typedef struct ClpReqHdr { + uint16_t len; + uint16_t cmd; +} QEMU_PACKED ClpReqHdr; + +typedef struct ClpRspHdr { + uint16_t len; + uint16_t rsp; +} QEMU_PACKED ClpRspHdr; + +/* CLP Response Codes */ +#define CLP_RC_OK 0x0010 /* Command request successfully */ +#define CLP_RC_CMD 0x0020 /* Command code not recognized */ +#define CLP_RC_PERM 0x0030 /* Command not authorized */ +#define CLP_RC_FMT 0x0040 /* Invalid command request format */ +#define CLP_RC_LEN 0x0050 /* Invalid command request length */ +#define CLP_RC_8K 0x0060 /* Command requires 8K LPCB */ +#define CLP_RC_RESNOT0 0x0070 /* Reserved field not zero */ +#define CLP_RC_NODATA 0x0080 /* No data available */ +#define CLP_RC_FC_UNKNOWN 0x0100 /* Function code not recognized */ + +/* + * Call Logical Processor - Command Codes + */ +#define CLP_LIST_PCI 0x0002 +#define CLP_QUERY_PCI_FN 0x0003 +#define CLP_QUERY_PCI_FNGRP 0x0004 +#define CLP_SET_PCI_FN 0x0005 + +/* PCI function handle list entry */ +typedef struct ClpFhListEntry { + uint16_t device_id; + uint16_t vendor_id; +#define CLP_FHLIST_MASK_CONFIG 0x80000000 + uint32_t config; + uint32_t fid; + uint32_t fh; +} QEMU_PACKED ClpFhListEntry; + +#define CLP_RC_SETPCIFN_FH 0x0101 /* Invalid PCI fn handle */ +#define CLP_RC_SETPCIFN_FHOP 0x0102 /* Fn handle not valid for op */ +#define CLP_RC_SETPCIFN_DMAAS 0x0103 /* Invalid DMA addr space */ +#define CLP_RC_SETPCIFN_RES 0x0104 /* Insufficient resources */ +#define CLP_RC_SETPCIFN_ALRDY 0x0105 /* Fn already in requested state */ +#define CLP_RC_SETPCIFN_ERR 0x0106 /* Fn in permanent error state */ +#define CLP_RC_SETPCIFN_RECPND 0x0107 /* Error recovery pending */ +#define CLP_RC_SETPCIFN_BUSY 0x0108 /* Fn busy */ +#define CLP_RC_LISTPCI_BADRT 0x010a /* Resume token not recognized */ +#define CLP_RC_QUERYPCIFG_PFGID 0x010b /* Unrecognized PFGID */ + +/* request or response block header length */ +#define LIST_PCI_HDR_LEN 32 + +/* Number of function handles fitting in response block */ +#define CLP_FH_LIST_NR_ENTRIES \ + ((CLP_BLK_SIZE - 2 * LIST_PCI_HDR_LEN) \ + / sizeof(ClpFhListEntry)) + +#define CLP_SET_ENABLE_PCI_FN 0 /* Yes, 0 enables it */ +#define CLP_SET_DISABLE_PCI_FN 1 /* Yes, 1 disables it */ + +#define CLP_UTIL_STR_LEN 64 + +#define CLP_MASK_FMT 0xf0000000 + +/* List PCI functions request */ +typedef struct ClpReqListPci { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint64_t resume_token; + uint64_t reserved2; +} QEMU_PACKED ClpReqListPci; + +/* List PCI functions response */ +typedef struct ClpRspListPci { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint64_t resume_token; + uint32_t mdd; + uint16_t max_fn; + uint8_t flags; + uint8_t entry_size; + ClpFhListEntry fh_list[CLP_FH_LIST_NR_ENTRIES]; +} QEMU_PACKED ClpRspListPci; + +/* Query PCI function request */ +typedef struct ClpReqQueryPci { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint32_t fh; /* function handle */ + uint32_t reserved2; + uint64_t reserved3; +} QEMU_PACKED ClpReqQueryPci; + +/* Query PCI function response */ +typedef struct ClpRspQueryPci { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint16_t vfn; /* virtual fn number */ +#define CLP_RSP_QPCI_MASK_UTIL 0x100 +#define CLP_RSP_QPCI_MASK_PFGID 0xff + uint16_t ug; + uint32_t fid; /* pci function id */ + uint8_t bar_size[PCI_BAR_COUNT]; + uint16_t pchid; + uint32_t bar[PCI_BAR_COUNT]; + uint64_t reserved2; + uint64_t sdma; /* start dma as */ + uint64_t edma; /* end dma as */ + uint32_t reserved3[11]; + uint32_t uid; + uint8_t util_str[CLP_UTIL_STR_LEN]; /* utility string */ +} QEMU_PACKED ClpRspQueryPci; + +/* Query PCI function group request */ +typedef struct ClpReqQueryPciGrp { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; +#define CLP_REQ_QPCIG_MASK_PFGID 0xff + uint32_t g; + uint32_t reserved2; + uint64_t reserved3; +} QEMU_PACKED ClpReqQueryPciGrp; + +/* Query PCI function group response */ +typedef struct ClpRspQueryPciGrp { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; +#define CLP_RSP_QPCIG_MASK_NOI 0xfff + uint16_t i; + uint8_t version; +#define CLP_RSP_QPCIG_MASK_FRAME 0x2 +#define CLP_RSP_QPCIG_MASK_REFRESH 0x1 + uint8_t fr; + uint16_t maxstbl; + uint16_t mui; + uint64_t reserved3; + uint64_t dasm; /* dma address space mask */ + uint64_t msia; /* MSI address */ + uint64_t reserved4; + uint64_t reserved5; +} QEMU_PACKED ClpRspQueryPciGrp; + +/* Set PCI function request */ +typedef struct ClpReqSetPci { + ClpReqHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint32_t fh; /* function handle */ + uint16_t reserved2; + uint8_t oc; /* operation controls */ + uint8_t ndas; /* number of dma spaces */ + uint64_t reserved3; +} QEMU_PACKED ClpReqSetPci; + +/* Set PCI function response */ +typedef struct ClpRspSetPci { + ClpRspHdr hdr; + uint32_t fmt; + uint64_t reserved1; + uint32_t fh; /* function handle */ + uint32_t reserved3; + uint64_t reserved4; +} QEMU_PACKED ClpRspSetPci; + +typedef struct ClpReqRspListPci { + ClpReqListPci request; + ClpRspListPci response; +} QEMU_PACKED ClpReqRspListPci; + +typedef struct ClpReqRspSetPci { + ClpReqSetPci request; + ClpRspSetPci response; +} QEMU_PACKED ClpReqRspSetPci; + +typedef struct ClpReqRspQueryPci { + ClpReqQueryPci request; + ClpRspQueryPci response; +} QEMU_PACKED ClpReqRspQueryPci; + +typedef struct ClpReqRspQueryPciGrp { + ClpReqQueryPciGrp request; + ClpRspQueryPciGrp response; +} QEMU_PACKED ClpReqRspQueryPciGrp; + +#endif diff --git a/include/hw/s390x/s390-pci-inst.h b/include/hw/s390x/s390-pci-inst.h index fa3bf8b..6c4273a 100644 --- a/include/hw/s390x/s390-pci-inst.h +++ b/include/hw/s390x/s390-pci-inst.h @@ -17,202 +17,6 @@ #include "s390-pci-bus.h" #include "sysemu/dma.h" -/* CLP common request & response block size */ -#define CLP_BLK_SIZE 4096 -#define PCI_BAR_COUNT 6 -#define PCI_MAX_FUNCTIONS 4096 - -typedef struct ClpReqHdr { - uint16_t len; - uint16_t cmd; -} QEMU_PACKED ClpReqHdr; - -typedef struct ClpRspHdr { - uint16_t len; - uint16_t rsp; -} QEMU_PACKED ClpRspHdr; - -/* CLP Response Codes */ -#define CLP_RC_OK 0x0010 /* Command request successfully */ -#define CLP_RC_CMD 0x0020 /* Command code not recognized */ -#define CLP_RC_PERM 0x0030 /* Command not authorized */ -#define CLP_RC_FMT 0x0040 /* Invalid command request format */ -#define CLP_RC_LEN 0x0050 /* Invalid command request length */ -#define CLP_RC_8K 0x0060 /* Command requires 8K LPCB */ -#define CLP_RC_RESNOT0 0x0070 /* Reserved field not zero */ -#define CLP_RC_NODATA 0x0080 /* No data available */ -#define CLP_RC_FC_UNKNOWN 0x0100 /* Function code not recognized */ - -/* - * Call Logical Processor - Command Codes - */ -#define CLP_LIST_PCI 0x0002 -#define CLP_QUERY_PCI_FN 0x0003 -#define CLP_QUERY_PCI_FNGRP 0x0004 -#define CLP_SET_PCI_FN 0x0005 - -/* PCI function handle list entry */ -typedef struct ClpFhListEntry { - uint16_t device_id; - uint16_t vendor_id; -#define CLP_FHLIST_MASK_CONFIG 0x80000000 - uint32_t config; - uint32_t fid; - uint32_t fh; -} QEMU_PACKED ClpFhListEntry; - -#define CLP_RC_SETPCIFN_FH 0x0101 /* Invalid PCI fn handle */ -#define CLP_RC_SETPCIFN_FHOP 0x0102 /* Fn handle not valid for op */ -#define CLP_RC_SETPCIFN_DMAAS 0x0103 /* Invalid DMA addr space */ -#define CLP_RC_SETPCIFN_RES 0x0104 /* Insufficient resources */ -#define CLP_RC_SETPCIFN_ALRDY 0x0105 /* Fn already in requested state */ -#define CLP_RC_SETPCIFN_ERR 0x0106 /* Fn in permanent error state */ -#define CLP_RC_SETPCIFN_RECPND 0x0107 /* Error recovery pending */ -#define CLP_RC_SETPCIFN_BUSY 0x0108 /* Fn busy */ -#define CLP_RC_LISTPCI_BADRT 0x010a /* Resume token not recognized */ -#define CLP_RC_QUERYPCIFG_PFGID 0x010b /* Unrecognized PFGID */ - -/* request or response block header length */ -#define LIST_PCI_HDR_LEN 32 - -/* Number of function handles fitting in response block */ -#define CLP_FH_LIST_NR_ENTRIES \ - ((CLP_BLK_SIZE - 2 * LIST_PCI_HDR_LEN) \ - / sizeof(ClpFhListEntry)) - -#define CLP_SET_ENABLE_PCI_FN 0 /* Yes, 0 enables it */ -#define CLP_SET_DISABLE_PCI_FN 1 /* Yes, 1 disables it */ - -#define CLP_UTIL_STR_LEN 64 - -#define CLP_MASK_FMT 0xf0000000 - -/* List PCI functions request */ -typedef struct ClpReqListPci { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint64_t resume_token; - uint64_t reserved2; -} QEMU_PACKED ClpReqListPci; - -/* List PCI functions response */ -typedef struct ClpRspListPci { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint64_t resume_token; - uint32_t mdd; - uint16_t max_fn; - uint8_t flags; - uint8_t entry_size; - ClpFhListEntry fh_list[CLP_FH_LIST_NR_ENTRIES]; -} QEMU_PACKED ClpRspListPci; - -/* Query PCI function request */ -typedef struct ClpReqQueryPci { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint32_t fh; /* function handle */ - uint32_t reserved2; - uint64_t reserved3; -} QEMU_PACKED ClpReqQueryPci; - -/* Query PCI function response */ -typedef struct ClpRspQueryPci { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint16_t vfn; /* virtual fn number */ -#define CLP_RSP_QPCI_MASK_UTIL 0x100 -#define CLP_RSP_QPCI_MASK_PFGID 0xff - uint16_t ug; - uint32_t fid; /* pci function id */ - uint8_t bar_size[PCI_BAR_COUNT]; - uint16_t pchid; - uint32_t bar[PCI_BAR_COUNT]; - uint64_t reserved2; - uint64_t sdma; /* start dma as */ - uint64_t edma; /* end dma as */ - uint32_t reserved3[11]; - uint32_t uid; - uint8_t util_str[CLP_UTIL_STR_LEN]; /* utility string */ -} QEMU_PACKED ClpRspQueryPci; - -/* Query PCI function group request */ -typedef struct ClpReqQueryPciGrp { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; -#define CLP_REQ_QPCIG_MASK_PFGID 0xff - uint32_t g; - uint32_t reserved2; - uint64_t reserved3; -} QEMU_PACKED ClpReqQueryPciGrp; - -/* Query PCI function group response */ -typedef struct ClpRspQueryPciGrp { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; -#define CLP_RSP_QPCIG_MASK_NOI 0xfff - uint16_t i; - uint8_t version; -#define CLP_RSP_QPCIG_MASK_FRAME 0x2 -#define CLP_RSP_QPCIG_MASK_REFRESH 0x1 - uint8_t fr; - uint16_t maxstbl; - uint16_t mui; - uint64_t reserved3; - uint64_t dasm; /* dma address space mask */ - uint64_t msia; /* MSI address */ - uint64_t reserved4; - uint64_t reserved5; -} QEMU_PACKED ClpRspQueryPciGrp; - -/* Set PCI function request */ -typedef struct ClpReqSetPci { - ClpReqHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint32_t fh; /* function handle */ - uint16_t reserved2; - uint8_t oc; /* operation controls */ - uint8_t ndas; /* number of dma spaces */ - uint64_t reserved3; -} QEMU_PACKED ClpReqSetPci; - -/* Set PCI function response */ -typedef struct ClpRspSetPci { - ClpRspHdr hdr; - uint32_t fmt; - uint64_t reserved1; - uint32_t fh; /* function handle */ - uint32_t reserved3; - uint64_t reserved4; -} QEMU_PACKED ClpRspSetPci; - -typedef struct ClpReqRspListPci { - ClpReqListPci request; - ClpRspListPci response; -} QEMU_PACKED ClpReqRspListPci; - -typedef struct ClpReqRspSetPci { - ClpReqSetPci request; - ClpRspSetPci response; -} QEMU_PACKED ClpReqRspSetPci; - -typedef struct ClpReqRspQueryPci { - ClpReqQueryPci request; - ClpRspQueryPci response; -} QEMU_PACKED ClpReqRspQueryPci; - -typedef struct ClpReqRspQueryPciGrp { - ClpReqQueryPciGrp request; - ClpRspQueryPciGrp response; -} QEMU_PACKED ClpReqRspQueryPciGrp; - /* Load/Store status codes */ #define ZPCI_PCI_ST_FUNC_NOT_ENABLED 4 #define ZPCI_PCI_ST_FUNC_IN_ERR 8 From patchwork Fri Oct 2 20:06:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814597 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3ACEE112E for ; Fri, 2 Oct 2020 20:13:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DA162206E3 for ; Fri, 2 Oct 2020 20:13:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="eALEgxmd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA162206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:33226 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORQW-0005uj-S5 for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:13:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44992) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKL-0006sI-TS; Fri, 02 Oct 2020 16:06:50 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:18100 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKJ-0003pw-SY; Fri, 02 Oct 2020 16:06:49 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K26h2129853; Fri, 2 Oct 2020 16:06:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=/c4aQd29M1auT0/CyDwBkm84KfpaCpql/9dOy4nQC0E=; b=eALEgxmdc+44lD6T0bcXOupKuv9yscmfYbSHfNOq/97FttwmItdOhq1oRjOInhDjoHEN 39zZ+zsiKWb4ZnRSwiDYwAJhLSbXtjLQ+lbKtSjDtO8d3an9yqjkAfXb19MRqCh/7hfo YbH3ioQ67lL4rVwEjg8DIfO0felfvfhedGOGmh/NVNiRcUiIMtlFyTu45P/ZOplvofsy UHi8luMb62JciJ2LeBdHQrz1+NTynV9KYAAVEdZIqBlQBrGbK2cmG0PXa2YzvOuJDSRr /s0bHtFHQ3FEUoA5a6VtVPYPI1zqAjTqO4IH8HSZgl+H2GqN9T4OUVVeTXZfYMJDTYe7 PA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 33xaq1rcbw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:46 -0400 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K2T4e131655; Fri, 2 Oct 2020 16:06:46 -0400 Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 33xaq1rcbq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:46 -0400 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092JmV0c013682; Fri, 2 Oct 2020 20:06:45 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma04wdc.us.ibm.com with ESMTP id 33sw9a0aqp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:45 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6eva34341184 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:40 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 397F5BE051; Fri, 2 Oct 2020 20:06:44 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E4C48BE04F; Fri, 2 Oct 2020 20:06:42 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:42 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 6/9] s390x/pci: use a PCI Group structure Date: Fri, 2 Oct 2020 16:06:28 -0400 Message-Id: <1601669191-6731-7-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0 bulkscore=0 clxscore=1015 malwarescore=0 mlxscore=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=2 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020142 Received-SPF: pass client-ip=148.163.158.5; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:37 X-ACL-Warn: Detected OS = Linux 3.x [generic] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Pierre Morel We use a S390PCIGroup structure to hold the information related to a zPCI Function group. This allows us to be ready to support multiple groups and to retrieve the group information from the host. Signed-off-by: Pierre Morel Signed-off-by: Matthew Rosato --- hw/s390x/s390-pci-bus.c | 42 +++++++++++++++++++++++++++++++++++++++++ hw/s390x/s390-pci-inst.c | 23 +++++++++++++--------- include/hw/s390x/s390-pci-bus.h | 10 ++++++++++ 3 files changed, 66 insertions(+), 9 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index a929340..c99f2f0 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -737,6 +737,46 @@ static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn) object_unref(OBJECT(iommu)); } +static S390PCIGroup *s390_group_create(int id) +{ + S390PCIGroup *group; + S390pciState *s = s390_get_phb(); + + group = g_new0(S390PCIGroup, 1); + group->id = id; + QTAILQ_INSERT_TAIL(&s->zpci_groups, group, link); + return group; +} + +S390PCIGroup *s390_group_find(int id) +{ + S390PCIGroup *group; + S390pciState *s = s390_get_phb(); + + QTAILQ_FOREACH(group, &s->zpci_groups, link) { + if (group->id == id) { + return group; + } + } + return NULL; +} + +static void s390_pci_init_default_group(void) +{ + S390PCIGroup *group; + ClpRspQueryPciGrp *resgrp; + + group = s390_group_create(ZPCI_DEFAULT_FN_GRP); + resgrp = &group->zpci_group; + resgrp->fr = 1; + stq_p(&resgrp->dasm, 0); + stq_p(&resgrp->msia, ZPCI_MSI_ADDR); + stw_p(&resgrp->mui, DEFAULT_MUI); + stw_p(&resgrp->i, 128); + stw_p(&resgrp->maxstbl, 128); + resgrp->version = 0; +} + static void s390_pcihost_realize(DeviceState *dev, Error **errp) { PCIBus *b; @@ -764,7 +804,9 @@ static void s390_pcihost_realize(DeviceState *dev, Error **errp) s->bus_no = 0; QTAILQ_INIT(&s->pending_sei); QTAILQ_INIT(&s->zpci_devs); + QTAILQ_INIT(&s->zpci_groups); + s390_pci_init_default_group(); css_register_io_adapters(CSS_IO_ADAPTER_PCI, true, false, S390_ADAPTER_SUPPRESSIBLE, errp); } diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index 639b13c..aeb8b5f 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -284,21 +284,25 @@ int clp_service_call(S390CPU *cpu, uint8_t r2, uintptr_t ra) stq_p(&resquery->edma, ZPCI_EDMA_ADDR); stl_p(&resquery->fid, pbdev->fid); stw_p(&resquery->pchid, 0); - stw_p(&resquery->ug, 1); + stw_p(&resquery->ug, ZPCI_DEFAULT_FN_GRP); stl_p(&resquery->uid, pbdev->uid); stw_p(&resquery->hdr.rsp, CLP_RC_OK); break; } case CLP_QUERY_PCI_FNGRP: { ClpRspQueryPciGrp *resgrp = (ClpRspQueryPciGrp *)resh; - resgrp->fr = 1; - stq_p(&resgrp->dasm, 0); - stq_p(&resgrp->msia, ZPCI_MSI_ADDR); - stw_p(&resgrp->mui, DEFAULT_MUI); - stw_p(&resgrp->i, 128); - stw_p(&resgrp->maxstbl, 128); - resgrp->version = 0; + ClpReqQueryPciGrp *reqgrp = (ClpReqQueryPciGrp *)reqh; + S390PCIGroup *group; + + group = s390_group_find(reqgrp->g); + if (!group) { + /* We do not allow access to unknown groups */ + /* The group must have been obtained with a vfio device */ + stw_p(&resgrp->hdr.rsp, CLP_RC_QUERYPCIFG_PFGID); + goto out; + } + memcpy(resgrp, &group->zpci_group, sizeof(ClpRspQueryPciGrp)); stw_p(&resgrp->hdr.rsp, CLP_RC_OK); break; } @@ -754,7 +758,8 @@ int pcistb_service_call(S390CPU *cpu, uint8_t r1, uint8_t r3, uint64_t gaddr, } /* Length must be greater than 8, a multiple of 8 */ /* and not greater than maxstbl */ - if ((len <= 8) || (len % 8) || (len > pbdev->maxstbl)) { + if ((len <= 8) || (len % 8) || + (len > pbdev->pci_group->zpci_group.maxstbl)) { goto specification_error; } /* Do not cross a 4K-byte boundary */ diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bus.h index 04e7cb6..52f6e7b 100644 --- a/include/hw/s390x/s390-pci-bus.h +++ b/include/hw/s390x/s390-pci-bus.h @@ -308,6 +308,14 @@ typedef struct ZpciFmb { } ZpciFmb; QEMU_BUILD_BUG_MSG(offsetof(ZpciFmb, fmt0) != 48, "padding in ZpciFmb"); +#define ZPCI_DEFAULT_FN_GRP 0x20 +typedef struct S390PCIGroup { + ClpRspQueryPciGrp zpci_group; + int id; + QTAILQ_ENTRY(S390PCIGroup) link; +} S390PCIGroup; +S390PCIGroup *s390_group_find(int id); + struct S390PCIBusDevice { DeviceState qdev; PCIDevice *pdev; @@ -325,6 +333,7 @@ struct S390PCIBusDevice { uint16_t noi; uint16_t maxstbl; uint8_t sum; + S390PCIGroup *pci_group; S390MsixInfo msix; AdapterRoutes routes; S390PCIIOMMU *iommu; @@ -349,6 +358,7 @@ struct S390pciState { GHashTable *zpci_table; QTAILQ_HEAD(, SeiContainer) pending_sei; QTAILQ_HEAD(, S390PCIBusDevice) zpci_devs; + QTAILQ_HEAD(, S390PCIGroup) zpci_groups; }; S390pciState *s390_get_phb(void); From patchwork Fri Oct 2 20:06:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814595 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60FE9139F for ; Fri, 2 Oct 2020 20:13:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2391D206E3 for ; Fri, 2 Oct 2020 20:13:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="barTABkb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2391D206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:32948 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORQO-0005nh-5C for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:13:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45004) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKN-0006ts-WB; Fri, 02 Oct 2020 16:06:54 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:64852) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKM-0003qK-89; Fri, 02 Oct 2020 16:06:51 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K1w3t008793; Fri, 2 Oct 2020 16:06:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=mPRqIry+1htThAWT37oYyw8XVX4dUwbBHMUdDjTjQWY=; b=barTABkbKQVwLKpgbh1ZqTiXwNxsMZVUqf0Ui0+N5wxRq2JVn55yKFa4CBGn9NSzDERh taPuIUwdtW7aNCeU+gFYFkr1WZS8Ay0rkES69/bzloF132zel7pphWkts7JYRQt2zbaR j9QxpY7vtlv0N1ouGl8pzLIUgfkxaR1+AdhjRDqqB8l5V0dudJyLVea1fud9DssCliCH HS10lh4sKwg4HAcPh7swtD8rSUvIvYrRhI8Ei6ZmJFFuPV3sK3kPeAJRjg2Hi68ZTEZj eoS6aPywmkjsuYXshVvt0XmUsbQu+ufA8b89wWq2Er/3OSW5laUurMjIPEAWWIj1VsJK xA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 33x9uasp0r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:48 -0400 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K2CDU009899; Fri, 2 Oct 2020 16:06:48 -0400 Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 33x9uasp07-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:48 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092Jm86p011502; Fri, 2 Oct 2020 20:06:47 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma03wdc.us.ibm.com with ESMTP id 33sw99rab6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:46 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6jWJ57147790 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:45 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9D581BE053; Fri, 2 Oct 2020 20:06:45 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5BA40BE04F; Fri, 2 Oct 2020 20:06:44 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:44 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 7/9] s390x/pci: clean up s390 PCI groups Date: Fri, 2 Oct 2020 16:06:29 -0400 Message-Id: <1601669191-6731-8-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 impostorscore=0 spamscore=0 malwarescore=0 bulkscore=0 phishscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020142 Received-SPF: pass client-ip=148.163.156.1; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:40 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Add a step to remove all stashed PCI groups to avoid stale data between machine resets. Signed-off-by: Matthew Rosato --- hw/s390x/s390-pci-bus.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index c99f2f0..c34f14a 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -811,6 +811,17 @@ static void s390_pcihost_realize(DeviceState *dev, Error **errp) S390_ADAPTER_SUPPRESSIBLE, errp); } +static void s390_pcihost_unrealize(DeviceState *dev) +{ + S390PCIGroup *group; + S390pciState *s = S390_PCI_HOST_BRIDGE(dev); + + while (!QTAILQ_EMPTY(&s->zpci_groups)) { + group = QTAILQ_FIRST(&s->zpci_groups); + QTAILQ_REMOVE(&s->zpci_groups, group, link); + } +} + static int s390_pci_msix_init(S390PCIBusDevice *pbdev) { char *name; @@ -1165,6 +1176,7 @@ static void s390_pcihost_class_init(ObjectClass *klass, void *data) dc->reset = s390_pcihost_reset; dc->realize = s390_pcihost_realize; + dc->unrealize = s390_pcihost_unrealize; hc->pre_plug = s390_pcihost_pre_plug; hc->plug = s390_pcihost_plug; hc->unplug_request = s390_pcihost_unplug_request; From patchwork Fri Oct 2 20:06:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17896139F for ; Fri, 2 Oct 2020 20:10:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C1AE206E3 for ; Fri, 2 Oct 2020 20:10:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="EqVCgqge" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C1AE206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:55712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORNz-0003OY-IO for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:10:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45016) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKQ-0006u1-Gj; Fri, 02 Oct 2020 16:06:54 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:16980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKO-0003qX-AQ; Fri, 02 Oct 2020 16:06:54 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K261K063401; Fri, 2 Oct 2020 16:06:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=C5JQCQIst+KDnNJxTvrs5f3/srqXPxtOEYLI+yitwj4=; b=EqVCgqgeytlP11UlYZNK84wMVT9BDnx/5O3wlFpZnsOaI8FW+tZwAZxHZqm1+Wmmd1vc 85s4rg0V82C1AB8uQNEF65lYwnv9oTFH1uyk7eMXLcKEKQgXHPTXX/ZTZjc22msSCEla hKHjDWV6x+ZVqe6q1Qd5t9NTagEPhkmTIMzvmezzz5h2ZaLyu2rI2ycQ3/pNkfW9Qxkm ujlUa51eiLr9vQxtyeZNwi4BSwM6XQ06bUmEHmM/6uJxMBHaEwK3Pa0meYKyFFuNAj/4 VKnQwc7MMJlBfa63ysPJkHk1MmG/FchsTB+y+j2Wv+WgZNw25DDCLuam/AECf1q3X9U+ iA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xanm0dmw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:50 -0400 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K3NJS072133; Fri, 2 Oct 2020 16:06:50 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 33xanm0dmc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:49 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092JlIuJ012339; Fri, 2 Oct 2020 20:06:48 GMT Received: from b03cxnp08025.gho.boulder.ibm.com (b03cxnp08025.gho.boulder.ibm.com [9.17.130.17]) by ppma01dal.us.ibm.com with ESMTP id 33sw9aecbs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:48 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6gle36831668 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:42 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0D541BE051; Fri, 2 Oct 2020 20:06:47 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BF684BE054; Fri, 2 Oct 2020 20:06:45 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:45 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 8/9] s390x/pci: use a PCI Function structure Date: Fri, 2 Oct 2020 16:06:30 -0400 Message-Id: <1601669191-6731-9-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 priorityscore=1501 mlxlogscore=964 impostorscore=0 phishscore=0 spamscore=0 suspectscore=0 adultscore=0 bulkscore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020142 Received-SPF: pass client-ip=148.163.156.1; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:40 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Pierre Morel We use a ClpRspQueryPci structure to hold the information related to a zPCI Function. This allows us to be ready to support different zPCI functions and to retrieve the zPCI function information from the host. Signed-off-by: Pierre Morel Signed-off-by: Matthew Rosato --- hw/s390x/s390-pci-bus.c | 22 +++++++++++++++++----- hw/s390x/s390-pci-inst.c | 8 ++------ include/hw/s390x/s390-pci-bus.h | 1 + 3 files changed, 20 insertions(+), 11 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index c34f14a..3dc8a10 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -777,6 +777,17 @@ static void s390_pci_init_default_group(void) resgrp->version = 0; } +static void set_pbdev_info(S390PCIBusDevice *pbdev) +{ + pbdev->zpci_fn.sdma = ZPCI_SDMA_ADDR; + pbdev->zpci_fn.edma = ZPCI_EDMA_ADDR; + pbdev->zpci_fn.pchid = 0; + pbdev->zpci_fn.ug = ZPCI_DEFAULT_FN_GRP; + pbdev->zpci_fn.fid = pbdev->fid; + pbdev->zpci_fn.uid = pbdev->uid; + pbdev->pci_group = s390_group_find(ZPCI_DEFAULT_FN_GRP); +} + static void s390_pcihost_realize(DeviceState *dev, Error **errp) { PCIBus *b; @@ -994,17 +1005,18 @@ static void s390_pcihost_plug(HotplugHandler *hotplug_dev, DeviceState *dev, } } + pbdev->pdev = pdev; + pbdev->iommu = s390_pci_get_iommu(s, pci_get_bus(pdev), pdev->devfn); + pbdev->iommu->pbdev = pbdev; + pbdev->state = ZPCI_FS_DISABLED; + set_pbdev_info(pbdev); + if (object_dynamic_cast(OBJECT(dev), "vfio-pci")) { pbdev->fh |= FH_SHM_VFIO; } else { pbdev->fh |= FH_SHM_EMUL; } - pbdev->pdev = pdev; - pbdev->iommu = s390_pci_get_iommu(s, pci_get_bus(pdev), pdev->devfn); - pbdev->iommu->pbdev = pbdev; - pbdev->state = ZPCI_FS_DISABLED; - if (s390_pci_msix_init(pbdev)) { error_setg(errp, "MSI-X support is mandatory " "in the S390 architecture"); diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index aeb8b5f..cc3d274 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -267,6 +267,8 @@ int clp_service_call(S390CPU *cpu, uint8_t r2, uintptr_t ra) goto out; } + memcpy(resquery, &pbdev->zpci_fn, sizeof(*resquery)); + for (i = 0; i < PCI_BAR_COUNT; i++) { uint32_t data = pci_get_long(pbdev->pdev->config + PCI_BASE_ADDRESS_0 + (i * 4)); @@ -280,12 +282,6 @@ int clp_service_call(S390CPU *cpu, uint8_t r2, uintptr_t ra) resquery->bar_size[i]); } - stq_p(&resquery->sdma, ZPCI_SDMA_ADDR); - stq_p(&resquery->edma, ZPCI_EDMA_ADDR); - stl_p(&resquery->fid, pbdev->fid); - stw_p(&resquery->pchid, 0); - stw_p(&resquery->ug, ZPCI_DEFAULT_FN_GRP); - stl_p(&resquery->uid, pbdev->uid); stw_p(&resquery->hdr.rsp, CLP_RC_OK); break; } diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bus.h index 52f6e7b..79206be 100644 --- a/include/hw/s390x/s390-pci-bus.h +++ b/include/hw/s390x/s390-pci-bus.h @@ -334,6 +334,7 @@ struct S390PCIBusDevice { uint16_t maxstbl; uint8_t sum; S390PCIGroup *pci_group; + ClpRspQueryPci zpci_fn; S390MsixInfo msix; AdapterRoutes routes; S390PCIIOMMU *iommu; From patchwork Fri Oct 2 20:06:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Rosato X-Patchwork-Id: 11814599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99300139F for ; Fri, 2 Oct 2020 20:15:32 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1979C206E3 for ; Fri, 2 Oct 2020 20:15:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="rjRsYQIm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1979C206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:37634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kORSj-0007up-5n for patchwork-qemu-devel@patchwork.kernel.org; Fri, 02 Oct 2020 16:15:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45032) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKU-0006vU-2P; Fri, 02 Oct 2020 16:06:58 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:31372) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kORKQ-0003qj-Fc; Fri, 02 Oct 2020 16:06:56 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 092K2877147657; Fri, 2 Oct 2020 16:06:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=pp1; bh=WDZWZB78IeDoi7y8HzMtrWGyEsURrjdAosX/wiZIGME=; b=rjRsYQImXUdJbT6Hgw14KpWkudV8g/QGz+kFrrwZbPACpv83GRVY2mdOQcdnh2Hq1Tow aXVDvJN5UqaRuf8zJNZFHmbznrUdCgt7FXOKA/wFOwklYir0wMGmNcNgUP8WAJ/gUwHl CcKe+dUF6rT2SdAavl7SiM6CG/FrA8cwJkVu9ZgRB4pVQhiuZfjj2/Bh+npxC1fiyL8q oG/wFgjT5giILBJtC8dlVhBRGYYyC8qkn/dI8JkgqMWjrPUat5L1Y4bozd6a9u3FAhGe ujiiNq2NieCSYPn68OIgfH1r6YwXd61eYuu2Z8H/+Gs1hdCwYVkyibKUw2ucGzn05SBq Ag== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 33x8ufuca9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:52 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 092K30rw151077; Fri, 2 Oct 2020 16:06:51 -0400 Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 33x8ufuc95-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 16:06:51 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 092JlDJ4012159; Fri, 2 Oct 2020 20:06:50 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma01dal.us.ibm.com with ESMTP id 33sw9aecc5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 02 Oct 2020 20:06:50 +0000 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 092K6mrC54722936 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 2 Oct 2020 20:06:48 GMT Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 82BBBBE053; Fri, 2 Oct 2020 20:06:48 +0000 (GMT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 30539BE04F; Fri, 2 Oct 2020 20:06:47 +0000 (GMT) Received: from oc4221205838.ibm.com (unknown [9.163.4.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 2 Oct 2020 20:06:47 +0000 (GMT) From: Matthew Rosato To: cohuck@redhat.com, thuth@redhat.com Subject: [PATCH v2 9/9] s390x/pci: get zPCI function info from host Date: Fri, 2 Oct 2020 16:06:31 -0400 Message-Id: <1601669191-6731-10-git-send-email-mjrosato@linux.ibm.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> References: <1601669191-6731-1-git-send-email-mjrosato@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-02_14:2020-10-02, 2020-10-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=2 phishscore=0 lowpriorityscore=0 bulkscore=0 mlxlogscore=999 impostorscore=0 malwarescore=0 spamscore=0 priorityscore=1501 clxscore=1015 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010020142 Received-SPF: pass client-ip=148.163.156.1; envelope-from=mjrosato@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/02 16:06:40 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, pmorel@linux.ibm.com, david@redhat.com, schnelle@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, pasic@linux.ibm.com, borntraeger@de.ibm.com, alex.williamson@redhat.com, mst@redhat.com, pbonzini@redhat.com, rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We use the VFIO_REGION_SUBTYPE_ZDEV_CLP subregion of PCI_VENDOR_ID_IBM to retrieve the CLP information the kernel exports. To be compatible with previous kernel versions we fall back on previous predefined values, same as the emulation values, when the region is not found. If individual CLP feature(s) are not found in the region, we fall back on default values for only those features missing from the region. This patch is based on work previously done by Pierre Morel. Signed-off-by: Matthew Rosato --- hw/s390x/meson.build | 1 + hw/s390x/s390-pci-bus.c | 10 +- hw/s390x/s390-pci-vfio.c | 235 +++++++++++++++++++++++++++++++++++++++ include/hw/s390x/s390-pci-bus.h | 1 + include/hw/s390x/s390-pci-clp.h | 12 +- include/hw/s390x/s390-pci-vfio.h | 19 ++++ 6 files changed, 271 insertions(+), 7 deletions(-) create mode 100644 hw/s390x/s390-pci-vfio.c create mode 100644 include/hw/s390x/s390-pci-vfio.h diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build index 948ceae..3ee4594 100644 --- a/hw/s390x/meson.build +++ b/hw/s390x/meson.build @@ -10,6 +10,7 @@ s390x_ss.add(files( 's390-ccw.c', 's390-pci-bus.c', 's390-pci-inst.c', + 's390-pci-vfio.c', 's390-skeys.c', 's390-stattrib.c', 's390-virtio-hcall.c', diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 3dc8a10..182e76e 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -17,6 +17,7 @@ #include "cpu.h" #include "hw/s390x/s390-pci-bus.h" #include "hw/s390x/s390-pci-inst.h" +#include "hw/s390x/s390-pci-vfio.h" #include "hw/pci/pci_bus.h" #include "hw/qdev-properties.h" #include "hw/pci/pci_bridge.h" @@ -737,7 +738,7 @@ static void s390_pci_iommu_free(S390pciState *s, PCIBus *bus, int32_t devfn) object_unref(OBJECT(iommu)); } -static S390PCIGroup *s390_group_create(int id) +S390PCIGroup *s390_group_create(int id) { S390PCIGroup *group; S390pciState *s = s390_get_phb(); @@ -782,7 +783,7 @@ static void set_pbdev_info(S390PCIBusDevice *pbdev) pbdev->zpci_fn.sdma = ZPCI_SDMA_ADDR; pbdev->zpci_fn.edma = ZPCI_EDMA_ADDR; pbdev->zpci_fn.pchid = 0; - pbdev->zpci_fn.ug = ZPCI_DEFAULT_FN_GRP; + pbdev->zpci_fn.pfgid = ZPCI_DEFAULT_FN_GRP; pbdev->zpci_fn.fid = pbdev->fid; pbdev->zpci_fn.uid = pbdev->uid; pbdev->pci_group = s390_group_find(ZPCI_DEFAULT_FN_GRP); @@ -861,7 +862,8 @@ static int s390_pci_msix_init(S390PCIBusDevice *pbdev) name = g_strdup_printf("msix-s390-%04x", pbdev->uid); memory_region_init_io(&pbdev->msix_notify_mr, OBJECT(pbdev), &s390_msi_ctrl_ops, pbdev, name, PAGE_SIZE); - memory_region_add_subregion(&pbdev->iommu->mr, ZPCI_MSI_ADDR, + memory_region_add_subregion(&pbdev->iommu->mr, + pbdev->pci_group->zpci_group.msia, &pbdev->msix_notify_mr); g_free(name); @@ -1013,6 +1015,8 @@ static void s390_pcihost_plug(HotplugHandler *hotplug_dev, DeviceState *dev, if (object_dynamic_cast(OBJECT(dev), "vfio-pci")) { pbdev->fh |= FH_SHM_VFIO; + /* Fill in CLP information passed via the vfio region */ + s390_pci_get_clp_info(pbdev); } else { pbdev->fh |= FH_SHM_EMUL; } diff --git a/hw/s390x/s390-pci-vfio.c b/hw/s390x/s390-pci-vfio.c new file mode 100644 index 0000000..8435df0 --- /dev/null +++ b/hw/s390x/s390-pci-vfio.c @@ -0,0 +1,235 @@ +/* + * s390 vfio-pci interfaces + * + * Copyright 2020 IBM Corp. + * Author(s): Matthew Rosato + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#include +#include +#include + +#include "qemu/osdep.h" +#include "hw/s390x/s390-pci-bus.h" +#include "hw/s390x/s390-pci-clp.h" +#include "hw/s390x/s390-pci-vfio.h" +#include "hw/vfio/pci.h" + +#ifndef DEBUG_S390PCI_VFIO +#define DEBUG_S390PCI_VFIO 0 +#endif + +#define DPRINTF(fmt, ...) \ + do { \ + if (DEBUG_S390PCI_VFIO) { \ + fprintf(stderr, "S390pci-vfio: " fmt, ## __VA_ARGS__); \ + } \ + } while (0) + +static void *get_next_clp_buf(struct vfio_region_zpci_info *zpci_info, + struct vfio_region_zpci_info_hdr *hdr) +{ + /* If the next payload would be beyond the region, we're done */ + if (zpci_info->argsz <= hdr->next) { + return NULL; + } + + return (void *)zpci_info + hdr->next; +} + +static void *find_clp_data(struct vfio_region_zpci_info *zpci_info, int id) +{ + struct vfio_region_zpci_info_hdr *hdr; + void *clp; + + assert(zpci_info); + + /* Jump to the first CLP feature, which starts with header information */ + clp = (void *)zpci_info + zpci_info->offset; + hdr = (struct vfio_region_zpci_info_hdr *)clp; + + while (hdr != NULL) { + if (hdr->id == id) { + return hdr; + } + hdr = get_next_clp_buf(zpci_info, hdr); + } + + return NULL; +} + +static void s390_pci_read_base(S390PCIBusDevice *pbdev, + struct vfio_region_zpci_info *zpci_info) +{ + struct vfio_region_zpci_info_base *clp; + + clp = find_clp_data(zpci_info, VFIO_REGION_ZPCI_INFO_BASE); + + /* If CLP feature not provided, just leave the defaults in place */ + if (clp == NULL) { + DPRINTF("Base PCI clp feature not found\n"); + return; + } + + pbdev->zpci_fn.sdma = clp->start_dma; + pbdev->zpci_fn.edma = clp->end_dma; + pbdev->zpci_fn.pchid = clp->pchid; + pbdev->zpci_fn.vfn = clp->vfn; + pbdev->zpci_fn.pfgid = clp->gid; + /* The following values remain 0 until we support other FMB formats */ + pbdev->zpci_fn.fmbl = 0; + pbdev->zpci_fn.pft = 0; +} + +static void s390_pci_read_group(S390PCIBusDevice *pbdev, + struct vfio_region_zpci_info *zpci_info) +{ + struct vfio_region_zpci_info_group *clp; + ClpRspQueryPciGrp *resgrp; + + clp = find_clp_data(zpci_info, VFIO_REGION_ZPCI_INFO_GROUP); + + /* If CLP feature not provided, just use the default group */ + if (clp == NULL) { + DPRINTF("Base PCI Group clp feature not found\n"); + pbdev->zpci_fn.pfgid = ZPCI_DEFAULT_FN_GRP; + pbdev->pci_group = s390_group_find(ZPCI_DEFAULT_FN_GRP); + return; + } + + /* See if the PCI group is already defined, create if not */ + pbdev->pci_group = s390_group_find(pbdev->zpci_fn.pfgid); + + if (!pbdev->pci_group) { + pbdev->pci_group = s390_group_create(pbdev->zpci_fn.pfgid); + + resgrp = &pbdev->pci_group->zpci_group; + if (clp->flags & VFIO_PCI_ZDEV_FLAGS_REFRESH) { + resgrp->fr = 1; + } + stq_p(&resgrp->dasm, clp->dasm); + stq_p(&resgrp->msia, clp->msi_addr); + stw_p(&resgrp->mui, clp->mui); + stw_p(&resgrp->i, clp->noi); + stw_p(&resgrp->maxstbl, clp->maxstbl); + stb_p(&resgrp->version, clp->version); + } +} + +static void s390_pci_read_util(S390PCIBusDevice *pbdev, + struct vfio_region_zpci_info *zpci_info) +{ + struct vfio_region_zpci_info_util *clp; + + clp = find_clp_data(zpci_info, VFIO_REGION_ZPCI_INFO_UTIL); + + /* If CLP feature not provided or unusable, leave the defaults in place */ + if (clp == NULL) { + DPRINTF("UTIL clp feature not found\n"); + return; + } + if (clp->size > CLP_UTIL_STR_LEN) { + DPRINTF("UTIL clp feature unexpected size\n"); + return; + } + + pbdev->zpci_fn.flags |= CLP_RSP_QPCI_MASK_UTIL; + memcpy(pbdev->zpci_fn.util_str, clp->util_str, CLP_UTIL_STR_LEN); +} + +static void s390_pci_read_pfip(S390PCIBusDevice *pbdev, + struct vfio_region_zpci_info *zpci_info) +{ + struct vfio_region_zpci_info_pfip *clp; + + clp = find_clp_data(zpci_info, VFIO_REGION_ZPCI_INFO_PFIP); + + /* If CLP feature not provided or unusable, leave the defaults in place */ + if (clp == NULL) { + DPRINTF("PFIP clp feature not found\n"); + return; + } + if (clp->size > CLP_PFIP_NR_SEGMENTS) { + DPRINTF("PFIP clp feature unexpected size\n"); + return; + } + + memcpy(pbdev->zpci_fn.pfip, clp->pfip, CLP_PFIP_NR_SEGMENTS); +} + +/* + * This function will look for the VFIO_REGION_SUBTYPE_IBM_ZPCI_CLP vfio device + * region, which has information about CLP features provided by the underlying + * host. On entry, defaults have already been placed into the guest CLP + * response buffers. On exit, defaults will have been overwritten for any CLP + * features found in the region; defaults will remain for any CLP features not + * found in the region. + */ +void s390_pci_get_clp_info(S390PCIBusDevice *pbdev) +{ + VFIOPCIDevice *vfio_pci; + VFIODevice *vdev; + struct vfio_region_info *info; + struct vfio_region_zpci_info *zpci_info; + int size, argsz; + int ret; + + vfio_pci = container_of(pbdev->pdev, VFIOPCIDevice, pdev); + vdev = &vfio_pci->vbasedev; + + if (vdev->num_regions < VFIO_PCI_NUM_REGIONS + 1) { + /* Fall back to old handling */ + DPRINTF("No zPCI vfio region available\n"); + return; + } + + ret = vfio_get_dev_region_info(vdev, + PCI_VENDOR_ID_IBM | + VFIO_REGION_TYPE_PCI_VENDOR_TYPE, + VFIO_REGION_SUBTYPE_IBM_ZPCI_CLP, &info); + if (ret) { + /* Fall back to old handling */ + DPRINTF("zPCI vfio region not found\n"); + return; + } + + /* Start by determining the region size */ + zpci_info = g_malloc(sizeof(*zpci_info)); + size = pread(vdev->fd, zpci_info, sizeof(*zpci_info), info->offset); + if (size != sizeof(*zpci_info)) { + DPRINTF("Failed to read vfio zPCI device region header\n"); + goto end; + } + + /* Allocate a buffer for the entire region */ + argsz = zpci_info->argsz; + zpci_info = g_realloc(zpci_info, argsz); + + /* Read the entire region now */ + size = pread(vdev->fd, zpci_info, argsz, info->offset); + if (size != argsz) { + DPRINTF("Failed to read vfio zPCI device region\n"); + goto end; + } + + /* + * Find the CLP features provided and fill in the guest CLP responses. + * Always call s390_pci_read_base first as information from this could + * determine which function group is used in s390_pci_read_group. + * For any feature not found, the default values will remain in the CLP + * response. + */ + s390_pci_read_base(pbdev, zpci_info); + s390_pci_read_group(pbdev, zpci_info); + s390_pci_read_util(pbdev, zpci_info); + s390_pci_read_pfip(pbdev, zpci_info); + +end: + g_free(info); + g_free(zpci_info); + return; +} diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bus.h index 79206be..945d5c1 100644 --- a/include/hw/s390x/s390-pci-bus.h +++ b/include/hw/s390x/s390-pci-bus.h @@ -314,6 +314,7 @@ typedef struct S390PCIGroup { int id; QTAILQ_ENTRY(S390PCIGroup) link; } S390PCIGroup; +S390PCIGroup *s390_group_create(int id); S390PCIGroup *s390_group_find(int id); struct S390PCIBusDevice { diff --git a/include/hw/s390x/s390-pci-clp.h b/include/hw/s390x/s390-pci-clp.h index e442307..5dd87c8 100644 --- a/include/hw/s390x/s390-pci-clp.h +++ b/include/hw/s390x/s390-pci-clp.h @@ -79,6 +79,7 @@ typedef struct ClpFhListEntry { #define CLP_SET_DISABLE_PCI_FN 1 /* Yes, 1 disables it */ #define CLP_UTIL_STR_LEN 64 +#define CLP_PFIP_NR_SEGMENTS 4 #define CLP_MASK_FMT 0xf0000000 @@ -120,14 +121,17 @@ typedef struct ClpRspQueryPci { uint32_t fmt; uint64_t reserved1; uint16_t vfn; /* virtual fn number */ -#define CLP_RSP_QPCI_MASK_UTIL 0x100 -#define CLP_RSP_QPCI_MASK_PFGID 0xff - uint16_t ug; +#define CLP_RSP_QPCI_MASK_UTIL 0x01 + uint8_t flags; + uint8_t pfgid; uint32_t fid; /* pci function id */ uint8_t bar_size[PCI_BAR_COUNT]; uint16_t pchid; uint32_t bar[PCI_BAR_COUNT]; - uint64_t reserved2; + uint8_t pfip[CLP_PFIP_NR_SEGMENTS]; + uint16_t reserved2; + uint8_t fmbl; + uint8_t pft; uint64_t sdma; /* start dma as */ uint64_t edma; /* end dma as */ uint32_t reserved3[11]; diff --git a/include/hw/s390x/s390-pci-vfio.h b/include/hw/s390x/s390-pci-vfio.h new file mode 100644 index 0000000..f690dae --- /dev/null +++ b/include/hw/s390x/s390-pci-vfio.h @@ -0,0 +1,19 @@ +/* + * s390 vfio-pci interfaces + * + * Copyright 2020 IBM Corp. + * Author(s): Matthew Rosato + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#ifndef HW_S390_PCI_VFIO_H +#define HW_S390_PCI_VFIO_H + +#include "hw/s390x/s390-pci-bus.h" + +void s390_pci_get_clp_info(S390PCIBusDevice *pbdev); + +#endif