From patchwork Wed Jan 19 21:41:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 12717792 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9116DC433EF for ; Wed, 19 Jan 2022 21:49:37 +0000 (UTC) Received: from localhost ([::1]:34654 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAIpj-0002kT-Nv for qemu-devel@archiver.kernel.org; Wed, 19 Jan 2022 16:49:35 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58082) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAIix-0007d7-6J for qemu-devel@nongnu.org; Wed, 19 Jan 2022 16:42:38 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:38828) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAIit-0007H0-Ty for qemu-devel@nongnu.org; Wed, 19 Jan 2022 16:42:34 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 20JJXoEm032613; Wed, 19 Jan 2022 21:42:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=dyWplkGsMn9ULuUq0iexMLNiaHFr0c6f8czUHH+U+aQ=; b=ywcPAl33kTjYAA8jKvUYgs6DSiUx3//Y6pFmmAw8nuHL6YkMEEcO4sTfumk5pGUfInk6 O8VfmPdTLrarMF4POenWQmGubDpDoWSbICQwwuDHIkSYcoed8Tfb8EuwrkA4BFx4YltP HY+hHaB6vfZdpsQayaBA5UaDrp9jwyPegLkt80duTqU2E/en/kwGJzKR35Edc4/nU/zR n5Gwi81Bs4LBnjs+xmU++aPVn9L7nUi+JfMENZdX+oBVTSzIszG1KNiROWhZTYjqXjWD GNka5UL3Yd/qIKJ3L61b5C/+nhF+FEsL21YQMzBcRCxj0lvmEhsgKtjAskG9zk+/jzxd LA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3dnc4q6hwh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jan 2022 21:42:20 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 20JLa6fx131728; Wed, 19 Jan 2022 21:42:19 GMT Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2177.outbound.protection.outlook.com [104.47.59.177]) by userp3020.oracle.com with ESMTP id 3dkqqr64c6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jan 2022 21:42:19 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IqUne23E41W+EZ5QJPUQpwaJKpu67VWAAuE6cyfulMPfLpQGtL1/mDU9gPRa/W9HbSz86J15Q1+noux5UKICw0IIYIiov0wCbXhb+5y7vYsYsaArE0VGkzrn/85JAZkHV+SqnJsX+S9iw83EisLDRWCy5L0sjLFgxd9p8rOzRELM/GtK/jQ+fRS/+HuXyQjRLnQ+0PijD12x+1V2KX60nfQRmX0pOXtmK0amitfnupF0nN2D0VU3GWulm1Fkx25ej/FiFRzqumHUYpwBl8euaijafoB90NWpxqTTpjEbtCbjS5xMMeYfyv2UvBGyBadRjIZ6TPhLjaYMiFRHnY0XJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dyWplkGsMn9ULuUq0iexMLNiaHFr0c6f8czUHH+U+aQ=; b=hq6Cz2rvLw0CsfakPAhSTYkZrpuLCh4k9NeR3ft2IG9ImXVAP5IK5hd2lN92YRyDDfbeaGUqr3Gim8+a3LOpeUuwsQ1PJISQu3MdAVsrvvWeNi73svwznT8SWw7xpGonuZ39pJOY+p5idA2MpFX4RRfd3TG7sRf74GXXXI8qeYQ61PRu/6tzYVDrn65n2NyKy3sK3IOT3z5jzgBRkx0gjg8cCJKrGqyEMt4UV/eZddecIVKVrkQ/6BNyqEygFxFHTcZReQ9lrnsO6KX52QA/0Z5Erg+Wip3M6D8WXzP/rKIsi8RpWq1tz8Ad0I+127hCsHSPgkpz3AmLPHl3mZo/eQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dyWplkGsMn9ULuUq0iexMLNiaHFr0c6f8czUHH+U+aQ=; b=Q5WOtdPz+HvYut6kosX+Og3H7XbQxEp6Fba4eiseliuCS9l8coqMoOhxLrnJE7WMLbps6qrlMDP/NDSup+paVGVq8d2QzL6Uckt9THesrEvPRzbu6A/KQoQ+LW9d4hAH0sKzsHRUK5yPxWlctN0l791Gb8NSOpi9dBt0QmNF3vo= Received: from MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) by SA1PR10MB5887.namprd10.prod.outlook.com (2603:10b6:806:22b::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.7; Wed, 19 Jan 2022 21:42:17 +0000 Received: from MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::98b1:4348:5117:5fc]) by MN2PR10MB4013.namprd10.prod.outlook.com ([fe80::98b1:4348:5117:5fc%7]) with mapi id 15.20.4888.014; Wed, 19 Jan 2022 21:42:16 +0000 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v5 00/18] vfio-user server in QEMU Date: Wed, 19 Jan 2022 16:41:49 -0500 Message-Id: X-Mailer: git-send-email 2.20.1 X-ClientProxiedBy: BYAPR08CA0043.namprd08.prod.outlook.com (2603:10b6:a03:117::20) To MN2PR10MB4013.namprd10.prod.outlook.com (2603:10b6:208:185::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d17bc3b4-114d-4c19-ae8f-08d9db948ffe X-MS-TrafficTypeDiagnostic: SA1PR10MB5887:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Xfs/ZQ9PWj6NGNIZ0oZszp6CEjcx6B726bHQOCRX3CWOCOilMf2JQNeYW/kez+++E3wvPT5vSyymKVpoCS0cdSLVPXgPUF0uD/zQrtgUDImzmW9O5dYz2B0z9cfAp/7F01f7T1v/2Oen3lY2fBqOQxq83gHluI0GKQXtI5qsowaM6U1yaOAmXjrTB8QGFzcTxjZo7/4g7PolV8ROrfvEgkpgjlComFba6JwtS55L3oFjD4KhouxDMgwZMJyTiXtp7E05Itdq2nhafrJPmj+gE8sRuxagjPy01h6wX5fCsdf4xvFYtso5uHxMH5uTp/jjfTkcFOYdmMqhSQ/BIcdDqD5uG6OMmZsui+CkoNlYNQRmbV5JpJoL0Gs3BLw2ehuzViP1jIdAEO+CHH15ERNMP7kgHQZdIWzwRdEXQjwAOpVlf6ni09EL9fGJBhz0WtOIv120nhgUB3rBJGTO8N2CKsREF0cQqnXwuL+eE7UD8/ARPhCdOVcgVlxfYHpZltU9m6yZwv3VY/FF2F+GYzkqyqsdRUC0KUk1FSbHq73/bUzNTJAKyyERIFoH9/3ztAxk1/FTzW43nsPKToMovHgvWOV+hxFkqemiJpzBocaRfAj07CHpBEOOldM0rivuRgW+VoLTTtAZUR1U//5lCmwtMCwobwfWCdbdDrclFF+Z/rsp5KSmy9IZvsvlOR28azuz8AOt39hCLDwb34afUmOvUQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR10MB4013.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(5660300002)(186003)(6666004)(6916009)(508600001)(66946007)(66556008)(26005)(7416002)(4326008)(66476007)(86362001)(107886003)(8936002)(38100700002)(38350700002)(8676002)(36756003)(83380400001)(6486002)(2616005)(6512007)(2906002)(6506007)(316002)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?FNkbaVwltq2HEIEAQGldbu6KxD6h?= =?utf-8?q?0YvVjqZyNoHc5mjQRbbKQOrIXAMmh3YKbW9crcOClKPapVtkrb4C++YjqbW3mNung?= =?utf-8?q?QlpXrVPfxaSe28g02+JxXCX3wt3iISHWUGSjWmrCQe0g9fkkpMO8OkwHA2nmZlfwh?= =?utf-8?q?CZ7JEzj5asFyETV3p1nwNHeSHiQ9bZiTKpsyhHCQ2hlTn7lOg7yE4DxGDstKxBX5f?= =?utf-8?q?+cxxegNuRpQPU4qWO353mixA/MU+JtgMAdicwSZvy8vAlYaIMfSKbvxx4xzdIxmJ+?= =?utf-8?q?UYfQG29pa4jX9GeYsBJPBSf+rvDMgEEYL3oATjYqt22mk2p7q9mhSzFcQKzl2c7Fx?= =?utf-8?q?d2nh2+qogVsccrQg60vgLXi7Ntv3NsfXLrVsiXSkQzjOR7JTkB46/Isa7Czy19AGb?= =?utf-8?q?rmOXEoHXtSYM88vD+W5mjkdMJur36bhJ3Hc5e8iIG0N9yF1eR1OYS5WdW6w3eEr14?= =?utf-8?q?04FVCkYDzmG/F7ww4Uo1uZv6xaivE12LyVQp5exaxsEaibDL0UvPGl9b7trq3Vz/b?= =?utf-8?q?aVgEs0Xs9niPQ66h9gAcfGCIbR94C/sGJi6JbRYwrtCFRO1pRZQo0uJrU+IYjWvNt?= =?utf-8?q?F1UUUXHl+VjmTnhrjCSbBGZHrKO0rq1Fk+bMVCuJRNXzxoDxUsxMp7JHjEnpTBMN+?= =?utf-8?q?Ypz7qnKNWkJ80vrRk1mhcXNY590EoWsg67tDvsWr7Yy3q1nmBzUQ+hTJrMaIWDyu1?= =?utf-8?q?lqfuU4Dgtg6ON1rY1s7h+BgEZnn9z82+BZugOfM2+jTFMxFmUR8ku/B9tHWZvKv/d?= =?utf-8?q?OUCHZUqC+e41QNGmb3k/65KoxT9eAKRHa9QDoeQOhydA3ZqqpE8ANaY9YKYKzyiY3?= =?utf-8?q?0sIOSIlbF7yR/6Vka/ll/39pP/tYiAey3xcv8mAzTwGeYqFJShATxYgIUXImFI4yV?= =?utf-8?q?3ir4VhYvmiOmP76T7CvOe6nTe5XVM4At7pQy9OVEsEajGQgqMCwyt+CXhVfYcLin7?= =?utf-8?q?F/bnwf4KsQ1oV1QZDR68jEVMcmyGM3Kgvhu7+Mpfpa3eOrfnprGbU6mz3KFNeSNC5?= =?utf-8?q?sOwFEf7hSZKZfAV1esnfcv8TCb9o3uu4yd+ESkAIHpdX7Hy9mzbucDWRD0s95OraC?= =?utf-8?q?UA6znS4Bnjj4WKloM5yyYnvu7Oq98KE5nnuQwZ+glkosuBqnN6nyVQmN3/pqxmarq?= =?utf-8?q?9y8OcXONEoU+0W5sofrybpuuP4AcO4ZjEXIWISXN9AKV11UYRpW7k4djTIvdjfQNz?= =?utf-8?q?9hGQNwwbzMkyM5DRmqNYBf1Uw21UjB/mrpda4pCw5wnjFPqCAdzp0V5ZloDKGzUIg?= =?utf-8?q?V0MXLxiXML8ncEQB3sxfxZ7dkljkQmpH9oT3Z+wklKIiyt3EFGJqO5a0rZCoGF5/O?= =?utf-8?q?aiCTIah2zwrV+cTPSc/oDi2R9LUurlMZ4NPXllAiumwE/N2bu+D6KI7hwkjumxrpH?= =?utf-8?q?Hzu880B+8QI7SFzQP4jzx1ZCZZV4ivAjyT3aaQ0uP0LSk2WgPiGJqwN/ugD5r+BVN?= =?utf-8?q?AF+TQAeYNasdaYw/WZuEZl3bfGqts7GDX8zR0/DHXUXV3AKCO0AAXqSJJQwompuyQ?= =?utf-8?q?YPl9KU0AA2HUN5/IWYl//GtkmGwz90RPTKtDOlosLuM53GlhcHnPEKM=3D?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: d17bc3b4-114d-4c19-ae8f-08d9db948ffe X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4013.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2022 21:42:16.8986 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: HiuEdSN9pGgNSGHuyuoNMfRFAhAbyAOIHPTp1DdGGjZNhZGHzrBNGYRSQQO8G3+b/ZeLzwh+9h9E902qr8rwhQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB5887 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10232 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2201190117 X-Proofpoint-GUID: 9MznWntmrWz843D2zQgSC2uuwiKwi8gD X-Proofpoint-ORIG-GUID: 9MznWntmrWz843D2zQgSC2uuwiKwi8gD Received-SPF: pass client-ip=205.220.177.32; envelope-from=jag.raman@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: eduardo@habkost.net, elena.ufimtseva@oracle.com, john.g.johnson@oracle.com, berrange@redhat.com, bleal@redhat.com, john.levon@nutanix.com, mst@redhat.com, armbru@redhat.com, quintela@redhat.com, f4bug@amsat.org, marcandre.lureau@gmail.com, stefanha@redhat.com, thanos.makatos@nutanix.com, pbonzini@redhat.com, jag.raman@oracle.com, eblake@redhat.com, dgilbert@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hi, Thank you for taking the time to provide a comprehensive feedback of the last series of patches. We have addressed all the comments. We are posting this v5 of the series, which incorporates all the feedback. Kindly share your feedback for this latest series We added the following patches to the series: - [PATCH v5 03/18] pci: isolated address space for PCI bus - [PATCH v5 04/18] pci: create and free isolated PCI buses - [PATCH v5 05/18] qdev: unplug blocker for devices - [PATCH v5 06/18] vfio-user: add HotplugHandler for remote machine - [PATCH v5 07/18] vfio-user: set qdev bus callbacks for remote machine We made the following changes to the existing patches: [PATCH v5 09/18] vfio-user: define vfio-user-server object - renamed object class member 'daemon' as 'auto_shutdown' - set VfioUserServerProperties version to 6.3 - use SocketAddressType_str to compose error message - refuse setting 'socket' and 'device' properties after server starts - added VFU_OBJECT_ERROR macro to report error [PATCH v5 10/18] vfio-user: instantiate vfio-user context - set error variable to NULL after transferring ownership with error_propagate() [PATCH v5 11/18] vfio-user: find and init PCI device - block hot-unplug of PCI device when it is attached to the server object [PATCH v5 12/18] vfio-user: run vfio-user context - emit a hangup event to the monitor when the client disconnects - reset vfu_poll_fd member and disable FD handler during finalize - add a comment to explain that attach could block - use VFU_OBJECT_ERROR instead of setting error_abort [PATCH v5 14/18] vfio-user: handle DMA mappings - use pci_address_space() to access device's root memory region - given we're using one bus per device, mapped memory regions get destroyed automatically when device is unplugged [PATCH v5 15/18] vfio-user: handle PCI BAR accesses - use pci_isol_as_io() & pci_isol_as_mem() to access the device's PCI/CPU address space. This simultaneously fixes the AddressSpace issue noted in the last review cycle [PATCH v5 16/18] vfio-user: handle device interrupts - setting own IRQ handlers for each bus - renamed vfu_object_dev_table to vfu_object_dev_to_ctx_table - indexing into vfu_object_dev_to_ctx_table with device's address pointer instead of devfn - not looking up before removing from table [PATCH v5 17/18] vfio-user: register handlers to facilitate migration - use VFU_OBJECT_ERROR instead of setting error_abort We dropped the following patch from previous series: - vfio-user: IOMMU support for remote device Thank you very much! Jagannathan Raman (18): configure, meson: override C compiler for cmake tests/avocado: Specify target VM argument to helper routines pci: isolated address space for PCI bus pci: create and free isolated PCI buses qdev: unplug blocker for devices vfio-user: add HotplugHandler for remote machine vfio-user: set qdev bus callbacks for remote machine vfio-user: build library vfio-user: define vfio-user-server object vfio-user: instantiate vfio-user context vfio-user: find and init PCI device vfio-user: run vfio-user context vfio-user: handle PCI config space accesses vfio-user: handle DMA mappings vfio-user: handle PCI BAR accesses vfio-user: handle device interrupts vfio-user: register handlers to facilitate migration vfio-user: avocado tests for vfio-user configure | 21 +- meson.build | 44 +- qapi/misc.json | 23 + qapi/qom.json | 20 +- include/hw/pci/pci.h | 12 + include/hw/pci/pci_bus.h | 17 + include/hw/qdev-core.h | 21 + include/migration/vmstate.h | 2 + migration/savevm.h | 2 + hw/pci/msi.c | 13 +- hw/pci/msix.c | 12 +- hw/pci/pci.c | 186 ++++ hw/pci/pci_bridge.c | 5 + hw/remote/machine.c | 86 ++ hw/remote/vfio-user-obj.c | 1019 ++++++++++++++++++++ migration/savevm.c | 73 ++ migration/vmstate.c | 19 + softmmu/qdev-monitor.c | 74 +- .gitlab-ci.d/buildtest.yml | 2 + .gitmodules | 3 + Kconfig.host | 4 + MAINTAINERS | 3 + hw/remote/Kconfig | 4 + hw/remote/meson.build | 3 + hw/remote/trace-events | 11 + meson_options.txt | 2 + subprojects/libvfio-user | 1 + tests/avocado/avocado_qemu/__init__.py | 14 +- tests/avocado/vfio-user.py | 225 +++++ tests/docker/dockerfiles/centos8.docker | 2 + tests/docker/dockerfiles/ubuntu2004.docker | 2 + 31 files changed, 1912 insertions(+), 13 deletions(-) create mode 100644 hw/remote/vfio-user-obj.c create mode 160000 subprojects/libvfio-user create mode 100644 tests/avocado/vfio-user.py