From patchwork Fri Oct 28 19:16:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Bulekov X-Patchwork-Id: 13024238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7121BECAAA1 for ; Fri, 28 Oct 2022 19:18:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ooUrA-0001gs-Eb; Fri, 28 Oct 2022 15:17:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooUr8-0001g6-Ji for qemu-devel@nongnu.org; Fri, 28 Oct 2022 15:17:26 -0400 Received: from esa10.hc2706-39.iphmx.com ([216.71.140.198]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ooUr5-00031O-QU for qemu-devel@nongnu.org; Fri, 28 Oct 2022 15:17:26 -0400 X-IronPort-RemoteIP: 209.85.166.200 X-IronPort-MID: 235579293 X-IronPort-Reputation: None X-IronPort-Listener: OutgoingMail X-IronPort-SenderGroup: RELAY_GSUITE X-IronPort-MailFlowPolicy: $RELAYED IronPort-Data: A9a23:z9sd96tjiB1rCFDcPCN9IDVsqufnVORcMUV32f8akzHdYApBsoF/q tZmKTiDOv+KMTD1fY0lO4Sy90tVucCDx9dmTgNr/3sxEiwT9ZOVVN+UEBzMMnLJJKUvbq7GA +byyDXkBJppJpMJjk71atANlVEliefSAOKU5NfsYkhZXRVjRDoqlSVtkus4hp8AqdWiCkaGt MiaT/f3YTdJ4BYpdDNJg06/gEk35q6r4GlI5gdWic1j5zcyqVFEVPrzGonsdxMUcqEMdsamS uDKyq2O/2+x13/B3fv8z94X2mVTKlLjFVHmZkh+AsBOsTAbzsAG6ZvXAdJHAathZ5RlqPgqo DlFncTYpQ7EpcQgksxEO/VTO3gW0aGrZNYriJVw2CCe5xSuTpfi/xlhJAIdGokFxscwPX5f2 PUGKSkwRQyCif3jldpXSsE07igiBMziPYdau3U5iD+DVa5gTpfETKHHo9Rf2V/chOgURaeYN 5dfMGQxKk6YO3WjOX9OYH46tO6sln/zaRVStRSYqbdfD237llcsjee8aouOEjCMbehWm2eIi k/rxjy6DRUxBtOn7QGD/G3504cjmgu+Aur+DoaQ7/NvnRify3IeDDUQUl20p+T/jVSxM++zM GQR8ysq6Kk2rQmlFIGlGRK/p3GAs1gXXN84//AG1TxhA5H8u26xblXohBYaADD6nKfanQAX6 2I= IronPort-HdrOrdr: A9a23:o6yudKi+ay339+FYna0SUpiRiXBQXrUji2hC6mlwRA09TyX4ra CTdZsgvyMc5Ax9ZJhCo7G90cu7Lk80nKQdieIs1NyZMjUO1lHFEL1f Received: from mail-il1-f200.google.com ([209.85.166.200]) by ob1.hc2706-39.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 28 Oct 2022 15:17:14 -0400 Received: by mail-il1-f200.google.com with SMTP id i8-20020a056e0212c800b002f9a4c75658so5780105ilm.3 for ; Fri, 28 Oct 2022 12:17:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bu.edu; s=s1gsbu; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=RsEApOYgJmSDnt3tpUpETyCg4QSWVIS3JX2hY7/l8vo=; b=B2cou2TgmMI99n7jCQ0EcFSES+ecBVorZXZjPnwsEJaptmojzrisoEG/XykxGNjrv8 r/NfzIRIx4KXLDKeqj48cCa+b5o3HfIuZ0MFiG50101A1uoc2Lrwehr8+4uIjdwdTv2h /agwN/zuZgOnOO9UfGhNFZzD2XcjIokLBXvkYpUErB5qvNgBYdw5rC9pT/YZs/ilir56 +n1DvFTTVIynV1mr90kbqdnLk7xkLSYPULeIXjMbKkw+5IkwpLdmZwJDo2Wm/+fRviMz 9xnsy40V5azlSmr7h5Gbd8ASXJ++JMTClGPRE57YXESSjVwzDGt8kxU+d9RVKhRFfnC1 /9eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RsEApOYgJmSDnt3tpUpETyCg4QSWVIS3JX2hY7/l8vo=; b=PyAd76qQNHSzDbXgV/oEs5baaf7wE6YwMIRYkh1mK7hF+fUzVaeK3ZlxVCeRuEb+He 9KPFe2QQGRHVlBcD45ZOffQggJC83B6ig32ul8RGxUr3Y09QXLoIugIaih3czefkd2ki K4rjdM0dNacu6CR3c5NBHimQopgC9qWvqKtRLZMDpmwGMqEeirGgc0rHp531q9x8v4kT LUehFvACKNZLf9IyNjz8pf2sslc30QsOZ5K62qQzaaSmxLNZSvAYFNK1IO/8URZ87Sl6 G3SWlkBdo01kTXw+ukPAFGH5HJU7Ybis9fm6yQOyScSFbQGFYLGnZC3B4nxHDr1u9s+z g2CA== X-Gm-Message-State: ACrzQf1QwM8php2iLkVhuInM9mmkpMF5CqEDuvUKvXlABkAWu+6+bZVt VXH+BvfsKJH/5Fb5sDMV6wnU0pceHPE5b9aGBIbo+m/k//vfCBJFj72X/2Pa6rpU8TUKoK9Ml3L lqm5Gd5i7qKEGSYnm9kkyd8gJHBvopg== X-Received: by 2002:a5e:8517:0:b0:6c6:fbe7:1c0e with SMTP id i23-20020a5e8517000000b006c6fbe71c0emr489574ioj.95.1666984633563; Fri, 28 Oct 2022 12:17:13 -0700 (PDT) X-Google-Smtp-Source: AMsMyM49tQmmocWVYdn2xvfEQPX2dg2+wGStWGO3N10yBcyJrL0bS70QhG3GhAssHlFIfpI/dhUGoQ== X-Received: by 2002:a5e:8517:0:b0:6c6:fbe7:1c0e with SMTP id i23-20020a5e8517000000b006c6fbe71c0emr489551ioj.95.1666984633323; Fri, 28 Oct 2022 12:17:13 -0700 (PDT) Received: from mozz.bu.edu (mozz.bu.edu. [128.197.127.33]) by smtp.gmail.com with ESMTPSA id l5-20020a056e021c0500b002fff117027dsm1878505ilh.13.2022.10.28.12.17.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Oct 2022 12:17:12 -0700 (PDT) From: Alexander Bulekov To: qemu-devel@nongnu.org Cc: Alexander Bulekov , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Mauro Matteo Cascella , Qiuhao Li , Peter Xu , Jason Wang , David Hildenbrand , Gerd Hoffmann , Li Qiang , Thomas Huth , Laurent Vivier , Bandan Das , "Edgar E . Iglesias" , Darren Kenny , Bin Meng , Paolo Bonzini , "Michael S . Tsirkin" , Marcel Apfelbaum , =?utf-8?q?Daniel_P_=2E_Berra?= =?utf-8?q?ng=C3=A9?= , Eduardo Habkost , Jon Maloy , Siqi Chen Subject: [PATCH v3 0/7] memory: prevent dma-reentracy issues Date: Fri, 28 Oct 2022 15:16:41 -0400 Message-Id: <20221028191648.964076-1-alxndr@bu.edu> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 X-CES-GSUITE_AUTH: bf3aNvsZpxl8 Received-SPF: pass client-ip=216.71.140.198; envelope-from=alxndr@bu.edu; helo=esa10.hc2706-39.iphmx.com X-Spam_score_int: 0 X-Spam_score: -0.1 X-Spam_bar: / X-Spam_report: (-0.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HK_RANDOM_ENVFROM=0.998, HK_RANDOM_FROM=0.998, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Qemu-devel" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org These patches aim to solve two types of DMA-reentrancy issues: 1.) mmio -> dma -> mmio case To solve this, we track whether the device is engaged in io by checking/setting a flag within APIs used for MMIO access. 2.) bh -> dma write -> mmio case This case is trickier, since we dont have a generic way to associate a bh with the underlying Device/DeviceState. Thus, this version introduces a change to QEMU's DMA APIs to associate each request with the origiantor DeviceState. In total, the affected APIs are used in approximately 250 locations: dma_memory_valid (1 usage) dma_memory_rw (~5 uses) dma_memory_read (~92 uses) dma_memory_write (~71 uses) dma_memory_set (~4 uses) dma_memory_map (~18 uses) dma_memory_unmap (~21 uses) {ld,st}_{le,be}_{uw,l,q}_dma (~10 uses) ldub_dma (does not appear to be used anywhere) stb_dma (1 usage) dma_buf_read (~18 uses) dma_buf_write (~7 uses) It is not trivial to mechanically replace all of the invocations: For many cases, this will be as simple as adding DEVICE(s) to the arguments, but there are locations where the code will need to be slightly changed. As such, for now I added "_guarded" versions of most of the APIs which can be used until all of the invocations are fixed. The end goal is to go through all of hw/ and make the required changes (I will need help with this). Once that is done, the "_guarded" APIs can take the place of the standard DMA APIs and we can mecahnically remove the "_guarded" suffix from all invocations. These changes do not address devices that bypass DMA apis and directly call into address_space.. APIs. This occurs somewhat commonly, and prevents me from fixing issues in Virtio devices, such as: https://gitlab.com/qemu-project/qemu/-/issues/827 I'm not sure what approach we should take for these cases - maybe they should be switched to DMA APIs (or the DMA API expanded). v2 -> v3: Bite the bullet and modify the DMA APIs, rather than attempting to guess DeviceStates in BHs. Alexander Bulekov (7): memory: associate DMA accesses with the initiator Device dma-helpers: switch to guarded DMA accesses ahci: switch to guarded DMA acccesses sdhci: switch to guarded DMA accesses ehci: switch to guarded DMA accesses xhci: switch to guarded DMA accesses usb/libhw: switch to guarded DMA accesses hw/ide/ahci.c | 16 +++++++++------- hw/sd/sdhci.c | 43 ++++++++++++++++++++++-------------------- hw/usb/hcd-ehci.c | 8 ++++---- hw/usb/hcd-xhci.c | 24 +++++++++++------------ hw/usb/libhw.c | 4 ++-- include/hw/qdev-core.h | 2 ++ include/sysemu/dma.h | 41 ++++++++++++++++++++++++++++++++++++++++ softmmu/dma-helpers.c | 15 ++++++++------- softmmu/memory.c | 15 +++++++++++++++ softmmu/trace-events | 1 + 10 files changed, 117 insertions(+), 52 deletions(-)