From patchwork Mon Apr 24 11:21:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viktor Prutyanov X-Patchwork-Id: 13222044 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D792CC77B61 for ; Mon, 24 Apr 2023 11:23:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pquGu-0000qD-VS; Mon, 24 Apr 2023 07:22:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pquGt-0000pu-UE for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:15 -0400 Received: from mail-lf1-x131.google.com ([2a00:1450:4864:20::131]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pquGj-00056I-0o for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:15 -0400 Received: by mail-lf1-x131.google.com with SMTP id 2adb3069b0e04-4ec94eb6dcaso4574731e87.3 for ; Mon, 24 Apr 2023 04:21:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20221208.gappssmtp.com; s=20221208; t=1682335318; x=1684927318; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b49RuNteQiiwmolZv5ATskx2CIb9sDZ4N568I2W2fpQ=; b=YpzdVPVe48eSpdd+9QLfbey3ENC1u1S0MTgJDV3Q9Hv7HEWvvgmJ5dajFRpjrik4bP ZecT9okJGCYjJpTG5dLynQav4VmG6rzUZTd11TUaSYo9IGXSr/8Ji/iKdnz4eYNMUAbC tpvRAwRSQ5UfVj3LJxOJoUinq9CAS3DmCMyo2GqddVqmU+JG9z6InLPs9o0TMlWeW6w3 VK97m6tSjIIf6H3d4AziktlDxLvTlu2ShXBcc/C97/jT/LQ7sFC+x4/LYGuoWGHvnxY/ SkACqwI9RgNVo/5Ze9ZoewlLGbWqnAQu1rTtOjsL6t7NIRkD4ehB+tVIxzZVyvKKKSWA gT8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682335318; x=1684927318; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b49RuNteQiiwmolZv5ATskx2CIb9sDZ4N568I2W2fpQ=; b=WTicL0tsj0Z34S0rdwXe+I/l+c/x4xwmVL6XfVOpDIe9qiDt+jUN2B4o/gXeTqpfyM /ii0JiaSkVG5F2Ay0zPNKwFflKF94YQUJlHcu+/wXVCjynNjtR4yedj98I6GZJdHJYfN 2efFi0iC4bx8G+5gBjwIVSrHUWCE1Z3G27C5NKn8tiO8CFUBQiobX21gUaceqi4Qeb8i rSIAoPTvv35fILkbu36C4f5W8V1V8qm4bBukuPwe/lTAQp5TGSL0qk1q5mXNPbGFXy+Q 7+sdN423Q8/JtHqMGZ3IPTbned58dz+6Px4XzqC7ebF8lQvsKCDz0aTzQNjjvDZ0ntsI R63A== X-Gm-Message-State: AAQBX9eazalc9c42kSNJoihsAF2fYMZ1jQxFu5crx7zlNdIGOPvxcpgg FNmc4Djt28OaI2h+dFtykLU5eA== X-Google-Smtp-Source: AKy350bppNDqHbwOEgJ5BIcccop9b4P00el97xvc4x6TCKxnQfVINbvxdtcvSZpkrjfQRrgDZnXg9A== X-Received: by 2002:a19:ac43:0:b0:4ee:d799:eca with SMTP id r3-20020a19ac43000000b004eed7990ecamr2953685lfc.40.1682335317914; Mon, 24 Apr 2023 04:21:57 -0700 (PDT) Received: from localhost.localdomain ([80.250.80.22]) by smtp.gmail.com with ESMTPSA id j1-20020a19f501000000b004eff0bcb276sm415542lfb.7.2023.04.24.04.21.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 04:21:57 -0700 (PDT) From: Viktor Prutyanov To: mst@redhat.com, jasowang@redhat.com, marcel.apfelbaum@gmail.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com Cc: philmd@linaro.org, qemu-devel@nongnu.org, yan@daynix.com, yuri.benditovich@daynix.com, Viktor Prutyanov Subject: [RFC PATCH 1/4] pci: add handling of Enable bit in ATS Control Register Date: Mon, 24 Apr 2023 14:21:44 +0300 Message-Id: <20230424112147.17083-2-viktor@daynix.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230424112147.17083-1-viktor@daynix.com> References: <20230424112147.17083-1-viktor@daynix.com> MIME-Version: 1.0 Received-SPF: none client-ip=2a00:1450:4864:20::131; envelope-from=viktor@daynix.com; helo=mail-lf1-x131.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org According to PCIe Address Translation Services specification 5.1.3., ATS Control Register has Enable bit to enable/disable ATS. Add a new field for a trigger function which is called at the Enable bit change, so that PCIe devices can handle ATS enable/disable. Signed-off-by: Viktor Prutyanov --- hw/pci/pci.c | 1 + hw/pci/pcie.c | 21 +++++++++++++++++++++ include/hw/pci/pci_device.h | 3 +++ include/hw/pci/pcie.h | 4 ++++ 4 files changed, 29 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 208c16f450..79a47d2589 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -1550,6 +1550,7 @@ void pci_default_write_config(PCIDevice *d, uint32_t addr, uint32_t val_in, int msi_write_config(d, addr, val_in, l); msix_write_config(d, addr, val_in, l); pcie_sriov_config_write(d, addr, val_in, l); + pcie_ats_config_write(d, addr, val_in, l); } /***********************************************************/ diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index 924fdabd15..e0217161e5 100644 --- a/hw/pci/pcie.c +++ b/hw/pci/pcie.c @@ -1057,6 +1057,27 @@ void pcie_ats_init(PCIDevice *dev, uint16_t offset, bool aligned) pci_set_word(dev->wmask + dev->exp.ats_cap + PCI_ATS_CTRL, 0x800f); } +void pcie_ats_config_write(PCIDevice *dev, uint32_t address, uint32_t val, + int len) +{ + uint32_t off; + uint16_t ats_cap = dev->exp.ats_cap; + + if (!ats_cap || address < ats_cap) { + return; + } + off = address - ats_cap; + if (off >= PCI_EXT_CAP_ATS_SIZEOF) { + return; + } + + if (range_covers_byte(off, len, PCI_ATS_CTRL + 1)) { + if (dev->ats_ctrl_trigger) { + dev->ats_ctrl_trigger(dev, !!(val & PCI_ATS_CTRL_ENABLE)); + } + } +} + /* ACS (Access Control Services) */ void pcie_acs_init(PCIDevice *dev, uint16_t offset) { diff --git a/include/hw/pci/pci_device.h b/include/hw/pci/pci_device.h index d3dd0f64b2..2bb1d68f3b 100644 --- a/include/hw/pci/pci_device.h +++ b/include/hw/pci/pci_device.h @@ -160,6 +160,9 @@ struct PCIDevice { /* ID of standby device in net_failover pair */ char *failover_pair_id; uint32_t acpi_index; + + /* PCI ATS enable/disable trigger */ + void (*ats_ctrl_trigger)(PCIDevice *dev, bool enable); }; static inline int pci_intx(PCIDevice *pci_dev) diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h index 798a262a0a..5f2dbd87cf 100644 --- a/include/hw/pci/pcie.h +++ b/include/hw/pci/pcie.h @@ -154,4 +154,8 @@ void pcie_cap_slot_unplug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp); void pcie_cap_slot_unplug_request_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp); + +void pcie_ats_config_write(PCIDevice *dev, uint32_t address, uint32_t val, + int len); + #endif /* QEMU_PCIE_H */ From patchwork Mon Apr 24 11:21:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viktor Prutyanov X-Patchwork-Id: 13222045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 11A38C77B61 for ; Mon, 24 Apr 2023 11:23:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pquGv-0000qR-Hg; Mon, 24 Apr 2023 07:22:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pquGq-0000p0-Fu for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:15 -0400 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pquGn-00056V-Hr for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:12 -0400 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-4ec816d64afso18432259e87.1 for ; Mon, 24 Apr 2023 04:21:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20221208.gappssmtp.com; s=20221208; t=1682335318; x=1684927318; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/q4tfU1tBwdT+EHQodeGfa8pPty+c1D1ASDSQOSgHB0=; b=ciyJjKs124mc/NT8RSutQTAZqxq9UfYpRGX/PUK7IthVZ5MhH5kTvMwLNd+kKgnWFA QcGibYh1S3NrV6cLe7kI4L165/QiKyOlqw+858MOM+ES7KhOJRTB9mmdkLQ2xk9tyO7X F1/YowZsEdq0qqCz8JKki4d9iP3Wpoxlk13JKZsOJERAOM1U+DtB9JjjhDpsisDy4bGK jQ8IFN0Ar4W/CwXq3RoIGv/SpgY8HTHc6uM6KIt0kWDgWfx7ihglsbJZO77N4ljlh7N8 3nbmF/pwPvl7H34lPagDiWriTg3YFs156596ge8Q+MDsyo3ISTqRpeIsHEbaxXjaKyhu DErw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682335318; x=1684927318; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/q4tfU1tBwdT+EHQodeGfa8pPty+c1D1ASDSQOSgHB0=; b=i5qdDz/85fAUBVdvRJCrDJHkDIeLZLIPZTFgymeTe0pV4/3y63k5J9ZWVTMPLTkWwh Sacic+tPU6HWab1OM1nEbognKUtYFpvIfZOWgO8TnIpveDJnuGVinSCxjVWIbN0s0a+l nqXJ6oC917/iclBGUO6y7n+L8BKBPVfjL9Jqow7kyQQ5V5nh6YCfEQwl7jZ7Eeiaf2AU DXnLYPV+JTsro4q7Y37WKI8VsqfkdnT3h8dOEV0BoK+sCGUHRXmovXRRHQR5iYxVd3V/ POLKe8zletgRGCCaJRg5u1Avad/4xWQ8TtItxRDMJLRxfrVl7mHWTgmtyJTRzI6WkAw+ f93A== X-Gm-Message-State: AAQBX9dgYzWtJpkN/QhYmlUO9dTYqS3mHmKhutpMY1ySjl61KUyjrDSZ vQr1Y2NqA2b939+kil1/8NEK5A== X-Google-Smtp-Source: AKy350b2A/7gi7zGaw6PRXH8Rd+IqR08QUgew+pmTLB3ltkRaTbuA3Cb6mqvvLpmR7pgdjEM7q+jSg== X-Received: by 2002:ac2:424a:0:b0:4eb:e8e:4139 with SMTP id m10-20020ac2424a000000b004eb0e8e4139mr3878783lfl.2.1682335318680; Mon, 24 Apr 2023 04:21:58 -0700 (PDT) Received: from localhost.localdomain ([80.250.80.22]) by smtp.gmail.com with ESMTPSA id j1-20020a19f501000000b004eff0bcb276sm415542lfb.7.2023.04.24.04.21.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 04:21:58 -0700 (PDT) From: Viktor Prutyanov To: mst@redhat.com, jasowang@redhat.com, marcel.apfelbaum@gmail.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com Cc: philmd@linaro.org, qemu-devel@nongnu.org, yan@daynix.com, yuri.benditovich@daynix.com, Viktor Prutyanov Subject: [RFC PATCH 2/4] virtio-pci: add handling of ATS state change Date: Mon, 24 Apr 2023 14:21:45 +0300 Message-Id: <20230424112147.17083-3-viktor@daynix.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230424112147.17083-1-viktor@daynix.com> References: <20230424112147.17083-1-viktor@daynix.com> MIME-Version: 1.0 Received-SPF: none client-ip=2a00:1450:4864:20::136; envelope-from=viktor@daynix.com; helo=mail-lf1-x136.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Guest may enable or disable ATS for the device. Add logic for handling these events. Signed-off-by: Viktor Prutyanov --- hw/virtio/virtio-pci.c | 12 ++++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 14 insertions(+) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 247325c193..70f63a4986 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1731,6 +1731,17 @@ static void virtio_pci_device_write(void *opaque, hwaddr addr, } } +static void virtio_pci_ats_ctrl_trigger(PCIDevice *pci_dev, bool enable) +{ + VirtIOPCIProxy *proxy = VIRTIO_PCI(pci_dev); + VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus); + + vdev->ats_enabled = enable; + + if (vdev->ats_ctrl_trigger) + vdev->ats_ctrl_trigger(enable, vdev); +} + static void virtio_pci_modern_regions_init(VirtIOPCIProxy *proxy, const char *vdev_name) { @@ -2166,6 +2177,7 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) if (proxy->flags & VIRTIO_PCI_FLAG_ATS) { pcie_ats_init(pci_dev, last_pcie_cap_offset, proxy->flags & VIRTIO_PCI_FLAG_ATS_PAGE_ALIGNED); + pci_dev->ats_ctrl_trigger = virtio_pci_ats_ctrl_trigger; last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF; } diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index 77c6c55929..2812561aae 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -155,6 +155,8 @@ struct VirtIODevice QLIST_HEAD(, VirtQueue) *vector_queues; QTAILQ_ENTRY(VirtIODevice) next; EventNotifier config_notifier; + void (*ats_ctrl_trigger)(bool enable, VirtIODevice *vdev); + bool ats_enabled; }; struct VirtioDeviceClass { From patchwork Mon Apr 24 11:21:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viktor Prutyanov X-Patchwork-Id: 13222041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A12E8C77B61 for ; Mon, 24 Apr 2023 11:23:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pquGw-0000rE-KU; Mon, 24 Apr 2023 07:22:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pquGr-0000p1-7V for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:15 -0400 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pquGl-00056f-Mx for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:13 -0400 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-4edc63c82d1so4378389e87.0 for ; Mon, 24 Apr 2023 04:22:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20221208.gappssmtp.com; s=20221208; t=1682335319; x=1684927319; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CwD/L5o2/pRtObZ6GaVbf9uZzAG747kqUQpZ52beJ+0=; b=YbPswaCEfTvIyRqScUWXa3VJwc0ZVesbG/XWWU24qY5AL2guxiWI7ZcPEQZhnO3wLa /lUi8CyxJMa2TdGeHvMow/dJxGY6RCTUK3gAZSgs7hKCEyPI73MgHrzC5P63hbgSzEM8 8AX7ssrM4BVsqtCBAQ+R9+L7VFvsj6NrK9sPTLYAxqkaT4fDHIMbVz3cSi1XeMck2JD7 SvIYaQF43fuY+wfO3qxWgWxYqvjlPJqTbUuSRtI9HKm8Mqx4o8vVqsfW7p6WQ92xqWRG wwY4w1ge7Ey1jMEGFLXsOSKi90Zi/OXMZgFvd7GahsqrrGU86aQeTrSW0XLlRCnt0daq qkeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682335319; x=1684927319; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CwD/L5o2/pRtObZ6GaVbf9uZzAG747kqUQpZ52beJ+0=; b=Vz0lVKec3PzxmwJgcWM20PvJayC6vr6mDhta6aHbkHZYcJJyqiR7A/ycXmYfS3XdtA HqbCxwB3DFw4/0GDOWaGucOJmAkdgAHIrcpaEwRwElloSoARgkfJn9YgZvPw9es4RLrv FZ8NoLAPUTf1xWJXeKpTSrd5nTcUE83eSsTjX7KO9qcXx+DkPI3HovIIi6L95lpmFaGv tqvMXI6woutMXH94JSFb83f0or7UQYBGknBtB5bkgvNPWWnoXxOb1wKdMO4KBIaY6VjM mtmm9ji6M5QL0DRJ05MfkhC2WE/PrG1YQX6AdpE7O1xdpFtOPq3L0JOSYCwYK+3D3dWp +yLA== X-Gm-Message-State: AAQBX9dd6cplTIs20fMJ1breqdi3Rb9iks9L/Awhi1Ql/ecxlQS8Zn8/ yWRgtF9vrnCQUzqWnJWPII8wwA== X-Google-Smtp-Source: AKy350awPhHPM3dijDynX5CBhrdIcVS7YvxvZwmZ4BtQR1BcfHKRceZHhwCaxDuEluyb5aDJJyJA/Q== X-Received: by 2002:ac2:5ecf:0:b0:4ec:a786:1f23 with SMTP id d15-20020ac25ecf000000b004eca7861f23mr3419143lfq.59.1682335319416; Mon, 24 Apr 2023 04:21:59 -0700 (PDT) Received: from localhost.localdomain ([80.250.80.22]) by smtp.gmail.com with ESMTPSA id j1-20020a19f501000000b004eff0bcb276sm415542lfb.7.2023.04.24.04.21.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 04:21:58 -0700 (PDT) From: Viktor Prutyanov To: mst@redhat.com, jasowang@redhat.com, marcel.apfelbaum@gmail.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com Cc: philmd@linaro.org, qemu-devel@nongnu.org, yan@daynix.com, yuri.benditovich@daynix.com, Viktor Prutyanov Subject: [RFC PATCH 3/4] memory: add interface for triggering IOMMU notify_flag_changed handler Date: Mon, 24 Apr 2023 14:21:46 +0300 Message-Id: <20230424112147.17083-4-viktor@daynix.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230424112147.17083-1-viktor@daynix.com> References: <20230424112147.17083-1-viktor@daynix.com> MIME-Version: 1.0 Received-SPF: none client-ip=2a00:1450:4864:20::12d; envelope-from=viktor@daynix.com; helo=mail-lf1-x12d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This interface helps if IOMMU notify_flag_changed should be triggered after changing IOMMU notifier flags in the runtime. Signed-off-by: Viktor Prutyanov --- include/exec/memory.h | 2 ++ softmmu/memory.c | 12 ++++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 2e602a2fad..c8d5f69add 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1772,6 +1772,8 @@ void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n); void memory_region_unregister_iommu_notifier(MemoryRegion *mr, IOMMUNotifier *n); +void memory_region_iommu_notify_flags_changed(MemoryRegion *mr); + /** * memory_region_iommu_get_attr: return an IOMMU attr if get_attr() is * defined on the IOMMU. diff --git a/softmmu/memory.c b/softmmu/memory.c index 9d64efca26..e4a53cde6c 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1950,6 +1950,18 @@ void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) } } +void memory_region_iommu_notify_flags_changed(MemoryRegion *mr) +{ + IOMMUMemoryRegion *iommu_mr; + + if (mr->alias) { + memory_region_iommu_notify_flags_changed(mr->alias); + return; + } + iommu_mr = IOMMU_MEMORY_REGION(mr); + memory_region_update_iommu_notify_flags(iommu_mr, NULL); +} + void memory_region_unregister_iommu_notifier(MemoryRegion *mr, IOMMUNotifier *n) { From patchwork Mon Apr 24 11:21:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viktor Prutyanov X-Patchwork-Id: 13222043 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DEC67C7618E for ; Mon, 24 Apr 2023 11:23:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pquGx-0000rF-88; Mon, 24 Apr 2023 07:22:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pquGt-0000pL-Jj for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:15 -0400 Received: from mail-lf1-x135.google.com ([2a00:1450:4864:20::135]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pquGn-00056p-Hw for qemu-devel@nongnu.org; Mon, 24 Apr 2023 07:22:15 -0400 Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-4ec94eb6dcaso4574784e87.3 for ; Mon, 24 Apr 2023 04:22:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20221208.gappssmtp.com; s=20221208; t=1682335320; x=1684927320; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QDgBd0KM6E87x+Aap6dd/kXysuUUzOZolqfyQRQpgT4=; b=u57P0ANgqYDoVEtqclKYBCNf2qFObtVMRYwOWWF2bvjjnGkKSKFFEMMglPkqmSsF8P rhixtU5SAqBbfc3TCTXK0hL9RN0r3y5B65DWUk16ufxLuSl/4luLy7qINS1txgUh/N67 RsK1qwx4I/35NhLrGYmbWHLJHj1AEwFk0/NOrzJRo2uYHg6+A3No6jjbY3+Tbk50DNMT oXpW55Yupjpi6w+fx1cBYLiRwPHFGkriMae3AlmKI1mLmqxwrjERs6FrioRGpYgRpF3s H0YXyTKutT6oGt7s+mFgcLOS9K9F6hBf7fQtXEDbRuQJqyk7T96l5oOubVWJaTdqOemW yYKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682335320; x=1684927320; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QDgBd0KM6E87x+Aap6dd/kXysuUUzOZolqfyQRQpgT4=; b=eNMQzi44X/BtosBMcMj1uD3S/ebBN/xEBKpRGzrJ16rc6I0Lx3rCcYc26q2RQ8DdwB QwNZk+9wFkd4d5wrvHg8qJA3WuOST88ZeOqCAy6Z7L2bpSJdvec2KX6hxwn4+CgNupJF Mdlfcj+DG34vX9BuswmUS5yqQuGE3SWOgjZjyIfw2NNYmnCMiwlQUOO2AMdOUN5gkUOe zP29Yg4FEXLkaSKoUETJSe6ZIXBqtnvkYJWD35BDJKbdS1fFEhSPY5L+tq8/fcphx/EP FIqvCbONJM1CAKeWcqCRPLdrtkUHQcLzZkh7lk6cS0B40xmrRJImovaikEtdbDFD4KYE cKcw== X-Gm-Message-State: AAQBX9fQb7G785OX/nnpHUSN6Sx0B5I1nzX2kJj2fLXIDBM/RLY4UDGg EAF9o7F2DJn0MEJMFtWZ6gK26Q== X-Google-Smtp-Source: AKy350ak0KCJQF230x3JvHEY99VEfqeyzGDYXNR4o54v5nK06xxba3SNHDbpAncS0MBeGL0QTV95Yg== X-Received: by 2002:a19:c518:0:b0:4eb:3b58:c558 with SMTP id w24-20020a19c518000000b004eb3b58c558mr3027695lfe.59.1682335320193; Mon, 24 Apr 2023 04:22:00 -0700 (PDT) Received: from localhost.localdomain ([80.250.80.22]) by smtp.gmail.com with ESMTPSA id j1-20020a19f501000000b004eff0bcb276sm415542lfb.7.2023.04.24.04.21.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Apr 2023 04:21:59 -0700 (PDT) From: Viktor Prutyanov To: mst@redhat.com, jasowang@redhat.com, marcel.apfelbaum@gmail.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com Cc: philmd@linaro.org, qemu-devel@nongnu.org, yan@daynix.com, yuri.benditovich@daynix.com, Viktor Prutyanov Subject: [RFC PATCH 4/4] vhost: register and change IOMMU flag depending on ATS state Date: Mon, 24 Apr 2023 14:21:47 +0300 Message-Id: <20230424112147.17083-5-viktor@daynix.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20230424112147.17083-1-viktor@daynix.com> References: <20230424112147.17083-1-viktor@daynix.com> MIME-Version: 1.0 Received-SPF: none client-ip=2a00:1450:4864:20::135; envelope-from=viktor@daynix.com; helo=mail-lf1-x135.google.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The guest can disable or never enable ATS. In these cases, Device-TLB can't be used even if enabled in QEMU. So, check ATS state before registering IOMMU notifier and select flag depending on that. Also, change IOMMU notifier flag if ATS state is changed. Signed-off-by: Viktor Prutyanov --- hw/virtio/vhost.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index eb8c4c378c..1b14937020 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -796,7 +796,9 @@ static void vhost_iommu_region_add(MemoryListener *listener, iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, MEMTXATTRS_UNSPECIFIED); iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, - IOMMU_NOTIFIER_DEVIOTLB_UNMAP, + dev->vdev->ats_enabled ? + IOMMU_NOTIFIER_DEVIOTLB_UNMAP : + IOMMU_NOTIFIER_UNMAP, section->offset_within_region, int128_get64(end), iommu_idx); @@ -804,7 +806,8 @@ static void vhost_iommu_region_add(MemoryListener *listener, iommu->iommu_offset = section->offset_within_address_space - section->offset_within_region; iommu->hdev = dev; - ret = memory_region_register_iommu_notifier(section->mr, &iommu->n, NULL); + ret = memory_region_register_iommu_notifier(section->mr, &iommu->n, + dev->vdev->ats_enabled ? NULL : &error_fatal); if (ret) { /* * Some vIOMMUs do not support dev-iotlb yet. If so, try to use the @@ -818,6 +821,19 @@ static void vhost_iommu_region_add(MemoryListener *listener, /* TODO: can replay help performance here? */ } +static void vhost_deviotlb_ctrl_trigger(bool enable, struct VirtIODevice *vdev) +{ + VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(vdev); + struct vhost_dev *hdev = vdc->get_vhost(vdev); + struct vhost_iommu *iommu; + + QLIST_FOREACH(iommu, &hdev->iommu_list, iommu_next) { + iommu->n.notifier_flags = enable ? + IOMMU_NOTIFIER_DEVIOTLB_UNMAP : IOMMU_NOTIFIER_UNMAP; + memory_region_iommu_notify_flags_changed(iommu->mr); + } +} + static void vhost_iommu_region_del(MemoryListener *listener, MemoryRegionSection *section) { @@ -2000,6 +2016,8 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings) struct vhost_virtqueue *vq = hdev->vqs + i; vhost_device_iotlb_miss(hdev, vq->used_phys, true); } + + vdev->ats_ctrl_trigger = vhost_deviotlb_ctrl_trigger; } vhost_start_config_intr(hdev); return 0; @@ -2055,6 +2073,7 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings) hdev->vhost_ops->vhost_set_iotlb_callback(hdev, false); } memory_listener_unregister(&hdev->iommu_listener); + vdev->ats_ctrl_trigger = NULL; } vhost_stop_config_intr(hdev); vhost_log_put(hdev, true);