From patchwork Wed Dec 18 07:42:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13913134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B06DCE77183 for ; Wed, 18 Dec 2024 07:43:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNoho-0007jS-0s; Wed, 18 Dec 2024 02:42:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNohm-0007jK-Nq for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:42:50 -0500 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tNohk-0006hY-KQ for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:42:50 -0500 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-728eedfca37so6321322b3a.2 for ; Tue, 17 Dec 2024 23:42:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734507767; x=1735112567; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bRTDyBdfnK9kAprnP+Hh3qG7LUon4ziNP1QIA/V4/Ww=; b=OkhgyRatjPJGOAUol+614WJ5NEiWOoyp4dZPqK6VguJfjgwBQOYmaCwZmt67nxqa+x QiWsAKSA0ZFkU/7pndK0BvjbUIjM6vhMLaLJ2F0f2fmWRYQj3OkVV+8JG+diWTQArI3z fuPXLdo+Bn3VGSrGw2eMVsc4PukodPGf1n7w2eTMe5zPy2wFxFRbv6Lw6srTbosEJAbu 7w89T7NnnBWD7Rsu+I5Rc7+96vgNz1YcpITxWvipSHd9fGojfFr5J67k5zU0g5Z6Pufp 32/UyGT7AnpNblXELsusABQ0euDOevtwNoaOq9wuQPBYfI2hpC1eC3+kd3awvqyB+qhu LUUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734507767; x=1735112567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bRTDyBdfnK9kAprnP+Hh3qG7LUon4ziNP1QIA/V4/Ww=; b=EzJHmdbvq4py7KAD48/v9uuutvmHCi9qN2THcuUkqcl3kuQFBZbhe2smTL+2iUCmG2 pST9UfZ504Zbozzo3hLpxSRIz74dRV/P0Unte+HZwzQe7Qm6ouu4rnQqW3HV4QMuPcbe borvmUlxAC/ysYw/PUNZb4eXk3bf9mftOji7fgMvw/0x4W5m+i8L2Y3ZjCcY9MYpH/WN i2099DBTlBp1cknYFCmK8euX2IZLQZGPQEFtYMD+L3mpNf5+IFwVGZFJ+n16gIbjn84R airDkze7oKBaUSCdG4pS3rPYg5GFE4FHIjYFZ3JHQoIIsYmJJot4iPzRFsj2LbgDqOsi YhyQ== X-Gm-Message-State: AOJu0YzGnXHIO/ELjEzAQj++aNkuE/TxkBi57Rvw0lyxHKRdnWFNdoi5 TFPm3M0v0/FR/WxEwXQLvEpUX2snLpQxILljofBjjRUunKAlEHWo+hWYOQ== X-Gm-Gg: ASbGncu66HZ2e+ZXSDC/lrLenvPK9aAGR8EcXTfmpPAOIdGYQpu2nKwa9uNDCTeAQrw XPpUsuTR1XqtqNI6elSCe/0wA1uWpf8ZXBfbiRXLtLFXHAiPFcvlZmfuiQ9LO3SjknA7oZzf4Fd 5xbsA8zxV1xEyfo0YsrPPGQkvneVo6dQislv7rjs9ScL5fdRq1yDY5DR54gNvPTpiwA5OfIWbnO ANO/30V3LGTcAHwpbLFk/y6fzzaveDJGeEsax9WG2s0TQD4rclhPZ5dOvswGvQvAaeqapvfu8aj LqpXV3UVnA== X-Google-Smtp-Source: AGHT+IEkBa11aQAthEAHT3iqR8+FsIc7vO78JQzpOFjXyOhemtexAb4tpwjEh7tgs7YLBRr6C7RuSA== X-Received: by 2002:a05:6a00:2e9a:b0:725:cfd0:dffa with SMTP id d2e1a72fcca58-72a8d223ad7mr3138470b3a.5.1734507766756; Tue, 17 Dec 2024 23:42:46 -0800 (PST) Received: from wheely.local0.net (14-200-18-130.tpgi.com.au. [14.200.18.130]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918af0caesm7900933b3a.86.2024.12.17.23.42.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Dec 2024 23:42:46 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , Fabiano Rosas , Laurent Vivier , Paolo Bonzini Subject: [PATCH 1/5] qtest/pci: Enforce balanced iomap/unmap Date: Wed, 18 Dec 2024 17:42:27 +1000 Message-ID: <20241218074232.1784427-2-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241218074232.1784427-1-npiggin@gmail.com> References: <20241218074232.1784427-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=npiggin@gmail.com; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add assertions to ensure a BAR is not mapped twice, and only previously mapped BARs are unmapped. This can help catch some bugs. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/ahci.h | 1 + tests/qtest/libqos/pci.h | 2 ++ tests/qtest/libqos/virtio-pci.h | 1 + tests/qtest/ahci-test.c | 2 ++ tests/qtest/libqos/ahci.c | 6 ++++++ tests/qtest/libqos/pci.c | 32 +++++++++++++++++++++++++++++++- tests/qtest/libqos/virtio-pci.c | 6 +++++- 7 files changed, 48 insertions(+), 2 deletions(-) diff --git a/tests/qtest/libqos/ahci.h b/tests/qtest/libqos/ahci.h index a0487a1557d..5d7e26aee2a 100644 --- a/tests/qtest/libqos/ahci.h +++ b/tests/qtest/libqos/ahci.h @@ -575,6 +575,7 @@ QPCIDevice *get_ahci_device(QTestState *qts, uint32_t *fingerprint); void free_ahci_device(QPCIDevice *dev); void ahci_pci_enable(AHCIQState *ahci); void start_ahci_device(AHCIQState *ahci); +void stop_ahci_device(AHCIQState *ahci); void ahci_hba_enable(AHCIQState *ahci); /* Port Management */ diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h index 83896145235..9dc82ea723a 100644 --- a/tests/qtest/libqos/pci.h +++ b/tests/qtest/libqos/pci.h @@ -65,6 +65,8 @@ struct QPCIDevice { QPCIBus *bus; int devfn; + bool bars_mapped[6]; + QPCIBar bars[6]; bool msix_enabled; QPCIBar msix_table_bar, msix_pba_bar; uint64_t msix_table_off, msix_pba_off; diff --git a/tests/qtest/libqos/virtio-pci.h b/tests/qtest/libqos/virtio-pci.h index f5115cacba2..efdf904b254 100644 --- a/tests/qtest/libqos/virtio-pci.h +++ b/tests/qtest/libqos/virtio-pci.h @@ -26,6 +26,7 @@ typedef struct QVirtioPCIDevice { uint64_t config_msix_addr; uint32_t config_msix_data; + bool enabled; int bar_idx; /* VIRTIO 1.0 */ diff --git a/tests/qtest/ahci-test.c b/tests/qtest/ahci-test.c index 5a1923f721b..b3dae7a8ce4 100644 --- a/tests/qtest/ahci-test.c +++ b/tests/qtest/ahci-test.c @@ -1483,6 +1483,8 @@ static void test_reset_pending_callback(void) /* Wait for throttled write to finish. */ sleep(1); + stop_ahci_device(ahci); + /* Start again. */ ahci_clean_mem(ahci); ahci_pci_enable(ahci); diff --git a/tests/qtest/libqos/ahci.c b/tests/qtest/libqos/ahci.c index 34a75b7f43b..cfc435b6663 100644 --- a/tests/qtest/libqos/ahci.c +++ b/tests/qtest/libqos/ahci.c @@ -217,6 +217,12 @@ void start_ahci_device(AHCIQState *ahci) qpci_device_enable(ahci->dev); } +void stop_ahci_device(AHCIQState *ahci) +{ + /* Map AHCI's ABAR (BAR5) */ + qpci_iounmap(ahci->dev, ahci->hba_bar); +} + /** * Test and initialize the AHCI's HBA memory areas. * Initialize and start any ports with devices attached. diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index b23d72346b6..a42ca08261d 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -93,12 +93,17 @@ QPCIDevice *qpci_device_find(QPCIBus *bus, int devfn) void qpci_device_init(QPCIDevice *dev, QPCIBus *bus, QPCIAddress *addr) { uint16_t vendor_id, device_id; + int i; qpci_device_set(dev, bus, addr->devfn); vendor_id = qpci_config_readw(dev, PCI_VENDOR_ID); device_id = qpci_config_readw(dev, PCI_DEVICE_ID); g_assert(!addr->vendor_id || vendor_id == addr->vendor_id); g_assert(!addr->device_id || device_id == addr->device_id); + + for (i = 0; i < 6; i++) { + g_assert(!dev->bars_mapped[i]); + } } static uint8_t qpci_find_resource_reserve_capability(QPCIDevice *dev) @@ -531,6 +536,8 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr) uint64_t loc; g_assert(barno >= 0 && barno <= 5); + g_assert(!dev->bars_mapped[barno]); + bar_reg = bar_reg_map[barno]; qpci_config_writel(dev, bar_reg, 0xFFFFFFFF); @@ -574,12 +581,35 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr) } bar.addr = loc; + + dev->bars_mapped[barno] = true; + dev->bars[barno] = bar; + return bar; } void qpci_iounmap(QPCIDevice *dev, QPCIBar bar) { - /* FIXME */ + static const int bar_reg_map[] = { + PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2, + PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5, + }; + int bar_reg; + int i; + + for (i = 0; i < 6; i++) { + if (!dev->bars_mapped[i]) { + continue; + } + if (dev->bars[i].addr == bar.addr) { + dev->bars_mapped[i] = false; + bar_reg = bar_reg_map[i]; + qpci_config_writel(dev, bar_reg, 0xFFFFFFFF); + /* FIXME: the address space is leaked */ + return; + } + } + g_assert_not_reached(); } QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr) diff --git a/tests/qtest/libqos/virtio-pci.c b/tests/qtest/libqos/virtio-pci.c index 485b8f6b7e0..2b59fb181c9 100644 --- a/tests/qtest/libqos/virtio-pci.c +++ b/tests/qtest/libqos/virtio-pci.c @@ -304,11 +304,15 @@ void qvirtio_pci_device_enable(QVirtioPCIDevice *d) { qpci_device_enable(d->pdev); d->bar = qpci_iomap(d->pdev, d->bar_idx, NULL); + d->enabled = true; } void qvirtio_pci_device_disable(QVirtioPCIDevice *d) { - qpci_iounmap(d->pdev, d->bar); + if (d->enabled) { + qpci_iounmap(d->pdev, d->bar); + d->enabled = false; + } } void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci, From patchwork Wed Dec 18 07:42:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13913131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 939A3E77183 for ; Wed, 18 Dec 2024 07:43:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNohv-0007kG-4Z; Wed, 18 Dec 2024 02:43:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNohr-0007ju-8o for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:42:55 -0500 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tNohp-0006i7-Bm for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:42:54 -0500 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-725f4623df7so5498256b3a.2 for ; Tue, 17 Dec 2024 23:42:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734507771; x=1735112571; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e6QI+StavfES5vlYy9klhZaUtTtmII/hVb11fVMVfTA=; b=SxDDSFSb5+x8QIA5kJVokxeMx9Q0BK2rtYLnGhusKCccNjw5bFHSW8aX20Lb6JwM00 UODQuhxquBt2qSFTE6j8axuIxiAM3cdvcxnmVpdW0rGqt4teU4LTnFOAJ8JoYanPyQLK DYzCvZr4cPuHlFeap+gt0Zi91ztUmKcllVKveUS8Ro6ARGBUY3uvc5cCL9v6E6a8OnxP CXvjIEGm62Wfohu6MtisafsgU2aFVvl4+AF7uP5lyGRAG52/OWkYFa2GQmAgF0wgQ5s2 doTutSB9vmaOrYcJOhJ+P3S9fs3h+5rZz/l+Tnwb0jthtem8FfIXp4a8H3Je2sc7ocLO Yozw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734507771; x=1735112571; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e6QI+StavfES5vlYy9klhZaUtTtmII/hVb11fVMVfTA=; b=Uka5/+qKzKI+FsWq0cPnOW49IDdkuYEN6Z7InSqzX2//vakXBlInghNi4hkaBzNWlZ xvS3oZVmnvDWNRC2ZL10PjOSFeI5C06MQJLPqbYjCTA/7x3MMNKBKSZlr/SeaDSe4iha hA4fheGHctvB/Hr61vMN/vWUG2a0M/tKg2AjQNUkl8VkAhlId+0fyB+bRBz/uE55QZHe u+ZFMPpuHJvwBmm59Cy4ACEqcBi5+faxBoGhyeSPqi1GLh/8/II5uzaYbWvlUKUrBwBS DK8haHxBuqDNE2sk4Zn8WVfcJ9yXlqYUi0fwlV0N8LLYFfqCvduEDaONrh5NsVMCLsCR oQGw== X-Gm-Message-State: AOJu0YxJz/axN1/TS2jv2j907LcXefFLVkXMjcYRrAccQDr9n+whtkfT qtB4NbmP10BaMm8w2YiPQNrLrLf93FT9E3GmILrRooS0k9EWn1gZO7TCOQ== X-Gm-Gg: ASbGnctoEenMsWUTdknIGGav/tGvDKB3zx9GjkCiOtlpHhUJ2ssX8+663jxjLwrpMHX 4tytlLAC/RR5Lt+r8LitrLksEZm16srxcvVjNa1YonEL0AFMWrgw1JcD+lkkANrYcgo9LVYPz60 IGiq+CGj9B+6qw8p7sqsGW0/1SVdfTS8jpRUvvhxHRpWIY486W2dwSwxk1KMPUyGzGoTmJSJd5z 0mNALdSPpWF9QBRZQj4qwXIvy+Jp7PdXHADOwLp2SJzhC67CzUuWlom+oGLfAqJAXvFZk43gxWM kiK1Pd0F+w== X-Google-Smtp-Source: AGHT+IFJ0eKApvsuy3Zy2t9apqWQMyZhLMs6XbT/YBKfzMKhqx7/C2He/Wg2VAn4/egUZICJRL+KSw== X-Received: by 2002:a05:6a00:3307:b0:725:90f9:daf9 with SMTP id d2e1a72fcca58-72a8d261072mr2725391b3a.15.1734507771477; Tue, 17 Dec 2024 23:42:51 -0800 (PST) Received: from wheely.local0.net (14-200-18-130.tpgi.com.au. [14.200.18.130]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918af0caesm7900933b3a.86.2024.12.17.23.42.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Dec 2024 23:42:50 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , Fabiano Rosas , Laurent Vivier , Paolo Bonzini Subject: [PATCH 2/5] qtest/libqos/pci: Fix qpci_msix_enable sharing bar0 Date: Wed, 18 Dec 2024 17:42:28 +1000 Message-ID: <20241218074232.1784427-3-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241218074232.1784427-1-npiggin@gmail.com> References: <20241218074232.1784427-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=npiggin@gmail.com; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Devices where the MSI-X addresses are shared with other MMIO on BAR0 can not use msi_enable because it unmaps and remaps BAR0, which interferes with device MMIO mappings. xhci-nec is one such device we would like to test msix with. Use the BAR iomap tracking structure introduced in the previous change to have qpci_misx_enable() use existing iomaps if msix bars are already mapped. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/pci.h | 1 + tests/qtest/libqos/pci.c | 40 ++++++++++++++++++++++++++++++++++------ 2 files changed, 35 insertions(+), 6 deletions(-) diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h index 9dc82ea723a..5a7b2454ad5 100644 --- a/tests/qtest/libqos/pci.h +++ b/tests/qtest/libqos/pci.h @@ -68,6 +68,7 @@ struct QPCIDevice bool bars_mapped[6]; QPCIBar bars[6]; bool msix_enabled; + bool msix_table_bar_iomap, msix_pba_bar_iomap; QPCIBar msix_table_bar, msix_pba_bar; uint64_t msix_table_off, msix_pba_off; }; diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index a42ca08261d..023c1617680 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -288,15 +288,21 @@ void qpci_msix_enable(QPCIDevice *dev) table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE); bir_table = table & PCI_MSIX_FLAGS_BIRMASK; - dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL); + if (dev->bars_mapped[bir_table]) { + dev->msix_table_bar = dev->bars[bir_table]; + } else { + dev->msix_table_bar_iomap = true; + dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL); + } dev->msix_table_off = table & ~PCI_MSIX_FLAGS_BIRMASK; table = qpci_config_readl(dev, addr + PCI_MSIX_PBA); bir_pba = table & PCI_MSIX_FLAGS_BIRMASK; - if (bir_pba != bir_table) { - dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL); + if (dev->bars_mapped[bir_pba]) { + dev->msix_pba_bar = dev->bars[bir_pba]; } else { - dev->msix_pba_bar = dev->msix_table_bar; + dev->msix_pba_bar_iomap = true; + dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL); } dev->msix_pba_off = table & ~PCI_MSIX_FLAGS_BIRMASK; @@ -307,6 +313,7 @@ void qpci_msix_disable(QPCIDevice *dev) { uint8_t addr; uint16_t val; + uint32_t table; g_assert(dev->msix_enabled); addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX, 0); @@ -315,10 +322,31 @@ void qpci_msix_disable(QPCIDevice *dev) qpci_config_writew(dev, addr + PCI_MSIX_FLAGS, val & ~PCI_MSIX_FLAGS_ENABLE); - if (dev->msix_pba_bar.addr != dev->msix_table_bar.addr) { + if (dev->msix_pba_bar_iomap) { + dev->msix_pba_bar_iomap = false; qpci_iounmap(dev, dev->msix_pba_bar); + } else { + /* + * If we had reused an existing iomap, ensure it is still mapped + * otherwise it would be a bug if it were unmapped before msix is + * disabled. A refcounting iomap implementation could avoid this + * issue entirely, but let's wait until that's needed. + */ + uint8_t bir_pba; + table = qpci_config_readl(dev, addr + PCI_MSIX_PBA); + bir_pba = table & PCI_MSIX_FLAGS_BIRMASK; + g_assert(dev->bars_mapped[bir_pba]); + } + + if (dev->msix_table_bar_iomap) { + dev->msix_table_bar_iomap = false; + qpci_iounmap(dev, dev->msix_table_bar); + } else { + uint8_t bir_table; + table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE); + bir_table = table & PCI_MSIX_FLAGS_BIRMASK; + g_assert(dev->bars_mapped[bir_table]); } - qpci_iounmap(dev, dev->msix_table_bar); dev->msix_enabled = 0; dev->msix_table_off = 0; From patchwork Wed Dec 18 07:42:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13913133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0A042E77187 for ; Wed, 18 Dec 2024 07:43:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNoiF-0007ma-6g; Wed, 18 Dec 2024 02:43:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNohw-0007ky-Ob for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:43:01 -0500 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tNohv-0006j7-2R for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:43:00 -0500 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-725ed193c9eso5288111b3a.1 for ; Tue, 17 Dec 2024 23:42:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734507776; x=1735112576; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DhQYxkte1TR03Kv00xe+YPvtM5yD9Xcv9xhHtnyiN9Q=; b=TmQrIXrSGo3zz2AUsS91MTUK+NNiK0Y881szTp2mhytDbGkB2ciJ/JEYr/fFQOcm6y B8ggUkUsDeIzPhFxmUdQsLWxHNvGChRxpBvvFmIaaeMuinVhZDjAq8nH+dLy5SkTNI5i uomkHDvt1BydEoDHFKwEfasjdSGFMH5pFCxgVpAGbxBleMs+x12BBZ5CLMg5HDn9sEQ/ 1BamklvAFRLvtV8JIVbGhTVrBDPKY2t0f9biKKOcGd1EULlynaOh2WNn4N01Zj92YIbj SuvdUaihuiy129PIlh0yhyocUiVux67Lrax/Iigjsv0fSkyxg3x1YmIw3WJZ+ib3mSsi Ey4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734507776; x=1735112576; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DhQYxkte1TR03Kv00xe+YPvtM5yD9Xcv9xhHtnyiN9Q=; b=SKW6Xl2+GfMDSeHSVB0fGaZoE3ylfgxPh4QZkdTr8BNf/KIDdnUgBJr4opkJQ3bvuH Hl5c5HxINTi6fGGUsSJAT4whdkFTaO0veArtQO6nN89E+brjtHN6kEn8ae96a3LxEQt0 cp8pOT4Mvq8bc/nuSfz01Q+x1UUaFZEtMFwNCIVgHtXJzL+LOzXXx9bfH/ShqWvv/9/f lMXaALWjUyk5ObOHtBrWAjZRuozw+wgu2fOouOwu6fhUCM0dPbqRLQh0z/au0BF7H/GE jaA01qB7XxrQSM9NLzAs3T3isyMpkOYfMWUvPCggbs4UttNxNWvXUs+j2h3qcW1Shpe4 iJmA== X-Gm-Message-State: AOJu0YyQwc/mqkO3AEWn4v3m5mpkebijGq7vFhIrlL//4SaU+CB6Z5sp ndIbegV4O8EmfgLj2IiBFbMMYEbpOX90HV1IdCUtVfo1jk8uk2MUD5Q6cw== X-Gm-Gg: ASbGncu2EBBr1znHUX0nt049FU6fQgtvQ6nWEjb8L/x4oWEUto6mBCQAMApQoSVRnGH aEwZBTJjFUw4zVvXqzvYvGL0uzlVER2iwskj/X+ZnSuiKU71iKQ/KXKBZeZVpNZvfSzjA+C0UyT Fqfjxza+DQZfCWk4tiw0lxTvq3vlIgHD5a6PeV4TcHpbUV9xpWVo0uE3tqv5nsWRZZCEotRiULR uYU8VbL7hkHubIdgQjowP6uQ2g0AZOIVQ2LZOAhMJzrT6JIb8XrjgVnm1+dnOJcriQyPSSw0PTs V9yaOw7AiA== X-Google-Smtp-Source: AGHT+IGjJW5O0zzFIkf1ALUeSbO0hZNu1IMO+2h4EsFG/Ng7jISvSniljPv106yzaHIuDra3hj7THg== X-Received: by 2002:a05:6a00:1909:b0:729:c7b:9385 with SMTP id d2e1a72fcca58-72a8d22374bmr2831416b3a.6.1734507776024; Tue, 17 Dec 2024 23:42:56 -0800 (PST) Received: from wheely.local0.net (14-200-18-130.tpgi.com.au. [14.200.18.130]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918af0caesm7900933b3a.86.2024.12.17.23.42.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Dec 2024 23:42:55 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , Fabiano Rosas , Laurent Vivier , Paolo Bonzini Subject: [PATCH 3/5] qtest/libqos/pci: Do not write to PBA memory Date: Wed, 18 Dec 2024 17:42:29 +1000 Message-ID: <20241218074232.1784427-4-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241218074232.1784427-1-npiggin@gmail.com> References: <20241218074232.1784427-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=npiggin@gmail.com; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The PCI Local Bus Specification says the result of writes to MSI-X PBA memory is undefined. QEMU implements them as no-ops, so remove the pointless write from qpci_msix_pending(). Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Cc: Dmitry Fleytman Cc: Akihiko Odaki Cc: Sriram Yagnaraman Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/pci.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index 023c1617680..a187349d30a 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -361,8 +361,6 @@ bool qpci_msix_pending(QPCIDevice *dev, uint16_t entry) g_assert(dev->msix_enabled); pba_entry = qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off + off); - qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off, - pba_entry & ~(1 << bit_n)); return (pba_entry & (1 << bit_n)) != 0; } From patchwork Wed Dec 18 07:42:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13913140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47E76E77183 for ; Wed, 18 Dec 2024 07:44:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNoiH-0007nr-KG; Wed, 18 Dec 2024 02:43:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNoi0-0007lP-CL for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:43:05 -0500 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tNohy-0006js-PR for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:43:04 -0500 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-72764c995e5so3775677b3a.2 for ; Tue, 17 Dec 2024 23:43:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734507781; x=1735112581; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZIRfQ0hXwoFmbBWphEiN9u3hsNhMmmCqqRQ4tN1OE6M=; b=fpk5Gy4S5HjzdeXbCOV4kgZdIc8sQ+Gawpkeh710Hi4Ov/iADtHWXrUvZm2Jz8PZiM JqPCIvLdx1dr+YAt5Y00TIMvWkbHh+jP/Hwa08ZQKe9VQQhRHNTa9rKfXQxOHO2HZ/i1 6SqRY0dKcpjlSQPv8EfkmrY010delkec5b5BZmQY0Q+3kSNfxtBfqXY9VDuaxzxlrmhh fS+vyb0HLC4dsIZ8IDGuBdipJxbzjsTlDQ68hbjhJWtZxCiXx7FPOvZE4wvTtvuM+ziN zgF3EaORWaSITMW/Rw1R/szz+R3lciCBHmOoVOLmNmpXysTioUllk1AQPjIDUP9CAEMI 4I1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734507781; x=1735112581; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZIRfQ0hXwoFmbBWphEiN9u3hsNhMmmCqqRQ4tN1OE6M=; b=U9omYGiu/jpKUb++P/fNCF5JPsLajrG7wJVzkTxSzKtfaN7Kor4J20D007BsL1WFZp Qnar1+eyI10tqcDxq4h1G12xpgznJ1yHz2Ngc1bX9kT85ll2n7s73hrBL90N4lyileRM laqdxwYtXpJsazWniM2RXwELkRKu4uyW8awOjEAmlhqgrr16ZAXZ+VWZQZ9x7z3q24Ig LvfAJL4vdQNErCmnayqtU/6i8bD1RJflBTMV6zEgpKaBnevTp5X6Hmg5iR25nF1vvZF8 lbizBBA0SdxJ9NKmb0zvcQywNTMhfZn+KQ5U8Winy4LotrdAGNPVRnV7aSW/bJXZHGgm v5qw== X-Gm-Message-State: AOJu0YwuTbYnCzWKx7bh0ZUGJ4Yy4xK42tYCOHnbPHDcqtJzIPe/5fn8 PUsVa6RzoBfFycfxlCkHENQcSQdQddfAAvYo9GyYO90ggRXuwMFjL/x/pQ== X-Gm-Gg: ASbGncs8Il7lnBlL8HnMyw0r605mdFjHFqp5/cIdry0+ecCxHw43+dfBvgYa19GrYjl z/owz5wX2PWaaSgywu3FpbLEGDQI/dmqhH4c8raqlXC+4qltjWD2glpr45Il+c1u+bKczTj77V1 LpK6K39LIp8DRXQ0b9Bx3FeuQ/h1EEP7pglFIv0UANWoKbCuQrZ4GTq7vch0tBQVLtzjSenFNa8 sCT+7/HlPaJufAnXhGlTlEHKtGoWhIR9HUGMahUUXE9ocvPjH6AvF+QsIUpuYd8VkcmS+6LPR2I KaeoUUcn/A== X-Google-Smtp-Source: AGHT+IEhCK/CLlWw64i+ZtrpZ4r2ECdKiHRvxOX50YDvusdew0Oi8XT2RH/2Od9LMbBlw1xGNogjcQ== X-Received: by 2002:a05:6a00:4087:b0:72a:8461:d179 with SMTP id d2e1a72fcca58-72a8d12803emr3088434b3a.1.1734507780555; Tue, 17 Dec 2024 23:43:00 -0800 (PST) Received: from wheely.local0.net (14-200-18-130.tpgi.com.au. [14.200.18.130]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918af0caesm7900933b3a.86.2024.12.17.23.42.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Dec 2024 23:43:00 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , Fabiano Rosas , Laurent Vivier , Paolo Bonzini Subject: [PATCH 4/5] qtest/e1000e|igb: Clear interrupt-cause bits after irq Date: Wed, 18 Dec 2024 17:42:30 +1000 Message-ID: <20241218074232.1784427-5-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241218074232.1784427-1-npiggin@gmail.com> References: <20241218074232.1784427-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=npiggin@gmail.com; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The e1000e and igb tests do not clear the ICR/EICR cause bits (or set auto-clear) on seeing queue interrupts, which inhibits the triggering of a new interrupt. Fix this by clearing the cause bits, and verify that the expected cause bit was set. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Cc: Dmitry Fleytman Cc: Akihiko Odaki Cc: Sriram Yagnaraman Signed-off-by: Nicholas Piggin --- tests/qtest/e1000e-test.c | 8 ++++++-- tests/qtest/igb-test.c | 8 ++++++-- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/tests/qtest/e1000e-test.c b/tests/qtest/e1000e-test.c index de9738fdb74..a69759da70e 100644 --- a/tests/qtest/e1000e-test.c +++ b/tests/qtest/e1000e-test.c @@ -64,8 +64,10 @@ static void e1000e_send_verify(QE1000E *d, int *test_sockets, QGuestAllocator *a /* Put descriptor to the ring */ e1000e_tx_ring_push(d, &descr); - /* Wait for TX WB interrupt */ + /* Wait for TX WB interrupt (this clears the MSIX PBA) */ e1000e_wait_isr(d, E1000E_TX0_MSG_ID); + /* Read ICR which clears it ready for next interrupt, assert TXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_ICR) & E1000_ICR_TXQ0); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.upper.data) & E1000_TXD_STAT_DD, ==, @@ -115,8 +117,10 @@ static void e1000e_receive_verify(QE1000E *d, int *test_sockets, QGuestAllocator /* Put descriptor to the ring */ e1000e_rx_ring_push(d, &descr); - /* Wait for TX WB interrupt */ + /* Wait for TX WB interrupt (this clears the MSIX PBA) */ e1000e_wait_isr(d, E1000E_RX0_MSG_ID); + /* Read ICR which clears it ready for next interrupt, assert RXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_ICR) & E1000_ICR_RXQ0); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.wb.upper.status_error) & diff --git a/tests/qtest/igb-test.c b/tests/qtest/igb-test.c index 3d397ea6973..2f22c4fb208 100644 --- a/tests/qtest/igb-test.c +++ b/tests/qtest/igb-test.c @@ -67,8 +67,10 @@ static void igb_send_verify(QE1000E *d, int *test_sockets, QGuestAllocator *allo /* Put descriptor to the ring */ e1000e_tx_ring_push(d, &descr); - /* Wait for TX WB interrupt */ + /* Wait for TX WB interrupt (this clears the MSIX PBA) */ e1000e_wait_isr(d, E1000E_TX0_MSG_ID); + /* Read EICR which clears it ready for next interrupt, assert TXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_EICR) & (1 << E1000E_TX0_MSG_ID)); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.wb.status) & E1000_TXD_STAT_DD, ==, @@ -118,8 +120,10 @@ static void igb_receive_verify(QE1000E *d, int *test_sockets, QGuestAllocator *a /* Put descriptor to the ring */ e1000e_rx_ring_push(d, &descr); - /* Wait for TX WB interrupt */ + /* Wait for TX WB interrupt (this clears the MSIX PBA) */ e1000e_wait_isr(d, E1000E_RX0_MSG_ID); + /* Read EICR which clears it ready for next interrupt, assert RXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_EICR) & (1 << E1000E_RX0_MSG_ID)); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.wb.upper.status_error) & From patchwork Wed Dec 18 07:42:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13913135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 15AA3E77187 for ; Wed, 18 Dec 2024 07:43:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tNoiH-0007nt-SO; Wed, 18 Dec 2024 02:43:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tNoi9-0007m9-G0 for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:43:15 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tNoi3-0006kZ-VZ for qemu-devel@nongnu.org; Wed, 18 Dec 2024 02:43:12 -0500 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-728e729562fso5308943b3a.0 for ; Tue, 17 Dec 2024 23:43:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734507785; x=1735112585; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=08v+xDHjDPrZZfAoWPfyiDvPE6FENcKOOzunpUkbwIo=; b=Z9u26s6FQbF10aK1G4MiOHA4B7H9C/WIVNxw4iEf8DzDUMeZx3X8dR5khpCJamv3Bg w59huRYCjs+Ovfsxu8g7eQc8yTqgA8GDiT0ywdbdi5o8BOwLxnH/C1xDnev+0ywayrFT uf2sH5s8fvyat3OvTXabcG6rY6rZlCQaYshjlRIf7FpExPtNElbaQ85zZY7fJP7HdzkV 1lJ+KjX3DOk6F9QbgD6tqqCLcCHZEQWyD1P1VMFUhk23kGLVTd8idDWgxVJIAgnKUV5R nbTuD5ShVDMRSB9EKNEhWrbWymr/RFGsSJykx+tDJSRy7EYbdM8LlTiPs5Vls5jKZ3re fEOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734507785; x=1735112585; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=08v+xDHjDPrZZfAoWPfyiDvPE6FENcKOOzunpUkbwIo=; b=uYkIs7uT122ZImRQ7hcTeiGktlAspa0AL7TuEIwAZCXqgDl6V3IlHiqLQsZ0J8xccz Hf3B2NHbCpgJKkCMuUiFfXUhPPLc+fO3OLDnx8gBEAN1py3PjVlIEEy6cqzRScNVBtvq /iZxE35+r2D+nuM8ayFL+Va6jfGATC+wvad3oNkR91UrXkKttVqVDEILoeHhh3zK9uOF LLmFwQ9FSfkrafZI9OIBLlXouZW878ZDvFlza/Qhup+tcbnSsjI0F+jyB1ATU7dPtvGQ cvPj74ku7CGUmQxHqgjXpgc2GbQcNq8p9s9raw4DfsEtqNhWLkkwLEnD4uXKr+EPIRzR QfYg== X-Gm-Message-State: AOJu0YxTGKrqoNaypTSDR+bPEo1WHAhsUIuLOC1R8h1bZARNLJm3T3rF fo1yLuY2ws0dNH6IyQhHDtbl3RJQGSMozy2v/ljsyqD9yaL2zQw1Hs7IOA== X-Gm-Gg: ASbGnctUorV+45VsGxy0wLdkReBWMQ3+f3TAQBhYvmCt5duzVbHrpCWiND5tGWmaG/K 4NWZ+6ozOpPHfXPavP+kaafinHyN7ue9BGNp+9O+Kr0ur0SNezclbYfPLOZy8fn2NfjZzkqeLMx JWSNOaoJ07Qyci7cmnVoSCpRIabOOVFlgrl8LbDuCW8tL3+eFaEqjyFtGS7YDrKvCZRSI5MiP// nSilTCvjSGzUT4qUwISTGehcp63e1lUjvoDDQbyK3BxL6KoVJQMcM6QHsDR8hXOyOkB7qZhrJQ8 fvSvCmiL9A== X-Google-Smtp-Source: AGHT+IGVg8Gc8pHg52oXwWAukS8IUHRQP0TWGKsu1WZE4Mgwv71gynqBFA6vuUgUdn69Xj+xvXXlgA== X-Received: by 2002:a05:6a21:1011:b0:1e1:a671:7122 with SMTP id adf61e73a8af0-1e5b4603989mr3396529637.2.1734507785053; Tue, 17 Dec 2024 23:43:05 -0800 (PST) Received: from wheely.local0.net (14-200-18-130.tpgi.com.au. [14.200.18.130]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72918af0caesm7900933b3a.86.2024.12.17.23.43.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Dec 2024 23:43:04 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , Fabiano Rosas , Laurent Vivier , Paolo Bonzini Subject: [PATCH 5/5] qtest/e1000e|igb: Fix msix to re-trigger interrupts Date: Wed, 18 Dec 2024 17:42:31 +1000 Message-ID: <20241218074232.1784427-6-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241218074232.1784427-1-npiggin@gmail.com> References: <20241218074232.1784427-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=npiggin@gmail.com; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The e1000e and igb tests don't clear the msix pending bit after waiting for it, as it is masked so the irq doesn't get sent. Failing to clear the pending interrupt means all subsequent waits for that interrupt after the first do not actually wait for an interrupt genreated by the device. This affects the multiple_transfers tests, they never actually verify more than one interrupt is generated. So for those tests, enable the msix vectors for the queue interrupts so they are delivered and the pending bit is cleared. Add assertions to ensure the masked pending tests are not used to test an interrupt multiple times. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Cc: Dmitry Fleytman Cc: Akihiko Odaki Cc: Sriram Yagnaraman Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/e1000e.h | 8 +++ tests/qtest/e1000e-test.c | 2 + tests/qtest/igb-test.c | 2 + tests/qtest/libqos/e1000e.c | 113 +++++++++++++++++++++++++++++++++++- 4 files changed, 122 insertions(+), 3 deletions(-) diff --git a/tests/qtest/libqos/e1000e.h b/tests/qtest/libqos/e1000e.h index 30643c80949..6cc1a9732b1 100644 --- a/tests/qtest/libqos/e1000e.h +++ b/tests/qtest/libqos/e1000e.h @@ -25,6 +25,9 @@ #define E1000E_RX0_MSG_ID (0) #define E1000E_TX0_MSG_ID (1) +#define E1000E_RX0_MSIX_DATA (0x12345678) +#define E1000E_TX0_MSIX_DATA (0xabcdef00) + #define E1000E_ADDRESS { 0x52, 0x54, 0x00, 0x12, 0x34, 0x56 } typedef struct QE1000E QE1000E; @@ -40,6 +43,10 @@ struct QE1000E_PCI { QPCIDevice pci_dev; QPCIBar mac_regs; QE1000E e1000e; + uint64_t msix_rx0_addr; + uint64_t msix_tx0_addr; + bool msix_found_rx0_pending; + bool msix_found_tx0_pending; }; static inline void e1000e_macreg_write(QE1000E *d, uint32_t reg, uint32_t val) @@ -57,5 +64,6 @@ static inline uint32_t e1000e_macreg_read(QE1000E *d, uint32_t reg) void e1000e_wait_isr(QE1000E *d, uint16_t msg_id); void e1000e_tx_ring_push(QE1000E *d, void *descr); void e1000e_rx_ring_push(QE1000E *d, void *descr); +void e1000e_pci_msix_enable_rxtxq_vectors(QE1000E *d, QGuestAllocator *alloc); #endif diff --git a/tests/qtest/e1000e-test.c b/tests/qtest/e1000e-test.c index a69759da70e..4921a141ffe 100644 --- a/tests/qtest/e1000e-test.c +++ b/tests/qtest/e1000e-test.c @@ -185,6 +185,8 @@ static void test_e1000e_multiple_transfers(void *obj, void *data, return; } + /* Triggering msix interrupts multiple times so must enable vectors */ + e1000e_pci_msix_enable_rxtxq_vectors(d, alloc); for (i = 0; i < iterations; i++) { e1000e_send_verify(d, data, alloc); e1000e_receive_verify(d, data, alloc); diff --git a/tests/qtest/igb-test.c b/tests/qtest/igb-test.c index 2f22c4fb208..06082cbe7ff 100644 --- a/tests/qtest/igb-test.c +++ b/tests/qtest/igb-test.c @@ -188,6 +188,8 @@ static void test_igb_multiple_transfers(void *obj, void *data, return; } + /* Triggering msix interrupts multiple times so must enable vectors */ + e1000e_pci_msix_enable_rxtxq_vectors(d, alloc); for (i = 0; i < iterations; i++) { igb_send_verify(d, data, alloc); igb_receive_verify(d, data, alloc); diff --git a/tests/qtest/libqos/e1000e.c b/tests/qtest/libqos/e1000e.c index 925654c7fd4..7b7e811bce7 100644 --- a/tests/qtest/libqos/e1000e.c +++ b/tests/qtest/libqos/e1000e.c @@ -19,6 +19,7 @@ #include "qemu/osdep.h" #include "hw/net/e1000_regs.h" #include "hw/pci/pci_ids.h" +#include "hw/pci/pci_regs.h" #include "../libqtest.h" #include "pci-pc.h" #include "qemu/sockets.h" @@ -77,16 +78,77 @@ static void e1000e_foreach_callback(QPCIDevice *dev, int devfn, void *data) g_free(dev); } +static bool e1000e_test_msix_irq(QE1000E *d, uint16_t msg_id, + uint64_t guest_msix_addr, + uint32_t msix_data) +{ + QE1000E_PCI *d_pci = container_of(d, QE1000E_PCI, e1000e); + QPCIDevice *pci_dev = &d_pci->pci_dev; + + if (msg_id == E1000E_RX0_MSG_ID) { + g_assert(!d_pci->msix_found_rx0_pending); + } else if (msg_id == E1000E_TX0_MSG_ID) { + g_assert(!d_pci->msix_found_tx0_pending); + } else { + /* Must enable MSI-X vector to test multiple messages */ + g_assert_not_reached(); + } + + if (pci_dev->msix_enabled) { + if (qpci_msix_masked(pci_dev, msg_id)) { + /* No ISR checking should be done if masked, but read anyway */ + bool p = qpci_msix_pending(pci_dev, msg_id); + if (p) { + if (msg_id == E1000E_RX0_MSG_ID) { + d_pci->msix_found_rx0_pending = true; + } else if (msg_id == E1000E_TX0_MSG_ID) { + d_pci->msix_found_tx0_pending = true; + } else { + g_assert_not_reached(); + } + } + return p; + } else { + uint32_t data = qtest_readl(pci_dev->bus->qts, guest_msix_addr); + if (data == msix_data) { + qtest_writel(pci_dev->bus->qts, guest_msix_addr, 0); + return true; + } else if (data == 0) { + return false; + } else { + g_assert_not_reached(); + } + } + } else { + g_assert_not_reached(); + } +} + void e1000e_wait_isr(QE1000E *d, uint16_t msg_id) { QE1000E_PCI *d_pci = container_of(d, QE1000E_PCI, e1000e); - guint64 end_time = g_get_monotonic_time() + 5 * G_TIME_SPAN_SECOND; + QPCIDevice *pci_dev = &d_pci->pci_dev; + uint64_t end_time = g_get_monotonic_time() + 5 * G_TIME_SPAN_SECOND; + uint64_t guest_msix_addr; + uint32_t msix_data; + + assert(pci_dev->msix_enabled); + + if (msg_id == E1000E_RX0_MSG_ID) { + guest_msix_addr = d_pci->msix_rx0_addr; + msix_data = E1000E_RX0_MSIX_DATA; + } else if (msg_id == E1000E_TX0_MSG_ID) { + guest_msix_addr = d_pci->msix_tx0_addr; + msix_data = E1000E_TX0_MSIX_DATA; + } else { + g_assert_not_reached(); + } do { - if (qpci_msix_pending(&d_pci->pci_dev, msg_id)) { + if (e1000e_test_msix_irq(d, msg_id, guest_msix_addr, msix_data)) { return; } - qtest_clock_step(d_pci->pci_dev.bus->qts, 10000); + qtest_clock_step(pci_dev->bus->qts, 10000); } while (g_get_monotonic_time() < end_time); g_error("Timeout expired"); @@ -99,6 +161,51 @@ static void e1000e_pci_destructor(QOSGraphObject *obj) qpci_msix_disable(&epci->pci_dev); } +static void e1000e_pci_msix_enable_vector(QE1000E *d, uint16_t msg_id, + uint64_t guest_msix_addr, + uint32_t msix_data) +{ + QE1000E_PCI *d_pci = container_of(d, QE1000E_PCI, e1000e); + QPCIDevice *pci_dev = &d_pci->pci_dev; + uint32_t control; + uint64_t off; + + g_assert_cmpint(msg_id , >=, 0); + g_assert_cmpint(msg_id , <, qpci_msix_table_size(pci_dev)); + + off = pci_dev->msix_table_off + (msg_id * 16); + + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_LOWER_ADDR, guest_msix_addr & ~0UL); + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_UPPER_ADDR, + (guest_msix_addr >> 32) & ~0UL); + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_DATA, msix_data); + + control = qpci_io_readl(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_VECTOR_CTRL); + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_VECTOR_CTRL, + control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT); +} + +void e1000e_pci_msix_enable_rxtxq_vectors(QE1000E *d, QGuestAllocator *alloc) +{ + QE1000E_PCI *d_pci = container_of(d, QE1000E_PCI, e1000e); + QPCIDevice *pci_dev = &d_pci->pci_dev; + + g_assert(pci_dev->msix_enabled); + + d_pci->msix_rx0_addr = guest_alloc(alloc, 4); + d_pci->msix_tx0_addr = guest_alloc(alloc, 4); + + e1000e_pci_msix_enable_vector(d, E1000E_RX0_MSG_ID, + d_pci->msix_rx0_addr, E1000E_RX0_MSIX_DATA); + e1000e_pci_msix_enable_vector(d, E1000E_TX0_MSG_ID, + d_pci->msix_tx0_addr, E1000E_TX0_MSIX_DATA); +} + static void e1000e_pci_start_hw(QOSGraphObject *obj) { QE1000E_PCI *d = (QE1000E_PCI *) obj;