From patchwork Wed Jan 15 15:01:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13940530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AC115C02180 for ; Wed, 15 Jan 2025 15:03:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tY4th-0004E2-3q; Wed, 15 Jan 2025 10:01:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tY4tf-0004D8-FX; Wed, 15 Jan 2025 10:01:31 -0500 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tY4td-0002tI-PO; Wed, 15 Jan 2025 10:01:31 -0500 Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-2166360285dso123124625ad.1; Wed, 15 Jan 2025 07:01:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736953287; x=1737558087; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bRTDyBdfnK9kAprnP+Hh3qG7LUon4ziNP1QIA/V4/Ww=; b=LqP5KzG4xj1klLDyaKBr1PDy7gLDptupJSMc7MrhJfd8B+CZ2TR28iUv74ULLwPhWm IgvJvub+PqTR4o+565e8L3cK0ldOQFjZpUj0f2FvWHmyc4fz+dO0Q7WqzgyF0X21ar0u g6pSkPoQNl0oixICt7BQxzVIjs3i73Fi3fM8v+QAy0reCKdeS3Q0lgyn0rxCNF7/8kwm dj3u6e3AsGhwEq5bSmHKSGxeOYCL7WB1iFOq1WM277oyFokl8uztfb/dMN4scqFe2vAA ey79kkFc9cXXSL7Zqj0NErIhoY+vmKqFrfkuZDmUiPbxuqELZDGGvCjIu2dX11VeTZ+D IJdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736953287; x=1737558087; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bRTDyBdfnK9kAprnP+Hh3qG7LUon4ziNP1QIA/V4/Ww=; b=FRupjqm5n49H0edGhWcpzVGiHnLM25NFSu+VsuIsvkeIjMSsRjll7ZLpBHfXrsTZUa UQcZr0jAyMvDw+JDISl6tIw94S8hrB3lX+sd/qKYP7PSJZY22LsqLiSbP/5oiKb/lDd8 n/DH5jL4Zrnx+fcCkYjikcJueiFoaftMTdNBZnFLMzVBli+pjfZOxPGQvPH1ssPhEdBZ LEhYi6802UgxyjO28VHiFoo69CMuYMTMf6aK04BBJOBF59SDpvVYpEJ7FTGTuy3H9cvH NnQoOczqazrm/xK0utm+C0b1rEdyU6Ju1y098ux5fnFhgKmlfGP6CxeT3Ey199JDSF78 AlOw== X-Forwarded-Encrypted: i=1; AJvYcCWyyZvZwygKJZHiym5fN2tz0OUVUyybiPsorcQO0WhTvr1twmTb4asVYA1M8BqXN8Ik5R9sKuZWD/0n@nongnu.org X-Gm-Message-State: AOJu0YxcVrHEU8ZXqPaJQ+fSh7p6K10GyOltJ8Efj78IXY6rnl7DUN4I z2dhwZNhTgOeecPSMDYBYyucyPR4kKznHo2m+0Lc+UFxU4wGD3gySNkHuQ== X-Gm-Gg: ASbGncsPG5Fa4povmv9bswKNQhFSCRHqDoyCU5jycVHpGnr/gw00mKQmXMDtsBPJh5X XuVcgd/J1B/FQvOUb1GZ+yjEK7gFY6AR1GkCLQCfXI73p9E7k+JAQoJ8tTw7jV3JgEIKN39fIsC pBUJyPbYHAZ2uvnDcK2ttxan7WHPGsZf7Pe10xKxiSFU+UvXSm+k3isZ2JqRkvUZG/8oLL9TA9I TjSLdYksMbpQDQVcm0gwzsne8DaNOniFfWu0ZWNOMJV4x8dcwfIBWeF5ghuAg== X-Google-Smtp-Source: AGHT+IFXaXzY6Y5xpzCSgv6aikowSpTB99cwOlYwLyAWXoXKcEo2L9NLx+DBCwXcrd4dsCS0GayUBg== X-Received: by 2002:a05:6a00:ac2:b0:725:e1de:c0bf with SMTP id d2e1a72fcca58-72d21f2dcbcmr37039713b3a.9.1736953286796; Wed, 15 Jan 2025 07:01:26 -0800 (PST) Received: from wheely.local0.net ([118.210.104.29]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72d40658a7esm9519592b3a.106.2025.01.15.07.01.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jan 2025 07:01:26 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , John Snow , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , qemu-block@nongnu.org Subject: [PATCH v2 1/5] qtest/pci: Enforce balanced iomap/unmap Date: Thu, 16 Jan 2025 01:01:07 +1000 Message-ID: <20250115150112.346497-2-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115150112.346497-1-npiggin@gmail.com> References: <20250115150112.346497-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=npiggin@gmail.com; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add assertions to ensure a BAR is not mapped twice, and only previously mapped BARs are unmapped. This can help catch some bugs. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/ahci.h | 1 + tests/qtest/libqos/pci.h | 2 ++ tests/qtest/libqos/virtio-pci.h | 1 + tests/qtest/ahci-test.c | 2 ++ tests/qtest/libqos/ahci.c | 6 ++++++ tests/qtest/libqos/pci.c | 32 +++++++++++++++++++++++++++++++- tests/qtest/libqos/virtio-pci.c | 6 +++++- 7 files changed, 48 insertions(+), 2 deletions(-) diff --git a/tests/qtest/libqos/ahci.h b/tests/qtest/libqos/ahci.h index a0487a1557d..5d7e26aee2a 100644 --- a/tests/qtest/libqos/ahci.h +++ b/tests/qtest/libqos/ahci.h @@ -575,6 +575,7 @@ QPCIDevice *get_ahci_device(QTestState *qts, uint32_t *fingerprint); void free_ahci_device(QPCIDevice *dev); void ahci_pci_enable(AHCIQState *ahci); void start_ahci_device(AHCIQState *ahci); +void stop_ahci_device(AHCIQState *ahci); void ahci_hba_enable(AHCIQState *ahci); /* Port Management */ diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h index 83896145235..9dc82ea723a 100644 --- a/tests/qtest/libqos/pci.h +++ b/tests/qtest/libqos/pci.h @@ -65,6 +65,8 @@ struct QPCIDevice { QPCIBus *bus; int devfn; + bool bars_mapped[6]; + QPCIBar bars[6]; bool msix_enabled; QPCIBar msix_table_bar, msix_pba_bar; uint64_t msix_table_off, msix_pba_off; diff --git a/tests/qtest/libqos/virtio-pci.h b/tests/qtest/libqos/virtio-pci.h index f5115cacba2..efdf904b254 100644 --- a/tests/qtest/libqos/virtio-pci.h +++ b/tests/qtest/libqos/virtio-pci.h @@ -26,6 +26,7 @@ typedef struct QVirtioPCIDevice { uint64_t config_msix_addr; uint32_t config_msix_data; + bool enabled; int bar_idx; /* VIRTIO 1.0 */ diff --git a/tests/qtest/ahci-test.c b/tests/qtest/ahci-test.c index 5a1923f721b..b3dae7a8ce4 100644 --- a/tests/qtest/ahci-test.c +++ b/tests/qtest/ahci-test.c @@ -1483,6 +1483,8 @@ static void test_reset_pending_callback(void) /* Wait for throttled write to finish. */ sleep(1); + stop_ahci_device(ahci); + /* Start again. */ ahci_clean_mem(ahci); ahci_pci_enable(ahci); diff --git a/tests/qtest/libqos/ahci.c b/tests/qtest/libqos/ahci.c index 34a75b7f43b..cfc435b6663 100644 --- a/tests/qtest/libqos/ahci.c +++ b/tests/qtest/libqos/ahci.c @@ -217,6 +217,12 @@ void start_ahci_device(AHCIQState *ahci) qpci_device_enable(ahci->dev); } +void stop_ahci_device(AHCIQState *ahci) +{ + /* Map AHCI's ABAR (BAR5) */ + qpci_iounmap(ahci->dev, ahci->hba_bar); +} + /** * Test and initialize the AHCI's HBA memory areas. * Initialize and start any ports with devices attached. diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index b23d72346b6..a42ca08261d 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -93,12 +93,17 @@ QPCIDevice *qpci_device_find(QPCIBus *bus, int devfn) void qpci_device_init(QPCIDevice *dev, QPCIBus *bus, QPCIAddress *addr) { uint16_t vendor_id, device_id; + int i; qpci_device_set(dev, bus, addr->devfn); vendor_id = qpci_config_readw(dev, PCI_VENDOR_ID); device_id = qpci_config_readw(dev, PCI_DEVICE_ID); g_assert(!addr->vendor_id || vendor_id == addr->vendor_id); g_assert(!addr->device_id || device_id == addr->device_id); + + for (i = 0; i < 6; i++) { + g_assert(!dev->bars_mapped[i]); + } } static uint8_t qpci_find_resource_reserve_capability(QPCIDevice *dev) @@ -531,6 +536,8 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr) uint64_t loc; g_assert(barno >= 0 && barno <= 5); + g_assert(!dev->bars_mapped[barno]); + bar_reg = bar_reg_map[barno]; qpci_config_writel(dev, bar_reg, 0xFFFFFFFF); @@ -574,12 +581,35 @@ QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr) } bar.addr = loc; + + dev->bars_mapped[barno] = true; + dev->bars[barno] = bar; + return bar; } void qpci_iounmap(QPCIDevice *dev, QPCIBar bar) { - /* FIXME */ + static const int bar_reg_map[] = { + PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2, + PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5, + }; + int bar_reg; + int i; + + for (i = 0; i < 6; i++) { + if (!dev->bars_mapped[i]) { + continue; + } + if (dev->bars[i].addr == bar.addr) { + dev->bars_mapped[i] = false; + bar_reg = bar_reg_map[i]; + qpci_config_writel(dev, bar_reg, 0xFFFFFFFF); + /* FIXME: the address space is leaked */ + return; + } + } + g_assert_not_reached(); } QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr) diff --git a/tests/qtest/libqos/virtio-pci.c b/tests/qtest/libqos/virtio-pci.c index 485b8f6b7e0..2b59fb181c9 100644 --- a/tests/qtest/libqos/virtio-pci.c +++ b/tests/qtest/libqos/virtio-pci.c @@ -304,11 +304,15 @@ void qvirtio_pci_device_enable(QVirtioPCIDevice *d) { qpci_device_enable(d->pdev); d->bar = qpci_iomap(d->pdev, d->bar_idx, NULL); + d->enabled = true; } void qvirtio_pci_device_disable(QVirtioPCIDevice *d) { - qpci_iounmap(d->pdev, d->bar); + if (d->enabled) { + qpci_iounmap(d->pdev, d->bar); + d->enabled = false; + } } void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci, From patchwork Wed Jan 15 15:01:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13940531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79B77C02180 for ; Wed, 15 Jan 2025 15:03:30 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tY4tm-0004FB-6K; Wed, 15 Jan 2025 10:01:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tY4tk-0004Ex-Q5; Wed, 15 Jan 2025 10:01:36 -0500 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tY4ti-0002u4-8S; Wed, 15 Jan 2025 10:01:36 -0500 Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-2ee50ffcf14so1650812a91.0; Wed, 15 Jan 2025 07:01:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736953292; x=1737558092; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e6QI+StavfES5vlYy9klhZaUtTtmII/hVb11fVMVfTA=; b=C1/L9VE8bHIjGYEShm9MxP/oAc1c7GseTkYFLSMx2fSjsfQsLQnZG6VVnFXlPbATGS eI3+EEzdCP6TX+yPmACcsSb0bivDjOGnCAcO15yQ7u4WsV0pMUrCmcD/aaQvfcSDfgl/ 1IvMD1vdkDu2Yw2n+CVix/Wl95Mkoqpd2yKsy0UdsLC9cs2TeGJGG/onvtHVWd5VK380 jX8lpYeTIy6cJc5pp+OE1N25HIl+HUB4Wj+78rnS0mBRMylMYP3Nvi6tSS2E/xgRuqIp FLzTorUPCE+2sD/os90YV8DoWWnt/A5Wwo0J4HSW9ZKYxBxL6pb/lFdXeoY4m+Za9FND AO9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736953292; x=1737558092; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e6QI+StavfES5vlYy9klhZaUtTtmII/hVb11fVMVfTA=; b=lzIIpRGcd4IZHuMUBIHjd7xKC1KzPiC2wlwBA94ntq6CgtCv+otdVa9WApYs0UbOsc /hZ6k/SdASKS7CpjFacpjEpoviRvwonHYqeamNdK5AZY/Nqt2MLLmPb3KLvCu1M30cHF kGcX4nN/Oio74HCqSL+ep/X7sQMjNLybq8pv2kzYyAR5qC4QoiCAoN8VGWvuqKP3izs4 lc1dL4DBG1fz7dXfFAwr7WtPmEINRd+HSuGXZsstbInKrg19L4MKpwRCQN86L5tUrDnT XA52YpXl5htjLw4vQ2zYuazMmH+bHRbjGywXTetM0NGBagMOJge/STELOD7xxxja68TV qwig== X-Forwarded-Encrypted: i=1; AJvYcCV+LAo29EmgUYrXxjPd2SwTy6rhjvX1tL62H2gTPYD+VHuUbkBmqnwPLil/Q9DYwHn6+30eTv7ufSeW@nongnu.org X-Gm-Message-State: AOJu0YwXReWQPelfcgVpS1CIVLltfxryM9Q/rDTyo9gy/ScFYvjn7WNM Tavu+wM+QhMsdVDp2Zvxts8fg/VKo9zNHQpJO/PPaoChZkGtK50TAJfw+A== X-Gm-Gg: ASbGncu4UQv4mvc/c26ZB8IJuwexTLKShlxAzXlTEJ0ZSapb4FkV0JgpS8dMBANPVcQ 6YLmMLO3LjGqj5XctBo6mRgkexkuvikP5r4q1zVsxChH+ZU8YDQ+IV1SGYHz5gZ0KwOfU96oqeD g4gUI23FA39j9+DFiTL03s3Eh5PnpQmU4jGQEiCtoYwsSt09Sei3YvZp1PdYZiOY+GlZCUs36sx VmK7S4VpQnsHz8ScutommzgQtOICWlyfLltIdAbm8k27k0jSjkxkpCjkY2tcw== X-Google-Smtp-Source: AGHT+IHDYgdrnuyIKi1Ffq+jXsk5UJh/wQ3U5IhAFNbKCBdAgNez/up0T5A72BS4aTMoR2futqTNKQ== X-Received: by 2002:a05:6a00:a0d:b0:726:a820:921d with SMTP id d2e1a72fcca58-72d8c6fcf40mr5240177b3a.10.1736953291904; Wed, 15 Jan 2025 07:01:31 -0800 (PST) Received: from wheely.local0.net ([118.210.104.29]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72d40658a7esm9519592b3a.106.2025.01.15.07.01.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jan 2025 07:01:31 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , John Snow , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , qemu-block@nongnu.org Subject: [PATCH v2 2/5] qtest/libqos/pci: Fix qpci_msix_enable sharing bar0 Date: Thu, 16 Jan 2025 01:01:08 +1000 Message-ID: <20250115150112.346497-3-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115150112.346497-1-npiggin@gmail.com> References: <20250115150112.346497-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=npiggin@gmail.com; helo=mail-pj1-x102c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Devices where the MSI-X addresses are shared with other MMIO on BAR0 can not use msi_enable because it unmaps and remaps BAR0, which interferes with device MMIO mappings. xhci-nec is one such device we would like to test msix with. Use the BAR iomap tracking structure introduced in the previous change to have qpci_misx_enable() use existing iomaps if msix bars are already mapped. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/pci.h | 1 + tests/qtest/libqos/pci.c | 40 ++++++++++++++++++++++++++++++++++------ 2 files changed, 35 insertions(+), 6 deletions(-) diff --git a/tests/qtest/libqos/pci.h b/tests/qtest/libqos/pci.h index 9dc82ea723a..5a7b2454ad5 100644 --- a/tests/qtest/libqos/pci.h +++ b/tests/qtest/libqos/pci.h @@ -68,6 +68,7 @@ struct QPCIDevice bool bars_mapped[6]; QPCIBar bars[6]; bool msix_enabled; + bool msix_table_bar_iomap, msix_pba_bar_iomap; QPCIBar msix_table_bar, msix_pba_bar; uint64_t msix_table_off, msix_pba_off; }; diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index a42ca08261d..023c1617680 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -288,15 +288,21 @@ void qpci_msix_enable(QPCIDevice *dev) table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE); bir_table = table & PCI_MSIX_FLAGS_BIRMASK; - dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL); + if (dev->bars_mapped[bir_table]) { + dev->msix_table_bar = dev->bars[bir_table]; + } else { + dev->msix_table_bar_iomap = true; + dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL); + } dev->msix_table_off = table & ~PCI_MSIX_FLAGS_BIRMASK; table = qpci_config_readl(dev, addr + PCI_MSIX_PBA); bir_pba = table & PCI_MSIX_FLAGS_BIRMASK; - if (bir_pba != bir_table) { - dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL); + if (dev->bars_mapped[bir_pba]) { + dev->msix_pba_bar = dev->bars[bir_pba]; } else { - dev->msix_pba_bar = dev->msix_table_bar; + dev->msix_pba_bar_iomap = true; + dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL); } dev->msix_pba_off = table & ~PCI_MSIX_FLAGS_BIRMASK; @@ -307,6 +313,7 @@ void qpci_msix_disable(QPCIDevice *dev) { uint8_t addr; uint16_t val; + uint32_t table; g_assert(dev->msix_enabled); addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX, 0); @@ -315,10 +322,31 @@ void qpci_msix_disable(QPCIDevice *dev) qpci_config_writew(dev, addr + PCI_MSIX_FLAGS, val & ~PCI_MSIX_FLAGS_ENABLE); - if (dev->msix_pba_bar.addr != dev->msix_table_bar.addr) { + if (dev->msix_pba_bar_iomap) { + dev->msix_pba_bar_iomap = false; qpci_iounmap(dev, dev->msix_pba_bar); + } else { + /* + * If we had reused an existing iomap, ensure it is still mapped + * otherwise it would be a bug if it were unmapped before msix is + * disabled. A refcounting iomap implementation could avoid this + * issue entirely, but let's wait until that's needed. + */ + uint8_t bir_pba; + table = qpci_config_readl(dev, addr + PCI_MSIX_PBA); + bir_pba = table & PCI_MSIX_FLAGS_BIRMASK; + g_assert(dev->bars_mapped[bir_pba]); + } + + if (dev->msix_table_bar_iomap) { + dev->msix_table_bar_iomap = false; + qpci_iounmap(dev, dev->msix_table_bar); + } else { + uint8_t bir_table; + table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE); + bir_table = table & PCI_MSIX_FLAGS_BIRMASK; + g_assert(dev->bars_mapped[bir_table]); } - qpci_iounmap(dev, dev->msix_table_bar); dev->msix_enabled = 0; dev->msix_table_off = 0; From patchwork Wed Jan 15 15:01:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13940528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F352C02180 for ; Wed, 15 Jan 2025 15:02:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tY4tq-0004Gp-St; Wed, 15 Jan 2025 10:01:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tY4to-0004GG-Lz; Wed, 15 Jan 2025 10:01:40 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tY4tn-0002uu-47; Wed, 15 Jan 2025 10:01:40 -0500 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-21644aca3a0so152231395ad.3; Wed, 15 Jan 2025 07:01:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736953297; x=1737558097; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DhQYxkte1TR03Kv00xe+YPvtM5yD9Xcv9xhHtnyiN9Q=; b=TxlI5p2QMoXCS8KWIjGtOMVeMEZe+WfSwcdkNrDICAVUnjNL4HcPcs3cBngVjZBDcP 71WVvZw9rd9Nolt7M66Z8Rf+QCMW5qY89LymaqTqF3GVzUrOQGucEO+vQL5tZsQNuqBb xvdeufb28QzBql6Irw2TL+ASPCyG5IC69Q4myQPwg53t83MfZutJ8cH2plk8MjQFq8uj Mhuct1EQSskb+M78aNg9t8jyW0N/boA3y965DJnTUoywzZrO9+8zBlWF0BmXMSTehNhp re4ktZPTnN3GtAjPNajmRfsFfiAGI1FRYGc7/6/8e4n6UXZj16W+5Nx31+T6SH5mRGKK rMsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736953297; x=1737558097; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DhQYxkte1TR03Kv00xe+YPvtM5yD9Xcv9xhHtnyiN9Q=; b=Ibe2AGcmWv/0MHSx4VPA0BbVYhaZVMLHBnOKj15RinCEbaReVtNr6b/Rw5dMFkS/5X P5gQLsfEFtkQ6+Ta52nWM4qywCKXAGCgCWwWSF+AtyTW0JunhpVZR0cPB8bBFr1cg3ml DELKoXuVttvpQJneo1siDOv1uDZAn20CLKU4JNVaTp1xSpRG4Jn6R819PU8NgnPJLkgY TV2EVNxSfHRx8kCflTa5aVMIXEZuPvjdAKcBkh/gwiHCff7le+z356hdqn/8aOr/qpm+ VSjPJLeu1GaWTSWzS6Plzr/Ax6meCsI7Pw1ESila8dtCCg10vo9vuLKdFJAgm9f813Lv 2lCw== X-Forwarded-Encrypted: i=1; AJvYcCVsKlqgR0IChyDkn5p5ILdtwckgHcaCktatcAsQh+NgDfMZdNe+z5t5RHwOS5BCulwDhJreFse4NkJT@nongnu.org X-Gm-Message-State: AOJu0Yxj+JP9f8NmpdOKqRBkpRuwGEN6fMvGpsyv25VcxvYKUgUEZkGH 8CsUInVmF66TCCYeEnLacB5Ss83GMtWC4q9zqZNtpuYseCyom/d3FhFHoA== X-Gm-Gg: ASbGnct8LzhNd1RbvzanvU05P29iLJINF/kOIRQFSDVcu56pbbnwFTRSOsbqIZxmwCm U4sT2QqbDiYVE0BvtsGAfWa5TfFXZGf6ZTo/WCch4gCQT7lgzFk1E1dUIYlfv4e8Ioosb4XeDHi rliFeg0/TS8L+kTJCu4oWX5ZRuLy9bNRvS6PTJojiR+jw5DbkbGRn85U2ffJzGSQQB7Y1tTDuJl DGQbQff1bMU2kiU13ohIYXcReg9WseTZfHQIxnZJ7zYAiCVxf0X0q60dQRAIg== X-Google-Smtp-Source: AGHT+IGinQGv5Bp2toKtZBhGVwp2SkkBmyPIea5EtoB8LFvficmge50GnB6ywsZyacHdJN/pjDuc+g== X-Received: by 2002:a05:6a00:9a0:b0:725:cfd0:dffa with SMTP id d2e1a72fcca58-72d21f453e4mr40122714b3a.5.1736953297019; Wed, 15 Jan 2025 07:01:37 -0800 (PST) Received: from wheely.local0.net ([118.210.104.29]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72d40658a7esm9519592b3a.106.2025.01.15.07.01.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jan 2025 07:01:36 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , John Snow , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , qemu-block@nongnu.org Subject: [PATCH v2 3/5] qtest/libqos/pci: Do not write to PBA memory Date: Thu, 16 Jan 2025 01:01:09 +1000 Message-ID: <20250115150112.346497-4-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115150112.346497-1-npiggin@gmail.com> References: <20250115150112.346497-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=npiggin@gmail.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The PCI Local Bus Specification says the result of writes to MSI-X PBA memory is undefined. QEMU implements them as no-ops, so remove the pointless write from qpci_msix_pending(). Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Cc: Dmitry Fleytman Cc: Akihiko Odaki Cc: Sriram Yagnaraman Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/pci.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/tests/qtest/libqos/pci.c b/tests/qtest/libqos/pci.c index 023c1617680..a187349d30a 100644 --- a/tests/qtest/libqos/pci.c +++ b/tests/qtest/libqos/pci.c @@ -361,8 +361,6 @@ bool qpci_msix_pending(QPCIDevice *dev, uint16_t entry) g_assert(dev->msix_enabled); pba_entry = qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off + off); - qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off, - pba_entry & ~(1 << bit_n)); return (pba_entry & (1 << bit_n)) != 0; } From patchwork Wed Jan 15 15:01:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13940529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F0696C02180 for ; Wed, 15 Jan 2025 15:02:54 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tY4tu-0004Ha-Tu; Wed, 15 Jan 2025 10:01:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tY4tt-0004HL-TD; Wed, 15 Jan 2025 10:01:45 -0500 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tY4ts-0002vs-8v; Wed, 15 Jan 2025 10:01:45 -0500 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-2161eb94cceso85107665ad.2; Wed, 15 Jan 2025 07:01:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736953302; x=1737558102; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fHQKJ7ZE130ntDkhQLA0rjO4bmRpsWSpyiW+ZgLoauA=; b=Q5XjKDG5fR4f6InVeHcfrUqUEH7nTkZ9U/6yjwTdU9sEOUl8hPGKXyImsq03Vqk02r N/J0+iqm+fuAzJMxgkqUfquaYyzgAVn3sxOeSKZYQOoBKzzPpu8aITG6VBDr53wS6L0z kR5MOoQrUaY4AYFsKybPqoiyh2N6Vi8onWz0UguUSTGRcemwtFcBIhFMeMIQv1Xahzd9 uj3EYsX4eh4lnuBiEIyfLfKd6eT8i6qOS6A6c944d4vMJsiPjI77rlK2osnWQoW1l6w6 ohHzq1Llouzb2w/SiUu6fOoGIdnNYpwXJi5o/Ymx+n6jL3tYKH1NH9LYLGyvtXXWw3kQ KRIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736953302; x=1737558102; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fHQKJ7ZE130ntDkhQLA0rjO4bmRpsWSpyiW+ZgLoauA=; b=NJp3/gRE2M/9Xm5D13BT9GorfaVgqLa011rL4Sc7bTu1kcrXJcKRfRkO15oNdIjT70 MSIYJ7YlpwLtetvNdYxTXSnXEQO9HgVU+kFo35adRq/ItGO1ejkW7dICrnEoHgf31ZeY 54eJeGgSljYOv1s2oc4esJrMNhRNEROU4nuwwn7PDaLziafvcCm0ZQ19Cvsmzk78RWPw sDtYBCYoGuSCeIP55SHZu4dbhd5XYPrGsHQ98WuR6AXVM8M75YK1eLYLnmExnIeeHv8g VrJn0CoENujHlNoGM9lS8SXkJelfdusx+HCRtIgZ1MnkUhKPyhQ81B56hSiszMnCWj66 w7ZA== X-Forwarded-Encrypted: i=1; AJvYcCWsR4VFOuaDmzQD6kKZhA89Ud9hCQHnsepPIgdiKQ1COD8mVJ3MVgSkUrTBe/c/gOP8T5tbqVcNfYvf@nongnu.org X-Gm-Message-State: AOJu0YzNSoizs9kvHDELT2A/dJHPnz9QOYp5vdrjatESJevO82QIy1Q3 nakGo2k81IwZIhaZzIMhEk15m2FmJ+rVM9LRyFASbqa+WSbgphevj5mxww== X-Gm-Gg: ASbGncupyNvzj+z6BJKGHvHyzebTI+3UaVML/LCLLO4mJbs5IiLlteCkdwupdWl4y1v gh7kWi9YrWA6r8q90UpJaah+IGDl+OIBHd87gqpvBjudENB31RZkrRM4Vfjmoc98I+uvR4DJnB1 jMIPae2y5RKOc6GBom7NuhKFlQRQEDd4v7/tQ1V3AXa1BsZZP0HtK4v6j8Zu+m7uGYfOZj4p11V wROGcYpSqlJGZIhv2w2yBzIdKyaeX0nFEgU6kVjdHfQ0bdwbLtpLGH97Gn/+w== X-Google-Smtp-Source: AGHT+IFmMBSqe4D11QliWOWJ2+3etQjHH7dg1nxLr08YI6XxrOK4Si5S2mC2TeNY66nfqzOez2G0Jw== X-Received: by 2002:a05:6a00:928c:b0:727:3935:dc83 with SMTP id d2e1a72fcca58-72d21fb1e07mr38988916b3a.10.1736953302097; Wed, 15 Jan 2025 07:01:42 -0800 (PST) Received: from wheely.local0.net ([118.210.104.29]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72d40658a7esm9519592b3a.106.2025.01.15.07.01.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jan 2025 07:01:41 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , John Snow , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , qemu-block@nongnu.org Subject: [PATCH v2 4/5] qtest/e1000e|igb: Clear interrupt-cause bits after irq Date: Thu, 16 Jan 2025 01:01:10 +1000 Message-ID: <20250115150112.346497-5-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115150112.346497-1-npiggin@gmail.com> References: <20250115150112.346497-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=npiggin@gmail.com; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The e1000e and igb tests do not clear the ICR/EICR cause bits (or set auto-clear) on seeing queue interrupts, which inhibits the triggering of a new interrupt. Fix this by clearing the cause bits, and verify that the expected cause bit was set. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Cc: Dmitry Fleytman Cc: Akihiko Odaki Cc: Sriram Yagnaraman Signed-off-by: Nicholas Piggin --- tests/qtest/e1000e-test.c | 4 ++++ tests/qtest/igb-test.c | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/tests/qtest/e1000e-test.c b/tests/qtest/e1000e-test.c index de9738fdb74..35548d1c20b 100644 --- a/tests/qtest/e1000e-test.c +++ b/tests/qtest/e1000e-test.c @@ -66,6 +66,8 @@ static void e1000e_send_verify(QE1000E *d, int *test_sockets, QGuestAllocator *a /* Wait for TX WB interrupt */ e1000e_wait_isr(d, E1000E_TX0_MSG_ID); + /* Read ICR to make it ready for next interrupt, assert TXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_ICR) & E1000_ICR_TXQ0); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.upper.data) & E1000_TXD_STAT_DD, ==, @@ -117,6 +119,8 @@ static void e1000e_receive_verify(QE1000E *d, int *test_sockets, QGuestAllocator /* Wait for TX WB interrupt */ e1000e_wait_isr(d, E1000E_RX0_MSG_ID); + /* Read ICR to make it ready for next interrupt, assert RXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_ICR) & E1000_ICR_RXQ0); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.wb.upper.status_error) & diff --git a/tests/qtest/igb-test.c b/tests/qtest/igb-test.c index 3d397ea6973..fb77c5e507a 100644 --- a/tests/qtest/igb-test.c +++ b/tests/qtest/igb-test.c @@ -69,6 +69,8 @@ static void igb_send_verify(QE1000E *d, int *test_sockets, QGuestAllocator *allo /* Wait for TX WB interrupt */ e1000e_wait_isr(d, E1000E_TX0_MSG_ID); + /* Read EICR which clears it ready for next interrupt, assert TXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_EICR) & (1 << E1000E_TX0_MSG_ID)); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.wb.status) & E1000_TXD_STAT_DD, ==, @@ -120,6 +122,8 @@ static void igb_receive_verify(QE1000E *d, int *test_sockets, QGuestAllocator *a /* Wait for TX WB interrupt */ e1000e_wait_isr(d, E1000E_RX0_MSG_ID); + /* Read EICR which clears it ready for next interrupt, assert RXQ0 cause */ + g_assert(e1000e_macreg_read(d, E1000_EICR) & (1 << E1000E_RX0_MSG_ID)); /* Check DD bit */ g_assert_cmphex(le32_to_cpu(descr.wb.upper.status_error) & From patchwork Wed Jan 15 15:01:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 13940532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4B70DC02183 for ; Wed, 15 Jan 2025 15:03:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tY4u2-0004PP-Kq; Wed, 15 Jan 2025 10:01:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tY4u0-0004IR-03; Wed, 15 Jan 2025 10:01:52 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tY4tx-0002wI-SN; Wed, 15 Jan 2025 10:01:51 -0500 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-21634338cfdso122205865ad.2; Wed, 15 Jan 2025 07:01:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736953307; x=1737558107; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jCAzdtRpgOa7ExFBXaLuO4nmF6Dq3KEzUWWjfN/UCuo=; b=F9S3IhviYCzI4VvJ1Llr0l+AKcFuFdiMcrNLWfFalwTpbQ4ZiF264+fCap8Emskyhu 3pkczWlgupLDs8g/NnMWZbV6QTZapE/SjkxBnLaej7K2ffT8Rmn+0YK/AeU22rBBdUgg gunzICYUvCVHuWo1PAxSzsYAmgWPfbELXRpvb5pmOkQ9iz6RUjQASiPnWBtggmbztYms LD62+K/neVwWMr8BZS/TV14S5tHmBaT8dkPCDeTvuGu+phy6qZf1ydhijqPQVSAHgcFv AeoIFm3dyXqZnAZvyEA6J5W+yoySoMd8gaDyCFPjt/LC14D/Koyxm89s3fWcillPM7hg fTOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736953307; x=1737558107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jCAzdtRpgOa7ExFBXaLuO4nmF6Dq3KEzUWWjfN/UCuo=; b=Rggk9ALNYCwW2H4rJz2YFD16c7FP7hxoOQqBX/c5zsbH35DuTUuYJUxRgnBol5aK/B aa/OqK0AzAdpM8Tf/4E1Hn7/c1LzO8ov8AIf4KlEPWqK0XWW6kmmnuV2BQSSaOoNAFH/ Lna8hA5JBLi9jWWt7cZu5UOGLcTPTEeOL4kwuxoJ6iq+fFtvkWR9XU6Rz4xzdX4T07QQ Lj7AwTw2U5Fxd9tiE4TqV4htmcXF1ysdr+jyD/d7hgeozOyuiL8/RqqzjU1ofi0tw8R9 bGNhGJtHcCgN3pKTtMHnoC9seFdgfrOJboOk2C0L20U7U6gcTu9fhh9Oh6RgtMLvmvLF 3fqg== X-Forwarded-Encrypted: i=1; AJvYcCWUMjTsqZgLfplGZUIvRa5gEIMgmWOQqA69wdwpx+Dybrwtsk/6ui1FuvMZtuyC0vZfR5cWYoJpSYlM@nongnu.org X-Gm-Message-State: AOJu0YxkpatX8XKT6K7EqLDv0PMkmCh5NmEOIAuLFT6gyi0ZnoqXMjEh QE/EArZVcUI5K8+64gv7w0NwtQ9fNmhcK8KtIvTifQwGx1fIV3C6RPD2DQ== X-Gm-Gg: ASbGnctoyrY/xMQTPMvm5cVbFU9BzdHLklkIhBhC7uWXQ7rXMAIAnQWLNq7U1ICAHZY XBXjAVohJwpncJmvOcUwTsI77wj+h3Fx8NwwpheYsATBMbkfI8cik0VoNc+N4sghOw8ac3QqvXZ LNTk/nBLcd5spgFCXxN3q7ZR7XoX1rYtBXGLSo+IryuNw5dd+llAqAoznbwME0A5Ehcy/R8sRVJ m1ZdEHcbli5VFgDVK9g42pIyX4t6f4DZvFEvQArem+6lfoJLLgq+tqA+Th0Bg== X-Google-Smtp-Source: AGHT+IFxxLGaXEUXpA7SwZgUM7yXX3PBualxUwgoJYTPQDsEdl/CufGqtyp7uthURbsIlHDjxd2efg== X-Received: by 2002:a05:6a00:3c83:b0:71e:4296:2e with SMTP id d2e1a72fcca58-72d21f4bc9emr40833840b3a.11.1736953307190; Wed, 15 Jan 2025 07:01:47 -0800 (PST) Received: from wheely.local0.net ([118.210.104.29]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72d40658a7esm9519592b3a.106.2025.01.15.07.01.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jan 2025 07:01:46 -0800 (PST) From: Nicholas Piggin To: qemu-devel@nongnu.org Cc: Nicholas Piggin , "Michael S . Tsirkin" , Marcel Apfelbaum , John Snow , Fabiano Rosas , Laurent Vivier , Paolo Bonzini , Dmitry Fleytman , Akihiko Odaki , Sriram Yagnaraman , qemu-block@nongnu.org Subject: [PATCH v2 5/5] qtest/e1000e|igb: Fix msix to re-trigger interrupts Date: Thu, 16 Jan 2025 01:01:11 +1000 Message-ID: <20250115150112.346497-6-npiggin@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115150112.346497-1-npiggin@gmail.com> References: <20250115150112.346497-1-npiggin@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=npiggin@gmail.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The e1000e and igb tests don't clear the msix pending bit after waiting for it, as it is masked so the irq doesn't get sent. This means all subsequent waits for the interrupt does not wait or verify the interrupt was generated, affecting the multiple_transfers tests. To fix this, have device setup always enable and unmask the RXQ and TXQ irq vectors, and verify interrupt was seen by checking the msix message was seen. Cc: Michael S. Tsirkin Cc: Marcel Apfelbaum Cc: Dmitry Fleytman Cc: Akihiko Odaki Cc: Sriram Yagnaraman Signed-off-by: Nicholas Piggin --- tests/qtest/libqos/e1000e.h | 11 ++++- tests/qtest/libqos/e1000e.c | 87 +++++++++++++++++++++++++++++++++++-- tests/qtest/libqos/igb.c | 9 ++++ 3 files changed, 102 insertions(+), 5 deletions(-) diff --git a/tests/qtest/libqos/e1000e.h b/tests/qtest/libqos/e1000e.h index 30643c80949..63aa8c28a39 100644 --- a/tests/qtest/libqos/e1000e.h +++ b/tests/qtest/libqos/e1000e.h @@ -22,8 +22,13 @@ #include "qgraph.h" #include "pci.h" -#define E1000E_RX0_MSG_ID (0) -#define E1000E_TX0_MSG_ID (1) +enum { + E1000E_RX0_MSG_ID, + E1000E_TX0_MSG_ID, + E1000E_MSG_ID_MAX +}; + +#define E1000E_MSIX_DATA ((uint32_t[]) { 0x12345678, 0xabcdef00 }) #define E1000E_ADDRESS { 0x52, 0x54, 0x00, 0x12, 0x34, 0x56 } @@ -40,6 +45,7 @@ struct QE1000E_PCI { QPCIDevice pci_dev; QPCIBar mac_regs; QE1000E e1000e; + uint64_t msix_msg_addr[E1000E_MSG_ID_MAX]; }; static inline void e1000e_macreg_write(QE1000E *d, uint32_t reg, uint32_t val) @@ -57,5 +63,6 @@ static inline uint32_t e1000e_macreg_read(QE1000E *d, uint32_t reg) void e1000e_wait_isr(QE1000E *d, uint16_t msg_id); void e1000e_tx_ring_push(QE1000E *d, void *descr); void e1000e_rx_ring_push(QE1000E *d, void *descr); +void e1000e_pci_msix_enable_rxtxq_vectors(QE1000E_PCI *d); #endif diff --git a/tests/qtest/libqos/e1000e.c b/tests/qtest/libqos/e1000e.c index 925654c7fd4..49bedb5e009 100644 --- a/tests/qtest/libqos/e1000e.c +++ b/tests/qtest/libqos/e1000e.c @@ -19,6 +19,7 @@ #include "qemu/osdep.h" #include "hw/net/e1000_regs.h" #include "hw/pci/pci_ids.h" +#include "hw/pci/pci_regs.h" #include "../libqtest.h" #include "pci-pc.h" #include "qemu/sockets.h" @@ -77,16 +78,48 @@ static void e1000e_foreach_callback(QPCIDevice *dev, int devfn, void *data) g_free(dev); } +static bool e1000e_test_msix_irq(QE1000E *d, uint16_t msg_id, + uint64_t guest_msix_addr, + uint32_t msix_data) +{ + QE1000E_PCI *d_pci = container_of(d, QE1000E_PCI, e1000e); + QPCIDevice *pci_dev = &d_pci->pci_dev; + uint32_t data; + + /* msix irq test must enable msix */ + g_assert(pci_dev->msix_enabled); + + /* Vector must be enabled (e.g., with enable_rxtxq_vectors) */ + g_assert(!qpci_msix_masked(pci_dev, msg_id)); + + data = qtest_readl(pci_dev->bus->qts, guest_msix_addr); + if (data == msix_data) { + /* Clear msix addr ready for next interrupt */ + qtest_writel(pci_dev->bus->qts, guest_msix_addr, 0); + return true; + } else if (data == 0) { + return false; + } else { + /* Must only be either 0 (no interrupt) or the msix data. */ + g_assert_not_reached(); + } +} + void e1000e_wait_isr(QE1000E *d, uint16_t msg_id) { QE1000E_PCI *d_pci = container_of(d, QE1000E_PCI, e1000e); - guint64 end_time = g_get_monotonic_time() + 5 * G_TIME_SPAN_SECOND; + QPCIDevice *pci_dev = &d_pci->pci_dev; + uint64_t end_time = g_get_monotonic_time() + 5 * G_TIME_SPAN_SECOND; + uint64_t guest_msix_addr = d_pci->msix_msg_addr[msg_id]; + uint32_t msix_data = E1000E_MSIX_DATA[msg_id]; + + assert(pci_dev->msix_enabled); do { - if (qpci_msix_pending(&d_pci->pci_dev, msg_id)) { + if (e1000e_test_msix_irq(d, msg_id, guest_msix_addr, msix_data)) { return; } - qtest_clock_step(d_pci->pci_dev.bus->qts, 10000); + qtest_clock_step(pci_dev->bus->qts, 10000); } while (g_get_monotonic_time() < end_time); g_error("Timeout expired"); @@ -99,6 +132,45 @@ static void e1000e_pci_destructor(QOSGraphObject *obj) qpci_msix_disable(&epci->pci_dev); } +static void e1000e_pci_msix_enable_vector(QE1000E_PCI *d, uint16_t msg_id) +{ + QPCIDevice *pci_dev = &d->pci_dev; + uint64_t guest_msix_addr = d->msix_msg_addr[msg_id]; + uint32_t msix_data = E1000E_MSIX_DATA[msg_id]; + uint32_t control; + uint64_t off; + + g_assert_cmpint(msg_id , >=, 0); + g_assert_cmpint(msg_id , <, qpci_msix_table_size(pci_dev)); + g_assert_cmpint(msg_id , <, E1000E_MSG_ID_MAX); + g_assert(guest_msix_addr != 0); + g_assert(msix_data != 0); + + off = pci_dev->msix_table_off + (msg_id * 16); + + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_LOWER_ADDR, guest_msix_addr & ~0UL); + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_UPPER_ADDR, + (guest_msix_addr >> 32) & ~0UL); + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_DATA, msix_data); + + control = qpci_io_readl(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_VECTOR_CTRL); + qpci_io_writel(pci_dev, pci_dev->msix_table_bar, + off + PCI_MSIX_ENTRY_VECTOR_CTRL, + control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT); +} + +void e1000e_pci_msix_enable_rxtxq_vectors(QE1000E_PCI *d) +{ + g_assert(d->pci_dev.msix_enabled); + + e1000e_pci_msix_enable_vector(d, E1000E_RX0_MSG_ID); + e1000e_pci_msix_enable_vector(d, E1000E_TX0_MSG_ID); +} + static void e1000e_pci_start_hw(QOSGraphObject *obj) { QE1000E_PCI *d = (QE1000E_PCI *) obj; @@ -113,6 +185,7 @@ static void e1000e_pci_start_hw(QOSGraphObject *obj) /* Enable and configure MSI-X */ qpci_msix_enable(&d->pci_dev); + e1000e_pci_msix_enable_rxtxq_vectors(d); e1000e_macreg_write(&d->e1000e, E1000_IVAR, E1000E_IVAR_TEST_CFG); /* Check the device status - link and speed */ @@ -196,6 +269,14 @@ static void *e1000e_pci_create(void *pci_bus, QGuestAllocator *alloc, d->e1000e.rx_ring = guest_alloc(alloc, E1000E_RING_LEN); g_assert(d->e1000e.rx_ring != 0); + /* Allocate and clear msix msg addr for TX */ + d->msix_msg_addr[E1000E_TX0_MSG_ID] = guest_alloc(alloc, 4); + g_assert(d->msix_msg_addr[E1000E_TX0_MSG_ID] != 0); + + /* Allocate and clear msix msg addr for RX */ + d->msix_msg_addr[E1000E_RX0_MSG_ID] = guest_alloc(alloc, 4); + g_assert(d->msix_msg_addr[E1000E_RX0_MSG_ID] != 0); + d->obj.get_driver = e1000e_pci_get_driver; d->obj.start_hw = e1000e_pci_start_hw; d->obj.destructor = e1000e_pci_destructor; diff --git a/tests/qtest/libqos/igb.c b/tests/qtest/libqos/igb.c index f40c4ec4cd2..c902634b3df 100644 --- a/tests/qtest/libqos/igb.c +++ b/tests/qtest/libqos/igb.c @@ -75,6 +75,7 @@ static void igb_pci_start_hw(QOSGraphObject *obj) /* Enable and configure MSI-X */ qpci_msix_enable(&d->pci_dev); + e1000e_pci_msix_enable_rxtxq_vectors(d); e1000e_macreg_write(&d->e1000e, E1000_IVAR0, IGB_IVAR_TEST_CFG); /* Check the device link status */ @@ -161,6 +162,14 @@ static void *igb_pci_create(void *pci_bus, QGuestAllocator *alloc, void *addr) d->e1000e.rx_ring = guest_alloc(alloc, E1000E_RING_LEN); g_assert(d->e1000e.rx_ring != 0); + /* Allocate and clear msix msg addr for TX */ + d->msix_msg_addr[E1000E_TX0_MSG_ID] = guest_alloc(alloc, 4); + g_assert(d->msix_msg_addr[E1000E_TX0_MSG_ID] != 0); + + /* Allocate and clear msix msg addr for RX */ + d->msix_msg_addr[E1000E_RX0_MSG_ID] = guest_alloc(alloc, 4); + g_assert(d->msix_msg_addr[E1000E_RX0_MSG_ID] != 0); + d->obj.get_driver = igb_pci_get_driver; d->obj.start_hw = igb_pci_start_hw; d->obj.destructor = e1000e_pci_destructor;