From patchwork Thu Dec 8 17:50:30 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 9466855 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DEE8E60459 for ; Thu, 8 Dec 2016 17:51:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2AEF285A8 for ; Thu, 8 Dec 2016 17:51:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C7891285E5; Thu, 8 Dec 2016 17:51:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34300285E3 for ; Thu, 8 Dec 2016 17:51:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752738AbcLHRvC (ORCPT ); Thu, 8 Dec 2016 12:51:02 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34538 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752013AbcLHRvC (ORCPT ); Thu, 8 Dec 2016 12:51:02 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E725E285A5; Thu, 8 Dec 2016 17:51:01 +0000 (UTC) Received: from kamzik.brq.redhat.com (kamzik.brq.redhat.com [10.34.1.143]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id uB8HoWnB022068; Thu, 8 Dec 2016 12:50:59 -0500 From: Andrew Jones To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, qemu-devel@nongnu.org, qemu-arm@nongnu.org Cc: peter.maydell@linaro.org, marc.zyngier@arm.com, andre.przywara@arm.com, eric.auger@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org, christoffer.dall@linaro.org Subject: [PATCH kvm-unit-tests v8 10/10] arm/arm64: gic: don't just use zero Date: Thu, 8 Dec 2016 18:50:30 +0100 Message-Id: <20161208175030.12269-11-drjones@redhat.com> In-Reply-To: <20161208175030.12269-1-drjones@redhat.com> References: <20161208175030.12269-1-drjones@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 08 Dec 2016 17:51:02 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Instead of using cpu0 and irq=0 for the IPI test, use something a bit more interesting. To make sure we can still run the test with the minimal number of cpus (2), we need to use sender=1. As the irq in the test (an SGI) can only be 0 to 15 and there's really no difference, then use irq=1. sender=1 is a bit limited for the long term, as we may want to test IPIs across host sockets with pinned vcpus. But we can improve on this later, possibly with the introduction of a command line parameter. Reviewed-by: Eric Auger Signed-off-by: Andrew Jones --- v8: - remove crufy command line parsing. No need for sender or irq to be input right now. v7: - cleanup cmdline parsing and add complain on bad args [Eric] v6: - make sender/irq names more future-proof [drew] - sanity check inputs [drew] - introduce check_sender/irq and bad_sender/irq to more cleanly do checks [drew] - default sender and irq to 1, instead of still zero [drew] v4: improve structure and make sure spurious checking is done even when the sender isn't cpu0 v2: actually check that the irq received was the irq sent, and (for gicv2) that the sender is the expected one. --- arm/gic.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 73 insertions(+), 23 deletions(-) diff --git a/arm/gic.c b/arm/gic.c index d0d3be0fa36e..3054d45f7387 100644 --- a/arm/gic.c +++ b/arm/gic.c @@ -19,6 +19,9 @@ #include #include +#define IPI_SENDER 1 +#define IPI_IRQ 1 + struct gic { struct { void (*send_self)(void); @@ -28,6 +31,7 @@ struct gic { static struct gic *gic; static int acked[NR_CPUS], spurious[NR_CPUS]; +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS]; static cpumask_t ready; static void nr_cpu_check(int nr) @@ -43,10 +47,23 @@ static void wait_on_ready(void) cpu_relax(); } +static void stats_reset(void) +{ + int i; + + for (i = 0; i < nr_cpus; ++i) { + acked[i] = 0; + bad_sender[i] = -1; + bad_irq[i] = -1; + } + smp_wmb(); +} + static void check_acked(cpumask_t *mask) { int missing = 0, extra = 0, unexpected = 0; int nr_pass, cpu, i; + bool bad = false; /* Wait up to 5s for all interrupts to be delivered */ for (i = 0; i < 50; ++i) { @@ -56,9 +73,21 @@ static void check_acked(cpumask_t *mask) smp_rmb(); nr_pass += cpumask_test_cpu(cpu, mask) ? acked[cpu] == 1 : acked[cpu] == 0; + + if (bad_sender[cpu] != -1) { + printf("cpu%d received IPI from wrong sender %d\n", + cpu, bad_sender[cpu]); + bad = true; + } + + if (bad_irq[cpu] != -1) { + printf("cpu%d received wrong irq %d\n", + cpu, bad_irq[cpu]); + bad = true; + } } if (nr_pass == nr_cpus) { - report("Completed in %d ms", true, ++i * 100); + report("Completed in %d ms", !bad, ++i * 100); return; } } @@ -91,6 +120,22 @@ static void check_spurious(void) } } +static void check_ipi_sender(u32 irqstat) +{ + if (gic_version() == 2) { + int src = (irqstat >> 10) & 7; + + if (src != IPI_SENDER) + bad_sender[smp_processor_id()] = src; + } +} + +static void check_irqnr(u32 irqnr) +{ + if (irqnr != IPI_IRQ) + bad_irq[smp_processor_id()] = irqnr; +} + static void ipi_handler(struct pt_regs *regs __unused) { u32 irqstat = gic_read_iar(); @@ -98,8 +143,10 @@ static void ipi_handler(struct pt_regs *regs __unused) if (irqnr != GICC_INT_SPURIOUS) { gic_write_eoir(irqstat); - smp_rmb(); /* pairs with wmb in ipi_test functions */ + smp_rmb(); /* pairs with wmb in stats_reset */ ++acked[smp_processor_id()]; + check_ipi_sender(irqstat); + check_irqnr(irqnr); smp_wmb(); /* pairs with rmb in check_acked */ } else { ++spurious[smp_processor_id()]; @@ -109,22 +156,22 @@ static void ipi_handler(struct pt_regs *regs __unused) static void gicv2_ipi_send_self(void) { - writel(2 << 24, gicv2_dist_base() + GICD_SGIR); + writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR); } static void gicv2_ipi_send_broadcast(void) { - writel(1 << 24, gicv2_dist_base() + GICD_SGIR); + writel(1 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR); } static void gicv3_ipi_send_self(void) { - gic_ipi_send_single(0, smp_processor_id()); + gic_ipi_send_single(IPI_IRQ, smp_processor_id()); } static void gicv3_ipi_send_broadcast(void) { - gicv3_write_sgi1r(1ULL << 40); + gicv3_write_sgi1r(1ULL << 40 | IPI_IRQ << 24); isb(); } @@ -133,10 +180,9 @@ static void ipi_test_self(void) cpumask_t mask; report_prefix_push("self"); - memset(acked, 0, sizeof(acked)); - smp_wmb(); + stats_reset(); cpumask_clear(&mask); - cpumask_set_cpu(0, &mask); + cpumask_set_cpu(smp_processor_id(), &mask); gic->ipi.send_self(); check_acked(&mask); report_prefix_pop(); @@ -148,20 +194,18 @@ static void ipi_test_smp(void) int i; report_prefix_push("target-list"); - memset(acked, 0, sizeof(acked)); - smp_wmb(); + stats_reset(); cpumask_copy(&mask, &cpu_present_mask); - for (i = 0; i < nr_cpus; i += 2) + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2) cpumask_clear_cpu(i, &mask); - gic_ipi_send_mask(0, &mask); + gic_ipi_send_mask(IPI_IRQ, &mask); check_acked(&mask); report_prefix_pop(); report_prefix_push("broadcast"); - memset(acked, 0, sizeof(acked)); - smp_wmb(); + stats_reset(); cpumask_copy(&mask, &cpu_present_mask); - cpumask_clear_cpu(0, &mask); + cpumask_clear_cpu(smp_processor_id(), &mask); gic->ipi.send_broadcast(); check_acked(&mask); report_prefix_pop(); @@ -178,6 +222,16 @@ static void ipi_enable(void) local_irq_enable(); } +static void ipi_send(void) +{ + ipi_enable(); + wait_on_ready(); + ipi_test_self(); + ipi_test_smp(); + check_spurious(); + exit(report_summary()); +} + static void ipi_recv(void) { ipi_enable(); @@ -232,14 +286,10 @@ int main(int argc, char **argv) for_each_present_cpu(cpu) { if (cpu == 0) continue; - smp_boot_secondary(cpu, ipi_recv); + smp_boot_secondary(cpu, + cpu == IPI_SENDER ? ipi_send : ipi_recv); } - ipi_enable(); - wait_on_ready(); - ipi_test_self(); - ipi_test_smp(); - check_spurious(); - report_prefix_pop(); + ipi_recv(); } else { report_abort("Unknown subtest '%s'", argv[1]); }