From patchwork Wed Mar 1 22:15:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Stabellini X-Patchwork-Id: 9599157 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9083060414 for ; Wed, 1 Mar 2017 22:18:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8288727F90 for ; Wed, 1 Mar 2017 22:18:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 777D3284E7; Wed, 1 Mar 2017 22:18:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EFB6227F90 for ; Wed, 1 Mar 2017 22:18:08 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjCXK-00036y-TI; Wed, 01 Mar 2017 22:15:54 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjCXK-00036J-10 for xen-devel@lists.xenproject.org; Wed, 01 Mar 2017 22:15:54 +0000 Received: from [85.158.137.68] by server-4.bemta-3.messagelabs.com id 04/9A-03705-91847B85; Wed, 01 Mar 2017 22:15:53 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrNLMWRWlGSWpSXmKPExsVybKJsh66Ex/Y Ig+vP1C2+b5nM5MDocfjDFZYAxijWzLyk/IoE1ox9U54wF0yXqFjy5Rd7A2OzcBcjF4eQwFRG iUUXH7JDOAeZJKbcaAVyODnYBAwl/j7ZxAZiiwhISGxqWMEEYjMLWEvce/MDrEZYIE5i1+1zz CA2i4CqxJFt/8BsXgE3ifNzp4D1SgjISZw8NpkVxOYUcJdo6jgIZgsJtDNKHDqQBlGTITGvZw 4rhO0lsejGJShbTeLquU3MExj5FjAyrGLUKE4tKkst0jUy0UsqykzPKMlNzMzRNTQw1stNLS5 OTE/NSUwq1kvOz93ECAyUegYGxh2Mr477HWKU5GBSEuXdvWpbhBBfUn5KZUZicUZ8UWlOavEh RhkODiUJ3hNu2yOEBItS01Mr0jJzgCELk5bg4FES4b3uCpTmLS5IzC3OTIdInWJUlBLnvQ3SJ wCSyCjNg2uDxcklRlkpYV5GBgYGIZ6C1KLczBJU+VeM4hyMSsK890Gm8GTmlcBNfwW0mAlo8Q uVrSCLSxIRUlINjOtjfjk3ZO+rOvpU5YhU3Y1P4bnrTNymTXU65xD2aYNv3W+e4hV1rpqN57b pdLasUiliLLNnt6rcnVImyJLkuzTKxOXrc2aGS01BCaUPTzM8FrGZ8yRtyTaPlNXsufHlf1nC GaTs3G4Ya6TP2P7LW0w3cWFuU4SdWtmPXcxXN+6/k/Zup/wpJZbijERDLeai4kQA0Hd2h44CA AA= X-Env-Sender: sstabellini@kernel.org X-Msg-Ref: server-12.tower-31.messagelabs.com!1488406551!71317221!1 X-Originating-IP: [198.145.29.136] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 58411 invoked from network); 1 Mar 2017 22:15:52 -0000 Received: from mail.kernel.org (HELO mail.kernel.org) (198.145.29.136) by server-12.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 1 Mar 2017 22:15:52 -0000 Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B4218203A4; Wed, 1 Mar 2017 22:15:49 +0000 (UTC) Received: from sstabellini-ThinkPad-X260.hsd1.ca.comcast.net (96-82-76-110-static.hfc.comcastbusiness.net [96.82.76.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A4F98203A0; Wed, 1 Mar 2017 22:15:48 +0000 (UTC) From: Stefano Stabellini To: julien.grall@arm.com Date: Wed, 1 Mar 2017 14:15:45 -0800 Message-Id: <1488406545-26164-3-git-send-email-sstabellini@kernel.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1488406545-26164-1-git-send-email-sstabellini@kernel.org> References: <1488406545-26164-1-git-send-email-sstabellini@kernel.org> X-Virus-Scanned: ClamAV using ClamSMTP Cc: xen-devel@lists.xenproject.org, sstabellini@kernel.org Subject: [Xen-devel] [PATCH v5 3/3] xen/arm: vgic_migrate_irq: do not race against GIC_IRQ_GUEST_MIGRATING X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP A potential race condition occurs when vgic_migrate_irq is called a second time, while GIC_IRQ_GUEST_MIGRATING is already set. In that case, vgic_migrate_irq takes a different vgic lock from gic_update_one_lr. vgic_migrate_irq running concurrently with gic_update_one_lr could cause data corruptions, as they both access the inflight list. This patch fixes this problem. In vgic_migrate_irq after setting the new vcpu target, it checks both GIC_IRQ_GUEST_MIGRATING and GIC_IRQ_GUEST_VISIBLE. If they are both set we can just return because we have already set the new target: when gic_update_one_lr reaches the GIC_IRQ_GUEST_MIGRATING test, it will do the right thing. Otherwise, if GIC_IRQ_GUEST_MIGRATING is set but GIC_IRQ_GUEST_VISIBLE is not, gic_update_one_lr is running at the very same time on another pcpu, so it just waits until it completes (GIC_IRQ_GUEST_MIGRATING is cleared). Signed-off-by: Stefano Stabellini --- xen/arch/arm/gic.c | 5 ++++- xen/arch/arm/vgic.c | 16 ++++++++++++++-- 2 files changed, 18 insertions(+), 3 deletions(-) diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index 16bb150..a805300 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -508,10 +508,13 @@ static void gic_update_one_lr(struct vcpu *v, int i) * next pcpu, inflight is already cleared. No concurrent * accesses to inflight. */ smp_mb(); - if ( test_and_clear_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) ) + if ( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) ) { struct vcpu *v_target = vgic_get_target_vcpu(v, irq); irq_set_affinity(p->desc, cpumask_of(v_target->processor)); + /* Set the new affinity, then clear MIGRATING. */ + smp_mb(); + clear_bit(GIC_IRQ_GUEST_MIGRATING, &p->status); } } } diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c index a323e7e..9141b34 100644 --- a/xen/arch/arm/vgic.c +++ b/xen/arch/arm/vgic.c @@ -252,13 +252,25 @@ void vgic_migrate_irq(struct vcpu *old, struct vcpu *new, spin_lock_irqsave(&old->arch.vgic.lock, flags); write_atomic(t_vcpu, new->vcpu_id); - /* migration already in progress, no need to do anything */ - if ( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) ) + /* Set the new target, then check MIGRATING and VISIBLE, it pairs + * with the barrier in gic_update_one_lr. */ + smp_mb(); + + /* no need to do anything, gic_update_one_lr will take care of it */ + if ( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) && + test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) ) { spin_unlock_irqrestore(&old->arch.vgic.lock, flags); return; } + /* gic_update_one_lr is currently running, wait until its completion */ + while ( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) ) + { + cpu_relax(); + smp_rmb(); + } + if ( list_empty(&p->inflight) ) { irq_set_affinity(p->desc, cpumask_of(new->processor));