From patchwork Mon Jan 27 18:56:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Druzhinin X-Patchwork-Id: 11353147 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC8AE1580 for ; Mon, 27 Jan 2020 18:57:39 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A733021739 for ; Mon, 27 Jan 2020 18:57:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="HXRdoYxs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A733021739 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iw9YY-0004Zq-Kj; Mon, 27 Jan 2020 18:56:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iw9YW-0004Zl-TR for xen-devel@lists.xenproject.org; Mon, 27 Jan 2020 18:56:16 +0000 X-Inumbo-ID: b182abf8-4136-11ea-859d-12813bfff9fa Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id b182abf8-4136-11ea-859d-12813bfff9fa; Mon, 27 Jan 2020 18:56:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1580151376; h=from:to:cc:subject:date:message-id:mime-version; bh=ywQ0ZJqmvvPJIZ5OQ/KNiQV+lAY3DsCIgW5Mnl9+td0=; b=HXRdoYxsE5u+eQXlkg9TRLjrFq4cSz+mnAu5ezKT0KdLLcQvYLqVpICW kNbUF4XcHQppTp8FNN/eLw3vm0QrBIlGNJseY+WMnAvGKuz2yRNLdSGPd GALSJd0lnztUh0CcaPuQICqPBXOmQjZH/3t8SK1OlUbVa/uFv0EBp3SEr c=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=igor.druzhinin@citrix.com; spf=Pass smtp.mailfrom=igor.druzhinin@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of igor.druzhinin@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of igor.druzhinin@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="igor.druzhinin@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="igor.druzhinin@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: nk0n1KqGm7LM/fbcGK9peGFQ7c0rqJDjuPTn2ri8/L3B/WnJaX52DuwMXsp9cNE76JX4Xm3AT+ 6uB+VJDzsmdEj9t2mEEUpy+V171UrHWDJObdknz6sq4rFMqlzKLvcNSACzgMc9JMdWw3KCnIMP rthqWh7lqLgAC+ABeNsiAIeUn5J+3xMAsHuEGmp8r1pPnybDw6Nr0ZxRgeG7OJ56ZZa3lZ+Xf3 seODrtF9WVGMcKXS0c31U5pHjOZJ3y+k9k7XoebQ1QEbNLYIb+ThKnZUmG9n3fJ5OfLeghrrO/ psI= X-SBRS: 2.7 X-MesageID: 11503677 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.70,370,1574139600"; d="scan'208";a="11503677" From: Igor Druzhinin To: Date: Mon, 27 Jan 2020 18:56:10 +0000 Message-ID: <1580151370-6409-1-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH] RCU: reimplement RCU barrier to avoid deadlock X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Igor Druzhinin , sstabellini@kernel.org, julien@xen.org, wl@xen.org, konrad.wilk@oracle.com, George.Dunlap@eu.citrix.com, andrew.cooper3@citrix.com, ian.jackson@eu.citrix.com Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" The existing RCU barrier implementation is prone to a deadlock scenario due to IRQs being re-enabled inside stopmachine context. If due to a race IRQs are re-enabled on some of CPUs and softirqs are allowed to be processed in stopmachine, i.e. what currently happens in rcu_barrier(), timer interrupt is able to invoke TSC synchronization rendezvous. At this moment sending TSC synchronization IPI will stall waiting for other CPUs to synchronize while they in turn are waiting in stopmachine busy loop with IRQs disabled. To avoid the scenario above - reimplement rcu_barrier() in a way where IRQs are not being disabled at any moment. The proposed implementation is just a simplified and specialized version of stopmachine. The semantic of the call is preserved. Signed-off-by: Igor Druzhinin --- This change has been stress tested by doing actions invoking rcu_barrier() functionality and didn't show any issues. --- xen/common/rcupdate.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/xen/common/rcupdate.c b/xen/common/rcupdate.c index cb712c8..95a1f85 100644 --- a/xen/common/rcupdate.c +++ b/xen/common/rcupdate.c @@ -145,6 +145,9 @@ struct rcu_barrier_data { atomic_t *cpu_count; }; +static DEFINE_PER_CPU(struct tasklet, rcu_barrier_tasklet); +static atomic_t rcu_barrier_cpu_count, rcu_barrier_cpu_done; + static void rcu_barrier_callback(struct rcu_head *head) { struct rcu_barrier_data *data = container_of( @@ -152,12 +155,9 @@ static void rcu_barrier_callback(struct rcu_head *head) atomic_inc(data->cpu_count); } -static int rcu_barrier_action(void *_cpu_count) +static void rcu_barrier_action(void *unused) { - struct rcu_barrier_data data = { .cpu_count = _cpu_count }; - - ASSERT(!local_irq_is_enabled()); - local_irq_enable(); + struct rcu_barrier_data data = { .cpu_count = &rcu_barrier_cpu_count }; /* * When callback is executed, all previously-queued RCU work on this CPU @@ -172,15 +172,30 @@ static int rcu_barrier_action(void *_cpu_count) cpu_relax(); } - local_irq_disable(); - - return 0; + atomic_inc(&rcu_barrier_cpu_done); } int rcu_barrier(void) { - atomic_t cpu_count = ATOMIC_INIT(0); - return stop_machine_run(rcu_barrier_action, &cpu_count, NR_CPUS); + unsigned int i; + + if ( !get_cpu_maps() ) + return -EBUSY; + + atomic_set(&rcu_barrier_cpu_count, 0); + atomic_set(&rcu_barrier_cpu_done, 0); + + for_each_online_cpu ( i ) + if ( i != smp_processor_id() ) + tasklet_schedule_on_cpu(&per_cpu(rcu_barrier_tasklet, i), i); + + rcu_barrier_action(NULL); + + while ( atomic_read(&rcu_barrier_cpu_done) != num_online_cpus() ) + cpu_relax(); + + put_cpu_maps(); + return 0; } /* Is batch a before batch b ? */ @@ -564,6 +579,7 @@ static void rcu_init_percpu_data(int cpu, struct rcu_ctrlblk *rcp, rdp->cpu = cpu; rdp->blimit = blimit; init_timer(&rdp->idle_timer, rcu_idle_timer_handler, rdp, cpu); + tasklet_init(&per_cpu(rcu_barrier_tasklet, cpu), rcu_barrier_action, NULL); } static int cpu_callback(