From patchwork Tue Aug 9 09:37:47 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 9270743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7FD1660754 for ; Tue, 9 Aug 2016 09:38:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 706B328327 for ; Tue, 9 Aug 2016 09:38:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 648C0283DF; Tue, 9 Aug 2016 09:38:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D9CE328327 for ; Tue, 9 Aug 2016 09:37:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932290AbcHIJh5 (ORCPT ); Tue, 9 Aug 2016 05:37:57 -0400 Received: from mail-pa0-f68.google.com ([209.85.220.68]:33039 "EHLO mail-pa0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752297AbcHIJhz (ORCPT ); Tue, 9 Aug 2016 05:37:55 -0400 Received: by mail-pa0-f68.google.com with SMTP id vy10so679161pac.0; Tue, 09 Aug 2016 02:37:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=9r59A7oN3i5FnyDDO4EtsNMV0dSj1jGpOj3oSmGWTXY=; b=kw7W03osnjFcK+/LYbjsxVBJNYigxcfoGlpC2bc6ZCWCec8PoGbaSx6XOcqh38SP3M JDc8mv37PJ+7jPdJRxCNd+nPFUSp8ttm1OeWF9csOVRTvCTScjdwNFkbdS4+lFgjnj55 fLDkVJ8StQvv555JGjS8/5/aomp3tH5QfH33cdHzSj36dATg6e9f63sOzaO9YoWVgSOd GxJ4w7Vo/PhxYQJWVZD9yTZIx/AkTOp3RpedtzWP8hqoK7m30sALovSG9i00Fj+JIGf3 VU3/824Epit0zJpFC8AZCF8hy3U9AUY9y4Wr5/6byr64DXLf+ndg1fT4ogniqVC+V0ft X4rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=9r59A7oN3i5FnyDDO4EtsNMV0dSj1jGpOj3oSmGWTXY=; b=juPgvhVXpxS31YIq4B2FcgepGL1frDdcUmqY8iz/nH669gtg2Ujym7yJPrIAUwTIRv KSUjoH4oyeQXBPummJO+07Voit8KrTg05fsA3gVoDztSr6Xe+13UbAixsZO88HtlYhar kiUKZU1/XC9gDut35kZTMbTO8VWaAw/sn4d8E9+2q9bjW267nDKC5FLZmGuTu5xgpXCl at4oBDtZ4WiotDU3oTGPZCc3CaeYeDeCrAdtKV5geuNE1P3d0Wz92JxrY+K8oVcPywuQ 3MWcGZg2ra+FLcsVqNySw7TpmfC4rACNP94ASMpyfqv+z4XQKaHkmBAbyVe9mbVA72w4 AOOQ== X-Gm-Message-State: AEkoouvtGWrhO3RaBN4t1mb3MukND7SZGtzE30tRCGHIFCj2h09lRUvHgnYeMPp01Hi3pQ== X-Received: by 10.66.193.7 with SMTP id hk7mr153761254pac.78.1470735474751; Tue, 09 Aug 2016 02:37:54 -0700 (PDT) Received: from kernel.kingsoft.cn ([114.255.44.132]) by smtp.gmail.com with ESMTPSA id y2sm54467167pan.31.2016.08.09.02.37.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 09 Aug 2016 02:37:53 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Wanpeng Li , "Peter Zijlstra (Intel)" , Ingo Molnar , Waiman Long , Davidlohr Bueso Subject: [PATCH RESEND v4] locking/pvqspinlock: Fix double hash race Date: Tue, 9 Aug 2016 17:37:47 +0800 Message-Id: <1470735467-4370-1-git-send-email-wanpeng.li@hotmail.com> X-Mailer: git-send-email 1.9.1 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wanpeng Li When the lock holder vCPU is racing with the queue head vCPU: lock holder vCPU queue head vCPU ===================== ================== node->locked = 1; READ_ONCE(node->locked) ... pv_wait_head_or_lock(): SPIN_THRESHOLD loop; pv_hash(); lock->locked = _Q_SLOW_VAL; node->state = vcpu_hashed; pv_kick_node(): cmpxchg(node->state, vcpu_halted, vcpu_hashed); lock->locked = _Q_SLOW_VAL; pv_hash(); With preemption at the right moment, it is possible that both the lock holder and queue head vCPUs can be racing to set node->state which can result in hash entry race. Making sure the state is never set to vcpu_halted will prevent this racing from happening. This patch fix it by setting vcpu_hashed after we did all hash thing. Acked-by: Waiman Long Reviewed-by: Davidlohr Bueso Reviewed-by: Pan Xinhui Cc: Peter Zijlstra (Intel) Cc: Ingo Molnar Cc: Waiman Long Cc: Davidlohr Bueso Signed-off-by: Wanpeng Li --- v3 -> v4: * update patch subject * add code comments v2 -> v3: * fix typo in patch description v1 -> v2: * adjust patch description kernel/locking/qspinlock_paravirt.h | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/kernel/locking/qspinlock_paravirt.h b/kernel/locking/qspinlock_paravirt.h index 21ede57..ca96db4 100644 --- a/kernel/locking/qspinlock_paravirt.h +++ b/kernel/locking/qspinlock_paravirt.h @@ -450,7 +450,28 @@ pv_wait_head_or_lock(struct qspinlock *lock, struct mcs_spinlock *node) goto gotlock; } } - WRITE_ONCE(pn->state, vcpu_halted); + /* + * lock holder vCPU queue head vCPU + * ---------------- --------------- + * node->locked = 1; + * READ_ONCE(node->locked) + * ... pv_wait_head_or_lock(): + * SPIN_THRESHOLD loop; + * pv_hash(); + * lock->locked = _Q_SLOW_VAL; + * node->state = vcpu_hashed; + * pv_kick_node(): + * cmpxchg(node->state, + * vcpu_halted, vcpu_hashed); + * lock->locked = _Q_SLOW_VAL; + * pv_hash(); + * + * With preemption at the right moment, it is possible that both the + * lock holder and queue head vCPUs can be racing to set node->state. + * Making sure the state is never set to vcpu_halted will prevent this + * racing from happening. + */ + WRITE_ONCE(pn->state, vcpu_hashed); qstat_inc(qstat_pv_wait_head, true); qstat_inc(qstat_pv_wait_again, waitcnt); pv_wait(&l->locked, _Q_SLOW_VAL);