From patchwork Wed Nov 2 09:08:29 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xinhui X-Patchwork-Id: 9408517 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 34A4C60234 for ; Wed, 2 Nov 2016 05:15:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 265CB29E05 for ; Wed, 2 Nov 2016 05:15:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1A73729E08; Wed, 2 Nov 2016 05:15:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.2 required=2.0 tests=BAYES_00, DATE_IN_FUTURE_03_06, RCVD_IN_DNSWL_MED autolearn=no version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C36E329E05 for ; Wed, 2 Nov 2016 05:15:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1nrq-0001xX-OG; Wed, 02 Nov 2016 05:13:42 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1c1nrp-0001x1-7f for xen-devel@lists.xenproject.org; Wed, 02 Nov 2016 05:13:41 +0000 Received: from [85.158.139.211] by server-4.bemta-5.messagelabs.com id EB/45-22514-40679185; Wed, 02 Nov 2016 05:13:40 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBIsWRWlGSWpSXmKPExsUyZfEcRl3mMsk Ig+8rTS2+b5nM5MDocfjDFZYAxijWzLyk/IoE1oz/7yazFEwWqVj/9hpjA2OPYBcjJ4eQwCVG idazkl2MXEB2C5PEqu0/GCGcN4wSB39PY4dwTjNKXPx9kgnCWc8o8frEVKiynYwSe49/Z4Nw1 jFKbHi2Cipzm1Hi7KyrrCBr2AQMJG5c2MQCkhAR6GaSeLVhDSNIglngPLPE+S8GILawgJvEsb YOsDiLgKrE0YedQNs5OCQE5CWe3a4HCXMK+Ek8+riMGeJ0X4llF76DlUsIiEhM/PueDcJWlbg 4ZyYzhK0jcXPKTSjbQmJFA8Q9EgJGEt/eLgSr5wWac7L5GlRvlsT+vbfB7pQQ+M8o8eX0QvYJ jBILGBlWMaoXpxaVpRbpmuslFWWmZ5TkJmbm6BoamOrlphYXJ6an5iQmFesl5+duYgTGCAMQ7 GA8Ntn5EKMkB5OSKK9NsmSEEF9SfkplRmJxRnxRaU5q8SFGGQ4OJQlekxKgnGBRanpqRVpmDj BaYdISHDxKIrzmIGne4oLE3OLMdIjUKUZFKXFeFZCEAEgiozQPrg2WIC4xykoJ8zICHSLEU5B alJtZgir/ilGcg1FJmNcGZApPZl4J3PRXQIuZQBYnSYAsLklESEk1MCab/Rb7kXmlz7jm/Jzi A0+lekxeLW7ptJh650PNlCP9pjPMU1wKXS49ZHCLeF3T6TDDUJc9N7fWv30e+6abK35OeKrQq rPjYjbDlvWKGzcy5JgXSCmJTLq9T9pmRvZ+Dz/O33+OXi3dHHhLLqLzzXtD4zkpO32UQ9eWi+ y+2TNN8aT9UjGti0osxRmJhlrMRcWJACLvsR8LAwAA X-Env-Sender: xinhui.pan@linux.vnet.ibm.com X-Msg-Ref: server-9.tower-206.messagelabs.com!1478063618!68143373!1 X-Originating-IP: [148.163.156.1] X-SpamReason: No, hits=2.1 required=7.0 tests=DATE_IN_FUTURE_03_06, UNPARSEABLE_RELAY X-StarScan-Received: X-StarScan-Version: 9.0.13; banners=-,-,- X-VirusChecked: Checked Received: (qmail 13293 invoked from network); 2 Nov 2016 05:13:39 -0000 Received: from mx0a-001b2d01.pphosted.com (HELO mx0a-001b2d01.pphosted.com) (148.163.156.1) by server-9.tower-206.messagelabs.com with AES256-SHA encrypted SMTP; 2 Nov 2016 05:13:39 -0000 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id uA25DQZT132015 for ; Wed, 2 Nov 2016 01:13:38 -0400 Received: from e23smtp05.au.ibm.com (e23smtp05.au.ibm.com [202.81.31.147]) by mx0a-001b2d01.pphosted.com with ESMTP id 26f7awgq8x-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 02 Nov 2016 01:13:37 -0400 Received: from localhost by e23smtp05.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 2 Nov 2016 15:13:35 +1000 Received: from d23dlp02.au.ibm.com (202.81.31.213) by e23smtp05.au.ibm.com (202.81.31.211) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 2 Nov 2016 15:13:32 +1000 Received: from d23relay10.au.ibm.com (d23relay10.au.ibm.com [9.190.26.77]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id 5A0142BB0059; Wed, 2 Nov 2016 16:13:32 +1100 (EST) Received: from d23av02.au.ibm.com (d23av02.au.ibm.com [9.190.235.138]) by d23relay10.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id uA25DWdY3145998; Wed, 2 Nov 2016 16:13:32 +1100 Received: from d23av02.au.ibm.com (localhost [127.0.0.1]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id uA25DUX1007248; Wed, 2 Nov 2016 16:13:32 +1100 Received: from ltcalpine2-lp13.aus.stglabs.ibm.com (ltcalpine2-lp13.aus.stglabs.ibm.com [9.40.195.196]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id uA25CnXg006036; Wed, 2 Nov 2016 16:13:25 +1100 From: Pan Xinhui To: linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, xen-devel-request@lists.xenproject.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, x86@kernel.org Date: Wed, 2 Nov 2016 05:08:29 -0400 X-Mailer: git-send-email 2.4.11 In-Reply-To: <1478077718-37424-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> References: <1478077718-37424-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16110205-0016-0000-0000-000001E61B9F X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16110205-0017-0000-0000-000005C0798C Message-Id: <1478077718-37424-3-git-send-email-xinhui.pan@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-11-02_02:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1611020096 Cc: kernellwp@gmail.com, jgross@suse.com, dave@stgolabs.net, David.Laight@ACULAB.COM, rkrcmar@redhat.com, peterz@infradead.org, benh@kernel.crashing.org, bsingharora@gmail.com, will.deacon@arm.com, Pan Xinhui , borntraeger@de.ibm.com, mingo@redhat.com, paulus@samba.org, mpe@ellerman.id.au, pbonzini@redhat.com, paulmck@linux.vnet.ibm.com, boqun.feng@gmail.com Subject: [Xen-devel] [PATCH v7 02/11] locking/osq: Drop the overload of osq_lock() X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP An over-committed guest with more vCPUs than pCPUs has a heavy overload in osq_lock(). This is because vCPU A hold the osq lock and yield out, vCPU B wait per_cpu node->locked to be set. IOW, vCPU B wait vCPU A to run and unlock the osq lock. Kernel has an interface bool vcpu_is_preempted(int cpu) to detect if a vCPU is currently running or not. So break the spin loops on true condition. test case: perf record -a perf bench sched messaging -g 400 -p && perf report before patch: 18.09% sched-messaging [kernel.vmlinux] [k] osq_lock 12.28% sched-messaging [kernel.vmlinux] [k] rwsem_spin_on_owner 5.27% sched-messaging [kernel.vmlinux] [k] mutex_unlock 3.89% sched-messaging [kernel.vmlinux] [k] wait_consider_task 3.64% sched-messaging [kernel.vmlinux] [k] _raw_write_lock_irq 3.41% sched-messaging [kernel.vmlinux] [k] mutex_spin_on_owner.is 2.49% sched-messaging [kernel.vmlinux] [k] system_call after patch: 20.68% sched-messaging [kernel.vmlinux] [k] mutex_spin_on_owner 8.45% sched-messaging [kernel.vmlinux] [k] mutex_unlock 4.12% sched-messaging [kernel.vmlinux] [k] system_call 3.01% sched-messaging [kernel.vmlinux] [k] system_call_common 2.83% sched-messaging [kernel.vmlinux] [k] copypage_power7 2.64% sched-messaging [kernel.vmlinux] [k] rwsem_spin_on_owner 2.00% sched-messaging [kernel.vmlinux] [k] osq_lock Suggested-by: Boqun Feng Signed-off-by: Pan Xinhui Acked-by: Christian Borntraeger Acked-by: Paolo Bonzini Tested-by: Juergen Gross --- kernel/locking/osq_lock.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c index 05a3785..091f97f 100644 --- a/kernel/locking/osq_lock.c +++ b/kernel/locking/osq_lock.c @@ -21,6 +21,11 @@ static inline int encode_cpu(int cpu_nr) return cpu_nr + 1; } +static inline int node_cpu(struct optimistic_spin_node *node) +{ + return node->cpu - 1; +} + static inline struct optimistic_spin_node *decode_cpu(int encoded_cpu_val) { int cpu_nr = encoded_cpu_val - 1; @@ -118,8 +123,9 @@ bool osq_lock(struct optimistic_spin_queue *lock) while (!READ_ONCE(node->locked)) { /* * If we need to reschedule bail... so we can block. + * Use vcpu_is_preempted to detect lock holder preemption issue. */ - if (need_resched()) + if (need_resched() || vcpu_is_preempted(node_cpu(node->prev))) goto unqueue; cpu_relax_lowlatency();