From patchwork Mon May 20 04:22:11 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qinchuanyu X-Patchwork-Id: 2590731 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 1CABDDF2A2 for ; Mon, 20 May 2013 04:22:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751362Ab3ETEWd (ORCPT ); Mon, 20 May 2013 00:22:33 -0400 Received: from szxga02-in.huawei.com ([119.145.14.65]:54894 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750887Ab3ETEWb (ORCPT ); Mon, 20 May 2013 00:22:31 -0400 Received: from 172.24.2.119 (EHLO szxeml210-edg.china.huawei.com) ([172.24.2.119]) by szxrg02-dlp.huawei.com (MOS 4.3.4-GA FastPath queued) with ESMTP id BBU80538; Mon, 20 May 2013 12:22:20 +0800 (CST) Received: from SZXEML461-HUB.china.huawei.com (10.82.67.204) by szxeml210-edg.china.huawei.com (172.24.2.183) with Microsoft SMTP Server (TLS) id 14.1.323.7; Mon, 20 May 2013 12:22:17 +0800 Received: from NKGEML401-HUB.china.huawei.com (10.98.56.32) by szxeml461-hub.china.huawei.com (10.82.67.204) with Microsoft SMTP Server (TLS) id 14.1.323.7; Mon, 20 May 2013 12:22:19 +0800 Received: from NKGEML511-MBX.china.huawei.com ([169.254.5.214]) by nkgeml401-hub.china.huawei.com ([10.98.56.32]) with mapi id 14.01.0323.007; Mon, 20 May 2013 12:22:11 +0800 From: Qinchuanyu To: Jason Wang CC: "rusty@rustcorp.com.au" , "mst@redhat.com" , "dhowells@redhat.com" , "(kvm@vger.kernel.org)" , "(netdev@vger.kernel.org)" , Heguansen Subject: Re: [PATCH] vhost: get 2% performance improved by reducing spin_lock race in vhost_work_queue Thread-Topic: [PATCH] vhost: get 2% performance improved by reducing spin_lock race in vhost_work_queue Thread-Index: Ac5VBvou1TggFPyiQYWhLbN9xw3Ujv//gtUA//9vwMA= Date: Mon, 20 May 2013 04:22:11 +0000 Message-ID: <5872DA217C2FF7488B20897D84F904E7338FD2C5@nkgeml511-mbx.china.huawei.com> References: <5872DA217C2FF7488B20897D84F904E7338FD1E5@nkgeml511-mbx.china.huawei.com> <51999AA2.8070806@redhat.com> In-Reply-To: <51999AA2.8070806@redhat.com> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-cr-hashedpuzzle: APeR FAQO F0z6 Kuk9 QkfZ QpS1 R3up SDFG VWPK XYcP X3zw e8ay iH4Z pbrD q2zA 2aZ0; 6; ZABoAG8AdwBlAGwAbABzAEAAcgBlAGQAaABhAHQALgBjAG8AbQA7AGoAYQBzAG8AdwBhAG4AZwBAAHIAZQBkAGgAYQB0AC4AYwBvAG0AOwBrAHYAbQBAAHYAZwBlAHIALgBrAGUAcgBuAGUAbAAuAG8AcgBnADsAbQBzAHQAQAByAGUAZABoAGEAdAAuAGMAbwBtADsAbgBlAHQAZABlAHYAQAB2AGcAZQByAC4AawBlAHIAbgBlAGwALgBvAHIAZwA7AHIAdQBzAHQAeQBAAHIAdQBzAHQAYwBvAHIAcAAuAGMAbwBtAC4AYQB1AA==; Sosha1_v1; 7; {48F28E5A-178D-4AD8-9119-6BB66622F1DA}; cQBpAG4AYwBoAHUAYQBuAHkAdQBAAGgAdQBhAHcAZQBpAC4AYwBvAG0A; Mon, 20 May 2013 04:21:56 GMT; UgBlADoAIABbAFAAQQBUAEMASABdACAAdgBoAG8AcwB0ADoAIABnAGUAdAAgADIAJQAgAHAAZQByAGYAbwByAG0AYQBuAGMAZQAgAGkAbQBwAHIAbwB2AGUAZAAgAGIAeQAgAHIAZQBkAHUAYwBpAG4AZwAgAHMAcABpAG4AXwBsAG8AYwBrACAAcgBhAGMAZQAgAGkAbgAgAHYAaABvAHMAdABfAHcAbwByAGsAXwBxAHUAZQB1AGUA x-cr-puzzleid: {48F28E5A-178D-4AD8-9119-6BB66622F1DA} x-originating-ip: [10.135.68.166] MIME-Version: 1.0 X-CFilter-Loop: Reflected Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The patch below is base on https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/drivers/vhost/vhost.c?id=refs/tags/next-20130517 Signed-off-by: Chuanyu Qin I did the test by using iperf in 10G environment, the test num as below: orignal modified thread_num tp(Gbps) vhost(%) | tp(Gbps) vhost(%) 1 9.59 28.82 | 9.59 27.49 8 9.61 32.92 | 9.62 26.77 64 9.58 46.48 | 9.55 38.99 256 9.6 63.7 | 9.6 52.59 The cost of vhost reduced while the throughput is almost unchanged. On 05/20/2013 11:06 AM, Qinchuanyu wrote: > Right now the wake_up_process func is included in spin_lock/unlock, but it could be done outside the spin_lock. > I have test it with kernel 3.0.27 and guest suse11-sp2, it provide 2%-3% net performance improved. > > Signed-off-by: Chuanyu Qin Make sense to me but need generate a patch against net-next.git or vhost.git in git.kernel.org. Btw. How did you test this? Care to share the perf numbers? Thanks > mu > --- a/drivers/vhost/vhost.c 2013-05-20 10:36:30.000000000 +0800 > +++ b/drivers/vhost/vhost.c 2013-05-20 10:36:54.000000000 +0800 > @@ -144,9 +144,10 @@ > if (list_empty(&work->node)) { > list_add_tail(&work->node, &dev->work_list); > work->queue_seq++; > + spin_unlock_irqrestore(&dev->work_lock, flags); > wake_up_process(dev->worker); > - } > - spin_unlock_irqrestore(&dev->work_lock, flags); > + } else > + spin_unlock_irqrestore(&dev->work_lock, flags); > } > > void vhost_poll_queue(struct vhost_poll *poll) > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html --- a/drivers/vhost/vhost.c 2013-05-20 11:47:05.000000000 +0800 +++ b/drivers/vhost/vhost.c 2013-05-20 11:48:24.000000000 +0800 @@ -154,9 +154,10 @@ if (list_empty(&work->node)) { list_add_tail(&work->node, &dev->work_list); work->queue_seq++; + spin_unlock_irqrestore(&dev->work_lock, flags); wake_up_process(dev->worker); - } - spin_unlock_irqrestore(&dev->work_lock, flags); + } else + spin_unlock_irqrestore(&dev->work_lock, flags); } void vhost_poll_queue(struct vhost_poll *poll)