From patchwork Sun Jun 2 03:42:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Chen" X-Patchwork-Id: 10971611 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F337912 for ; Sun, 2 Jun 2019 03:53:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 185EF28C12 for ; Sun, 2 Jun 2019 03:53:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0625B28C61; Sun, 2 Jun 2019 03:53:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5F24A28C12 for ; Sun, 2 Jun 2019 03:53:15 +0000 (UTC) Received: from localhost ([127.0.0.1]:44515 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHYY-0002vh-Ps for patchwork-qemu-devel@patchwork.kernel.org; Sat, 01 Jun 2019 23:53:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47056) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHSp-0007Dy-MC for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXHSn-00039n-W2 for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:19 -0400 Received: from mga12.intel.com ([192.55.52.136]:51834) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hXHSn-00032p-Mo for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:17 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Jun 2019 20:47:11 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 01 Jun 2019 20:47:09 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev Date: Sun, 2 Jun 2019 11:42:34 +0800 Message-Id: <20190602034239.23225-2-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190602034239.23225-1-chen.zhang@intel.com> References: <20190602034239.23225-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.136 Subject: [Qemu-devel] [PATCH 1/6] COLO-compare: Add new parameter to communicate with remote colo-frame X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen We add the "notify_dev=chardevID" parameter. After that colo-compare can connect with remote(currently just for Xen, KVM-COLO didn't need it.) colo-frame through chardev socket, it can notify remote(Xen) colo-frame to handle checkpoint event. Signed-off-by: Zhang Chen --- net/colo-compare.c | 21 +++++++++++++++++++++ qemu-options.hx | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index fcb491121b..21849748b6 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -83,6 +83,7 @@ typedef struct CompareState { char *pri_indev; char *sec_indev; char *outdev; + char *notify_dev; CharBackend chr_pri_in; CharBackend chr_sec_in; CharBackend chr_out; @@ -897,6 +898,21 @@ static void compare_set_vnet_hdr(Object *obj, s->vnet_hdr = value; } +static char *compare_get_notify_dev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->notify_dev); +} + +static void compare_set_notify_dev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->notify_dev); + s->notify_dev = g_strdup(value); +} + static void compare_pri_rs_finalize(SocketReadState *pri_rs) { CompareState *s = container_of(pri_rs, CompareState, pri_rs); @@ -1057,6 +1073,10 @@ static void colo_compare_init(Object *obj) (Object **)&s->iothread, object_property_allow_set_link, OBJ_PROP_LINK_STRONG, NULL); + /* This parameter just for Xen COLO */ + object_property_add_str(obj, "notify_dev", + compare_get_notify_dev, compare_set_notify_dev, + NULL); s->vnet_hdr = false; object_property_add_bool(obj, "vnet_hdr_support", compare_get_vnet_hdr, @@ -1103,6 +1123,7 @@ static void colo_compare_finalize(Object *obj) g_free(s->pri_indev); g_free(s->sec_indev); g_free(s->outdev); + g_free(s->notify_dev); } static const TypeInfo colo_compare_info = { diff --git a/qemu-options.hx b/qemu-options.hx index 7ae3373a00..30e1ca754f 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -4475,7 +4475,7 @@ Dump the network traffic on netdev @var{dev} to the file specified by The file format is libpcap, so it can be analyzed with tools such as tcpdump or Wireshark. -@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},outdev=@var{chardevid},iothread=@var{id}[,vnet_hdr_support] +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},outdev=@var{chardevid},iothread=@var{id}[,vnet_hdr_support][,notify_dev=@var{id}] Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with secondary packet. If the packets are same, we will output primary @@ -4484,11 +4484,15 @@ do checkpoint and send primary packet to outdev@var{chardevid}. In order to improve efficiency, we need to put the task of comparison in another thread. If it has the vnet_hdr_support flag, colo compare will send/recv packet with vnet_hdr_len. +If you want to use Xen COLO, will need the notify_dev to notify Xen +colo-frame to do checkpoint. we must use it with the help of filter-mirror and filter-redirector. @example +KVM COLO + primary: -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 @@ -4512,6 +4516,33 @@ secondary: -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 + +Xen COLO + +primary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 +-chardev socket,id=notify_way,host=3.3.3.3,port=9009,server,nowait +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 +-object iothread,id=iothread1 +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0,notify_dev=nofity_way,iothread=iothread1 + +secondary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=red0,host=3.3.3.3,port=9003 +-chardev socket,id=red1,host=3.3.3.3,port=9004 +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 + @end example If you want to know the detail of above command line, you can read From patchwork Sun Jun 2 03:42:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Chen" X-Patchwork-Id: 10971601 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1ECD17DF for ; Sun, 2 Jun 2019 03:48:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D13CD28B82 for ; Sun, 2 Jun 2019 03:48:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C51F728B87; Sun, 2 Jun 2019 03:48:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 56C6028B82 for ; Sun, 2 Jun 2019 03:48:33 +0000 (UTC) Received: from localhost ([127.0.0.1]:44454 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHU0-000865-NP for patchwork-qemu-devel@patchwork.kernel.org; Sat, 01 Jun 2019 23:48:32 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47045) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHSp-0007Dw-3g for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXHSn-00039f-R5 for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:19 -0400 Received: from mga12.intel.com ([192.55.52.136]:51836) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hXHSn-00035L-I5 for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:17 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Jun 2019 20:47:13 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 01 Jun 2019 20:47:11 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev Date: Sun, 2 Jun 2019 11:42:35 +0800 Message-Id: <20190602034239.23225-3-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190602034239.23225-1-chen.zhang@intel.com> References: <20190602034239.23225-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.136 Subject: [Qemu-devel] [PATCH 2/6] COLO-compare: Add remote notification chardev handler frame X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen Add chardev handler to send notification to remote(current from Xen) colo-frame. Signed-off-by: Zhang Chen --- net/colo-compare.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/net/colo-compare.c b/net/colo-compare.c index 21849748b6..8b7bd9f0c7 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -87,8 +87,10 @@ typedef struct CompareState { CharBackend chr_pri_in; CharBackend chr_sec_in; CharBackend chr_out; + CharBackend chr_notify_dev; SocketReadState pri_rs; SocketReadState sec_rs; + SocketReadState notify_rs; bool vnet_hdr; /* @@ -745,6 +747,19 @@ static void compare_sec_chr_in(void *opaque, const uint8_t *buf, int size) } } +static void compare_notify_chr(void *opaque, const uint8_t *buf, int size) +{ + CompareState *s = COLO_COMPARE(opaque); + int ret; + + ret = net_fill_rstate(&s->notify_rs, buf, size); + if (ret == -1) { + qemu_chr_fe_set_handlers(&s->chr_notify_dev, NULL, NULL, NULL, NULL, + NULL, NULL, true); + error_report("colo-compare notify_dev error"); + } +} + /* * Check old packet regularly so it can watch for any packets * that the secondary hasn't produced equivalents of. @@ -832,6 +847,11 @@ static void colo_compare_iothread(CompareState *s) qemu_chr_fe_set_handlers(&s->chr_sec_in, compare_chr_can_read, compare_sec_chr_in, NULL, NULL, s, s->worker_context, true); + if (s->notify_dev) { + qemu_chr_fe_set_handlers(&s->chr_notify_dev, compare_chr_can_read, + compare_notify_chr, NULL, NULL, + s, s->worker_context, true); + } colo_compare_timer_init(s); s->event_bh = qemu_bh_new(colo_compare_handle_event, s); @@ -943,6 +963,10 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs) } } +static void compare_notify_rs_finalize(SocketReadState *notify_rs) +{ + /* Get Xen colo-frame's notify and handle the message */ +} /* * Return 0 is success. @@ -1013,6 +1037,17 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp) net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, s->vnet_hdr); net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, s->vnet_hdr); + /* Try to enable remote notify chardev, currently just for Xen COLO */ + if (s->notify_dev) { + if (find_and_check_chardev(&chr, s->notify_dev, errp) || + !qemu_chr_fe_init(&s->chr_notify_dev, chr, errp)) { + return; + } + + net_socket_rs_init(&s->notify_rs, compare_notify_rs_finalize, + s->vnet_hdr); + } + QTAILQ_INSERT_TAIL(&net_compares, s, next); g_queue_init(&s->conn_list); @@ -1091,6 +1126,10 @@ static void colo_compare_finalize(Object *obj) qemu_chr_fe_deinit(&s->chr_pri_in, false); qemu_chr_fe_deinit(&s->chr_sec_in, false); qemu_chr_fe_deinit(&s->chr_out, false); + if (s->notify_dev) { + qemu_chr_fe_deinit(&s->chr_notify_dev, false); + } + if (s->iothread) { colo_compare_timer_del(s); } From patchwork Sun Jun 2 03:42:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Chen" X-Patchwork-Id: 10971609 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 245476C5 for ; Sun, 2 Jun 2019 03:52:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E34928C12 for ; Sun, 2 Jun 2019 03:52:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 00D9428C61; Sun, 2 Jun 2019 03:52:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 864CE28C12 for ; Sun, 2 Jun 2019 03:52:06 +0000 (UTC) Received: from localhost ([127.0.0.1]:44509 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHXR-0002LH-IS for patchwork-qemu-devel@patchwork.kernel.org; Sat, 01 Jun 2019 23:52:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47060) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHSp-0007Dz-OL for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXHSo-0003AG-Bi for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:19 -0400 Received: from mga12.intel.com ([192.55.52.136]:51833) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hXHSo-00030z-1H for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:18 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Jun 2019 20:47:15 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 01 Jun 2019 20:47:13 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev Date: Sun, 2 Jun 2019 11:42:36 +0800 Message-Id: <20190602034239.23225-4-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190602034239.23225-1-chen.zhang@intel.com> References: <20190602034239.23225-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.136 Subject: [Qemu-devel] [PATCH 3/6] COLO-compare: Make the compare_chr_send() can send notification message. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen We need use this function to send notification message for remote colo-frame(Xen). So we add new parameter for this job. Signed-off-by: Zhang Chen --- net/colo-compare.c | 41 +++++++++++++++++++++++++++++++++-------- 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 8b7bd9f0c7..16285f4a96 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -129,7 +129,8 @@ static void colo_compare_inconsistency_notify(void) static int compare_chr_send(CompareState *s, const uint8_t *buf, uint32_t size, - uint32_t vnet_hdr_len); + uint32_t vnet_hdr_len, + bool notify_remote_frame); static gint seq_sorter(Packet *a, Packet *b, gpointer data) { @@ -241,7 +242,8 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt) ret = compare_chr_send(s, pkt->data, pkt->size, - pkt->vnet_hdr_len); + pkt->vnet_hdr_len, + false); if (ret < 0) { error_report("colo send primary packet failed"); } @@ -671,7 +673,8 @@ static void colo_compare_connection(void *opaque, void *user_data) static int compare_chr_send(CompareState *s, const uint8_t *buf, uint32_t size, - uint32_t vnet_hdr_len) + uint32_t vnet_hdr_len, + bool notify_remote_frame) { int ret = 0; uint32_t len = htonl(size); @@ -680,7 +683,14 @@ static int compare_chr_send(CompareState *s, return 0; } - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len)); + if (notify_remote_frame) { + ret = qemu_chr_fe_write_all(&s->chr_notify_dev, + (uint8_t *)&len, + sizeof(len)); + } else { + ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len)); + } + if (ret != sizeof(len)) { goto err; } @@ -691,13 +701,26 @@ static int compare_chr_send(CompareState *s, * know how to parse net packet correctly. */ len = htonl(vnet_hdr_len); - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len)); + + if (!notify_remote_frame) { + ret = qemu_chr_fe_write_all(&s->chr_out, + (uint8_t *)&len, + sizeof(len)); + } + if (ret != sizeof(len)) { goto err; } } - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size); + if (notify_remote_frame) { + ret = qemu_chr_fe_write_all(&s->chr_notify_dev, + (uint8_t *)buf, + size); + } else { + ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size); + } + if (ret != size) { goto err; } @@ -943,7 +966,8 @@ static void compare_pri_rs_finalize(SocketReadState *pri_rs) compare_chr_send(s, pri_rs->buf, pri_rs->packet_len, - pri_rs->vnet_hdr_len); + pri_rs->vnet_hdr_len, + false); } else { /* compare packet in the specified connection */ colo_compare_connection(conn, s); @@ -1075,7 +1099,8 @@ static void colo_flush_packets(void *opaque, void *user_data) compare_chr_send(s, pkt->data, pkt->size, - pkt->vnet_hdr_len); + pkt->vnet_hdr_len, + false); packet_destroy(pkt, NULL); } while (!g_queue_is_empty(&conn->secondary_list)) { From patchwork Sun Jun 2 03:42:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Chen" X-Patchwork-Id: 10971605 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 887861515 for ; Sun, 2 Jun 2019 03:50:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67CD928968 for ; Sun, 2 Jun 2019 03:50:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 58517289A2; Sun, 2 Jun 2019 03:50:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E253028968 for ; Sun, 2 Jun 2019 03:50:35 +0000 (UTC) Received: from localhost ([127.0.0.1]:44499 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHVz-0001D0-7u for patchwork-qemu-devel@patchwork.kernel.org; Sat, 01 Jun 2019 23:50:35 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47055) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHSp-0007Dx-MF for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXHSo-0003A2-7b for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:19 -0400 Received: from mga12.intel.com ([192.55.52.136]:51836) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hXHSo-00035L-1x for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:18 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Jun 2019 20:47:17 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 01 Jun 2019 20:47:15 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev Date: Sun, 2 Jun 2019 11:42:37 +0800 Message-Id: <20190602034239.23225-5-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190602034239.23225-1-chen.zhang@intel.com> References: <20190602034239.23225-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.136 Subject: [Qemu-devel] [PATCH 4/6] COLO-compare: Add colo-compare remote notify support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen This patch make colo-compare can send message to remote COLO frame(Xen) when occur checkpoint. Signed-off-by: Zhang Chen Signed-off-by: Zhang Chen --- net/colo-compare.c | 51 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 46 insertions(+), 5 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 16285f4a96..19075c7a66 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -251,6 +251,17 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt) packet_destroy(pkt, NULL); } +static void notify_remote_frame(CompareState *s) +{ + char msg[] = "DO_CHECKPOINT"; + int ret = 0; + + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true); + if (ret < 0) { + error_report("Notify Xen COLO-frame failed"); + } +} + /* * The IP packets sent by primary and secondary * will be compared in here @@ -435,7 +446,11 @@ sec: qemu_hexdump((char *)spkt->data, stderr, "colo-compare spkt", spkt->size); - colo_compare_inconsistency_notify(); + if (s->notify_dev) { + notify_remote_frame(s); + } else { + colo_compare_inconsistency_notify(); + } } } @@ -577,7 +592,7 @@ void colo_compare_unregister_notifier(Notifier *notify) } static int colo_old_packet_check_one_conn(Connection *conn, - void *user_data) + CompareState *s) { GList *result = NULL; int64_t check_time = REGULAR_PACKET_CHECK_MS; @@ -588,7 +603,11 @@ static int colo_old_packet_check_one_conn(Connection *conn, if (result) { /* Do checkpoint will flush old packet */ - colo_compare_inconsistency_notify(); + if (s->notify_dev) { + notify_remote_frame(s); + } else { + colo_compare_inconsistency_notify(); + } return 0; } @@ -608,7 +627,7 @@ static void colo_old_packet_check(void *opaque) * If we find one old packet, stop finding job and notify * COLO frame do checkpoint. */ - g_queue_find_custom(&s->conn_list, NULL, + g_queue_find_custom(&s->conn_list, s, (GCompareFunc)colo_old_packet_check_one_conn); } @@ -637,7 +656,12 @@ static void colo_compare_packet(CompareState *s, Connection *conn, */ trace_colo_compare_main("packet different"); g_queue_push_head(&conn->primary_list, pkt); - colo_compare_inconsistency_notify(); + + if (s->notify_dev) { + notify_remote_frame(s); + } else { + colo_compare_inconsistency_notify(); + } break; } } @@ -989,7 +1013,24 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs) static void compare_notify_rs_finalize(SocketReadState *notify_rs) { + CompareState *s = container_of(notify_rs, CompareState, notify_rs); + /* Get Xen colo-frame's notify and handle the message */ + char *data = g_memdup(notify_rs->buf, notify_rs->packet_len); + char msg[] = "COLO_COMPARE_GET_XEN_INIT"; + int ret; + + if (!strcmp(data, "COLO_USERSPACE_PROXY_INIT")) { + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true); + if (ret < 0) { + error_report("Notify Xen COLO-frame INIT failed"); + } + } + + if (!strcmp(data, "COLO_CHECKPOINT")) { + /* colo-compare do checkpoint, flush pri packet and remove sec packet */ + g_queue_foreach(&s->conn_list, colo_flush_packets, s); + } } /* From patchwork Sun Jun 2 03:42:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Chen" X-Patchwork-Id: 10971607 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 188E4912 for ; Sun, 2 Jun 2019 03:50:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 069182897F for ; Sun, 2 Jun 2019 03:50:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EEA9228968; Sun, 2 Jun 2019 03:50:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A013828968 for ; Sun, 2 Jun 2019 03:50:36 +0000 (UTC) Received: from localhost ([127.0.0.1]:44501 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHW0-0001DO-1G for patchwork-qemu-devel@patchwork.kernel.org; Sat, 01 Jun 2019 23:50:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47085) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHSr-0007EC-1I for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXHSq-0003Cu-0F for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:20 -0400 Received: from mga12.intel.com ([192.55.52.136]:51833) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hXHSp-00030z-Ns for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:19 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Jun 2019 20:47:19 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 01 Jun 2019 20:47:17 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev Date: Sun, 2 Jun 2019 11:42:38 +0800 Message-Id: <20190602034239.23225-6-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190602034239.23225-1-chen.zhang@intel.com> References: <20190602034239.23225-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.136 Subject: [Qemu-devel] [PATCH 5/6] COLO-compare: Rename the colo_compare_inconsistency_notify X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen We have add the notify_remote_frame function for Xen, so we rename the colo_compare_inconsistency_notify to notify_native_frame(KVM-qemu COLO frame) looks better. Signed-off-by: Zhang Chen --- net/colo-compare.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 19075c7a66..101c1f477e 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -120,7 +120,7 @@ enum { SECONDARY_IN, }; -static void colo_compare_inconsistency_notify(void) +static void notify_native_frame(void) { notifier_list_notify(&colo_compare_notifiers, migrate_get_current()); @@ -449,7 +449,7 @@ sec: if (s->notify_dev) { notify_remote_frame(s); } else { - colo_compare_inconsistency_notify(); + notify_native_frame(); } } } @@ -606,7 +606,7 @@ static int colo_old_packet_check_one_conn(Connection *conn, if (s->notify_dev) { notify_remote_frame(s); } else { - colo_compare_inconsistency_notify(); + notify_native_frame(); } return 0; } @@ -660,7 +660,7 @@ static void colo_compare_packet(CompareState *s, Connection *conn, if (s->notify_dev) { notify_remote_frame(s); } else { - colo_compare_inconsistency_notify(); + notify_native_frame(); } break; } From patchwork Sun Jun 2 03:42:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Chen" X-Patchwork-Id: 10971603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B806C912 for ; Sun, 2 Jun 2019 03:48:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A879528B82 for ; Sun, 2 Jun 2019 03:48:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9CB8228B87; Sun, 2 Jun 2019 03:48:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5C30628B82 for ; Sun, 2 Jun 2019 03:48:37 +0000 (UTC) Received: from localhost ([127.0.0.1]:44456 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHU4-00089e-MT for patchwork-qemu-devel@patchwork.kernel.org; Sat, 01 Jun 2019 23:48:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47097) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXHSs-0007Gf-QT for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXHSr-0003FJ-RF for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:22 -0400 Received: from mga12.intel.com ([192.55.52.136]:51833) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hXHSr-00030z-Ji for qemu-devel@nongnu.org; Sat, 01 Jun 2019 23:47:21 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Jun 2019 20:47:21 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 01 Jun 2019 20:47:19 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev Date: Sun, 2 Jun 2019 11:42:39 +0800 Message-Id: <20190602034239.23225-7-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190602034239.23225-1-chen.zhang@intel.com> References: <20190602034239.23225-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.136 Subject: [Qemu-devel] [PATCH 6/6] migration/colo.c: Add missed filter notify for Xen COLO. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen We need to notify net filter to do checkpoint for Xen COLO, like KVM side. Signed-off-by: Zhang Chen --- migration/colo.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 8c1644091f..9f84b1fa3c 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -259,6 +259,8 @@ ReplicationStatus *qmp_query_xen_replication_status(Error **errp) void qmp_xen_colo_do_checkpoint(Error **errp) { replication_do_checkpoint_all(errp); + /* Notify all filters of all NIC to do checkpoint */ + colo_notify_filters_event(COLO_EVENT_CHECKPOINT, errp); } #endif