From patchwork Sun Jun 9 16:44:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 10983749 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24A851398 for ; Sun, 9 Jun 2019 16:50:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 135741FFFF for ; Sun, 9 Jun 2019 16:50:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 07373204FA; Sun, 9 Jun 2019 16:50:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=no version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8BB3A1FFFF for ; Sun, 9 Jun 2019 16:50:42 +0000 (UTC) Received: from localhost ([::1]:37060 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha11l-0002PP-Qy for patchwork-qemu-devel@patchwork.kernel.org; Sun, 09 Jun 2019 12:50:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56515) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha10S-0000QF-5S for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ha10Q-0007Fx-Ou for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:20 -0400 Received: from mga14.intel.com ([192.55.52.115]:62032) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ha10Q-0007CA-FQ for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:18 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jun 2019 09:49:11 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 09 Jun 2019 09:49:09 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev , Stefano Stabellini , Paul Durrant Date: Mon, 10 Jun 2019 00:44:29 +0800 Message-Id: <20190609164433.5866-2-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190609164433.5866-1-chen.zhang@intel.com> References: <20190609164433.5866-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH V2 1/5] COLO-compare: Add new parameter to communicate with remote colo-frame X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen We add the "notify_dev=chardevID" parameter. After that colo-compare can connect with remote(currently just for Xen, KVM-COLO didn't need it.) colo-frame through chardev socket, it can notify remote(Xen) colo-frame to handle checkpoint event. Signed-off-by: Zhang Chen --- net/colo-compare.c | 21 +++++++++++++++++++++ qemu-options.hx | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index fcb491121b..21849748b6 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -83,6 +83,7 @@ typedef struct CompareState { char *pri_indev; char *sec_indev; char *outdev; + char *notify_dev; CharBackend chr_pri_in; CharBackend chr_sec_in; CharBackend chr_out; @@ -897,6 +898,21 @@ static void compare_set_vnet_hdr(Object *obj, s->vnet_hdr = value; } +static char *compare_get_notify_dev(Object *obj, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + return g_strdup(s->notify_dev); +} + +static void compare_set_notify_dev(Object *obj, const char *value, Error **errp) +{ + CompareState *s = COLO_COMPARE(obj); + + g_free(s->notify_dev); + s->notify_dev = g_strdup(value); +} + static void compare_pri_rs_finalize(SocketReadState *pri_rs) { CompareState *s = container_of(pri_rs, CompareState, pri_rs); @@ -1057,6 +1073,10 @@ static void colo_compare_init(Object *obj) (Object **)&s->iothread, object_property_allow_set_link, OBJ_PROP_LINK_STRONG, NULL); + /* This parameter just for Xen COLO */ + object_property_add_str(obj, "notify_dev", + compare_get_notify_dev, compare_set_notify_dev, + NULL); s->vnet_hdr = false; object_property_add_bool(obj, "vnet_hdr_support", compare_get_vnet_hdr, @@ -1103,6 +1123,7 @@ static void colo_compare_finalize(Object *obj) g_free(s->pri_indev); g_free(s->sec_indev); g_free(s->outdev); + g_free(s->notify_dev); } static const TypeInfo colo_compare_info = { diff --git a/qemu-options.hx b/qemu-options.hx index 0d8beb4afd..c18b79099a 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -4477,7 +4477,7 @@ Dump the network traffic on netdev @var{dev} to the file specified by The file format is libpcap, so it can be analyzed with tools such as tcpdump or Wireshark. -@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},outdev=@var{chardevid},iothread=@var{id}[,vnet_hdr_support] +@item -object colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid},outdev=@var{chardevid},iothread=@var{id}[,vnet_hdr_support][,notify_dev=@var{id}] Colo-compare gets packet from primary_in@var{chardevid} and secondary_in@var{chardevid}, than compare primary packet with secondary packet. If the packets are same, we will output primary @@ -4486,11 +4486,15 @@ do checkpoint and send primary packet to outdev@var{chardevid}. In order to improve efficiency, we need to put the task of comparison in another thread. If it has the vnet_hdr_support flag, colo compare will send/recv packet with vnet_hdr_len. +If you want to use Xen COLO, will need the notify_dev to notify Xen +colo-frame to do checkpoint. we must use it with the help of filter-mirror and filter-redirector. @example +KVM COLO + primary: -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 @@ -4514,6 +4518,33 @@ secondary: -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 + +Xen COLO + +primary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 +-chardev socket,id=notify_way,host=3.3.3.3,port=9009,server,nowait +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 +-object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out +-object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 +-object iothread,id=iothread1 +-object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0,notify_dev=nofity_way,iothread=iothread1 + +secondary: +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down script=/etc/qemu-ifdown +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 +-chardev socket,id=red0,host=3.3.3.3,port=9003 +-chardev socket,id=red1,host=3.3.3.3,port=9004 +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 + @end example If you want to know the detail of above command line, you can read From patchwork Sun Jun 9 16:44:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 10983755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E34CD13AD for ; Sun, 9 Jun 2019 16:55:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5EFB28565 for ; Sun, 9 Jun 2019 16:55:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A37B42852A; Sun, 9 Jun 2019 16:55:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=no version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1BA6D2852A for ; Sun, 9 Jun 2019 16:55:44 +0000 (UTC) Received: from localhost ([::1]:37078 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha16a-0005hM-0H for patchwork-qemu-devel@patchwork.kernel.org; Sun, 09 Jun 2019 12:55:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56547) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha10T-0000QO-UM for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ha10S-0007HL-II for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:21 -0400 Received: from mga14.intel.com ([192.55.52.115]:62034) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ha10S-0007DM-AC for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:20 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jun 2019 09:49:13 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 09 Jun 2019 09:49:11 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev , Stefano Stabellini , Paul Durrant Date: Mon, 10 Jun 2019 00:44:30 +0800 Message-Id: <20190609164433.5866-3-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190609164433.5866-1-chen.zhang@intel.com> References: <20190609164433.5866-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH V2 2/5] COLO-compare: Add remote notification chardev handler frame X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen Add chardev handler to send notification to remote(current from Xen) colo-frame. Signed-off-by: Zhang Chen --- net/colo-compare.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/net/colo-compare.c b/net/colo-compare.c index 21849748b6..8b7bd9f0c7 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -87,8 +87,10 @@ typedef struct CompareState { CharBackend chr_pri_in; CharBackend chr_sec_in; CharBackend chr_out; + CharBackend chr_notify_dev; SocketReadState pri_rs; SocketReadState sec_rs; + SocketReadState notify_rs; bool vnet_hdr; /* @@ -745,6 +747,19 @@ static void compare_sec_chr_in(void *opaque, const uint8_t *buf, int size) } } +static void compare_notify_chr(void *opaque, const uint8_t *buf, int size) +{ + CompareState *s = COLO_COMPARE(opaque); + int ret; + + ret = net_fill_rstate(&s->notify_rs, buf, size); + if (ret == -1) { + qemu_chr_fe_set_handlers(&s->chr_notify_dev, NULL, NULL, NULL, NULL, + NULL, NULL, true); + error_report("colo-compare notify_dev error"); + } +} + /* * Check old packet regularly so it can watch for any packets * that the secondary hasn't produced equivalents of. @@ -832,6 +847,11 @@ static void colo_compare_iothread(CompareState *s) qemu_chr_fe_set_handlers(&s->chr_sec_in, compare_chr_can_read, compare_sec_chr_in, NULL, NULL, s, s->worker_context, true); + if (s->notify_dev) { + qemu_chr_fe_set_handlers(&s->chr_notify_dev, compare_chr_can_read, + compare_notify_chr, NULL, NULL, + s, s->worker_context, true); + } colo_compare_timer_init(s); s->event_bh = qemu_bh_new(colo_compare_handle_event, s); @@ -943,6 +963,10 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs) } } +static void compare_notify_rs_finalize(SocketReadState *notify_rs) +{ + /* Get Xen colo-frame's notify and handle the message */ +} /* * Return 0 is success. @@ -1013,6 +1037,17 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp) net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, s->vnet_hdr); net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, s->vnet_hdr); + /* Try to enable remote notify chardev, currently just for Xen COLO */ + if (s->notify_dev) { + if (find_and_check_chardev(&chr, s->notify_dev, errp) || + !qemu_chr_fe_init(&s->chr_notify_dev, chr, errp)) { + return; + } + + net_socket_rs_init(&s->notify_rs, compare_notify_rs_finalize, + s->vnet_hdr); + } + QTAILQ_INSERT_TAIL(&net_compares, s, next); g_queue_init(&s->conn_list); @@ -1091,6 +1126,10 @@ static void colo_compare_finalize(Object *obj) qemu_chr_fe_deinit(&s->chr_pri_in, false); qemu_chr_fe_deinit(&s->chr_sec_in, false); qemu_chr_fe_deinit(&s->chr_out, false); + if (s->notify_dev) { + qemu_chr_fe_deinit(&s->chr_notify_dev, false); + } + if (s->iothread) { colo_compare_timer_del(s); } From patchwork Sun Jun 9 16:44:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 10983751 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BDF276 for ; Sun, 9 Jun 2019 16:50:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49CD21FFFF for ; Sun, 9 Jun 2019 16:50:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3C42F204FA; Sun, 9 Jun 2019 16:50:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=no version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DE6571FFFF for ; Sun, 9 Jun 2019 16:50:43 +0000 (UTC) Received: from localhost ([::1]:37062 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha11n-0002RE-8e for patchwork-qemu-devel@patchwork.kernel.org; Sun, 09 Jun 2019 12:50:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56519) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha10S-0000QG-Bl for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ha10R-0007GO-7j for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:20 -0400 Received: from mga14.intel.com ([192.55.52.115]:62029) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ha10Q-0007BK-U0 for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:19 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jun 2019 09:49:16 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 09 Jun 2019 09:49:14 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev , Stefano Stabellini , Paul Durrant Date: Mon, 10 Jun 2019 00:44:31 +0800 Message-Id: <20190609164433.5866-4-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190609164433.5866-1-chen.zhang@intel.com> References: <20190609164433.5866-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH V2 3/5] COLO-compare: Make the compare_chr_send() can send notification message. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen We need use this function to send notification message for remote colo-frame(Xen). So we add new parameter for this job. Signed-off-by: Zhang Chen --- net/colo-compare.c | 41 +++++++++++++++++++++++++++++++++-------- 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 8b7bd9f0c7..16285f4a96 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -129,7 +129,8 @@ static void colo_compare_inconsistency_notify(void) static int compare_chr_send(CompareState *s, const uint8_t *buf, uint32_t size, - uint32_t vnet_hdr_len); + uint32_t vnet_hdr_len, + bool notify_remote_frame); static gint seq_sorter(Packet *a, Packet *b, gpointer data) { @@ -241,7 +242,8 @@ static void colo_release_primary_pkt(CompareState *s, Packet *pkt) ret = compare_chr_send(s, pkt->data, pkt->size, - pkt->vnet_hdr_len); + pkt->vnet_hdr_len, + false); if (ret < 0) { error_report("colo send primary packet failed"); } @@ -671,7 +673,8 @@ static void colo_compare_connection(void *opaque, void *user_data) static int compare_chr_send(CompareState *s, const uint8_t *buf, uint32_t size, - uint32_t vnet_hdr_len) + uint32_t vnet_hdr_len, + bool notify_remote_frame) { int ret = 0; uint32_t len = htonl(size); @@ -680,7 +683,14 @@ static int compare_chr_send(CompareState *s, return 0; } - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len)); + if (notify_remote_frame) { + ret = qemu_chr_fe_write_all(&s->chr_notify_dev, + (uint8_t *)&len, + sizeof(len)); + } else { + ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len)); + } + if (ret != sizeof(len)) { goto err; } @@ -691,13 +701,26 @@ static int compare_chr_send(CompareState *s, * know how to parse net packet correctly. */ len = htonl(vnet_hdr_len); - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)&len, sizeof(len)); + + if (!notify_remote_frame) { + ret = qemu_chr_fe_write_all(&s->chr_out, + (uint8_t *)&len, + sizeof(len)); + } + if (ret != sizeof(len)) { goto err; } } - ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size); + if (notify_remote_frame) { + ret = qemu_chr_fe_write_all(&s->chr_notify_dev, + (uint8_t *)buf, + size); + } else { + ret = qemu_chr_fe_write_all(&s->chr_out, (uint8_t *)buf, size); + } + if (ret != size) { goto err; } @@ -943,7 +966,8 @@ static void compare_pri_rs_finalize(SocketReadState *pri_rs) compare_chr_send(s, pri_rs->buf, pri_rs->packet_len, - pri_rs->vnet_hdr_len); + pri_rs->vnet_hdr_len, + false); } else { /* compare packet in the specified connection */ colo_compare_connection(conn, s); @@ -1075,7 +1099,8 @@ static void colo_flush_packets(void *opaque, void *user_data) compare_chr_send(s, pkt->data, pkt->size, - pkt->vnet_hdr_len); + pkt->vnet_hdr_len, + false); packet_destroy(pkt, NULL); } while (!g_queue_is_empty(&conn->secondary_list)) { From patchwork Sun Jun 9 16:44:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 10983759 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A4E6213AD for ; Sun, 9 Jun 2019 16:55:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9283D2852A for ; Sun, 9 Jun 2019 16:55:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86C052889C; Sun, 9 Jun 2019 16:55:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=no version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3117A2852A for ; Sun, 9 Jun 2019 16:55:47 +0000 (UTC) Received: from localhost ([::1]:37082 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha16g-0005xa-IC for patchwork-qemu-devel@patchwork.kernel.org; Sun, 09 Jun 2019 12:55:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56524) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha10S-0000QM-LF for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ha10R-0007GT-8I for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:20 -0400 Received: from mga14.intel.com ([192.55.52.115]:62032) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ha10Q-0007CA-W0 for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:19 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jun 2019 09:49:18 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 09 Jun 2019 09:49:16 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev , Stefano Stabellini , Paul Durrant Date: Mon, 10 Jun 2019 00:44:32 +0800 Message-Id: <20190609164433.5866-5-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190609164433.5866-1-chen.zhang@intel.com> References: <20190609164433.5866-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH V2 4/5] COLO-compare: Add colo-compare remote notify support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen This patch make colo-compare can send message to remote COLO frame(Xen) when occur checkpoint. Signed-off-by: Zhang Chen --- net/colo-compare.c | 54 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 44 insertions(+), 10 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 16285f4a96..516b651ecd 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -120,11 +120,6 @@ enum { SECONDARY_IN, }; -static void colo_compare_inconsistency_notify(void) -{ - notifier_list_notify(&colo_compare_notifiers, - migrate_get_current()); -} static int compare_chr_send(CompareState *s, const uint8_t *buf, @@ -132,6 +127,27 @@ static int compare_chr_send(CompareState *s, uint32_t vnet_hdr_len, bool notify_remote_frame); +static void notify_remote_frame(CompareState *s) +{ + char msg[] = "DO_CHECKPOINT"; + int ret = 0; + + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true); + if (ret < 0) { + error_report("Notify Xen COLO-frame failed"); + } +} + +static void colo_compare_inconsistency_notify(CompareState *s) +{ + if (s->notify_dev) { + notify_remote_frame(s); + } else { + notifier_list_notify(&colo_compare_notifiers, + migrate_get_current()); + } +} + static gint seq_sorter(Packet *a, Packet *b, gpointer data) { struct tcp_hdr *atcp, *btcp; @@ -435,7 +451,7 @@ sec: qemu_hexdump((char *)spkt->data, stderr, "colo-compare spkt", spkt->size); - colo_compare_inconsistency_notify(); + colo_compare_inconsistency_notify(s); } } @@ -577,7 +593,7 @@ void colo_compare_unregister_notifier(Notifier *notify) } static int colo_old_packet_check_one_conn(Connection *conn, - void *user_data) + CompareState *s) { GList *result = NULL; int64_t check_time = REGULAR_PACKET_CHECK_MS; @@ -588,7 +604,7 @@ static int colo_old_packet_check_one_conn(Connection *conn, if (result) { /* Do checkpoint will flush old packet */ - colo_compare_inconsistency_notify(); + colo_compare_inconsistency_notify(s); return 0; } @@ -608,7 +624,7 @@ static void colo_old_packet_check(void *opaque) * If we find one old packet, stop finding job and notify * COLO frame do checkpoint. */ - g_queue_find_custom(&s->conn_list, NULL, + g_queue_find_custom(&s->conn_list, s, (GCompareFunc)colo_old_packet_check_one_conn); } @@ -637,7 +653,8 @@ static void colo_compare_packet(CompareState *s, Connection *conn, */ trace_colo_compare_main("packet different"); g_queue_push_head(&conn->primary_list, pkt); - colo_compare_inconsistency_notify(); + + colo_compare_inconsistency_notify(s); break; } } @@ -989,7 +1006,24 @@ static void compare_sec_rs_finalize(SocketReadState *sec_rs) static void compare_notify_rs_finalize(SocketReadState *notify_rs) { + CompareState *s = container_of(notify_rs, CompareState, notify_rs); + /* Get Xen colo-frame's notify and handle the message */ + char *data = g_memdup(notify_rs->buf, notify_rs->packet_len); + char msg[] = "COLO_COMPARE_GET_XEN_INIT"; + int ret; + + if (!strcmp(data, "COLO_USERSPACE_PROXY_INIT")) { + ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true); + if (ret < 0) { + error_report("Notify Xen COLO-frame INIT failed"); + } + } + + if (!strcmp(data, "COLO_CHECKPOINT")) { + /* colo-compare do checkpoint, flush pri packet and remove sec packet */ + g_queue_foreach(&s->conn_list, colo_flush_packets, s); + } } /* From patchwork Sun Jun 9 16:44:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 10983757 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 80B1E924 for ; Sun, 9 Jun 2019 16:55:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D24C2852A for ; Sun, 9 Jun 2019 16:55:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 615C128657; Sun, 9 Jun 2019 16:55:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=2.0 tests=BAYES_00,FROM_ADDR_WS, FROM_WSP_TRAIL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=no version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 23B4B2852A for ; Sun, 9 Jun 2019 16:55:46 +0000 (UTC) Received: from localhost ([::1]:37080 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha16f-0005vd-Fo for patchwork-qemu-devel@patchwork.kernel.org; Sun, 09 Jun 2019 12:55:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56556) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha10U-0000Qh-Ot for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ha10T-0007I0-RQ for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:22 -0400 Received: from mga14.intel.com ([192.55.52.115]:62034) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ha10T-0007DM-8N for qemu-devel@nongnu.org; Sun, 09 Jun 2019 12:49:21 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Jun 2019 09:49:20 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.239.13.19]) by orsmga005.jf.intel.com with ESMTP; 09 Jun 2019 09:49:18 -0700 From: Zhang Chen To: Li Zhijian , "Dr. David Alan Gilbert" , Juan Quintela , zhanghailiang , Jason Wang , qemu-dev , Stefano Stabellini , Paul Durrant Date: Mon, 10 Jun 2019 00:44:33 +0800 Message-Id: <20190609164433.5866-6-chen.zhang@intel.com> X-Mailer: git-send-email 2.17.GIT In-Reply-To: <20190609164433.5866-1-chen.zhang@intel.com> References: <20190609164433.5866-1-chen.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH V2 5/5] migration/colo.c: Add missed filter notify for Xen COLO. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , Zhang Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Zhang Chen We need to notify net filter to do checkpoint for Xen COLO, like KVM side. Signed-off-by: Zhang Chen --- migration/colo.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 8c1644091f..9f84b1fa3c 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -259,6 +259,8 @@ ReplicationStatus *qmp_query_xen_replication_status(Error **errp) void qmp_xen_colo_do_checkpoint(Error **errp) { replication_do_checkpoint_all(errp); + /* Notify all filters of all NIC to do checkpoint */ + colo_notify_filters_event(COLO_EVENT_CHECKPOINT, errp); } #endif