From patchwork Tue Jun 5 15:28:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: 858585 jemmy X-Patchwork-Id: 10448515 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5B05660284 for ; Tue, 5 Jun 2018 15:39:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E93C22B39 for ; Tue, 5 Jun 2018 15:39:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 435192976B; Tue, 5 Jun 2018 15:39:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 989E1298C5 for ; Tue, 5 Jun 2018 15:38:00 +0000 (UTC) Received: from localhost ([::1]:47571 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQE23-0007mp-KA for patchwork-qemu-devel@patchwork.kernel.org; Tue, 05 Jun 2018 11:37:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40223) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQDt7-0008My-27 for qemu-devel@nongnu.org; Tue, 05 Jun 2018 11:28:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQDt5-0003zu-Sc for qemu-devel@nongnu.org; Tue, 05 Jun 2018 11:28:45 -0400 Received: from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]:34678) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fQDt5-0003zN-JP for qemu-devel@nongnu.org; Tue, 05 Jun 2018 11:28:43 -0400 Received: by mail-pf0-x241.google.com with SMTP id a63-v6so1492802pfl.1 for ; Tue, 05 Jun 2018 08:28:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BU/AaHONtx2y8sW4Yxt/LJSbPRVczbwYHznqDXS1FTI=; b=GYzvj2pgTFgLsZbjw7vrs4Tlb+LVieuBET18YnIJOWUxDZbPr2t0YJArpQ0QlJ8TAx +w0B4zCSgvLiuPgux3F1j8LVLJMyuecKp+UoKq5V7ZlwbTb8z/PV8bSdVxXtT/LycvhJ TmciPnOHZdMGYh+sfxPCg5tvpes5RvXgXXOeZXfgrQdy08hf28UZK9f/+E202q4lskgA inKPqqzNWc/14SXBrg4GGt27lJrlpl+v7aU9NnGOLWxjrzQFwmWXEmmnasIvDUnmxUV4 p676R9vvFhZlbIuAeJP5DJtZ2uHvsgYyLR2NWYCRU8xUOjbkgDShzStuOoehGZVqJg7r u5Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BU/AaHONtx2y8sW4Yxt/LJSbPRVczbwYHznqDXS1FTI=; b=BLvrU1C3Xc7Po9WT8NWfZWaER5iUXR3QHgJ9tLmZzhxTLj2iK43IsUY59I1I84MhvI Y9GOJ8k0Z/wjK28GbZV+0igqZ8XFKYN3pZi3fo5grF1mVSkvnePDCH7Ke46BtC8DJdRn RjqAKx27SMQNohPiLZ6GHGcCR1Sh9fZbWnQBm6OpJOLfm1LVUQGZ2NokjHxLQA+1F2XV G0B3g2yHM4LE67+KvBjCmJuhr7Nsfgx5J+d1FFR0mgjjAHr3iDQBoc++knlY3+UfKgHx S2ySRPIEBjY0D3WJFZYXKJXqNDJbwfup+xC3WOFsAUGo683gTlIiFtsQJbYSV5YkPvF4 OOig== X-Gm-Message-State: ALKqPweCp1b/vODUfAsuJ+RgM/4svW3NKk3G3u/++HuofSqyDi394UV+ ubzb7C4521ejBJPkubBkqPE= X-Google-Smtp-Source: ADUXVKLMr3VxpCo2MwdorsnCGHgaMlm7iZ079xF+5C3PO705rHVQnji1YLFhvVQFVso/Bah8c9JbGQ== X-Received: by 2002:a63:3759:: with SMTP id g25-v6mr21789763pgn.59.1528212522778; Tue, 05 Jun 2018 08:28:42 -0700 (PDT) Received: from VM_93_245_centos.localdomain ([150.109.57.149]) by smtp.gmail.com with ESMTPSA id t14-v6sm101952007pfh.109.2018.06.05.08.28.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Jun 2018 08:28:42 -0700 (PDT) From: Lidong Chen X-Google-Original-From: Lidong Chen To: zhang.zhanghailiang@huawei.com, quintela@redhat.com, dgilbert@redhat.com, berrange@redhat.com, aviadye@mellanox.com, pbonzini@redhat.com Date: Tue, 5 Jun 2018 23:28:09 +0800 Message-Id: <1528212489-19137-11-git-send-email-lidongchen@tencent.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1528212489-19137-1-git-send-email-lidongchen@tencent.com> References: <1528212489-19137-1-git-send-email-lidongchen@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::241 Subject: [Qemu-devel] [PATCH v5 10/10] migration: implement the shutdown for RDMA QIOChannel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: galsha@mellanox.com, adido@mellanox.com, qemu-devel@nongnu.org, Lidong Chen Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Because RDMA QIOChannel not implement shutdown function, If the to_dst_file was set error, the return path thread will wait forever. and the migration thread will wait return path thread exit. the backtrace of return path thread is: (gdb) bt #0 0x00007f372a76bb0f in ppoll () from /lib64/libc.so.6 #1 0x000000000071dc24 in qemu_poll_ns (fds=0x7ef7091d0580, nfds=2, timeout=100000000) at qemu-timer.c:325 #2 0x00000000006b2fba in qemu_rdma_wait_comp_channel (rdma=0xd424000) at migration/rdma.c:1501 #3 0x00000000006b3191 in qemu_rdma_block_for_wrid (rdma=0xd424000, wrid_requested=4000, byte_len=0x7ef7091d0640) at migration/rdma.c:1580 #4 0x00000000006b3638 in qemu_rdma_exchange_get_response (rdma=0xd424000, head=0x7ef7091d0720, expecting=3, idx=0) at migration/rdma.c:1726 #5 0x00000000006b3ad6 in qemu_rdma_exchange_recv (rdma=0xd424000, head=0x7ef7091d0720, expecting=3) at migration/rdma.c:1903 #6 0x00000000006b5d03 in qemu_rdma_get_buffer (opaque=0x6a57dc0, buf=0x5c80030 "", pos=8, size=32768) at migration/rdma.c:2714 #7 0x00000000006a9635 in qemu_fill_buffer (f=0x5c80000) at migration/qemu-file.c:232 #8 0x00000000006a9ecd in qemu_peek_byte (f=0x5c80000, offset=0) at migration/qemu-file.c:502 #9 0x00000000006a9f1f in qemu_get_byte (f=0x5c80000) at migration/qemu-file.c:515 #10 0x00000000006aa162 in qemu_get_be16 (f=0x5c80000) at migration/qemu-file.c:591 #11 0x00000000006a46d3 in source_return_path_thread ( opaque=0xd826a0 ) at migration/migration.c:1331 #12 0x00007f372aa49e25 in start_thread () from /lib64/libpthread.so.0 #13 0x00007f372a77635d in clone () from /lib64/libc.so.6 the backtrace of migration thread is: (gdb) bt #0 0x00007f372aa4af57 in pthread_join () from /lib64/libpthread.so.0 #1 0x00000000007d5711 in qemu_thread_join (thread=0xd826f8 ) at util/qemu-thread-posix.c:504 #2 0x00000000006a4bc5 in await_return_path_close_on_source ( ms=0xd826a0 ) at migration/migration.c:1460 #3 0x00000000006a53e4 in migration_completion (s=0xd826a0 , current_active_state=4, old_vm_running=0x7ef7089cf976, start_time=0x7ef7089cf980) at migration/migration.c:1695 #4 0x00000000006a5c54 in migration_thread (opaque=0xd826a0 ) at migration/migration.c:1837 #5 0x00007f372aa49e25 in start_thread () from /lib64/libpthread.so.0 #6 0x00007f372a77635d in clone () from /lib64/libc.so.6 Signed-off-by: Lidong Chen Reviewed-by: Dr. David Alan Gilbert --- migration/rdma.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/migration/rdma.c b/migration/rdma.c index bb6989e..0b35d65 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3050,6 +3050,45 @@ static int qio_channel_rdma_close(QIOChannel *ioc, return 0; } +static int +qio_channel_rdma_shutdown(QIOChannel *ioc, + QIOChannelShutdown how, + Error **errp) +{ + QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc); + RDMAContext *rdmain, *rdmaout; + + rcu_read_lock(); + + rdmain = atomic_rcu_read(&rioc->rdmain); + rdmaout = atomic_rcu_read(&rioc->rdmain); + + switch (how) { + case QIO_CHANNEL_SHUTDOWN_READ: + if (rdmain) { + rdmain->error_state = -1; + } + break; + case QIO_CHANNEL_SHUTDOWN_WRITE: + if (rdmaout) { + rdmaout->error_state = -1; + } + break; + case QIO_CHANNEL_SHUTDOWN_BOTH: + default: + if (rdmain) { + rdmain->error_state = -1; + } + if (rdmaout) { + rdmaout->error_state = -1; + } + break; + } + + rcu_read_unlock(); + return 0; +} + /* * Parameters: * @offset == 0 : @@ -3876,6 +3915,7 @@ static void qio_channel_rdma_class_init(ObjectClass *klass, ioc_klass->io_close = qio_channel_rdma_close; ioc_klass->io_create_watch = qio_channel_rdma_create_watch; ioc_klass->io_set_aio_fd_handler = qio_channel_rdma_set_aio_fd_handler; + ioc_klass->io_shutdown = qio_channel_rdma_shutdown; } static const TypeInfo qio_channel_rdma_info = {