From patchwork Thu Feb 20 15:10:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oksana Vohchana X-Patchwork-Id: 11394293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9466F924 for ; Thu, 20 Feb 2020 15:12:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B846208CD for ; Thu, 20 Feb 2020 15:12:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dNAOU9kN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B846208CD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:44078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j4nVZ-000609-4N for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Feb 2020 10:12:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46605) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j4nTc-0002jD-0Z for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j4nTa-0006qA-PR for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:55 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:41957 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j4nTa-0006nV-La for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582211453; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yMcsuqrYa1e/c6WIxi9XC5EJlGhyeHPg8JOTNTCllQ8=; b=dNAOU9kNtnXOIG6zeHm63Xv8XC3s6N1Jt+EvBReH3JeK5qyKF89Eb8HI0XWVXG7E921iyI c5e77c0wI5ft951TulYVQYwH9YQEkAaMpBKjZ7Blj4tijuB/0BfS28OtfjmUNQ/Ipi0BEt +ioGXqS8bsSWm5g5ZKE8nwTrppcaZc4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-97-ns8n_V4hOvmoE9yMw6s2hA-1; Thu, 20 Feb 2020 10:10:48 -0500 X-MC-Unique: ns8n_V4hOvmoE9yMw6s2hA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 08D8B1937FC9 for ; Thu, 20 Feb 2020 15:10:47 +0000 (UTC) Received: from kh066.redhat.com (ovpn-204-205.brq.redhat.com [10.40.204.205]) by smtp.corp.redhat.com (Postfix) with ESMTP id 602C38ED1E; Thu, 20 Feb 2020 15:10:45 +0000 (UTC) From: Oksana Vohchana To: qemu-devel@nongnu.org Subject: [PATCH v2 1/3] Adding functions _send_fds and _recv_fds Date: Thu, 20 Feb 2020 17:10:37 +0200 Message-Id: <20200220151039.20552-2-ovoshcha@redhat.com> In-Reply-To: <20200220151039.20552-1-ovoshcha@redhat.com> References: <20200220151039.20552-1-ovoshcha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ovoshcha@redhat.com, philmd@redhat.com, ehabkost@redhat.com, wainersm@redhat.com, crosa@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" It provides new possibilities to send or receive data through the Unix domain socket file descriptor. This is useful for obtaining a socket that belongs to a different network namespace. Signed-off-by: Oksana Vohchana --- python/qemu/machine.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 183d8f3d38..976316e5f5 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -24,6 +24,7 @@ import subprocess import shutil import socket import tempfile +import array from . import qmp @@ -155,6 +156,29 @@ class QEMUMachine(object): self._args.append(','.join(options)) return self + def _recv_fds(self, sock, msglen=8192, maxfds=4096): + """ + Function from: + https://docs.python.org/3/library/socket.html#socket.socket.recvmsg + """ + fds = array.array("i") + msg, ancdata, flags, addr = sock.recvmsg(msglen, socket.CMSG_LEN( + maxfds * fds.itemsize)) + for cmsg_level, cmsg_type, cmsg_data in ancdata: + if (cmsg_level == socket.SOL_SOCKET and + cmsg_type == socket.SCM_RIGHTS): + fds.frombytes(cmsg_data[:len(cmsg_data) - (len(cmsg_data) + % fds.itemsize)]) + return msg, list(fds) + + def _send_fds(self, sock, msg, fds): + """ + Function from: + https://docs.python.org/3/library/socket.html#socket.socket.sendmsg + """ + return sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, + array.array("i", fds))]) + def send_fd_scm(self, fd=None, file_path=None): """ Send an fd or file_path to socket_scm_helper. From patchwork Thu Feb 20 15:10:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oksana Vohchana X-Patchwork-Id: 11394295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88481109A for ; Thu, 20 Feb 2020 15:13:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5FD9F208CD for ; Thu, 20 Feb 2020 15:13:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="S5SVQy/3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FD9F208CD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:44080 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j4nVd-00067d-Hf for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Feb 2020 10:13:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46626) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j4nTd-0002m8-RQ for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j4nTb-0006sz-LC for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:57 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:28030 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j4nTb-0006ri-HC for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582211455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=heDlndT4Vn36W18ursSUZuZUPiNVz/9nzMrCO7RbA5E=; b=S5SVQy/3N83sN1KFtt3C5Qo5ZHXKPLmi94ozI6iTOAIem/rSFPMUqni7+czYTheBWQaOEJ eX6D6ys1yIpdlXzhtcfapVkmK3cDRrfqMrVjQqGOkpaeBLSQaLSscdFaHiCd+TWdP64HxY YuzPKNMgPtN/+HvTJOWDXk90cmrnuGo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-337-vSbbs7lWMvWHTW8IDydEdQ-1; Thu, 20 Feb 2020 10:10:50 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 104B58017CC for ; Thu, 20 Feb 2020 15:10:49 +0000 (UTC) Received: from kh066.redhat.com (ovpn-204-205.brq.redhat.com [10.40.204.205]) by smtp.corp.redhat.com (Postfix) with ESMTP id 67BB18ECFC; Thu, 20 Feb 2020 15:10:47 +0000 (UTC) From: Oksana Vohchana To: qemu-devel@nongnu.org Subject: [PATCH v2 2/3] Updates send_fd_scm function Date: Thu, 20 Feb 2020 17:10:38 +0200 Message-Id: <20200220151039.20552-3-ovoshcha@redhat.com> In-Reply-To: <20200220151039.20552-1-ovoshcha@redhat.com> References: <20200220151039.20552-1-ovoshcha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: vSbbs7lWMvWHTW8IDydEdQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ovoshcha@redhat.com, philmd@redhat.com, ehabkost@redhat.com, wainersm@redhat.com, crosa@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" A qemu-iotest uses for FD-migration test a helper program "socket_scm_helper". And it makes some problems if you didn't build it with a QEMU. And now we can use new methods for the socket that allow us to send a file/socket descriptor (with access and permissions) from one process to another. Signed-off-by: Oksana Vohchana --- python/qemu/machine.py | 64 ++++++++++++++++++++++++++---------------- 1 file changed, 40 insertions(+), 24 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 976316e5f5..906ca118db 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -179,20 +179,27 @@ class QEMUMachine(object): return sock.sendmsg([msg], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, array.array("i", fds))]) - def send_fd_scm(self, fd=None, file_path=None): + def send_fd_scm(self, fd=None, file_path=None, data=None): """ - Send an fd or file_path to socket_scm_helper. + Can be used in two different cases. + Send an fd or file_path to socket_scm_helper or + provide data and fd to send it to the socket. - Exactly one of fd and file_path must be given. - If it is file_path, the helper will open that file and pass its own fd. + Exactly one of fd and file_path must be given to the case of + socket_scm_helper. If it is file_path, the helper will open that file + and pass its own fd. + + To second case need adds data that include a QMP request and fd """ # In iotest.py, the qmp should always use unix socket. assert self._qmp.is_scm_available() - if self._socket_scm_helper is None: - raise QEMUMachineError("No path to socket_scm_helper set") - if not os.path.exists(self._socket_scm_helper): - raise QEMUMachineError("%s does not exist" % - self._socket_scm_helper) + if data is None: + if self._socket_scm_helper is None: + raise QEMUMachineError( + "No path to socket_scm_helper set or data provided") + if not os.path.exists(self._socket_scm_helper): + raise QEMUMachineError("%s does not exist" % + self._socket_scm_helper) # This did not exist before 3.4, but since then it is # mandatory for our purpose @@ -201,24 +208,33 @@ class QEMUMachine(object): if fd is not None: os.set_inheritable(fd, True) - fd_param = ["%s" % self._socket_scm_helper, - "%d" % self._qmp.get_sock_fd()] + if data is None: + fd_param = ["%s" % self._socket_scm_helper, + "%d" % self._qmp.get_sock_fd()] + if file_path is not None: + assert fd is None + fd_param.append(file_path) + else: + assert fd is not None + fd_param.append(str(fd)) - if file_path is not None: - assert fd is None - fd_param.append(file_path) - else: - assert fd is not None - fd_param.append(str(fd)) + devnull = open(os.path.devnull, 'rb') + proc = subprocess.Popen(fd_param, stdin=devnull, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, close_fds=False) + output = proc.communicate()[0] + if output: + LOG.debug(output) - devnull = open(os.path.devnull, 'rb') - proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, close_fds=False) - output = proc.communicate()[0] - if output: - LOG.debug(output) + return proc.returncode - return proc.returncode + else: + sock_fd = socket.fromfd(self._qmp.get_sock_fd(), socket.AF_UNIX, + socket.SOCK_STREAM) + fds_param = [fd, self._qmp.get_sock_fd()] + self._send_fds(sock_fd, data, fds_param) + self._recv_fds(sock_fd) + return self @staticmethod def _remove_if_exists(path): From patchwork Thu Feb 20 15:10:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oksana Vohchana X-Patchwork-Id: 11394291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F1C3109A for ; Thu, 20 Feb 2020 15:11:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16C14208CD for ; Thu, 20 Feb 2020 15:11:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KnYgvZDT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16C14208CD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:44062 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j4nUY-0004Be-AZ for patchwork-qemu-devel@patchwork.kernel.org; Thu, 20 Feb 2020 10:11:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46622) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j4nTd-0002l8-Bk for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j4nTc-0006ut-8K for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:57 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:33548 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j4nTc-0006tJ-4U for qemu-devel@nongnu.org; Thu, 20 Feb 2020 10:10:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582211455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mk7mxObj+F1rSmq9Vu2WnQwTKR+cBHHEHHwDVqoXI+8=; b=KnYgvZDTdLDJvTG1FvXcDIiEoDziEYkvTHnWIWoZy5hn/UPqGgpVv4H5g/D9nAZSUg3Ci/ D6OoxH04JQ+0De/7kGxpokl/jVFClj9ZWuBbMNXTs73KIEkzwK4rmY1eS6yzLIrJJzxoTD CaJQpf4C49akpS/1C5MXkskcJ+4iWsc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-239-zkZelA5ZMVe5PEtslRxB_w-1; Thu, 20 Feb 2020 10:10:52 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 23945800D4E for ; Thu, 20 Feb 2020 15:10:51 +0000 (UTC) Received: from kh066.redhat.com (ovpn-204-205.brq.redhat.com [10.40.204.205]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6E7B08ECFC; Thu, 20 Feb 2020 15:10:49 +0000 (UTC) From: Oksana Vohchana To: qemu-devel@nongnu.org Subject: [PATCH v2 3/3] Acceptance test: FD migration Date: Thu, 20 Feb 2020 17:10:39 +0200 Message-Id: <20200220151039.20552-4-ovoshcha@redhat.com> In-Reply-To: <20200220151039.20552-1-ovoshcha@redhat.com> References: <20200220151039.20552-1-ovoshcha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: zkZelA5ZMVe5PEtslRxB_w-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ovoshcha@redhat.com, philmd@redhat.com, ehabkost@redhat.com, wainersm@redhat.com, crosa@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Adds a new migration test through the file descriptor. Signed-off-by: Oksana Vohchana --- tests/acceptance/migration.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tests/acceptance/migration.py b/tests/acceptance/migration.py index a8367ca023..7f4879ce5d 100644 --- a/tests/acceptance/migration.py +++ b/tests/acceptance/migration.py @@ -10,7 +10,10 @@ # later. See the COPYING file in the top-level directory. +import os import tempfile +from socket import socketpair, AF_UNIX, SOCK_STREAM + from avocado_qemu import Test from avocado import skipUnless @@ -75,3 +78,21 @@ class Migration(Test): """ free_port = self._get_free_port() dest_uri = 'exec:nc -l localhost %u' % free_port + + def test_migration_with_fd(self): + opaque = 'fd-migration' + data_to_send = b"{\"execute\": \"getfd\", \"arguments\": \ + {\"fdname\": \"fd-migration\"}}" + send_socket, recv_socket = socketpair(AF_UNIX, SOCK_STREAM) + fd1 = send_socket.fileno() + fd2 = recv_socket.fileno() + os.set_inheritable(fd2, True) + + source_vm = self.get_vm() + source_vm.launch() + source_vm.send_fd_scm(fd=fd1, data=data_to_send) + + dest_vm = self.get_vm('-incoming', 'fd:%s' % fd2) + dest_vm.launch() + source_vm.qmp('migrate', uri='fd:%s' % opaque) + self.assert_migration(source_vm, dest_vm)