From patchwork Thu Oct 6 08:12:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= X-Patchwork-Id: 13000037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39C93C433F5 for ; Thu, 6 Oct 2022 08:19:29 +0000 (UTC) Received: from localhost ([::1]:54746 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ogM6K-0003Jl-8f for qemu-devel@archiver.kernel.org; Thu, 06 Oct 2022 04:19:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55608) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzg-0006LX-QT for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56344) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLze-0002oW-QY for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665043952; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XExzAlB9hioR6/71riasaaqxWc+GgRvUIex+5d/LKR4=; b=PdaZLum5c7bDLfVF7Zj33OQ9nlKkEkJdV8MWXrv2QxElN1lSXjKyfxknn4Mzv0310JChSf qKmNeQJRDYU3wGe4VxCluE74LfwP4KpNjWpOB/Bmfu6Ci1liN2lRDcaogQ9boi2MzhYnGt lZ8oQvhSI0vtIAF+BV8OmNHOCqVnlIw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-644-_5QyKeMgMNCNc26MZIurCw-1; Thu, 06 Oct 2022 04:12:31 -0400 X-MC-Unique: _5QyKeMgMNCNc26MZIurCw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E5F7E185A794 for ; Thu, 6 Oct 2022 08:12:30 +0000 (UTC) Received: from localhost (unknown [10.39.208.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40EBD40C6F9F; Thu, 6 Oct 2022 08:12:29 +0000 (UTC) From: marcandre.lureau@redhat.com To: qemu-devel@nongnu.org Cc: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?M?= =?utf-8?q?arc-Andr=C3=A9_Lureau?= Subject: [PATCH v3 1/5] osdep: make readv_writev() work with partial read/write Date: Thu, 6 Oct 2022 12:12:18 +0400 Message-Id: <20221006081222.2606746-2-marcandre.lureau@redhat.com> In-Reply-To: <20221006081222.2606746-1-marcandre.lureau@redhat.com> References: <20221006081222.2606746-1-marcandre.lureau@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=marcandre.lureau@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Marc-André Lureau With a pipe or other reasons, read/write may return less than the requested bytes. This happens with the test-io-channel-command test on Windows. glib spawn code uses a binary pipe of 4096 bytes, and the first read returns that much (although more are requested), for some unclear reason... Signed-off-by: Marc-André Lureau Reviewed-by: Daniel P. Berrangé --- util/osdep.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/util/osdep.c b/util/osdep.c index 60fcbbaebe..b85715a743 100644 --- a/util/osdep.c +++ b/util/osdep.c @@ -544,12 +544,17 @@ readv_writev(int fd, const struct iovec *iov, int iov_cnt, bool do_write) { unsigned i = 0; ssize_t ret = 0; + ssize_t off = 0; while (i < iov_cnt) { ssize_t r = do_write - ? write(fd, iov[i].iov_base, iov[i].iov_len) - : read(fd, iov[i].iov_base, iov[i].iov_len); + ? write(fd, iov[i].iov_base + off, iov[i].iov_len - off) + : read(fd, iov[i].iov_base + off, iov[i].iov_len - off); if (r > 0) { ret += r; + off += r; + if (off < iov[i].iov_len) { + continue; + } } else if (!r) { break; } else if (errno == EINTR) { @@ -562,6 +567,7 @@ readv_writev(int fd, const struct iovec *iov, int iov_cnt, bool do_write) } break; } + off = 0; i++; } return ret; From patchwork Thu Oct 6 08:12:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= X-Patchwork-Id: 13000057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70F70C433FE for ; Thu, 6 Oct 2022 08:38:58 +0000 (UTC) Received: from localhost ([::1]:35810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ogMPB-00074K-CR for qemu-devel@archiver.kernel.org; Thu, 06 Oct 2022 04:38:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60868) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzy-0006Qk-03 for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60689) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzv-0002pV-Rl for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665043971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ewLeS4ncP+E7VOsBF6eqtywiAtqKEQEmMxssfQaquZ4=; b=ZB45bODfR33Lmb419J4GmJ6QLcyPdrjEs0AIPsoxhaixrEic5HyDzSuSdwtZTo8U1Ipg8o GoYLcmvrqFRq3ZolUMLgCvVdSeMwsIsrjjloPbmPbH5NTfnsiFqFy0Do3BG1cz0njaVyCr BWLBfGniSkyerZaIh4CsmQKclFIPg6k= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-286-b5aFTXTjP_6jgOB1OO0UYg-1; Thu, 06 Oct 2022 04:12:50 -0400 X-MC-Unique: b5aFTXTjP_6jgOB1OO0UYg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E57EC29AB3E3 for ; Thu, 6 Oct 2022 08:12:34 +0000 (UTC) Received: from localhost (unknown [10.39.208.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0C72239DB3; Thu, 6 Oct 2022 08:12:33 +0000 (UTC) From: marcandre.lureau@redhat.com To: qemu-devel@nongnu.org Cc: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?M?= =?utf-8?q?arc-Andr=C3=A9_Lureau?= Subject: [PATCH v3 2/5] tests/channel-helper: set blocking in main thread Date: Thu, 6 Oct 2022 12:12:19 +0400 Message-Id: <20221006081222.2606746-3-marcandre.lureau@redhat.com> In-Reply-To: <20221006081222.2606746-1-marcandre.lureau@redhat.com> References: <20221006081222.2606746-1-marcandre.lureau@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 Received-SPF: pass client-ip=170.10.133.124; envelope-from=marcandre.lureau@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Marc-André Lureau The /io/channel/command/echo tests run the reader side and the writer side with the same underlying command channel. Setting the blocking mode of the fd/handles while the other end is already reading/writing may create issues (deadlock in win32 when earlier attempt of this series were using SetNamedPipeHandleState). Let's just do it before spawning the threads to avoid further concurrency issues. Signed-off-by: Marc-André Lureau Reviewed-by: Daniel P. Berrangé --- tests/unit/io-channel-helpers.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/tests/unit/io-channel-helpers.c b/tests/unit/io-channel-helpers.c index ff156ed3c4..c0799c21c2 100644 --- a/tests/unit/io-channel-helpers.c +++ b/tests/unit/io-channel-helpers.c @@ -25,7 +25,6 @@ struct QIOChannelTest { QIOChannel *src; QIOChannel *dst; - bool blocking; size_t len; size_t niov; char *input; @@ -42,8 +41,6 @@ static gpointer test_io_thread_writer(gpointer opaque) { QIOChannelTest *data = opaque; - qio_channel_set_blocking(data->src, data->blocking, NULL); - qio_channel_writev_all(data->src, data->inputv, data->niov, @@ -58,8 +55,6 @@ static gpointer test_io_thread_reader(gpointer opaque) { QIOChannelTest *data = opaque; - qio_channel_set_blocking(data->dst, data->blocking, NULL); - qio_channel_readv_all(data->dst, data->outputv, data->niov, @@ -113,7 +108,9 @@ void qio_channel_test_run_threads(QIOChannelTest *test, test->src = src; test->dst = dst; - test->blocking = blocking; + + qio_channel_set_blocking(test->dst, blocking, NULL); + qio_channel_set_blocking(test->src, blocking, NULL); reader = g_thread_new("reader", test_io_thread_reader, From patchwork Thu Oct 6 08:12:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= X-Patchwork-Id: 13000038 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1EBD9C433FE for ; Thu, 6 Oct 2022 08:22:15 +0000 (UTC) Received: from localhost ([::1]:60606 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ogM8z-00048R-FY for qemu-devel@archiver.kernel.org; Thu, 06 Oct 2022 04:22:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzo-0006O8-Ha for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:58213) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzm-0002om-Su for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665043960; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S0LxJV+KBjtlyQg/pWeX9lsL8wJ+xZKe/k1sVYgNX/Q=; b=USc/G43vPRkYfYnPNJrWfstJqBsHqZON+xA0BLCPu2yTkzlO2khg//otCdqWj2bb+r61Nv BFDA9NdkoTO8UW+KuTKGVGOdOZuhV0FAllKeYAOcVEpoIk+Te8EgUO6R4dQu13rHmN7M1J V0ZjFy3WcugVzgLy3eSo9mUfEcoqd2w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-47-O1HgomDkM4qpgRGXCJSu-g-1; Thu, 06 Oct 2022 04:12:39 -0400 X-MC-Unique: O1HgomDkM4qpgRGXCJSu-g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 085CB887401 for ; Thu, 6 Oct 2022 08:12:39 +0000 (UTC) Received: from localhost (unknown [10.39.208.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id C0F011121314; Thu, 6 Oct 2022 08:12:37 +0000 (UTC) From: marcandre.lureau@redhat.com To: qemu-devel@nongnu.org Cc: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?M?= =?utf-8?q?arc-Andr=C3=A9_Lureau?= Subject: [PATCH v3 3/5] io/command: use glib GSpawn, instead of open-coding fork/exec Date: Thu, 6 Oct 2022 12:12:20 +0400 Message-Id: <20221006081222.2606746-4-marcandre.lureau@redhat.com> In-Reply-To: <20221006081222.2606746-1-marcandre.lureau@redhat.com> References: <20221006081222.2606746-1-marcandre.lureau@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass client-ip=170.10.129.124; envelope-from=marcandre.lureau@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Marc-André Lureau Simplify qio_channel_command_new_spawn() with GSpawn API. This will allow to build for WIN32 in the following patches. As pointed out by Daniel Berrangé: there is a change in semantics here too. The current code only touches stdin/stdout/stderr. Any other FDs which do NOT have O_CLOEXEC set will be inherited. With the new code, all FDs except stdin/out/err will be explicitly closed, because we don't set the flag G_SPAWN_LEAVE_DESCRIPTORS_OPEN. The only place we use QIOChannelCommand today is the migration exec: protocol, and that is only declared to use stdin/stdout. Reviewed-by: Daniel P. Berrangé Signed-off-by: Marc-André Lureau --- include/io/channel-command.h | 2 +- io/channel-command.c | 105 ++++++----------------------------- 2 files changed, 19 insertions(+), 88 deletions(-) diff --git a/include/io/channel-command.h b/include/io/channel-command.h index 305ac1d280..8dc58273c0 100644 --- a/include/io/channel-command.h +++ b/include/io/channel-command.h @@ -41,7 +41,7 @@ struct QIOChannelCommand { QIOChannel parent; int writefd; int readfd; - pid_t pid; + GPid pid; }; diff --git a/io/channel-command.c b/io/channel-command.c index 9f2f4a1793..f84d1f03a0 100644 --- a/io/channel-command.c +++ b/io/channel-command.c @@ -31,7 +31,7 @@ * qio_channel_command_new_pid: * @writefd: the FD connected to the command's stdin * @readfd: the FD connected to the command's stdout - * @pid: the PID of the running child command + * @pid: the PID/HANDLE of the running child command * @errp: pointer to a NULL-initialized error object * * Create a channel for performing I/O with the @@ -50,7 +50,7 @@ static QIOChannelCommand * qio_channel_command_new_pid(int writefd, int readfd, - pid_t pid) + GPid pid) { QIOChannelCommand *ioc; @@ -69,94 +69,24 @@ qio_channel_command_new_spawn(const char *const argv[], int flags, Error **errp) { - pid_t pid = -1; - int stdinfd[2] = { -1, -1 }; - int stdoutfd[2] = { -1, -1 }; - int devnull = -1; - bool stdinnull = false, stdoutnull = false; - QIOChannelCommand *ioc; + g_autoptr(GError) err = NULL; + GPid pid = 0; + GSpawnFlags gflags = G_SPAWN_CLOEXEC_PIPES | G_SPAWN_DO_NOT_REAP_CHILD; + int stdinfd = -1, stdoutfd = -1; flags = flags & O_ACCMODE; - - if (flags == O_RDONLY) { - stdinnull = true; - } - if (flags == O_WRONLY) { - stdoutnull = true; - } - - if (stdinnull || stdoutnull) { - devnull = open("/dev/null", O_RDWR); - if (devnull < 0) { - error_setg_errno(errp, errno, - "Unable to open /dev/null"); - goto error; - } - } - - if ((!stdinnull && !g_unix_open_pipe(stdinfd, FD_CLOEXEC, NULL)) || - (!stdoutnull && !g_unix_open_pipe(stdoutfd, FD_CLOEXEC, NULL))) { - error_setg_errno(errp, errno, - "Unable to open pipe"); - goto error; - } - - pid = qemu_fork(errp); - if (pid < 0) { - goto error; - } - - if (pid == 0) { /* child */ - dup2(stdinnull ? devnull : stdinfd[0], STDIN_FILENO); - dup2(stdoutnull ? devnull : stdoutfd[1], STDOUT_FILENO); - /* Leave stderr connected to qemu's stderr */ - - if (!stdinnull) { - close(stdinfd[0]); - close(stdinfd[1]); - } - if (!stdoutnull) { - close(stdoutfd[0]); - close(stdoutfd[1]); - } - if (devnull != -1) { - close(devnull); - } - - execv(argv[0], (char * const *)argv); - _exit(1); + gflags |= flags == O_WRONLY ? G_SPAWN_STDOUT_TO_DEV_NULL : 0; + + if (!g_spawn_async_with_pipes(NULL, (char **)argv, NULL, gflags, NULL, NULL, + &pid, + flags == O_RDONLY ? NULL : &stdinfd, + flags == O_WRONLY ? NULL : &stdoutfd, + NULL, &err)) { + error_setg(errp, "%s", err->message); + return NULL; } - if (!stdinnull) { - close(stdinfd[0]); - } - if (!stdoutnull) { - close(stdoutfd[1]); - } - - ioc = qio_channel_command_new_pid(stdinnull ? devnull : stdinfd[1], - stdoutnull ? devnull : stdoutfd[0], - pid); - trace_qio_channel_command_new_spawn(ioc, argv[0], flags); - return ioc; - - error: - if (devnull != -1) { - close(devnull); - } - if (stdinfd[0] != -1) { - close(stdinfd[0]); - } - if (stdinfd[1] != -1) { - close(stdinfd[1]); - } - if (stdoutfd[0] != -1) { - close(stdoutfd[0]); - } - if (stdoutfd[1] != -1) { - close(stdoutfd[1]); - } - return NULL; + return qio_channel_command_new_pid(stdinfd, stdoutfd, pid); } #else /* WIN32 */ @@ -221,7 +151,7 @@ static void qio_channel_command_init(Object *obj) QIOChannelCommand *ioc = QIO_CHANNEL_COMMAND(obj); ioc->readfd = -1; ioc->writefd = -1; - ioc->pid = -1; + ioc->pid = 0; } static void qio_channel_command_finalize(Object *obj) @@ -239,6 +169,7 @@ static void qio_channel_command_finalize(Object *obj) #ifndef WIN32 qio_channel_command_abort(ioc, NULL); #endif + g_spawn_close_pid(ioc->pid); } } From patchwork Thu Oct 6 08:12:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= X-Patchwork-Id: 13000040 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2580C433F5 for ; Thu, 6 Oct 2022 08:28:49 +0000 (UTC) Received: from localhost ([::1]:45694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ogMFM-0001eN-CV for qemu-devel@archiver.kernel.org; Thu, 06 Oct 2022 04:28:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60864) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzv-0006QG-RE for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44312) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzs-0002pB-Bg for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665043967; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BgsGnfrXW57LeRzvUty34ARc83X7XjL6HPmKywQsdsI=; b=PV6agEjx6f7xcTEC8GmJaCMDyp5aqCMFhyUZQiIKqP6HS50tMhhe2lCiyKwds0haCdoEm1 /zR/fVfNW+B14qaIuQTgVbk1tShTs5a9inrWfboXg2bFMOd6sr1cN1sVF/cty8XmGfSzFL //UFWK80K8HV21XF6cfi3xWGCdia4PA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-224-5wmv80HSPXict0_uFC41kw-1; Thu, 06 Oct 2022 04:12:43 -0400 X-MC-Unique: 5wmv80HSPXict0_uFC41kw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 668A3811E87 for ; Thu, 6 Oct 2022 08:12:43 +0000 (UTC) Received: from localhost (unknown [10.39.208.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 537762166B30; Thu, 6 Oct 2022 08:12:42 +0000 (UTC) From: marcandre.lureau@redhat.com To: qemu-devel@nongnu.org Cc: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?M?= =?utf-8?q?arc-Andr=C3=A9_Lureau?= Subject: [PATCH v3 4/5] io/command: implement support for win32 Date: Thu, 6 Oct 2022 12:12:21 +0400 Message-Id: <20221006081222.2606746-5-marcandre.lureau@redhat.com> In-Reply-To: <20221006081222.2606746-1-marcandre.lureau@redhat.com> References: <20221006081222.2606746-1-marcandre.lureau@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.133.124; envelope-from=marcandre.lureau@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Marc-André Lureau The initial implementation was changing the pipe state created by GLib to PIPE_NOWAIT, but it turns out it doesn't work (read/write returns an error). Since reading may return less than the requested amount, it seems to be non-blocking already. However, the IO operation may block until the FD is ready, I can't find good sources of information, to be safe we can just poll for readiness before. Alternatively, we could setup the FDs ourself, and use UNIX sockets on Windows, which can be used in blocking/non-blocking mode. I haven't tried it, as I am not sure it is necessary. Signed-off-by: Marc-André Lureau Reviewed-by: Daniel P. Berrangé --- include/io/channel-command.h | 3 ++ io/channel-command.c | 80 ++++++++++++++++++++++++++---------- 2 files changed, 62 insertions(+), 21 deletions(-) diff --git a/include/io/channel-command.h b/include/io/channel-command.h index 8dc58273c0..98934e6d9e 100644 --- a/include/io/channel-command.h +++ b/include/io/channel-command.h @@ -42,6 +42,9 @@ struct QIOChannelCommand { int writefd; int readfd; GPid pid; +#ifdef WIN32 + bool blocking; +#endif }; diff --git a/io/channel-command.c b/io/channel-command.c index f84d1f03a0..74516252ba 100644 --- a/io/channel-command.c +++ b/io/channel-command.c @@ -26,7 +26,6 @@ #include "qemu/sockets.h" #include "trace.h" -#ifndef WIN32 /** * qio_channel_command_new_pid: * @writefd: the FD connected to the command's stdin @@ -60,7 +59,13 @@ qio_channel_command_new_pid(int writefd, ioc->writefd = writefd; ioc->pid = pid; - trace_qio_channel_command_new_pid(ioc, writefd, readfd, pid); + trace_qio_channel_command_new_pid(ioc, writefd, readfd, +#ifdef WIN32 + GetProcessId(pid) +#else + pid +#endif + ); return ioc; } @@ -89,18 +94,6 @@ qio_channel_command_new_spawn(const char *const argv[], return qio_channel_command_new_pid(stdinfd, stdoutfd, pid); } -#else /* WIN32 */ -QIOChannelCommand * -qio_channel_command_new_spawn(const char *const argv[], - int flags, - Error **errp) -{ - error_setg_errno(errp, ENOSYS, - "Command spawn not supported on this platform"); - return NULL; -} -#endif /* WIN32 */ - #ifndef WIN32 static int qio_channel_command_abort(QIOChannelCommand *ioc, Error **errp) @@ -143,6 +136,23 @@ static int qio_channel_command_abort(QIOChannelCommand *ioc, return 0; } +#else +static int qio_channel_command_abort(QIOChannelCommand *ioc, + Error **errp) +{ + DWORD ret; + + TerminateProcess(ioc->pid, 0); + ret = WaitForSingleObject(ioc->pid, 1000); + if (ret != WAIT_OBJECT_0) { + error_setg(errp, + "Process %llu refused to die", + (unsigned long long)GetProcessId(ioc->pid)); + return -1; + } + + return 0; +} #endif /* ! WIN32 */ @@ -166,13 +176,27 @@ static void qio_channel_command_finalize(Object *obj) } ioc->writefd = ioc->readfd = -1; if (ioc->pid > 0) { -#ifndef WIN32 qio_channel_command_abort(ioc, NULL); -#endif g_spawn_close_pid(ioc->pid); } } +#ifdef WIN32 +static bool win32_fd_poll(int fd, gushort events) +{ + GPollFD pfd = { .fd = _get_osfhandle(fd), .events = events }; + int res; + + do { + res = g_poll(&pfd, 1, 0); + } while (res < 0 && errno == EINTR); + if (res == 0) { + return false; + } + + return true; +} +#endif static ssize_t qio_channel_command_readv(QIOChannel *ioc, const struct iovec *iov, @@ -184,6 +208,12 @@ static ssize_t qio_channel_command_readv(QIOChannel *ioc, QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc); ssize_t ret; +#ifdef WIN32 + if (!cioc->blocking && !win32_fd_poll(cioc->readfd, G_IO_IN)) { + return QIO_CHANNEL_ERR_BLOCK; + } +#endif + retry: ret = readv(cioc->readfd, iov, niov); if (ret < 0) { @@ -213,6 +243,12 @@ static ssize_t qio_channel_command_writev(QIOChannel *ioc, QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc); ssize_t ret; +#ifdef WIN32 + if (!cioc->blocking && !win32_fd_poll(cioc->writefd, G_IO_OUT)) { + return QIO_CHANNEL_ERR_BLOCK; + } +#endif + retry: ret = writev(cioc->writefd, iov, niov); if (ret <= 0) { @@ -233,14 +269,14 @@ static int qio_channel_command_set_blocking(QIOChannel *ioc, bool enabled, Error **errp) { + QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc); + #ifdef WIN32 - /* command spawn is not supported on win32 */ - g_assert_not_reached(); + cioc->blocking = enabled; #else - QIOChannelCommand *cioc = QIO_CHANNEL_COMMAND(ioc); - if (!g_unix_set_fd_nonblocking(cioc->writefd, !enabled, NULL) || - !g_unix_set_fd_nonblocking(cioc->readfd, !enabled, NULL)) { + if ((cioc->writefd >= 0 && !g_unix_set_fd_nonblocking(cioc->writefd, !enabled, NULL)) || + (cioc->readfd >= 0 && !g_unix_set_fd_nonblocking(cioc->readfd, !enabled, NULL))) { error_setg_errno(errp, errno, "Failed to set FD nonblocking"); return -1; } @@ -281,6 +317,8 @@ static int qio_channel_command_close(QIOChannel *ioc, (unsigned long long)cioc->pid); return -1; } +#else + WaitForSingleObject(cioc->pid, INFINITE); #endif if (rv < 0) { From patchwork Thu Oct 6 08:12:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= X-Patchwork-Id: 13000039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C829C433F5 for ; Thu, 6 Oct 2022 08:27:58 +0000 (UTC) Received: from localhost ([::1]:60744 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ogMEX-0000qO-Iz for qemu-devel@archiver.kernel.org; Thu, 06 Oct 2022 04:27:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60866) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzv-0006QR-J1 for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29117) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ogLzt-0002pI-Tx for qemu-devel@nongnu.org; Thu, 06 Oct 2022 04:12:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1665043969; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e5k/wLn7IEccbADJ5zePlBZPAqaHVDssqFVHapiNmq8=; b=AuHh9cI/LJxzUztzWCp7H1rpPED5TjbjuTMOrHQvw+E0Yt4iM32vVkMR0Tq8q0lksRYXxa AzwiK13K+wobjlqmqPCqz5kFGqFvtPShQ/6YqJY4K71FrUhFqo/6d96pE1d+YOtWS2i5Jn 12HiZYMsk1JhV3xy4l8UMikRRnk7M5s= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-516-fxVLurNWMlCR1ZU8XpqHog-1; Thu, 06 Oct 2022 04:12:47 -0400 X-MC-Unique: fxVLurNWMlCR1ZU8XpqHog-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 970443811F28 for ; Thu, 6 Oct 2022 08:12:47 +0000 (UTC) Received: from localhost (unknown [10.39.208.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E54E2166B2E; Thu, 6 Oct 2022 08:12:46 +0000 (UTC) From: marcandre.lureau@redhat.com To: qemu-devel@nongnu.org Cc: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , =?utf-8?q?M?= =?utf-8?q?arc-Andr=C3=A9_Lureau?= Subject: [PATCH v3 5/5] tests/unit: make test-io-channel-command work on win32 Date: Thu, 6 Oct 2022 12:12:22 +0400 Message-Id: <20221006081222.2606746-6-marcandre.lureau@redhat.com> In-Reply-To: <20221006081222.2606746-1-marcandre.lureau@redhat.com> References: <20221006081222.2606746-1-marcandre.lureau@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 Received-SPF: pass client-ip=170.10.133.124; envelope-from=marcandre.lureau@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Marc-André Lureau This has been tested under msys2 & windows 11. I haven't tried to make it work with other environments yet, but that should be enough to validate the channel-command implementation anyway. Signed-off-by: Marc-André Lureau --- tests/unit/test-io-channel-command.c | 32 ++++++++++++---------------- tests/unit/meson.build | 2 +- 2 files changed, 15 insertions(+), 19 deletions(-) diff --git a/tests/unit/test-io-channel-command.c b/tests/unit/test-io-channel-command.c index aa09c559cd..be98c3452a 100644 --- a/tests/unit/test-io-channel-command.c +++ b/tests/unit/test-io-channel-command.c @@ -24,29 +24,27 @@ #include "qapi/error.h" #include "qemu/module.h" -#ifndef WIN32 +#define TEST_PATH "test-io-channel-command.fifo" + +#define SOCAT_SRC "PIPE:" TEST_PATH ",wronly" +#define SOCAT_DST "PIPE:" TEST_PATH ",rdonly" + static void test_io_channel_command_fifo(bool async) { -#define TEST_FIFO "tests/test-io-channel-command.fifo" QIOChannel *src, *dst; QIOChannelTest *test; - const char *srcfifo = "PIPE:" TEST_FIFO ",wronly"; - const char *dstfifo = "PIPE:" TEST_FIFO ",rdonly"; const char *srcargv[] = { - "/bin/socat", "-", srcfifo, NULL, + g_getenv("SOCAT"), "-", SOCAT_SRC, NULL, }; const char *dstargv[] = { - "/bin/socat", dstfifo, "-", NULL, + g_getenv("SOCAT"), SOCAT_DST, "-", NULL, }; - unlink(TEST_FIFO); - if (access("/bin/socat", X_OK) < 0) { + unlink(TEST_PATH); + if (!g_file_test(g_getenv("SOCAT"), G_FILE_TEST_IS_EXECUTABLE)) { g_test_skip("socat is missing"); return; } - if (mkfifo(TEST_FIFO, 0600) < 0) { - abort(); - } src = QIO_CHANNEL(qio_channel_command_new_spawn(srcargv, O_WRONLY, &error_abort)); @@ -61,7 +59,7 @@ static void test_io_channel_command_fifo(bool async) object_unref(OBJECT(src)); object_unref(OBJECT(dst)); - unlink(TEST_FIFO); + unlink(TEST_PATH); } @@ -81,11 +79,12 @@ static void test_io_channel_command_echo(bool async) QIOChannel *ioc; QIOChannelTest *test; const char *socatargv[] = { - "/bin/socat", "-", "-", NULL, + g_getenv("SOCAT"), "-", "-", NULL, }; - if (access("/bin/socat", X_OK) < 0) { - return; /* Pretend success if socat is not present */ + if (!g_file_test(g_getenv("SOCAT"), G_FILE_TEST_IS_EXECUTABLE)) { + g_test_skip("socat is missing"); + return; } ioc = QIO_CHANNEL(qio_channel_command_new_spawn(socatargv, @@ -108,7 +107,6 @@ static void test_io_channel_command_echo_sync(void) { test_io_channel_command_echo(false); } -#endif int main(int argc, char **argv) { @@ -116,7 +114,6 @@ int main(int argc, char **argv) g_test_init(&argc, &argv, NULL); -#ifndef WIN32 g_test_add_func("/io/channel/command/fifo/sync", test_io_channel_command_fifo_sync); g_test_add_func("/io/channel/command/fifo/async", @@ -125,7 +122,6 @@ int main(int argc, char **argv) test_io_channel_command_echo_sync); g_test_add_func("/io/channel/command/echo/async", test_io_channel_command_echo_async); -#endif return g_test_run(); } diff --git a/tests/unit/meson.build b/tests/unit/meson.build index b497a41378..42e8218ac2 100644 --- a/tests/unit/meson.build +++ b/tests/unit/meson.build @@ -1,4 +1,3 @@ - testblock = declare_dependency(dependencies: [block], sources: 'iothread.c') tests = { @@ -164,6 +163,7 @@ endif test_env = environment() test_env.set('G_TEST_SRCDIR', meson.current_source_dir()) test_env.set('G_TEST_BUILDDIR', meson.current_build_dir()) +test_env.set('SOCAT', find_program('socat').full_path()) slow_tests = { 'test-crypto-tlscredsx509': 45,