From patchwork Sun Oct 22 20:12:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13432061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D4F9CDB474 for ; Sun, 22 Oct 2023 20:14:06 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1queoY-0005FY-Oq; Sun, 22 Oct 2023 16:12:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoU-0005Cd-GH for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoS-00060s-Nx for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698005558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fLhg34I1cMdFLR/HfPBn3YE3lnP5nXAiKWY2LyUuwjY=; b=dfpHBSyMo2A8c6yoKGKVFXxONKesbe4FTdpqggA1vmMtRjVOku91u+N2Mq0l9+gz4UMYEv H6I+UPIS8cQ3aMnnop3V5Rw3wQgZdhJb6AEle/qxD48aPp+WUKB4eDz6Il+3QlKgtTO0ZZ ApZEMJZX9GIzF3EnheXC0o1qvj5LzR4= Received: from mail-oo1-f71.google.com (mail-oo1-f71.google.com [209.85.161.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-259-KjmLqhOZOo295Dr1YAYN2w-1; Sun, 22 Oct 2023 16:12:36 -0400 X-MC-Unique: KjmLqhOZOo295Dr1YAYN2w-1 Received: by mail-oo1-f71.google.com with SMTP id 006d021491bc7-581e7b03855so1204120eaf.0 for ; Sun, 22 Oct 2023 13:12:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698005556; x=1698610356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fLhg34I1cMdFLR/HfPBn3YE3lnP5nXAiKWY2LyUuwjY=; b=AzW4Lk10hrUzHBDIgMLvJRTPX7mqe+PHKglc5P1Ff0/6NXfQcKFxtT+gdzv4cleqjv 5DVz6F4h3By5XYUqvQdCJXoNRwyfuq0t1JWAFsxsgCrAzkvnkCk6JfrJ8NYMKBAitVPH TkALoA8Ormut38sesETlESZgcazlIceZ4BASk446fK38LPWIRqnFxG1B0f8rXxkfGz+S pkmzRt2w3/MgESpy35YYOCFqXudwv4hxVEn7BMl4ZHt1/7Xa8kDQ80CQth0OigS7Oujj qrHqNoBkJ5P6Caj1Ujj3EEOs6Iw6k24qgaqcN19mfJTY78KZEOjgG8mOKHxi1t8feVgZ ZCAA== X-Gm-Message-State: AOJu0Yx58SWuo0eMOfinWR394RJkx6HOyw9GaExyz1bQBVU9/LwocEDz ZNbPSHvGjlH9Ma+SAoVolaXpG5GXVbZLk5NwuU3kIDxBbDiMquTG/K8AEjL62na/P0BkH2p9eHc QYtVGnwhdDY77pWwZ7iGEwHVJobu3goMQRJ5rDv8eyHQTgAo4O1fCtmgA9XQkOBvSE316a7Hx X-Received: by 2002:a05:6870:9d98:b0:1e9:a8ff:67e3 with SMTP id pv24-20020a0568709d9800b001e9a8ff67e3mr9408199oab.4.1698005555867; Sun, 22 Oct 2023 13:12:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHWxS4X7MsUD3DKTl4ASCdQWSYdLlztOu+wWdeVqXCMJ5wF2cLrQysPIwGaGDPZvJ6+pRbGvw== X-Received: by 2002:a05:6870:9d98:b0:1e9:a8ff:67e3 with SMTP id pv24-20020a0568709d9800b001e9a8ff67e3mr9408176oab.4.1698005555399; Sun, 22 Oct 2023 13:12:35 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id x11-20020a0cfe0b000000b0065af657ddf7sm2330515qvr.144.2023.10.22.13.12.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Oct 2023 13:12:34 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , peterx@redhat.com Subject: [PATCH RFC 1/7] migration: Drop stale comment for multifd zero copy Date: Sun, 22 Oct 2023 16:12:05 -0400 Message-ID: <20231022201211.452861-2-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231022201211.452861-1-peterx@redhat.com> References: <20231022201211.452861-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We've already done that with multifd_flush_after_each_section, for multifd in general. Drop the stale "TODO-like" comment. Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas --- migration/multifd.c | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 1fe53d3b98..c8bdd88041 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -600,17 +600,6 @@ int multifd_send_sync_main(QEMUFile *f) } } - /* - * When using zero-copy, it's necessary to flush the pages before any of - * the pages can be sent again, so we'll make sure the new version of the - * pages will always arrive _later_ than the old pages. - * - * Currently we achieve this by flushing the zero-page requested writes - * per ram iteration, but in the future we could potentially optimize it - * to be less frequent, e.g. only after we finished one whole scanning of - * all the dirty bitmaps. - */ - flush_zero_copy = migrate_zero_copy_send(); for (i = 0; i < migrate_multifd_channels(); i++) { From patchwork Sun Oct 22 20:12:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13432059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BADFCDB474 for ; Sun, 22 Oct 2023 20:13:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1queoX-0005F4-4l; Sun, 22 Oct 2023 16:12:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoU-0005Ce-GH for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoS-00060v-Ki for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698005558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GEK3BC2R+ljz1Pe0etq5Hqc5Baqx24HXWF1XvvP10jw=; b=huSvny0VB9XHgNijTA08jdBfFjYruTHxRmjp38GQnmKAoDDQydSimb2DKtJnS3bsfG6wlF N1L+KLwPESlFXyptQr2D7od6O04fGR7hfIeJkCuTYcob0W+WNbOwLBKaVqDkil+o6qTtXr 1cIydGTD6R1JPnzVkySaF300gAS5nVU= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-480-F7XOLwcwMbOp1pyVwgrpcQ-1; Sun, 22 Oct 2023 16:12:37 -0400 X-MC-Unique: F7XOLwcwMbOp1pyVwgrpcQ-1 Received: by mail-oi1-f199.google.com with SMTP id 5614622812f47-3ade1013032so901243b6e.1 for ; Sun, 22 Oct 2023 13:12:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698005556; x=1698610356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GEK3BC2R+ljz1Pe0etq5Hqc5Baqx24HXWF1XvvP10jw=; b=AttO+wcLJYo6zGvTaZhKUfc3x1Nv/A+kh/1XHjSSH8ASTisqfXZpARdPxM3CPlLwmU eAes0qNYaUXfCQu4btYKbk4+IE6j+aiQZ/6mk2J+dPHIeQRtWYuzxJgmG1mi9DJKriVV XPX1/E3p3ckHpMCUfcohVKeMKvVvptBBPHZ8UHUZdAj8Fn8b2fTNpJJiHpAuR7uQcOfK XEDZg7EnhShMWo0qWWbVdUMc6yQ4z5h+S/lRj0WSVmC1ZoN5xHB5HJpYymLf/9EF6xKj fofNGuK1XB/GKiwk7alAFPRf61fXFjvrlhC4rx4PEWRRMP5tAilQJCCjgYp+37B163NR 7qsg== X-Gm-Message-State: AOJu0YxdHoXk6FQ0+Hrxer50hKg2YKNUUun6QOR6BmOUq3Wi4OoiPZsy A6D5Nl6lXMPSKYLzrw9ArLEyvwGoa09Ji0Ve/ycGFvMjCRAK/koeGwni2TCNy85imlGyT5dbyAF KN9rjFhFCOg1CsPzBLEzm8GQdZm0HsmrnokrjJ/rtEKtYMpXR3MIT039A/Uu82QiiTMSuk/As X-Received: by 2002:a05:6808:38c6:b0:3a9:f25d:d917 with SMTP id el6-20020a05680838c600b003a9f25dd917mr8377637oib.4.1698005556494; Sun, 22 Oct 2023 13:12:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEaH17Qcs7m4G/kZvhXh+gI3ScRQtv4S+b7ty/qaUpH+mSDsLBteeMs7h5qXVNNoeS2cinuPQ== X-Received: by 2002:a05:6808:38c6:b0:3a9:f25d:d917 with SMTP id el6-20020a05680838c600b003a9f25dd917mr8377625oib.4.1698005556187; Sun, 22 Oct 2023 13:12:36 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id x11-20020a0cfe0b000000b0065af657ddf7sm2330515qvr.144.2023.10.22.13.12.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Oct 2023 13:12:35 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , peterx@redhat.com Subject: [PATCH RFC 2/7] migration: Fix error leak in multifd_tls_outgoing_handshake() Date: Sun, 22 Oct 2023 16:12:06 -0400 Message-ID: <20231022201211.452861-3-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231022201211.452861-1-peterx@redhat.com> References: <20231022201211.452861-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SORBS_WEB=1.5, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The Error* is leaked when error triggerred. It logically should have a Fixes here, but since the code changed a few times, the backport won't be straightforward anyway. Let's not bother with leaking an error in the failure path for now. Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas --- migration/multifd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration/multifd.c b/migration/multifd.c index c8bdd88041..4afdd88602 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -789,6 +789,7 @@ static void multifd_tls_outgoing_handshake(QIOTask *task, p->quit = true; qemu_sem_post(&multifd_send_state->channels_ready); qemu_sem_post(&p->sem_sync); + error_free(err); } static void *multifd_tls_handshake_thread(void *opaque) From patchwork Sun Oct 22 20:12:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13432064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CE6DECDB474 for ; Sun, 22 Oct 2023 20:14:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1queoZ-0005GI-Ux; Sun, 22 Oct 2023 16:12:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoV-0005D3-55 for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoT-000618-HK for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698005560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w3Z7cHyCISTXqbo3tI+yi6RkFj6TgXVXNtX2A4hjFww=; b=PN3GE4OnzJyXlymJZFz1ZaaD74dyLgNIPOpcR5s9B0Xx3grnqIhKmn9x+KEhyp/xGIqElB JTx/YbboBJo49b6C286zV/hYS7RoR7+Z92hsKQ7WAhK7wbNx+EDCBHUPBsA+JM3wy1dg/x Sz+QZfo5SNlbegwMpIsrZxrVFUNg5bE= Received: from mail-oo1-f69.google.com (mail-oo1-f69.google.com [209.85.161.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-100-lH_YLdjsPYOMSp8zgs6XLQ-1; Sun, 22 Oct 2023 16:12:38 -0400 X-MC-Unique: lH_YLdjsPYOMSp8zgs6XLQ-1 Received: by mail-oo1-f69.google.com with SMTP id 006d021491bc7-581e7b03855so1204145eaf.0 for ; Sun, 22 Oct 2023 13:12:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698005557; x=1698610357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w3Z7cHyCISTXqbo3tI+yi6RkFj6TgXVXNtX2A4hjFww=; b=PzIAmCLSB0BLQU9aVooeL33OeXOsHPmTPiLgaiMGirpC2//qPDXLZG0+C1Ko5PlnCK RCphhFTVquzEHnV4IIcziCAMs4G57ngQgWlZiqK3fuX9cdgTkFZU5mWSLeHtFYEGdxKs sGPcmM0gM7fCT3BDXWRW3jFIAuvNWi41Ql+N5NtaTXLOjztRAX1s8QmxnqQpX9l+Vbqa d59rK85Ofe1xj85R7x80tLjVI+NPM1LnRLj3HhzKzLjLSyjkB69aIvIhRQtQjq0+KIdD azNpInPb/jhzj/N3XRvBXQ5+l16RcRbX+AD3Q8zNzvUrAW2wE64K2JEH4hrECQd+No7Z eZfQ== X-Gm-Message-State: AOJu0Yx7J/Hcn9ggcxbrb6sA7uR92MxSFcnJIbIzfANMX09GBc0InBQY BUQjMEpj6peum/2QDhTkKvKfzSLmf30LTcZyqH4T4gt0t+nlBfPhbtO2M+UWT5wD5dxj9pGHUyY C9aX3j8pRsaaHDistUT6VO1OQG1e+rzMMQAhTwqeNufiS5hhxjHnwK7e0ZUfGDvdXFXMdsuuA X-Received: by 2002:a05:6870:9d98:b0:1e9:a8ff:67e3 with SMTP id pv24-20020a0568709d9800b001e9a8ff67e3mr9408267oab.4.1698005557639; Sun, 22 Oct 2023 13:12:37 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFkNJYSq0/lLd+0nSdmJm31J7B4r3mkNSq3AAVPho0SEqm8bWrECoEmgczYLxJa5HB12SfRRg== X-Received: by 2002:a05:6870:9d98:b0:1e9:a8ff:67e3 with SMTP id pv24-20020a0568709d9800b001e9a8ff67e3mr9408252oab.4.1698005557310; Sun, 22 Oct 2023 13:12:37 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id x11-20020a0cfe0b000000b0065af657ddf7sm2330515qvr.144.2023.10.22.13.12.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Oct 2023 13:12:36 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , peterx@redhat.com Subject: [PATCH RFC 3/7] migration: multifd_send_kick_main() Date: Sun, 22 Oct 2023 16:12:07 -0400 Message-ID: <20231022201211.452861-4-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231022201211.452861-1-peterx@redhat.com> References: <20231022201211.452861-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When a multifd sender thread hit errors, it always needs to kick the main thread by kicking all the semaphores that it can be waiting upon. Provide a helper for it and deduplicate the code. Signed-off-by: Peter Xu Reviewed-by: Fabiano Rosas --- migration/multifd.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 4afdd88602..33fb21d0e4 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -374,6 +374,18 @@ struct { MultiFDMethods *ops; } *multifd_send_state; +/* + * The migration thread can wait on either of the two semaphores. This + * function can be used to kick the main thread out of waiting on either of + * them. Should mostly only be called when something wrong happened with + * the current multifd send thread. + */ +static void multifd_send_kick_main(MultiFDSendParams *p) +{ + qemu_sem_post(&p->sem_sync); + qemu_sem_post(&multifd_send_state->channels_ready); +} + /* * How we use multifd_send_state->pages and channel->pages? * @@ -746,8 +758,7 @@ out: assert(local_err); trace_multifd_send_error(p->id); multifd_send_terminate_threads(local_err); - qemu_sem_post(&p->sem_sync); - qemu_sem_post(&multifd_send_state->channels_ready); + multifd_send_kick_main(p); error_free(local_err); } @@ -787,8 +798,7 @@ static void multifd_tls_outgoing_handshake(QIOTask *task, * is not created, and then tell who pay attention to me. */ p->quit = true; - qemu_sem_post(&multifd_send_state->channels_ready); - qemu_sem_post(&p->sem_sync); + multifd_send_kick_main(p); error_free(err); } @@ -859,8 +869,7 @@ static void multifd_new_send_channel_cleanup(MultiFDSendParams *p, { migrate_set_error(migrate_get_current(), err); /* Error happen, we need to tell who pay attention to me */ - qemu_sem_post(&multifd_send_state->channels_ready); - qemu_sem_post(&p->sem_sync); + multifd_send_kick_main(p); /* * Although multifd_send_thread is not created, but main migration * thread need to judge whether it is running, so we need to mark From patchwork Sun Oct 22 20:12:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13432058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1BEC3CDB474 for ; Sun, 22 Oct 2023 20:13:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1queoY-0005FX-PE; Sun, 22 Oct 2023 16:12:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoW-0005Ea-9g for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoU-00061J-A6 for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698005561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F96rZlJDGWqR9lE4rEHkK4a/bYoywEb7PcXAjwNkcmM=; b=V38lsjf9QGJVJ0M+GmW8EsJBF37Z2gmbqQI3f14XaqVwSJYT3qMIEuyKiwN6XAk8IRY6VB YwfzZ7N3Lu8kUJcNQszqYBFqDuj3OsKUx1xKIQ60ZFFpvpD5ZKI2BFaZuyiV0wD48gpy+o n0R++9LlacTERawIVQ8JZPzsRBCs6k4= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-331-grsD6fm_N9u8gj2aJojSIA-1; Sun, 22 Oct 2023 16:12:39 -0400 X-MC-Unique: grsD6fm_N9u8gj2aJojSIA-1 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-66cfa898cfeso7839746d6.0 for ; Sun, 22 Oct 2023 13:12:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698005559; x=1698610359; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F96rZlJDGWqR9lE4rEHkK4a/bYoywEb7PcXAjwNkcmM=; b=wT6ONSQCbvcAaPO6EjXd5lZtGZYofIajrv941KT6arYyet9Zfa/SxzKy0y+HEH4ykc 2IeovzKvCUasEm+FYBzZmfEW4qdmJrJqslviFHqDplHb1mFr6iIT5O0zuHcXi3RY1QCs Y0J50cOQetlddXLk/qr8RjGUln02QSAby1Kdt/OwuBBoLfJvzEfjf/KTle4POGR8C/9k b/OjGdqjSY+0VJmfBmwW6UlZGEjpMMbSJdxvX0nrbTxd/6LzMbVpWYD6QhehysDdrbNY QAx1vZZUoSsDLceYu0u71JoejJsBtgYp+dZ2HNqRNAOxs8Ib3B/hIrOYu2jOLUEOEeh1 fT1g== X-Gm-Message-State: AOJu0YwgCKRKPdWJ8aPuLDQ2lyXJ2Q+m/41wECpsFCEBEzdejEi4iln3 x//oqX4JKP4JxyakyeuHkAd2UjkVXMTwblo8JRXZ0l34n8cQc/d16ubzVJvLUJQMjgukmPtldKO 6GuN84uqEHsgzm9muCuuZBCgqvsMTqDIviWX+CEnfzAM/Ha90GcIbcNLH+hvSNO5W8uR0wISq X-Received: by 2002:ad4:58f2:0:b0:66d:44f2:7fcb with SMTP id di18-20020ad458f2000000b0066d44f27fcbmr7838428qvb.2.1698005558894; Sun, 22 Oct 2023 13:12:38 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGAENAIwIcNGFVixg6pmS9mVIF2qQxYLvxbj3IkO+bkcFElgBRxoWnkoz12xc/6BGDeiQxn+w== X-Received: by 2002:ad4:58f2:0:b0:66d:44f2:7fcb with SMTP id di18-20020ad458f2000000b0066d44f27fcbmr7838413qvb.2.1698005558552; Sun, 22 Oct 2023 13:12:38 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id x11-20020a0cfe0b000000b0065af657ddf7sm2330515qvr.144.2023.10.22.13.12.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Oct 2023 13:12:37 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , peterx@redhat.com Subject: [PATCH RFC 4/7] migration: Drop MultiFDSendParams.quit and cleanup error paths Date: Sun, 22 Oct 2023 16:12:08 -0400 Message-ID: <20231022201211.452861-5-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231022201211.452861-1-peterx@redhat.com> References: <20231022201211.452861-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Multifd send side has two fields to indicate error quits: - MultiFDSendParams.quit - &multifd_send_state->exiting Merge them into the global one. The replacement is done by changing all p->quit checks into the global var check. The global check doesn't need any lock. A few more things done on top of this altogether: - multifd_send_terminate_threads() Moving the xchg() of &multifd_send_state->exiting upper, so as to cover the tracepoint, migrate_set_error() and migrate_set_state(). - multifd_send_sync_main() In the 2nd loop, add one more check over the global var to make sure we don't keep the looping if QEMU already decided to quit. - multifd_tls_outgoing_handshake() Use multifd_send_terminate_threads() to set the error state. That has a benefit of updating MigrationState.error to that error too, so we can persist that 1st error we hit in that specific channel. - multifd_new_send_channel_async() Take similar approach like above, drop the migrate_set_error() because multifd_send_terminate_threads() already covers that. Unwrap the helper multifd_new_send_channel_cleanup() along the way; not really needed. Signed-off-by: Peter Xu --- migration/multifd.h | 2 -- migration/multifd.c | 82 ++++++++++++++------------------------------- 2 files changed, 26 insertions(+), 58 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index a835643b48..2acf400085 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -97,8 +97,6 @@ typedef struct { QemuMutex mutex; /* is this channel thread running */ bool running; - /* should this thread finish */ - bool quit; /* multifd flags for each packet */ uint32_t flags; /* global number of generated multifd packets */ diff --git a/migration/multifd.c b/migration/multifd.c index 33fb21d0e4..9d458914a9 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -411,10 +411,6 @@ static int multifd_send_pages(QEMUFile *f) MultiFDSendParams *p = NULL; /* make happy gcc */ MultiFDPages_t *pages = multifd_send_state->pages; - if (qatomic_read(&multifd_send_state->exiting)) { - return -1; - } - qemu_sem_wait(&multifd_send_state->channels_ready); /* * next_channel can remain from a previous migration that was @@ -423,14 +419,11 @@ static int multifd_send_pages(QEMUFile *f) */ next_channel %= migrate_multifd_channels(); for (i = next_channel;; i = (i + 1) % migrate_multifd_channels()) { - p = &multifd_send_state->params[i]; - - qemu_mutex_lock(&p->mutex); - if (p->quit) { - error_report("%s: channel %d has already quit!", __func__, i); - qemu_mutex_unlock(&p->mutex); + if (qatomic_read(&multifd_send_state->exiting)) { return -1; } + p = &multifd_send_state->params[i]; + qemu_mutex_lock(&p->mutex); if (!p->pending_job) { p->pending_job++; next_channel = (i + 1) % migrate_multifd_channels(); @@ -485,6 +478,16 @@ static void multifd_send_terminate_threads(Error *err) { int i; + /* + * We don't want to exit each threads twice. Depending on where + * we get the error, or if there are two independent errors in two + * threads at the same time, we can end calling this function + * twice. + */ + if (qatomic_xchg(&multifd_send_state->exiting, 1)) { + return; + } + trace_multifd_send_terminate_threads(err != NULL); if (err) { @@ -499,26 +502,13 @@ static void multifd_send_terminate_threads(Error *err) } } - /* - * We don't want to exit each threads twice. Depending on where - * we get the error, or if there are two independent errors in two - * threads at the same time, we can end calling this function - * twice. - */ - if (qatomic_xchg(&multifd_send_state->exiting, 1)) { - return; - } - for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; - qemu_mutex_lock(&p->mutex); - p->quit = true; qemu_sem_post(&p->sem); if (p->c) { qio_channel_shutdown(p->c, QIO_CHANNEL_SHUTDOWN_BOTH, NULL); } - qemu_mutex_unlock(&p->mutex); } } @@ -617,16 +607,13 @@ int multifd_send_sync_main(QEMUFile *f) for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; - trace_multifd_send_sync_main_signal(p->id); - - qemu_mutex_lock(&p->mutex); - - if (p->quit) { - error_report("%s: channel %d has already quit", __func__, i); - qemu_mutex_unlock(&p->mutex); + if (qatomic_read(&multifd_send_state->exiting)) { return -1; } + trace_multifd_send_sync_main_signal(p->id); + + qemu_mutex_lock(&p->mutex); p->packet_num = multifd_send_state->packet_num++; p->flags |= MULTIFD_FLAG_SYNC; p->pending_job++; @@ -636,6 +623,10 @@ int multifd_send_sync_main(QEMUFile *f) for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; + if (qatomic_read(&multifd_send_state->exiting)) { + return -1; + } + qemu_sem_wait(&multifd_send_state->channels_ready); trace_multifd_send_sync_main_wait(p->id); qemu_sem_wait(&p->sem_sync); @@ -744,9 +735,6 @@ static void *multifd_send_thread(void *opaque) if (flags & MULTIFD_FLAG_SYNC) { qemu_sem_post(&p->sem_sync); } - } else if (p->quit) { - qemu_mutex_unlock(&p->mutex); - break; } else { qemu_mutex_unlock(&p->mutex); /* sometimes there are spurious wakeups */ @@ -793,11 +781,7 @@ static void multifd_tls_outgoing_handshake(QIOTask *task, trace_multifd_tls_outgoing_handshake_error(ioc, error_get_pretty(err)); - /* - * Error happen, mark multifd_send_thread status as 'quit' although it - * is not created, and then tell who pay attention to me. - */ - p->quit = true; + multifd_send_terminate_threads(err); multifd_send_kick_main(p); error_free(err); } @@ -864,22 +848,6 @@ static bool multifd_channel_connect(MultiFDSendParams *p, return true; } -static void multifd_new_send_channel_cleanup(MultiFDSendParams *p, - QIOChannel *ioc, Error *err) -{ - migrate_set_error(migrate_get_current(), err); - /* Error happen, we need to tell who pay attention to me */ - multifd_send_kick_main(p); - /* - * Although multifd_send_thread is not created, but main migration - * thread need to judge whether it is running, so we need to mark - * its status. - */ - p->quit = true; - object_unref(OBJECT(ioc)); - error_free(err); -} - static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) { MultiFDSendParams *p = opaque; @@ -897,7 +865,10 @@ static void multifd_new_send_channel_async(QIOTask *task, gpointer opaque) } trace_multifd_new_send_channel_async_error(p->id, local_err); - multifd_new_send_channel_cleanup(p, ioc, local_err); + multifd_send_terminate_threads(local_err); + multifd_send_kick_main(p); + object_unref(OBJECT(ioc)); + error_free(local_err); } static void multifd_new_send_channel_create(gpointer opaque) @@ -929,7 +900,6 @@ int multifd_save_setup(Error **errp) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); - p->quit = false; p->pending_job = 0; p->id = i; p->pages = multifd_pages_init(page_count); From patchwork Sun Oct 22 20:12:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13432060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7C1D8C25B41 for ; Sun, 22 Oct 2023 20:13:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1queoZ-0005Fk-Bo; Sun, 22 Oct 2023 16:12:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoX-0005F6-6S for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoV-00061j-ES for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698005562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uOiiFysHiha1Yh2C+xE/FVmKDgYGIOfNW5Tic1Olavc=; b=ZtcrD+599a741Gy5zco16F684BLwk30hyzILxO+jKoB+Sz0Zpd/uL9v0pfO4mzTEvDCyMt KsGLnzif1TVK3omd+88YnfQaEEKf2X8KJyudZ3sKdPIVX5bd6NaC8sDzWTUZ9bPwXSFPdA vgJV4VMmX7KLpA7JpZ/GtqcfgFuYdhY= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-483-0j4quv0INnKAsju2hCpGoA-1; Sun, 22 Oct 2023 16:12:41 -0400 X-MC-Unique: 0j4quv0INnKAsju2hCpGoA-1 Received: by mail-qk1-f198.google.com with SMTP id af79cd13be357-778b5c628f4so36773485a.0 for ; Sun, 22 Oct 2023 13:12:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698005560; x=1698610360; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uOiiFysHiha1Yh2C+xE/FVmKDgYGIOfNW5Tic1Olavc=; b=lfN/lPZKXnhjrrUX66le6kqN6/Oho5vnxdYf2wTqxeawDjuk4oELwpZ7WcbKYPjQjw Bkb32ilFTDeQHurarnDxMcExPI6JQQWl7Rtr/3D5z91YIhU3nRdchbauqGR4Uaa5Gynx aN0I26u7P3EgpwW9Fkmfz9rmXRSIfJnl6yEO15cr8/3mlnwKert+gAc8+0FO4VZ2IP1k PhlAIrQ+jh20vw2EziEJtyTnxBDfpWl6eqFLNCWTf183p27kRrNg39b6N3DV7S+8gmnn HrfmUOeRCii5vcO2JSUZEKrrNqerLZBCiC1aDF4MpLpsNccgreo12zWw8rwy5/y5iW4S y6vQ== X-Gm-Message-State: AOJu0YwzTJE9zC0h2gRTPa5J/3fpbaS5OL55ywM/Tyj8o92JI1VxoxVq aZH9bIuE1rKj3oaRJ2ZrjyXs2p68fgGR4jXu/Aqd7Pif4hRcv7uRhaNcCA6Zdb528/uNr0hFZJk WHI3pRMV+vAQe6N0hbY4qtn/ODGaoZhu7hmI24zSUkdfU8Zu3/v0SNiPVC/IaI7daHAr9tVX2 X-Received: by 2002:ad4:57d1:0:b0:66d:b23:a62e with SMTP id y17-20020ad457d1000000b0066d0b23a62emr7643793qvx.6.1698005560180; Sun, 22 Oct 2023 13:12:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHV6uXGBS9VAROHWbBzuzzVZk01cDs9ue6F0TFMEKUiCQdUgykBwyJ29sud8EoIn/NR6fa2Wg== X-Received: by 2002:ad4:57d1:0:b0:66d:b23:a62e with SMTP id y17-20020ad457d1000000b0066d0b23a62emr7643778qvx.6.1698005559780; Sun, 22 Oct 2023 13:12:39 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id x11-20020a0cfe0b000000b0065af657ddf7sm2330515qvr.144.2023.10.22.13.12.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Oct 2023 13:12:39 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , peterx@redhat.com Subject: [PATCH RFC 5/7] migration: Modulize multifd send threads with a few helpers Date: Sun, 22 Oct 2023 16:12:09 -0400 Message-ID: <20231022201211.452861-6-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231022201211.452861-1-peterx@redhat.com> References: <20231022201211.452861-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Abstract the multifd send packet logic into two phases: - multifd_send_prepare(): prepare the packet headers, with mutex - multifd_do_send(): do the send job finally, without mutex When at it, always allow the send thread to use Error* for detecting errors, dropping "int ret" altogether. One trivial change is the send thread now kicks the sem_sync within mutex critical section, but that shouldn't be a problem. Signed-off-by: Peter Xu --- migration/multifd.c | 160 ++++++++++++++++++++++++++------------------ 1 file changed, 96 insertions(+), 64 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index 9d458914a9..8140520843 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -640,13 +640,89 @@ int multifd_send_sync_main(QEMUFile *f) return 0; } +/* + * Returns true if succeed, false otherwise (with errp set). Caller must + * be with p->mutex held. + */ +static bool multifd_send_prepare(MultiFDSendParams *p, Error **errp) +{ + bool use_zero_copy_send = migrate_zero_copy_send(); + uint64_t packet_num = p->packet_num; + uint32_t flags; + int ret; + + p->normal_num = 0; + + if (use_zero_copy_send) { + p->iovs_num = 0; + } else { + p->iovs_num = 1; + } + + for (int i = 0; i < p->pages->num; i++) { + p->normal[p->normal_num] = p->pages->offset[i]; + p->normal_num++; + } + + if (p->normal_num) { + ret = multifd_send_state->ops->send_prepare(p, errp); + if (ret != 0) { + return false; + } + } + multifd_send_fill_packet(p); + flags = p->flags; + p->flags = 0; + p->num_packets++; + p->total_normal_pages += p->normal_num; + p->pages->num = 0; + p->pages->block = NULL; + + trace_multifd_send(p->id, packet_num, p->normal_num, flags, + p->next_packet_size); + + return true; +} + +/* Returns true if succeed, false otherwise (with errp set) */ +static bool multifd_do_send(MultiFDSendParams *p, Error **errp) +{ + bool use_zero_copy_send = migrate_zero_copy_send(); + int ret; + + if (use_zero_copy_send) { + /* Send header first, without zerocopy */ + ret = qio_channel_write_all(p->c, (void *)p->packet, + p->packet_len, errp); + if (ret != 0) { + return false; + } + } else { + /* Send header using the same writev call */ + p->iov[0].iov_len = p->packet_len; + p->iov[0].iov_base = p->packet; + } + + ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, + 0, p->write_flags, errp); + if (ret != 0) { + return false; + } + + stat64_add(&mig_stats.multifd_bytes, + p->next_packet_size + p->packet_len); + stat64_add(&mig_stats.transferred, + p->next_packet_size + p->packet_len); + p->next_packet_size = 0; + + return true; +} + static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p = opaque; MigrationThread *thread = NULL; Error *local_err = NULL; - int ret = 0; - bool use_zero_copy_send = migrate_zero_copy_send(); thread = migration_threads_add(p->name, qemu_get_thread_id()); @@ -654,9 +730,10 @@ static void *multifd_send_thread(void *opaque) rcu_register_thread(); if (multifd_send_initial_packet(p, &local_err) < 0) { - ret = -1; + assert(local_err); goto out; } + /* initial packet */ p->num_packets = 1; @@ -667,83 +744,38 @@ static void *multifd_send_thread(void *opaque) if (qatomic_read(&multifd_send_state->exiting)) { break; } - qemu_mutex_lock(&p->mutex); + qemu_mutex_lock(&p->mutex); if (p->pending_job) { - uint64_t packet_num = p->packet_num; - uint32_t flags; - p->normal_num = 0; - - if (use_zero_copy_send) { - p->iovs_num = 0; - } else { - p->iovs_num = 1; - } + bool need_sync = p->flags & MULTIFD_FLAG_SYNC; - for (int i = 0; i < p->pages->num; i++) { - p->normal[p->normal_num] = p->pages->offset[i]; - p->normal_num++; + if (!multifd_send_prepare(p, &local_err)) { + assert(local_err); + qemu_mutex_unlock(&p->mutex); + goto out; } - if (p->normal_num) { - ret = multifd_send_state->ops->send_prepare(p, &local_err); - if (ret != 0) { - qemu_mutex_unlock(&p->mutex); - break; - } - } - multifd_send_fill_packet(p); - flags = p->flags; - p->flags = 0; - p->num_packets++; - p->total_normal_pages += p->normal_num; - p->pages->num = 0; - p->pages->block = NULL; + /* Send the packets without mutex */ qemu_mutex_unlock(&p->mutex); - - trace_multifd_send(p->id, packet_num, p->normal_num, flags, - p->next_packet_size); - - if (use_zero_copy_send) { - /* Send header first, without zerocopy */ - ret = qio_channel_write_all(p->c, (void *)p->packet, - p->packet_len, &local_err); - if (ret != 0) { - break; - } - } else { - /* Send header using the same writev call */ - p->iov[0].iov_len = p->packet_len; - p->iov[0].iov_base = p->packet; + if (!multifd_do_send(p, &local_err)) { + assert(local_err); + goto out; } - - ret = qio_channel_writev_full_all(p->c, p->iov, p->iovs_num, NULL, - 0, p->write_flags, &local_err); - if (ret != 0) { - break; - } - - stat64_add(&mig_stats.multifd_bytes, - p->next_packet_size + p->packet_len); - stat64_add(&mig_stats.transferred, - p->next_packet_size + p->packet_len); - p->next_packet_size = 0; qemu_mutex_lock(&p->mutex); + + /* Send successful, mark the task completed */ p->pending_job--; - qemu_mutex_unlock(&p->mutex); - if (flags & MULTIFD_FLAG_SYNC) { + /* If this is a sync task, we need one more kick */ + if (need_sync) { qemu_sem_post(&p->sem_sync); } - } else { - qemu_mutex_unlock(&p->mutex); - /* sometimes there are spurious wakeups */ } + qemu_mutex_unlock(&p->mutex); } out: - if (ret) { - assert(local_err); + if (local_err) { trace_multifd_send_error(p->id); multifd_send_terminate_threads(local_err); multifd_send_kick_main(p); From patchwork Sun Oct 22 20:12:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13432063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 921DECDB474 for ; Sun, 22 Oct 2023 20:14:30 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1queoa-0005Ge-Ul; Sun, 22 Oct 2023 16:12:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoZ-0005Fn-Hb for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoY-00062D-2t for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698005565; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=phapzCWUE8d8Jlkfp2/jJ1KPEdP1a8qLD5w17cHMUKI=; b=C584AgxY1fXLtrjoRMYjUTm9m2VzLmNseS7rehWcKMyd5lGaigjjSWnTCmHXm72vCfmrPb wRe+4WkfzckXf+aU3nd2586BJvxdxsrtFgyz3sEQuD/xQ8R6J5RXtguvy0VRfoqJqgf0SO Oz0rb3h+owcOovwFfbdNvAreh5qpuuY= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-62-NaL2fcVVODaVh8qH70r3Jg-1; Sun, 22 Oct 2023 16:12:41 -0400 X-MC-Unique: NaL2fcVVODaVh8qH70r3Jg-1 Received: by mail-qv1-f70.google.com with SMTP id 6a1803df08f44-66d48cfd295so11084726d6.0 for ; Sun, 22 Oct 2023 13:12:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698005561; x=1698610361; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=phapzCWUE8d8Jlkfp2/jJ1KPEdP1a8qLD5w17cHMUKI=; b=BJlx9DDuF1ybGKU4ocS1HNI0QPasoZuT2XF6THR1u/0aOYIeY+TwBW8hsxi6qmTugq SIcYT0L7MEMjsFryB43XrArZdKbjSmNcPeR02Xsic3mh9uZzEDH4IB6ke1P+Oj/6LsYS jlrzfiqNsalwH/V5zzpjPujCR8iEajJJrPRYrc59RnJPbYZ+T40UzF1X7/jHBejua8nt 8HU3GngAxMoV49jA1GU+nCwU/VBE0+zB3u1ITUtzLLANfHiym/mcLyD8xQuHcRp2CGKZ fw1GYyPdVpgZEoKsS9B90HlsT6GNqZJOhSBk7sli7DHlNffyJNcoBp+eOWjSPwkCzANd svvg== X-Gm-Message-State: AOJu0YytIJYTRirdf3JFO5g2JLdXFX/t5qbci5A9x7dieZWBmBZR9y9x 3+WtqYxmlqw29wgB2ztMzAw6VW5dLreLt66k5eluSjB+GE9p6pcrYhB45BXhxlkpCv462YMdHAs bKSYPQeFUoIG7f+rjq2sB08rfp0ECCaPJazRly/EqJnz0x0P28FZOAbuH2ZuLhc79kxUwOZJ/ X-Received: by 2002:a05:6214:3d8d:b0:66d:1ff9:321f with SMTP id om13-20020a0562143d8d00b0066d1ff9321fmr8206092qvb.6.1698005561010; Sun, 22 Oct 2023 13:12:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG2wtZOCYLDeeiAwgm0HjXbcr2Fl+kmdggWRm/WpeLAJBORKeyRqCbNQN9SV7/x4QFdFwZBPg== X-Received: by 2002:a05:6214:3d8d:b0:66d:1ff9:321f with SMTP id om13-20020a0562143d8d00b0066d1ff9321fmr8206074qvb.6.1698005560601; Sun, 22 Oct 2023 13:12:40 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id x11-20020a0cfe0b000000b0065af657ddf7sm2330515qvr.144.2023.10.22.13.12.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Oct 2023 13:12:40 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , peterx@redhat.com Subject: [PATCH RFC 6/7] migration: Split multifd pending_job into two booleans Date: Sun, 22 Oct 2023 16:12:10 -0400 Message-ID: <20231022201211.452861-7-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231022201211.452861-1-peterx@redhat.com> References: <20231022201211.452861-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Peter Xu Signed-off-by: Peter Xu --- migration/multifd.h | 16 ++++++++++------ migration/multifd.c | 33 +++++++++++++++++++++++---------- 2 files changed, 33 insertions(+), 16 deletions(-) diff --git a/migration/multifd.h b/migration/multifd.h index 2acf400085..ddee7b8d8a 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -101,12 +101,16 @@ typedef struct { uint32_t flags; /* global number of generated multifd packets */ uint64_t packet_num; - /* thread has work to do */ - int pending_job; - /* array of pages to sent. - * The owner of 'pages' depends of 'pending_job' value: - * pending_job == 0 -> migration_thread can use it. - * pending_job != 0 -> multifd_channel can use it. + /* thread has a request to sync all data */ + bool pending_sync; + /* thread has something to send */ + bool pending_job; + /* + * Array of pages to sent. The owner of 'pages' depends of + * 'pending_job' value: + * + * - true -> multifd_channel owns it. + * - false -> migration_thread owns it. */ MultiFDPages_t *pages; diff --git a/migration/multifd.c b/migration/multifd.c index 8140520843..fe8d746ff9 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -425,7 +425,7 @@ static int multifd_send_pages(QEMUFile *f) p = &multifd_send_state->params[i]; qemu_mutex_lock(&p->mutex); if (!p->pending_job) { - p->pending_job++; + p->pending_job = true; next_channel = (i + 1) % migrate_multifd_channels(); break; } @@ -615,8 +615,7 @@ int multifd_send_sync_main(QEMUFile *f) qemu_mutex_lock(&p->mutex); p->packet_num = multifd_send_state->packet_num++; - p->flags |= MULTIFD_FLAG_SYNC; - p->pending_job++; + p->pending_sync = true; qemu_mutex_unlock(&p->mutex); qemu_sem_post(&p->sem); } @@ -747,8 +746,6 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); if (p->pending_job) { - bool need_sync = p->flags & MULTIFD_FLAG_SYNC; - if (!multifd_send_prepare(p, &local_err)) { assert(local_err); qemu_mutex_unlock(&p->mutex); @@ -764,12 +761,27 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); /* Send successful, mark the task completed */ - p->pending_job--; + p->pending_job = false; + + } else if (p->pending_sync) { + p->flags |= MULTIFD_FLAG_SYNC; + + if (!multifd_send_prepare(p, &local_err)) { + assert(local_err); + qemu_mutex_unlock(&p->mutex); + goto out; + } - /* If this is a sync task, we need one more kick */ - if (need_sync) { - qemu_sem_post(&p->sem_sync); + /* Send the packets without mutex */ + qemu_mutex_unlock(&p->mutex); + if (!multifd_do_send(p, &local_err)) { + assert(local_err); + goto out; } + qemu_mutex_lock(&p->mutex); + + qemu_sem_post(&p->sem_sync); + p->pending_sync = false; } qemu_mutex_unlock(&p->mutex); } @@ -932,7 +944,8 @@ int multifd_save_setup(Error **errp) qemu_mutex_init(&p->mutex); qemu_sem_init(&p->sem, 0); qemu_sem_init(&p->sem_sync, 0); - p->pending_job = 0; + p->pending_job = false; + p->pending_sync = false; p->id = i; p->pages = multifd_pages_init(page_count); p->packet_len = sizeof(MultiFDPacket_t) From patchwork Sun Oct 22 20:12:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13432057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0AC2ECDB474 for ; Sun, 22 Oct 2023 20:13:34 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1queoa-0005Gb-CS; Sun, 22 Oct 2023 16:12:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoY-0005FW-Ld for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1queoX-00061t-5I for qemu-devel@nongnu.org; Sun, 22 Oct 2023 16:12:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698005563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QmqhK+JgD9oXmIthMn+SAkCj7IdZVJuTdH9Qh84fJac=; b=XhOhN7xV81Iwm2/VzVQBr0NL+S37b1Zu2Nkz4hksj7xvI+PpVJc9WUxRQasV72Rc4cXVhi nhYphMxAQR8uOe3AnqMSiGQfZNMlIzx5/zir3vjPMkpQ5XzJikteU2QMPJoQHibBGlMhz1 Mtcrw7Fm+QsNqYTOcVHTB5eXGrQ5bQ8= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-682-04pYdnSyMcCyKsJrlRzrlA-1; Sun, 22 Oct 2023 16:12:42 -0400 X-MC-Unique: 04pYdnSyMcCyKsJrlRzrlA-1 Received: by mail-qv1-f69.google.com with SMTP id 6a1803df08f44-66cfa898cfeso7839796d6.0 for ; Sun, 22 Oct 2023 13:12:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698005561; x=1698610361; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QmqhK+JgD9oXmIthMn+SAkCj7IdZVJuTdH9Qh84fJac=; b=JTu1ToRZi+Ga4UtV9fQVTanLxRkONl6EHA1HQjdjgUW6EzHk8LGpf8LKn+cgDLj5yr YqwXOnzwS/6aOXd5a/Fvc6GJ/GJv5DnkVdR9znE5DgH6wpunU7f/G080B14DArS4sOgw Vgalc03LAXeuFtiB9xJNqCCqWI6C+FwZju7RB5jA8Ejo9YK7rC4bWotEV+g07zUGOee7 zdYdocuUas/7Iudla5SMe2CZ73oqTEQ0QLo0DMdwHrPIqWGbWKTAZNl9vjc3W4qkPCyG +08GFlW3IanVHFHVdU1K6HNXq3ZSvIT0/4YFrupPA2uHc6E4z6fZ/efcjAuqllwx8/4x e6FA== X-Gm-Message-State: AOJu0Yw3hmfrP4l9o10Eae6jevoKPRfMRk69JUFmnawfwy5Zj3cVIv4B ecKJuumgLVEgNaw+/YKw8+MlC6wLZvg9H470RfOfGZDI9Er7XNvRXLBNnXenN0N33cTSDNZMdKs BCLy8b3gJA6lsLNasqZZvB+3eindSyzBHDVEUikquen3Lp7WQFSqiOM3PBiQ7eQxm89+JrC2L X-Received: by 2002:ad4:5844:0:b0:66d:169a:a661 with SMTP id de4-20020ad45844000000b0066d169aa661mr7492848qvb.4.1698005561755; Sun, 22 Oct 2023 13:12:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEKhfuWjZQciM3w3sDLaoO6iwhIX9y+OBqCDq0opdA/e5z4SibcHCGsmzgT5ixDH0+Jg5DY9w== X-Received: by 2002:ad4:5844:0:b0:66d:169a:a661 with SMTP id de4-20020ad45844000000b0066d169aa661mr7492837qvb.4.1698005561406; Sun, 22 Oct 2023 13:12:41 -0700 (PDT) Received: from x1n.redhat.com (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id x11-20020a0cfe0b000000b0065af657ddf7sm2330515qvr.144.2023.10.22.13.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Oct 2023 13:12:41 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Juan Quintela , Fabiano Rosas , peterx@redhat.com Subject: [PATCH RFC 7/7] migration: Further unify paths for multifd normal or sync requests Date: Sun, 22 Oct 2023 16:12:11 -0400 Message-ID: <20231022201211.452861-8-peterx@redhat.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231022201211.452861-1-peterx@redhat.com> References: <20231022201211.452861-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Provide multifd_send_execute() for merging duplicated codes. The trick here is multifd_send_execute() will conditionally hold the mutex when returned, depending on the retval. Signed-off-by: Peter Xu --- migration/multifd.c | 51 ++++++++++++++++++++++----------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index fe8d746ff9..0052e5daee 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -717,6 +717,29 @@ static bool multifd_do_send(MultiFDSendParams *p, Error **errp) return true; } +/* + * When succeed: returns true, mutex held. + * When failed: returns false, mutex released. + */ +static bool multifd_send_execute(MultiFDSendParams *p, Error **errp) +{ + if (!multifd_send_prepare(p, errp)) { + qemu_mutex_unlock(&p->mutex); + assert(*errp); + return false; + } + + /* Send the packets without mutex */ + qemu_mutex_unlock(&p->mutex); + if (!multifd_do_send(p, errp)) { + assert(*errp); + return false; + } + qemu_mutex_lock(&p->mutex); + + return true; +} + static void *multifd_send_thread(void *opaque) { MultiFDSendParams *p = opaque; @@ -746,40 +769,16 @@ static void *multifd_send_thread(void *opaque) qemu_mutex_lock(&p->mutex); if (p->pending_job) { - if (!multifd_send_prepare(p, &local_err)) { - assert(local_err); - qemu_mutex_unlock(&p->mutex); + if (!multifd_send_execute(p, &local_err)) { goto out; } - - /* Send the packets without mutex */ - qemu_mutex_unlock(&p->mutex); - if (!multifd_do_send(p, &local_err)) { - assert(local_err); - goto out; - } - qemu_mutex_lock(&p->mutex); - - /* Send successful, mark the task completed */ p->pending_job = false; } else if (p->pending_sync) { p->flags |= MULTIFD_FLAG_SYNC; - - if (!multifd_send_prepare(p, &local_err)) { - assert(local_err); - qemu_mutex_unlock(&p->mutex); - goto out; - } - - /* Send the packets without mutex */ - qemu_mutex_unlock(&p->mutex); - if (!multifd_do_send(p, &local_err)) { - assert(local_err); + if (!multifd_send_execute(p, &local_err)) { goto out; } - qemu_mutex_lock(&p->mutex); - qemu_sem_post(&p->sem_sync); p->pending_sync = false; }