From patchwork Thu Jul 11 21:52:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5713EC3DA45 for ; Thu, 11 Jul 2024 21:53:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1in-0003wk-FQ; Thu, 11 Jul 2024 17:53:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1il-0003up-Jv for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:52:59 -0400 Received: from mail-oa1-x33.google.com ([2001:4860:4864:20::33]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1if-0005Y0-BM for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:52:59 -0400 Received: by mail-oa1-x33.google.com with SMTP id 586e51a60fabf-25982aa59efso596245fac.3 for ; Thu, 11 Jul 2024 14:52:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734771; x=1721339571; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U8KIXwkZXyq8Ux81YoTSAxOBE0pbfYwmygweH+SbG/w=; b=gGUjl4+leOeBis/ziVySpUV/SMGw4kcamfs6ux3toT5fWmjby0bFs7rNoZv8UQ85cq XkTDgdjctGtLSERcQhbuSICh7anplyUtvyYwNMjoQGJCpFdDhgB23UdDY27aLq1lybWr bMsyZHQEyUi+qhP6n/s1qxxoX+kn2PoIK+YJmm+LEGjrpKfyqodZ60N/WftAQAS3q+VP 1XHMwBLZvwSALDL3w3FrdCnDOCEngZzsXxeumW1deCThzBr5b79Sg07Mcg+TTzz5dxy1 tnJcDHivpZ+7lQcsgFafr2pJMnhy9BCcCU+sxqlzWnIKP5ve6vYNlGIJSIqQkj7oSafF 65kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734771; x=1721339571; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U8KIXwkZXyq8Ux81YoTSAxOBE0pbfYwmygweH+SbG/w=; b=PpMDdqcvs6omsFrmSVCMeKaPtklf2gfJuws0101XnSsLQHPlPzS73thNGih3C/6fdb uhW7STHLAwAcgfPaxHTH8q3JBJEm57ktVrwOKTtD8XmutfsxcPSrPpKNuW6ZWRt6jYDN RUacqnQ0IJpkXT4AFcF9KOmGJ1YcU9lzhZd4iNx7c4q6P2Y3qThV/EGbnQImzt2qukuo +Zk+X3K4MaTzRMhonbdgDEGkWPEF1XgWVW2+yEmUTwmdu0lrijykK7Zf6tiX8Hg7Vpf+ lkExkdy4T5A+SwY8p6/FqW3L3Iskene4rJ0sQhj9UMfWOLfEnwJDChmXrUmUyNSi96bv 1FCA== X-Forwarded-Encrypted: i=1; AJvYcCUgH5HO5h92LhNWR1wfE8I5+VqUZkoPoxofUL2w79pgVImYdsy97Z1BSMnOjA4dr/mOCB3gPuogII6LqdzpkaKeUMOVzjg= X-Gm-Message-State: AOJu0Ywad6I7/7xX8MLvU/U+pM/DVKgYMbiF/ZBo44FZynsVoknsiP8m 1qKntnMdjtQ7CjX1nZr3jsqS0kuywke0eDH5vm3UuL6oo1hOH1IiPNFG6mBEC7k= X-Google-Smtp-Source: AGHT+IHXS4KqfD5dL20PDI1NMSzo5ZKEp2wXV7Vp30XmmPnRS57Fkyyq7shLr2biZJnSZliW0DjPiQ== X-Received: by 2002:a05:6871:88d:b0:25e:bdf7:7e08 with SMTP id 586e51a60fabf-25ebdf84150mr7104927fac.0.1720734771027; Thu, 11 Jul 2024 14:52:51 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.52.49 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:52:50 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 01/13] meson: Introduce new instruction set enqcmd to the build system. Date: Thu, 11 Jul 2024 14:52:36 -0700 Message-Id: <20240711215244.19237-2-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::33; envelope-from=yichen.wang@bytedance.com; helo=mail-oa1-x33.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Enable instruction set enqcmd in build. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- meson.build | 14 ++++++++++++++ meson_options.txt | 2 ++ scripts/meson-buildoptions.sh | 3 +++ 3 files changed, 19 insertions(+) diff --git a/meson.build b/meson.build index 6a93da48e1..af650cfabf 100644 --- a/meson.build +++ b/meson.build @@ -2893,6 +2893,20 @@ config_host_data.set('CONFIG_AVX512BW_OPT', get_option('avx512bw') \ int main(int argc, char *argv[]) { return bar(argv[0]); } '''), error_message: 'AVX512BW not available').allowed()) +config_host_data.set('CONFIG_DSA_OPT', get_option('enqcmd') \ + .require(have_cpuid_h, error_message: 'cpuid.h not available, cannot enable ENQCMD') \ + .require(cc.links(''' + #include + #include + #include + static int __attribute__((target("enqcmd"))) bar(void *a) { + uint64_t dst[8] = { 0 }; + uint64_t src[8] = { 0 }; + return _enqcmd(dst, src); + } + int main(int argc, char *argv[]) { return bar(argv[argc - 1]); } + '''), error_message: 'ENQCMD not available').allowed()) + # For both AArch64 and AArch32, detect if builtins are available. config_host_data.set('CONFIG_ARM_AES_BUILTIN', cc.compiles(''' #include diff --git a/meson_options.txt b/meson_options.txt index 0269fa0f16..4ed820bb8d 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -121,6 +121,8 @@ option('avx2', type: 'feature', value: 'auto', description: 'AVX2 optimizations') option('avx512bw', type: 'feature', value: 'auto', description: 'AVX512BW optimizations') +option('enqcmd', type: 'feature', value: 'disabled', + description: 'ENQCMD optimizations') option('keyring', type: 'feature', value: 'auto', description: 'Linux keyring support') option('libkeyutils', type: 'feature', value: 'auto', diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index cfadb5ea86..280e117687 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -95,6 +95,7 @@ meson_options_help() { printf "%s\n" ' auth-pam PAM access control' printf "%s\n" ' avx2 AVX2 optimizations' printf "%s\n" ' avx512bw AVX512BW optimizations' + printf "%s\n" ' enqcmd ENQCMD optimizations' printf "%s\n" ' blkio libblkio block device driver' printf "%s\n" ' bochs bochs image format support' printf "%s\n" ' bpf eBPF support' @@ -239,6 +240,8 @@ _meson_option_parse() { --disable-avx2) printf "%s" -Davx2=disabled ;; --enable-avx512bw) printf "%s" -Davx512bw=enabled ;; --disable-avx512bw) printf "%s" -Davx512bw=disabled ;; + --enable-enqcmd) printf "%s" -Denqcmd=enabled ;; + --disable-enqcmd) printf "%s" -Denqcmd=disabled ;; --enable-gcov) printf "%s" -Db_coverage=true ;; --disable-gcov) printf "%s" -Db_coverage=false ;; --enable-lto) printf "%s" -Db_lto=true ;; From patchwork Thu Jul 11 21:52:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25947C3DA45 for ; Thu, 11 Jul 2024 21:54:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1io-00040f-OH; Thu, 11 Jul 2024 17:53:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1im-0003w6-HE for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:00 -0400 Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1ig-0005YU-7v for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:00 -0400 Received: by mail-ot1-x32f.google.com with SMTP id 46e09a7af769-703775e4d5dso678051a34.0 for ; Thu, 11 Jul 2024 14:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734773; x=1721339573; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FOMeFKnXZZxuQzQ4Ug4ts6vhOgDMsElUKC3GROOqZJI=; b=FTLt1zdLF5Tg11PWxTq+7mQ2KaRGtDh6MjPwsC8S9t5rScb+rbX18ccykE6+x9XVFI QfXR9fGCE7gXzjdhp7/nS9xft+zAUB9G6zTYLJY8SNojW+zY0h+NIm0d8qRhoYqQp06P O4QcMz6FuKt1un3bkg9OyBE6sBbzevjgoZOdv8WBE/TgI1S5tXn3FCh0OH9AuPXiuWmS hbKiI3iNFKnXQi7FuyN3WbgCIAVZ+px8rpRSiZ32CifllVoGZQ4wlImauUj38zBA72Mf CmDPWpllqwQ70fWPVjlnVM+LPfCOEAuzqCFm6nMjhNwWGGcNcHT33PhS9YAPODdL/lWK T5rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734773; x=1721339573; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FOMeFKnXZZxuQzQ4Ug4ts6vhOgDMsElUKC3GROOqZJI=; b=AzZ2jFwLRtuCEIMb9kapMw6MNqUKQuMfqTav5vdCvZr6sXERbNr9px2ibzatdj95kV oqZVXvaAiaMf17YhdUDxFBazY9iKLDQ8nYmpxyOj8La0pAkVauPG1pl2FUbIjXCBIvym fQik7K8C4A8iYXzRYO4mqSuY0RpuEDfcsMeKIiRxFpk7IvCI0+pPyN3ooRGiUqux5zE5 dcomqqsrL/I9ay7bCOijW6yL+7yLQv7aswVMKgws4OBpvspBw9tujD3adKMb1cs3Xh8O PrpFm98UpKvbAv93S9dK0Eg/mbCEztUleiDWDlDf+WBjYeS8U2L204bHXxrsZf3PqfgZ Wv6w== X-Forwarded-Encrypted: i=1; AJvYcCWH9AURWZtJE1vlE28r+e5ii1A85C/w6zIdt9sGfgt41ndHmYS7gPHBSndgB9pAnJhHy4dJaEzGsuqGckXyaG68FUrtj84= X-Gm-Message-State: AOJu0YyGs2EWajVdhhgCktuuRZThe34uFghTsLHB0fHx2J5TIzHZc9Ow wZGOkfKjcM54mAkUxoJP3W+f4Q355RO8akqJ0GPc5T9n4BMff8nXQ3dhQ0HyiBA= X-Google-Smtp-Source: AGHT+IHzFlqKU2qpZF18tXY6Jqt2whh20atIX/FwpcZMEpSuSAolTOEUtWHlyfT6Omp8BCtePHdHsw== X-Received: by 2002:a9d:4819:0:b0:708:b286:f6f1 with SMTP id 46e09a7af769-708b286fcefmr2768816a34.13.1720734773065; Thu, 11 Jul 2024 14:52:53 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.52.51 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:52:52 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 02/13] util/dsa: Add idxd into linux header copy list. Date: Thu, 11 Jul 2024 14:52:37 -0700 Message-Id: <20240711215244.19237-3-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32f; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Yichen Wang --- scripts/update-linux-headers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh index c34ac6454e..5aba95d9cb 100755 --- a/scripts/update-linux-headers.sh +++ b/scripts/update-linux-headers.sh @@ -193,7 +193,7 @@ rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" for header in const.h stddef.h kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \ psci.h psp-sev.h userfaultfd.h memfd.h mman.h nvme_ioctl.h \ - vduse.h iommufd.h bits.h; do + vduse.h iommufd.h bits.h idxd.h; do cp "$hdrdir/include/linux/$header" "$output/linux-headers/linux" done From patchwork Thu Jul 11 21:52:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731094 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5BF39C3DA4A for ; Thu, 11 Jul 2024 21:53:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1ip-00046I-Te; Thu, 11 Jul 2024 17:53:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1in-0003wv-2X for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:01 -0400 Received: from mail-ot1-x336.google.com ([2607:f8b0:4864:20::336]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1ij-0005Z2-5N for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:00 -0400 Received: by mail-ot1-x336.google.com with SMTP id 46e09a7af769-706dea920f3so708276a34.1 for ; Thu, 11 Jul 2024 14:52:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734775; x=1721339575; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g36O2jlhRF6JWB+BcC9DsTu4cbuPShYgI3NtvZMuOF4=; b=cIdvR/IP9afshpESdtFB361N+tU7JJaPPhpLjjbT+bAf6kONPG+wuEf+rtKmNE3Ikp PnpqrcL89z2M4e7DQL24gs+4m76fUNcmnrKG2u8Tt2SKMGa0Bwc22v6B1J1YBHjX8NRW 6Iv//AkGl+MnkxPov88Xee+sL8pzQX7vqeGIKi0GjgfJ8iBcmI139fXxb0Jf6ofa1MB8 62XzTVkFeaY0r6SglSmQ4uO7OG23wiNXW9sqQotSFFWEsxAAVU7A8ZON5rJgtADvbUj/ g8YJdvAsZydQVHYcrrzAGzXXtrV1yRh4KzxSozX6A4Mdo7eqNYlETyT87Wvfx/XPzjou K+gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734775; x=1721339575; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g36O2jlhRF6JWB+BcC9DsTu4cbuPShYgI3NtvZMuOF4=; b=AgNJR8IMBwbOpCI1llfxYxbHh40C491uq8YR/Lpca+T9Nqg+z4NBOsd8y4j9jczd3S OETxTprL3El3YohlB+qTqaeAOb6723D8ILlr5x8HtjMyL/8/CGK0uKyN5ytnKwz1jr0S 9zsglYk5bxZtMdIBisws4EAUQa3G9R5Tm31ZbzPR/DOy+C19OqykOZ3ZHZFyeBrVC/Xp aywUi1rCmORZImoY2Pbj6/KDEzdBPYbK2He9Oqf1tVM+pBCuGC86NMA3ToiDZRMXahaD g+8nOlPQ9P5MGqiYPNNtbYZAf5FAaoNbrr0uojOD0v1IOIfD0VaJbMu6/kJ1Af6nU42H 78cg== X-Forwarded-Encrypted: i=1; AJvYcCUx3lU47QjTdXW5plxQgkdWS4OzFqD+tvhxQEiDYqhjEbnS+/L6niCWnoHRt1MdXLWSOyGLG7bSNjZhMmj1dAH5UwFa240= X-Gm-Message-State: AOJu0Yxr44OKOyJW18ZWLYPSI27Y/f6VZYfGQsdMz+9pOek6h6tYOoz6 HC8mbB9tU+wH47EvNFUtWq77049sEx5QEW7q5tfiPLHsidqUcHbRddWtkMvF56Gg5sjQPpI8c4a i2uc= X-Google-Smtp-Source: AGHT+IGPleItq9Gp5o2Ob4TLwKEyNHgk3mWYcyTg6L9hpiKMvWNil1MHaqSsKviTXe93LGLKGL6UiA== X-Received: by 2002:a05:6830:148:b0:704:4bb7:c99a with SMTP id 46e09a7af769-708b521adedmr523021a34.0.1720734775201; Thu, 11 Jul 2024 14:52:55 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.52.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:52:54 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v5 03/13] util/dsa: Implement DSA device start and stop logic. Date: Thu, 11 Jul 2024 14:52:38 -0700 Message-Id: <20240711215244.19237-4-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::336; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x336.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * DSA device open and close. * DSA group contains multiple DSA devices. * DSA group configure/start/stop/clean. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 86 +++++++++++++ util/dsa.c | 303 +++++++++++++++++++++++++++++++++++++++++++++ util/meson.build | 3 + 3 files changed, 392 insertions(+) create mode 100644 include/qemu/dsa.h create mode 100644 util/dsa.c diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h new file mode 100644 index 0000000000..29b60654e9 --- /dev/null +++ b/include/qemu/dsa.h @@ -0,0 +1,86 @@ +/* + * Interface for using Intel Data Streaming Accelerator to offload certain + * background operations. + * + * Copyright (C) Bytedance Ltd. + * + * Authors: + * Hao Xiang + * Yichen Wang + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_DSA_H +#define QEMU_DSA_H + +#include "qemu/error-report.h" +#include "qemu/thread.h" +#include "qemu/queue.h" + +#ifdef CONFIG_DSA_OPT + +#pragma GCC push_options +#pragma GCC target("enqcmd") + +#include +#include "x86intrin.h" + +/** + * @brief Initializes DSA devices. + * + * @param dsa_parameter A list of DSA device path from migration parameter. + * + * @return int Zero if successful, otherwise non zero. + */ +int qemu_dsa_init(const char *dsa_parameter, Error **errp); + +/** + * @brief Start logic to enable using DSA. + */ +void qemu_dsa_start(void); + +/** + * @brief Stop the device group and the completion thread. + */ +void qemu_dsa_stop(void); + +/** + * @brief Clean up system resources created for DSA offloading. + */ +void qemu_dsa_cleanup(void); + +/** + * @brief Check if DSA is running. + * + * @return True if DSA is running, otherwise false. + */ +bool qemu_dsa_is_running(void); + +#else + +static inline bool qemu_dsa_is_running(void) +{ + return false; +} + +static inline int qemu_dsa_init(const char *dsa_parameter, Error **errp) +{ + if (dsa_parameter != NULL && strlen(dsa_parameter) != 0) { + error_setg(errp, "DSA is not supported."); + return -1; + } + + return 0; +} + +static inline void qemu_dsa_start(void) {} + +static inline void qemu_dsa_stop(void) {} + +static inline void qemu_dsa_cleanup(void) {} + +#endif + +#endif diff --git a/util/dsa.c b/util/dsa.c new file mode 100644 index 0000000000..cdb0b9dda2 --- /dev/null +++ b/util/dsa.c @@ -0,0 +1,303 @@ +/* + * Use Intel Data Streaming Accelerator to offload certain background + * operations. + * + * Copyright (C) Bytedance Ltd. + * + * Authors: + * Hao Xiang + * Bryan Zhang + * Yichen Wang + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/queue.h" +#include "qemu/memalign.h" +#include "qemu/lockable.h" +#include "qemu/cutils.h" +#include "qemu/dsa.h" +#include "qemu/bswap.h" +#include "qemu/error-report.h" +#include "qemu/rcu.h" + +#pragma GCC push_options +#pragma GCC target("enqcmd") + +#include +#include "x86intrin.h" + +#define DSA_WQ_SIZE 4096 +#define MAX_DSA_DEVICES 16 + +typedef QSIMPLEQ_HEAD(dsa_task_queue, dsa_batch_task) dsa_task_queue; + +typedef struct { + void *work_queue; +} QemuDsaDevice; + +typedef struct { + QemuDsaDevice *dsa_devices; + int num_dsa_devices; + /* The index of the next DSA device to be used. */ + uint32_t device_allocator_index; + bool running; + QemuMutex task_queue_lock; + QemuCond task_queue_cond; + dsa_task_queue task_queue; +} QemuDsaDeviceGroup; + +uint64_t max_retry_count; +static QemuDsaDeviceGroup dsa_group; + + +/** + * @brief This function opens a DSA device's work queue and + * maps the DSA device memory into the current process. + * + * @param dsa_wq_path A pointer to the DSA device work queue's file path. + * @return A pointer to the mapped memory, or MAP_FAILED on failure. + */ +static void * +map_dsa_device(const char *dsa_wq_path) +{ + void *dsa_device; + int fd; + + fd = open(dsa_wq_path, O_RDWR); + if (fd < 0) { + error_report("Open %s failed with errno = %d.", + dsa_wq_path, errno); + return MAP_FAILED; + } + dsa_device = mmap(NULL, DSA_WQ_SIZE, PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, 0); + close(fd); + if (dsa_device == MAP_FAILED) { + error_report("mmap failed with errno = %d.", errno); + return MAP_FAILED; + } + return dsa_device; +} + +/** + * @brief Initializes a DSA device structure. + * + * @param instance A pointer to the DSA device. + * @param work_queue A pointer to the DSA work queue. + */ +static void +dsa_device_init(QemuDsaDevice *instance, + void *dsa_work_queue) +{ + instance->work_queue = dsa_work_queue; +} + +/** + * @brief Cleans up a DSA device structure. + * + * @param instance A pointer to the DSA device to cleanup. + */ +static void +dsa_device_cleanup(QemuDsaDevice *instance) +{ + if (instance->work_queue != MAP_FAILED) { + munmap(instance->work_queue, DSA_WQ_SIZE); + } +} + +/** + * @brief Initializes a DSA device group. + * + * @param group A pointer to the DSA device group. + * @param dsa_parameter A list of DSA device path from are separated by space + * character migration parameter. Multiple DSA device path. + * + * @return Zero if successful, non-zero otherwise. + */ +static int +dsa_device_group_init(QemuDsaDeviceGroup *group, + const char *dsa_parameter, + Error **errp) +{ + if (dsa_parameter == NULL || strlen(dsa_parameter) == 0) { + return 0; + } + + int ret = 0; + char *local_dsa_parameter = g_strdup(dsa_parameter); + const char *dsa_path[MAX_DSA_DEVICES]; + int num_dsa_devices = 0; + char delim[2] = " "; + + char *current_dsa_path = strtok(local_dsa_parameter, delim); + + while (current_dsa_path != NULL) { + dsa_path[num_dsa_devices++] = current_dsa_path; + if (num_dsa_devices == MAX_DSA_DEVICES) { + break; + } + current_dsa_path = strtok(NULL, delim); + } + + group->dsa_devices = + g_new0(QemuDsaDevice, num_dsa_devices); + group->num_dsa_devices = num_dsa_devices; + group->device_allocator_index = 0; + + group->running = false; + qemu_mutex_init(&group->task_queue_lock); + qemu_cond_init(&group->task_queue_cond); + QSIMPLEQ_INIT(&group->task_queue); + + void *dsa_wq = MAP_FAILED; + for (int i = 0; i < num_dsa_devices; i++) { + dsa_wq = map_dsa_device(dsa_path[i]); + if (dsa_wq == MAP_FAILED) { + error_setg(errp, "map_dsa_device failed MAP_FAILED."); + ret = -1; + goto exit; + } + dsa_device_init(&dsa_group.dsa_devices[i], dsa_wq); + } + +exit: + g_free(local_dsa_parameter); + return ret; +} + +/** + * @brief Starts a DSA device group. + * + * @param group A pointer to the DSA device group. + */ +static void +dsa_device_group_start(QemuDsaDeviceGroup *group) +{ + group->running = true; +} + +/** + * @brief Stops a DSA device group. + * + * @param group A pointer to the DSA device group. + */ +__attribute__((unused)) +static void +dsa_device_group_stop(QemuDsaDeviceGroup *group) +{ + group->running = false; +} + +/** + * @brief Cleans up a DSA device group. + * + * @param group A pointer to the DSA device group. + */ +static void +dsa_device_group_cleanup(QemuDsaDeviceGroup *group) +{ + if (!group->dsa_devices) { + return; + } + for (int i = 0; i < group->num_dsa_devices; i++) { + dsa_device_cleanup(&group->dsa_devices[i]); + } + g_free(group->dsa_devices); + group->dsa_devices = NULL; + + qemu_mutex_destroy(&group->task_queue_lock); + qemu_cond_destroy(&group->task_queue_cond); +} + +/** + * @brief Returns the next available DSA device in the group. + * + * @param group A pointer to the DSA device group. + * + * @return struct QemuDsaDevice* A pointer to the next available DSA device + * in the group. + */ +__attribute__((unused)) +static QemuDsaDevice * +dsa_device_group_get_next_device(QemuDsaDeviceGroup *group) +{ + if (group->num_dsa_devices == 0) { + return NULL; + } + uint32_t current = qatomic_fetch_inc(&group->device_allocator_index); + current %= group->num_dsa_devices; + return &group->dsa_devices[current]; +} + +/** + * @brief Check if DSA is running. + * + * @return True if DSA is running, otherwise false. + */ +bool qemu_dsa_is_running(void) +{ + return false; +} + +static void +dsa_globals_init(void) +{ + max_retry_count = UINT64_MAX; +} + +/** + * @brief Initializes DSA devices. + * + * @param dsa_parameter A list of DSA device path from migration parameter. + * + * @return int Zero if successful, otherwise non zero. + */ +int qemu_dsa_init(const char *dsa_parameter, Error **errp) +{ + dsa_globals_init(); + + return dsa_device_group_init(&dsa_group, dsa_parameter, errp); +} + +/** + * @brief Start logic to enable using DSA. + * + */ +void qemu_dsa_start(void) +{ + if (dsa_group.num_dsa_devices == 0) { + return; + } + if (dsa_group.running) { + return; + } + dsa_device_group_start(&dsa_group); +} + +/** + * @brief Stop the device group and the completion thread. + * + */ +void qemu_dsa_stop(void) +{ + QemuDsaDeviceGroup *group = &dsa_group; + + if (!group->running) { + return; + } +} + +/** + * @brief Clean up system resources created for DSA offloading. + * + */ +void qemu_dsa_cleanup(void) +{ + qemu_dsa_stop(); + dsa_device_group_cleanup(&dsa_group); +} + diff --git a/util/meson.build b/util/meson.build index 5d8bef9891..3360f62923 100644 --- a/util/meson.build +++ b/util/meson.build @@ -88,6 +88,9 @@ if have_block or have_ga endif if have_block util_ss.add(files('aio-wait.c')) + if config_host_data.get('CONFIG_DSA_OPT') + util_ss.add(files('dsa.c')) + endif util_ss.add(files('buffer.c')) util_ss.add(files('bufferiszero.c')) util_ss.add(files('hbitmap.c')) From patchwork Thu Jul 11 21:52:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731100 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45701C3DA4A for ; Thu, 11 Jul 2024 21:54:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1ip-00043i-8F; Thu, 11 Jul 2024 17:53:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1in-0003xT-EC for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:01 -0400 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1ik-0005ZB-HJ for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:01 -0400 Received: by mail-ot1-x331.google.com with SMTP id 46e09a7af769-7036815bde7so764076a34.3 for ; Thu, 11 Jul 2024 14:52:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734777; x=1721339577; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ezdtSRzbDM4mbxnqXDkQjV62eVN37sRjYFDSf1h9wlc=; b=ITayidxDmvs9yIeGheqt/dc5kfrpRvL0mrTrI46/vlA2lmAt7krTWW2gICzgQBxGum 9D8Ifd6qjKZ1LRhfxqV/K/m9mreV/u7gtLKggg5woVly2m+QymT36xg2a6QrFtHsx8wM eYB4WoSHMBJMa9pv6sU+QJiFX9FKZ7PyssTNobfrK+ZIygIzM5ELd4Vb48uXVPkZeSBq DsQQbMCpQ5ZuFSZJiAebts8mwilC6u41GQt4v7LRkCgzd1rQJ2ks3QCdxKcDbCN0wjQq xtlhLE+RklzW8Rial685oE+9DPYrBX5twSQqliyxqMJXjdJL7Bj4zPC4ihY+IvN5wVLU ORgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734777; x=1721339577; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ezdtSRzbDM4mbxnqXDkQjV62eVN37sRjYFDSf1h9wlc=; b=OjeTxkxIkgA2PSWGGK03Kmc4tFAdLMiA7ofhj6zUerJYAZjGEYZDWU0sf/vIIv0SaB km7Xoed9oR6f4qMrHWG1nEmcc8tIJSB5acS+3M+YWCnyD8TXpUev55rXbq7Qj67SzCXp bcJzMKkteZt+y6/cCB7xTR9WiLX/LOZmXjYAcn7FExecwpZHKDZ6jS9GFHt5gmJluNug PCI9CenAz/JT8I/V/UaPSGWC1EssOGHiIa9jxK/Kk4lbDPVUxaSzGCLfiW2ZxpragLJs zJ9Nl1eYsLA02mbtkXT7O+4yMQcsWWuMJ30EFf6rbY28Y1AbdDxfoYw+GfJaE+v0sVma vw6A== X-Forwarded-Encrypted: i=1; AJvYcCVg0m1MLqCPTr4wIMDaDik+JJtzYWNZ7MTbyZsrMOrfDUdrWehjiecNvUzP2t6JYEdQE116qqAsyLNIoMM9tPKNbKKpYt4= X-Gm-Message-State: AOJu0YyEW0wRj3QTmSjLb8klwlsLW/pr7H97inAFkEK3IqgsSbJlSiJB 0ce5GwRViJAjOYUSfk3zdIHpcGaD1dMjm9tcPj6kRrhdzgUs+jxsmdPP8Rcsip8= X-Google-Smtp-Source: AGHT+IGXbu5g32bH53t05aCGfesH7madKXiU/qxxZfX8BjX6ygJ5Ghow2wt31583z9NLFoxKjB8+Gg== X-Received: by 2002:a05:6830:cb:b0:703:77bd:9522 with SMTP id 46e09a7af769-70377bd9d30mr10668090a34.17.1720734777343; Thu, 11 Jul 2024 14:52:57 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.52.55 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:52:57 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 04/13] util/dsa: Implement DSA task enqueue and dequeue. Date: Thu, 11 Jul 2024 14:52:39 -0700 Message-Id: <20240711215244.19237-5-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Use a safe thread queue for DSA task enqueue/dequeue. * Implement DSA task submission. * Implement DSA batch task submission. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 46 ++++++++++ util/dsa.c | 222 ++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 246 insertions(+), 22 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 29b60654e9..9cc836b64c 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -27,6 +27,52 @@ #include #include "x86intrin.h" +typedef enum QemuDsaTaskType { + QEMU_DSA_TASK = 0, + QEMU_DSA_BATCH_TASK +} QemuDsaTaskType; + +typedef enum QemuDsaTaskStatus { + QEMU_DSA_TASK_READY = 0, + QEMU_DSA_TASK_PROCESSING, + QEMU_DSA_TASK_COMPLETION +} QemuDsaTaskStatus; + +typedef struct { + void *work_queue; +} QemuDsaDevice; + +typedef QSIMPLEQ_HEAD(QemuDsaTaskQueue, QemuDsaBatchTask) QemuDsaTaskQueue; + +typedef struct { + QemuDsaDevice *dsa_devices; + int num_dsa_devices; + /* The index of the next DSA device to be used. */ + uint32_t device_allocator_index; + bool running; + QemuMutex task_queue_lock; + QemuCond task_queue_cond; + QemuDsaTaskQueue task_queue; +} QemuDsaDeviceGroup; + +typedef void (*qemu_dsa_completion_fn)(void *); + +typedef struct QemuDsaBatchTask { + struct dsa_hw_desc batch_descriptor; + struct dsa_hw_desc *descriptors; + struct dsa_completion_record batch_completion __attribute__((aligned(32))); + struct dsa_completion_record *completions; + QemuDsaDeviceGroup *group; + QemuDsaDevice *device; + qemu_dsa_completion_fn completion_callback; + QemuSemaphore sem_task_complete; + QemuDsaTaskType task_type; + QemuDsaTaskStatus status; + int batch_size; + QSIMPLEQ_ENTRY(QemuDsaBatchTask) entry; +} QemuDsaBatchTask; + + /** * @brief Initializes DSA devices. * diff --git a/util/dsa.c b/util/dsa.c index cdb0b9dda2..43280b31cd 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -30,27 +30,11 @@ #include #include "x86intrin.h" -#define DSA_WQ_SIZE 4096 +#define DSA_WQ_PORTAL_SIZE 4096 +#define DSA_WQ_DEPTH 128 #define MAX_DSA_DEVICES 16 -typedef QSIMPLEQ_HEAD(dsa_task_queue, dsa_batch_task) dsa_task_queue; - -typedef struct { - void *work_queue; -} QemuDsaDevice; - -typedef struct { - QemuDsaDevice *dsa_devices; - int num_dsa_devices; - /* The index of the next DSA device to be used. */ - uint32_t device_allocator_index; - bool running; - QemuMutex task_queue_lock; - QemuCond task_queue_cond; - dsa_task_queue task_queue; -} QemuDsaDeviceGroup; - -uint64_t max_retry_count; +uint32_t max_retry_count; static QemuDsaDeviceGroup dsa_group; @@ -73,7 +57,7 @@ map_dsa_device(const char *dsa_wq_path) dsa_wq_path, errno); return MAP_FAILED; } - dsa_device = mmap(NULL, DSA_WQ_SIZE, PROT_WRITE, + dsa_device = mmap(NULL, DSA_WQ_PORTAL_SIZE, PROT_WRITE, MAP_SHARED | MAP_POPULATE, fd, 0); close(fd); if (dsa_device == MAP_FAILED) { @@ -105,7 +89,7 @@ static void dsa_device_cleanup(QemuDsaDevice *instance) { if (instance->work_queue != MAP_FAILED) { - munmap(instance->work_queue, DSA_WQ_SIZE); + munmap(instance->work_queue, DSA_WQ_PORTAL_SIZE); } } @@ -233,6 +217,198 @@ dsa_device_group_get_next_device(QemuDsaDeviceGroup *group) return &group->dsa_devices[current]; } +/** + * @brief Empties out the DSA task queue. + * + * @param group A pointer to the DSA device group. + */ +static void +dsa_empty_task_queue(QemuDsaDeviceGroup *group) +{ + qemu_mutex_lock(&group->task_queue_lock); + QemuDsaTaskQueue *task_queue = &group->task_queue; + while (!QSIMPLEQ_EMPTY(task_queue)) { + QSIMPLEQ_REMOVE_HEAD(task_queue, entry); + } + qemu_mutex_unlock(&group->task_queue_lock); +} + +/** + * @brief Adds a task to the DSA task queue. + * + * @param group A pointer to the DSA device group. + * @param task A pointer to the DSA task to enqueue. + * + * @return int Zero if successful, otherwise a proper error code. + */ +static int +dsa_task_enqueue(QemuDsaDeviceGroup *group, + QemuDsaBatchTask *task) +{ + bool notify = false; + + qemu_mutex_lock(&group->task_queue_lock); + + if (!group->running) { + error_report("DSA: Tried to queue task to stopped device queue."); + qemu_mutex_unlock(&group->task_queue_lock); + return -1; + } + + /* The queue is empty. This enqueue operation is a 0->1 transition. */ + if (QSIMPLEQ_EMPTY(&group->task_queue)) { + notify = true; + } + + QSIMPLEQ_INSERT_TAIL(&group->task_queue, task, entry); + + /* We need to notify the waiter for 0->1 transitions. */ + if (notify) { + qemu_cond_signal(&group->task_queue_cond); + } + + qemu_mutex_unlock(&group->task_queue_lock); + + return 0; +} + +/** + * @brief Takes a DSA task out of the task queue. + * + * @param group A pointer to the DSA device group. + * @return QemuDsaBatchTask* The DSA task being dequeued. + */ +__attribute__((unused)) +static QemuDsaBatchTask * +dsa_task_dequeue(QemuDsaDeviceGroup *group) +{ + QemuDsaBatchTask *task = NULL; + + qemu_mutex_lock(&group->task_queue_lock); + + while (true) { + if (!group->running) { + goto exit; + } + task = QSIMPLEQ_FIRST(&group->task_queue); + if (task != NULL) { + break; + } + qemu_cond_wait(&group->task_queue_cond, &group->task_queue_lock); + } + + QSIMPLEQ_REMOVE_HEAD(&group->task_queue, entry); + +exit: + qemu_mutex_unlock(&group->task_queue_lock); + return task; +} + +/** + * @brief Submits a DSA work item to the device work queue. + * + * @param wq A pointer to the DSA work queue's device memory. + * @param descriptor A pointer to the DSA work item descriptor. + * + * @return Zero if successful, non-zero otherwise. + */ +static int +submit_wi_int(void *wq, struct dsa_hw_desc *descriptor) +{ + uint32_t retry = 0; + + _mm_sfence(); + + while (true) { + if (_enqcmd(wq, descriptor) == 0) { + break; + } + retry++; + if (retry > max_retry_count) { + error_report("Submit work retry %u times.", retry); + return -1; + } + } + + return 0; +} + +/** + * @brief Synchronously submits a DSA work item to the + * device work queue. + * + * @param wq A pointer to the DSA work queue's device memory. + * @param descriptor A pointer to the DSA work item descriptor. + * + * @return int Zero if successful, non-zero otherwise. + */ +__attribute__((unused)) +static int +submit_wi(void *wq, struct dsa_hw_desc *descriptor) +{ + return submit_wi_int(wq, descriptor); +} + +/** + * @brief Asynchronously submits a DSA work item to the + * device work queue. + * + * @param task A pointer to the task. + * + * @return int Zero if successful, non-zero otherwise. + */ +__attribute__((unused)) +static int +submit_wi_async(QemuDsaBatchTask *task) +{ + QemuDsaDeviceGroup *device_group = task->group; + QemuDsaDevice *device_instance = task->device; + int ret; + + assert(task->task_type == QEMU_DSA_TASK); + + task->status = QEMU_DSA_TASK_PROCESSING; + + ret = submit_wi_int(device_instance->work_queue, + &task->descriptors[0]); + if (ret != 0) { + return ret; + } + + return dsa_task_enqueue(device_group, task); +} + +/** + * @brief Asynchronously submits a DSA batch work item to the + * device work queue. + * + * @param batch_task A pointer to the batch task. + * + * @return int Zero if successful, non-zero otherwise. + */ +__attribute__((unused)) +static int +submit_batch_wi_async(QemuDsaBatchTask *batch_task) +{ + QemuDsaDeviceGroup *device_group = batch_task->group; + QemuDsaDevice *device_instance = batch_task->device; + int ret; + + assert(batch_task->task_type == QEMU_DSA_BATCH_TASK); + assert(batch_task->batch_descriptor.desc_count <= batch_task->batch_size); + assert(batch_task->status == QEMU_DSA_TASK_READY); + + batch_task->status = QEMU_DSA_TASK_PROCESSING; + + ret = submit_wi_int(device_instance->work_queue, + &batch_task->batch_descriptor); + if (ret != 0) { + return ret; + } + + return dsa_task_enqueue(device_group, batch_task); +} + /** * @brief Check if DSA is running. * @@ -246,7 +422,7 @@ bool qemu_dsa_is_running(void) static void dsa_globals_init(void) { - max_retry_count = UINT64_MAX; + max_retry_count = DSA_WQ_DEPTH; } /** @@ -289,6 +465,8 @@ void qemu_dsa_stop(void) if (!group->running) { return; } + + dsa_empty_task_queue(group); } /** From patchwork Thu Jul 11 21:52:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2DEFC3DA4A for ; Thu, 11 Jul 2024 21:54:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1is-0004Ek-4p; Thu, 11 Jul 2024 17:53:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1iq-0004Ag-WD for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:05 -0400 Received: from mail-ot1-x332.google.com ([2607:f8b0:4864:20::332]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1io-0005Zh-Mp for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:04 -0400 Received: by mail-ot1-x332.google.com with SMTP id 46e09a7af769-70368b53cc0so662244a34.1 for ; Thu, 11 Jul 2024 14:53:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734780; x=1721339580; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lAClVOPjaYmAqeHOJnomznCPk2DbCytJlAbJtT0EBcU=; b=XCJ961Jxo5Q9tgv5PF01oPBtPyCBkYhApIjoMErVBb0r/d1WuHE7i/W0hXpK2/fWpp doFfqowKum1tDy6K71BwFVuEcwPi7lwXrZSciStwepMP7rv6VNz7YI5JaM0nEPNQ2ETL TMlADwDCiR/moYcexdg15GceN0QKeOWJlGNpUpnVRycJ5n4Eys4ILZK8n8JzXSkpl//X gYnYCRaWimN6w1LDixru6dxxoQ1qUvu1RcK3X+Wf20QNxEs+YgYN3NQFFeZVzpbBq59S Vt/sanZ1zUxa5EoR0dbNDjnjo+9dRA1tLZWpBwMAAdR2Qs9ZmIc5DgCUrhXCtO2qRsjL pAJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734780; x=1721339580; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lAClVOPjaYmAqeHOJnomznCPk2DbCytJlAbJtT0EBcU=; b=ntu0jfRlf9104CMJYuKTjkWMuVtSbiwj1Zeg/SjjVDIn9VliCsqtavjVz0icO3nr1w yXee1y/2YcTIxctefxCLH96uQna6VvHQCxGssw2fohXmOCaHUNlbjMv0ZWbc2o0fGA90 VfWDd2vOfPJmAMWwukYOAYXK0JUc4bhzRPOgAlBqI9q+Pzuf4Ce/TA/Ym1PFZMzOPX4j AVpcoBjM4OOm5Hsva68Q6F3H067pjpnIxv+YoXu9DC7oL0JxN6aSxI1bhDxdUXcHSkWY ENZUO11g4h2oiSAmHHaL6KW0KlIB2OvGAI+sOsFTFa1vfle2Tc8SYMCH/Jp8fUfDDhOo Bm6A== X-Forwarded-Encrypted: i=1; AJvYcCXr89t9XPHTzSa0QpOfyWPT0az8rGdGHQPhn5cQOeTL/awJhN9Cm8UyrOgDWITXl3S1g2/86X6uVHefrW5o2VAmsAP8pUo= X-Gm-Message-State: AOJu0YzZEzNs8dDdRvh04xPbknY1SaCN46D7A9vnz0lVwRFWJ5MYSUnl YHOGEaRaQJMx6qtQKYOOuyglE/uSEYQ1cRVUdly/lHL0iFMsUOWVikEHD4F6Z/I= X-Google-Smtp-Source: AGHT+IHhGaHZPvsOOFoJBlZC6j7jksJpDnEgjxRqPRRvjpMOSkZgwavxCc6jUd++pCbDbuTy3E32aQ== X-Received: by 2002:a05:6830:3b84:b0:703:7051:4018 with SMTP id 46e09a7af769-70375a4634dmr10897638a34.28.1720734780412; Thu, 11 Jul 2024 14:53:00 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.52.57 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:53:00 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 05/13] util/dsa: Implement DSA task asynchronous completion thread model. Date: Thu, 11 Jul 2024 14:52:40 -0700 Message-Id: <20240711215244.19237-6-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::332; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x332.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Create a dedicated thread for DSA task completion. * DSA completion thread runs a loop and poll for completed tasks. * Start and stop DSA completion thread during DSA device start stop. User space application can directly submit task to Intel DSA accelerator by writing to DSA's device memory (mapped in user space). Once a task is submitted, the device starts processing it and write the completion status back to the task. A user space application can poll the task's completion status to check for completion. This change uses a dedicated thread to perform DSA task completion checking. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 1 + util/dsa.c | 274 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 274 insertions(+), 1 deletion(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 9cc836b64c..d46a9f42a5 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -69,6 +69,7 @@ typedef struct QemuDsaBatchTask { QemuDsaTaskType task_type; QemuDsaTaskStatus status; int batch_size; + bool *results; QSIMPLEQ_ENTRY(QemuDsaBatchTask) entry; } QemuDsaBatchTask; diff --git a/util/dsa.c b/util/dsa.c index 43280b31cd..1eb85f37f1 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -33,9 +33,20 @@ #define DSA_WQ_PORTAL_SIZE 4096 #define DSA_WQ_DEPTH 128 #define MAX_DSA_DEVICES 16 +#define DSA_COMPLETION_THREAD "qemu_dsa_completion" + +typedef struct { + bool stopping; + bool running; + QemuThread thread; + int thread_id; + QemuSemaphore sem_init_done; + QemuDsaDeviceGroup *group; +} QemuDsaCompletionThread; uint32_t max_retry_count; static QemuDsaDeviceGroup dsa_group; +static QemuDsaCompletionThread completion_thread; /** @@ -409,6 +420,265 @@ submit_batch_wi_async(QemuDsaBatchTask *batch_task) return dsa_task_enqueue(device_group, batch_task); } +/** + * @brief Poll for the DSA work item completion. + * + * @param completion A pointer to the DSA work item completion record. + * @param opcode The DSA opcode. + * + * @return Zero if successful, non-zero otherwise. + */ +static int +poll_completion(struct dsa_completion_record *completion, + enum dsa_opcode opcode) +{ + uint8_t status; + uint64_t retry = 0; + + while (true) { + /* The DSA operation completes successfully or fails. */ + status = completion->status; + if (status == DSA_COMP_SUCCESS || + status == DSA_COMP_PAGE_FAULT_NOBOF || + status == DSA_COMP_BATCH_PAGE_FAULT || + status == DSA_COMP_BATCH_FAIL) { + break; + } else if (status != DSA_COMP_NONE) { + error_report("DSA opcode %d failed with status = %d.", + opcode, status); + return 1; + } + retry++; + if (retry > max_retry_count) { + error_report("DSA wait for completion retry %lu times.", retry); + return 1; + } + _mm_pause(); + } + + return 0; +} + +/** + * @brief Complete a single DSA task in the batch task. + * + * @param task A pointer to the batch task structure. + * + * @return Zero if successful, otherwise non-zero. + */ +static int +poll_task_completion(QemuDsaBatchTask *task) +{ + assert(task->task_type == QEMU_DSA_TASK); + + struct dsa_completion_record *completion = &task->completions[0]; + uint8_t status; + int ret; + + ret = poll_completion(completion, task->descriptors[0].opcode); + if (ret != 0) { + goto exit; + } + + status = completion->status; + if (status == DSA_COMP_SUCCESS) { + task->results[0] = (completion->result == 0); + goto exit; + } + + assert(status == DSA_COMP_PAGE_FAULT_NOBOF); + +exit: + return ret; +} + +/** + * @brief Poll a batch task status until it completes. If DSA task doesn't + * complete properly, use CPU to complete the task. + * + * @param batch_task A pointer to the DSA batch task. + * + * @return Zero if successful, otherwise non-zero. + */ +static int +poll_batch_task_completion(QemuDsaBatchTask *batch_task) +{ + struct dsa_completion_record *batch_completion = + &batch_task->batch_completion; + struct dsa_completion_record *completion; + uint8_t batch_status; + uint8_t status; + bool *results = batch_task->results; + uint32_t count = batch_task->batch_descriptor.desc_count; + int ret; + + ret = poll_completion(batch_completion, + batch_task->batch_descriptor.opcode); + if (ret != 0) { + goto exit; + } + + batch_status = batch_completion->status; + + if (batch_status == DSA_COMP_SUCCESS) { + if (batch_completion->bytes_completed == count) { + /* + * Let's skip checking for each descriptors' completion status + * if the batch descriptor says all succedded. + */ + for (int i = 0; i < count; i++) { + assert(batch_task->completions[i].status == DSA_COMP_SUCCESS); + results[i] = (batch_task->completions[i].result == 0); + } + goto exit; + } + } else { + assert(batch_status == DSA_COMP_BATCH_FAIL || + batch_status == DSA_COMP_BATCH_PAGE_FAULT); + } + + for (int i = 0; i < count; i++) { + + completion = &batch_task->completions[i]; + status = completion->status; + + if (status == DSA_COMP_SUCCESS) { + results[i] = (completion->result == 0); + continue; + } + + assert(status == DSA_COMP_PAGE_FAULT_NOBOF); + + if (status != DSA_COMP_PAGE_FAULT_NOBOF) { + error_report("Unexpected DSA completion status = %u.", status); + ret = 1; + goto exit; + } + } + +exit: + return ret; +} + +/** + * @brief Handles an asynchronous DSA batch task completion. + * + * @param task A pointer to the batch buffer zero task structure. + */ +static void +dsa_batch_task_complete(QemuDsaBatchTask *batch_task) +{ + batch_task->status = QEMU_DSA_TASK_COMPLETION; + batch_task->completion_callback(batch_task); +} + +/** + * @brief The function entry point called by a dedicated DSA + * work item completion thread. + * + * @param opaque A pointer to the thread context. + * + * @return void* Not used. + */ +static void * +dsa_completion_loop(void *opaque) +{ + QemuDsaCompletionThread *thread_context = + (QemuDsaCompletionThread *)opaque; + QemuDsaBatchTask *batch_task; + QemuDsaDeviceGroup *group = thread_context->group; + int ret; + + rcu_register_thread(); + + thread_context->thread_id = qemu_get_thread_id(); + qemu_sem_post(&thread_context->sem_init_done); + + while (thread_context->running) { + batch_task = dsa_task_dequeue(group); + assert(batch_task != NULL || !group->running); + if (!group->running) { + assert(!thread_context->running); + break; + } + if (batch_task->task_type == QEMU_DSA_TASK) { + ret = poll_task_completion(batch_task); + } else { + assert(batch_task->task_type == QEMU_DSA_BATCH_TASK); + ret = poll_batch_task_completion(batch_task); + } + + if (ret != 0) { + goto exit; + } + + dsa_batch_task_complete(batch_task); + } + +exit: + if (ret != 0) { + error_report("DSA completion thread exited due to internal error."); + } + rcu_unregister_thread(); + return NULL; +} + +/** + * @brief Initializes a DSA completion thread. + * + * @param completion_thread A pointer to the completion thread context. + * @param group A pointer to the DSA device group. + */ +static void +dsa_completion_thread_init( + QemuDsaCompletionThread *completion_thread, + QemuDsaDeviceGroup *group) +{ + completion_thread->stopping = false; + completion_thread->running = true; + completion_thread->thread_id = -1; + qemu_sem_init(&completion_thread->sem_init_done, 0); + completion_thread->group = group; + + qemu_thread_create(&completion_thread->thread, + DSA_COMPLETION_THREAD, + dsa_completion_loop, + completion_thread, + QEMU_THREAD_JOINABLE); + + /* Wait for initialization to complete */ + qemu_sem_wait(&completion_thread->sem_init_done); +} + +/** + * @brief Stops the completion thread (and implicitly, the device group). + * + * @param opaque A pointer to the completion thread. + */ +static void dsa_completion_thread_stop(void *opaque) +{ + QemuDsaCompletionThread *thread_context = + (QemuDsaCompletionThread *)opaque; + + QemuDsaDeviceGroup *group = thread_context->group; + + qemu_mutex_lock(&group->task_queue_lock); + + thread_context->stopping = true; + thread_context->running = false; + + /* Prevent the compiler from setting group->running first. */ + barrier(); + dsa_device_group_stop(group); + + qemu_cond_signal(&group->task_queue_cond); + qemu_mutex_unlock(&group->task_queue_lock); + + qemu_thread_join(&thread_context->thread); + + qemu_sem_destroy(&thread_context->sem_init_done); +} + /** * @brief Check if DSA is running. * @@ -416,7 +686,7 @@ submit_batch_wi_async(QemuDsaBatchTask *batch_task) */ bool qemu_dsa_is_running(void) { - return false; + return completion_thread.running; } static void @@ -452,6 +722,7 @@ void qemu_dsa_start(void) return; } dsa_device_group_start(&dsa_group); + dsa_completion_thread_init(&completion_thread, &dsa_group); } /** @@ -466,6 +737,7 @@ void qemu_dsa_stop(void) return; } + dsa_completion_thread_stop(&completion_thread); dsa_empty_task_queue(group); } From patchwork Thu Jul 11 21:52:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D13DC3DA45 for ; Thu, 11 Jul 2024 21:54:42 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1iu-0004MB-6m; Thu, 11 Jul 2024 17:53:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1is-0004HG-PZ for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:06 -0400 Received: from mail-ot1-x335.google.com ([2607:f8b0:4864:20::335]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1iq-0005a1-GT for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:06 -0400 Received: by mail-ot1-x335.google.com with SMTP id 46e09a7af769-703ba2477bdso641050a34.2 for ; Thu, 11 Jul 2024 14:53:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734783; x=1721339583; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7ksX8+D+3HSKg/c07IUgWYwiC42rzANbtLF+jp56qHU=; b=KdQHeebORG91L4nM4R/IM1zSkaBx6yQR8g/2JMI9DPmPGWvI935iY+dMKrARUZ0YD8 H7XSM19ODKFyFOSIyuqpb7k4V0urdydqPmFQpNpNcmYaUzOb0I5zjyJe3jOyLE7knJeX KyttoXuhkhYqKMttzA1qUJpt6RwQq6Cn6jFU90pKpvJ7aELmNNfbu+Cnwo+fwdmOnpAW r1xZlWhJfTTzGGjsPlmim0hQbJLtXP09FUhRLaiN5rm6qdZ3gvjVcy6dsYjtaUOP6MwR V6+2L416Yc9m1bwhVu10VwosMAlG0hpsvCeiDSQQ3xHvmx65eZ+EzXTLKmS1lN1XFqlj 5zfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734783; x=1721339583; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7ksX8+D+3HSKg/c07IUgWYwiC42rzANbtLF+jp56qHU=; b=NQ+eM0OddoFIndr/6nprKAaulpO09Dm2VtaA4ggyTIaLRy1eh446AwQPEyGQWicuL3 kJn/oZwaFzxGnfKccu2w0B/d1YoPrwD6oiXS7VeccLy3w9GhVCKIC/uG5JBEE568hWdV /qqq1U8C3+FksPKllnbUy2+lz7om09iWv/dDo+7Lk0uRZoocCg4nKtdb/5JC9GJaq24P 4SHRYeJwYMlF8dyuzGF1/dLj+YebvYadXcZ6tvulS+mRmYeqjAe6BFAoDPHMeYCE6ruu 64ykVXR4SZUVM3QoWuSNKqZLU2y5zlVAIVDutS3yNWkowibbslewr5vmm97meQWxQk9f Gw9Q== X-Forwarded-Encrypted: i=1; AJvYcCVEF5kcLEMprxeoc22E2ncmO4q2lDOnykiIA7nI2XHoR+rwPlPHtX37pfvDHwUoOobmbWxXXXun3hOUWQu2dd3FBHaljM4= X-Gm-Message-State: AOJu0YwnfXjUd+Xzhy4RX7zbUxcMf9sLHLykDdbMJ6I22AYUVRd+FSdy F38X8MwgsgINpHo4wpkxAgh7lTQwNwIe7HZx8Ps9RQfAUtpV5iyFXgvzzl1y/Kc= X-Google-Smtp-Source: AGHT+IEVIUXjXQowbbH6rLfMkmeq3YWA3T2pIBuIw7E1mjvhPBgcZHOrcMVzN4BnTmpgo1P4fITLjg== X-Received: by 2002:a9d:5912:0:b0:703:5f9f:82f2 with SMTP id 46e09a7af769-70375a0baccmr11210651a34.18.1720734783029; Thu, 11 Jul 2024 14:53:03 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.53.00 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:53:02 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v5 06/13] util/dsa: Implement zero page checking in DSA task. Date: Thu, 11 Jul 2024 14:52:41 -0700 Message-Id: <20240711215244.19237-7-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::335; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Create DSA task with operation code DSA_OPCODE_COMPVAL. Here we create two types of DSA tasks, a single DSA task and a batch DSA task. Batch DSA task reduces task submission overhead and hence should be the default option. However, due to the way DSA hardware works, a DSA batch task must contain at least two individual tasks. There are times we need to submit a single task and hence a single DSA task submission is also required. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 18 ++++ util/dsa.c | 247 +++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 244 insertions(+), 21 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index d46a9f42a5..1b4baf1c80 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -105,6 +105,24 @@ void qemu_dsa_cleanup(void); */ bool qemu_dsa_is_running(void); +/** + * @brief Initializes a buffer zero batch task. + * + * @param task A pointer to the batch task to initialize. + * @param results A pointer to an array of zero page checking results. + * @param batch_size The number of DSA tasks in the batch. + */ +void +buffer_zero_batch_task_init(QemuDsaBatchTask *task, + bool *results, int batch_size); + +/** + * @brief Performs the proper cleanup on a DSA batch task. + * + * @param task A pointer to the batch task to cleanup. + */ +void buffer_zero_batch_task_destroy(QemuDsaBatchTask *task); + #else static inline bool qemu_dsa_is_running(void) diff --git a/util/dsa.c b/util/dsa.c index 1eb85f37f1..f0d8cce210 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -48,6 +48,7 @@ uint32_t max_retry_count; static QemuDsaDeviceGroup dsa_group; static QemuDsaCompletionThread completion_thread; +static void buffer_zero_dsa_completion(void *context); /** * @brief This function opens a DSA device's work queue and @@ -180,7 +181,6 @@ dsa_device_group_start(QemuDsaDeviceGroup *group) * * @param group A pointer to the DSA device group. */ -__attribute__((unused)) static void dsa_device_group_stop(QemuDsaDeviceGroup *group) { @@ -216,7 +216,6 @@ dsa_device_group_cleanup(QemuDsaDeviceGroup *group) * @return struct QemuDsaDevice* A pointer to the next available DSA device * in the group. */ -__attribute__((unused)) static QemuDsaDevice * dsa_device_group_get_next_device(QemuDsaDeviceGroup *group) { @@ -289,7 +288,6 @@ dsa_task_enqueue(QemuDsaDeviceGroup *group, * @param group A pointer to the DSA device group. * @return QemuDsaBatchTask* The DSA task being dequeued. */ -__attribute__((unused)) static QemuDsaBatchTask * dsa_task_dequeue(QemuDsaDeviceGroup *group) { @@ -344,22 +342,6 @@ submit_wi_int(void *wq, struct dsa_hw_desc *descriptor) return 0; } -/** - * @brief Synchronously submits a DSA work item to the - * device work queue. - * - * @param wq A pointer to the DSA work queue's device memory. - * @param descriptor A pointer to the DSA work item descriptor. - * - * @return int Zero if successful, non-zero otherwise. - */ -__attribute__((unused)) -static int -submit_wi(void *wq, struct dsa_hw_desc *descriptor) -{ - return submit_wi_int(wq, descriptor); -} - /** * @brief Asynchronously submits a DSA work item to the * device work queue. @@ -368,7 +350,6 @@ submit_wi(void *wq, struct dsa_hw_desc *descriptor) * * @return int Zero if successful, non-zero otherwise. */ -__attribute__((unused)) static int submit_wi_async(QemuDsaBatchTask *task) { @@ -397,7 +378,6 @@ submit_wi_async(QemuDsaBatchTask *task) * * @return int Zero if successful, non-zero otherwise. */ -__attribute__((unused)) static int submit_batch_wi_async(QemuDsaBatchTask *batch_task) { @@ -679,6 +659,231 @@ static void dsa_completion_thread_stop(void *opaque) qemu_sem_destroy(&thread_context->sem_init_done); } +/** + * @brief Initializes a buffer zero comparison DSA task. + * + * @param descriptor A pointer to the DSA task descriptor. + * @param completion A pointer to the DSA task completion record. + */ +static void +buffer_zero_task_init_int(struct dsa_hw_desc *descriptor, + struct dsa_completion_record *completion) +{ + descriptor->opcode = DSA_OPCODE_COMPVAL; + descriptor->flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; + descriptor->comp_pattern = (uint64_t)0; + descriptor->completion_addr = (uint64_t)completion; +} + +/** + * @brief Initializes a buffer zero batch task. + * + * @param task A pointer to the batch task to initialize. + * @param results A pointer to an array of zero page checking results. + * @param batch_size The number of DSA tasks in the batch. + */ +void +buffer_zero_batch_task_init(QemuDsaBatchTask *task, + bool *results, int batch_size) +{ + int descriptors_size = sizeof(*task->descriptors) * batch_size; + memset(task, 0, sizeof(*task)); + + task->descriptors = + (struct dsa_hw_desc *)qemu_memalign(64, descriptors_size); + memset(task->descriptors, 0, descriptors_size); + task->completions = (struct dsa_completion_record *)qemu_memalign( + 32, sizeof(*task->completions) * batch_size); + task->results = results; + task->batch_size = batch_size; + + task->batch_completion.status = DSA_COMP_NONE; + task->batch_descriptor.completion_addr = (uint64_t)&task->batch_completion; + /* TODO: Ensure that we never send a batch with count <= 1 */ + task->batch_descriptor.desc_count = 0; + task->batch_descriptor.opcode = DSA_OPCODE_BATCH; + task->batch_descriptor.flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; + task->batch_descriptor.desc_list_addr = (uintptr_t)task->descriptors; + task->status = QEMU_DSA_TASK_READY; + task->group = &dsa_group; + task->device = dsa_device_group_get_next_device(&dsa_group); + + for (int i = 0; i < task->batch_size; i++) { + buffer_zero_task_init_int(&task->descriptors[i], + &task->completions[i]); + } + + qemu_sem_init(&task->sem_task_complete, 0); + task->completion_callback = buffer_zero_dsa_completion; +} + +/** + * @brief Performs the proper cleanup on a DSA batch task. + * + * @param task A pointer to the batch task to cleanup. + */ +void +buffer_zero_batch_task_destroy(QemuDsaBatchTask *task) +{ + qemu_vfree(task->descriptors); + qemu_vfree(task->completions); + task->results = NULL; + + qemu_sem_destroy(&task->sem_task_complete); +} + +/** + * @brief Resets a buffer zero comparison DSA batch task. + * + * @param task A pointer to the batch task. + * @param count The number of DSA tasks this batch task will contain. + */ +static void +buffer_zero_batch_task_reset(QemuDsaBatchTask *task, size_t count) +{ + task->batch_completion.status = DSA_COMP_NONE; + task->batch_descriptor.desc_count = count; + task->task_type = QEMU_DSA_BATCH_TASK; + task->status = QEMU_DSA_TASK_READY; +} + +/** + * @brief Sets a buffer zero comparison DSA task. + * + * @param descriptor A pointer to the DSA task descriptor. + * @param buf A pointer to the memory buffer. + * @param len The length of the buffer. + */ +static void +buffer_zero_task_set_int(struct dsa_hw_desc *descriptor, + const void *buf, + size_t len) +{ + struct dsa_completion_record *completion = + (struct dsa_completion_record *)descriptor->completion_addr; + + descriptor->xfer_size = len; + descriptor->src_addr = (uintptr_t)buf; + completion->status = 0; + completion->result = 0; +} + +/** + * @brief Resets a buffer zero comparison DSA batch task. + * + * @param task A pointer to the DSA batch task. + */ +static void +buffer_zero_task_reset(QemuDsaBatchTask *task) +{ + task->completions[0].status = DSA_COMP_NONE; + task->task_type = QEMU_DSA_TASK; + task->status = QEMU_DSA_TASK_READY; +} + +/** + * @brief Sets a buffer zero comparison DSA task. + * + * @param task A pointer to the DSA task. + * @param buf A pointer to the memory buffer. + * @param len The buffer length. + */ +static void +buffer_zero_task_set(QemuDsaBatchTask *task, + const void *buf, + size_t len) +{ + buffer_zero_task_reset(task); + buffer_zero_task_set_int(&task->descriptors[0], buf, len); +} + +/** + * @brief Sets a buffer zero comparison batch task. + * + * @param batch_task A pointer to the batch task. + * @param buf An array of memory buffers. + * @param count The number of buffers in the array. + * @param len The length of the buffers. + */ +static void +buffer_zero_batch_task_set(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + assert(count > 0); + assert(count <= batch_task->batch_size); + + buffer_zero_batch_task_reset(batch_task, count); + for (int i = 0; i < count; i++) { + buffer_zero_task_set_int(&batch_task->descriptors[i], buf[i], len); + } +} + +/** + * @brief Asychronously perform a buffer zero DSA operation. + * + * @param task A pointer to the batch task structure. + * @param buf A pointer to the memory buffer. + * @param len The length of the memory buffer. + * + * @return int Zero if successful, otherwise an appropriate error code. + */ +__attribute__((unused)) +static int +buffer_zero_dsa_async(QemuDsaBatchTask *task, + const void *buf, size_t len) +{ + buffer_zero_task_set(task, buf, len); + + return submit_wi_async(task); +} + +/** + * @brief Sends a memory comparison batch task to a DSA device and wait + * for completion. + * + * @param batch_task The batch task to be submitted to DSA device. + * @param buf An array of memory buffers to check for zero. + * @param count The number of buffers. + * @param len The buffer length. + */ +__attribute__((unused)) +static int +buffer_zero_dsa_batch_async(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + assert(count <= batch_task->batch_size); + buffer_zero_batch_task_set(batch_task, buf, count, len); + + return submit_batch_wi_async(batch_task); +} + +/** + * @brief The completion callback function for buffer zero + * comparison DSA task completion. + * + * @param context A pointer to the callback context. + */ +static void +buffer_zero_dsa_completion(void *context) +{ + assert(context != NULL); + + QemuDsaBatchTask *task = (QemuDsaBatchTask *)context; + qemu_sem_post(&task->sem_task_complete); +} + +/** + * @brief Wait for the asynchronous DSA task to complete. + * + * @param batch_task A pointer to the buffer zero comparison batch task. + */ +__attribute__((unused)) +static void +buffer_zero_dsa_wait(QemuDsaBatchTask *batch_task) +{ + qemu_sem_wait(&batch_task->sem_task_complete); +} + /** * @brief Check if DSA is running. * From patchwork Thu Jul 11 21:52:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 59D3AC41513 for ; Thu, 11 Jul 2024 21:53:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1iv-0004Sk-Tn; Thu, 11 Jul 2024 17:53:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1iu-0004MX-8q for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:08 -0400 Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1is-0005aB-Cq for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:08 -0400 Received: by mail-ot1-x32f.google.com with SMTP id 46e09a7af769-7036ab4a2acso695418a34.2 for ; Thu, 11 Jul 2024 14:53:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734785; x=1721339585; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9m458/uhhAXId9laqR5q13dVQY2eOCevIh5lPaBzFwg=; b=ZJujagfH6RLfJsb/WC/5sO42CQKOE6uATjzbdgwn9kzbyWxoUM0gS7Lyt5PE5zMV/t ik4A8Oq3pN7EyxOJSSvj37x2WmI4d9nP4N/xP4hWFzCiwzV4TrM7VXkQi13ajSmp5rXk 92phYR0Fx3e3YKsf4GXblevlYeHIEHPv0qhwQMKPIXik45y+m4iTwkwp/JxOzoZ8tTL6 OzrMSe3bU9I3hZhBWXxy9e2l9RPR8Fx3Td0T2x+eesv7MqfsdriX+CG2a2Ua3CzoEZfu 9gkHL/sr2Bb7Tr1IjbIr58EhoeNuVG+9DSm/6SELIA+gGAIzd5f4I8N+SoNXEiMLrO5f blKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734785; x=1721339585; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9m458/uhhAXId9laqR5q13dVQY2eOCevIh5lPaBzFwg=; b=pFQHpPq3bRK9G8QPJsvJKx4yaEzm8GkK40ffmzpe5h3WV/o2uyr5/sIzvCGjc25VoN gYSPWmkFJ0Hohfq8qGoKvEC+awDHB7qCb4Lyp2PGanQIqk7ogcXYkk/DSZFCbTxznIH6 YfSeLqPpCyQFpjdFU00L7sIobmveCjo3Az+rZD69Eau/Udt13lfaBQSnZ0WHfYEAngoN BJ51wH58Ufia89hT0KxjGsCe1NPl34QhfqLi3H9NsVEHG3RbHiAlivJjl3wvC9SMOWr+ DmVuLEDFOEX1F2kJHlgXNwB4asBFlg8bKC5cU+PTkLA2iuYPu/XS6eMiYpJ4Y7aqSGsv gB8g== X-Forwarded-Encrypted: i=1; AJvYcCUGMpEWsd4z/EiUSrx0fgMvzVw/KFhTNAXKAgxrzCNTtb/mYPsrD35cOrRsOJ2o+g195yx5g3YLlyhPLWIyGN2whwSQNmM= X-Gm-Message-State: AOJu0YzfcBOfCR9hjG+iOXoijg2V+3F7OHn2QFPPKicyTwxtQJF3Y+u6 AAL/hmYUzu2iddFEtcCAyHsi6o/Xm1dI1iOx/3sAqVZAu0z94KIZOukDdsOyDWgUP62wtDS7VZW M X-Google-Smtp-Source: AGHT+IGfge220JKGhpa1aVvmXZTMP7C5bj467mbmfAjwUFGjU6QSaq8JXgcvTSLMtk2UF4LQgoDKsw== X-Received: by 2002:a05:6830:1389:b0:703:68ff:c078 with SMTP id 46e09a7af769-70375b4d1b8mr10711216a34.30.1720734785219; Thu, 11 Jul 2024 14:53:05 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.53.03 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:53:04 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v5 07/13] util/dsa: Implement DSA task asynchronous submission and wait for completion. Date: Thu, 11 Jul 2024 14:52:42 -0700 Message-Id: <20240711215244.19237-8-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32f; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Add a DSA task completion callback. * DSA completion thread will call the tasks's completion callback on every task/batch task completion. * DSA submission path to wait for completion. * Implement CPU fallback if DSA is not able to complete the task. Signed-off-by: Hao Xiang Signed-off-by: Bryan Zhang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 14 +++++ util/dsa.c | 125 +++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 135 insertions(+), 4 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 1b4baf1c80..20bb88d48c 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -123,6 +123,20 @@ buffer_zero_batch_task_init(QemuDsaBatchTask *task, */ void buffer_zero_batch_task_destroy(QemuDsaBatchTask *task); +/** + * @brief Performs buffer zero comparison on a DSA batch task synchronously. + * + * @param batch_task A pointer to the batch task. + * @param buf An array of memory buffers. + * @param count The number of buffers in the array. + * @param len The buffer length. + * + * @return Zero if successful, otherwise non-zero. + */ +int +buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len); + #else static inline bool qemu_dsa_is_running(void) diff --git a/util/dsa.c b/util/dsa.c index f0d8cce210..74b9aa1331 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -439,6 +439,42 @@ poll_completion(struct dsa_completion_record *completion, return 0; } +/** + * @brief Helper function to use CPU to complete a single + * zero page checking task. + * + * @param completion A pointer to a DSA task completion record. + * @param descriptor A pointer to a DSA task descriptor. + * @param result A pointer to the result of a zero page checking. + */ +static void +task_cpu_fallback_int(struct dsa_completion_record *completion, + struct dsa_hw_desc *descriptor, bool *result) +{ + const uint8_t *buf; + size_t len; + + if (completion->status == DSA_COMP_SUCCESS) { + return; + } + + /* + * DSA was able to partially complete the operation. Check the + * result. If we already know this is not a zero page, we can + * return now. + */ + if (completion->bytes_completed != 0 && completion->result != 0) { + *result = false; + return; + } + + /* Let's fallback to use CPU to complete it. */ + buf = (const uint8_t *)descriptor->src_addr; + len = descriptor->xfer_size; + *result = buffer_is_zero(buf + completion->bytes_completed, + len - completion->bytes_completed); +} + /** * @brief Complete a single DSA task in the batch task. * @@ -567,7 +603,7 @@ dsa_completion_loop(void *opaque) (QemuDsaCompletionThread *)opaque; QemuDsaBatchTask *batch_task; QemuDsaDeviceGroup *group = thread_context->group; - int ret; + int ret = 0; rcu_register_thread(); @@ -827,7 +863,6 @@ buffer_zero_batch_task_set(QemuDsaBatchTask *batch_task, * * @return int Zero if successful, otherwise an appropriate error code. */ -__attribute__((unused)) static int buffer_zero_dsa_async(QemuDsaBatchTask *task, const void *buf, size_t len) @@ -846,7 +881,6 @@ buffer_zero_dsa_async(QemuDsaBatchTask *task, * @param count The number of buffers. * @param len The buffer length. */ -__attribute__((unused)) static int buffer_zero_dsa_batch_async(QemuDsaBatchTask *batch_task, const void **buf, size_t count, size_t len) @@ -877,13 +911,61 @@ buffer_zero_dsa_completion(void *context) * * @param batch_task A pointer to the buffer zero comparison batch task. */ -__attribute__((unused)) static void buffer_zero_dsa_wait(QemuDsaBatchTask *batch_task) { qemu_sem_wait(&batch_task->sem_task_complete); } +/** + * @brief Use CPU to complete the zero page checking task if DSA + * is not able to complete it. + * + * @param batch_task A pointer to the batch task. + */ +static void +buffer_zero_cpu_fallback(QemuDsaBatchTask *batch_task) +{ + if (batch_task->task_type == QEMU_DSA_TASK) { + if (batch_task->completions[0].status == DSA_COMP_SUCCESS) { + return; + } + task_cpu_fallback_int(&batch_task->completions[0], + &batch_task->descriptors[0], + &batch_task->results[0]); + } else if (batch_task->task_type == QEMU_DSA_BATCH_TASK) { + struct dsa_completion_record *batch_completion = + &batch_task->batch_completion; + struct dsa_completion_record *completion; + uint8_t status; + bool *results = batch_task->results; + uint32_t count = batch_task->batch_descriptor.desc_count; + + /* DSA is able to complete the entire batch task. */ + if (batch_completion->status == DSA_COMP_SUCCESS) { + assert(count == batch_completion->bytes_completed); + return; + } + + /* + * DSA encounters some error and is not able to complete + * the entire batch task. Use CPU fallback. + */ + for (int i = 0; i < count; i++) { + + completion = &batch_task->completions[i]; + status = completion->status; + + assert(status == DSA_COMP_SUCCESS || + status == DSA_COMP_PAGE_FAULT_NOBOF); + + task_cpu_fallback_int(completion, + &batch_task->descriptors[i], + &results[i]); + } + } +} + /** * @brief Check if DSA is running. * @@ -956,3 +1038,38 @@ void qemu_dsa_cleanup(void) dsa_device_group_cleanup(&dsa_group); } +/** + * @brief Performs buffer zero comparison on a DSA batch task synchronously. + * + * @param batch_task A pointer to the batch task. + * @param buf An array of memory buffers. + * @param count The number of buffers in the array. + * @param len The buffer length. + * + * @return Zero if successful, otherwise non-zero. + */ +int +buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + if (count <= 0 || count > batch_task->batch_size) { + return -1; + } + + assert(batch_task != NULL); + assert(len != 0); + assert(buf != NULL); + + if (count == 1) { + /* DSA doesn't take batch operation with only 1 task. */ + buffer_zero_dsa_async(batch_task, buf[0], len); + } else { + buffer_zero_dsa_batch_async(batch_task, buf, count, len); + } + + buffer_zero_dsa_wait(batch_task); + buffer_zero_cpu_fallback(batch_task); + + return 0; +} + From patchwork Thu Jul 11 21:52:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DAD9EC3DA45 for ; Thu, 11 Jul 2024 21:54:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1iz-0004h9-Tj; Thu, 11 Jul 2024 17:53:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1ix-0004Zy-U8 for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:11 -0400 Received: from mail-ot1-x333.google.com ([2607:f8b0:4864:20::333]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1iu-0005aR-Qo for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:11 -0400 Received: by mail-ot1-x333.google.com with SMTP id 46e09a7af769-703775e4d5dso678142a34.0 for ; Thu, 11 Jul 2024 14:53:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734788; x=1721339588; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wy8hCP/+6sQNwtALpMZF1H5u+/ijMcnvyWkW4ClZRxQ=; b=jYrCISgqZDMDLul0Ibl04easc8OVHAkD+PwwuIqdVxNlZUjmswFtSAQfS1n8ePEs7i dPBUJIE0G6xJeqnxvDUYzqEXJ4/0qgBk8a8PBxXWtcd8H4orEh9fTvxOiOVHUz7INXXr 7zA1LbtKAeHydoJhZp+gLT17/ln03tJd4WT8ZgudthyvycmgBHEz2MmzLyV7VVO6LiC5 Vgj8hLArQJ3Of4o9ymFqA8Z9EAzuRk31Q8g1GSqRXt1RknN2UeVDLh/FOVbkaVS09M/F PDM6+T+TfHAFvPzsAV1sSzSAhIwU6t6S/lxBs7Tu1th1wtqH55xswc9NTERQ43eBV5vM nKyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734788; x=1721339588; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Wy8hCP/+6sQNwtALpMZF1H5u+/ijMcnvyWkW4ClZRxQ=; b=d6lp7sB9SB8bOb+R6tOZuPtX+Hz4xM/KSUgBoBfWT9h44h6O0e9SiGvgm4nBoXuZJo hxUXZ/vBIwiaMagW6AhyyTlHMf8OzLAsNsAoLseEzadkr3D3KAK25/zRIACg8wFZllmD CCxpzHfQTQ3i/S+8IbgIyp7N/qU2U3mKy6GFU85C2pOU+e8ib76c6BABc7o8mSpxGOGa fP9ualeDBSvQMHiJOSRCZC7hB8ZD/uzCVTzoO1T6BJXSl0WtcAUNnKu2hXrC8MNGpAHB IfwsmfVHBy8f+TL8rr1GvFStHGl9E0aBs3UvbINcIB9Km03SWbEdZYNJM0CBNEkkX6uW y88g== X-Forwarded-Encrypted: i=1; AJvYcCW6oTn4XohpuxjtUQFoB0Jr5eZhNBSvaBxboTTxXkcmyjukHz7xgnkksN6Hl5OCWZK974v7ZSUIaW0Z72ojsJSxPrJMji8= X-Gm-Message-State: AOJu0Yy0VRLqk9KE6sVgRLhEo/tpqCfT7UAzCLL45vBmuvVH1rFmCBvZ 6+vmEZG5LzUJGkgUp/BugREkr4WCAuPJ5r7hebYiMLvdeUD8HEXTHezuIThS2yo= X-Google-Smtp-Source: AGHT+IF9qP5Rrk81NU7tcOi3HgqDWY/wy+MeMKx+SFv8SPNlOXlduJZMKVc481+5UT80iVuvsGDHuQ== X-Received: by 2002:a05:6830:95:b0:703:5d0e:b9cd with SMTP id 46e09a7af769-70375b55bc3mr10295443a34.34.1720734787522; Thu, 11 Jul 2024 14:53:07 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.53.05 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:53:07 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 08/13] migration/multifd: Add new migration option for multifd DSA offloading. Date: Thu, 11 Jul 2024 14:52:43 -0700 Message-Id: <20240711215244.19237-9-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::333; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Intel DSA offloading is an optional feature that turns on if proper hardware and software stack is available. To turn on DSA offloading in multifd live migration: dsa-accel-path="[dsa_dev_path1] [dsa_dev_path2] ... [dsa_dev_pathX]" This feature is turned off by default. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 15 ++++++++++- migration/options.c | 47 ++++++++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 32 ++++++++++++++++++++--- 4 files changed, 90 insertions(+), 5 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index 7d608d26e1..c422db4ecd 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -312,7 +312,16 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: '%s'\n", MigrationParameter_str(MIGRATION_PARAMETER_TLS_AUTHZ), params->tls_authz); - + if (params->has_dsa_accel_path) { + strList *dsa_accel_path = params->dsa_accel_path; + monitor_printf(mon, "%s:", + MigrationParameter_str(MIGRATION_PARAMETER_DSA_ACCEL_PATH)); + while (dsa_accel_path) { + monitor_printf(mon, " %s", dsa_accel_path->value); + dsa_accel_path = dsa_accel_path->next; + } + monitor_printf(mon, "\n"); + } if (params->has_block_bitmap_mapping) { const BitmapMigrationNodeAliasList *bmnal; @@ -563,6 +572,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_x_checkpoint_delay = true; visit_type_uint32(v, param, &p->x_checkpoint_delay, &err); break; + case MIGRATION_PARAMETER_DSA_ACCEL_PATH: + p->has_dsa_accel_path = true; + visit_type_strList(v, param, &p->dsa_accel_path, &err); + break; case MIGRATION_PARAMETER_MULTIFD_CHANNELS: p->has_multifd_channels = true; visit_type_uint8(v, param, &p->multifd_channels, &err); diff --git a/migration/options.c b/migration/options.c index 645f55003d..f839493016 100644 --- a/migration/options.c +++ b/migration/options.c @@ -29,6 +29,7 @@ #include "ram.h" #include "options.h" #include "sysemu/kvm.h" +#include /* Maximum migrate downtime set to 2000 seconds */ #define MAX_MIGRATE_DOWNTIME_SECONDS 2000 @@ -162,6 +163,10 @@ Property migration_properties[] = { DEFINE_PROP_ZERO_PAGE_DETECTION("zero-page-detection", MigrationState, parameters.zero_page_detection, ZERO_PAGE_DETECTION_MULTIFD), + /* DEFINE_PROP_ARRAY("dsa-accel-path", MigrationState, x, */ + /* parameters.dsa_accel_path, qdev_prop_string, char *), */ + /* DEFINE_PROP_STRING("dsa-accel-path", MigrationState, */ + /* parameters.dsa_accel_path), */ /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -815,6 +820,13 @@ const char *migrate_tls_creds(void) return s->parameters.tls_creds; } +const strList *migrate_dsa_accel_path(void) +{ + MigrationState *s = migrate_get_current(); + + return s->parameters.dsa_accel_path; +} + const char *migrate_tls_hostname(void) { MigrationState *s = migrate_get_current(); @@ -926,6 +938,7 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->zero_page_detection = s->parameters.zero_page_detection; params->has_direct_io = true; params->direct_io = s->parameters.direct_io; + params->dsa_accel_path = QAPI_CLONE(strList, s->parameters.dsa_accel_path); return params; } @@ -934,6 +947,7 @@ void migrate_params_init(MigrationParameters *params) { params->tls_hostname = g_strdup(""); params->tls_creds = g_strdup(""); + params->dsa_accel_path = NULL; /* Set has_* up only for parameter checks */ params->has_throttle_trigger_threshold = true; @@ -1137,6 +1151,22 @@ bool migrate_params_check(MigrationParameters *params, Error **errp) return false; } + if (params->has_zero_page_detection && + params->zero_page_detection == ZERO_PAGE_DETECTION_DSA_ACCEL) { +#ifdef CONFIG_DSA_OPT + unsigned int eax, ebx, ecx, edx; + /* ENQCMD is indicated by bit 29 of ecx in CPUID leaf 7, subleaf 0. */ + if (!__get_cpuid_count(7, 0, &eax, &ebx, &ecx, &edx) || + !(ecx & (1 << 29))) { + error_setg(errp, "DSA acceleration is not supported by CPU"); + return false; + } +#else + error_setg(errp, "DSA acceleration is not enabled"); + return false; +#endif + } + return true; } @@ -1247,6 +1277,11 @@ static void migrate_params_test_apply(MigrateSetParameters *params, if (params->has_direct_io) { dest->direct_io = params->direct_io; } + + if (params->has_dsa_accel_path) { + dest->has_dsa_accel_path = true; + dest->dsa_accel_path = params->dsa_accel_path; + } } static void migrate_params_apply(MigrateSetParameters *params, Error **errp) @@ -1376,6 +1411,12 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) if (params->has_direct_io) { s->parameters.direct_io = params->direct_io; } + if (params->has_dsa_accel_path) { + qapi_free_strList(s->parameters.dsa_accel_path); + s->parameters.has_dsa_accel_path = true; + s->parameters.dsa_accel_path = + QAPI_CLONE(strList, params->dsa_accel_path); + } } void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) @@ -1401,6 +1442,12 @@ void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) params->tls_authz->type = QTYPE_QSTRING; params->tls_authz->u.s = strdup(""); } + /* if (params->dsa_accel_path */ + /* && params->dsa_accel_path->type == QTYPE_QNULL) { */ + /* qobject_unref(params->dsa_accel_path->u.n); */ + /* params->dsa_accel_path->type = QTYPE_QLIST; */ + /* params->dsa_accel_path->u.s = strdup(""); */ + /* } */ migrate_params_test_apply(params, &tmp); diff --git a/migration/options.h b/migration/options.h index a2397026db..78b9e4080b 100644 --- a/migration/options.h +++ b/migration/options.h @@ -85,6 +85,7 @@ const char *migrate_tls_creds(void); const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); ZeroPageDetection migrate_zero_page_detection(void); +const strList *migrate_dsa_accel_path(void); /* parameters helpers */ diff --git a/qapi/migration.json b/qapi/migration.json index 1234bef888..ff41780347 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -619,10 +619,14 @@ # multifd migration is enabled, else in the main migration thread # as for @legacy. # +# @dsa-accel: Perform zero page checking with the DSA accelerator +# offloading in multifd sender thread if multifd migration is +# enabled, else in the main migration thread as for @legacy. +# # Since: 9.0 ## { 'enum': 'ZeroPageDetection', - 'data': [ 'none', 'legacy', 'multifd' ] } + 'data': [ 'none', 'legacy', 'multifd', 'dsa-accel' ] } ## # @BitmapMigrationBitmapAliasTransform: @@ -825,6 +829,12 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @dsa-accel-path: If enabled, use DSA accelerator offloading for +# certain memory operations. Enable DSA accelerator for zero +# page detection offloading by setting the @zero-page-detection +# to dsa-accel. This parameter defines the dsa device path, and +# defaults to an empty list. (since 9.2) +# # @direct-io: Open migration files with O_DIRECT when possible. This # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) @@ -843,7 +853,7 @@ 'cpu-throttle-initial', 'cpu-throttle-increment', 'cpu-throttle-tailslow', 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth', - 'avail-switchover-bandwidth', 'downtime-limit', + 'avail-switchover-bandwidth', 'downtime-limit', 'dsa-accel-path', { 'name': 'x-checkpoint-delay', 'features': [ 'unstable' ] }, 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', @@ -1000,6 +1010,12 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @dsa-accel-path: If enabled, use DSA accelerator offloading for +# certain memory operations. Enable DSA accelerator for zero +# page detection offloading by setting the @zero-page-detection +# to dsa-accel. This parameter defines the dsa device path, and +# defaults to an empty list. (since 9.2) +# # @direct-io: Open migration files with O_DIRECT when possible. This # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) @@ -1044,7 +1060,8 @@ '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', - '*direct-io': 'bool' } } + '*direct-io': 'bool', + '*dsa-accel-path': ['str'] } } ## # @migrate-set-parameters: @@ -1204,6 +1221,12 @@ # See description in @ZeroPageDetection. Default is 'multifd'. # (since 9.0) # +# @dsa-accel-path: If enabled, use DSA accelerator offloading for +# certain memory operations. Enable DSA accelerator for zero +# page detection offloading by setting the @zero-page-detection +# to dsa-accel. This parameter defines the dsa device path, and +# defaults to an empty list. (since 9.2) +# # @direct-io: Open migration files with O_DIRECT when possible. This # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) @@ -1245,7 +1268,8 @@ '*vcpu-dirty-limit': 'uint64', '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', - '*direct-io': 'bool' } } + '*direct-io': 'bool', + '*dsa-accel-path': ['str'] } } ## # @query-migrate-parameters: From patchwork Thu Jul 11 21:52:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA4F7C41513 for ; Thu, 11 Jul 2024 21:54:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1j1-0004np-Tw; Thu, 11 Jul 2024 17:53:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1iz-0004gO-N3 for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:13 -0400 Received: from mail-ot1-x32c.google.com ([2607:f8b0:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1ix-0005al-4U for qemu-devel@nongnu.org; Thu, 11 Jul 2024 17:53:13 -0400 Received: by mail-ot1-x32c.google.com with SMTP id 46e09a7af769-7036e383089so757309a34.2 for ; Thu, 11 Jul 2024 14:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720734790; x=1721339590; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JrzhnVMIN0zieDhkTpHmtBi2Mh7MPJkNAnz/C4OZgI0=; b=NjCckZzCoKo2mBdAZAhMl2l/sL8VZLwzWe+ylUMn77XJ6B3QMuVFjJFem6yFSBC5WA VqVwzBpA9MzwXFl6pPmh8uY5A3C1QkedSx3+l8fueDiGhmFuZTUNiPX1ARdK4BWcCVKD oT6isFUIrdhLqBE4D709Jv5Mz3eizjb3QF1Em0z/n+VFGTUUm3q26sNMQZn0+WsL/f0I BP6Ts5/UO26nBpMN5vxp+dOUoFTnAeiegt0UeGRv91C3Zg8o6klzPdCp7YLHjeWSBGMO G1z4zBDjl3uErNkgRH5otvNDApIq50bzM+GndSYtEJzkVNUfLi+7sVJ09TfamkvCwZVW zNBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720734790; x=1721339590; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JrzhnVMIN0zieDhkTpHmtBi2Mh7MPJkNAnz/C4OZgI0=; b=w/KdF4c4tAczXTx4sLHI9uHFc3B990mLlxjXHzW5ndiWnoiFQ/k7rCQufXqyzFVw3o Rp+/+OGitEVCUIMPfLq8OsGxeP0gX/eRnw7qhpR72YQBNr4hTwgb40EOdUIOOW7Yt5Wo gmVJ6Hh00LmoWq7d/B62hh7Lqc6xgNUfGAIZGOOS2LIowKQ0QpUKsSN8TM2dGKJTTjiL hlyNgZ5hfYxZ7elg6OrVoHrmqb0+dy6l9qIPTplWSb1a9VE0Uwag/G5VaEB2617UQic2 Sxe0wvBaOBZw6Zi6/Z8f+eadnVCyb4NyRv8ZPQly5jbTcRJifWEI8AAoZ3PF644M7Nqj N0RQ== X-Forwarded-Encrypted: i=1; AJvYcCX1f1vMIJ3hRxNxPKBNZlENlnWsY3fdrjxfsdDn3USAsv+Pkv6PecNIXjaKXdUkPfYjBXZN3vQtsKdNd2nV9CWtyW2M6qA= X-Gm-Message-State: AOJu0Yyr2YwHfGgfhMRn8RjZQ5rGqMSqvD7M5W/Eb1JI7iVDkGUORkEo X5URuFeWmYu7VQRWpBnoYFIO7+fQTGKo6YQ4XAdyNpTo7v7HSveKwNzWEAzh3bI= X-Google-Smtp-Source: AGHT+IHsSMKlrqRzTEMdePQ4spC/Rq8EILcx2P/tUbDZZguToZbmRh75MKkWJ3das3SV7q5S6Ny7xw== X-Received: by 2002:a05:6830:39d7:b0:703:6989:5b04 with SMTP id 46e09a7af769-70375b48a32mr10113294a34.31.1720734789880; Thu, 11 Jul 2024 14:53:09 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7044ac2876dsm713125a34.2.2024.07.11.14.53.08 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 14:53:09 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 09/13] migration/multifd: Prepare to introduce DSA acceleration on the multifd path. Date: Thu, 11 Jul 2024 14:52:44 -0700 Message-Id: <20240711215244.19237-10-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711215244.19237-1-yichen.wang@bytedance.com> References: <20240711215244.19237-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32c; envelope-from=yichen.wang@bytedance.com; helo=mail-ot1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang 1. Refactor multifd_send_thread function. 2. Introduce the batch task structure in MultiFDSendParams. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 41 ++++--- migration/multifd.c | 4 + migration/multifd.h | 3 + util/dsa.c | 270 +++++++++++++++++++++++--------------------- 4 files changed, 172 insertions(+), 146 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index 20bb88d48c..fd0305a7c7 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -16,6 +16,7 @@ #define QEMU_DSA_H #include "qemu/error-report.h" +#include "exec/cpu-common.h" #include "qemu/thread.h" #include "qemu/queue.h" @@ -70,10 +71,11 @@ typedef struct QemuDsaBatchTask { QemuDsaTaskStatus status; int batch_size; bool *results; + /* Address of each pages in pages */ + ram_addr_t *addr; QSIMPLEQ_ENTRY(QemuDsaBatchTask) entry; } QemuDsaBatchTask; - /** * @brief Initializes DSA devices. * @@ -106,15 +108,13 @@ void qemu_dsa_cleanup(void); bool qemu_dsa_is_running(void); /** - * @brief Initializes a buffer zero batch task. + * @brief Initializes a buffer zero DSA batch task. * - * @param task A pointer to the batch task to initialize. - * @param results A pointer to an array of zero page checking results. - * @param batch_size The number of DSA tasks in the batch. + * @param batch_size The number of zero page checking tasks in the batch. + * @return A pointer to the zero page checking tasks initialized. */ -void -buffer_zero_batch_task_init(QemuDsaBatchTask *task, - bool *results, int batch_size); +QemuDsaBatchTask * +buffer_zero_batch_task_init(int batch_size); /** * @brief Performs the proper cleanup on a DSA batch task. @@ -139,6 +139,8 @@ buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, #else +typedef struct QemuDsaBatchTask {} QemuDsaBatchTask; + static inline bool qemu_dsa_is_running(void) { return false; @@ -146,19 +148,28 @@ static inline bool qemu_dsa_is_running(void) static inline int qemu_dsa_init(const char *dsa_parameter, Error **errp) { - if (dsa_parameter != NULL && strlen(dsa_parameter) != 0) { - error_setg(errp, "DSA is not supported."); - return -1; - } - - return 0; + error_setg(errp, "DSA accelerator is not enabled."); + return -1; } static inline void qemu_dsa_start(void) {} static inline void qemu_dsa_stop(void) {} -static inline void qemu_dsa_cleanup(void) {} +static inline QemuDsaBatchTask *buffer_zero_batch_task_init(int batch_size) +{ + return NULL; +} + +static inline void buffer_zero_batch_task_destroy(QemuDsaBatchTask *task) {} + +static inline int +buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, + const void **buf, size_t count, size_t len) +{ + error_setg(errp, "DSA accelerator is not enabled."); + return -1; +} #endif diff --git a/migration/multifd.c b/migration/multifd.c index 0b4cbaddfe..6f8edd4b6a 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -13,6 +13,7 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "qemu/rcu.h" +#include "qemu/dsa.h" #include "exec/target_page.h" #include "sysemu/sysemu.h" #include "exec/ramblock.h" @@ -792,6 +793,8 @@ static bool multifd_send_cleanup_channel(MultiFDSendParams *p, Error **errp) p->name = NULL; multifd_pages_clear(p->pages); p->pages = NULL; + buffer_zero_batch_task_destroy(p->dsa_batch_task); + p->dsa_batch_task = NULL; p->packet_len = 0; g_free(p->packet); p->packet = NULL; @@ -1182,6 +1185,7 @@ bool multifd_send_setup(void) qemu_sem_init(&p->sem_sync, 0); p->id = i; p->pages = multifd_pages_init(page_count); + p->dsa_batch_task = buffer_zero_batch_task_init(page_count); if (use_packets) { p->packet_len = sizeof(MultiFDPacket_t) diff --git a/migration/multifd.h b/migration/multifd.h index 0ecd6f47d7..027f57bf4e 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -14,6 +14,7 @@ #define QEMU_MIGRATION_MULTIFD_H #include "ram.h" +#include "qemu/dsa.h" typedef struct MultiFDRecvData MultiFDRecvData; @@ -137,6 +138,8 @@ typedef struct { * pending_job != 0 -> multifd_channel can use it. */ MultiFDPages_t *pages; + /* Zero page checking batch task */ + QemuDsaBatchTask *dsa_batch_task; /* thread local variables. No locking required */ diff --git a/util/dsa.c b/util/dsa.c index 74b9aa1331..5aba1ae23a 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -696,93 +696,81 @@ static void dsa_completion_thread_stop(void *opaque) } /** - * @brief Initializes a buffer zero comparison DSA task. + * @brief Check if DSA is running. * - * @param descriptor A pointer to the DSA task descriptor. - * @param completion A pointer to the DSA task completion record. + * @return True if DSA is running, otherwise false. */ +bool qemu_dsa_is_running(void) +{ + return completion_thread.running; +} + static void -buffer_zero_task_init_int(struct dsa_hw_desc *descriptor, - struct dsa_completion_record *completion) +dsa_globals_init(void) { - descriptor->opcode = DSA_OPCODE_COMPVAL; - descriptor->flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; - descriptor->comp_pattern = (uint64_t)0; - descriptor->completion_addr = (uint64_t)completion; + max_retry_count = DSA_WQ_DEPTH; } /** - * @brief Initializes a buffer zero batch task. + * @brief Initializes DSA devices. + * + * @param dsa_parameter A list of DSA device path from migration parameter. * - * @param task A pointer to the batch task to initialize. - * @param results A pointer to an array of zero page checking results. - * @param batch_size The number of DSA tasks in the batch. + * @return int Zero if successful, otherwise non zero. */ -void -buffer_zero_batch_task_init(QemuDsaBatchTask *task, - bool *results, int batch_size) +int qemu_dsa_init(const char *dsa_parameter, Error **errp) { - int descriptors_size = sizeof(*task->descriptors) * batch_size; - memset(task, 0, sizeof(*task)); - - task->descriptors = - (struct dsa_hw_desc *)qemu_memalign(64, descriptors_size); - memset(task->descriptors, 0, descriptors_size); - task->completions = (struct dsa_completion_record *)qemu_memalign( - 32, sizeof(*task->completions) * batch_size); - task->results = results; - task->batch_size = batch_size; + dsa_globals_init(); - task->batch_completion.status = DSA_COMP_NONE; - task->batch_descriptor.completion_addr = (uint64_t)&task->batch_completion; - /* TODO: Ensure that we never send a batch with count <= 1 */ - task->batch_descriptor.desc_count = 0; - task->batch_descriptor.opcode = DSA_OPCODE_BATCH; - task->batch_descriptor.flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; - task->batch_descriptor.desc_list_addr = (uintptr_t)task->descriptors; - task->status = QEMU_DSA_TASK_READY; - task->group = &dsa_group; - task->device = dsa_device_group_get_next_device(&dsa_group); + return dsa_device_group_init(&dsa_group, dsa_parameter, errp); +} - for (int i = 0; i < task->batch_size; i++) { - buffer_zero_task_init_int(&task->descriptors[i], - &task->completions[i]); +/** + * @brief Start logic to enable using DSA. + * + */ +void qemu_dsa_start(void) +{ + if (dsa_group.num_dsa_devices == 0) { + return; } - - qemu_sem_init(&task->sem_task_complete, 0); - task->completion_callback = buffer_zero_dsa_completion; + if (dsa_group.running) { + return; + } + dsa_device_group_start(&dsa_group); + dsa_completion_thread_init(&completion_thread, &dsa_group); } /** - * @brief Performs the proper cleanup on a DSA batch task. + * @brief Stop the device group and the completion thread. * - * @param task A pointer to the batch task to cleanup. */ -void -buffer_zero_batch_task_destroy(QemuDsaBatchTask *task) +void qemu_dsa_stop(void) { - qemu_vfree(task->descriptors); - qemu_vfree(task->completions); - task->results = NULL; + QemuDsaDeviceGroup *group = &dsa_group; - qemu_sem_destroy(&task->sem_task_complete); + if (!group->running) { + return; + } + + dsa_completion_thread_stop(&completion_thread); + dsa_empty_task_queue(group); } /** - * @brief Resets a buffer zero comparison DSA batch task. + * @brief Clean up system resources created for DSA offloading. * - * @param task A pointer to the batch task. - * @param count The number of DSA tasks this batch task will contain. */ -static void -buffer_zero_batch_task_reset(QemuDsaBatchTask *task, size_t count) +void qemu_dsa_cleanup(void) { - task->batch_completion.status = DSA_COMP_NONE; - task->batch_descriptor.desc_count = count; - task->task_type = QEMU_DSA_BATCH_TASK; - task->status = QEMU_DSA_TASK_READY; + qemu_dsa_stop(); + dsa_device_group_cleanup(&dsa_group); } + +/* Buffer zero comparison DSA task implementations */ +/* =============================================== */ + /** * @brief Sets a buffer zero comparison DSA task. * @@ -817,6 +805,21 @@ buffer_zero_task_reset(QemuDsaBatchTask *task) task->status = QEMU_DSA_TASK_READY; } +/** + * @brief Resets a buffer zero comparison DSA batch task. + * + * @param task A pointer to the batch task. + * @param count The number of DSA tasks this batch task will contain. + */ +static void +buffer_zero_batch_task_reset(QemuDsaBatchTask *task, size_t count) +{ + task->batch_completion.status = DSA_COMP_NONE; + task->batch_descriptor.desc_count = count; + task->task_type = QEMU_DSA_BATCH_TASK; + task->status = QEMU_DSA_TASK_READY; +} + /** * @brief Sets a buffer zero comparison DSA task. * @@ -923,6 +926,7 @@ buffer_zero_dsa_wait(QemuDsaBatchTask *batch_task) * * @param batch_task A pointer to the batch task. */ + static void buffer_zero_cpu_fallback(QemuDsaBatchTask *batch_task) { @@ -966,78 +970,6 @@ buffer_zero_cpu_fallback(QemuDsaBatchTask *batch_task) } } -/** - * @brief Check if DSA is running. - * - * @return True if DSA is running, otherwise false. - */ -bool qemu_dsa_is_running(void) -{ - return completion_thread.running; -} - -static void -dsa_globals_init(void) -{ - max_retry_count = DSA_WQ_DEPTH; -} - -/** - * @brief Initializes DSA devices. - * - * @param dsa_parameter A list of DSA device path from migration parameter. - * - * @return int Zero if successful, otherwise non zero. - */ -int qemu_dsa_init(const char *dsa_parameter, Error **errp) -{ - dsa_globals_init(); - - return dsa_device_group_init(&dsa_group, dsa_parameter, errp); -} - -/** - * @brief Start logic to enable using DSA. - * - */ -void qemu_dsa_start(void) -{ - if (dsa_group.num_dsa_devices == 0) { - return; - } - if (dsa_group.running) { - return; - } - dsa_device_group_start(&dsa_group); - dsa_completion_thread_init(&completion_thread, &dsa_group); -} - -/** - * @brief Stop the device group and the completion thread. - * - */ -void qemu_dsa_stop(void) -{ - QemuDsaDeviceGroup *group = &dsa_group; - - if (!group->running) { - return; - } - - dsa_completion_thread_stop(&completion_thread); - dsa_empty_task_queue(group); -} - -/** - * @brief Clean up system resources created for DSA offloading. - * - */ -void qemu_dsa_cleanup(void) -{ - qemu_dsa_stop(); - dsa_device_group_cleanup(&dsa_group); -} - /** * @brief Performs buffer zero comparison on a DSA batch task synchronously. * @@ -1073,3 +1005,79 @@ buffer_is_zero_dsa_batch_sync(QemuDsaBatchTask *batch_task, return 0; } +/** + * @brief Initializes a buffer zero comparison DSA task. + * + * @param descriptor A pointer to the DSA task descriptor. + * @param completion A pointer to the DSA task completion record. + */ +static void +buffer_zero_task_init_int(struct dsa_hw_desc *descriptor, + struct dsa_completion_record *completion) +{ + descriptor->opcode = DSA_OPCODE_COMPVAL; + descriptor->flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; + descriptor->comp_pattern = (uint64_t)0; + descriptor->completion_addr = (uint64_t)completion; +} + +/** + * @brief Initializes a buffer zero DSA batch task. + * + * @param batch_size The number of zero page checking tasks in the batch. + * @return A pointer to the zero page checking tasks initialized. + */ +QemuDsaBatchTask * +buffer_zero_batch_task_init(int batch_size) +{ + QemuDsaBatchTask *task = qemu_memalign(64, sizeof(QemuDsaBatchTask)); + int descriptors_size = sizeof(*task->descriptors) * batch_size; + + memset(task, 0, sizeof(*task)); + task->addr = g_new0(ram_addr_t, batch_size); + task->results = g_new0(bool, batch_size); + task->batch_size = batch_size; + task->descriptors = + (struct dsa_hw_desc *)qemu_memalign(64, descriptors_size); + memset(task->descriptors, 0, descriptors_size); + task->completions = (struct dsa_completion_record *)qemu_memalign( + 32, sizeof(*task->completions) * batch_size); + + task->batch_completion.status = DSA_COMP_NONE; + task->batch_descriptor.completion_addr = (uint64_t)&task->batch_completion; + /* TODO: Ensure that we never send a batch with count <= 1 */ + task->batch_descriptor.desc_count = 0; + task->batch_descriptor.opcode = DSA_OPCODE_BATCH; + task->batch_descriptor.flags = IDXD_OP_FLAG_RCR | IDXD_OP_FLAG_CRAV; + task->batch_descriptor.desc_list_addr = (uintptr_t)task->descriptors; + task->status = QEMU_DSA_TASK_READY; + task->group = &dsa_group; + task->device = dsa_device_group_get_next_device(&dsa_group); + + for (int i = 0; i < task->batch_size; i++) { + buffer_zero_task_init_int(&task->descriptors[i], + &task->completions[i]); + } + + qemu_sem_init(&task->sem_task_complete, 0); + task->completion_callback = buffer_zero_dsa_completion; + + return task; +} + +/** + * @brief Performs the proper cleanup on a DSA batch task. + * + * @param task A pointer to the batch task to cleanup. + */ +void +buffer_zero_batch_task_destroy(QemuDsaBatchTask *task) +{ + g_free(task->addr); + g_free(task->results); + qemu_vfree(task->descriptors); + qemu_vfree(task->completions); + task->results = NULL; + qemu_sem_destroy(&task->sem_task_complete); + qemu_vfree(task); +} From patchwork Thu Jul 11 22:04:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B0CAC3DA45 for ; Thu, 11 Jul 2024 22:05:34 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1uQ-0001wO-EU; Thu, 11 Jul 2024 18:05:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1uO-0001v4-Hj for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:00 -0400 Received: from mail-oi1-x235.google.com ([2607:f8b0:4864:20::235]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1uM-0000Jq-CY for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:00 -0400 Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3d932f991f6so796585b6e.1 for ; Thu, 11 Jul 2024 15:04:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720735496; x=1721340296; darn=nongnu.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Y30kNx6mYhPmtns0WOVLEJLF7Vyb+wOWww6unPsKfy8=; b=HpIpebykxijvR/OOJx7rYwH1IUHxXjbWwJ23OC1GDbOaE+APDQ+sQojGJ0JDBgl0Ry tPeCP72W0QQPPbfG/zVeBEVKtRprgqM3Q0pjTCJIu7Cz3s4FfmMIYD+QCjhrtMtYxM+f gGcUTRNalsLKyylUnHlRoQla7X5JVGC6HTCMgpHfHQn0jNMLRqQCtknXb3fSiBZXpWGR 32Bvy/BK4hwB0GDSHcwVGgjMKSIJUQ3x9jAUa8EuuRonrZhR6811lV1t1M6NeXMI2QDw fbBa4lisNIx5M5ToW+Zp5NDAC9y4SCV0dqs+fDhvb24S8Mo+L3goMsBGcjEAW76TB9Qe 4Usw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720735496; x=1721340296; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Y30kNx6mYhPmtns0WOVLEJLF7Vyb+wOWww6unPsKfy8=; b=MKc4WyFV3oVHQcY+/xDWQU5mVF9RnMjBSy0X1VngjccDwxDe7x27k9ll1eTFMjQOS6 UPmsxseqkZ066YtfmWNNM+9wetbqADOMtLQ2FwKFGTyyrW+Bkvp/14sfXaDPcK932P4O cOfmhlLJTe3lhT+SiVNvbapFv0XRK59TcpcNJWiSFN5QjRTHgmtFLJx6vKhiYb/SuGfZ 599mzuKofJvOsaTE9BOdmaCNN8V3DE0SbNNPzZ2C0G+LdeVe5hyu6a5cBqFZPAxhlwQz lYI06z6VFsgSCNlZ4ebU67B+vffcEJ6mXbuUIJeavfZirYJE3POI0zla2f5WSD45cjxO AQvA== X-Forwarded-Encrypted: i=1; AJvYcCUzfnf/Yy3b5ATV9Ls/zZGylxAK+2KZYPBrmeQ2yH1IxiX51xWiSPhZQNXqofBGYGaAFzXUKgzbwUnHv+TkrI5aKcCPWO8= X-Gm-Message-State: AOJu0YwjKJKy79wKhqunYWT8Yk16tkSdW1WX9CyhG/AwgXjyH+lMqaRR Ej6Ra4X3JiH9RgFsix2nRzHgQxWG/N6fp1TbMvuWSKCHOBBu7CdYA4zhhZz6cso= X-Google-Smtp-Source: AGHT+IFl3wbDhp8TTN7KnWx2tA+l/HRGxAYk4XoZo5eMyQfIk6S8pcZN54krFIY3nWQhgmUCYioJyg== X-Received: by 2002:a05:6808:198a:b0:3d9:2c5a:c81a with SMTP id 5614622812f47-3d93c0a76e5mr11086807b6e.50.1720735496369; Thu, 11 Jul 2024 15:04:56 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3d93acff4d9sm1218578b6e.5.2024.07.11.15.04.54 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 15:04:56 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 10/13] migration/multifd: Enable DSA offloading in multifd sender path. Date: Thu, 11 Jul 2024 15:04:48 -0700 Message-Id: <20240711220451.19780-1-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::235; envelope-from=yichen.wang@bytedance.com; helo=mail-oi1-x235.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang Multifd sender path gets an array of pages queued by the migration thread. It performs zero page checking on every page in the array. The pages are classfied as either a zero page or a normal page. This change uses Intel DSA to offload the zero page checking from CPU to the DSA accelerator. The sender thread submits a batch of pages to DSA hardware and waits for the DSA completion thread to signal for work completion. Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- include/qemu/dsa.h | 4 +- migration/migration.c | 2 +- migration/multifd-zero-page.c | 100 ++++++++++++++++++++++++++++++++-- migration/multifd.c | 43 ++++++++++++++- migration/multifd.h | 2 +- util/dsa.c | 23 ++++---- 6 files changed, 150 insertions(+), 24 deletions(-) diff --git a/include/qemu/dsa.h b/include/qemu/dsa.h index fd0305a7c7..a3b502ee41 100644 --- a/include/qemu/dsa.h +++ b/include/qemu/dsa.h @@ -83,7 +83,7 @@ typedef struct QemuDsaBatchTask { * * @return int Zero if successful, otherwise non zero. */ -int qemu_dsa_init(const char *dsa_parameter, Error **errp); +int qemu_dsa_init(const strList *dsa_parameter, Error **errp); /** * @brief Start logic to enable using DSA. @@ -146,7 +146,7 @@ static inline bool qemu_dsa_is_running(void) return false; } -static inline int qemu_dsa_init(const char *dsa_parameter, Error **errp) +static inline int qemu_dsa_init(const strList *dsa_parameter, Error **errp) { error_setg(errp, "DSA accelerator is not enabled."); return -1; diff --git a/migration/migration.c b/migration/migration.c index 3dea06d577..085395b900 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -3469,7 +3469,7 @@ static void *migration_thread(void *opaque) object_ref(OBJECT(s)); update_iteration_initial_status(s); - if (!multifd_send_setup()) { + if (!multifd_send_setup(&local_err)) { goto out; } diff --git a/migration/multifd-zero-page.c b/migration/multifd-zero-page.c index e1b8370f88..ffb5611d44 100644 --- a/migration/multifd-zero-page.c +++ b/migration/multifd-zero-page.c @@ -37,25 +37,84 @@ static void swap_page_offset(ram_addr_t *pages_offset, int a, int b) } /** - * multifd_send_zero_page_detect: Perform zero page detection on all pages. + * zero_page_detect_cpu: Perform zero page detection using CPU. * * Sorts normal pages before zero pages in p->pages->offset and updates * p->pages->normal_num. * * @param p A pointer to the send params. */ -void multifd_send_zero_page_detect(MultiFDSendParams *p) +static void zero_page_detect_cpu(MultiFDSendParams *p) { MultiFDPages_t *pages = p->pages; RAMBlock *rb = pages->block; int i = 0; int j = pages->num - 1; - if (!multifd_zero_page_enabled()) { - pages->normal_num = pages->num; + /* + * Sort the page offset array by moving all normal pages to + * the left and all zero pages to the right of the array. + */ + while (i <= j) { + uint64_t offset = pages->offset[i]; + + if (!buffer_is_zero(rb->host + offset, p->page_size)) { + i++; + continue; + } + + swap_page_offset(pages->offset, i, j); + ram_release_page(rb->idstr, offset); + j--; + } + + pages->normal_num = i; +} + + +#ifdef CONFIG_DSA_OPT + +static void swap_result(bool *results, int a, int b) +{ + bool temp; + + if (a == b) { return; } + temp = results[a]; + results[a] = results[b]; + results[b] = temp; +} + +/** + * zero_page_detect_dsa: Perform zero page detection using + * Intel Data Streaming Accelerator (DSA). + * + * Sorts normal pages before zero pages in p->pages->offset and updates + * p->pages->normal_num. + * + * @param p A pointer to the send params. + */ +static void zero_page_detect_dsa(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = p->pages; + RAMBlock *rb = pages->block; + bool *results = p->dsa_batch_task->results; + + for (int i = 0; i < p->pages->num; i++) { + p->dsa_batch_task->addr[i] = + (ram_addr_t)(rb->host + p->pages->offset[i]); + } + + buffer_is_zero_dsa_batch_sync(p->dsa_batch_task, + (const void **)p->dsa_batch_task->addr, + p->pages->num, + p->page_size); + + int i = 0; + int j = pages->num - 1; + /* * Sort the page offset array by moving all normal pages to * the left and all zero pages to the right of the array. @@ -63,11 +122,12 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) while (i <= j) { uint64_t offset = pages->offset[i]; - if (!buffer_is_zero(rb->host + offset, p->page_size)) { + if (!results[i]) { i++; continue; } + swap_result(results, i, j); swap_page_offset(pages->offset, i, j); ram_release_page(rb->idstr, offset); j--; @@ -76,6 +136,15 @@ void multifd_send_zero_page_detect(MultiFDSendParams *p) pages->normal_num = i; } +#else + +static void zero_page_detect_dsa(MultiFDSendParams *p) +{ + exit(1); +} + +#endif + void multifd_recv_zero_page_process(MultiFDRecvParams *p) { for (int i = 0; i < p->zero_num; i++) { @@ -87,3 +156,24 @@ void multifd_recv_zero_page_process(MultiFDRecvParams *p) } } } + +/** + * multifd_send_zero_page_detect: Perform zero page detection on all pages. + * + * @param p A pointer to the send params. + */ +void multifd_send_zero_page_detect(MultiFDSendParams *p) +{ + MultiFDPages_t *pages = p->pages; + + if (!multifd_zero_page_enabled()) { + pages->normal_num = pages->num; + return; + } + + if (qemu_dsa_is_running()) { + zero_page_detect_dsa(p); + } else { + zero_page_detect_cpu(p); + } +} diff --git a/migration/multifd.c b/migration/multifd.c index 6f8edd4b6a..014fee757a 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -817,6 +817,32 @@ static void multifd_send_cleanup_state(void) multifd_send_state = NULL; } +static bool multifd_dsa_setup(MigrationState *s, const char *role, Error **errp) +{ + /* + * Only setup DSA when needed. Currently, DSA is only used for zero page + * detection, which is only needed on sender side. + */ + if (!s || + s->parameters.zero_page_detection != ZERO_PAGE_DETECTION_DSA_ACCEL) { + return true; + } + + const strList *dsa_parameter = migrate_dsa_accel_path(); + if (qemu_dsa_init(dsa_parameter, errp)) { + error_setg(errp, "multifd: %s failed to initialize DSA.", role); + return false; + } + qemu_dsa_start(); + + return true; +} + +static void multifd_dsa_cleanup(void) +{ + qemu_dsa_cleanup(); +} + void multifd_send_shutdown(void) { int i; @@ -827,6 +853,8 @@ void multifd_send_shutdown(void) multifd_send_terminate_threads(); + multifd_dsa_cleanup(); + for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; Error *local_err = NULL; @@ -1156,7 +1184,7 @@ static bool multifd_new_send_channel_create(gpointer opaque, Error **errp) return true; } -bool multifd_send_setup(void) +bool multifd_send_setup(Error **errp) { MigrationState *s = migrate_get_current(); Error *local_err = NULL; @@ -1169,6 +1197,10 @@ bool multifd_send_setup(void) return true; } + if (!multifd_dsa_setup(s, "Sender", errp)) { + return false; + } + thread_count = migrate_multifd_channels(); multifd_send_state = g_malloc0(sizeof(*multifd_send_state)); multifd_send_state->params = g_new0(MultiFDSendParams, thread_count); @@ -1395,6 +1427,7 @@ void multifd_recv_cleanup(void) qemu_thread_join(&p->thread); } } + multifd_dsa_cleanup(); for (i = 0; i < migrate_multifd_channels(); i++) { multifd_recv_cleanup_channel(&multifd_recv_state->params[i]); } @@ -1570,6 +1603,7 @@ int multifd_recv_setup(Error **errp) uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); bool use_packets = multifd_use_packets(); uint8_t i; + int ret; /* * Return successfully if multiFD recv state is already initialised @@ -1579,6 +1613,10 @@ int multifd_recv_setup(Error **errp) return 0; } + if (!multifd_dsa_setup(NULL, "Receiver", errp)) { + return -1; + } + thread_count = migrate_multifd_channels(); multifd_recv_state = g_malloc0(sizeof(*multifd_recv_state)); multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count); @@ -1617,13 +1655,12 @@ int multifd_recv_setup(Error **errp) for (i = 0; i < thread_count; i++) { MultiFDRecvParams *p = &multifd_recv_state->params[i]; - int ret; - ret = multifd_recv_state->ops->recv_setup(p, errp); if (ret) { return ret; } } + return 0; } diff --git a/migration/multifd.h b/migration/multifd.h index 027f57bf4e..871e3aa063 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -18,7 +18,7 @@ typedef struct MultiFDRecvData MultiFDRecvData; -bool multifd_send_setup(void); +bool multifd_send_setup(Error **errp); void multifd_send_shutdown(void); void multifd_send_channel_created(void); int multifd_recv_setup(Error **errp); diff --git a/util/dsa.c b/util/dsa.c index 5aba1ae23a..44b1130a51 100644 --- a/util/dsa.c +++ b/util/dsa.c @@ -116,27 +116,27 @@ dsa_device_cleanup(QemuDsaDevice *instance) */ static int dsa_device_group_init(QemuDsaDeviceGroup *group, - const char *dsa_parameter, + const strList *dsa_parameter, Error **errp) { - if (dsa_parameter == NULL || strlen(dsa_parameter) == 0) { - return 0; + if (dsa_parameter == NULL) { + /* HACKING ALERT. */ + /* return 0; */ + dsa_parameter = &(strList) { + .value = (char *)"/dev/dsa/wq0.0", .next = NULL + }; } int ret = 0; - char *local_dsa_parameter = g_strdup(dsa_parameter); const char *dsa_path[MAX_DSA_DEVICES]; int num_dsa_devices = 0; - char delim[2] = " "; - char *current_dsa_path = strtok(local_dsa_parameter, delim); - - while (current_dsa_path != NULL) { - dsa_path[num_dsa_devices++] = current_dsa_path; + while (dsa_parameter) { + dsa_path[num_dsa_devices++] = dsa_parameter->value; if (num_dsa_devices == MAX_DSA_DEVICES) { break; } - current_dsa_path = strtok(NULL, delim); + dsa_parameter = dsa_parameter->next; } group->dsa_devices = @@ -161,7 +161,6 @@ dsa_device_group_init(QemuDsaDeviceGroup *group, } exit: - g_free(local_dsa_parameter); return ret; } @@ -718,7 +717,7 @@ dsa_globals_init(void) * * @return int Zero if successful, otherwise non zero. */ -int qemu_dsa_init(const char *dsa_parameter, Error **errp) +int qemu_dsa_init(const strList *dsa_parameter, Error **errp) { dsa_globals_init(); From patchwork Thu Jul 11 22:04:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2404C3DA4A for ; Thu, 11 Jul 2024 22:05:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1uc-0002CO-Ew; Thu, 11 Jul 2024 18:05:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1uY-0002A1-7S for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:10 -0400 Received: from mail-oi1-x231.google.com ([2607:f8b0:4864:20::231]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1uO-0000KR-J1 for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:08 -0400 Received: by mail-oi1-x231.google.com with SMTP id 5614622812f47-3d93147ac6bso774956b6e.0 for ; Thu, 11 Jul 2024 15:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720735498; x=1721340298; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=duRwLvK2JsLh0/v9rntXL0ihMvEEQ5jod3rzDyBUOeM=; b=G5zFqvQkCDvMNA05B7nw3rJFbT8AWeo7K9LU8lQYIj2oQRtuNzGS7gpKSdaerG+DfL dm+X4n20IDZvRg9wrb/qDIp3wx1YkLZA6fihDYJU2k/TQlk/DUPsZpWUA+rp1ROvwWi0 3ghnNMy3hFtWrmS/aurkEqrFrUeGGAkNvAaiQARUv3gJpd/BaKQ4jjWHUfuwQgJloKCv 4/Y67Dw+6xFylsFDnKM+X9cNxWBcgDKd+/yLR1xNwvQBmX6ofekYleU0J62Gct30XJbj YjodyjVxh9ULRql8KtU+Snr9i/e2aiGrh6NEJVGZsyg29/4Nr8emkBoe46cXHAuiVuF2 p8AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720735498; x=1721340298; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=duRwLvK2JsLh0/v9rntXL0ihMvEEQ5jod3rzDyBUOeM=; b=v27FXIDyq8O+Y0UUm3eF6sbfEn6q+hRz+MvO3Ucx8KM9EuGIeF6UkJwfdUNZPSRCap pHDy2GJEPKv6/rxjsjPXhfgnIp2meTX627CJy8r158dYy+t2ENbm3IOhYqP2+3uOLi5M yHhc7TVKT9f0QpyZJ3Ti+dhymJwOm7VpxBLeuMa6XbS/wYjgtPmFmhm/5dwA9W3O2CnG X8vr8cXlPhJWamHIu1KjGy0gbvhc8fNmynDlAbIFOSJohSjo41wQGozOADdWTQn7inF6 atzWnwKx6NgXXb7oMuiGHQarTnQaMr/kX92e079E0xgYPV1fT91ETLT9fhtsEdaQ7bXl ZFlQ== X-Forwarded-Encrypted: i=1; AJvYcCW5MTrfTuaMzrZ52i8upnsiMN++82HLRFrEuECuTH1AdumjgC7CODmMEwyWbSAJ1VhmM+Sb1PmOPQZj5ANLZnYC4oqecIc= X-Gm-Message-State: AOJu0YwmeLKLzdtXzaX8TXS0nQW4r6gBMHHpLRwTszOEqpFABWwZ4OSv yyU1HSuaxMmYYTgMy0Vz3opyXJmefymwwnD6crl6KC1GTxfuj042/Jb6B7V2vZw= X-Google-Smtp-Source: AGHT+IHyJk5T5vJ5C6FD4TFWC96IsECCCnb7x/G9UilZvOQr87ajc5FezEQqYiq9KoXOXkBDs8TBwg== X-Received: by 2002:a05:6808:1312:b0:3d9:243a:7ae8 with SMTP id 5614622812f47-3d93c08c8cfmr10753026b6e.39.1720735498480; Thu, 11 Jul 2024 15:04:58 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3d93acff4d9sm1218578b6e.5.2024.07.11.15.04.56 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 15:04:58 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" Subject: [PATCH v5 11/13] migration/multifd: Add migration option set packet size. Date: Thu, 11 Jul 2024 15:04:49 -0700 Message-Id: <20240711220451.19780-2-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711220451.19780-1-yichen.wang@bytedance.com> References: <20240711220451.19780-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::231; envelope-from=yichen.wang@bytedance.com; helo=mail-oi1-x231.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang During live migration, if the latency between sender and receiver is high and bandwidth is also high (a long and fat pipe), using a bigger packet size can help reduce migration total time. The current multifd packet size is 128 * 4kb. In addition, Intel DSA offloading performs better with a large batch task. This change adds an option to set the packet size, which is also useful for performance tunin. Both sender and receiver needs to set the same packet size for things to work. Set the option: migrate_set_parameter multifd-packet-size 4190208 Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- migration/migration-hmp-cmds.c | 7 +++++++ migration/multifd-zlib.c | 6 ++++-- migration/multifd-zstd.c | 6 ++++-- migration/multifd.c | 6 ++++-- migration/multifd.h | 3 --- migration/options.c | 38 ++++++++++++++++++++++++++++++++++ migration/options.h | 1 + qapi/migration.json | 21 ++++++++++++++++--- 8 files changed, 76 insertions(+), 12 deletions(-) diff --git a/migration/migration-hmp-cmds.c b/migration/migration-hmp-cmds.c index c422db4ecd..27ba0ce79a 100644 --- a/migration/migration-hmp-cmds.c +++ b/migration/migration-hmp-cmds.c @@ -292,6 +292,9 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict) monitor_printf(mon, "%s: %u ms\n", MigrationParameter_str(MIGRATION_PARAMETER_X_CHECKPOINT_DELAY), params->x_checkpoint_delay); + monitor_printf(mon, "%s: %" PRIu64 "\n", + MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_PACKET_SIZE), + params->multifd_packet_size); monitor_printf(mon, "%s: %u\n", MigrationParameter_str(MIGRATION_PARAMETER_MULTIFD_CHANNELS), params->multifd_channels); @@ -576,6 +579,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_dsa_accel_path = true; visit_type_strList(v, param, &p->dsa_accel_path, &err); break; + case MIGRATION_PARAMETER_MULTIFD_PACKET_SIZE: + p->has_multifd_packet_size = true; + visit_type_size(v, param, &p->multifd_packet_size, &err); + break; case MIGRATION_PARAMETER_MULTIFD_CHANNELS: p->has_multifd_channels = true; visit_type_uint8(v, param, &p->multifd_channels, &err); diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c index 2ced69487e..bd900fe575 100644 --- a/migration/multifd-zlib.c +++ b/migration/multifd-zlib.c @@ -49,6 +49,7 @@ static int zlib_send_setup(MultiFDSendParams *p, Error **errp) struct zlib_data *z = g_new0(struct zlib_data, 1); z_stream *zs = &z->zs; const char *err_msg; + uint64_t multifd_packet_size = migrate_multifd_packet_size(); zs->zalloc = Z_NULL; zs->zfree = Z_NULL; @@ -58,7 +59,7 @@ static int zlib_send_setup(MultiFDSendParams *p, Error **errp) goto err_free_z; } /* This is the maximum size of the compressed buffer */ - z->zbuff_len = compressBound(MULTIFD_PACKET_SIZE); + z->zbuff_len = compressBound(multifd_packet_size); z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { err_msg = "out of memory for zbuff"; @@ -200,6 +201,7 @@ out: */ static int zlib_recv_setup(MultiFDRecvParams *p, Error **errp) { + uint64_t multifd_packet_size = migrate_multifd_packet_size(); struct zlib_data *z = g_new0(struct zlib_data, 1); z_stream *zs = &z->zs; @@ -214,7 +216,7 @@ static int zlib_recv_setup(MultiFDRecvParams *p, Error **errp) return -1; } /* To be safe, we reserve twice the size of the packet */ - z->zbuff_len = MULTIFD_PACKET_SIZE * 2; + z->zbuff_len = multifd_packet_size * 2; z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { inflateEnd(zs); diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c index ca17b7e310..8aaa7363be 100644 --- a/migration/multifd-zstd.c +++ b/migration/multifd-zstd.c @@ -49,6 +49,7 @@ struct zstd_data { */ static int zstd_send_setup(MultiFDSendParams *p, Error **errp) { + uint64_t multifd_packet_size = migrate_multifd_packet_size(); struct zstd_data *z = g_new0(struct zstd_data, 1); int res; @@ -68,7 +69,7 @@ static int zstd_send_setup(MultiFDSendParams *p, Error **errp) return -1; } /* This is the maximum size of the compressed buffer */ - z->zbuff_len = ZSTD_compressBound(MULTIFD_PACKET_SIZE); + z->zbuff_len = ZSTD_compressBound(multifd_packet_size); z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { ZSTD_freeCStream(z->zcs); @@ -188,6 +189,7 @@ out: */ static int zstd_recv_setup(MultiFDRecvParams *p, Error **errp) { + uint64_t multifd_packet_size = migrate_multifd_packet_size(); struct zstd_data *z = g_new0(struct zstd_data, 1); int ret; @@ -209,7 +211,7 @@ static int zstd_recv_setup(MultiFDRecvParams *p, Error **errp) } /* To be safe, we reserve twice the size of the packet */ - z->zbuff_len = MULTIFD_PACKET_SIZE * 2; + z->zbuff_len = multifd_packet_size * 2; z->zbuff = g_try_malloc(z->zbuff_len); if (!z->zbuff) { ZSTD_freeDStream(z->zds); diff --git a/migration/multifd.c b/migration/multifd.c index 014fee757a..87ed421364 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -1189,7 +1189,8 @@ bool multifd_send_setup(Error **errp) MigrationState *s = migrate_get_current(); Error *local_err = NULL; int thread_count, ret = 0; - uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + uint32_t page_count = + migrate_multifd_packet_size() / qemu_target_page_size(); bool use_packets = multifd_use_packets(); uint8_t i; @@ -1600,7 +1601,8 @@ static void *multifd_recv_thread(void *opaque) int multifd_recv_setup(Error **errp) { int thread_count; - uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + uint32_t page_count = + migrate_multifd_packet_size() / qemu_target_page_size(); bool use_packets = multifd_use_packets(); uint8_t i; int ret; diff --git a/migration/multifd.h b/migration/multifd.h index 871e3aa063..63cec33c61 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -44,9 +44,6 @@ MultiFDRecvData *multifd_get_recv_data(void); #define MULTIFD_FLAG_QPL (4 << 1) #define MULTIFD_FLAG_UADK (8 << 1) -/* This value needs to be a multiple of qemu_target_page_size() */ -#define MULTIFD_PACKET_SIZE (512 * 1024) - typedef struct { uint32_t magic; uint32_t version; diff --git a/migration/options.c b/migration/options.c index f839493016..1417fa6ab0 100644 --- a/migration/options.c +++ b/migration/options.c @@ -73,6 +73,12 @@ #define DEFAULT_MIGRATE_ANNOUNCE_ROUNDS 5 #define DEFAULT_MIGRATE_ANNOUNCE_STEP 100 +/* + * Parameter for multifd packet size. + */ +#define DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE (128 * 4 * 1024) +#define MAX_MIGRATE_MULTIFD_PACKET_SIZE (1023 * 4 * 1024) + #define DEFINE_PROP_MIG_CAP(name, x) \ DEFINE_PROP_BOOL(name, MigrationState, capabilities[x], false) @@ -167,6 +173,9 @@ Property migration_properties[] = { /* parameters.dsa_accel_path, qdev_prop_string, char *), */ /* DEFINE_PROP_STRING("dsa-accel-path", MigrationState, */ /* parameters.dsa_accel_path), */ + DEFINE_PROP_SIZE("multifd-packet-size", MigrationState, + parameters.multifd_packet_size, + DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE), /* Migration capabilities */ DEFINE_PROP_MIG_CAP("x-xbzrle", MIGRATION_CAPABILITY_XBZRLE), @@ -777,6 +786,13 @@ int migrate_multifd_channels(void) return s->parameters.multifd_channels; } +uint64_t migrate_multifd_packet_size(void) +{ + MigrationState *s = migrate_get_current(); + + return s->parameters.multifd_packet_size; +} + MultiFDCompression migrate_multifd_compression(void) { MigrationState *s = migrate_get_current(); @@ -898,6 +914,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->downtime_limit = s->parameters.downtime_limit; params->has_x_checkpoint_delay = true; params->x_checkpoint_delay = s->parameters.x_checkpoint_delay; + params->has_multifd_packet_size = true; + params->multifd_packet_size = s->parameters.multifd_packet_size; params->has_multifd_channels = true; params->multifd_channels = s->parameters.multifd_channels; params->has_multifd_compression = true; @@ -957,6 +975,7 @@ void migrate_params_init(MigrationParameters *params) params->has_max_bandwidth = true; params->has_downtime_limit = true; params->has_x_checkpoint_delay = true; + params->has_multifd_packet_size = true; params->has_multifd_channels = true; params->has_multifd_compression = true; params->has_multifd_zlib_level = true; @@ -1038,6 +1057,19 @@ bool migrate_params_check(MigrationParameters *params, Error **errp) /* x_checkpoint_delay is now always positive */ + if (params->has_multifd_packet_size && + ((params->multifd_packet_size < DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE) || + (params->multifd_packet_size > MAX_MIGRATE_MULTIFD_PACKET_SIZE) || + (params->multifd_packet_size % qemu_target_page_size() != 0))) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, + "multifd_packet_size", + "an integer in the range of " + stringify(DEFAULT_MIGRATE_MULTIFD_PACKET_SIZE) + " to "stringify(MAX_MIGRATE_MULTIFD_PACKET_SIZE)", " + "and must be a multiple of guest VM's page size."); + return false; + } + if (params->has_multifd_channels && (params->multifd_channels < 1)) { error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_channels", @@ -1219,6 +1251,9 @@ static void migrate_params_test_apply(MigrateSetParameters *params, dest->x_checkpoint_delay = params->x_checkpoint_delay; } + if (params->has_multifd_packet_size) { + dest->multifd_packet_size = params->multifd_packet_size; + } if (params->has_multifd_channels) { dest->multifd_channels = params->multifd_channels; } @@ -1344,6 +1379,9 @@ static void migrate_params_apply(MigrateSetParameters *params, Error **errp) colo_checkpoint_delay_set(); } + if (params->has_multifd_packet_size) { + s->parameters.multifd_packet_size = params->multifd_packet_size; + } if (params->has_multifd_channels) { s->parameters.multifd_channels = params->multifd_channels; } diff --git a/migration/options.h b/migration/options.h index 78b9e4080b..b37cffc887 100644 --- a/migration/options.h +++ b/migration/options.h @@ -86,6 +86,7 @@ const char *migrate_tls_hostname(void); uint64_t migrate_xbzrle_cache_size(void); ZeroPageDetection migrate_zero_page_detection(void); const strList *migrate_dsa_accel_path(void); +uint64_t migrate_multifd_packet_size(void); /* parameters helpers */ diff --git a/qapi/migration.json b/qapi/migration.json index ff41780347..1a9dc5d74c 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -839,6 +839,10 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @multifd-packet-size: Packet size in bytes used to migrate data. +# The value needs to be a multiple of guest page size. +# The default value is 524288 and max value is 4190208. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -864,7 +868,8 @@ 'vcpu-dirty-limit', 'mode', 'zero-page-detection', - 'direct-io'] } + 'direct-io', + 'multifd-packet-size'] } ## # @MigrateSetParameters: @@ -1020,6 +1025,10 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @multifd-packet-size: Packet size in bytes used to migrate data. +# The value needs to be a multiple of guest page size. +# The default value is 524288 and max value is 4190208. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1061,7 +1070,8 @@ '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', '*direct-io': 'bool', - '*dsa-accel-path': ['str'] } } + '*dsa-accel-path': ['str'], + '*multifd-packet-size' : 'uint64'} } ## # @migrate-set-parameters: @@ -1231,6 +1241,10 @@ # only has effect if the @mapped-ram capability is enabled. # (Since 9.1) # +# @multifd-packet-size: Packet size in bytes used to migrate data. +# The value needs to be a multiple of guest page size. +# The default value is 524288 and max value is 4190208. (Since 9.2) +# # Features: # # @unstable: Members @x-checkpoint-delay and @@ -1269,7 +1283,8 @@ '*mode': 'MigMode', '*zero-page-detection': 'ZeroPageDetection', '*direct-io': 'bool', - '*dsa-accel-path': ['str'] } } + '*dsa-accel-path': ['str'], + '*multifd-packet-size': 'uint64'} } ## # @query-migrate-parameters: From patchwork Thu Jul 11 22:04:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F00BFC3DA4A for ; Thu, 11 Jul 2024 22:06:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1uj-0002Rc-T8; Thu, 11 Jul 2024 18:05:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1ug-0002N2-HH for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:18 -0400 Received: from mail-oi1-x230.google.com ([2607:f8b0:4864:20::230]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1uQ-0000L8-8U for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:10 -0400 Received: by mail-oi1-x230.google.com with SMTP id 5614622812f47-3d9ddfbbc58so805012b6e.2 for ; Thu, 11 Jul 2024 15:05:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720735501; x=1721340301; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VJ68XmNRUjWRQT4cS7/rlxhPmFKSGh8Da+hZaqurBZ8=; b=R90E3VBRayagvmS8sxZ+49LifYw1q9ScgxzCxNKJK3oNY0yGacfGceXMuARXH/SqJG +BiW8MsFTBzO4CSCz5O4zsqPnjNwN4aG+cpFXMP4PdjQCRHXbDPaJLnSsSpkPvsgcGgq mUhkukrgbY19FQPR40wlHF7Uw0WG7ducancibVR99WcArTWZgmIZ8EoxdOg75vbbr8tZ +nPIbnEIyNIySGh2Zucr+fzXbysEUqP0Vos47m4Zd509Iqu5VpVOY89uYoAo8iFk6NDi FfBIH3Dhddei584lzwXEKEjbaBkmgoyjZf/MlxJ/FNjQbe03QIfftiYdHoKVTzvbdKM8 6YhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720735501; x=1721340301; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VJ68XmNRUjWRQT4cS7/rlxhPmFKSGh8Da+hZaqurBZ8=; b=QLvQe68Fu/InCXcZG8A8/s8Es6RTaCQFxRM5Lz+y4oljpobea1w8vBy0GX5Ybvej8k KWZS1Xa+DBR8/eoyqxGjbD12WGrbCqv3iA3UkMZAW4ORhVne6sYNdg5PpF+Bj2g0Flzt f6QdTIxx3CjzcGvMG81AZRq28cKp1ar6OEWe6A/pu/zVFPXSjXrE9sQo5IeJLDTa9mb6 6hql0vHK2NpcfrcAiCNBFwkpU9LmdWJG4sL86HaPBpj7+cDr4v2d2D/xr8PHgAlevXPI 6SkQqV8ZWB7p6fxkkjS8jyHuNOK4Tf4bE7U73ewlh9sJI4jLxqPI0ZC9DqW1aKdL4eU6 wgyg== X-Forwarded-Encrypted: i=1; AJvYcCVEiwatImrluSXKaLBzaUPWCNHWZNOUI8zs/APsrXxD5Adif01P0x56I4/a94P4/ZvS6X33wXVWDojJJ8QVEgtUwX5/IFY= X-Gm-Message-State: AOJu0YzkcC5vqxsyrOwuCkL6MHcjweQBwfXEp3R+numGqX+FFFFOfFti NZR8tQtuSUjngyeWah2AYAgvh8Zgi61uxJ0QJe4wVPn/asQt9MiB+hLdbgsgr+M= X-Google-Smtp-Source: AGHT+IEydHSpwAcXmjBpWdVvkuoH+rjr5/xfXqx8Kgrl2CWb4nPWuCiwILLQ7YdwODc69O+dX+oBLg== X-Received: by 2002:a05:6808:218a:b0:3d5:5c77:fc2f with SMTP id 5614622812f47-3d93c097a16mr13345967b6e.48.1720735500857; Thu, 11 Jul 2024 15:05:00 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3d93acff4d9sm1218578b6e.5.2024.07.11.15.04.58 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 15:05:00 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v5 12/13] util/dsa: Add unit test coverage for Intel DSA task submission and completion. Date: Thu, 11 Jul 2024 15:04:50 -0700 Message-Id: <20240711220451.19780-3-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711220451.19780-1-yichen.wang@bytedance.com> References: <20240711220451.19780-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::230; envelope-from=yichen.wang@bytedance.com; helo=mail-oi1-x230.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Test DSA start and stop path. * Test DSA configure and cleanup path. * Test DSA task submission and completion path. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- tests/unit/meson.build | 6 + tests/unit/test-dsa.c | 503 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 509 insertions(+) create mode 100644 tests/unit/test-dsa.c diff --git a/tests/unit/meson.build b/tests/unit/meson.build index 26c109c968..1d4d48898b 100644 --- a/tests/unit/meson.build +++ b/tests/unit/meson.build @@ -49,6 +49,12 @@ tests = { 'test-interval-tree': [], } +if config_host_data.get('CONFIG_DSA_OPT') + tests += { + 'test-dsa': [], + } +endif + if have_system or have_tools tests += { 'test-qmp-event': [testqapi], diff --git a/tests/unit/test-dsa.c b/tests/unit/test-dsa.c new file mode 100644 index 0000000000..181a547528 --- /dev/null +++ b/tests/unit/test-dsa.c @@ -0,0 +1,503 @@ +/* + * Test DSA functions. + * + * Copyright (C) Bytedance Ltd. + * + * Authors: + * Hao Xiang + * Bryan Zhang + * Yichen Wang + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/host-utils.h" + +#include "qemu/cutils.h" +#include "qemu/memalign.h" +#include "qemu/dsa.h" + +/* + * TODO Communicate that DSA must be configured to support this batch size. + * TODO Alternatively, poke the DSA device to figure out batch size. + */ +#define batch_size 128 +#define page_size 4096 + +#define oversized_batch_size (batch_size + 1) +#define num_devices 2 +#define max_buffer_size (64 * 1024) + +/* TODO Make these not-hardcoded. */ +static const strList path1[] = { + {.value = (char *)"/dev/dsa/wq4.0", .next = NULL} +}; +static const strList path2[] = { + {.value = (char *)"/dev/dsa/wq4.0", .next = (strList*)&path2[1]}, + {.value = (char *)"/dev/dsa/wq4.1", .next = NULL} +}; + +static Error **errp; + +static QemuDsaBatchTask *task; + +/* A helper for running a single task and checking for correctness. */ +static void do_single_task(void) +{ + task = buffer_zero_batch_task_init(batch_size); + char buf[page_size]; + char *ptr = buf; + + buffer_is_zero_dsa_batch_sync(task, + (const void **)&ptr, + 1, + page_size); + g_assert(task->results[0] == buffer_is_zero(buf, page_size)); + + buffer_zero_batch_task_destroy(task); +} + +static void test_single_zero(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size]; + char *ptr = buf; + + memset(buf, 0x0, page_size); + buffer_is_zero_dsa_batch_sync(task, + (const void **)&ptr, + 1, page_size); + g_assert(task->results[0]); + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_single_zero_async(void) +{ + test_single_zero(); +} + +static void test_single_nonzero(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size]; + char *ptr = buf; + + memset(buf, 0x1, page_size); + buffer_is_zero_dsa_batch_sync(task, + (const void **)&ptr, + 1, page_size); + g_assert(!task->results[0]); + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_single_nonzero_async(void) +{ + test_single_nonzero(); +} + +/* count == 0 should return quickly without calling into DSA. */ +static void test_zero_count_async(void) +{ + char buf[page_size]; + buffer_is_zero_dsa_batch_sync(task, + (const void **)&buf, + 0, + page_size); +} + +static void test_null_task_async(void) +{ + if (g_test_subprocess()) { + g_assert(!qemu_dsa_init(path1, errp)); + + char buf[page_size * batch_size]; + char *addrs[batch_size]; + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf + (page_size * i); + } + + buffer_is_zero_dsa_batch_sync(NULL, (const void **)addrs, + batch_size, + page_size); + } else { + g_test_trap_subprocess(NULL, 0, 0); + g_test_trap_assert_failed(); + } +} + +static void test_oversized_batch(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size * oversized_batch_size]; + char *addrs[batch_size]; + for (int i = 0; i < oversized_batch_size; i++) { + addrs[i] = buf + (page_size * i); + } + + int ret = buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + oversized_batch_size, + page_size); + g_assert(ret != 0); + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_oversized_batch_async(void) +{ + test_oversized_batch(); +} + +static void test_zero_len_async(void) +{ + if (g_test_subprocess()) { + g_assert(!qemu_dsa_init(path1, errp)); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size]; + + buffer_is_zero_dsa_batch_sync(task, + (const void **)&buf, + 1, + 0); + + buffer_zero_batch_task_destroy(task); + } else { + g_test_trap_subprocess(NULL, 0, 0); + g_test_trap_assert_failed(); + } +} + +static void test_null_buf_async(void) +{ + if (g_test_subprocess()) { + g_assert(!qemu_dsa_init(path1, errp)); + + task = buffer_zero_batch_task_init(batch_size); + + buffer_is_zero_dsa_batch_sync(task, NULL, 1, page_size); + + buffer_zero_batch_task_destroy(task); + } else { + g_test_trap_subprocess(NULL, 0, 0); + g_test_trap_assert_failed(); + } +} + +static void test_batch(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + task = buffer_zero_batch_task_init(batch_size); + + char buf[page_size * batch_size]; + char *addrs[batch_size]; + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf + (page_size * i); + } + + /* + * Using whatever is on the stack is somewhat random. + * Manually set some pages to zero and some to nonzero. + */ + memset(buf + 0, 0, page_size * 10); + memset(buf + (10 * page_size), 0xff, page_size * 10); + + buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + batch_size, + page_size); + + bool is_zero; + for (int i = 0; i < batch_size; i++) { + is_zero = buffer_is_zero((const void *)&buf[page_size * i], page_size); + g_assert(task->results[i] == is_zero); + } + + buffer_zero_batch_task_destroy(task); + + qemu_dsa_cleanup(); +} + +static void test_batch_async(void) +{ + test_batch(); +} + +static void test_page_fault(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + char *buf[2]; + int prot = PROT_READ | PROT_WRITE; + int flags = MAP_SHARED | MAP_ANON; + buf[0] = (char *)mmap(NULL, page_size * batch_size, prot, flags, -1, 0); + assert(buf[0] != MAP_FAILED); + buf[1] = (char *)malloc(page_size * batch_size); + assert(buf[1] != NULL); + + for (int j = 0; j < 2; j++) { + task = buffer_zero_batch_task_init(batch_size); + + char *addrs[batch_size]; + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf[j] + (page_size * i); + } + + buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + batch_size, + page_size); + + bool is_zero; + for (int i = 0; i < batch_size; i++) { + is_zero = buffer_is_zero((const void *)&buf[j][page_size * i], + page_size); + g_assert(task->results[i] == is_zero); + } + buffer_zero_batch_task_destroy(task); + } + + assert(!munmap(buf[0], page_size * batch_size)); + free(buf[1]); + qemu_dsa_cleanup(); +} + +static void test_various_buffer_sizes(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + char *buf = malloc(max_buffer_size * batch_size); + char *addrs[batch_size]; + + for (int len = 16; len <= max_buffer_size; len *= 2) { + task = buffer_zero_batch_task_init(batch_size); + + for (int i = 0; i < batch_size; i++) { + addrs[i] = buf + (len * i); + } + + buffer_is_zero_dsa_batch_sync(task, + (const void **)addrs, + batch_size, + len); + + bool is_zero; + for (int j = 0; j < batch_size; j++) { + is_zero = buffer_is_zero((const void *)&buf[len * j], len); + g_assert(task->results[j] == is_zero); + } + + buffer_zero_batch_task_destroy(task); + } + + free(buf); + + qemu_dsa_cleanup(); +} + +static void test_various_buffer_sizes_async(void) +{ + test_various_buffer_sizes(); +} + +static void test_double_start_stop(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + /* Double start */ + qemu_dsa_start(); + qemu_dsa_start(); + g_assert(qemu_dsa_is_running()); + do_single_task(); + + /* Double stop */ + qemu_dsa_stop(); + g_assert(!qemu_dsa_is_running()); + qemu_dsa_stop(); + g_assert(!qemu_dsa_is_running()); + + /* Restart */ + qemu_dsa_start(); + g_assert(qemu_dsa_is_running()); + do_single_task(); + qemu_dsa_cleanup(); +} + +static void test_is_running(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + + g_assert(!qemu_dsa_is_running()); + qemu_dsa_start(); + g_assert(qemu_dsa_is_running()); + qemu_dsa_stop(); + g_assert(!qemu_dsa_is_running()); + qemu_dsa_cleanup(); +} + +static void test_multiple_engines(void) +{ + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_start(); + + QemuDsaBatchTask *tasks[num_devices]; + char bufs[num_devices][page_size * batch_size]; + char *addrs[num_devices][batch_size]; + + /* + * This is a somewhat implementation-specific way + * of testing that the tasks have unique engines + * assigned to them. + */ + tasks[0] = buffer_zero_batch_task_init(batch_size); + tasks[1] = buffer_zero_batch_task_init(batch_size); + g_assert(tasks[0]->device != tasks[1]->device); + + for (int i = 0; i < num_devices; i++) { + for (int j = 0; j < batch_size; j++) { + addrs[i][j] = bufs[i] + (page_size * j); + } + + buffer_is_zero_dsa_batch_sync(tasks[i], + (const void **)addrs[i], + batch_size, page_size); + + bool is_zero; + for (int j = 0; j < batch_size; j++) { + is_zero = buffer_is_zero((const void *)&bufs[i][page_size * j], + page_size); + g_assert(tasks[i]->results[j] == is_zero); + } + } + + buffer_zero_batch_task_destroy(tasks[0]); + buffer_zero_batch_task_destroy(tasks[1]); + + qemu_dsa_cleanup(); +} + +static void test_configure_dsa_twice(void) +{ + g_assert(!qemu_dsa_init(path2, errp)); + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_start(); + do_single_task(); + qemu_dsa_cleanup(); +} + +static void test_configure_dsa_bad_path(void) +{ + const strList *bad_path = &(strList) { + .value = (char *)"/not/a/real/path", .next = NULL + }; + g_assert(qemu_dsa_init(bad_path, errp)); +} + +static void test_cleanup_before_configure(void) +{ + qemu_dsa_cleanup(); + g_assert(!qemu_dsa_init(path2, errp)); +} + +static void test_configure_dsa_num_devices(void) +{ + g_assert(!qemu_dsa_init(path1, errp)); + qemu_dsa_start(); + + do_single_task(); + qemu_dsa_stop(); + qemu_dsa_cleanup(); +} + +static void test_cleanup_twice(void) +{ + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_cleanup(); + qemu_dsa_cleanup(); + + g_assert(!qemu_dsa_init(path2, errp)); + qemu_dsa_start(); + do_single_task(); + qemu_dsa_cleanup(); +} + +static int check_test_setup(void) +{ + const strList *path[2] = {path1, path2}; + for (int i = 0; i < sizeof(path) / sizeof(strList *); i++) { + if (qemu_dsa_init(path[i], errp)) { + return -1; + } + qemu_dsa_cleanup(); + } + return 0; +} + +int main(int argc, char **argv) +{ + g_test_init(&argc, &argv, NULL); + + if (check_test_setup() != 0) { + /* + * This test requires extra setup. The current + * setup is not correct. Just skip this test + * for now. + */ + exit(0); + } + + if (num_devices > 1) { + g_test_add_func("/dsa/multiple_engines", test_multiple_engines); + } + + g_test_add_func("/dsa/async/batch", test_batch_async); + g_test_add_func("/dsa/async/various_buffer_sizes", + test_various_buffer_sizes_async); + g_test_add_func("/dsa/async/null_buf", test_null_buf_async); + g_test_add_func("/dsa/async/zero_len", test_zero_len_async); + g_test_add_func("/dsa/async/oversized_batch", test_oversized_batch_async); + g_test_add_func("/dsa/async/zero_count", test_zero_count_async); + g_test_add_func("/dsa/async/single_zero", test_single_zero_async); + g_test_add_func("/dsa/async/single_nonzero", test_single_nonzero_async); + g_test_add_func("/dsa/async/null_task", test_null_task_async); + g_test_add_func("/dsa/async/page_fault", test_page_fault); + + g_test_add_func("/dsa/double_start_stop", test_double_start_stop); + g_test_add_func("/dsa/is_running", test_is_running); + + g_test_add_func("/dsa/configure_dsa_twice", test_configure_dsa_twice); + g_test_add_func("/dsa/configure_dsa_bad_path", test_configure_dsa_bad_path); + g_test_add_func("/dsa/cleanup_before_configure", + test_cleanup_before_configure); + g_test_add_func("/dsa/configure_dsa_num_devices", + test_configure_dsa_num_devices); + g_test_add_func("/dsa/cleanup_twice", test_cleanup_twice); + + return g_test_run(); +} From patchwork Thu Jul 11 22:04:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yichen Wang X-Patchwork-Id: 13731121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4CE39C3DA4D for ; Thu, 11 Jul 2024 22:05:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sS1uj-0002PQ-5t; Thu, 11 Jul 2024 18:05:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sS1ug-0002N9-Iw for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:18 -0400 Received: from mail-oi1-x236.google.com ([2607:f8b0:4864:20::236]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sS1uS-0000Vq-B9 for qemu-devel@nongnu.org; Thu, 11 Jul 2024 18:05:10 -0400 Received: by mail-oi1-x236.google.com with SMTP id 5614622812f47-3d9e13ef9aaso843774b6e.1 for ; Thu, 11 Jul 2024 15:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1720735503; x=1721340303; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Lky2qViTQvC4wtieFSLEUnrtuJBg0t8UrboKRrXUgoo=; b=HIQHYU5sRPRjjlpJ6ZcYSR8FSju+NIJftn12iVaeRPd1rUjsWIc6SqHlwFRissfI93 u5YvMqsdAw/hq/6YsrigTRtjzyFMxFCaUf6iidVE4oKk1pa9R47E/AC+f0zJGplagCG7 UrIp2FyZWQ3oYea0IXWzcR9+rrw8z/dQP6Y6059sK+Mewo95eLhK4dPMELrnHcG4CmJr PT4w8VmcuEzzYXuS1B3p68OVZHZcEtpf6kEKN4xuhoILOp/wVzB29xdx6CcHPwrK9NXh YE0/L2qEImLHqsDhln4wvZCrvLutFLIwy1Ini/ZNV4fZpgYdUSAKw5zkuaCzIipk0MEe hZvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720735503; x=1721340303; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Lky2qViTQvC4wtieFSLEUnrtuJBg0t8UrboKRrXUgoo=; b=J1UEOZIZIoBbuQqf8LJhNrxkkX868Z5WyViGEaZ4t88onUIC2SS38FZrfLf8G6D+0N taWs+3bYMPesolWKVkaYKrgnjhLbbU7w8YHf1mX1DFYnklFlDCI62C53qPBA2ByZQciJ AtoUVEBX7QkKPNuXcoyKxDh1zwIiztJF0DfpsXq8rUKvaUm8qIXgC1h4pdG/vHP9g8ST tyhUsLKfxiJp2tmlDyW+bPZU9N1UNM3boiV72Ad7+bd+GrZFsa99cs/aFx2gPNdjDT8U tw38n4YnLi+XkSGmVv8c0JNVWlK36sQPX3VJNLrwB3Bblqk70zCV28HRgPiJjpLCnT7I o/zA== X-Forwarded-Encrypted: i=1; AJvYcCW8nCVWvJMH/hFLF5NP0azBt87cf5dUMjv+EeN1n5cPgnxkMNPBliSTMqjgrGS52fcwzIKPDq43XVLU7GLevQgb4BAmvZ4= X-Gm-Message-State: AOJu0Yxl+gesY9JH0qsQ65D8oLmA/BNu87zogKXagU4BIx5TR87rjBhH KVNbE3hE2aUf2CSmbhcER6I4lhUkV33z9TODSQXX1UgWgVtPGKZJrEBySvUoZXI= X-Google-Smtp-Source: AGHT+IH7zaypfsaiYkptZ7R0CTbcb62xCDDJHF7wn0mG1YOlnSX+zQngcvdCwyJPVdFgzJ/Wuqvkcg== X-Received: by 2002:a05:6808:1:b0:3da:aae9:7165 with SMTP id 5614622812f47-3daaae97310mr427966b6e.14.1720735503179; Thu, 11 Jul 2024 15:05:03 -0700 (PDT) Received: from DY4X0N7X05.bytedance.net ([208.184.112.130]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3d93acff4d9sm1218578b6e.5.2024.07.11.15.05.01 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 11 Jul 2024 15:05:02 -0700 (PDT) From: Yichen Wang To: Paolo Bonzini , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Thomas Huth , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Peter Xu , Fabiano Rosas , Eric Blake , Markus Armbruster , "Michael S. Tsirkin" , Cornelia Huck , qemu-devel@nongnu.org Cc: "Hao Xiang" , "Liu, Yuan1" , "Shivam Kumar" , "Ho-Ren (Jack) Chuang" , "Yichen Wang" , Bryan Zhang Subject: [PATCH v5 13/13] migration/multifd: Add integration tests for multifd with Intel DSA offloading. Date: Thu, 11 Jul 2024 15:04:51 -0700 Message-Id: <20240711220451.19780-4-yichen.wang@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240711220451.19780-1-yichen.wang@bytedance.com> References: <20240711220451.19780-1-yichen.wang@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::236; envelope-from=yichen.wang@bytedance.com; helo=mail-oi1-x236.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Hao Xiang * Add test case to start and complete multifd live migration with DSA offloading enabled. * Add test case to start and cancel multifd live migration with DSA offloading enabled. Signed-off-by: Bryan Zhang Signed-off-by: Hao Xiang Signed-off-by: Yichen Wang --- tests/qtest/migration-test.c | 80 +++++++++++++++++++++++++++++++++++- 1 file changed, 79 insertions(+), 1 deletion(-) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 70b606b888..67cd976705 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -608,6 +608,13 @@ typedef struct { bool suspend_me; } MigrateStart; +/* + * It requires separate steps to configure and enable DSA device. + * This test assumes that the configuration is done already. + */ +static const char *dsa_dev_path_p = "['/dev/dsa/wq4.0']"; +static const char *dsa_dev_path = "/dev/dsa/wq4.0"; + /* * A hook that runs after the src and dst QEMUs have been * created, but before the migration is started. This can @@ -3279,7 +3286,7 @@ static void test_multifd_tcp_tls_x509_reject_anon_client(void) * * And see that it works */ -static void test_multifd_tcp_cancel(void) +static void test_multifd_tcp_cancel_common(bool use_dsa) { MigrateStart args = { .hide_stderr = true, @@ -3299,6 +3306,11 @@ static void test_multifd_tcp_cancel(void) migrate_set_capability(from, "multifd", true); migrate_set_capability(to, "multifd", true); + if (use_dsa) { + migrate_set_parameter_str(from, "zero-page-detection", "dsa-accel"); + migrate_set_parameter_str(from, "dsa-accel-path", dsa_dev_path_p); + } + /* Start incoming migration from the 1st socket */ migrate_incoming_qmp(to, "tcp:127.0.0.1:0", "{}"); @@ -3348,6 +3360,49 @@ static void test_multifd_tcp_cancel(void) test_migrate_end(from, to2, true); } +/* + * This test does: + * source target + * migrate_incoming + * migrate + * migrate_cancel + * launch another target + * migrate + * + * And see that it works + */ +static void test_multifd_tcp_cancel(void) +{ + test_multifd_tcp_cancel_common(false); +} + +#ifdef CONFIG_DSA_OPT + +static void *test_migrate_precopy_tcp_multifd_start_dsa(QTestState *from, + QTestState *to) +{ + migrate_set_parameter_str(from, "zero-page-detection", "dsa-accel"); + migrate_set_parameter_str(from, "dsa-accel-path", dsa_dev_path_p); + return test_migrate_precopy_tcp_multifd_start_common(from, to, "none"); +} + +static void test_multifd_tcp_zero_page_dsa(void) +{ + MigrateCommon args = { + .listen_uri = "defer", + .start_hook = test_migrate_precopy_tcp_multifd_start_dsa, + }; + + test_precopy_common(&args); +} + +static void test_multifd_tcp_cancel_dsa(void) +{ + test_multifd_tcp_cancel_common(true); +} + +#endif + static void calc_dirty_rate(QTestState *who, uint64_t calc_time) { qtest_qmp_assert_success(who, @@ -3772,6 +3827,19 @@ static bool kvm_dirty_ring_supported(void) #endif } +#ifdef CONFIG_DSA_OPT +static int test_dsa_setup(void) +{ + int fd; + fd = open(dsa_dev_path, O_RDWR); + if (fd < 0) { + return -1; + } + close(fd); + return 0; +} +#endif + int main(int argc, char **argv) { bool has_kvm, has_tcg; @@ -3984,6 +4052,16 @@ int main(int argc, char **argv) test_multifd_tcp_zero_page_legacy); migration_test_add("/migration/multifd/tcp/plain/zero-page/none", test_multifd_tcp_no_zero_page); + +#ifdef CONFIG_DSA_OPT + if (g_str_equal(arch, "x86_64") && test_dsa_setup() == 0) { + migration_test_add("/migration/multifd/tcp/plain/zero-page/dsa", + test_multifd_tcp_zero_page_dsa); + migration_test_add("/migration/multifd/tcp/plain/cancel/dsa", + test_multifd_tcp_cancel_dsa); + } +#endif + migration_test_add("/migration/multifd/tcp/plain/cancel", test_multifd_tcp_cancel); migration_test_add("/migration/multifd/tcp/plain/zlib",