From patchwork Thu Apr 17 11:13:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandr Moshkov X-Patchwork-Id: 14055325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B2F5C369C2 for ; Thu, 17 Apr 2025 11:14:23 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5NBe-0003XS-DO; Thu, 17 Apr 2025 07:13:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBc-0003Wj-2S for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:40 -0400 Received: from forwardcorp1a.mail.yandex.net ([178.154.239.72]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBX-0007AU-HO for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:39 -0400 Received: from mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net [IPv6:2a02:6b8:c15:381b:0:640:f69d:0]) by forwardcorp1a.mail.yandex.net (Yandex) with ESMTPS id 55C1A60AB0; Thu, 17 Apr 2025 14:13:31 +0300 (MSK) Received: from dtalexundeer-nx.yandex-team.ru (unknown [2a02:6b8:b081:8014::1:10]) by mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id MDH9PZ2FiKo0-8qlmYwjS; Thu, 17 Apr 2025 14:13:30 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1744888410; bh=OFnYkElq7KLQOliisIu7XXuHdcEvLFLPedH92jINYp0=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=PV6WmtkucY51a4DtZLPykerG8WZU1ykIdI8VykDW/6pfcy/4ZZqnUst2nXpNjMqY+ MBmfyHxDHsWWRkaQwpL3jLo84SY3xBCK7vDhM4PLyKTNK55CjLRCwkYM/zKY+MkWVR DHue1Qlac2Ji4GXGN1A24sUKo9J+zD0qBw5Pp4FI= Authentication-Results: mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Alexandr Moshkov To: qemu-devel@nongnu.org Cc: Cleber Rosa , "yc-core @ yandex-team . ru" , Paolo Bonzini , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= , Alexandr Moshkov Subject: [PATCH v5 1/2] tests/functional: add skipLockedMemoryTest decorator Date: Thu, 17 Apr 2025 16:13:20 +0500 Message-Id: <20250417111321.167008-2-dtalexundeer@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> References: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> MIME-Version: 1.0 Received-SPF: pass client-ip=178.154.239.72; envelope-from=dtalexundeer@yandex-team.ru; helo=forwardcorp1a.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Used in future commit to skipping execution of a tests if the system's locked memory limit is below the required threshold. Signed-off-by: Alexandr Moshkov --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/decorators.py | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py index af41c2c6a2..6e666a059f 100644 --- a/tests/functional/qemu_test/__init__.py +++ b/tests/functional/qemu_test/__init__.py @@ -15,6 +15,6 @@ from .linuxkernel import LinuxKernelTest from .decorators import skipIfMissingCommands, skipIfNotMachine, \ skipFlakyTest, skipUntrustedTest, skipBigDataTest, skipSlowTest, \ - skipIfMissingImports, skipIfOperatingSystem + skipIfMissingImports, skipIfOperatingSystem, skipLockedMemoryTest from .archive import archive_extract from .uncompress import uncompress diff --git a/tests/functional/qemu_test/decorators.py b/tests/functional/qemu_test/decorators.py index 50d29de533..c0d1567b14 100644 --- a/tests/functional/qemu_test/decorators.py +++ b/tests/functional/qemu_test/decorators.py @@ -5,6 +5,7 @@ import importlib import os import platform +import resource from unittest import skipIf, skipUnless from .cmd import which @@ -131,3 +132,20 @@ def skipIfMissingImports(*args): return skipUnless(has_imports, 'required import(s) "%s" not installed' % ", ".join(args)) + +''' +Decorator to skip execution of a test if the system's +locked memory limit is below the required threshold. +Takes required locked memory threshold in kB. +Example: + + @skipLockedMemoryTest(2_097_152) +''' +def skipLockedMemoryTest(locked_memory): + # get memlock hard limit in bytes + _, ulimit_memory = resource.getrlimit(resource.RLIMIT_MEMLOCK) + + return skipUnless( + ulimit_memory == resource.RLIM_INFINITY or ulimit_memory >= locked_memory * 1024, + f'Test required {locked_memory} kB of available locked memory', + ) From patchwork Thu Apr 17 11:13:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandr Moshkov X-Patchwork-Id: 14055326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48B64C369CB for ; Thu, 17 Apr 2025 11:14:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1u5NBc-0003X2-Qe; Thu, 17 Apr 2025 07:13:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBa-0003WC-Jf for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:38 -0400 Received: from forwardcorp1a.mail.yandex.net ([2a02:6b8:c0e:500:1:45:d181:df01]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1u5NBX-0007Af-HR for qemu-devel@nongnu.org; Thu, 17 Apr 2025 07:13:38 -0400 Received: from mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net [IPv6:2a02:6b8:c15:381b:0:640:f69d:0]) by forwardcorp1a.mail.yandex.net (Yandex) with ESMTPS id 4AE0260A83; Thu, 17 Apr 2025 14:13:32 +0300 (MSK) Received: from dtalexundeer-nx.yandex-team.ru (unknown [2a02:6b8:b081:8014::1:10]) by mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id MDH9PZ2FiKo0-V8zXIySh; Thu, 17 Apr 2025 14:13:31 +0300 Precedence: bulk X-Yandex-Fwd: 1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1744888411; bh=Mg42Zg1ZyEgQAtkcUl6Dccx4cbgucQhpYSocIFJlCM0=; h=Message-Id:Date:In-Reply-To:Cc:Subject:References:To:From; b=khAAhXu22ZXEbuIz/G99aKBnjGHOcgmoyMuXEDDV4hLPeiEYfdjna9U1Ber5d1cmD e2gdK2kNmAPsIlQg93WVYoM3j9CJBGJZRREKXOPNT/uDNyXQvEGHGtVdqWEYWCH9tE WWuwyNK6Vk4zv8GNK+n33Kbr/UjvbsPawT/ehZN4= Authentication-Results: mail-nwsmtp-smtp-corp-main-69.vla.yp-c.yandex.net; dkim=pass header.i=@yandex-team.ru From: Alexandr Moshkov To: qemu-devel@nongnu.org Cc: Cleber Rosa , "yc-core @ yandex-team . ru" , Paolo Bonzini , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= , Alexandr Moshkov Subject: [PATCH v5 2/2] tests/functional: add memlock tests Date: Thu, 17 Apr 2025 16:13:21 +0500 Message-Id: <20250417111321.167008-3-dtalexundeer@yandex-team.ru> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> References: <20250417111321.167008-1-dtalexundeer@yandex-team.ru> MIME-Version: 1.0 Received-SPF: pass client-ip=2a02:6b8:c0e:500:1:45:d181:df01; envelope-from=dtalexundeer@yandex-team.ru; helo=forwardcorp1a.mail.yandex.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add new tests to check the correctness of the `-overcommit memlock` option (possible values: off, on, on-fault) by using `/proc/{qemu_pid}/smaps` file to check in Size, Rss and Locked fields of anonymous segments: * if `memlock=off`, then Locked = 0 on every anonymous smaps; * if `memlock=on`, then Size, Rss and Locked values must be equal for every anon smaps where Rss is not 0; * if `memlock=on-fault`, then Rss and Locked must be equal on every anon smaps and anonymous segment with Rss < Size must exists. Signed-off-by: Alexandr Moshkov --- tests/functional/meson.build | 1 + tests/functional/test_memlock.py | 102 +++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+) create mode 100755 tests/functional/test_memlock.py diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 0f8be30fe2..339af7835f 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -61,6 +61,7 @@ tests_generic_system = [ 'empty_cpu_model', 'info_usernet', 'version', + 'memlock', ] tests_generic_linuxuser = [ diff --git a/tests/functional/test_memlock.py b/tests/functional/test_memlock.py new file mode 100755 index 0000000000..b62f12a715 --- /dev/null +++ b/tests/functional/test_memlock.py @@ -0,0 +1,102 @@ +# Functional test that check overcommit memlock options +# +# Copyright (c) Yandex Technologies LLC, 2025 +# +# Author: +# Alexandr Moshkov +# +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import re + +from typing import List, Dict + +from qemu_test import QemuSystemTest +from qemu_test import skipLockedMemoryTest + + +SMAPS_HEADER_PATTERN = re.compile(r'^\w+-\w+', re.MULTILINE) +SMAPS_VALUE_PATTERN = re.compile(r'^(\w+):\s+(\d+) kB', re.MULTILINE) + + +@skipLockedMemoryTest(2_097_152) # 2GB +class MemlockTest(QemuSystemTest): + """ + Boots a Linux system with memlock options. + Then verify, that this options is working correctly + by checking the smaps of the QEMU proccess. + """ + + def common_vm_setup_with_memlock(self, memlock): + self.vm.add_args('-overcommit', f'mem-lock={memlock}') + self.vm.launch() + + def get_anon_smaps_by_pid(self, pid): + smaps_raw = self._get_raw_smaps_by_pid(pid) + return self._parse_anonymous_smaps(smaps_raw) + + def test_memlock_off(self): + self.common_vm_setup_with_memlock('off') + + anon_smaps = self.get_anon_smaps_by_pid(self.vm.get_pid()) + + # locked = 0 on every smap + for smap in anon_smaps: + self.assertEqual(smap['Locked'], 0) + + def test_memlock_on(self): + self.common_vm_setup_with_memlock('on') + + anon_smaps = self.get_anon_smaps_by_pid(self.vm.get_pid()) + + # size = rss = locked on every smap where rss not 0 + for smap in anon_smaps: + if smap['Rss'] == 0: + continue + self.assertTrue(smap['Size'] == smap['Rss'] == smap['Locked']) + + def test_memlock_onfault(self): + self.common_vm_setup_with_memlock('on-fault') + + anon_smaps = self.get_anon_smaps_by_pid(self.vm.get_pid()) + + # rss = locked on every smap and segment with rss < size exists + exists = False + for smap in anon_smaps: + self.assertTrue(smap['Rss'] == smap['Locked']) + if smap['Rss'] < smap['Size']: + exists = True + self.assertTrue(exists) + + def _parse_anonymous_smaps(self, smaps_raw: str) -> List[Dict[str, int]]: + result_segments = [] + current_segment = {} + is_anonymous = False + + for line in smaps_raw.split('\n'): + if SMAPS_HEADER_PATTERN.match(line): + if current_segment and is_anonymous: + result_segments.append(current_segment) + current_segment = {} + # anonymous segment header looks like this: + # 7f3b8d3f0000-7f3b8d3f3000 rw-s 00000000 00:0f 1052 + # and non anonymous header looks like this: + # 7f3b8d3f0000-7f3b8d3f3000 rw-s 00000000 00:0f 1052 [stack] + is_anonymous = len(line.split()) == 5 + elif m := SMAPS_VALUE_PATTERN.match(line): + current_segment[m.group(1)] = int(m.group(2)) + + if current_segment and is_anonymous: + result_segments.append(current_segment) + + return result_segments + + def _get_raw_smaps_by_pid(self, pid: int) -> str: + with open(f'/proc/{pid}/smaps', 'r') as f: + return f.read() + + +if __name__ == '__main__': + MemlockTest.main()