From patchwork Tue Jul 21 16:04:11 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?THVrw6HFoSBEb2t0b3I=?= X-Patchwork-Id: 36561 Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n6LG4puC004953 for ; Tue, 21 Jul 2009 16:04:51 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754775AbZGUQEX (ORCPT ); Tue, 21 Jul 2009 12:04:23 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754618AbZGUQEX (ORCPT ); Tue, 21 Jul 2009 12:04:23 -0400 Received: from mx2.redhat.com ([66.187.237.31]:59026 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755422AbZGUQEW (ORCPT ); Tue, 21 Jul 2009 12:04:22 -0400 Received: from int-mx2.corp.redhat.com (int-mx2.corp.redhat.com [172.16.27.26]) by mx2.redhat.com (8.13.8/8.13.8) with ESMTP id n6LG4IDb024283; Tue, 21 Jul 2009 12:04:18 -0400 Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx2.corp.redhat.com (8.13.1/8.13.1) with ESMTP id n6LG4E3j013715; Tue, 21 Jul 2009 12:04:15 -0400 Received: from [10.34.33.254] (dhcp-lab-254.englab.brq.redhat.com [10.34.33.254]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id n6LG4CUb008101; Tue, 21 Jul 2009 12:04:13 -0400 Message-ID: <4A65E6FB.1090901@redhat.com> Date: Tue, 21 Jul 2009 18:04:11 +0200 From: =?UTF-8?B?THVrw6HFoSBEb2t0b3I=?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1b3pre) Gecko/20090513 Fedora/3.0-2.3.beta2.fc11 Thunderbird/3.0b2 MIME-Version: 1.0 To: Lucas Meneghel Rodrigues CC: KVM list , Jason Wang , Michael Goldish , Autotest mailing list Subject: Re: [KVM_AUTOTEST] add kvm hugepage variant References: <4A55B759.5080302@redhat.com> <4A57118F.3030907@redhat.com> <1248094728.5318.18.camel@localhost.localdomain> In-Reply-To: <1248094728.5318.18.camel@localhost.localdomain> X-Scanned-By: MIMEDefang 2.58 on 172.16.27.26 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Well, thank you for notifications, I'll keep them in my mind. Also the problem with mempath vs. mem-path is solved. It was just a misspell in one version of KVM. * fixed patch attached Dne 20.7.2009 14:58, Lucas Meneghel Rodrigues napsal(a): > On Fri, 2009-07-10 at 12:01 +0200, Lukáš Doktor wrote: >> After discussion I split the patches. > > Hi Lukáš, sorry for the delay answering your patch. Looks good to me in > general, I have some remarks to make: > > 1) When posting patches to the autotest kvm tests, please cross post the > autotest mailing list (autotest@test.kernel.org) and the KVM list. > > 2) About scripts to prepare the environment to perform tests - we've had > some discussion about including shell scripts on autotest. Bottom line, > autotest has a policy of not including non python code when possible > [1]. So, would you mind re-creating your hugepage setup code in python > and re-sending it? > > Thanks for your contribution, looking forward getting it integrated to > our tests. > > [1] Unless when it is not practical for testing purposes - writing tests > in C is just fine, for example. > >> This patch adds kvm_hugepage variant. It prepares the host system and >> start vm with -mem-path option. It does not clean after itself, because >> it's impossible to unmount and free hugepages before all guests are >> destroyed. >> >> I need to ask you what to do with change of qemu parameter. Newest >> versions are using -mempath insted of -mem-path. This is impossible to >> fix using current config file. I can see 2 solutions: >> 1) direct change in kvm_vm.py (parse output and try another param) >> 2) detect qemu capabilities outside and create additional layer (better >> for future occurrence) >> >> Dne 9.7.2009 11:24, Lukáš Doktor napsal(a): >>> This patch adds kvm_hugepage variant. It prepares the host system and >>> start vm with -mem-path option. It does not clean after itself, because >>> it's impossible to unmount and free hugepages before all guests are >>> destroyed. >>> >>> There is also added autotest.libhugetlbfs test. >>> >>> I need to ask you what to do with change of qemu parameter. Newest >>> versions are using -mempath insted of -mem-path. This is impossible to >>> fix using current config file. I can see 2 solutions: >>> 1) direct change in kvm_vm.py (parse output and try another param) >>> 2) detect qemu capabilities outside and create additional layer (better >>> for future occurrence) >>> >>> Tested by:ldoktor@redhat.com on RHEL5.4 with kvm-83-72.el5 > diff --git a/client/tests/kvm/kvm_tests.cfg.sample b/client/tests/kvm/kvm_tests.cfg.sample index 5bd6eb8..70e290d 100644 --- a/client/tests/kvm/kvm_tests.cfg.sample +++ b/client/tests/kvm/kvm_tests.cfg.sample @@ -555,6 +555,13 @@ variants: only default image_format = raw +variants: + - @kvm_smallpages: + - kvm_hugepages: + hugepage_path = /mnt/hugepage + pre_command = "/usr/bin/python scripts/hugepage.py" + extra_params += " -mem-path /mnt/hugepage" + variants: - @basic: @@ -568,6 +575,7 @@ variants: only Fedora.8.32 only install setup boot shutdown only rtl8139 + only kvm_smallpages - @sample1: only qcow2 only ide diff --git a/client/tests/kvm/kvm_vm.py b/client/tests/kvm/kvm_vm.py index 48f2916..2b97ccc 100644 --- a/client/tests/kvm/kvm_vm.py +++ b/client/tests/kvm/kvm_vm.py @@ -412,6 +412,13 @@ class VM: self.destroy() return False + if output: + logging.debug("qemu produced some output:\n%s", output) + if "alloc_mem_area" in output: + logging.error("Could not allocate hugepage memory" + " -- qemu command:\n%s", qemu_command) + return False + logging.debug("VM appears to be alive with PID %d", self.pid) return True diff -Narup a/client/tests/kvm/scripts/hugepage.py b/client/tests/kvm/scripts/ hugepage.py --- a/client/tests/kvm/scripts/hugepage.py 1970-01-01 01:00:00.000000000 +0100 +++ a/client/tests/kvm/scripts/hugepage.py 2009-07-21 16:47:00.000000000 +0200 @@ -0,0 +1,63 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# Alocates enough hugepages and mount hugetlbfs +import os, sys, time + +# Variables check & set +vms = os.environ['KVM_TEST_vms'].split().__len__() +try: + max_vms = int(os.environ['KVM_TEST_max_vms']) +except KeyError: + max_vms = 0 +mem = int(os.environ['KVM_TEST_mem']) +hugepage_path = os.environ['KVM_TEST_hugepage_path'] + +fmeminfo = open("/proc/meminfo", "r") +while fmeminfo: + line = fmeminfo.readline() + if line.startswith("Hugepagesize"): + dumm, hp_size, dumm = line.split() + break +fmeminfo.close() + +if not hp_size: + print "Could not get Hugepagesize from /proc/meminfo file" + raise ValueError + +if vms < max_vms: + vms = max_vms + +vmsm = ((vms * mem) + (vms * 64)) +target = (vmsm * 1024 / int(hp_size)) + +# Iteratively set # of hugepages +fhp = open("/proc/sys/vm/nr_hugepages", "r+") +hp = fhp.readline() +while int(hp) < target: + hp_ = hp + fhp.write(target.__str__()) + fhp.flush() + time.sleep(5) + fhp.seek(0) + hp = int(fhp.readline()) + if hp_ == hp: + raise MemoryError +fhp.close() + +# Mount hugepage filesystem, if necessarily +fmount = open("/proc/mounts", "r") +mount = 1 +line = fmount.readline() +while line: + if line.split()[1] == os.environ['KVM_TEST_hugepage_path']: + mount = 0 + break + line = fmount.readline() +fmount.close() + +if mount: + if not os.path.exists(hugepage_path): + os.makedirs(hugepage_path) + cmd = "mount -t hugetlbfs none %s" % (hugepage_path) + if os.system(cmd): + raise OSError