From patchwork Tue Oct 25 14:49:59 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Liu X-Patchwork-Id: 9394629 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E9D2C60231 for ; Tue, 25 Oct 2016 14:52:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCA0328A7B for ; Tue, 25 Oct 2016 14:52:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D0EF629634; Tue, 25 Oct 2016 14:52:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 38BF828A7B for ; Tue, 25 Oct 2016 14:52:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bz33R-00063b-ON; Tue, 25 Oct 2016 14:50:17 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bz33P-00063V-NJ for xen-devel@lists.xen.org; Tue, 25 Oct 2016 14:50:15 +0000 Received: from [193.109.254.147] by server-8.bemta-6.messagelabs.com id 94/32-28497-6217F085; Tue, 25 Oct 2016 14:50:14 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBIsWRWlGSWpSXmKPExsXitHSDva5aIX+ EwaytAhZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa8bt2ycZCxZqVRxe3cjcwHhXsYuRk0NCwF/i 6q+7rCA2i4CqRPP/P4wgNpuAssTPzl62LkYODhEBXYlvVwu6GLk4mAVaGCX2LGhhAXGEBfoYJ R49OMgG0sAroC+x+Mw7ZpCEkMAVZokPi06xQyQEJU7OfMICYjML6Egs2P0JbCqzgLTE8n8cIG FOATOJn0fmMIPYogIqElcmvAVrFRJQkOiYfowJ4lBuidunpzJPYOSfhWTqLCRTZyFMXcDIvIp RvTi1qCy1SNdYL6koMz2jJDcxM0fX0MBMLze1uDgxPTUnMalYLzk/dxMjMAQZgGAHY8c/p0OM khxMSqK8N7L4I4T4kvJTKjMSizPii0pzUosPMcpwcChJ8D7JB8oJFqWmp1akZeYAowEmLcHBo yTCy1cAlOYtLkjMLc5Mh0idYlSUEudlBUkIgCQySvPg2mAReIlRVkqYlxHoECGegtSi3MwSVP lXjOIcjErCvDIgU3gy80rgpr8CWswEtFgwngdkcUkiQkqqgbHH/L5vjv+O1Rz3Zz2z53vlVBO haLrv4KTihctZXA8zSAcu+JlsdLvQWunWpbxlGSGdNwWKppb2TrebonknbGr41hnvVKJe32XV 2sC9eOnVzuITv96bHVNazbwt9MGx2Kppk45NFezcLm9w4mG59eWzJ0Oz92x8Xfz4ot+j6R2vp bQUTu5vkApVYinOSDTUYi4qTgQAF1JgVrsCAAA= X-Env-Sender: prvs=0992e837c=wei.liu2@citrix.com X-Msg-Ref: server-8.tower-27.messagelabs.com!1477407012!57199686!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.0.13; banners=-,-,- X-VirusChecked: Checked Received: (qmail 40564 invoked from network); 25 Oct 2016 14:50:14 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-8.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 25 Oct 2016 14:50:14 -0000 X-IronPort-AV: E=Sophos;i="5.31,546,1473120000"; d="scan'208";a="394522071" Date: Tue, 25 Oct 2016 15:49:59 +0100 From: Wei Liu To: Sander Eikelenboom Message-ID: <20161025144959.GZ30231@citrix.com> References: <62d9fcb78806a784b7e7de5542ed5a34@eikelenboom.it> <57CD5B11020000780010BAC5@prv-mh.provo.novell.com> <57CD6429020000780010BB5A@prv-mh.provo.novell.com> <3b48ddc603993f01b0ba305638877ce3@eikelenboom.it> <826255623.20161013164331@eikelenboom.it> <365154539.20161017172817@eikelenboom.it> <20161018124823.GM23219@citrix.com> <20161025112412.GV30231@citrix.com> <567356225.20161025133745@eikelenboom.it> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <567356225.20161025133745@eikelenboom.it> User-Agent: Mutt/1.5.23 (2014-03-12) X-DLP: MIA2 Cc: Doug Goldstein , Wei Liu , Jan Beulich , xen-devel@lists.xen.org Subject: Re: [Xen-devel] Regression between Xen 4.6.0 and 4.7.0, Direct kernel boot on a qemu-xen and seabios HVM guest doesn't work anymore. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP On Tue, Oct 25, 2016 at 01:37:45PM +0200, Sander Eikelenboom wrote: > > Tuesday, October 25, 2016, 1:24:12 PM, you wrote: > > > On Tue, Oct 18, 2016 at 01:48:23PM +0100, Wei Liu wrote: > >> On Mon, Oct 17, 2016 at 05:28:17PM +0200, Sander Eikelenboom wrote: > >> > Thursday, October 13, 2016, 4:43:31 PM, you wrote: > >> > > >> > > Hi Jan / Wei, > >> > > >> > > Took a while before i had the chance to fiddle some more to find the actual culprit. > >> > > After analyzing the output of xl -vvvvv create somewhat more i came to the > >> > > insight it was probably Qemu and not Xen causing the fault. > >> > > >> > > As a test I just used a qemu-xen binary build with xen-4.6.0 booting up a guest with > >> > > direct kernel boot mode on xen-unstable. And that old qemu binary works fine. > >> > > >> > > After testing i can conclude, Jan was right, the bisection was a red herring, > >> > > the problem is caused by some change in Qemu and not by something in the Xen tree. > >> > > (strange thing is that for as far as i know i did a "make distclean" between > >> > > every build (taking a lot of time), which should have pulled a fresh qemu-xen > >> > > tree and therefor the bisection should have lead to a commit with a Config.mk > >> > > hash change for qemu-xen version.) > >> > > >> > > Will see if i can find some more time and bisect qemu and find the culprit. > >> > > >> > > -- > >> > > Sander > >> > > >> > > >> > Unfortunately i have to give up on this issue, for me it's impossible to bisect this > >> > issue with my present git-foo. > >> > > >> > The first try with bisection of the whole xen-tree seems to have hit the issue that the > >> > qemu-revision that gets pulled on a fresh build is "master" during the whole > >> > dev period. That creates havoc when trying to bisect, since you are testing > >> > combinations that were never developed (nor auto tested) in that combination > >> > (especially when a xen-tree and qemu-tree change have a dependency like Roger's > >> > "xen: fix usage of xc_domain_create in domain builder") > >> > > >> > While trying to bisect only qemu (keeping xen itself on RELEASE-4.6.0 and > >> > seabios on rel-1.8.2) it get stuck on issues with that tree. > >> > Between 4.6.0 and 4.7.0 the qemu tree switched from git://xenbits.xen.org/qemu-upstream-4.6-testing.git > >> > to git://xenbits.xen.org/qemu-xen.git),after that there seem to have > >> > been a lot of merges going back and forth and to me it seems a mess (but as i > >> > said it could also be a lack of git-foo). I tried by manual bisecting, removing > >> > and cloning trees again etc. but that doesn't suffice, it's all going no-where. > >> > (while the known good build (plain RELEASE-4.6.0) always works, so it doesn't > >> > seem to be some random problem) > >> > > >> > >> Thanks for trying. > >> > >> > So perhaps some dev can at least verify that the issue is there (since 4.7.0) > >> > and put it on the "known broken" list of things. > >> > > >> > >> I will put this into the list of things I need to look at. > >> > > > I investigated this a bit. The root cause is the memory accounting is > > wrong in QEMU. It would try to allocate more ram than allowed. I haven't > > tried to figure out exactly what is wrong, though. > > That confirms what i was thinking in the end, but bisection the qemu-tree > changes between the xen-4.6.0 and xen-4.7.0 release proved to be pretty > difficult as i explained. So i you have a hunch as to in what code it should > reside debugging instead of bisecting would probably be better. > (so one of the questions is what changes in the memory accounting when you > supply the kernel from the host instead of the guest, since booting a kernel > with grub from within the guest doesn't give any memory accounting issues.) > > Thanks for investigating ! I think I hunted down the offending function. Mind trying this patch for me? ---8<--- From 3c7f8b55109959cf470deeee452f452f7c0ade51 Mon Sep 17 00:00:00 2001 From: Wei Liu Date: Tue, 25 Oct 2016 15:45:04 +0100 Subject: [PATCH] acpi: don't build acpi tables for xen guests Xen's toolstack is in charge of building ACPI tables. Skip acpi table building if running on Xen. This issue is discovered due to direct kernel boot on Xen doesn't boot anymore, because the new ACPI tables cause the guest to exceed its memory allocation limit. Reported-by: Sander Eikelenboom Signed-off-by: Wei Liu --- Cc: Anthony PERARD Cc: Stefano Stabellini RFC because I'm not sure this is the best way to fix it. --- hw/i386/acpi-build.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index a26a4bb..6ba5031 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -45,6 +45,7 @@ #include "sysemu/tpm_backend.h" #include "hw/timer/mc146818rtc_regs.h" #include "sysemu/numa.h" +#include "hw/xen/xen.h" /* Supported chipsets: */ #include "hw/acpi/piix4.h" @@ -2865,6 +2866,12 @@ void acpi_setup(void) return; } + if (xen_enabled()) { + fprintf(stderr, "%s %d\n", __FILE__, __LINE__); + ACPI_BUILD_DPRINTF("Xen enabled. Bailing out.\n"); + return; + } + build_state = g_malloc0(sizeof *build_state); acpi_set_pci_info();