From patchwork Tue May 31 09:58:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Fan X-Patchwork-Id: 9144127 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A4F6960777 for ; Tue, 31 May 2016 10:01:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 869D62793B for ; Tue, 31 May 2016 10:01:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 794D927D17; Tue, 31 May 2016 10:01:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E911A2793B for ; Tue, 31 May 2016 10:01:08 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7gRo-0002bT-90; Tue, 31 May 2016 09:58:52 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b7gRn-0002ar-6y for xen-devel@lists.xen.org; Tue, 31 May 2016 09:58:51 +0000 Received: from [85.158.139.211] by server-10.bemta-5.messagelabs.com id ED/51-11369-A506D475; Tue, 31 May 2016 09:58:50 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNIsWRWlGSWpSXmKPExsVyMfSOs25kgm+ 4wZZNNhZLPi5mcWD0OLr7N1MAYxRrZl5SfkUCa8a0X7uYC/okKyZNTWlgvCjcxcjFISQwkVHi 1fJDrCAOi8B8VollB84wgjgSArtZJZa8P8nUxcgJ5MRIPHnzCcoul1i0YQeYLSSgIjF3Uw8jx KjpTBKfZ05lAUmwCKhKzP/xhQ3EZhNQkzjyfiYjiC0ioC7x58IEMJtZwFji3Y4bYLawQLTEu7 UQC3gFTCR+3elmhFhQIrF+8iJmiLigxMmZT1ggerUkbvx7CVTPAWRLSyz/xwES5hSwlli7ZQ4 7SFgU6LZXB+snMArPQtI8C0nzLITmBYzMqxg1ilOLylKLdA0N9ZKKMtMzSnITM3N0DQ1M9XJT i4sT01NzEpOK9ZLzczcxAoOcAQh2MK5sdz7EKMnBpCTKa/XbJ1yILyk/pTIjsTgjvqg0J7X4E KMMB4eSBC9rvG+4kGBRanpqRVpmDjDeYNISHDxKIrwiIGne4oLE3OLMdIjUKUZjji2/r61l4t g29d5aJiGWvPy8VClx3hCQUgGQ0ozSPLhBsDRwiVFWSpiXEeg0IZ6C1KLczBJU+VeM4hyMSsK 80iBTeDLzSuD2vQI6hQnolPgMH5BTShIRUlINjFMfRP10m3Lszy47JodsdvEzCcYfnna/ej1r yQGNLzM4UjYICFgWf1PrWbtHeImFiPyadPedlzXe1k36IVvRssWwOqh2ha2+UMpFxm3fXCfXz 1f0MC8JePu2eTfv3MpLrpZR9nYuN+0u70qwPlU+lWFxfNvlmjf607Zs+KF34+En+4XesVYXpJ RYijMSDbWYi4oTAaupLoL+AgAA X-Env-Sender: van.freenix@gmail.com X-Msg-Ref: server-13.tower-206.messagelabs.com!1464688728!33215105!1 X-Originating-IP: [209.85.220.67] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 8.46; banners=-,-,- X-VirusChecked: Checked Received: (qmail 30488 invoked from network); 31 May 2016 09:58:49 -0000 Received: from mail-pa0-f67.google.com (HELO mail-pa0-f67.google.com) (209.85.220.67) by server-13.tower-206.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 31 May 2016 09:58:49 -0000 Received: by mail-pa0-f67.google.com with SMTP id fg1so24338968pad.3 for ; Tue, 31 May 2016 02:58:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=MPvPoPotgifSq/m3v580oxPWPHUi6Xj28lG8DQ/RnsQ=; b=Eq1wOR+RCbYgi+8b6K6wcYEtqmiiVY8g7BMVMLkPtSegy1tsJD2bqix4jdAQX/wHRA uMyu89IGByek2EfOQWN1V8MfcPJVWEBJlrw4EtiMJdxKYStM6BcNZT0CiMse/0dpUIWu kpdSV+6cp/wMSEPpprU9q2CsvmXZm9dTEtb0Wzh5FnmU8JE9dviqZpdM37lgb9VRFH+C xqZZhjyjATUNUbbqqz0+WRlrS86uPFxPwp/i4iQWSg1/uyCy1HTGYnpy30t9ew/+wPjU 9FpFSx6j7ZMFtovZ2OInJxvfK8neZxaT333qWpv2vhVWyhxtwEzaM0gXuXGQzHrkY9AM s60g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=MPvPoPotgifSq/m3v580oxPWPHUi6Xj28lG8DQ/RnsQ=; b=TafBM2+6Jae4h9Ij5U+VRK0ymvw05lOOZS8kys0DlpPIlPKVYSbFc0cA1oatjKtW41 EBjjJObgKZFkNss3rQNL5Ph9EVEtH9m+SWmbviJHSHuqMv6fMiQSB0WdUfmOHjWak6J9 uRl0Bx0oLWGHgGx4TCJ/DDho7+OBXmikj3LDLsMJRMzr4IRmQ0lGrHxyWfwz2Uyy66Ft +g/K8UtJW0AmYfUoefqqREwC7Rmx4w7N+dB37wVCw8sBMOWRcjoWGFvP9HJs++qpCmqp TpuH7sgMwkubacBEbFxHFBoYJ4oq4HI3d2JUWqs024fm7Pi9hEBmKaZGWfp/K1a3tq2V FTRw== X-Gm-Message-State: ALyK8tLb1SnuKATDyUyW+FPiHbljMPOIg0/QMYP3luCMIgagnL45AONQEM5WYYxo7x4bIg== X-Received: by 10.66.26.37 with SMTP id i5mr53946247pag.15.1464688728139; Tue, 31 May 2016 02:58:48 -0700 (PDT) Received: from linux-7smt.suse (gate-zmy3.freescale.com. [192.88.167.1]) by smtp.googlemail.com with ESMTPSA id f27sm22980490pff.17.2016.05.31.02.58.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 May 2016 02:58:47 -0700 (PDT) Date: Tue, 31 May 2016 17:58:37 +0800 From: Peng Fan To: Julien Grall Message-ID: <20160531095834.GA10141@linux-7smt.suse> References: <1464327119-6242-1-git-send-email-van.freenix@gmail.com> <567844c6-6b60-6d7b-2f0c-5c940f4f7350@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <567844c6-6b60-6d7b-2f0c-5c940f4f7350@arm.com> User-Agent: Mutt/1.5.24 (2015-08-30) Cc: sstabellini@kernel.org, xen-devel@lists.xen.org Subject: Re: [Xen-devel] [PATCH] xen/arm: setup: initialize xenheap mappings after boot pages avaiable X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hi Julien, On Mon, May 30, 2016 at 10:53:24PM +0100, Julien Grall wrote: >Hi Peng, > >On 27/05/2016 06:31, Peng Fan wrote: >>To ARM64, setup_xenheap_mappings may call alloc_boot_pages to allocate >>first level page table, if there is a big chunk memory (ie, >512GB). > >Out of interest, have you found the bug by testing Xen on a platform with >512GB of RAM? :) Actually not. My board does not have so large memory. When I am reading the piece code, I think there is possibility that setup_xenheap_mappings may need boot pages, but boot pages are not ready. > >>So, need to make sure boot pages are ready before setup xenheap mappings. > >init_boot_pages is using mfn_to_virt (see bootmem_region_add), which cannot >work until xenheap_mfn_start is initialized. This is done by >setup_xenheap_mappings. My bad. I did not catch this point. Thanks for correcting me. > >I would be happy to give you hint on how to solve this, but I am not sure to >fully understand your issue. Can you give more details? I did not met issue on my platform. I just think the logic of this piece code may cause errors on platform with large memory (>512GB). How about the following patch? First loop all the memory banks and calculate ram_size/ram_start/ram_end, then set xenheap_virt_end/start and xenheap_mfn_end. Now readly for init boot pages and setup_xenheap_mappings. Thanks, Peng. diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index dcb23b7..d3a3af3 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -635,13 +635,24 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size) paddr_t bank_start = bootinfo.mem.bank[bank].start; paddr_t bank_size = bootinfo.mem.bank[bank].size; paddr_t bank_end = bank_start + bank_size; - paddr_t s, e; ram_size = ram_size + bank_size; ram_start = min(ram_start,bank_start); ram_end = max(ram_end,bank_end); + } - setup_xenheap_mappings(bank_start>>PAGE_SHIFT, bank_size>>PAGE_SHIFT); + total_pages += ram_size >> PAGE_SHIFT; + + xenheap_virt_end = XENHEAP_VIRT_START + ram_end - ram_start; + xenheap_mfn_start = ram_start >> PAGE_SHIFT; + xenheap_mfn_end = ram_end >> PAGE_SHIFT; + + for ( bank = 0 ; bank < bootinfo.mem.nr_banks; bank++ ) + { + paddr_t bank_start = bootinfo.mem.bank[bank].start; + paddr_t bank_size = bootinfo.mem.bank[bank].size; + paddr_t bank_end = bank_start + bank_size; + paddr_t s, e; s = bank_start; while ( s < bank_end ) @@ -658,18 +669,12 @@ static void __init setup_mm(unsigned long dtb_paddr, size_t dtb_size) if ( e > bank_end ) e = bank_end; - xenheap_mfn_end = e; - dt_unreserved_regions(s, e, init_boot_pages, 0); s = n; } - } - - total_pages += ram_size >> PAGE_SHIFT; - xenheap_virt_end = XENHEAP_VIRT_START + ram_end - ram_start; - xenheap_mfn_start = ram_start >> PAGE_SHIFT; - xenheap_mfn_end = ram_end >> PAGE_SHIFT; + setup_xenheap_mappings(bank_start>>PAGE_SHIFT, bank_size>>PAGE_SHIFT); + } /* * Need enough mapped pages for copying the DTB.