From patchwork Fri Jul 25 15:29:08 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arun Chandran X-Patchwork-Id: 4624121 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 6A3DBC0338 for ; Fri, 25 Jul 2014 15:32:04 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 64F69201F2 for ; Fri, 25 Jul 2014 15:32:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 03368201ED for ; Fri, 25 Jul 2014 15:32:02 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XAhRK-0007jK-Lh; Fri, 25 Jul 2014 15:29:46 +0000 Received: from mail-yk0-f179.google.com ([209.85.160.179]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XAhRD-0007Xi-Ix for linux-arm-kernel@lists.infradead.org; Fri, 25 Jul 2014 15:29:43 +0000 Received: by mail-yk0-f179.google.com with SMTP id 142so2895469ykq.38 for ; Fri, 25 Jul 2014 08:29:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=XkEVm1hNsWyc9LZbJvYM8G2sNWml9V7qCcKSFVjMnDE=; b=e4bFPIdnsqNn++5MGsZjyss0BJKkHey6GHA+Ps7ySrylWkwaum/9MEKxRPlPXOig/F M+DI4ve7vMvxLquyl3KK+imqf5mVki0UVKYZdg5upd8Gj3gdqXIm2yJ2R05bsGjP0apg ordIcekj/7ddAsb7H48NI/83qXN5H4Prtx4PKy45+zRTCrNBY4ml/Ws6EYIuLWuY4nYR u2yQdmUAmttiS/T+/5Q3hI6OTv2/MxT1v6Bn935yUxEKlCOJk2vS3ebZwS67xLeMST17 9osOaQc9hOERj11PQcO1UwKged0kQo9lPmMNy3eVZpBo8GEgOvIQ5pz7pHb2Kt+AIVJR TPMA== X-Gm-Message-State: ALoCoQmIDdUIW4dtwvZsTUlPJS5zFBWoUGn/vgEi+FNZ6vp3zLxcxGqIQT1wRvuIv07y6MpEUJZJ MIME-Version: 1.0 X-Received: by 10.236.105.229 with SMTP id k65mr24502157yhg.50.1406302148711; Fri, 25 Jul 2014 08:29:08 -0700 (PDT) Received: by 10.170.96.87 with HTTP; Fri, 25 Jul 2014 08:29:08 -0700 (PDT) In-Reply-To: <20140725121448.GB19632@leverpostej> References: <1405443898.22585.7.camel@smoke> <1405551861.7262.26.camel@smoke> <1406162287.4062.39.camel@smoke> <20140724093603.GC4079@leverpostej> <1406247468.4062.59.camel@smoke> <20140725121448.GB19632@leverpostej> Date: Fri, 25 Jul 2014 20:59:08 +0530 Message-ID: Subject: Re: Kexec on arm64 From: Arun Chandran To: Mark Rutland X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140725_082939_788950_84ACF736 X-CRM114-Status: GOOD ( 28.29 ) X-Spam-Score: -0.7 (/) Cc: Geoff Levand , Feng Kan , "ksankaran@apm.com" , Will Deacon , "linux-arm-kernel@lists.infradead.org" X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Fri, Jul 25, 2014 at 5:44 PM, Mark Rutland wrote: > On Fri, Jul 25, 2014 at 12:48:04PM +0100, Arun Chandran wrote: >> Hi Geoff, >> >> On Fri, Jul 25, 2014 at 5:47 AM, Geoff Levand wrote: >> > Hi, >> > >> > On Thu, 2014-07-24 at 10:36 +0100, Mark Rutland wrote: >> >> On Thu, Jul 24, 2014 at 01:38:07AM +0100, Geoff Levand wrote: >> > >> >> > All memory management for the main cpu is done by the arch code. Kexec >> >> > and cpu hot plug only work with the secondary cpus, so the problem would >> >> > be in the arch memory code, either in setup_restart() for shutdown, or >> >> > in the startup code. >> >> >> >> It's possible that soft_restart and setup_restart are a little dodgy, as >> >> they also rely on the compiler being smart and not touching the stack >> >> after setup_restart(). >> >> >> >> However, they provide absolutely no guarantee that any data has been >> >> flushed out to the PoC [1]. If you require any data to be flushed out to the >> >> PoC so as to be visible to noncacheable accesses, you will need to >> >> ensure that this is flushed by VA before soft_restart is called. Data >> >> may have migrated to another cache (e.g. another CPU, or the L3) where >> >> it is not visible. >> > >> > OK, kexec's reset routine relocate_new_kernel does use a few global >> > variables that are set by the main kexec routines. I added a call to >> > __flush_dcache_area(), which uses 'dc civac' for those. >> > >> > I had thought the call to __flush_dcache_all, which uses 'dc cisw', in >> > flush_cache_all() would be enough. >> > >> > Arun, I also fixed UP builds. Could you pull my latest and try with L3 >> > enabled? >> > >> I got this working. As 'Mark Rutland' pointed in another mail that >> it could be problem with flushing the cache; I did a read of >> 1GB data from start of RAM to a volatile var. I assume that >> this will clear and invalidate all that in cache (L1=32K, L2=256 K, L3=8M) > > You've managed to get the cache to evict some lines, which proves my > theory, but this is absolute nonsense and guarantees nothing. > > So NAK to this. > Yes I was just shooting wildly. > If you need to perform cache maintenance to guarantee data is visible to > non-cacheable accesses you _must_ use the architected mechanism for > cleaning data to the PoC: DC CVAC. We have wrappers for flushing ranges. > > Anything else is nonsense and does not provide the guarantee you need. > > That said, I still am not sure what guarantee you are attempting to get. > Which data do you need out at the PoC? > I tried flushing the jump addr ########## +static unsigned long jump_addr_save; void soft_restart(unsigned long addr) { typedef void (*phys_reset_t)(unsigned long); phys_reset_t phys_reset; + phys_reset = (phys_reset_t)virt_to_phys(cpu_reset); + jump_addr_save = addr; + __flush_dcache_area(&jump_addr_save, 16); + __flush_dcache_area(&phys_reset, 16); setup_restart(); /* Switch to the identity mapping */ - phys_reset = (phys_reset_t)virt_to_phys(cpu_reset); - phys_reset(addr); + phys_reset(jump_addr_save); /* Should never get here */ BUG(); ########### And flushing all the source and destination pages of the kexeced kernel ########## case IND_DONE: ########### It still fails(not reboot to kexeced kernel); That means I miss flushing of some other area. --Arun > Thanks, > Mark. > >> >> ################### >> diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c >> index 786daa6..90418f3 100644 >> --- a/arch/arm64/kernel/process.c >> +++ b/arch/arm64/kernel/process.c >> @@ -63,6 +63,10 @@ static inline void smp_secondary_shutdown(void) {} >> >> static void setup_restart(void) >> { >> + volatile u64 tmp; >> + volatile u64 *addr; >> + >> + addr = (u64 *)(0xffffffc000000000); >> /* >> * Tell the mm system that we are going to reboot - >> * we may need it to insert some 1:1 mappings so that >> @@ -75,6 +79,11 @@ static void setup_restart(void) >> /* Clean and invalidate caches */ >> flush_cache_all(); >> >> + for ( ;addr < (u64 *)0xffffffc040000000; addr++) >> + { >> + tmp = *addr; >> + } >> + >> /* Turn D-cache off */ >> cpu_cache_off(); >> >> ################### >> >> With the above change latest kernel @ >> https://git.linaro.org/people/geoff.levand/linux-kexec.git >> is able to do kexec with L3 enabled in UP scenario. >> >> --Arun >> >> >> >> >> >> >> >> >> >> >> >> >> >> > -Geoff >> > >> diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 2995c78..3edf567 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -221,6 +221,8 @@ static void _kexec_entry_dump(const char *func, int line, addr, (unsigned long)virt_to_phys(dest), dest); + __flush_dcache_area(addr, PAGE_SIZE); + __flush_dcache_area(dest, PAGE_SIZE); dest += PAGE_SIZE; break;