From patchwork Tue Jun 19 16:32:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Pieralisi X-Patchwork-Id: 10475039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2A62A60383 for ; Tue, 19 Jun 2018 16:33:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1809328E8F for ; Tue, 19 Jun 2018 16:33:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0CBC728EBB; Tue, 19 Jun 2018 16:33:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A415A28E8F for ; Tue, 19 Jun 2018 16:33:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF2FB6B0005; Tue, 19 Jun 2018 12:33:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A7BC26B0006; Tue, 19 Jun 2018 12:33:07 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 944F16B0007; Tue, 19 Jun 2018 12:33:07 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot0-f197.google.com (mail-ot0-f197.google.com [74.125.82.197]) by kanga.kvack.org (Postfix) with ESMTP id 6838E6B0005 for ; Tue, 19 Jun 2018 12:33:07 -0400 (EDT) Received: by mail-ot0-f197.google.com with SMTP id h38-v6so152793otb.4 for ; Tue, 19 Jun 2018 09:33:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:date:from:to :cc:subject:message-id:references:mime-version:content-disposition :in-reply-to:user-agent; bh=nJ6HkDyuFqi+xX6qlEDpxVh8BgRI/j9zOnZXzp26BTA=; b=Km/07Ors5xOmhp7CXao8b+ug03W7Zp76rmGWcCUGV6c7IUkAvhswlXSewCjyQOumEU +E3TdqV8Ez4t0tloTv+bEn3UWuOcZRNqR0eEJTK2w071bufU4poln8geVvZR1BW3/KW1 KtVMgELi+mQT6CT4VFpQVZvWo0Vt4NML1IciHjPVEpIyt3vk6J14FB70TTzhmMe8CWGk ShxNLdZsMVh4t26qNFB5qGhTndEBW0KXvlmhjOLFRvBz8seTibG2uv3YRg1kJjKsNAI/ 2WhatIjduoPbnQbj8bjcdN0xC2XuyrIbt5fl+d0gmXFoBpwg2rP73CmteVpXQTQz7A1e cPbQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of lorenzo.pieralisi@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=lorenzo.pieralisi@arm.com X-Gm-Message-State: APt69E1F23dNSwN72/q/1yNF3XuG5BendrrFcGG2lUCfO95jRC2x+81u jMKK+VzDWaKoe7Ak6cqWq6hgAb8FnOJzcDU6aL47SEGoXT7mpNH/C5gMMhgychVrgm4MRp9X+Qq EjUPUadTUY2JNeFT/EBxDsycvWFpW5mmqI/1bQ2kvsr3WXiHL6fzzx+0nOeHrsgc1AA== X-Received: by 2002:a9d:3ca7:: with SMTP id z36-v6mr10012383otc.179.1529425987202; Tue, 19 Jun 2018 09:33:07 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKPgTEOz+YBZ8GPyCMUSU9p1bhiXQpPuU06SLUPSRqoq/4mD4+/xDf1P0DfuY+9YVb19qKk X-Received: by 2002:a9d:3ca7:: with SMTP id z36-v6mr10012344otc.179.1529425986235; Tue, 19 Jun 2018 09:33:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529425986; cv=none; d=google.com; s=arc-20160816; b=P1jQeYEGNepBqZWi5RALCD7r1FdLZRtNgaR0WcfHYWvEgFjG8PK7vwye9juO0TvDgF psi8LIADkMBtbDxaorL5dTHxUW88k8sh+4KgvBU2DIEI2NAKM+tfVRyb1/NrGbJo/ZKH +3PRlaKqwJGPH0muCDq4qEkDhgfN3ToYnXQUSkPEY4atOxzccfQmhciU1JNhNDin3KyJ j/BebNOQjlRNe2i130xLknSdRnhb0PVoFudmjSg9c8vqaD3aphUbUo6wtO0gntMrjN2S EpuK7NcumpnKwBi108OsEi54aEyKoTLOzWkRd9xIISAgrVDA2GJwxiXyTRjX0k/Klv01 lOCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:arc-authentication-results; bh=nJ6HkDyuFqi+xX6qlEDpxVh8BgRI/j9zOnZXzp26BTA=; b=GDEXzpCwG9Pgi/0lQTpcMV0QvMnbG3Lqp4Z46S/Uo6LS2cWUUb20LZvZS9cVXTE0eV NJw/S1jFtWNxLtLfr0d8LaJdkqbSJIT4aCpq7mkxeasPiik8I2MgKLTvYCo97YnOy2Dm fjYqn2Qc6UIS0VIyp4xOhtgG/j4mS+aySQfaRtbS3WgdyS60rV0/INDslV2srd9gufXQ am6TsDfxKGH9E6sMvr72R9trzr2nYFhoLGRDdHTt7QmYI6uYMOvv9NvCfE7UvuXhluVT OX9iQYfL1H/jeegf25ILyubHawU1sdOnyesdKwnhqEQ0wciKDEUWb/e8UbTCbHUiURr4 0loQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lorenzo.pieralisi@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=lorenzo.pieralisi@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id 63-v6si37085oto.190.2018.06.19.09.33.05 for ; Tue, 19 Jun 2018 09:33:06 -0700 (PDT) Received-SPF: pass (google.com: domain of lorenzo.pieralisi@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lorenzo.pieralisi@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=lorenzo.pieralisi@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 594461596; Tue, 19 Jun 2018 09:33:05 -0700 (PDT) Received: from e107981-ln.cambridge.arm.com (e107981-ln.cambridge.arm.com [10.1.207.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 443693F246; Tue, 19 Jun 2018 09:33:02 -0700 (PDT) Date: Tue, 19 Jun 2018 17:32:56 +0100 From: Lorenzo Pieralisi To: Punit Agrawal Cc: Michal Hocko , Xie XiuQi , Hanjun Guo , Bjorn Helgaas , tnowicki@caviumnetworks.com, linux-pci@vger.kernel.org, Catalin Marinas , "Rafael J. Wysocki" , Will Deacon , Linux Kernel Mailing List , Jarkko Sakkinen , linux-mm@kvack.org, wanghuiqiang@huawei.com, Greg Kroah-Hartman , Bjorn Helgaas , Andrew Morton , zhongjiang , linux-arm Subject: Re: [PATCH 1/2] arm64: avoid alloc memory on offline node Message-ID: <20180619163256.GA18952@e107981-ln.cambridge.arm.com> References: <20180611145330.GO13364@dhcp22.suse.cz> <87lgbk59gs.fsf@e105922-lin.cambridge.arm.com> <87bmce60y3.fsf@e105922-lin.cambridge.arm.com> <8b715082-14d4-f10b-d2d6-b23be7e4bf7e@huawei.com> <20180619120714.GE13685@dhcp22.suse.cz> <874lhz3pmn.fsf@e105922-lin.cambridge.arm.com> <20180619140818.GA16927@e107981-ln.cambridge.arm.com> <87wouu3jz1.fsf@e105922-lin.cambridge.arm.com> <20180619151425.GH13685@dhcp22.suse.cz> <87r2l23i2b.fsf@e105922-lin.cambridge.arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87r2l23i2b.fsf@e105922-lin.cambridge.arm.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On Tue, Jun 19, 2018 at 04:35:40PM +0100, Punit Agrawal wrote: > Michal Hocko writes: > > > On Tue 19-06-18 15:54:26, Punit Agrawal wrote: > > [...] > >> In terms of $SUBJECT, I wonder if it's worth taking the original patch > >> as a temporary fix (it'll also be easier to backport) while we work on > >> fixing these other issues and enabling memoryless nodes. > > > > Well, x86 already does that but copying this antipatern is not really > > nice. So it is good as a quick fix but it would be definitely much > > better to have a robust fix. Who knows how many other places might hit > > this. You certainly do not want to add a hack like this all over... > > Completely agree! I was only suggesting it as a temporary measure, > especially as it looked like a proper fix might be invasive. > > Another fix might be to change the node specific allocation to node > agnostic allocations. It isn't clear why the allocation is being > requested from a specific node. I think Lorenzo suggested this in one of > the threads. I think that code was just copypasted but it is better to fix the underlying issue. > I've started putting together a set fixing the issues identified in this > thread. It should give a better idea on the best course of action. On ACPI ARM64, this diff should do if I read the code correctly, it should be (famous last words) just a matter of mapping PXMs to nodes for every SRAT GICC entry, feel free to pick it up if it works. Yes, we can take the original patch just because it is safer for an -rc cycle even though if the patch below would do delaying the fix for a couple of -rc (to get it tested across ACPI ARM64 NUMA platforms) is not a disaster. Lorenzo -- >8 -- diff --git a/arch/arm64/kernel/acpi_numa.c b/arch/arm64/kernel/acpi_numa.c index d190a7b231bf..877b268ef9fa 100644 --- a/arch/arm64/kernel/acpi_numa.c +++ b/arch/arm64/kernel/acpi_numa.c @@ -70,12 +70,6 @@ void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) if (!(pa->flags & ACPI_SRAT_GICC_ENABLED)) return; - if (cpus_in_srat >= NR_CPUS) { - pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n", - NR_CPUS); - return; - } - pxm = pa->proximity_domain; node = acpi_map_pxm_to_node(pxm); @@ -85,6 +79,14 @@ void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) return; } + node_set(node, numa_nodes_parsed); + + if (cpus_in_srat >= NR_CPUS) { + pr_warn_once("SRAT: cpu_to_node_map[%d] is too small, may not be able to use all cpus\n", + NR_CPUS); + return; + } + mpidr = acpi_map_madt_entry(pa->acpi_processor_uid); if (mpidr == PHYS_CPUID_INVALID) { pr_err("SRAT: PXM %d with ACPI ID %d has no valid MPIDR in MADT\n", @@ -95,7 +97,6 @@ void __init acpi_numa_gicc_affinity_init(struct acpi_srat_gicc_affinity *pa) early_node_cpu_hwid[cpus_in_srat].node_id = node; early_node_cpu_hwid[cpus_in_srat].cpu_hwid = mpidr; - node_set(node, numa_nodes_parsed); cpus_in_srat++; pr_info("SRAT: PXM %d -> MPIDR 0x%Lx -> Node %d\n", pxm, mpidr, node);