From patchwork Tue Sep 7 00:25:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 12477453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3377C433F5 for ; Tue, 7 Sep 2021 00:27:20 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2677660F92 for ; Tue, 7 Sep 2021 00:27:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2677660F92 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:36934 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mNOxL-0006SH-Bw for qemu-devel@archiver.kernel.org; Mon, 06 Sep 2021 20:27:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59068) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mNOvs-00044c-Tk; Mon, 06 Sep 2021 20:25:48 -0400 Received: from mail-qk1-x72b.google.com ([2607:f8b0:4864:20::72b]:34807) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mNOvr-0005km-3u; Mon, 06 Sep 2021 20:25:48 -0400 Received: by mail-qk1-x72b.google.com with SMTP id a66so8444243qkc.1; Mon, 06 Sep 2021 17:25:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e0OAJGK7QQ67l/s45XfqlUOoltbSPMh/cftld7lIQUY=; b=XL7F/brNfhGHKSiMqb3FqtJ6w6/tBGPZtz4ev06XurnVhmqYfsRK+ziZoIQYHpAhjE C8M+9n7Mm1R5KDKuBQz96YXIXZuoQeFBwH4I1Fd4nFyC38Q8lHzb1QucCip2D+3kCkzh fCsFUpxJCCEV0Z5CM6X73hIfZhDJro27rPtfk9HmflhJXCqh8tGpZ/QFVt68WsfS0JiI dDfZJmG5NVL4czHY4ZnCxKwqiCHdaNRlk12lXZSzjCJGgEotiSIiHeEPh/eeHzc5kl3y VdpgdjSoL6/1i7PH3V1KWXc4w0CCl+yIOyu4IHjnLKzM0EqnW4o1EUXav5hwN5UIL112 Gxbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e0OAJGK7QQ67l/s45XfqlUOoltbSPMh/cftld7lIQUY=; b=FD7iBFO9rUS1Nelarul/uYLzsbLpTgZ9aCIpM9ei9mI7Yl1s08qpCgG3wvkMRuLMt3 3/bomZrBh+Z/5ot4BRJDakLTJ3+Sl653i2EFnh5h52Fh8v02v4OF01bC1wt8UcZkz3Ux MoRoJzFG40ReWOPF5Ig5NzFxqlz1ifrr4NrxNYNLDiMgoKYhcF8WLRpHen7JXQevHT1U eDAMeb9gtdkpMrdZ6lXsGpZg7jF44kGm9PbqFbcNqRs6BDiA2dCxRCSYW9iU1L8zP3dM 00Uh/1OWp7X+2shfQ90sUeIHJt8YfHs/5i4C2fAcKt0meT8eaXVZfH4jDsticI+2E7ob srOQ== X-Gm-Message-State: AOAM5309YANfHJfS906WQLMBBZJ74gA/MdnUUoUpc8xV3vY/BN/lk19y XyzpN72ip/jKaxIkZVmihlRiTruwuTE= X-Google-Smtp-Source: ABdhPJwp8v8gvnyy57uIf9GK9eX3wdbvWVn1qKMsUxKUVKjsiZHnWl8yiv9HACfamkN5XRA5Js3xCQ== X-Received: by 2002:a05:620a:4541:: with SMTP id u1mr13569614qkp.106.1630974345630; Mon, 06 Sep 2021 17:25:45 -0700 (PDT) Received: from rekt.COMFAST ([179.247.136.122]) by smtp.gmail.com with ESMTPSA id 207sm7892056qkh.45.2021.09.06.17.25.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Sep 2021 17:25:45 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v5 4/4] spapr: move FORM1 verifications to do_client_architecture_support() Date: Mon, 6 Sep 2021 21:25:27 -0300 Message-Id: <20210907002527.412013-5-danielhb413@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210907002527.412013-1-danielhb413@gmail.com> References: <20210907002527.412013-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::72b; envelope-from=danielhb413@gmail.com; helo=mail-qk1-x72b.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, groug@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" FORM2 NUMA affinity is prepared to deal with empty (memory/cpu less) NUMA nodes. This is used by the DAX KMEM driver to locate a PAPR SCM device that has a different latency than the original NUMA node from the regular memory. FORM2 is also enable to deal with asymmetric NUMA distances gracefully, something that our FORM1 implementation doesn't do. Move these FORM1 verifications to a new function and wait until after CAS, when we're sure that we're sticking with FORM1, to enforce them. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 33 ------------------------- hw/ppc/spapr_hcall.c | 6 +++++ hw/ppc/spapr_numa.c | 49 ++++++++++++++++++++++++++++++++----- include/hw/ppc/spapr_numa.h | 1 + 4 files changed, 50 insertions(+), 39 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 8d98e3b08a..c974c07fb8 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2797,39 +2797,6 @@ static void spapr_machine_init(MachineState *machine) /* init CPUs */ spapr_init_cpus(spapr); - /* - * check we don't have a memory-less/cpu-less NUMA node - * Firmware relies on the existing memory/cpu topology to provide the - * NUMA topology to the kernel. - * And the linux kernel needs to know the NUMA topology at start - * to be able to hotplug CPUs later. - */ - if (machine->numa_state->num_nodes) { - for (i = 0; i < machine->numa_state->num_nodes; ++i) { - /* check for memory-less node */ - if (machine->numa_state->nodes[i].node_mem == 0) { - CPUState *cs; - int found = 0; - /* check for cpu-less node */ - CPU_FOREACH(cs) { - PowerPCCPU *cpu = POWERPC_CPU(cs); - if (cpu->node_id == i) { - found = 1; - break; - } - } - /* memory-less and cpu-less node */ - if (!found) { - error_report( - "Memory-less/cpu-less nodes are not supported (node %d)", - i); - exit(1); - } - } - } - - } - spapr->gpu_numa_id = spapr_numa_initial_nvgpu_numa_id(machine); if ((!kvm_enabled() || kvmppc_has_cap_mmu_radix()) && diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index 7efbe93f4b..27ee713600 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -1202,9 +1202,15 @@ target_ulong do_client_architecture_support(PowerPCCPU *cpu, * If the guest chooses FORM2 we need to reset the associativity * information - it is being defaulted to FORM1 during * spapr_machine_reset(). + * + * If we're sure that we'll be using FORM1, verify now if we have + * a configuration or condition that is not available for FORM1 + * (namely asymmetric NUMA topologies and empty NUMA nodes). */ if (spapr_ovec_test(spapr->ov5_cas, OV5_FORM2_AFFINITY)) { spapr_numa_associativity_reset(spapr); + } else { + spapr_numa_check_FORM1_constraints(MACHINE(spapr)); } /* diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index ca276e16cb..0c57d03184 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -155,6 +155,49 @@ static void spapr_numa_define_associativity_domains(SpaprMachineState *spapr) } +void spapr_numa_check_FORM1_constraints(MachineState *machine) +{ + int i; + + if (!spapr_numa_is_symmetrical(machine)) { + error_report("Asymmetrical NUMA topologies aren't supported " + "in the pSeries machine"); + exit(EXIT_FAILURE); + } + + /* + * check we don't have a memory-less/cpu-less NUMA node + * Firmware relies on the existing memory/cpu topology to provide the + * NUMA topology to the kernel. + * And the linux kernel needs to know the NUMA topology at start + * to be able to hotplug CPUs later. + */ + if (machine->numa_state->num_nodes) { + for (i = 0; i < machine->numa_state->num_nodes; ++i) { + /* check for memory-less node */ + if (machine->numa_state->nodes[i].node_mem == 0) { + CPUState *cs; + int found = 0; + /* check for cpu-less node */ + CPU_FOREACH(cs) { + PowerPCCPU *cpu = POWERPC_CPU(cs); + if (cpu->node_id == i) { + found = 1; + break; + } + } + /* memory-less and cpu-less node */ + if (!found) { + error_report( + "Memory-less/cpu-less nodes are not supported (node %d)", + i); + exit(EXIT_FAILURE); + } + } + } + } +} + /* * Set NUMA machine state data based on FORM1 affinity semantics. */ @@ -172,12 +215,6 @@ static void spapr_numa_FORM1_affinity_init(SpaprMachineState *spapr, return; } - if (!spapr_numa_is_symmetrical(machine)) { - error_report("Asymmetrical NUMA topologies aren't supported " - "in the pSeries machine"); - exit(EXIT_FAILURE); - } - spapr_numa_define_associativity_domains(spapr); } diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index 0e457bba57..b5a19cb3f1 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -25,5 +25,6 @@ int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, int spapr_numa_write_assoc_lookup_arrays(SpaprMachineState *spapr, void *fdt, int offset); unsigned int spapr_numa_initial_nvgpu_numa_id(MachineState *machine); +void spapr_numa_check_FORM1_constraints(MachineState *machine); #endif /* HW_SPAPR_NUMA_H */