From patchwork Wed Apr 12 18:57:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Radu Rendec X-Patchwork-Id: 13209425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 87A51C77B6E for ; Wed, 12 Apr 2023 19:00:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lpwZqpH6/jHDWd4dpDRrkB82hs0QZP+UB8N8k78F92I=; b=r1dhRtYqHz4wQv PqiXXEf/6zgEYcEo1F7KNZHO+4G7RBYzpcgMS46c7iiqYyNKog0oMeSshVsZqmbE2s2FXhX9CskK6 FOEstDTJkZ35ATYAOsgh2v4i2XGaUO3qHeJnP0nvkaGHfXlHtS0fX3wS6r62xTAgDQQCwiQ3TvJen X8tL07pso8XEw1MzXOkyFIIuQGv73NQYfSJF9AFdDDbm0rngwDsjMh4QqqyDsalPRjVkNofZl31M4 Lal+BSy879hLbItc7gWQlpLOp0oNPnZCMlSJsE5Hy0aqiQnJtXp0s8StuYpWplquo9/nSNnCRqHWE 6hjT2IT0e7D2LlTZn3yA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pmfgQ-0046qv-2y; Wed, 12 Apr 2023 18:59:06 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pmfgM-0046qJ-3A for linux-arm-kernel@lists.infradead.org; Wed, 12 Apr 2023 18:59:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681325942; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TIrahFdRAf91u6g7Hjbf+gtc3KP6J5YCSemB+yiV888=; b=Y4rGuvnYKNnfZUAPw/Mn7BPcGwN/53I7v3TsnSKbHDvZvWfsnDg1uXP2hm5YNeKltSxmPy V93GWjrGt28DtYg09RMf4+BfdTJAU2fbfTsOdNEZIDsQpZW/8Q4z0QXdccVJmCMjF4cIV7 8ZrXGZ3iZkFm2HGjzspDBGjkB3RlXQE= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-121-K_s-GYRiMoeJk1WqZv-0eA-1; Wed, 12 Apr 2023 14:59:00 -0400 X-MC-Unique: K_s-GYRiMoeJk1WqZv-0eA-1 Received: by mail-qt1-f197.google.com with SMTP id f2-20020ac87f02000000b003e6372b917dso27064328qtk.3 for ; Wed, 12 Apr 2023 11:59:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681325940; x=1683917940; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TIrahFdRAf91u6g7Hjbf+gtc3KP6J5YCSemB+yiV888=; b=kMW+qKB0YWHmM6RAbJkaCkv5OiKiOi4U0QZdM9Dx9Gp6P84lX0ALl8Zsp4OYJYcCxU 9h77Da1C/HIrbnlTH4Wliwnzi7EyJR+Uu6Cm1rXiPdV/ROD+mQt2IZM/QPXL01nQTZGh ITrIz9kD7/qAU8WmSlRv7oYi7LHm6LSPs06a6GHKHf/mUTjSnOEMsJMJndXvKdzTQ6xF LLS//o9N5Ow4Gcelvf7UUqr/1wlguPp7/0Iqm/GMzbQ30BxNkSE0+1kl9Fv4h2M0SMYL Qas4Zk6CCMxxSrLpGSWD5UNDBlFcniMgt5/xysZ5djNaGa7Laq0e5kJn0yEbtRyor7mA Bk7A== X-Gm-Message-State: AAQBX9dbKpyHMN9sRcii5UqN6P8w5m2digeoDis2WwLyWBUJtqTBOxet c7wd3k+v2Xox3Br4dAp7QvWS7hP/tsdmBu2Xd6X47R8nA4A/oQ58LVOZwpuCOJjj4A/TEgwGxtq rlVlL/L7TQvgJLfQttdEF/o5PL/54vvGUza8= X-Received: by 2002:a05:622a:514:b0:3bf:db0a:c105 with SMTP id l20-20020a05622a051400b003bfdb0ac105mr23439001qtx.6.1681325940431; Wed, 12 Apr 2023 11:59:00 -0700 (PDT) X-Google-Smtp-Source: AKy350bYTWjXEi+fFDnEWbAazjdSJa6gJGd3ECc3dgPFcbfvhSRgqSlfUFMQwEWyKO7xDlA/oN5ySQ== X-Received: by 2002:a05:622a:514:b0:3bf:db0a:c105 with SMTP id l20-20020a05622a051400b003bfdb0ac105mr23438976qtx.6.1681325940065; Wed, 12 Apr 2023 11:59:00 -0700 (PDT) Received: from thinkpad-p1.kanata.rendec.net (cpe00fc8d79db03-cm00fc8d79db00.cpe.net.fido.ca. [72.137.118.218]) by smtp.gmail.com with ESMTPSA id k21-20020ac86055000000b003d3a34d2eb2sm4417558qtm.41.2023.04.12.11.58.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Apr 2023 11:58:59 -0700 (PDT) From: Radu Rendec To: linux-kernel@vger.kernel.org Cc: Catalin Marinas , Will Deacon , Pierre Gondois , Sudeep Holla , linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 1/3] cacheinfo: Add arch specific early level initializer Date: Wed, 12 Apr 2023 14:57:57 -0400 Message-Id: <20230412185759.755408-2-rrendec@redhat.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230412185759.755408-1-rrendec@redhat.com> References: <20230412185759.755408-1-rrendec@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230412_115903_112777_445164CF X-CRM114-Status: GOOD ( 28.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch gives architecture specific code the ability to initialize the cache level and allocate cacheinfo memory early, when cache level initialization runs on the primary CPU for all possible CPUs. This is part of a patch series that attempts to further the work in commit 5944ce092b97 ("arch_topology: Build cacheinfo from primary CPU"). Previously, in the absence of any DT/ACPI cache info, architecture specific cache detection and info allocation for secondary CPUs would happen in non-preemptible context during early CPU initialization and trigger a "BUG: sleeping function called from invalid context" splat on an RT kernel. More specifically, this patch adds the early_cache_level() function, which is called by fetch_cache_info() as a fallback when the number of cache leaves cannot be extracted from DT/ACPI. In the default generic (weak) implementation, this new function returns -ENOENT, which preserves the original behavior for architectures that do not implement the function. Since early detection can get the number of cache leaves wrong in some cases*, additional logic is added to still call init_cache_level() later on the secondary CPU, therefore giving the architecture specific code an opportunity to go back and fix the initial guess. Again, the original behavior is preserved for architectures that do not implement the new function. * For example, on arm64, CLIDR_EL1 detection works only when it runs on the current CPU. In other words, a CPU cannot detect the cache depth for any other CPU than itself. Signed-off-by: Radu Rendec Reviewed-by: Pierre Gondois --- drivers/base/cacheinfo.c | 75 +++++++++++++++++++++++++++------------ include/linux/cacheinfo.h | 2 ++ 2 files changed, 55 insertions(+), 22 deletions(-) diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c index f3903d002819..d783896c8a1f 100644 --- a/drivers/base/cacheinfo.c +++ b/drivers/base/cacheinfo.c @@ -398,6 +398,11 @@ static void free_cache_attributes(unsigned int cpu) cache_shared_cpu_map_remove(cpu); } +int __weak early_cache_level(unsigned int cpu) +{ + return -ENOENT; +} + int __weak init_cache_level(unsigned int cpu) { return -ENOENT; @@ -423,56 +428,82 @@ int allocate_cache_info(int cpu) int fetch_cache_info(unsigned int cpu) { - struct cpu_cacheinfo *this_cpu_ci; + struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu); unsigned int levels = 0, split_levels = 0; int ret; if (acpi_disabled) { ret = init_of_cache_level(cpu); - if (ret < 0) - return ret; } else { ret = acpi_get_cache_info(cpu, &levels, &split_levels); - if (ret < 0) + if (!ret) { + this_cpu_ci->num_levels = levels; + /* + * This assumes that: + * - there cannot be any split caches (data/instruction) + * above a unified cache + * - data/instruction caches come by pair + */ + this_cpu_ci->num_leaves = levels + split_levels; + } + } + + if (ret || !cache_leaves(cpu)) { + ret = early_cache_level(cpu); + if (ret) return ret; - this_cpu_ci = get_cpu_cacheinfo(cpu); - this_cpu_ci->num_levels = levels; - /* - * This assumes that: - * - there cannot be any split caches (data/instruction) - * above a unified cache - * - data/instruction caches come by pair - */ - this_cpu_ci->num_leaves = levels + split_levels; + if (!cache_leaves(cpu)) + return -ENOENT; + + this_cpu_ci->early_ci_levels = true; } - if (!cache_leaves(cpu)) - return -ENOENT; return allocate_cache_info(cpu); } -int detect_cache_attributes(unsigned int cpu) +static inline int init_level_allocate_ci(unsigned int cpu) { - int ret; + unsigned int early_leaves = cache_leaves(cpu); /* Since early initialization/allocation of the cacheinfo is allowed * via fetch_cache_info() and this also gets called as CPU hotplug * callbacks via cacheinfo_cpu_online, the init/alloc can be skipped * as it will happen only once (the cacheinfo memory is never freed). - * Just populate the cacheinfo. + * Just populate the cacheinfo. However, if the cacheinfo has been + * allocated early through the arch-specific early_cache_level() call, + * there is a chance the info is wrong (this can happen on arm64). In + * that case, call init_cache_level() anyway to give the arch-specific + * code a chance to make things right. */ - if (per_cpu_cacheinfo(cpu)) - goto populate_leaves; + if (per_cpu_cacheinfo(cpu) && !ci_cacheinfo(cpu)->early_ci_levels) + return 0; if (init_cache_level(cpu) || !cache_leaves(cpu)) return -ENOENT; - ret = allocate_cache_info(cpu); + /* + * Now that we have properly initialized the cache level info, make + * sure we don't try to do that again the next time we are called + * (e.g. as CPU hotplug callbacks). + */ + ci_cacheinfo(cpu)->early_ci_levels = false; + + if (cache_leaves(cpu) <= early_leaves) + return 0; + + kfree(per_cpu_cacheinfo(cpu)); + return allocate_cache_info(cpu); +} + +int detect_cache_attributes(unsigned int cpu) +{ + int ret; + + ret = init_level_allocate_ci(cpu); if (ret) return ret; -populate_leaves: /* * If LLC is valid the cache leaves were already populated so just go to * update the cpu map. diff --git a/include/linux/cacheinfo.h b/include/linux/cacheinfo.h index 908e19d17f49..6147b2672555 100644 --- a/include/linux/cacheinfo.h +++ b/include/linux/cacheinfo.h @@ -76,9 +76,11 @@ struct cpu_cacheinfo { unsigned int num_levels; unsigned int num_leaves; bool cpu_map_populated; + bool early_ci_levels; }; struct cpu_cacheinfo *get_cpu_cacheinfo(unsigned int cpu); +int early_cache_level(unsigned int cpu); int init_cache_level(unsigned int cpu); int init_of_cache_level(unsigned int cpu); int populate_cache_leaves(unsigned int cpu);