From patchwork Thu Jul 11 08:26:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730172 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4224F1514D1; Thu, 11 Jul 2024 08:27:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686426; cv=none; b=FtttdWHqbc/HUuQ1YoypOGAI2jN7uW9e18VGuLVeW0zgHaPSzRuYzQBYt3BvkUviQsIUmGTnKIXxybYH/mXhhAyvtdHAAWJxnivOx8tkpYHA8Xux9vvvfdzfzJ4U7AtTAh6ZYPfp735I6Fez+CcXCjfUG8Qmrzj+VC/43Hq8J/A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686426; c=relaxed/simple; bh=UjTUvT0oPIWScfHNdgXV79JFsNdQ0sqOip6ZjRaNG8w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jkyCC7fx5zIpyiuFeYqDQwvZbhI1qX+N6Ki5FYamBkJsmqFhiCe2PYa5tm6uBkE07SVlX/S4UjDvxdDRZfcV+dzF5qvkClFyWeB19NcEjeJCktum0nQM6fKx5yMIKA4M6fO8M7x8XTTWLoeoThfP+v9/lfzFvYV4si2h9JA1Ufc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GsowhUB0; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GsowhUB0" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-36796bbf687so272545f8f.0; Thu, 11 Jul 2024 01:27:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686422; x=1721291222; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CbtJeURzuXzMSUVq6dwyLPsKk+tyXx3nczbjfITmnRo=; b=GsowhUB0foHrtkIF3DhjYblkGCtB4d2fBOHPERaH0ibxGnXDyu2VcIIe2jAx3jPtFM WOXMoTXyVtTjXoc0A3WTh00jAoupQB3TCLqbNOG6pKJ70XThbktNJLeBasPGCIrlcZmR VwqdvGvLh0IHxBlRy7fldqasSMypvYrYHpOpUGpAqhcH3TAMxXZKjGTGaW92pulMUcm+ tVaT4SF0xcoItkDK0zRl1OeXipyXHy5rNMou77QM31fFm0psUMk+R1OBJOzBa1cnyDfY QFX/sTWq5DIeMg7t0MsFG+SgqZ1B33gCF8x5XWEvN908AOaeh4jJSz+xjwqk292kdQP2 YyNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686422; x=1721291222; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CbtJeURzuXzMSUVq6dwyLPsKk+tyXx3nczbjfITmnRo=; b=V+afthasqWJgjKD8wWN0LhzWbG0B8U4TFJar9nzGrgWkUIyJyKwQtpM35GEEHGaQnH SxZBLxTrq5AB9mifDoA+1MrXfrLawUBvFh6ANPnfwEEyrDRSzOcLzamAZyBrk1/9Uw5z i++gvaxz5ZIO5R3nI9Tp/IJ0gJXX8poMg6vFg34ZQYPE/ijbo0iPEgMePAYBb56u9cT0 KcXMBxpsabV5FJpo+0fhcmA3yUWriwfI/CzOWdYywB6YPOErq7Q56HwubCpvOgJ5pX/L JfavW6KOzzzoNBoy4rnzCOpPP+nqD6fGuJ5fUpf19soS39rS6VjZmiqLwjsKIgYNQhB2 e21g== X-Forwarded-Encrypted: i=1; AJvYcCW5xx3dxxN1YrQh36NIc+LeKG2Yli0NuvR2rKnY34luzqf0AyG9O88tf9oW16Zwrs0OYp2oqFfNT9SJucuHQMntPX2Jcl1wSpfta9gcHIRD+NJGDi/oruI1DIaMfNJQi254GS1VGQ/++g== X-Gm-Message-State: AOJu0YzVYOxAS3NSWOm1Dtn5UrFSUEAr3cQ9S95aYBPsjIfBMzUYL1h6 JQ7SKNTGp0RRWx+0vsQ+JcS9cLmzKy6OfRwuKIkKKY9ydIRKVygM X-Google-Smtp-Source: AGHT+IGRg+Owi/W3sasP/VODbkqgQBx34vKrV1FtkjaI5HVpD7Y+fzuRtSgGnCAui9tmXD6iY31ylw== X-Received: by 2002:adf:f204:0:b0:367:8a3a:3fd7 with SMTP id ffacd0b85a97d-367cea8fa23mr5750432f8f.33.1720686422183; Thu, 11 Jul 2024 01:27:02 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:01 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 01/11] MIPS: CPS: Add a couple of multi-cluster utility functions Date: Thu, 11 Jul 2024 10:26:46 +0200 Message-Id: <20240711082656.1889440-2-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton This patch introduces a couple of utility functions which help later patches with introducing support for multi-cluster systems. - mips_cps_multicluster_cpus() allows its caller to determine whether the system includes CPUs spread across multiple clusters. This is useful because in some cases behaviour can be more optimal taking this knowledge into account. The means by which we check this is dependent upon the way we probe CPUs & assign their numbers, so keeping this knowledge confined here in arch/mips/ seems appropriate. - mips_cps_first_online_in_cluster() allows its caller to determine whether it is running upon the first CPU online within its cluster. This information is useful in cases where some cluster-wide configuration may need to occur, but should not be repeated if another CPU in the cluster is already online. Similarly to the above this is determined based upon knowledge of CPU numbering so it makes sense to keep that knowledge in arch/mips/. The function is defined in mips-cm.c rather than in asm/mips-cps.h in order to allow us to use asm/cpu-info.h & linux/smp.h without encountering an include nightmare. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/include/asm/mips-cps.h | 39 ++++++++++++++++++++++++++++++++ arch/mips/kernel/mips-cm.c | 37 ++++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+) diff --git a/arch/mips/include/asm/mips-cps.h b/arch/mips/include/asm/mips-cps.h index c077e8d100f5..917009b80e69 100644 --- a/arch/mips/include/asm/mips-cps.h +++ b/arch/mips/include/asm/mips-cps.h @@ -8,6 +8,7 @@ #define __MIPS_ASM_MIPS_CPS_H__ #include +#include #include #include @@ -228,4 +229,42 @@ static inline unsigned int mips_cps_numvps(unsigned int cluster, unsigned int co return FIELD_GET(CM_GCR_Cx_CONFIG_PVPE, cfg + 1); } +/** + * mips_cps_multicluster_cpus() - Detect whether CPUs are in multiple clusters + * + * Determine whether the system includes CPUs in multiple clusters - ie. + * whether we can treat the system as single or multi-cluster as far as CPUs + * are concerned. Note that this is slightly different to simply checking + * whether multiple clusters are present - it is possible for there to be + * clusters which contain no CPUs, which this function will effectively ignore. + * + * Returns true if CPUs are spread across multiple clusters, else false. + */ +static inline bool mips_cps_multicluster_cpus(void) +{ + unsigned int first_cl, last_cl; + + /* + * CPUs are numbered sequentially by cluster - ie. CPUs 0..X will be in + * cluster 0, CPUs X+1..Y in cluster 1, CPUs Y+1..Z in cluster 2 etc. + * + * Thus we can detect multiple clusters trivially by checking whether + * the first & last CPUs belong to the same cluster. + */ + first_cl = cpu_cluster(&boot_cpu_data); + last_cl = cpu_cluster(&cpu_data[nr_cpu_ids - 1]); + return first_cl != last_cl; +} + +/** + * mips_cps_first_online_in_cluster() - Detect if CPU is first online in cluster + * + * Determine whether the local CPU is the first to be brought online in its + * cluster - that is, whether there are any other online CPUs in the local + * cluster. + * + * Returns true if this CPU is first online, else false. + */ +extern unsigned int mips_cps_first_online_in_cluster(void); + #endif /* __MIPS_ASM_MIPS_CPS_H__ */ diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c index 3a115fab5573..3eb2cfb893e1 100644 --- a/arch/mips/kernel/mips-cm.c +++ b/arch/mips/kernel/mips-cm.c @@ -512,3 +512,40 @@ void mips_cm_error_report(void) /* reprime cause register */ write_gcr_error_cause(cm_error); } + +unsigned int mips_cps_first_online_in_cluster(void) +{ + unsigned int local_cl; + int i; + + local_cl = cpu_cluster(¤t_cpu_data); + + /* + * We rely upon knowledge that CPUs are numbered sequentially by + * cluster - ie. CPUs 0..X will be in cluster 0, CPUs X+1..Y in cluster + * 1, CPUs Y+1..Z in cluster 2 etc. This means that CPUs in the same + * cluster will immediately precede or follow one another. + * + * First we scan backwards, until we find an online CPU in the cluster + * or we move on to another cluster. + */ + for (i = smp_processor_id() - 1; i >= 0; i--) { + if (cpu_cluster(&cpu_data[i]) != local_cl) + break; + if (!cpu_online(i)) + continue; + return false; + } + + /* Then do the same for higher numbered CPUs */ + for (i = smp_processor_id() + 1; i < nr_cpu_ids; i++) { + if (cpu_cluster(&cpu_data[i]) != local_cl) + break; + if (!cpu_online(i)) + continue; + return false; + } + + /* We found no online CPUs in the local cluster */ + return true; +} From patchwork Thu Jul 11 08:26:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730173 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC3F2153581; Thu, 11 Jul 2024 08:27:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686427; cv=none; b=BXpqJBCtB8tKLKQeOTjOE/bUcZUKI5PsgrL/Cvw7JO30MP8bQv3THh5VTVudhKITZB4/P6Q6LAq5o+2Bjpj/9deLEYbDmTyOhgk67OUlldb2Hbq7wbKgPIUVeX9Ev78L2sm69hhQ2sQlDm2HJwEM9Jk+M4SQbi03goHPEg+6XhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686427; c=relaxed/simple; bh=YqXfpzlrn+tQ5pRH/5JSmt1daHU/VeJILheS7aM7wuo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RETTNeTSXCZPShDLhq4u0Z5iIA4mo/wgpqPShNRNqwJkRwHmK4x6/sulGMS/0jjsjFNG5KTa2NfccqPTQjfpjfy4603+tcElzc5w8CUqo0mNvEdB2ovdW80iJwuAJwiybBLAnxQhMHq9b9FafJWudg15fb/COo6qkEaOxrkZoJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=anA296jV; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="anA296jV" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-367e50f71bbso454690f8f.2; Thu, 11 Jul 2024 01:27:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686424; x=1721291224; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SoGjIyylq/2U1UdDbKKzqWV3owFDbnfS+xDpWv/w24A=; b=anA296jVbphQTINDJ5o6T0LKaF20n8I8FxWIWxdzcfaSTEqBd4aKU4XYAZhBYkZbjE d2Hqv5EPHcinPFu83CUSK8ea62Z1juRwS7vYQoqYFRRTJrzgRT6/MdMS2uYoj48FqRwI R0Ys3Ldlcxf1iFMAqf4/ITA4Z5VnRDl3GyQ9XP7+43NT3JfadQqRMN6JCk1uyb7uwWIy 0HE4jGLt4e+zoibzDwsPGFwcP/UwDMw5J7HAlNGJGyRbFgTHkVio2FwBp4Yd8zmxYxGf Esig8Wt/9pvNnaTuZI5w9YXQFyJ63hJkNC2HXGPa3+iA++VlzeIJE1CZhSnO1AfEcPzH etdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686424; x=1721291224; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SoGjIyylq/2U1UdDbKKzqWV3owFDbnfS+xDpWv/w24A=; b=BlhHrCZXgjcUK+9/phZvFq5kdjWvXnlqjrsZQspoD/tU5C1jFQ2ti10vvgBG6qSeFm aBJoWI6c0vNkuoe9E5ILaZbdlXmnFHJEMFjKRJ8lPkN1ZMR5KtxqQ3auX7NfegqqVB+D sjP8WWUB1Iii6gb/QMhBV416I7h+Vh5jU2slBeGoKn7zP2ktAF2kDQ0k/J121wJynOfj 87Jozm7GoPaIUeyAPEy7sUv7FIMejTJeUsa5XJySXe7jXKO8hJ1Ot2y4+SU01Gdcrl6Y /0OvlLrzpasLJe/02PMSId9a2mGF8fJoXvxQe73NunyDXv5UtoRl3xyhKJ7hLcgXoTfD GAwA== X-Forwarded-Encrypted: i=1; AJvYcCWz5oQoKjrhzjXNGxaJPqgdNuapZFTvC+aGdrixTpWMEm3WunL8ow6l3+5BGf0GQ4MjbjxQLfslWXAiwsn6gKGNAShjh6x2VO/AzPMM22jfKBpCmL1r6CNJT6srJKwx4k+TEpAwnUM9Ow== X-Gm-Message-State: AOJu0YzzrcOa7bb3mCwMncqiWu3P+KSDwo33ZFzVDoRRTirFxgsQOtwv U9WWVtLoUmgSlGAs6tFFtw/mS6zn/UJbKjiG0xi3pCFX0zdfbc/w X-Google-Smtp-Source: AGHT+IEm9yPJ1zLNIjg4u8xmCjz1g5S1kD7ahImT+qOHrp/AXsNNUzk/WyBwfj6wEUjb5sqFrfNgpg== X-Received: by 2002:a5d:470a:0:b0:35f:d57:6e3f with SMTP id ffacd0b85a97d-367cea96613mr4931059f8f.31.1720686424057; Thu, 11 Jul 2024 01:27:04 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:03 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 02/11] MIPS: GIC: Generate redirect block accessors Date: Thu, 11 Jul 2024 10:26:47 +0200 Message-Id: <20240711082656.1889440-3-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton With CM 3.5 the "core-other" register block evolves into the "redirect" register block, which is capable of accessing not only the core local registers of other cores but also the shared/global registers of other clusters. This patch generates accessor functions for shared/global registers accessed via the redirect block, with "redir_" inserted after "gic_" in their names. For example the accessor function: read_gic_config() ...accesses the GIC_CONFIG register of the GIC in the local cluster. With this patch a new function: read_gic_redir_config() ...is added which accesses the GIC_CONFIG register of the GIC in whichever cluster the GCR_CL_REDIRECT register is configured to access. This mirrors the similar redirect block accessors already provided for the CM & CPC. Reviewed-by: Jiaxun Yang Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/include/asm/mips-gic.h | 50 ++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 16 deletions(-) diff --git a/arch/mips/include/asm/mips-gic.h b/arch/mips/include/asm/mips-gic.h index 084cac1c5ea2..fd9da5e3beaa 100644 --- a/arch/mips/include/asm/mips-gic.h +++ b/arch/mips/include/asm/mips-gic.h @@ -28,11 +28,13 @@ extern void __iomem *mips_gic_base; /* For read-only shared registers */ #define GIC_ACCESSOR_RO(sz, off, name) \ - CPS_ACCESSOR_RO(gic, sz, MIPS_GIC_SHARED_OFS + off, name) + CPS_ACCESSOR_RO(gic, sz, MIPS_GIC_SHARED_OFS + off, name) \ + CPS_ACCESSOR_RO(gic, sz, MIPS_GIC_REDIR_OFS + off, redir_##name) /* For read-write shared registers */ #define GIC_ACCESSOR_RW(sz, off, name) \ - CPS_ACCESSOR_RW(gic, sz, MIPS_GIC_SHARED_OFS + off, name) + CPS_ACCESSOR_RW(gic, sz, MIPS_GIC_SHARED_OFS + off, name) \ + CPS_ACCESSOR_RW(gic, sz, MIPS_GIC_REDIR_OFS + off, redir_##name) /* For read-only local registers */ #define GIC_VX_ACCESSOR_RO(sz, off, name) \ @@ -45,7 +47,7 @@ extern void __iomem *mips_gic_base; CPS_ACCESSOR_RW(gic, sz, MIPS_GIC_REDIR_OFS + off, vo_##name) /* For read-only shared per-interrupt registers */ -#define GIC_ACCESSOR_RO_INTR_REG(sz, off, stride, name) \ +#define _GIC_ACCESSOR_RO_INTR_REG(sz, off, stride, name) \ static inline void __iomem *addr_gic_##name(unsigned int intr) \ { \ return mips_gic_base + (off) + (intr * (stride)); \ @@ -58,8 +60,8 @@ static inline unsigned int read_gic_##name(unsigned int intr) \ } /* For read-write shared per-interrupt registers */ -#define GIC_ACCESSOR_RW_INTR_REG(sz, off, stride, name) \ - GIC_ACCESSOR_RO_INTR_REG(sz, off, stride, name) \ +#define _GIC_ACCESSOR_RW_INTR_REG(sz, off, stride, name) \ + _GIC_ACCESSOR_RO_INTR_REG(sz, off, stride, name) \ \ static inline void write_gic_##name(unsigned int intr, \ unsigned int val) \ @@ -68,22 +70,30 @@ static inline void write_gic_##name(unsigned int intr, \ __raw_writel(val, addr_gic_##name(intr)); \ } +#define GIC_ACCESSOR_RO_INTR_REG(sz, off, stride, name) \ + _GIC_ACCESSOR_RO_INTR_REG(sz, off, stride, name) \ + _GIC_ACCESSOR_RO_INTR_REG(sz, MIPS_GIC_REDIR_OFS + off, stride, redir_##name) + +#define GIC_ACCESSOR_RW_INTR_REG(sz, off, stride, name) \ + _GIC_ACCESSOR_RW_INTR_REG(sz, off, stride, name) \ + _GIC_ACCESSOR_RW_INTR_REG(sz, MIPS_GIC_REDIR_OFS + off, stride, redir_##name) + /* For read-only local per-interrupt registers */ #define GIC_VX_ACCESSOR_RO_INTR_REG(sz, off, stride, name) \ - GIC_ACCESSOR_RO_INTR_REG(sz, MIPS_GIC_LOCAL_OFS + off, \ + _GIC_ACCESSOR_RO_INTR_REG(sz, MIPS_GIC_LOCAL_OFS + off, \ stride, vl_##name) \ - GIC_ACCESSOR_RO_INTR_REG(sz, MIPS_GIC_REDIR_OFS + off, \ + _GIC_ACCESSOR_RO_INTR_REG(sz, MIPS_GIC_REDIR_OFS + off, \ stride, vo_##name) /* For read-write local per-interrupt registers */ #define GIC_VX_ACCESSOR_RW_INTR_REG(sz, off, stride, name) \ - GIC_ACCESSOR_RW_INTR_REG(sz, MIPS_GIC_LOCAL_OFS + off, \ + _GIC_ACCESSOR_RW_INTR_REG(sz, MIPS_GIC_LOCAL_OFS + off, \ stride, vl_##name) \ - GIC_ACCESSOR_RW_INTR_REG(sz, MIPS_GIC_REDIR_OFS + off, \ + _GIC_ACCESSOR_RW_INTR_REG(sz, MIPS_GIC_REDIR_OFS + off, \ stride, vo_##name) /* For read-only shared bit-per-interrupt registers */ -#define GIC_ACCESSOR_RO_INTR_BIT(off, name) \ +#define _GIC_ACCESSOR_RO_INTR_BIT(off, name) \ static inline void __iomem *addr_gic_##name(void) \ { \ return mips_gic_base + (off); \ @@ -106,8 +116,8 @@ static inline unsigned int read_gic_##name(unsigned int intr) \ } /* For read-write shared bit-per-interrupt registers */ -#define GIC_ACCESSOR_RW_INTR_BIT(off, name) \ - GIC_ACCESSOR_RO_INTR_BIT(off, name) \ +#define _GIC_ACCESSOR_RW_INTR_BIT(off, name) \ + _GIC_ACCESSOR_RO_INTR_BIT(off, name) \ \ static inline void write_gic_##name(unsigned int intr) \ { \ @@ -146,6 +156,14 @@ static inline void change_gic_##name(unsigned int intr, \ } \ } +#define GIC_ACCESSOR_RO_INTR_BIT(off, name) \ + _GIC_ACCESSOR_RO_INTR_BIT(off, name) \ + _GIC_ACCESSOR_RO_INTR_BIT(MIPS_GIC_REDIR_OFS + off, redir_##name) + +#define GIC_ACCESSOR_RW_INTR_BIT(off, name) \ + _GIC_ACCESSOR_RW_INTR_BIT(off, name) \ + _GIC_ACCESSOR_RW_INTR_BIT(MIPS_GIC_REDIR_OFS + off, redir_##name) + /* For read-only local bit-per-interrupt registers */ #define GIC_VX_ACCESSOR_RO_INTR_BIT(sz, off, name) \ GIC_ACCESSOR_RO_INTR_BIT(sz, MIPS_GIC_LOCAL_OFS + off, \ @@ -155,10 +173,10 @@ static inline void change_gic_##name(unsigned int intr, \ /* For read-write local bit-per-interrupt registers */ #define GIC_VX_ACCESSOR_RW_INTR_BIT(sz, off, name) \ - GIC_ACCESSOR_RW_INTR_BIT(sz, MIPS_GIC_LOCAL_OFS + off, \ - vl_##name) \ - GIC_ACCESSOR_RW_INTR_BIT(sz, MIPS_GIC_REDIR_OFS + off, \ - vo_##name) + _GIC_ACCESSOR_RW_INTR_BIT(sz, MIPS_GIC_LOCAL_OFS + off, \ + vl_##name) \ + _GIC_ACCESSOR_RW_INTR_BIT(sz, MIPS_GIC_REDIR_OFS + off, \ + vo_##name) /* GIC_SH_CONFIG - Information about the GIC configuration */ GIC_ACCESSOR_RW(32, 0x000, config) From patchwork Thu Jul 11 08:26:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730174 Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FE58155A47; Thu, 11 Jul 2024 08:27:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686430; cv=none; b=NXWslGwdpa9yr0RMMcYW+wCgP7CWNSYUrgH7eFvG47bIA3X5BWNS7Nv/rfzcKoS5ERDM9pD05+IPo+KBQOjfPb3lhjVgpNB75hUr1dUI9Kz/aTEtVmUW7L1pIqHNxeAJ5l7IsW8pn6uoxgAyrI9TNIWqH93sMkDHReYRCJ3CYj4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686430; c=relaxed/simple; bh=lRP8PrGqJ+uo9Zp6g0Qm8qLkDotTHxE60FSRnQlZCsM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rhAShUnmIvlDPuyWji9HCgtlxt0DXDqeYNUYd6f2pb5FoCh6E5bLoi/XQ4FFT1TEGV8UZdfIbAlIsCAxoRRhdc6SGTZHxg8zaJjpS4jBipoENnL77i9bATFYGAr77q5m5n7SWH8vBJFAGqkVAd/dVJHCXfVPC2qzfil8qpZtNIU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ARpZTf85; arc=none smtp.client-ip=209.85.208.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ARpZTf85" Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2eec7e431d9so6496331fa.2; Thu, 11 Jul 2024 01:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686426; x=1721291226; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ReifRjJ+F7L7a0QBvncSYAOaGaiMv8UukF6KYGpSd/w=; b=ARpZTf85SKLKvne4AzMZ9UTfuEJEw4VV46YIzy7QpwUbjRXac7O1tvvF07XNxxx5bm vRCb8U/fxLA7mgB6w7v5MYeIPSczklM0zcBEvFdyMACBz0/5DV77vj058SmVuz+QIPiQ ek7gNoVsLeJ5y8AA4SEUHcWO+tfeJeYh6lkno7T2FukZbUzRL88TyI89uzrs4SC9fkZl v3KATvJC58IYvGQ7cLEWgcbiO+E76TKY+2Lf7FSWtmFT/ONpYuCKJaJLToxwdr9uRxMB YXq9Jni+os8bQCyYm+nsyAE8TL5TnTHX3y3TwFCDC5H93TjWLfqgQdPSwx3I1XEVD6Kk nIAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686426; x=1721291226; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ReifRjJ+F7L7a0QBvncSYAOaGaiMv8UukF6KYGpSd/w=; b=F9gBAfMk1WAhcCjK6qUknjOP3wnfD5mnvu3IvNauQzuMft/2ew8fsA7/9LmtYdLW6b +kf4JxA8UFqFTc70FFX796tFNE8bU6AhjLozDw3OutR8XncZ/LG0AalnZJeC1q4oZmE+ xpfUGVYpUyyuOV+WvI+Fhl4M8X41snQOukUXShlbLVG239UAODeaNS3h2Nnx8v8KRW0N ZhNHWkDszwl817bUt7+gcOWfqOz5dfr8oQm89GoEAauulTevgWiQ6lChIa/DBT5SkTef 2AZi3yq3GSUqF6kLSKHfo1Xz+z/Mae2fq7MkhKBdagPJuX2ynjnXrV3aoQI3hbwoZBRq T+BQ== X-Forwarded-Encrypted: i=1; AJvYcCVWywWTgDKd7T2MzakEm8eOZE85+SM7zCFMBTCwABqidxC5afGS0ZCOVcAavLMVa1putuUDrj2v/Z5PifebAw5R7DN6Q40S9IcKYzZp3lmcVe5a4E1Qes4SQJWjB/d7ZWy61ykn/4CnCw== X-Gm-Message-State: AOJu0YyZCnp5tsNorCs4Qj20r0fJRhntVcBkgKTDwBsZDbqwkNu1PYMy lVqR8Ot8IdgCvA6vTTR4gKLwRqzeX0juwKqYh3M+r74q1AbBvLZn X-Google-Smtp-Source: AGHT+IE85Ukum8VZjJ8MbuTew4337izeAeArigurhDA/mX33JddC/tDb/b11mpGLgtyMhoyaVk+Qzg== X-Received: by 2002:a2e:a7d0:0:b0:2ee:8a88:b060 with SMTP id 38308e7fff4ca-2eeb3191ab3mr62260971fa.53.1720686426174; Thu, 11 Jul 2024 01:27:06 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:05 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 03/11] irqchip/mips-gic: Introduce for_each_online_cpu_gic() Date: Thu, 11 Jul 2024 10:26:48 +0200 Message-Id: <20240711082656.1889440-4-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton Parts of code in the MIPS GIC driver operate on the GIC local register block for each online CPU, accessing each via the GIC's other/redirect register block. Abstract the process of iterating over online CPUs & configuring the other/redirect region to access their registers through a new for_each_online_cpu_gic() macro. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Signed-off-by: Thomas Gleixner Tested-by: Serge Semin --- drivers/irqchip/irq-mips-gic.c | 59 +++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 76253e864f23..6c7a7d2f0438 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -66,6 +66,44 @@ static struct gic_all_vpes_chip_data { bool mask; } gic_all_vpes_chip_data[GIC_NUM_LOCAL_INTRS]; +static int __gic_with_next_online_cpu(int prev) +{ + unsigned int cpu; + + /* Discover the next online CPU */ + cpu = cpumask_next(prev, cpu_online_mask); + + /* If there isn't one, we're done */ + if (cpu >= nr_cpu_ids) + return cpu; + + /* + * Move the access lock to the next CPU's GIC local register block. + * + * Set GIC_VL_OTHER. Since the caller holds gic_lock nothing can + * clobber the written value. + */ + write_gic_vl_other(mips_cm_vp_id(cpu)); + + return cpu; +} + +/** + * for_each_online_cpu_gic() - Iterate over online CPUs, access local registers + * @cpu: An integer variable to hold the current CPU number + * @gic_lock: A pointer to raw spin lock used as a guard + * + * Iterate over online CPUs & configure the other/redirect register region to + * access each CPUs GIC local register block, which can be accessed from the + * loop body using read_gic_vo_*() or write_gic_vo_*() accessor functions or + * their derivatives. + */ +#define for_each_online_cpu_gic(cpu, gic_lock) \ + guard(raw_spinlock_irqsave)(gic_lock); \ + for ((cpu) = __gic_with_next_online_cpu(-1); \ + (cpu) < nr_cpu_ids; \ + (cpu) = __gic_with_next_online_cpu(cpu)) + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -350,37 +388,27 @@ static struct irq_chip gic_local_irq_controller = { static void gic_mask_local_irq_all_vpes(struct irq_data *d) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; int intr, cpu; intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); cd = irq_data_get_irq_chip_data(d); cd->mask = false; - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_rmask(BIT(intr)); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); } static void gic_unmask_local_irq_all_vpes(struct irq_data *d) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; int intr, cpu; intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); cd = irq_data_get_irq_chip_data(d); cd->mask = true; - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_smask(BIT(intr)); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); } static void gic_all_vpes_irq_cpu_online(void) @@ -469,7 +497,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hwirq) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; unsigned int intr; int err, cpu; u32 map; @@ -533,12 +560,8 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, if (!gic_local_irq_is_routable(intr)) return -EPERM; - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_map(mips_gic_vx_map_reg(intr), map); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); return 0; } From patchwork Thu Jul 11 08:26:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730175 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B388155CBA; Thu, 11 Jul 2024 08:27:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686431; cv=none; b=L5b/7nMF/sWy3sc+gkZ108yb4jyGSBPspKhmHXlpXXODMyfyUidgpgCRdnIpZqT8dAhaYTRtzJu+aTYMAoPO7mH/VSSh/P7Ozar6nM5nSTcza8GeutKZ1+pGBvrmq+a7BOWBpruU2DKZIdDSLtJzJZw3tKCbyeKXoLXrexf2I1U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686431; c=relaxed/simple; bh=zUVIkISW3jZnhcMxLsleQnjjg0Ud5N6so9W6XSuUCpE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sExldKe27YA8duXDu8HHu2zVGZPClZpUSUD24bIPwax0gZgKUGNFWlnFpvArfPigZdyTnqMhkpNzKSO8dJ+3pibz1+7MYXL50Ya/3mgQJSVXSe8yS4IUeh3DWWqZZ0GHjJDKigTbVGcQLOODmbItbeVuu2k88ycUaE2ZaV7WX58= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZSb4hLS8; arc=none smtp.client-ip=209.85.221.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZSb4hLS8" Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-3678fbf4a91so282236f8f.1; Thu, 11 Jul 2024 01:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686428; x=1721291228; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sOO3jOTAGVFfwBWAICwL9MIO5uU5dpebDIzy/18ap1w=; b=ZSb4hLS8qzvcQnqu7tKj3tV8Uv9QgQS7YEfO209iD79EDkBsQYcosjiQjYApTKZgH1 SqZ1f6Mqj8XUpef25pVL8++12rPnSg32byCwlukyGY3+kKxy4JWi/uPYoSKKig1YDPEa S7TUHDmxNyMRZ35KfdhPL0Isz4D0eoIDkz3zkmYNrWdbrV3vULXYquN7gqRhAfe+jTUv 6JYXGcaGAb5+suKNpahTP3Be8Nlv0AAUUZk9rHnSVNFNzrpyrxO67DyTmuKmkxytvld8 KaB8gElu1luUg/ibl49ERD35u76tqlSLxiDuNCD09s8AGlrlBecL1uJ1d7FhrWR43N5K pU3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686428; x=1721291228; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sOO3jOTAGVFfwBWAICwL9MIO5uU5dpebDIzy/18ap1w=; b=WzjEuxWcWcokcV+srZRaKjzmPfyduSuewURoRKfL5Kyo1xNE3TxqkRc44mAWNntqsL nhEH+4EB63jqnPRD/cfKgLkZqrqyyHUU6YaSWK5i4apuk+QjD9pocYKpwLlFebnxy4ij KAdSLsxOOk0C9K4gT5IRNtJ78bXmpGKxwJrK21SnmjTXH/XP5bMVEBxZnXF+GC4721lB qhTIeqYyDg11eSpGJAhkJ7blFuiornlGwyYbv/rg5PeNQTtETxbe4Zv6yFGmjJnj7Jsz 2PRzwwUoD0m4bgqq48RXeewwEgVn05P1BP/o55kTGbR47+b6linK0n+aSlY/Wmp1xA6x GOoA== X-Forwarded-Encrypted: i=1; AJvYcCUlgUDJG1ApdhdL8p2d6kYG9ncb21QWHtM1pIQbwsM9f0xCJd6fs/EOLpSbeoWQe2yjSQcDJtjI6HDcLDMrJaJlarwmgvUiSFIinF97ZB/HJto/daM0EvPU5/w4hixWbJxyvlanvwqAwg== X-Gm-Message-State: AOJu0YzxyaTtnqOQaocMuuM817sntyXFq48hKLLEcdm5/kHEdiokpqGf 5onSH3VlPRJ4236JfKK4HlaU/OhpTZs2cyB4GjTuhgHh4eTr7hmw X-Google-Smtp-Source: AGHT+IE9wgH2mUydYc/dbEwsCa2ULDGix/rao40AXMd3CPfxnFN4bkrvzhEzSMFAzpGl2wJLnya4dw== X-Received: by 2002:adf:cc88:0:b0:367:94ee:734c with SMTP id ffacd0b85a97d-367cea46df5mr5009692f8f.14.1720686427799; Thu, 11 Jul 2024 01:27:07 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:07 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 04/11] irqchip/mips-gic: Support multi-cluster in for_each_online_cpu_gic() Date: Thu, 11 Jul 2024 10:26:49 +0200 Message-Id: <20240711082656.1889440-5-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton Use CM's GCR_CL_REDIRECT register to access registers in remote clusters, so users of gic_with_each_online_cpu() gain support for multi-cluster with no further changes. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Signed-off-by: Thomas Gleixner Tested-by: Serge Semin --- drivers/irqchip/irq-mips-gic.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 6c7a7d2f0438..e7358d3f4e74 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -98,10 +98,17 @@ static int __gic_with_next_online_cpu(int prev) * loop body using read_gic_vo_*() or write_gic_vo_*() accessor functions or * their derivatives. */ +static inline void gic_unlock_cluster(void) +{ + if (mips_cps_multicluster_cpus()) + mips_cm_unlock_other(); +} + #define for_each_online_cpu_gic(cpu, gic_lock) \ guard(raw_spinlock_irqsave)(gic_lock); \ for ((cpu) = __gic_with_next_online_cpu(-1); \ (cpu) < nr_cpu_ids; \ + gic_unlock_cluster(), \ (cpu) = __gic_with_next_online_cpu(cpu)) static void gic_clear_pcpu_masks(unsigned int intr) From patchwork Thu Jul 11 08:26:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730176 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C778156F3A; Thu, 11 Jul 2024 08:27:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686433; cv=none; b=b7uXb2/7IdGAkFzB3vcW/wTtehQwS7L6hrxzlJBNQHWrqvRyimLFZY2T2yTTOwYjcQ3N2NgMMePQKWLZ3BaLcl7ubqCbbWcQHloVbOUkR3G0PjQaJCyPvehK0pR8+ToBDza/WqGOYlFOa/NFKWBXvpXUEi5OO5E+baw+kicnbEI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686433; c=relaxed/simple; bh=/xE0w9SR+3yMBtreBYEwxXqcCREle6Eog1NUc4b7uGo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=j97OoLYfHJsVbh1fKIPyd0p2zWc3N4+wWcPlB+IvnQ9POSDNyS7Kwr7Dq3ristSr8NpegSempadjth8eTrLvip9sB1p9NCI7YrkYARyU0U6Yq4DMTJTE1dyKWW9h6pr4+PsQ48LRkWWH302QNVDnakOLiA/5pUcxqaijmdT4TxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KMjK1bUD; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KMjK1bUD" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4265c2b602aso4014125e9.3; Thu, 11 Jul 2024 01:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686430; x=1721291230; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D1Vgvtd2WAkghL37NXwrCMUEOPeUgVOtQe2UKqlBzbU=; b=KMjK1bUDsQ4AapU5VIJxNTMOyIBJnKkEVpDXno0ESQyS0HbfSxQAy8RooGOq2wDOOw Iljvcvv8L8uiigcLCHzBPfshOZqlkX81QT/iQ/XnX6b1Z7Fp6kTdm+XqYoGGK8wnnuYz CwEHGUfbBrrZ7MVwRhC2ib4lMf8x830FhlFL4HiVdVLzjJuPNH702uYD3IgL8vllc3Gw Y+5QQmo+VwLT9gQZHUYSoFqYJcex7qgmaJwUMJwUD/snYdKE+io8QuDecG1yVRjYYpln o7/mD+tWjsIFrbaf/5N0crXpR4+q3plQqB2xwL1v8b2IPe891oCyluIFTul9CWs1kzsB 18pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686430; x=1721291230; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D1Vgvtd2WAkghL37NXwrCMUEOPeUgVOtQe2UKqlBzbU=; b=aeJkpX7kVYJvegmxTTTfINDqCoKc6MLwU3aE41sXOL4MoFHenJmUFTj6Nf+K+UyS6v Yl6bOVMO+LOgpQ6cAosSxN4vAS4SiH69EyecfDp+65Etb98oSGeq9fziuis8xvxu6oxr QIKCo4KzWgsEAEjJa4v7QueauAX2yUCyldgIOpJEEc9BJNYpZegcbpL/Me79sI0t4ogu PAoizyR7XhgOS3YCxm99R/t/yMyQgGxRpB/mrxYfrNjrrKmd9FUXEZEi0ObjqfT5SCzR 4H1ZVKGzjBIJLzQE3nAOtKWSsYh426hL4sglZRzG3/sXrFfGdYRP56ZVuX7THiBqzyT4 BDDA== X-Forwarded-Encrypted: i=1; AJvYcCVAvSYNqYFLjFO1WL4gJa1O/yuRF67eYjSxZRDtqHKGa1ABP7kiktZ/iIr+LwURYJxWEELwCW3tr95FtuUBmHCfCKT+83CCcCISdYdJoyDFpJa8qvqYGywDk3uXBF3RtACb6LgmjXTrkA== X-Gm-Message-State: AOJu0YyWbqdmHCMlG0ZLep9bzxIzFS++gQj3KxGwvEMeHoqzpsahKQKJ Pzz5Pey6wUxzS58cVigcPSYpb8kdiD+mlz1dO6oyWHt12VzSs3Q5 X-Google-Smtp-Source: AGHT+IHj/QA+7aGk/J1y/qlEAHos5RnBKqOiKcSp7UWvu0DLrLUaiyEL4yPIz/F95q3R+7JyFLfrcA== X-Received: by 2002:adf:eb81:0:b0:367:9ce3:167a with SMTP id ffacd0b85a97d-367ceadb3e7mr4994435f8f.64.1720686429675; Thu, 11 Jul 2024 01:27:09 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:09 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 05/11] irqchip/mips-gic: Setup defaults in each cluster Date: Thu, 11 Jul 2024 10:26:50 +0200 Message-Id: <20240711082656.1889440-6-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chao-ying Fu In multi-cluster MIPS I6500 systems, there is a GIC per cluster. The default shared interrupt setup configured in gic_of_init() will only apply to the GIC in the cluster containing the boot CPU, leaving the GICs of other clusters unconfigured. Similarly configure other clusters. Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- drivers/irqchip/irq-mips-gic.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index e7358d3f4e74..4fd6e316616e 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -764,7 +764,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, gicconfig; + unsigned int cpu_vec, i, gicconfig, cl, nclusters; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -845,11 +845,29 @@ static int __init gic_of_init(struct device_node *node, board_bind_eic_interrupt = &gic_bind_eic_interrupt; - /* Setup defaults */ - for (i = 0; i < gic_shared_intrs; i++) { - change_gic_pol(i, GIC_POL_ACTIVE_HIGH); - change_gic_trig(i, GIC_TRIG_LEVEL); - write_gic_rmask(i); + /* + * Initialise each cluster's GIC shared registers to sane default + * values. + * Otherwise, the IPI set up will be erased if we move code + * to gic_cpu_startup for each cpu. + */ + nclusters = mips_cps_numclusters(); + for (cl = 0; cl < nclusters; cl++) { + if (cl == cpu_cluster(¤t_cpu_data)) { + for (i = 0; i < gic_shared_intrs; i++) { + change_gic_pol(i, GIC_POL_ACTIVE_HIGH); + change_gic_trig(i, GIC_TRIG_LEVEL); + write_gic_rmask(i); + } + } else { + mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + for (i = 0; i < gic_shared_intrs; i++) { + change_gic_redir_pol(i, GIC_POL_ACTIVE_HIGH); + change_gic_redir_trig(i, GIC_TRIG_LEVEL); + write_gic_redir_rmask(i); + } + mips_cm_unlock_other(); + } } return cpuhp_setup_state(CPUHP_AP_IRQ_MIPS_GIC_STARTING, From patchwork Thu Jul 11 08:26:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730177 Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87EF415820E; Thu, 11 Jul 2024 08:27:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686436; cv=none; b=Map42cVqAHdktd/42dmEqlVsAfLNw3wow4m/d9JIegFy9u+Rvk+Ck8Pajd1A6C9UIb0a5v84aIqhkNhO5s5VZo8133nmb1lD7n51Rjo9VKkhc529Vnu5PmRhgFBHYqjzg0VEGiv1rP2GyXYj96LU8jUejnF7DJBDXQFR5gFU8Vc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686436; c=relaxed/simple; bh=Xl62SQuZCHKLnDpCY6udbIHzOsJgA7vUABFTaZ5gaaw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cNuiEjw3khB6SxH5x/QAB2TbxX3cnkFIPPp53/vStg6Sms60megScnacVKEFwh6EoLy7SBwUyzCnGPEH5RjjUWSMMgn0e7w5r6A3Dj9tl2sB939UMEkSkrcmxGgEFb3KeF2FQ/nBcuX7Y8rY/eFoApj10TAE9ANzVteabt/a8ts= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=L6oXVR3w; arc=none smtp.client-ip=209.85.167.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="L6oXVR3w" Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-52ea2b6a9f5so706722e87.0; Thu, 11 Jul 2024 01:27:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686433; x=1721291233; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NLOwwdUhWzwcfHMczhONF97Y3kzxFyjEORxGYabTDDg=; b=L6oXVR3wHoB1vJfG2KHulBnEyAGjNKUD4RZKlodm2luSNK/7/JgZb1yB3zSpG5QOBT QpY5uo1VVNdElMFvBPxijJiJTrLpneK8W5h7YU4Pfvi2CaoKkLR4Ucos7Te6nUNqpkNz EuUI/5tI/s2hg/RePlOrMc/4pv58mO5FOn8WPlrUEhC9LTQm0a/gFwpy15ll+h6VeH6l sxGJbwzISA4xPOv2Mu63cxDDREDZRetX45vHFtEk3rRPE9mq7B5gROO+Lvda/t6t9kvA 3edr/T1RvX/HagU7FDopcg/jZrw9byIX5LW07TSdVrPSvfoS1Hi7w2gRuwu7ufpuLUiJ wZSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686433; x=1721291233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NLOwwdUhWzwcfHMczhONF97Y3kzxFyjEORxGYabTDDg=; b=KtZuiKy4Jk9vSQngDGGzW15PvyKXTd+Rl53VJrKTjM2u4XLjQJdZlwG5iHovLc9jQW 4Zh+OspG+i0Y3OQ7LINvpGjJhmaTCtYbqTnSPB1qxR1DkqwpBqS2WCDlYNdgpFNSDQXd yIkf3CUf449J176vqJ8pICLs104NO5xju7YIZaqOqbLvMKwH/S7QlG6KcgPTojWPZm08 rAXpzFEa+E26hQfPc6oO6d1B/1zfBKsZTa+yrvT2FUOKVGku8uQbZs80zT4h4qBOSzWH iVG/XFyMmK3rS989pxK+JaQnYHSYFHJAWILfOIbUBJXRFAFVxLhuT7zoPg0Hmwn8lqJn O98g== X-Forwarded-Encrypted: i=1; AJvYcCXFTnpowzsouEJy29g4uHaUoMOfNsAEEOXTvzIssVpDPzQ/JclVXmzQjhFJ0YqV0SFTm+3KGC6gnB/Ha8AS8ULTAIZZAa09yU0l3/+8U22yY3EdGcrQ2TMaBHl078Vro4W3Q5hsSN3i+w== X-Gm-Message-State: AOJu0Yx7g6BXFGTdKLVyN6Mj+QPZF2NcjoLS4UYhepFZ5q9Zp0NPobRF Wi8rbRH+RMKCP9cCuILJrMpRNxcCqcQFYC8o6E7abwhOiBM1Ng/r X-Google-Smtp-Source: AGHT+IFOxacAuDp8HNcvS4GCUXkV2BNTDfePTvxA0TaFCt38tY57Sheu03iBaIfumxpnWrzpinVJdA== X-Received: by 2002:ac2:57ce:0:b0:52e:9694:3f98 with SMTP id 2adb3069b0e04-52eb9997b66mr3856395e87.27.1720686432122; Thu, 11 Jul 2024 01:27:12 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:11 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 06/11] irqchip/mips-gic: Multi-cluster support Date: Thu, 11 Jul 2024 10:26:51 +0200 Message-Id: <20240711082656.1889440-7-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton The MIPS I6500 CPU & CM (Coherence Manager) 3.5 introduce the concept of multiple clusters to the system. In these systems, each cluster contains its own GIC, so the GIC isn't truly global any longer. Access to registers in the GICs of remote clusters is possible using a redirect register block much like the redirect register blocks provided by the CM & CPC, and configured through the same GCR_REDIRECT register that mips_cm_lock_other() abstraction builds upon. It is expected that external interrupts are connected identically on all clusters. That is, if there is a device providing an interrupt connected to GIC interrupt pin 0 then it should be connected to pin 0 of every GIC in the system. For the most part, the GIC can be treated as though it is still truly global, so long as interrupts in the cluster are configured properly. This patch introduces support for such multi-cluster systems in the MIPS GIC irqchip driver. A newly introduced gic_irq_lock_cluster() function allows: 1) Configure access to a GIC in a remote cluster via the redirect register block, using mips_cm_lock_other(). Or: 2) Detect that the interrupt in question is affine to the local cluster and plain old GIC register access to the GIC in the local cluster should be used. It is possible to access the local cluster's GIC registers via the redirect block, but keeping the special case for them is both good for performance (because we avoid the locking & indirection overhead of using the redirect block) and necessary to maintain compatibility with systems using CM revisions prior to 3.5 which don't support the redirect block. The gic_irq_lock_cluster() function relies upon an IRQs effective affinity in order to discover which cluster the IRQ is affine to. In order to track this & allow it to be updated at an appropriate point during gic_set_affinity() we select the generic support for effective affinity using CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK. gic_set_affinity() is the one function which gains much complexity. It now deconfigures routing to any VP(E), ie. CPU, on the old cluster when moving affinity to a new cluster. gic_shared_irq_domain_map() moves its update of the IRQs effective affinity to before its use of gic_irq_lock_cluster(), to ensure that operation is on the cluster the IRQ is affine to. The remaining changes are straightforward use of the gic_irq_lock_cluster() function to select between local cluster & remote cluster code-paths when configuring interrupts. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- drivers/irqchip/Kconfig | 1 + drivers/irqchip/irq-mips-gic.c | 161 +++++++++++++++++++++++++++++---- 2 files changed, 143 insertions(+), 19 deletions(-) diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 14464716bacb..fb124414c597 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -328,6 +328,7 @@ config KEYSTONE_IRQ config MIPS_GIC bool + select GENERIC_IRQ_EFFECTIVE_AFF_MASK select GENERIC_IRQ_IPI if SMP select IRQ_DOMAIN_HIERARCHY select MIPS_CM diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 4fd6e316616e..20fa6b12df49 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -111,6 +111,41 @@ static inline void gic_unlock_cluster(void) gic_unlock_cluster(), \ (cpu) = __gic_with_next_online_cpu(cpu)) +/** + * gic_irq_lock_cluster() - Lock redirect block access to IRQ's cluster + * @d: struct irq_data corresponding to the interrupt we're interested in + * + * Locks redirect register block access to the global register block of the GIC + * within the remote cluster that the IRQ corresponding to @d is affine to, + * returning true when this redirect block setup & locking has been performed. + * + * If @d is affine to the local cluster then no locking is performed and this + * function will return false, indicating to the caller that it should access + * the local clusters registers without the overhead of indirection through the + * redirect block. + * + * In summary, if this function returns true then the caller should access GIC + * registers using redirect register block accessors & then call + * mips_cm_unlock_other() when done. If this function returns false then the + * caller should trivially access GIC registers in the local cluster. + * + * Returns true if locking performed, else false. + */ +static bool gic_irq_lock_cluster(struct irq_data *d) +{ + unsigned int cpu, cl; + + cpu = cpumask_first(irq_data_get_effective_affinity_mask(d)); + BUG_ON(cpu >= NR_CPUS); + + cl = cpu_cluster(&cpu_data[cpu]); + if (cl == cpu_cluster(¤t_cpu_data)) + return false; + + mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + return true; +} + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -157,7 +192,12 @@ static void gic_send_ipi(struct irq_data *d, unsigned int cpu) { irq_hw_number_t hwirq = GIC_HWIRQ_TO_SHARED(irqd_to_hwirq(d)); - write_gic_wedge(GIC_WEDGE_RW | hwirq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(GIC_WEDGE_RW | hwirq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(GIC_WEDGE_RW | hwirq); + } } int gic_get_c0_compare_int(void) @@ -225,7 +265,13 @@ static void gic_mask_irq(struct irq_data *d) { unsigned int intr = GIC_HWIRQ_TO_SHARED(d->hwirq); - write_gic_rmask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_rmask(intr); + mips_cm_unlock_other(); + } else { + write_gic_rmask(intr); + } + gic_clear_pcpu_masks(intr); } @@ -234,7 +280,12 @@ static void gic_unmask_irq(struct irq_data *d) unsigned int intr = GIC_HWIRQ_TO_SHARED(d->hwirq); unsigned int cpu; - write_gic_smask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_smask(intr); + mips_cm_unlock_other(); + } else { + write_gic_smask(intr); + } gic_clear_pcpu_masks(intr); cpu = cpumask_first(irq_data_get_effective_affinity_mask(d)); @@ -245,7 +296,12 @@ static void gic_ack_irq(struct irq_data *d) { unsigned int irq = GIC_HWIRQ_TO_SHARED(d->hwirq); - write_gic_wedge(irq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(irq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(irq); + } } static int gic_set_type(struct irq_data *d, unsigned int type) @@ -285,9 +341,16 @@ static int gic_set_type(struct irq_data *d, unsigned int type) break; } - change_gic_pol(irq, pol); - change_gic_trig(irq, trig); - change_gic_dual(irq, dual); + if (gic_irq_lock_cluster(d)) { + change_gic_redir_pol(irq, pol); + change_gic_redir_trig(irq, trig); + change_gic_redir_dual(irq, dual); + mips_cm_unlock_other(); + } else { + change_gic_pol(irq, pol); + change_gic_trig(irq, trig); + change_gic_dual(irq, dual); + } if (trig == GIC_TRIG_EDGE) irq_set_chip_handler_name_locked(d, &gic_edge_irq_controller, @@ -305,25 +368,72 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *cpumask, bool force) { unsigned int irq = GIC_HWIRQ_TO_SHARED(d->hwirq); + unsigned int cpu, cl, old_cpu, old_cl; unsigned long flags; - unsigned int cpu; + /* + * The GIC specifies that we can only route an interrupt to one VP(E), + * ie. CPU in Linux parlance, at a time. Therefore we always route to + * the first online CPU in the mask. + */ cpu = cpumask_first_and(cpumask, cpu_online_mask); if (cpu >= NR_CPUS) return -EINVAL; - /* Assumption : cpumask refers to a single CPU */ - raw_spin_lock_irqsave(&gic_lock, flags); + old_cpu = cpumask_first(irq_data_get_effective_affinity_mask(d)); + old_cl = cpu_cluster(&cpu_data[old_cpu]); + cl = cpu_cluster(&cpu_data[cpu]); - /* Re-route this IRQ */ - write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + raw_spin_lock_irqsave(&gic_lock, flags); - /* Update the pcpu_masks */ - gic_clear_pcpu_masks(irq); - if (read_gic_mask(irq)) - set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + /* + * If we're moving affinity between clusters, stop routing the + * interrupt to any VP(E) in the old cluster. + */ + if (cl != old_cl) { + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_vp(irq, 0); + mips_cm_unlock_other(); + } else { + write_gic_map_vp(irq, 0); + } + } + /* + * Update effective affinity - after this gic_irq_lock_cluster() will + * begin operating on the new cluster. + */ irq_data_update_effective_affinity(d, cpumask_of(cpu)); + + /* + * If we're moving affinity between clusters, configure the interrupt + * trigger type in the new cluster. + */ + if (cl != old_cl) + gic_set_type(d, irqd_get_trigger_type(d)); + + /* Route the interrupt to its new VP(E) */ + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_pin(irq, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_redir_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + + mips_cm_unlock_other(); + } else { + write_gic_map_pin(irq, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); return IRQ_SET_MASK_OK; @@ -471,11 +581,21 @@ static int gic_shared_irq_domain_map(struct irq_domain *d, unsigned int virq, unsigned long flags; data = irq_get_irq_data(virq); + irq_data_update_effective_affinity(data, cpumask_of(cpu)); raw_spin_lock_irqsave(&gic_lock, flags); - write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); - write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); - irq_data_update_effective_affinity(data, cpumask_of(cpu)); + + /* Route the interrupt to its VP(E) */ + if (gic_irq_lock_cluster(data)) { + write_gic_redir_map_pin(intr, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + mips_cm_unlock_other(); + } else { + write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); return 0; @@ -651,6 +771,9 @@ static int gic_ipi_domain_alloc(struct irq_domain *d, unsigned int virq, if (ret) goto error; + /* Set affinity to cpu. */ + irq_data_update_effective_affinity(irq_get_irq_data(virq + i), + cpumask_of(cpu)); ret = irq_set_irq_type(virq + i, IRQ_TYPE_EDGE_RISING); if (ret) goto error; From patchwork Thu Jul 11 08:26:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730178 Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A5D9158851; Thu, 11 Jul 2024 08:27:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686437; cv=none; b=gPgPp+hAr3ADA2tp8GlZSuhnzd4+VRirQ6S8oP9K0UKWypH9Ymwlp0sIy8+97xIjpADyKhZa7fK0RGsOtDfmqp7ugga+IaXhG0MVDzz6AuvxYNUOaBNMCrQ6BFJsrPtwC/qS+N4mn40P7zOKhjO2aeUxam5zmT0xVXIdlF3sVZ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686437; c=relaxed/simple; bh=fwpzYPPCUgCelrVVCDuVaqDelwUZQ6NMXtjgABaLPEc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uAY+Lswvjp+RO8p09l6nL0uYKrzeJ+ui7Uk9YaZ3iCO7eoCo8qydIwd3D4eIxg2w4+DVJBSJjes/BxhQVesR0KLk/Z6VomENghTmbTDwXWO3knSknTYZDgsP/jnrgoH/hnKLATKDUvt6MH9KzEHzVmrvi3DolRYw74oPoSxVWk8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TjTXTT1A; arc=none smtp.client-ip=209.85.221.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TjTXTT1A" Received: by mail-wr1-f50.google.com with SMTP id ffacd0b85a97d-3678f36f154so296545f8f.2; Thu, 11 Jul 2024 01:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686434; x=1721291234; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/Eq5L/3x6rW9mJ9PUBft1DoFSqkQM9o69ITIfWtFGDU=; b=TjTXTT1AJ3UVHmH9XWAkLM111IOFx/iaLVNItXRaq8duVjlmrxM0FtLp9b8MxHiRte gc35Mdfo91sRnCUcaQmEwJVXJrGT0+5Uka37VYlwL53dLVhGXQSGSREcgcnbYFvDaHg3 MbO1zT3wtrB/my4cj1220zRKSSmgZ3Mlj4vygmBU8bWK/RqDOnHlKKKNNqmabiKInmfe Nxo4uTsIhG6c7IZZCGkDJaaf9mLTIh4HUyPlMLb4/v+io/XHgbC+jTPgUYlnz7YxGJqX Hp17ewiHwC/3aNaHpxLnIqBLYE/2zqadjbcf7bGJoGqGdxxDJBsUd9wuDY2MVTZS9SxX vehQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686434; x=1721291234; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Eq5L/3x6rW9mJ9PUBft1DoFSqkQM9o69ITIfWtFGDU=; b=djmFrJ01Z6Sqv2zHsqu0EFf3NjsLPtyefxAFWyVhrad8KFwNRma3rJJ55YR+03tS9x QqT2SZqoFgqdbuaHk0ptE/5MLeEG5cI3cIkjfUk453xuOKQiDG8gXMqiAv9CbvjliQbH tOvH+LGg+KP5Fl+pnjQM/3znXLfdnJ/Zr8F4w8yXX5eKLKbxdZO3j5rtLJqc4ih9jim/ ZLseKI50RWcrDMoUTX9eY9v3dkIuiJWADfsAclXOAgM4RM+1kmPWYXP9F1OQ/8Sjerdc rr5E+BXr2alJ3ERRdEWwaC4GiI+dQU2y4Z5+/+wpOGR53W2oClNfZDCaLZiCMUKAVT7T VQVg== X-Forwarded-Encrypted: i=1; AJvYcCUkNTnwMCtMZyWgy8XY3VCN1vHJ3aS8wjO4/nynslRIOX9p1rn5DCiZ2sULuiELokXYq9euA1HJlh7LLKMcsbh1BfQYjea8uZHpI7FbVj+UBU9sMUuMIn+AWlsJO/o6vmzf9Vc9NdlrnA== X-Gm-Message-State: AOJu0YwzBSMKyjvkvd5HHq9DFIu5VvR7kqQ6X6tI5x6HmMDT6j0OOQ71 3V8X50mZrVN4yVzX1AlkNlGX7m3mR0IwXnuTeJmoF2qYzHEMBr+O7m6mVzvt X-Google-Smtp-Source: AGHT+IFKtYUq1wOQolR8qmRQpgxvK8oXDtrlAIQLNh8NRbYRPc10lApqKQE8h9TOYq9fNCwdb2sXqA== X-Received: by 2002:adf:f0cb:0:b0:367:9634:f103 with SMTP id ffacd0b85a97d-367ceadc8c2mr4468334f8f.64.1720686434405; Thu, 11 Jul 2024 01:27:14 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:13 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 07/11] clocksource: mips-gic-timer: Always use cluster 0 counter as clocksource Date: Thu, 11 Jul 2024 10:26:52 +0200 Message-Id: <20240711082656.1889440-8-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton In a multi-cluster MIPS system, there are multiple GICs - one in each cluster - each of which has its independent counter. The counters in each GIC are not synchronized in any way, so they can drift relative to one another through the lifetime of the system. This is problematic for a clock source which ought to be global. Avoid problems by always accessing cluster 0's counter, using cross-cluster register access. This adds overhead so it is applied only on multi-cluster systems. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- drivers/clocksource/mips-gic-timer.c | 39 +++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c index b3ae38f36720..ebf308916fb1 100644 --- a/drivers/clocksource/mips-gic-timer.c +++ b/drivers/clocksource/mips-gic-timer.c @@ -165,6 +165,37 @@ static u64 gic_hpt_read(struct clocksource *cs) return gic_read_count(); } +static u64 gic_hpt_read_multicluster(struct clocksource *cs) +{ + unsigned int hi, hi2, lo; + u64 count; + + mips_cm_lock_other(0, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + if (mips_cm_is64) { + count = read_gic_redir_counter(); + goto out; + } + + hi = read_gic_redir_counter_32h(); + while (true) { + lo = read_gic_redir_counter_32l(); + + /* If hi didn't change then lo didn't wrap & we're done */ + hi2 = read_gic_redir_counter_32h(); + if (hi2 == hi) + break; + + /* Otherwise, repeat with the latest hi value */ + hi = hi2; + } + + count = (((u64)hi) << 32) + lo; +out: + mips_cm_unlock_other(); + return count; +} + static struct clocksource gic_clocksource = { .name = "GIC", .read = gic_hpt_read, @@ -199,6 +230,11 @@ static int __init __gic_clocksource_init(void) /* Calculate a somewhat reasonable rating value. */ gic_clocksource.rating = 200 + gic_frequency / 10000000; + if (mips_cps_multicluster_cpus()) { + gic_clocksource.read = &gic_hpt_read_multicluster; + gic_clocksource.vdso_clock_mode = VDSO_CLOCKMODE_NONE; + } + ret = clocksource_register_hz(&gic_clocksource, gic_frequency); if (ret < 0) pr_warn("Unable to register clocksource\n"); @@ -257,7 +293,8 @@ static int __init gic_clocksource_of_init(struct device_node *node) * stable CPU frequency or on the platforms with CM3 and CPU frequency * change performed by the CPC core clocks divider. */ - if (mips_cm_revision() >= CM_REV_CM3 || !IS_ENABLED(CONFIG_CPU_FREQ)) { + if ((mips_cm_revision() >= CM_REV_CM3 || !IS_ENABLED(CONFIG_CPU_FREQ)) && + !mips_cps_multicluster_cpus()) { sched_clock_register(mips_cm_is64 ? gic_read_count_64 : gic_read_count_2x32, 64, gic_frequency); From patchwork Thu Jul 11 08:26:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730179 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 232561534E1; Thu, 11 Jul 2024 08:27:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686439; cv=none; b=P6bHBGQFMsLcPHmLuf2Qe9IydNNtfhoGDSmX1TYtQbpPGIfmyGwjodrW92CHzeeTX/9Jsd3TZB98wjVdd6BCqloVZOgQoqObOKpaV9MJ1O+XL/H4YI2InZF22FP/hxm+PHt9qb1Nn1hmQeOD0xTuvb9EkLWdW0gyDLpwGrhGaqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686439; c=relaxed/simple; bh=BWru3gbDwB+xSMlpt5VLhdUAAVAvJvSrq4mELPxfulU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=k+hZxbGtHqrMja+qwlz2pLm4da2N+JXaLCdKjo38wJBVGHEP9S4fBugH/y4MwqYoWHbTD4vmSo7JEmEoK2XbQ76KwOSE8CD/lrFbzI+3MBAejnUfdjOZ/M2xU4I7TaIzKtKPiF1RYlNMoKJaLBqlPExbEXJbkkb8BR3BHnwWH5g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZLyZzj+Q; arc=none smtp.client-ip=209.85.221.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZLyZzj+Q" Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3678aa359b7so1048532f8f.1; Thu, 11 Jul 2024 01:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686436; x=1721291236; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QCY8yakOgFx6ixafaMq4NF6KqoNRvoG64yc9U8LV1IM=; b=ZLyZzj+QGam2kYwwTxi2dkSaMTHQY7nHWMDRSxyTzJ/Wb/Y58vCWS8IePByspDzqsC xyWJkLuGq7qY7hVFz3PJ4yN906eA1HQz5Rx/xT3v4dI8IOEoqgkZkNODghp3Yv6mWCdh tffCTYlGBvhv6HVmIrgR6dTby5IcvMX5aa9mZq6WW2z1CKga3qXlKMF9pLYp0/uhLBI1 kU4kZeY/R8rpPrkxKJii30Zund+iGmarKUiKcGehPkW7XUvjPiucXPoaFwHsV3oNSEFg Njswls/bgvg6Nxp+Bg3aVP9/VXEp+eiEn2zAf0uIMZJLAdFN3WSbtyukiBo4WUGHp1AH 7sdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686436; x=1721291236; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QCY8yakOgFx6ixafaMq4NF6KqoNRvoG64yc9U8LV1IM=; b=rLzumfULtfqq6NHt9gCdqBAoGjC46MBRHyevhB1gsmRUCfCuJzJdag3mUxjFvSt0iG 9xZdPEeJuUndemR/EMcx3KJrI9cEb9sT6Ze/mr+05yh25lc+J4p5emKA3z4ei5KFNRad rhap/qR6r8WLpf+6MX503lvkaIpzo+6LZu9rVPH7KKyN8QydWAQo6W9FzZKiPfPAgUXF h7j7P1diBwFRKXqAlWEMFe5hErGOa7ZGKrrqSELRtrpARn7qL9jZAD/PP6Lu1TF2btzW t2rGufhvt5s7fNtkWbhXzLOMCh6hSZkr2wJxL1nAVXbeFFm/LcorYbxBB/JGgmoQGRi3 6slg== X-Forwarded-Encrypted: i=1; AJvYcCXsgHz54TAEuFbYdLCNOEw8Go15pW+dIQ9m8TI/rm8lhM2HRhMlQvXfDOg5Qy9WL1psYspSf55Daj8sAAiudBQPWVWZFL+53kDNihOFQ0DMNiEMjrxMfpEqxSmcUlFROQ6An1AZo0g7Kw== X-Gm-Message-State: AOJu0Yz5HGNNIt/NCrv1Lrey0xkwSeRuKAV/I/UAAUIvtmOH31srhPRe FHcyxoIKB0dywzwH9ipCHpXQnfz8bF6ENi6Q7cMNQSfkdaGznT// X-Google-Smtp-Source: AGHT+IEjSYEEaX48FshcXj6CxYudxam3VGcC/uxrHOKGP4C6F0fyeI+wv5B6A6KMEDqcGR/KY0ulgw== X-Received: by 2002:a5d:42c9:0:b0:367:3282:a258 with SMTP id ffacd0b85a97d-367f717057emr704191f8f.18.1720686436374; Thu, 11 Jul 2024 01:27:16 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:15 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 08/11] clocksource: mips-gic-timer: Enable counter when CPUs start Date: Thu, 11 Jul 2024 10:26:53 +0200 Message-Id: <20240711082656.1889440-9-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton In multi-cluster MIPS I6500 systems there is a GIC in each cluster, each with its own counter. When a cluster powers up the counter will be stopped, with the COUNTSTOP bit set in the GIC_CONFIG register. In single cluster systems, it has been fine to clear COUNTSTOP once in gic_clocksource_of_init() to start the counter. In multi-cluster systems, this will only have started the counter in the boot cluster, and any CPUs in other clusters will find their counter stopped which will break the GIC clock_event_device. Resolve this by having CPUs clear the COUNTSTOP bit when they come online, using the existing gic_starting_cpu() CPU hotplug callback. This will allow CPUs in secondary clusters to ensure that the cluster's GIC counter is running as expected. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Reviewed-by: Philippe Mathieu-Daudé Tested-by: Serge Semin --- drivers/clocksource/mips-gic-timer.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c index ebf308916fb1..4d7659c119e1 100644 --- a/drivers/clocksource/mips-gic-timer.c +++ b/drivers/clocksource/mips-gic-timer.c @@ -114,6 +114,9 @@ static void gic_update_frequency(void *data) static int gic_starting_cpu(unsigned int cpu) { + /* Ensure the GIC counter is running */ + clear_gic_config(GIC_CONFIG_COUNTSTOP); + gic_clockevent_cpu_init(cpu, this_cpu_ptr(&gic_clockevent_device)); return 0; } @@ -284,9 +287,6 @@ static int __init gic_clocksource_of_init(struct device_node *node) pr_warn("Unable to register clock notifier\n"); } - /* And finally start the counter */ - clear_gic_config(GIC_CONFIG_COUNTSTOP); - /* * It's safe to use the MIPS GIC timer as a sched clock source only if * its ticks are stable, which is true on either the platforms with From patchwork Thu Jul 11 08:26:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730180 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1D40158D6D; Thu, 11 Jul 2024 08:27:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686441; cv=none; b=j7cDCAbFd1XKhAFFgPHa2oRhqGKcMK4SqmaKPfZrja/DNLxaqzIgJXkHS5gfbekchZ+X5nScL0CIgf1d1kvgZozgMYSxEohngtVl2wD9joXidyei4P9qGDb5aiom9sT5pi5/UG+ERR71rpoO0IbmsnZ4DJQqCOuX7PoIb9CBSdg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686441; c=relaxed/simple; bh=QKto5A9MIMK/IEg/kWynx+UpUoTjf00hu+AW07gvz90=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uOvCky361kaH7UWoGhxzs1uJYTlwu2D5EQQApuCzX56VeLnnkbFfI4vCR6YMukP2z7LtijDKpn6P1U2mOy2BjIA3nOEsa9CVEGhNjt3C+wVijTmLm4jQFJHqlnChzZH9ZVIcdmpnJp5LIn8f+M8bEf9vxUMrAZN8dH/zFdpBAYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DXhK+XP0; arc=none smtp.client-ip=209.85.128.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DXhK+XP0" Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-4266f535e82so3820385e9.1; Thu, 11 Jul 2024 01:27:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686438; x=1721291238; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5Wui8yC11QzS3zuL6au0E0B/rgxgmkQnWbsEcia4kwc=; b=DXhK+XP0jcmbn32uHOien6R2ft5KBHWdRHU4fqWCL10kthKoM0dHC41PNBg26V2uyX DmsH+U7Hl9iqlRbapaHDlQJ7J13F1lFyyA8DONLKiQLGYl+a4/JnYyR9EyCvryIEi1qk 7TGGO036Z1GykH0fUGK3PBlFf/THsZpR2vJ+lIzJqkZN4eIBnb18djcZ2dte6QAGsQaV gfn0U6u15gLCL+SAmngo6MU7c+O/+Q9iW+5zSoLciixsiqoyCD2L00LgoAzW34JntWSx ueKIU4Z1ktPF4k4/Aut09riHv8JhP4luTx02zHJG0BRKvRbVWVgkP66WUO3Wvi2XD2PC 9eOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686438; x=1721291238; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5Wui8yC11QzS3zuL6au0E0B/rgxgmkQnWbsEcia4kwc=; b=BBFejgbU7Cwi/+KP1qvAgGz/fcKBlhHed7lGT+LB22DGDEMCNoNaMpXX89r648wn70 OnfLwmorRfURHb2wHHIqT2SMq9wSc4cNOY3mdzpwp3nZuwtutFqRR7ESuC6Hw6JS1Yev db2mRgOEyFQrLZJreqTDYC0S+c9Gm3vmrGzWl7qbGgG63wyWC90AiiPI1u1InXsvqia3 mwXVHsE/nXVu6cqfZBz3l5ga81ew0NTXAIHxL+VA+yRLPPbdLZEFaLo2ecJA/jxPeNfE gJOo804xR3G6X9zLGc4U0B7785HsG5CEPpydWL4jea2hIuO+jciHglKboY2pSp9WI0lf q5Ww== X-Forwarded-Encrypted: i=1; AJvYcCXIAUZbQz8FivRTK8qivfoDau+YRt6T/+U/N97R3qC4uZ8Hh6ZOlUP+0F7mDwp3lmIyR8ubYZMO83GfGXzljoVmck84J9m8AWzS49H9yF4mNu/jf5I/oZ3VbVSnIVPNA6beyYAsRMRPoQ== X-Gm-Message-State: AOJu0Yy/Xn07iFmZEuQhq8L4OE4Gw35Chmgs1lJvtM7oXskwQYXJjFjp Fxaj9WbX3eDdXoJm8uBfNtPL80Bp2WUs21clp3F1rwemA5jqD5Pr X-Google-Smtp-Source: AGHT+IHYB4QOIksPssx59VmapvSFYlVkcwUpV42+h9Y5VBTqffraZPy8/t3FWAZ/aDtvaJ2BKywJdQ== X-Received: by 2002:a05:600c:4a99:b0:426:5de3:2ae5 with SMTP id 5b1f17b1804b1-426707d8ae6mr63499495e9.10.1720686438130; Thu, 11 Jul 2024 01:27:18 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:17 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 09/11] MIPS: pm-cps: Use per-CPU variables as per-CPU, not per-core Date: Thu, 11 Jul 2024 10:26:54 +0200 Message-Id: <20240711082656.1889440-10-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton The pm-cps code has up until now used per-CPU variables indexed by core, rather than CPU number, in order to share data amongst sibling CPUs (ie. VPs/threads in a core). This works fine for single cluster systems, but with multi-cluster systems a core number is no longer unique in the system, leading to sharing between CPUs that are not actually siblings. Avoid this issue by using per-CPU variables as they are more generally used - ie. access them using CPU numbers rather than core numbers. Sharing between siblings is then accomplished by: - Assigning the same pointer to entries for each sibling CPU for the nc_asm_enter & ready_count variables, which allow this by virtue of being per-CPU pointers. - Indexing by the first CPU set in a CPUs cpu_sibling_map in the case of pm_barrier, for which we can't use the previous approach because the per-CPU variable is not a pointer. Signed-off-by: Paul Burton Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/kernel/pm-cps.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index d09ca77e624d..9369a8dc385e 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -57,10 +57,7 @@ static DEFINE_PER_CPU_ALIGNED(u32*, ready_count); /* Indicates online CPUs coupled with the current CPU */ static DEFINE_PER_CPU_ALIGNED(cpumask_t, online_coupled); -/* - * Used to synchronize entry to deep idle states. Actually per-core rather - * than per-CPU. - */ +/* Used to synchronize entry to deep idle states */ static DEFINE_PER_CPU_ALIGNED(atomic_t, pm_barrier); /* Saved CPU state across the CPS_PM_POWER_GATED state */ @@ -112,9 +109,10 @@ int cps_pm_enter_state(enum cps_pm_state state) cps_nc_entry_fn entry; struct core_boot_config *core_cfg; struct vpe_boot_config *vpe_cfg; + atomic_t *barrier; /* Check that there is an entry function for this state */ - entry = per_cpu(nc_asm_enter, core)[state]; + entry = per_cpu(nc_asm_enter, cpu)[state]; if (!entry) return -EINVAL; @@ -150,7 +148,7 @@ int cps_pm_enter_state(enum cps_pm_state state) smp_mb__after_atomic(); /* Create a non-coherent mapping of the core ready_count */ - core_ready_count = per_cpu(ready_count, core); + core_ready_count = per_cpu(ready_count, cpu); nc_addr = kmap_noncoherent(virt_to_page(core_ready_count), (unsigned long)core_ready_count); nc_addr += ((unsigned long)core_ready_count & ~PAGE_MASK); @@ -158,7 +156,8 @@ int cps_pm_enter_state(enum cps_pm_state state) /* Ensure ready_count is zero-initialised before the assembly runs */ WRITE_ONCE(*nc_core_ready_count, 0); - coupled_barrier(&per_cpu(pm_barrier, core), online); + barrier = &per_cpu(pm_barrier, cpumask_first(&cpu_sibling_map[cpu])); + coupled_barrier(barrier, online); /* Run the generated entry code */ left = entry(online, nc_core_ready_count); @@ -629,12 +628,14 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) static int cps_pm_online_cpu(unsigned int cpu) { - enum cps_pm_state state; - unsigned core = cpu_core(&cpu_data[cpu]); + unsigned int sibling, core; void *entry_fn, *core_rc; + enum cps_pm_state state; + + core = cpu_core(&cpu_data[cpu]); for (state = CPS_PM_NC_WAIT; state < CPS_PM_STATE_COUNT; state++) { - if (per_cpu(nc_asm_enter, core)[state]) + if (per_cpu(nc_asm_enter, cpu)[state]) continue; if (!test_bit(state, state_support)) continue; @@ -646,16 +647,19 @@ static int cps_pm_online_cpu(unsigned int cpu) clear_bit(state, state_support); } - per_cpu(nc_asm_enter, core)[state] = entry_fn; + for_each_cpu(sibling, &cpu_sibling_map[cpu]) + per_cpu(nc_asm_enter, sibling)[state] = entry_fn; } - if (!per_cpu(ready_count, core)) { + if (!per_cpu(ready_count, cpu)) { core_rc = kmalloc(sizeof(u32), GFP_KERNEL); if (!core_rc) { pr_err("Failed allocate core %u ready_count\n", core); return -ENOMEM; } - per_cpu(ready_count, core) = core_rc; + + for_each_cpu(sibling, &cpu_sibling_map[cpu]) + per_cpu(ready_count, sibling) = core_rc; } return 0; From patchwork Thu Jul 11 08:26:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730181 Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 284D7158DCE; Thu, 11 Jul 2024 08:27:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686444; cv=none; b=Go/0DvA8eX+w2TrrBM6Gfy356w6JGRETeCOvZAhsLczL+aT+0L5yvv6/7TaVrkym4RdLrM14L4lQXqZlk9NOtygnGvzuCxozBQa+It4AfmDAWih1YTmNsItV4q0Y755or+LR4t1Y3Yg1TDeeQapqxDLX7HjZX2rm32fosuQl/jQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686444; c=relaxed/simple; bh=dGtkojoCL7ACGD+vgA/1jJm9suwAY579SJ3HvgF55Ac=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=at5DSsMR9HrSXI8IHiGvBXmQl5tHYGX3ai+lBcxqoIrQATPgOHnFtlXkV2uLhC83XKSiUQLlCMEZevWUtomA8ElgcvAIQy7DJRQnwq445v0R9wmXbepjzkIe+EFKsPMUyLqXibIWgJEuMxj0BadIO6oLtfXmWFaCySz6bCQ3sGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bb9eBMbS; arc=none smtp.client-ip=209.85.208.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bb9eBMbS" Received: by mail-lj1-f179.google.com with SMTP id 38308e7fff4ca-2eea7e2b073so6849631fa.0; Thu, 11 Jul 2024 01:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686440; x=1721291240; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E0Q5hRMxcBidzAbspqE37uGSfEWkuJsBaxyeY4W2RUI=; b=bb9eBMbSvMgj0cIMu3Q7BtgpQS4ReqG4xMrz0axVL22uvjkKazs2JeNAId9QpLHKla BP+RIMsKMwJ0DzKr7iBxDOGwWGWTks5axh6EnoxTSZu0XBsz1B2+CVrTRSinHoum3s+p 5bkWT2c5Y/GkOopOrvv5O0XIoRVOnu1NL+1N3/ZXmmD58sqNbSzHLxpsaPUROIEQ59hp MzsrFxmpY4b4OX4hop+e7muIf8R35qL+aiBCjUWpeVAojtxHnV7qoegOTO7dcQeM357V isjUjrhfMqn8upNtu2nloDCPyP2hVkVOkpzYHlbWZG0xtT5UxmWRNJFSd+b6MNDYG73M gTlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686440; x=1721291240; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E0Q5hRMxcBidzAbspqE37uGSfEWkuJsBaxyeY4W2RUI=; b=LXvjE0UaztDwbN6/wXB6AgMY3WFdxy37EQAQhSVnAlmyzCgcwWH1E6ew0kyRHmY9Ec u8xQlHk2sGmg+SWV4fGbNCDikExA55s56ba2lMTmJAtXrdH0zb/T3ktAMyx7J0X1K8qp Q4TUMAmblf9ndFF9Fnt4GlgKCDuSPEx3ZWHlF01GRtmVy/+iBT2Qm/C9Gf7D1tHydlWr zWVtuHFxqH3YEOQYCabMXgDJJkuvIkSNgCbC913HY0fW72GdEiFgs/iBDBMg/QISsJEt /wiaeI2a8XdJ35LhfJbF3/G2P7gw+du9ap8yW3Mi5bZmw33+U3LmReGvZWCbW5Bt6+AJ 3sKg== X-Forwarded-Encrypted: i=1; AJvYcCVIV6vEBIE5xwU5YlUHffFQbwnmgUa+4FjY7KTe2HGt5sOCDb/FC1HsNjpd44ylg1ZMZ2n11PtlqOT6InBwrJmo/cqMYhaA2jM46cZdgYPjJgo40B2t0U43a3Zwrc8h5TJjBvJgsNZeqg== X-Gm-Message-State: AOJu0YwJIr/j794ELxLm+5PTwxuLXcTHv9Za5ec6O14jjJxqqtd8QKwJ LbNZqy2lnSUnSDX464C4kMJrNqGui8UIQMP7lWz1QU7bW6GU3qfQ X-Google-Smtp-Source: AGHT+IGyhWa/QojzYd0rFAtFoAxcyeZpoGig1rKstnZE7FzF1QHHfeUhPKTXmR16XLaaFf6diG9Qqg== X-Received: by 2002:a05:651c:509:b0:2ee:8d07:3d5e with SMTP id 38308e7fff4ca-2eeb30ba031mr60613661fa.9.1720686440167; Thu, 11 Jul 2024 01:27:20 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:19 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 10/11] MIPS: CPS: Introduce struct cluster_boot_config Date: Thu, 11 Jul 2024 10:26:55 +0200 Message-Id: <20240711082656.1889440-11-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton In preparation for supporting multi-cluster systems, introduce a struct cluster_boot_config as an extra layer in the boot configuration maintained by the MIPS Coherent Processing System (CPS) SMP implementation. For now only one struct cluster_boot_config will be allocated & we'll simply defererence its core_config field to find the struct core_boot_config array which can be used to boot as usual. Signed-off-by: Paul Burton Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/include/asm/smp-cps.h | 6 ++- arch/mips/kernel/asm-offsets.c | 3 ++ arch/mips/kernel/cps-vec.S | 19 ++++++-- arch/mips/kernel/pm-cps.c | 5 +- arch/mips/kernel/smp-cps.c | 82 +++++++++++++++++++++------------ 5 files changed, 81 insertions(+), 34 deletions(-) diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cps.h index ab94e50f62b8..a629e948a6fd 100644 --- a/arch/mips/include/asm/smp-cps.h +++ b/arch/mips/include/asm/smp-cps.h @@ -22,7 +22,11 @@ struct core_boot_config { struct vpe_boot_config *vpe_config; }; -extern struct core_boot_config *mips_cps_core_bootcfg; +struct cluster_boot_config { + struct core_boot_config *core_config; +}; + +extern struct cluster_boot_config *mips_cps_cluster_bootcfg; extern void mips_cps_core_boot(int cca, void __iomem *gcr_base); extern void mips_cps_core_init(void); diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c index cb1045ebab06..b29944160b28 100644 --- a/arch/mips/kernel/asm-offsets.c +++ b/arch/mips/kernel/asm-offsets.c @@ -404,6 +404,9 @@ void output_cps_defines(void) { COMMENT(" MIPS CPS offsets. "); + OFFSET(CLUSTERBOOTCFG_CORECONFIG, cluster_boot_config, core_config); + DEFINE(CLUSTERBOOTCFG_SIZE, sizeof(struct cluster_boot_config)); + OFFSET(COREBOOTCFG_VPEMASK, core_boot_config, vpe_mask); OFFSET(COREBOOTCFG_VPECONFIG, core_boot_config, vpe_config); DEFINE(COREBOOTCFG_SIZE, sizeof(struct core_boot_config)); diff --git a/arch/mips/kernel/cps-vec.S b/arch/mips/kernel/cps-vec.S index f876309130ad..2ae7034a3d5c 100644 --- a/arch/mips/kernel/cps-vec.S +++ b/arch/mips/kernel/cps-vec.S @@ -19,6 +19,10 @@ #define GCR_CPC_BASE_OFS 0x0088 #define GCR_CL_COHERENCE_OFS 0x2008 #define GCR_CL_ID_OFS 0x2028 +#define CM3_GCR_Cx_ID_CLUSTER_SHF 8 +#define CM3_GCR_Cx_ID_CLUSTER_MSK (0xff << 8) +#define CM3_GCR_Cx_ID_CORENUM_SHF 0 +#define CM3_GCR_Cx_ID_CORENUM_MSK (0xff << 0) #define CPC_CL_VC_STOP_OFS 0x2020 #define CPC_CL_VC_RUN_OFS 0x2028 @@ -271,12 +275,21 @@ LEAF(mips_cps_core_init) */ LEAF(mips_cps_get_bootcfg) /* Calculate a pointer to this cores struct core_boot_config */ + PTR_LA v0, mips_cps_cluster_bootcfg + PTR_L v0, 0(v0) lw t0, GCR_CL_ID_OFS(s1) +#ifdef CONFIG_CPU_MIPSR6 + ext t1, t0, CM3_GCR_Cx_ID_CLUSTER_SHF, 8 + li t2, CLUSTERBOOTCFG_SIZE + mul t1, t1, t2 + PTR_ADDU \ + v0, v0, t1 +#endif + PTR_L v0, CLUSTERBOOTCFG_CORECONFIG(v0) + andi t0, t0, CM3_GCR_Cx_ID_CORENUM_MSK li t1, COREBOOTCFG_SIZE mul t0, t0, t1 - PTR_LA t1, mips_cps_core_bootcfg - PTR_L t1, 0(t1) - PTR_ADDU v0, t0, t1 + PTR_ADDU v0, v0, t0 /* Calculate this VPEs ID. If the core doesn't support MT use 0 */ li t9, 0 diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index 9369a8dc385e..3de0e05e0511 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -101,12 +101,14 @@ static void coupled_barrier(atomic_t *a, unsigned online) int cps_pm_enter_state(enum cps_pm_state state) { unsigned cpu = smp_processor_id(); + unsigned int cluster = cpu_cluster(¤t_cpu_data); unsigned core = cpu_core(¤t_cpu_data); unsigned online, left; cpumask_t *coupled_mask = this_cpu_ptr(&online_coupled); u32 *core_ready_count, *nc_core_ready_count; void *nc_addr; cps_nc_entry_fn entry; + struct cluster_boot_config *cluster_cfg; struct core_boot_config *core_cfg; struct vpe_boot_config *vpe_cfg; atomic_t *barrier; @@ -136,7 +138,8 @@ int cps_pm_enter_state(enum cps_pm_state state) if (!mips_cps_smp_in_use()) return -EINVAL; - core_cfg = &mips_cps_core_bootcfg[core]; + cluster_cfg = &mips_cps_cluster_bootcfg[cluster]; + core_cfg = &cluster_cfg->core_config[core]; vpe_cfg = &core_cfg->vpe_config[cpu_vpe_id(¤t_cpu_data)]; vpe_cfg->pc = (unsigned long)mips_cps_pm_restore; vpe_cfg->gp = (unsigned long)current_thread_info(); diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c index 9cc087dd1c19..1954c9e912b2 100644 --- a/arch/mips/kernel/smp-cps.c +++ b/arch/mips/kernel/smp-cps.c @@ -40,7 +40,7 @@ static DECLARE_BITMAP(core_power, NR_CPUS); static uint32_t core_entry_reg; static phys_addr_t cps_vec_pa; -struct core_boot_config *mips_cps_core_bootcfg; +struct cluster_boot_config *mips_cps_cluster_bootcfg; static unsigned __init core_vpe_count(unsigned int cluster, unsigned core) { @@ -212,8 +212,10 @@ static void __init cps_smp_setup(void) static void __init cps_prepare_cpus(unsigned int max_cpus) { - unsigned ncores, core_vpes, c, cca; + unsigned int nclusters, ncores, core_vpes, c, cl, cca; bool cca_unsuitable, cores_limited; + struct cluster_boot_config *cluster_bootcfg; + struct core_boot_config *core_bootcfg; mips_mt_set_cpuoptions(); @@ -255,40 +257,54 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) setup_cps_vecs(); - /* Allocate core boot configuration structs */ - ncores = mips_cps_numcores(0); - mips_cps_core_bootcfg = kcalloc(ncores, sizeof(*mips_cps_core_bootcfg), - GFP_KERNEL); - if (!mips_cps_core_bootcfg) { - pr_err("Failed to allocate boot config for %u cores\n", ncores); - goto err_out; - } + /* Allocate cluster boot configuration structs */ + nclusters = mips_cps_numclusters(); + mips_cps_cluster_bootcfg = kcalloc(nclusters, + sizeof(*mips_cps_cluster_bootcfg), + GFP_KERNEL); - /* Allocate VPE boot configuration structs */ - for (c = 0; c < ncores; c++) { - core_vpes = core_vpe_count(0, c); - mips_cps_core_bootcfg[c].vpe_config = kcalloc(core_vpes, - sizeof(*mips_cps_core_bootcfg[c].vpe_config), - GFP_KERNEL); - if (!mips_cps_core_bootcfg[c].vpe_config) { - pr_err("Failed to allocate %u VPE boot configs\n", - core_vpes); + for (cl = 0; cl < nclusters; cl++) { + /* Allocate core boot configuration structs */ + ncores = mips_cps_numcores(cl); + core_bootcfg = kcalloc(ncores, sizeof(*core_bootcfg), + GFP_KERNEL); + if (!core_bootcfg) goto err_out; + mips_cps_cluster_bootcfg[cl].core_config = core_bootcfg; + + /* Allocate VPE boot configuration structs */ + for (c = 0; c < ncores; c++) { + core_vpes = core_vpe_count(cl, c); + core_bootcfg[c].vpe_config = kcalloc(core_vpes, + sizeof(*core_bootcfg[c].vpe_config), + GFP_KERNEL); + if (!core_bootcfg[c].vpe_config) + goto err_out; } } /* Mark this CPU as booted */ - atomic_set(&mips_cps_core_bootcfg[cpu_core(¤t_cpu_data)].vpe_mask, - 1 << cpu_vpe_id(¤t_cpu_data)); + cl = cpu_cluster(¤t_cpu_data); + c = cpu_core(¤t_cpu_data); + cluster_bootcfg = &mips_cps_cluster_bootcfg[cl]; + core_bootcfg = &cluster_bootcfg->core_config[c]; + atomic_set(&core_bootcfg->vpe_mask, 1 << cpu_vpe_id(¤t_cpu_data)); return; err_out: /* Clean up allocations */ - if (mips_cps_core_bootcfg) { - for (c = 0; c < ncores; c++) - kfree(mips_cps_core_bootcfg[c].vpe_config); - kfree(mips_cps_core_bootcfg); - mips_cps_core_bootcfg = NULL; + if (mips_cps_cluster_bootcfg) { + for (cl = 0; cl < nclusters; cl++) { + cluster_bootcfg = &mips_cps_cluster_bootcfg[cl]; + ncores = mips_cps_numcores(cl); + for (c = 0; c < ncores; c++) { + core_bootcfg = &cluster_bootcfg->core_config[c]; + kfree(core_bootcfg->vpe_config); + } + kfree(mips_cps_cluster_bootcfg[c].core_config); + } + kfree(mips_cps_cluster_bootcfg); + mips_cps_cluster_bootcfg = NULL; } /* Effectively disable SMP by declaring CPUs not present */ @@ -373,17 +389,23 @@ static void boot_core(unsigned int core, unsigned int vpe_id) static void remote_vpe_boot(void *dummy) { + unsigned int cluster = cpu_cluster(¤t_cpu_data); unsigned core = cpu_core(¤t_cpu_data); - struct core_boot_config *core_cfg = &mips_cps_core_bootcfg[core]; + struct cluster_boot_config *cluster_cfg = + &mips_cps_cluster_bootcfg[cluster]; + struct core_boot_config *core_cfg = &cluster_cfg->core_config[core]; mips_cps_boot_vpes(core_cfg, cpu_vpe_id(¤t_cpu_data)); } static int cps_boot_secondary(int cpu, struct task_struct *idle) { + unsigned int cluster = cpu_cluster(&cpu_data[cpu]); unsigned core = cpu_core(&cpu_data[cpu]); unsigned vpe_id = cpu_vpe_id(&cpu_data[cpu]); - struct core_boot_config *core_cfg = &mips_cps_core_bootcfg[core]; + struct cluster_boot_config *cluster_cfg = + &mips_cps_cluster_bootcfg[cluster]; + struct core_boot_config *core_cfg = &cluster_cfg->core_config[core]; struct vpe_boot_config *vpe_cfg = &core_cfg->vpe_config[vpe_id]; unsigned int remote; int err; @@ -541,12 +563,14 @@ static void cps_kexec_nonboot_cpu(void) static int cps_cpu_disable(void) { unsigned cpu = smp_processor_id(); + struct cluster_boot_config *cluster_cfg; struct core_boot_config *core_cfg; if (!cps_pm_support_state(CPS_PM_POWER_GATED)) return -EINVAL; - core_cfg = &mips_cps_core_bootcfg[cpu_core(¤t_cpu_data)]; + cluster_cfg = &mips_cps_cluster_bootcfg[cpu_cluster(¤t_cpu_data)]; + core_cfg = &cluster_cfg->core_config[cpu_core(¤t_cpu_data)]; atomic_sub(1 << cpu_vpe_id(¤t_cpu_data), &core_cfg->vpe_mask); smp_mb__after_atomic(); set_cpu_online(cpu, false); From patchwork Thu Jul 11 08:26:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13730182 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7704159582; Thu, 11 Jul 2024 08:27:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686445; cv=none; b=YOr9AdtcKntYs75d7AvDHVLhmIblVzG50xIwrJkBFPYBLhmGOSuQaZj9vdVVWA838bQ1ktzVQ2ZtuIXYzTBw53q0V6vijhpzQZjpwX3mlOSUOqlRy/X4ks/7mJH7W+GnDGfS6YlMd8WGsBG+KCk8b9aQ5Ztqv/ZWoSQ7j03aUPM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720686445; c=relaxed/simple; bh=xq/DNrRorWnPcPa8MmAE/LGhjNMVOsK7yIV1ZLzNsCA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=i7jsIeQCNsHlqKjcLRJCVUN17XmgGDiT+J+KIwTP6gBBsidqglR/mAZsEaLpfPj0tFgPq05c6Q/TOzLIzDnNCNo8BCvBFZF8QhP8+uXwLgg9V6jGa3XooPGMVUtiREqe5cZpX2LP9Y1kwRofedOsQWrtZnGD5ZcP4OJ0U7Lq6H4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fpny4gCD; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fpny4gCD" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4266edee10cso3631395e9.2; Thu, 11 Jul 2024 01:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720686442; x=1721291242; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FaAJQ5SDpAqfWt+qUQOcxkGDcwpCestITtFu0QQSY7c=; b=fpny4gCD8pGkpSUHIJ2qwjXn2HOYYvrb5hJv5l0rWutFxaI+yZk4a1hUPzur+XZ0h9 c3J9kqNfd5rWlO6w1sANFzwTeJ3BmWjiAO1sgcqzZ3fbjlcnq9Phy47uJUWoTztBA4BS QbsJhYGlqPWa00zhFpIIAX0cwUbKpmdEOydTND1CC0x11aDlx4DVDWVtsGPx2UXgchwy nc/h64bxOlEqeoAJTaqKKcnUSz+FfQyZD3qa0JBVLEv5d6uW0XGoXZ0E3xDg2NtRQvPd W+cWXoLh6nkzW4ijW1A4x6ypfgqfkq+yTFhw2cQY49mW4M1gqVhiIvxJXmqc9Qx3GOFf tP/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720686442; x=1721291242; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FaAJQ5SDpAqfWt+qUQOcxkGDcwpCestITtFu0QQSY7c=; b=wcLh0ru/wvMyH5jtUqSVpXuEraz0uRi7uspkpGCl9Zt1s8yvhiBZwR+NgIOTa0gjff PBpazPYc0O3/WUjebMWP8519LnMSCeGgsYuGB5CCGLuUZk9EDSh2ujbkh1SgKQyjRFgk A66zspZy/IZQx+TLnT7CTrLBM75aDLyww2Pz6+mt1eyWn/aKvxF3Z8+MMuZM6xGtiMsb Ahj8LN+oKmNggGsipe4WTV346SgSX9HlFJY8sp9B3QIndf1GPyf79PwToDvEkAPSxQ7G jJRIhEBlkdD+lbQirKMNAMu+1kU2H4iAU8XoywWu7BSoX0T3/Rj26dLC2QrotxgFeTf3 JmFw== X-Forwarded-Encrypted: i=1; AJvYcCWYuUhuYUcao0F4i3i2MEystPPctSofrx+YPFdUjRnoxcStolU9BAimLj8ICCKzLMd3t2YQpQXn9zqfZ525ofdycHu+sAW5CF6WlPFrx5XEz20A3PTBvQkgtiiCAr/QSpw17CSDNpXIag== X-Gm-Message-State: AOJu0Yy8aCSlPyANpBE1ynic+EtHjIDCSe9HRKbS/OAMayThFaKKyqvD QOAd2eyWU6XoFeUmGkvkHWQ3MWGQ3diE9FkBkNnS91NBFWMvsAGe X-Google-Smtp-Source: AGHT+IF6cb2Coy/i5yCB0WW6AvyoTFe5pEqYcXLBNqnzUvY7clw+JaJVF9mZocUuuHLPFK98NnSUow== X-Received: by 2002:a05:600c:3b86:b0:426:55a3:71af with SMTP id 5b1f17b1804b1-426708fa552mr46818735e9.33.1720686441995; Thu, 11 Jul 2024 01:27:21 -0700 (PDT) Received: from localhost.localdomain ([79.175.114.8]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-367cde84712sm7134283f8f.33.2024.07.11.01.27.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jul 2024 01:27:21 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v5 11/11] MIPS: CPS: Boot CPUs in secondary clusters Date: Thu, 11 Jul 2024 10:26:56 +0200 Message-Id: <20240711082656.1889440-12-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240711082656.1889440-1-arikalo@gmail.com> References: <20240711082656.1889440-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton Probe for & boot CPUs (cores & VPs) in secondary clusters (ie. not the cluster that began booting Linux) when they are present in systems with CM 3.5 or higher. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/include/asm/mips-cm.h | 18 +++ arch/mips/include/asm/smp-cps.h | 1 + arch/mips/kernel/mips-cm.c | 4 +- arch/mips/kernel/smp-cps.c | 208 ++++++++++++++++++++++++++++---- 4 files changed, 207 insertions(+), 24 deletions(-) diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h index c2930a75b7e4..65a46b35a218 100644 --- a/arch/mips/include/asm/mips-cm.h +++ b/arch/mips/include/asm/mips-cm.h @@ -251,6 +251,12 @@ GCR_ACCESSOR_RW(32, 0x130, l2_config) GCR_ACCESSOR_RO(32, 0x150, sys_config2) #define CM_GCR_SYS_CONFIG2_MAXVPW GENMASK(3, 0) +/* GCR_L2-RAM_CONFIG - Configuration & status of L2 cache RAMs */ +GCR_ACCESSOR_RW(64, 0x240, l2_ram_config) +#define CM_GCR_L2_RAM_CONFIG_PRESENT BIT(31) +#define CM_GCR_L2_RAM_CONFIG_HCI_DONE BIT(30) +#define CM_GCR_L2_RAM_CONFIG_HCI_SUPPORTED BIT(29) + /* GCR_L2_PFT_CONTROL - Controls hardware L2 prefetching */ GCR_ACCESSOR_RW(32, 0x300, l2_pft_control) #define CM_GCR_L2_PFT_CONTROL_PAGEMASK GENMASK(31, 12) @@ -262,6 +268,18 @@ GCR_ACCESSOR_RW(32, 0x308, l2_pft_control_b) #define CM_GCR_L2_PFT_CONTROL_B_CEN BIT(8) #define CM_GCR_L2_PFT_CONTROL_B_PORTID GENMASK(7, 0) +/* GCR_L2_TAG_ADDR - Access addresses in L2 cache tags */ +GCR_ACCESSOR_RW(64, 0x600, l2_tag_addr) + +/* GCR_L2_TAG_STATE - Access L2 cache tag state */ +GCR_ACCESSOR_RW(64, 0x608, l2_tag_state) + +/* GCR_L2_DATA - Access data in L2 cache lines */ +GCR_ACCESSOR_RW(64, 0x610, l2_data) + +/* GCR_L2_ECC - Access ECC information from L2 cache lines */ +GCR_ACCESSOR_RW(64, 0x618, l2_ecc) + /* GCR_L2SM_COP - L2 cache op state machine control */ GCR_ACCESSOR_RW(32, 0x620, l2sm_cop) #define CM_GCR_L2SM_COP_PRESENT BIT(31) diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cps.h index a629e948a6fd..10d3ebd890cb 100644 --- a/arch/mips/include/asm/smp-cps.h +++ b/arch/mips/include/asm/smp-cps.h @@ -23,6 +23,7 @@ struct core_boot_config { }; struct cluster_boot_config { + unsigned long *core_power; struct core_boot_config *core_config; }; diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c index 3eb2cfb893e1..9854bc2b6895 100644 --- a/arch/mips/kernel/mips-cm.c +++ b/arch/mips/kernel/mips-cm.c @@ -308,7 +308,9 @@ void mips_cm_lock_other(unsigned int cluster, unsigned int core, FIELD_PREP(CM3_GCR_Cx_OTHER_VP, vp); if (cm_rev >= CM_REV_CM3_5) { - val |= CM_GCR_Cx_OTHER_CLUSTER_EN; + if (cluster != cpu_cluster(¤t_cpu_data)) + val |= CM_GCR_Cx_OTHER_CLUSTER_EN; + val |= CM_GCR_Cx_OTHER_GIC_EN; val |= FIELD_PREP(CM_GCR_Cx_OTHER_CLUSTER, cluster); val |= FIELD_PREP(CM_GCR_Cx_OTHER_BLOCK, block); } else { diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c index 1954c9e912b2..ae814fdb930c 100644 --- a/arch/mips/kernel/smp-cps.c +++ b/arch/mips/kernel/smp-cps.c @@ -36,12 +36,56 @@ enum label_id { UASM_L_LA(_not_nmi) -static DECLARE_BITMAP(core_power, NR_CPUS); static uint32_t core_entry_reg; static phys_addr_t cps_vec_pa; struct cluster_boot_config *mips_cps_cluster_bootcfg; +static void power_up_other_cluster(unsigned int cluster) +{ + u32 stat, seq_state; + unsigned int timeout; + + mips_cm_lock_other(cluster, CM_GCR_Cx_OTHER_CORE_CM, 0, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); + stat = read_cpc_co_stat_conf(); + mips_cm_unlock_other(); + + seq_state = stat & CPC_Cx_STAT_CONF_SEQSTATE; + seq_state >>= __ffs(CPC_Cx_STAT_CONF_SEQSTATE); + if (seq_state == CPC_Cx_STAT_CONF_SEQSTATE_U5) + return; + + /* Set endianness & power up the CM */ + mips_cm_lock_other(cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + write_cpc_redir_sys_config(IS_ENABLED(CONFIG_CPU_BIG_ENDIAN)); + write_cpc_redir_pwrup_ctl(1); + mips_cm_unlock_other(); + + /* Wait for the CM to start up */ + timeout = 1000; + mips_cm_lock_other(cluster, CM_GCR_Cx_OTHER_CORE_CM, 0, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); + while (1) { + stat = read_cpc_co_stat_conf(); + seq_state = stat & CPC_Cx_STAT_CONF_SEQSTATE; + seq_state >>= __ffs(CPC_Cx_STAT_CONF_SEQSTATE); + if (seq_state == CPC_Cx_STAT_CONF_SEQSTATE_U5) + break; + + if (timeout) { + mdelay(1); + timeout--; + } else { + pr_warn("Waiting for cluster %u CM to power up... STAT_CONF=0x%x\n", + cluster, stat); + mdelay(1000); + } + } + + mips_cm_unlock_other(); +} + static unsigned __init core_vpe_count(unsigned int cluster, unsigned core) { return min(smp_max_threads, mips_cps_numvps(cluster, core)); @@ -152,6 +196,9 @@ static void __init cps_smp_setup(void) pr_cont(","); pr_cont("{"); + if (mips_cm_revision() >= CM_REV_CM3_5) + power_up_other_cluster(cl); + ncores = mips_cps_numcores(cl); for (c = 0; c < ncores; c++) { core_vpes = core_vpe_count(cl, c); @@ -179,8 +226,8 @@ static void __init cps_smp_setup(void) /* Indicate present CPUs (CPU being synonymous with VPE) */ for (v = 0; v < min_t(unsigned, nvpes, NR_CPUS); v++) { - set_cpu_possible(v, cpu_cluster(&cpu_data[v]) == 0); - set_cpu_present(v, cpu_cluster(&cpu_data[v]) == 0); + set_cpu_possible(v, true); + set_cpu_present(v, true); __cpu_number_map[v] = v; __cpu_logical_map[v] = v; } @@ -188,9 +235,6 @@ static void __init cps_smp_setup(void) /* Set a coherent default CCA (CWB) */ change_c0_config(CONF_CM_CMASK, 0x5); - /* Core 0 is powered up (we're running on it) */ - bitmap_set(core_power, 0, 1); - /* Initialise core 0 */ mips_cps_core_init(); @@ -272,6 +316,10 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) goto err_out; mips_cps_cluster_bootcfg[cl].core_config = core_bootcfg; + mips_cps_cluster_bootcfg[cl].core_power = + kcalloc(BITS_TO_LONGS(ncores), sizeof(unsigned long), + GFP_KERNEL); + /* Allocate VPE boot configuration structs */ for (c = 0; c < ncores; c++) { core_vpes = core_vpe_count(cl, c); @@ -283,11 +331,12 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) } } - /* Mark this CPU as booted */ + /* Mark this CPU as powered up & booted */ cl = cpu_cluster(¤t_cpu_data); c = cpu_core(¤t_cpu_data); cluster_bootcfg = &mips_cps_cluster_bootcfg[cl]; core_bootcfg = &cluster_bootcfg->core_config[c]; + bitmap_set(cluster_bootcfg->core_power, cpu_core(¤t_cpu_data), 1); atomic_set(&core_bootcfg->vpe_mask, 1 << cpu_vpe_id(¤t_cpu_data)); return; @@ -315,13 +364,118 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) } } -static void boot_core(unsigned int core, unsigned int vpe_id) +static void init_cluster_l2(void) { - u32 stat, seq_state; - unsigned timeout; + u32 l2_cfg, l2sm_cop, result; + + while (1) { + l2_cfg = read_gcr_redir_l2_ram_config(); + + /* If HCI is not supported, use the state machine below */ + if (!(l2_cfg & CM_GCR_L2_RAM_CONFIG_PRESENT)) + break; + if (!(l2_cfg & CM_GCR_L2_RAM_CONFIG_HCI_SUPPORTED)) + break; + + /* If the HCI_DONE bit is set, we're finished */ + if (l2_cfg & CM_GCR_L2_RAM_CONFIG_HCI_DONE) + return; + } + + l2sm_cop = read_gcr_redir_l2sm_cop(); + if (WARN(!(l2sm_cop & CM_GCR_L2SM_COP_PRESENT), + "L2 init not supported on this system yet")) + return; + + /* Clear L2 tag registers */ + write_gcr_redir_l2_tag_state(0); + write_gcr_redir_l2_ecc(0); + + /* Ensure the L2 tag writes complete before the state machine starts */ + mb(); + + /* Wait for the L2 state machine to be idle */ + do { + l2sm_cop = read_gcr_redir_l2sm_cop(); + } while (l2sm_cop & CM_GCR_L2SM_COP_RUNNING); + + /* Start a store tag operation */ + l2sm_cop = CM_GCR_L2SM_COP_TYPE_IDX_STORETAG; + l2sm_cop <<= __ffs(CM_GCR_L2SM_COP_TYPE); + l2sm_cop |= CM_GCR_L2SM_COP_CMD_START; + write_gcr_redir_l2sm_cop(l2sm_cop); + + /* Ensure the state machine starts before we poll for completion */ + mb(); + + /* Wait for the operation to be complete */ + do { + l2sm_cop = read_gcr_redir_l2sm_cop(); + result = l2sm_cop & CM_GCR_L2SM_COP_RESULT; + result >>= __ffs(CM_GCR_L2SM_COP_RESULT); + } while (!result); + + WARN(result != CM_GCR_L2SM_COP_RESULT_DONE_OK, + "L2 state machine failed cache init with error %u\n", result); +} + +static void boot_core(unsigned int cluster, unsigned int core, + unsigned int vpe_id) +{ + struct cluster_boot_config *cluster_cfg; + u32 access, stat, seq_state; + unsigned int timeout, ncores; + + cluster_cfg = &mips_cps_cluster_bootcfg[cluster]; + ncores = mips_cps_numcores(cluster); + + if ((cluster != cpu_cluster(¤t_cpu_data)) && + bitmap_empty(cluster_cfg->core_power, ncores)) { + power_up_other_cluster(cluster); + + mips_cm_lock_other(cluster, core, 0, + CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + /* Ensure cluster GCRs are where we expect */ + write_gcr_redir_base(read_gcr_base()); + write_gcr_redir_cpc_base(read_gcr_cpc_base()); + write_gcr_redir_gic_base(read_gcr_gic_base()); + + init_cluster_l2(); + + /* Mirror L2 configuration */ + write_gcr_redir_l2_only_sync_base(read_gcr_l2_only_sync_base()); + write_gcr_redir_l2_pft_control(read_gcr_l2_pft_control()); + write_gcr_redir_l2_pft_control_b(read_gcr_l2_pft_control_b()); + + /* Mirror ECC/parity setup */ + write_gcr_redir_err_control(read_gcr_err_control()); + + /* Set BEV base */ + write_gcr_redir_bev_base(core_entry_reg); + + mips_cm_unlock_other(); + } + + if (cluster != cpu_cluster(¤t_cpu_data)) { + mips_cm_lock_other(cluster, core, 0, + CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + /* Ensure the core can access the GCRs */ + access = read_gcr_redir_access(); + access |= BIT(core); + write_gcr_redir_access(access); + + mips_cm_unlock_other(); + } else { + /* Ensure the core can access the GCRs */ + access = read_gcr_access(); + access |= BIT(core); + write_gcr_access(access); + } /* Select the appropriate core */ - mips_cm_lock_other(0, core, 0, CM_GCR_Cx_OTHER_BLOCK_LOCAL); + mips_cm_lock_other(cluster, core, 0, CM_GCR_Cx_OTHER_BLOCK_LOCAL); /* Set its reset vector */ write_gcr_co_reset_base(core_entry_reg); @@ -332,9 +486,6 @@ static void boot_core(unsigned int core, unsigned int vpe_id) /* Start it with the legacy memory map and exception base */ write_gcr_co_reset_ext_base(CM_GCR_Cx_RESET_EXT_BASE_UEB); - /* Ensure the core can access the GCRs */ - set_gcr_access(1 << core); - if (mips_cpc_present()) { /* Reset the core */ mips_cpc_lock_other(core); @@ -384,7 +535,17 @@ static void boot_core(unsigned int core, unsigned int vpe_id) mips_cm_unlock_other(); /* The core is now powered up */ - bitmap_set(core_power, core, 1); + bitmap_set(cluster_cfg->core_power, core, 1); + + /* + * Restore CM_PWRUP=0 so that the CM can power down if all the cores in + * the cluster do (eg. if they're all removed via hotplug. + */ + if (mips_cm_revision() >= CM_REV_CM3_5) { + mips_cm_lock_other(cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + write_cpc_redir_pwrup_ctl(0); + mips_cm_unlock_other(); + } } static void remote_vpe_boot(void *dummy) @@ -410,10 +571,6 @@ static int cps_boot_secondary(int cpu, struct task_struct *idle) unsigned int remote; int err; - /* We don't yet support booting CPUs in other clusters */ - if (cpu_cluster(&cpu_data[cpu]) != cpu_cluster(&raw_current_cpu_data)) - return -ENOSYS; - vpe_cfg->pc = (unsigned long)&smp_bootstrap; vpe_cfg->sp = __KSTK_TOS(idle); vpe_cfg->gp = (unsigned long)task_thread_info(idle); @@ -422,14 +579,15 @@ static int cps_boot_secondary(int cpu, struct task_struct *idle) preempt_disable(); - if (!test_bit(core, core_power)) { + if (!test_bit(core, cluster_cfg->core_power)) { /* Boot a VPE on a powered down core */ - boot_core(core, vpe_id); + boot_core(cluster, core, vpe_id); goto out; } if (cpu_has_vp) { - mips_cm_lock_other(0, core, vpe_id, CM_GCR_Cx_OTHER_BLOCK_LOCAL); + mips_cm_lock_other(cluster, core, vpe_id, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); write_gcr_co_reset_base(core_entry_reg); mips_cm_unlock_other(); } @@ -636,11 +794,15 @@ static void cps_cpu_die(unsigned int cpu) { } static void cps_cleanup_dead_cpu(unsigned cpu) { + unsigned int cluster = cpu_cluster(&cpu_data[cpu]); unsigned core = cpu_core(&cpu_data[cpu]); unsigned int vpe_id = cpu_vpe_id(&cpu_data[cpu]); ktime_t fail_time; unsigned stat; int err; + struct cluster_boot_config *cluster_cfg; + + cluster_cfg = &mips_cps_cluster_bootcfg[cluster]; /* * Now wait for the CPU to actually offline. Without doing this that @@ -692,7 +854,7 @@ static void cps_cleanup_dead_cpu(unsigned cpu) } while (1); /* Indicate the core is powered off */ - bitmap_clear(core_power, core, 1); + bitmap_clear(cluster_cfg->core_power, core, 1); } else if (cpu_has_mipsmt) { /* * Have a CPU with access to the offlined CPUs registers wait