From patchwork Thu Sep 12 09:30:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801803 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C1FB198A05; Thu, 12 Sep 2024 09:30:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133460; cv=none; b=kvJl4zSN9r8Ut/11QZvIwuK02ElXRFVA+P6QzDGNS5HCNo7aI56IpGW/9iJwA4tBABm1Pyj5lgLAl8tuJSnClJ6XnmMcSqU9nMfz+5X+VJz/XkI3MvGyTudQt6USmhuS440hoAZVC8zEZH4fcAPpEsEEv5lkxhXDCapWMpcblKs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133460; c=relaxed/simple; bh=cAw1fMa88K0eSA4QdBE4QrEFyFaLSup0xnRQDCeAFRE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jUoTlCF6E9/OHWVtbFSf1kAZBZFeLJK01IExlqqX233J58mKD7uyjA9RPT12Q31FjmjQSvN4bRaqYJMQG/GDq31BP/X3VvCn+sIBU1YkEw8NcccUqxK58ataLsipgS4FtooKSlXjMXW8Qyr37Kk6Odi8YJQ2PJvHRYPDGMftti4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=a3J0ciFm; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="a3J0ciFm" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-42cd46f3ac9so6547565e9.3; Thu, 12 Sep 2024 02:30:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133457; x=1726738257; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3bRNszhMVq/3+e44FU1ywn3qp6VaEiQfTqGQh9hECvc=; b=a3J0ciFmLcR6aKtFXjXpcmbYqAK/pYUDMsTiSfPjodANL+HquIEBGhme+dp2JAYRrx GkD9QtXFQsuo6bk9KGiqeJXh1NmVV3EuuxmUlukskwp6bu1Yyzu3JlX3ySk+QxTB5P1b HlFWcbsoDUFTmBMjC/5svhOOE/OuE3vPwRpX0OHNJJKCVgjtQXH+m8NvroiAOK4VkHAx T4mWidcdKMVGTjjvkbgyHn+GgEwbtoYpBThBDvAA4beDt56d5QUruF3HPxDlF3bOhZlD WM9ynWb7ycs6kcpZ6pw8vuvnivxwrfFmwXtH1fgfq79MYPgN9TN4YIohIXx3eyKwvSJw MgTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133457; x=1726738257; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3bRNszhMVq/3+e44FU1ywn3qp6VaEiQfTqGQh9hECvc=; b=W0mkm9T3LWIqVq35+iqr/mcmY9AVyb3XK8FpnB2JnG6hEDbDo8MwWs5CwmZe6v6+2m UdA4esDo/53JpGL6KPR2KKpBHt3jWU4teWIj1qlGkeL36pSkD0dR++n86R+vz4+ifA0x nWj7hQQvQ6z6N5FCkl0G7vGQwqRc/8853T2eFvwIK0N1MF4NOJhLM9Z1yr25NuvDEuRE YCncJBA3OB7VsLqNvmM9Lhx1Ov+4S9YnT2LiZWCVnGGcwmGQItmL1eP5HR5fiuqoG/36 /MyuHCCnCbEfMIAjpXJ3Q1O17j1NSf0MgiaVTWAB4eF7YEfFFO6h1LYjn5egUAhAZs6j nWzg== X-Forwarded-Encrypted: i=1; AJvYcCUc6I8GF01rqB8Tqp9O7kESJfbG1QNKklZmBLiGWjk/DwQzGLxn0xsDEiw306DQIzYgEs3tl9OgA1XrBA==@vger.kernel.org, AJvYcCXReMzGNRYwqd9jIYotRLpMPVsT5eF2pzT+Vb7e4H683ENyJ8D6lg1pBXHYpOGNae4hQ72AUXaX0xkueaU=@vger.kernel.org X-Gm-Message-State: AOJu0YwfoXZ9V/7bI7gF/STpztzEbNOvj6gNMxzCOemlkhLmhEmGgsXp 1RJ0BEt4rfBxyQKC7O2qxCdvCqYAM1q/Xo4/Pn12v2sLWzRUfORm X-Google-Smtp-Source: AGHT+IGJyBcjyFUHN1mKAxQ1N+NjIrechU3TDap7lzNpBTTBhXbaSyWFVEljHcj4VVhuumujNO+PCA== X-Received: by 2002:a05:600c:35c1:b0:42c:bfb1:766e with SMTP id 5b1f17b1804b1-42cdb55006fmr17061985e9.21.1726133456631; Thu, 12 Sep 2024 02:30:56 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.30.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:30:56 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 1/9] irqchip/mips-gic: Introduce for_each_online_cpu_gic() Date: Thu, 12 Sep 2024 11:30:43 +0200 Message-Id: <20240912093051.452172-2-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton Parts of code in the MIPS GIC driver operate on the GIC local register block for each online CPU, accessing each via the GIC's other/redirect register block. Abstract the process of iterating over online CPUs & configuring the other/redirect region to access their registers through a new for_each_online_cpu_gic() macro. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- drivers/irqchip/irq-mips-gic.c | 59 +++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 76253e864f23..6c7a7d2f0438 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -66,6 +66,44 @@ static struct gic_all_vpes_chip_data { bool mask; } gic_all_vpes_chip_data[GIC_NUM_LOCAL_INTRS]; +static int __gic_with_next_online_cpu(int prev) +{ + unsigned int cpu; + + /* Discover the next online CPU */ + cpu = cpumask_next(prev, cpu_online_mask); + + /* If there isn't one, we're done */ + if (cpu >= nr_cpu_ids) + return cpu; + + /* + * Move the access lock to the next CPU's GIC local register block. + * + * Set GIC_VL_OTHER. Since the caller holds gic_lock nothing can + * clobber the written value. + */ + write_gic_vl_other(mips_cm_vp_id(cpu)); + + return cpu; +} + +/** + * for_each_online_cpu_gic() - Iterate over online CPUs, access local registers + * @cpu: An integer variable to hold the current CPU number + * @gic_lock: A pointer to raw spin lock used as a guard + * + * Iterate over online CPUs & configure the other/redirect register region to + * access each CPUs GIC local register block, which can be accessed from the + * loop body using read_gic_vo_*() or write_gic_vo_*() accessor functions or + * their derivatives. + */ +#define for_each_online_cpu_gic(cpu, gic_lock) \ + guard(raw_spinlock_irqsave)(gic_lock); \ + for ((cpu) = __gic_with_next_online_cpu(-1); \ + (cpu) < nr_cpu_ids; \ + (cpu) = __gic_with_next_online_cpu(cpu)) + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -350,37 +388,27 @@ static struct irq_chip gic_local_irq_controller = { static void gic_mask_local_irq_all_vpes(struct irq_data *d) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; int intr, cpu; intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); cd = irq_data_get_irq_chip_data(d); cd->mask = false; - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_rmask(BIT(intr)); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); } static void gic_unmask_local_irq_all_vpes(struct irq_data *d) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; int intr, cpu; intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); cd = irq_data_get_irq_chip_data(d); cd->mask = true; - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_smask(BIT(intr)); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); } static void gic_all_vpes_irq_cpu_online(void) @@ -469,7 +497,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hwirq) { struct gic_all_vpes_chip_data *cd; - unsigned long flags; unsigned int intr; int err, cpu; u32 map; @@ -533,12 +560,8 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, if (!gic_local_irq_is_routable(intr)) return -EPERM; - raw_spin_lock_irqsave(&gic_lock, flags); - for_each_online_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); + for_each_online_cpu_gic(cpu, &gic_lock) write_gic_vo_map(mips_gic_vx_map_reg(intr), map); - } - raw_spin_unlock_irqrestore(&gic_lock, flags); return 0; } From patchwork Thu Sep 12 09:30:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801804 Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 778CD19F412; Thu, 12 Sep 2024 09:31:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133463; cv=none; b=tGpPbfxzpIFBxYG0b9IQqf1IZ3yzq4cORRzW5PtbBhNT881LWkVj356GIbu0gdGtM4Wl/QXwghwBLSwkcSXQ2eVukCwphlHSvMnCum/5yGrHIMCq08tBRzKlJ37TERjWvCybOTgtUj6T9PhPfo/Ph+boukZtSNuBBp8B7hX1gag= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133463; c=relaxed/simple; bh=6v8h21BYHwmBn6z3r5CIfCo7toAhvHcTjG+vWOZBQCA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eEtJTpaDWi3zi4Bh9vt7jSXNtFVqVfZLUWMB3OLKLBnRnABOCkys/HC2B+CfexY8tpVKdk/oXwsbO/jQUExo61ydZbyS5P6yf9Su0vD5H/Cq4lSn7XoSHEhHZR6ImfJG61PJX6sVJTGzeUEUzaghJEyKuYv0R9+zMF7Z61J9ZqY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IFw37+lk; arc=none smtp.client-ip=209.85.167.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IFw37+lk" Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-5365b71a6bdso829705e87.2; Thu, 12 Sep 2024 02:31:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133459; x=1726738259; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ajgv0hB3TeKXEQwcY+X8KT/8Qpdg4uMdZRkO0EFzzv0=; b=IFw37+lklsAZZ2OdoVqR2/bXtkgB+n5aGAMdHF2PLsmub9WNV7mnWjxY99WpuXTpao oRiv4uWTpkZRYdYS/AUavU8HM7PRqKqjAOR3EiWJskNduR13EukgH6k6xBHwQmY1evRW hkAM9v9Vf0PBwTB8d1vK5oFVH+wqzzR/q3/4M+pR1rzXNbbFPMpTHbbx0lUXHjoUwwfy xgbUwpc/XVDsXB+clUtVylrEtxGB3lQkT/fNawkvCxJdDy7HuzMCzZnKVj5n6Wg4GDGM az7UYWm0ON3znOQ1CyguuKi4xqvJ9krSsXfeeKQKS/xCQRzJ8/fAMEzVHd4BJC3jV57l DNXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133459; x=1726738259; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ajgv0hB3TeKXEQwcY+X8KT/8Qpdg4uMdZRkO0EFzzv0=; b=WbVYYeuncNUt2dwD3d08mdKBFBtVKFXlkgn0QEDHVut97exjtGE1WRgqSOL/aP4RX+ pSL5lYOdser9Fvh6vTUE1BJCa+hopZDjms9DKPXH+VZlXxRSlTuE38n73IsH/5VJ1kus rYX3ty3wiymGUa8rmhqT/xDGBKO4wecQmjBtTjayYiPWszdF4TnoMC458AucZbVwxVcQ CtXksAHpbWWQu6CCwaSTuIJvxo6j5NVYOVmqBn7skIU6keLV4KNGkl3CJKpk/YK+Scs7 SyASdk8M9KNtVWO7nw6zlQ3JnqNc1CeXsmlO1XOFZt6+LlggQaYcWSFjlSPIOedgYekm JZnA== X-Forwarded-Encrypted: i=1; AJvYcCU78tZXYUmrx0LHP8Z3Z5VUxapUO1O5u+flPbEuuJ+eDPV3SVAfMJi74Y2p2lwb+A8qWvq4Tf9M5RKCjA==@vger.kernel.org, AJvYcCXac76BBEW9z9wBz1OM6xOfWJNfOl4/O2uw4cO0A97ut5BZckOcCUE6KOaUbFum+9CmckSJHU7lEXDUKHg=@vger.kernel.org X-Gm-Message-State: AOJu0YyuXRVo5gDLMn8n1ud+fvxmQMwJS+MjHyMai19uM4hJvjZf3pFt 9ST3iCheHVWhBNPCja0ANJKZhbWxbtjJlkd1K8KHbbg1BcMJ2nZl X-Google-Smtp-Source: AGHT+IHDYnW78bOUZS4nhLcuxJll9/+0I9MSg/om/F296GkVTANK4fS4HTz7sKENbHh6UUayiXU6zA== X-Received: by 2002:a05:6512:ac4:b0:535:6935:7869 with SMTP id 2adb3069b0e04-53678ff5ec3mr1164147e87.55.1726133459051; Thu, 12 Sep 2024 02:30:59 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.30.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:30:58 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 2/9] irqchip/mips-gic: Support multi-cluster in for_each_online_cpu_gic() Date: Thu, 12 Sep 2024 11:30:44 +0200 Message-Id: <20240912093051.452172-3-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton Use CM's GCR_CL_REDIRECT register to access registers in remote clusters, so users of gic_with_each_online_cpu() gain support for multi-cluster with no further changes. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Signed-off-by: Thomas Gleixner Tested-by: Serge Semin --- drivers/irqchip/irq-mips-gic.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 6c7a7d2f0438..29bdfdce2123 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -88,6 +88,12 @@ static int __gic_with_next_online_cpu(int prev) return cpu; } +static inline void gic_unlock_cluster(void) +{ + if (mips_cps_multicluster_cpus()) + mips_cm_unlock_other(); +} + /** * for_each_online_cpu_gic() - Iterate over online CPUs, access local registers * @cpu: An integer variable to hold the current CPU number @@ -102,6 +108,7 @@ static int __gic_with_next_online_cpu(int prev) guard(raw_spinlock_irqsave)(gic_lock); \ for ((cpu) = __gic_with_next_online_cpu(-1); \ (cpu) < nr_cpu_ids; \ + gic_unlock_cluster(), \ (cpu) = __gic_with_next_online_cpu(cpu)) static void gic_clear_pcpu_masks(unsigned int intr) From patchwork Thu Sep 12 09:30:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801805 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B578519F421; Thu, 12 Sep 2024 09:31:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133464; cv=none; b=iDXmeIujLjx2+ioDlJz8SseUOgXEAjfRYYQkMzrC38A/YgR5V01EvXiM2LZ2AlSpo/laMElBthI58XZbrltLL1iSAX8iUHbQ4nTHlIuXtSxsNHnF7ZNKReDSNqT74kTjb0ppr6NxEA22I4qNBFydRnpcANb9PrDXMGBnTqo/zzA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133464; c=relaxed/simple; bh=F+ZcSGOqx9X/ZvXyBUNOAihkxk6As3xiu6/7A855sXA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eZCzcomFAugCuWisgJCZ2sg3u4CbmQWujm7FqRWDQDz2c9tAEtzwCSvJN6RH1UhAuISVX27jGdkrE93+qPXDf6h/ZvMgtfxqMFhZEaDaF4EUz88Gt8ehUlCJWGpILvUv6IPlCQsPN6TUjeGAP5iB0H7pi+O4s+8Ssoq0LD3gd1U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OsUHJ8pb; arc=none smtp.client-ip=209.85.128.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OsUHJ8pb" Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-42bbffe38e6so5690875e9.0; Thu, 12 Sep 2024 02:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133461; x=1726738261; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jz0kUjpzsFWVo1j1eqZUTSojq1aOuYqxFxKOuRH0s6M=; b=OsUHJ8pb13qQXuQjR4jWe5M+qmLGmWpLw8++YjCaOLFM2GEQcEzaQQQUeCFKusHsr+ wR3Z9Ndx6ikNPQoxmxibt7Elj/K3LPKdgPuwXztMdeGkirPNJO9QVNHfZ76lOwmzF4Or t8OJebMeWYXRd71ARwNGh7seUbw+QZeT+YAphALERYWC6TWt9R+1+B1j5n4P8g7BquP3 KJ0QZo063wUkiMAi9stb/h77EUh9dloZr9evimYdNrFIsC3x3zMZbiDnJsF3cexubgpX ngVlqU55Ll9XMdyP1mR7D2hNgcVsVnaASGzaruKmuik1SeKvJvppK20ALN3sJShkugW/ 9qHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133461; x=1726738261; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jz0kUjpzsFWVo1j1eqZUTSojq1aOuYqxFxKOuRH0s6M=; b=T6/jmSz6KQXX57pwL9sIHyo9EuQ/rpaHU1UNUpAxN0Ntc4LyK2Amruyynxn6fvx7AJ t5ykfE/qS61fsDTwHttJ2BqBa8yUuFcZ6Pbr4JAFg1Pj/xqifmsZK/KDqpRm5s7fU6zr eaXkIsDPIZoFvb0nUhUramDz9glHq53gcaGA3i64TJ6ld5yjsMKLvIsiqXlR6zkvktks BSWQ8fICvUq7FbmwpjUnfi0ZdnggqtzzfQMGR3TpqKLb/Ir7LW6/S3go2TpadRYjh42m mOTqJE+Jem6DClYNuDyJV5Ji201W/ahSdMw6axASIBko12JbaSkljJukfD2ozgQoLxg9 3A4A== X-Forwarded-Encrypted: i=1; AJvYcCW3IYQGexo5TCiGFrALJq4dCunN6tq2S0EB4umoK9QJZq8aSbqngjkp47oGVTZ6I8RvNO87glP+GE8Tbg==@vger.kernel.org, AJvYcCWotzWUsiYz5RWo50/LBJ60FYTqxa1dH81iadUBMULu4uZ0NnPYvDc0eWlO++beHQ3k/yvumIbKSfrPj2o=@vger.kernel.org X-Gm-Message-State: AOJu0YyRzGIg+7zSu75sSSIy/e9GcD86/3QPqB/wqRVZiXzrlHPAFKh8 aelo7a7QXqjqAJ5L3OYZyXsgKKdB6TCJ+WKrR6CwbIM0V5SnBd1C X-Google-Smtp-Source: AGHT+IEuBnqLhcEUuPRV/yZSG+3KlKTYwY3VBgwE8+43RgJAjvCcbUKrxv2gmdyYCRwqY6t0nVrOOg== X-Received: by 2002:a05:600c:1c27:b0:428:150e:4f13 with SMTP id 5b1f17b1804b1-42cdb572754mr16614615e9.33.1726133461065; Thu, 12 Sep 2024 02:31:01 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.30.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:31:00 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 3/9] irqchip/mips-gic: Setup defaults in each cluster Date: Thu, 12 Sep 2024 11:30:45 +0200 Message-Id: <20240912093051.452172-4-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chao-ying Fu In multi-cluster MIPS I6500 systems, there is a GIC per cluster. The default shared interrupt setup configured in gic_of_init() will only apply to the GIC in the cluster containing the boot CPU, leaving the GICs of other clusters unconfigured. Similarly configure other clusters. Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- drivers/irqchip/irq-mips-gic.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 29bdfdce2123..d93a076620c7 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -764,7 +764,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, gicconfig; + unsigned int cpu_vec, i, gicconfig, cl, nclusters; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -845,11 +845,29 @@ static int __init gic_of_init(struct device_node *node, board_bind_eic_interrupt = &gic_bind_eic_interrupt; - /* Setup defaults */ - for (i = 0; i < gic_shared_intrs; i++) { - change_gic_pol(i, GIC_POL_ACTIVE_HIGH); - change_gic_trig(i, GIC_TRIG_LEVEL); - write_gic_rmask(i); + /* + * Initialise each cluster's GIC shared registers to sane default + * values. + * Otherwise, the IPI set up will be erased if we move code + * to gic_cpu_startup for each cpu. + */ + nclusters = mips_cps_numclusters(); + for (cl = 0; cl < nclusters; cl++) { + if (cl == cpu_cluster(¤t_cpu_data)) { + for (i = 0; i < gic_shared_intrs; i++) { + change_gic_pol(i, GIC_POL_ACTIVE_HIGH); + change_gic_trig(i, GIC_TRIG_LEVEL); + write_gic_rmask(i); + } + } else { + mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + for (i = 0; i < gic_shared_intrs; i++) { + change_gic_redir_pol(i, GIC_POL_ACTIVE_HIGH); + change_gic_redir_trig(i, GIC_TRIG_LEVEL); + write_gic_redir_rmask(i); + } + mips_cm_unlock_other(); + } } return cpuhp_setup_state(CPUHP_AP_IRQ_MIPS_GIC_STARTING, From patchwork Thu Sep 12 09:30:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801806 Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DA1319E980; Thu, 12 Sep 2024 09:31:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133467; cv=none; b=C6F4FlPDonYeL6VKf2O5B6waPYOESZtJlU4OrJSoQtfgDPWPAs7qG3tAmWBQ2ZeGYEDGWOtF43Sw/6zsa0+9HAy3sS8DXDPjf2Twsoqy3nljxNR41ChBfWXGTFPtF9AMjLbWv7Cg7wL7WKHg0zmq59BhC11Wo1QLnziw5YCtSzE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133467; c=relaxed/simple; bh=PO79bYcd199TIcgNt4JG/l2bdfqzjFs867BX4VVkyFs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=a23G002op3iv1LMJD4UQIwLSu+gPKxgLaq66F6djGhd1me+NSvppVMucs9Fh9QD49UG2QyKpTdqQ2hKTnN9tvNcNABPkdxdsJLTbHy9nsh6gBzZPMFcER2hHbdteHc/N/bQMNcoQf6aQIO9khn3CcmqoslCzZNLHz/j/Uipvjos= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=VhB29QNG; arc=none smtp.client-ip=209.85.167.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VhB29QNG" Received: by mail-lf1-f47.google.com with SMTP id 2adb3069b0e04-5365c060f47so848402e87.2; Thu, 12 Sep 2024 02:31:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133463; x=1726738263; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rqX7walQ5/+iLe+3Z9+N8PEDe2lYoBurgkuCUvd7CsE=; b=VhB29QNG7NECSbaL++6ESsX3orxlB0yj4DoOUZ9g1RYorjueW/UrcmWyTsQnJuXSIJ M2Em329at35X3xFA+jQQ8KqlMXMkH4GZISRaZoHqEFQNbs7+DZEz6KV2CIF12ZIGaupw UGGlZXF2Z8BTUSbFswKH3rhdC7yI2YL7K+lwjxOG+rePWktJpMlV/pQ+3lRjxnnmuGLm rMknAdVK8LvDs7seQ2wkE1ra30ESD2B+j2qbWvL3QSJUpJSLb5PxBsq8u+TOVweJwAvH P9Pgzxd9FbA61rxTU+/7WqXtmK0wdW3YARIceBsiA2N0wC4TrABaZiuAdc+p4ocOf+0V T8MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133463; x=1726738263; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rqX7walQ5/+iLe+3Z9+N8PEDe2lYoBurgkuCUvd7CsE=; b=qqqfvCdKSWJ+8E5h1ELaN+uZQAUjlilf0W8aj0/UOSOHrF2F1vYYZjnzokECsgsTkg 59Dp30ZKvI0z0yLyFA6qBfv6cifcMmyZzGqV1ucEhW3RfeOpOxMaf3YyrpUGXmdjASIM 7NJt4QST7938ZhWy/ySmkIFJQf2oNkIYm3502HTreVWDmGLZxKyhiqeNv8WoRdLMX3tH rwXytpqrp4HMMSYA8gpfq7ot5rhhoAWDjAC0MGVfaKviyh+eG5BVPfZ5bLeJs+AaKWH0 vPdlHlTzOh2x+ftyRHZWOa+DxSkJS5WzFKpEXozbFhZDpgMvC7XOl5Koi5afpmDa/X7V JSSw== X-Forwarded-Encrypted: i=1; AJvYcCVZaqAICJphBP1edWMRs+s3rDsbN4+gvU8LQiEIbBqlsGBN56D66EozSJdEd3ca60QH7e/h5DFpP5OOBw==@vger.kernel.org, AJvYcCWxReRlwzDy3b6OTmrckPv1iAzprXCDZHKqpyL6ryfU6C6vOGz0jmjwa+roOGFUF98XsrHs5vDJC+vBPCk=@vger.kernel.org X-Gm-Message-State: AOJu0YwVqzL1se6rhoWFJZiOGFNdhEm7WADfHgQL6sUY1fvVnAOtAfxi 6gSC6b3OSC7NWfiwPxVUONDZuDs8ANyJXaRYanNGjMmJHxnV/NPY3kQS2aNN X-Google-Smtp-Source: AGHT+IELpaCvWg5K+/2T4Hs360SDbVvV4cCVqC3TqDIPxCzm1aFeTy1YacXeA1PeeERX6CW9NHwjmA== X-Received: by 2002:a05:6512:239d:b0:531:4c6d:b8ef with SMTP id 2adb3069b0e04-53678fb74c7mr1284830e87.6.1726133462986; Thu, 12 Sep 2024 02:31:02 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.31.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:31:02 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 4/9] irqchip/mips-gic: Multi-cluster support Date: Thu, 12 Sep 2024 11:30:46 +0200 Message-Id: <20240912093051.452172-5-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton The MIPS I6500 CPU & CM (Coherence Manager) 3.5 introduce the concept of multiple clusters to the system. In these systems, each cluster contains its own GIC, so the GIC isn't truly global any longer. Access to registers in the GICs of remote clusters is possible using a redirect register block much like the redirect register blocks provided by the CM & CPC, and configured through the same GCR_REDIRECT register that mips_cm_lock_other() abstraction builds upon. It is expected that external interrupts are connected identically on all clusters. That is, if there is a device providing an interrupt connected to GIC interrupt pin 0 then it should be connected to pin 0 of every GIC in the system. For the most part, the GIC can be treated as though it is still truly global, so long as interrupts in the cluster are configured properly. This patch introduces support for such multi-cluster systems in the MIPS GIC irqchip driver. A newly introduced gic_irq_lock_cluster() function allows: 1) Configure access to a GIC in a remote cluster via the redirect register block, using mips_cm_lock_other(). Or: 2) Detect that the interrupt in question is affine to the local cluster and plain old GIC register access to the GIC in the local cluster should be used. It is possible to access the local cluster's GIC registers via the redirect block, but keeping the special case for them is both good for performance (because we avoid the locking & indirection overhead of using the redirect block) and necessary to maintain compatibility with systems using CM revisions prior to 3.5 which don't support the redirect block. The gic_irq_lock_cluster() function relies upon an IRQs effective affinity in order to discover which cluster the IRQ is affine to. In order to track this & allow it to be updated at an appropriate point during gic_set_affinity() we select the generic support for effective affinity using CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK. gic_set_affinity() is the one function which gains much complexity. It now deconfigures routing to any VP(E), ie. CPU, on the old cluster when moving affinity to a new cluster. gic_shared_irq_domain_map() moves its update of the IRQs effective affinity to before its use of gic_irq_lock_cluster(), to ensure that operation is on the cluster the IRQ is affine to. The remaining changes are straightforward use of the gic_irq_lock_cluster() function to select between local cluster & remote cluster code-paths when configuring interrupts. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- drivers/irqchip/Kconfig | 1 + drivers/irqchip/irq-mips-gic.c | 161 +++++++++++++++++++++++++++++---- 2 files changed, 143 insertions(+), 19 deletions(-) diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index d078bdc48c38..455e512d5f45 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -345,6 +345,7 @@ config KEYSTONE_IRQ config MIPS_GIC bool + select GENERIC_IRQ_EFFECTIVE_AFF_MASK select GENERIC_IRQ_IPI if SMP select IRQ_DOMAIN_HIERARCHY select MIPS_CM diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index d93a076620c7..f42f69bbd6fb 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -111,6 +111,41 @@ static inline void gic_unlock_cluster(void) gic_unlock_cluster(), \ (cpu) = __gic_with_next_online_cpu(cpu)) +/** + * gic_irq_lock_cluster() - Lock redirect block access to IRQ's cluster + * @d: struct irq_data corresponding to the interrupt we're interested in + * + * Locks redirect register block access to the global register block of the GIC + * within the remote cluster that the IRQ corresponding to @d is affine to, + * returning true when this redirect block setup & locking has been performed. + * + * If @d is affine to the local cluster then no locking is performed and this + * function will return false, indicating to the caller that it should access + * the local clusters registers without the overhead of indirection through the + * redirect block. + * + * In summary, if this function returns true then the caller should access GIC + * registers using redirect register block accessors & then call + * mips_cm_unlock_other() when done. If this function returns false then the + * caller should trivially access GIC registers in the local cluster. + * + * Returns true if locking performed, else false. + */ +static bool gic_irq_lock_cluster(struct irq_data *d) +{ + unsigned int cpu, cl; + + cpu = cpumask_first(irq_data_get_effective_affinity_mask(d)); + BUG_ON(cpu >= NR_CPUS); + + cl = cpu_cluster(&cpu_data[cpu]); + if (cl == cpu_cluster(¤t_cpu_data)) + return false; + + mips_cm_lock_other(cl, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + return true; +} + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -157,7 +192,12 @@ static void gic_send_ipi(struct irq_data *d, unsigned int cpu) { irq_hw_number_t hwirq = GIC_HWIRQ_TO_SHARED(irqd_to_hwirq(d)); - write_gic_wedge(GIC_WEDGE_RW | hwirq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(GIC_WEDGE_RW | hwirq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(GIC_WEDGE_RW | hwirq); + } } int gic_get_c0_compare_int(void) @@ -225,7 +265,13 @@ static void gic_mask_irq(struct irq_data *d) { unsigned int intr = GIC_HWIRQ_TO_SHARED(d->hwirq); - write_gic_rmask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_rmask(intr); + mips_cm_unlock_other(); + } else { + write_gic_rmask(intr); + } + gic_clear_pcpu_masks(intr); } @@ -234,7 +280,12 @@ static void gic_unmask_irq(struct irq_data *d) unsigned int intr = GIC_HWIRQ_TO_SHARED(d->hwirq); unsigned int cpu; - write_gic_smask(intr); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_smask(intr); + mips_cm_unlock_other(); + } else { + write_gic_smask(intr); + } gic_clear_pcpu_masks(intr); cpu = cpumask_first(irq_data_get_effective_affinity_mask(d)); @@ -245,7 +296,12 @@ static void gic_ack_irq(struct irq_data *d) { unsigned int irq = GIC_HWIRQ_TO_SHARED(d->hwirq); - write_gic_wedge(irq); + if (gic_irq_lock_cluster(d)) { + write_gic_redir_wedge(irq); + mips_cm_unlock_other(); + } else { + write_gic_wedge(irq); + } } static int gic_set_type(struct irq_data *d, unsigned int type) @@ -285,9 +341,16 @@ static int gic_set_type(struct irq_data *d, unsigned int type) break; } - change_gic_pol(irq, pol); - change_gic_trig(irq, trig); - change_gic_dual(irq, dual); + if (gic_irq_lock_cluster(d)) { + change_gic_redir_pol(irq, pol); + change_gic_redir_trig(irq, trig); + change_gic_redir_dual(irq, dual); + mips_cm_unlock_other(); + } else { + change_gic_pol(irq, pol); + change_gic_trig(irq, trig); + change_gic_dual(irq, dual); + } if (trig == GIC_TRIG_EDGE) irq_set_chip_handler_name_locked(d, &gic_edge_irq_controller, @@ -305,25 +368,72 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *cpumask, bool force) { unsigned int irq = GIC_HWIRQ_TO_SHARED(d->hwirq); + unsigned int cpu, cl, old_cpu, old_cl; unsigned long flags; - unsigned int cpu; + /* + * The GIC specifies that we can only route an interrupt to one VP(E), + * ie. CPU in Linux parlance, at a time. Therefore we always route to + * the first online CPU in the mask. + */ cpu = cpumask_first_and(cpumask, cpu_online_mask); if (cpu >= NR_CPUS) return -EINVAL; - /* Assumption : cpumask refers to a single CPU */ - raw_spin_lock_irqsave(&gic_lock, flags); + old_cpu = cpumask_first(irq_data_get_effective_affinity_mask(d)); + old_cl = cpu_cluster(&cpu_data[old_cpu]); + cl = cpu_cluster(&cpu_data[cpu]); - /* Re-route this IRQ */ - write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + raw_spin_lock_irqsave(&gic_lock, flags); - /* Update the pcpu_masks */ - gic_clear_pcpu_masks(irq); - if (read_gic_mask(irq)) - set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + /* + * If we're moving affinity between clusters, stop routing the + * interrupt to any VP(E) in the old cluster. + */ + if (cl != old_cl) { + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_vp(irq, 0); + mips_cm_unlock_other(); + } else { + write_gic_map_vp(irq, 0); + } + } + /* + * Update effective affinity - after this gic_irq_lock_cluster() will + * begin operating on the new cluster. + */ irq_data_update_effective_affinity(d, cpumask_of(cpu)); + + /* + * If we're moving affinity between clusters, configure the interrupt + * trigger type in the new cluster. + */ + if (cl != old_cl) + gic_set_type(d, irqd_get_trigger_type(d)); + + /* Route the interrupt to its new VP(E) */ + if (gic_irq_lock_cluster(d)) { + write_gic_redir_map_pin(irq, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_redir_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + + mips_cm_unlock_other(); + } else { + write_gic_map_pin(irq, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(irq, BIT(mips_cm_vp_id(cpu))); + + /* Update the pcpu_masks */ + gic_clear_pcpu_masks(irq); + if (read_gic_mask(irq)) + set_bit(irq, per_cpu_ptr(pcpu_masks, cpu)); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); return IRQ_SET_MASK_OK; @@ -471,11 +581,21 @@ static int gic_shared_irq_domain_map(struct irq_domain *d, unsigned int virq, unsigned long flags; data = irq_get_irq_data(virq); + irq_data_update_effective_affinity(data, cpumask_of(cpu)); raw_spin_lock_irqsave(&gic_lock, flags); - write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); - write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); - irq_data_update_effective_affinity(data, cpumask_of(cpu)); + + /* Route the interrupt to its VP(E) */ + if (gic_irq_lock_cluster(data)) { + write_gic_redir_map_pin(intr, + GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_redir_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + mips_cm_unlock_other(); + } else { + write_gic_map_pin(intr, GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin); + write_gic_map_vp(intr, BIT(mips_cm_vp_id(cpu))); + } + raw_spin_unlock_irqrestore(&gic_lock, flags); return 0; @@ -651,6 +771,9 @@ static int gic_ipi_domain_alloc(struct irq_domain *d, unsigned int virq, if (ret) goto error; + /* Set affinity to cpu. */ + irq_data_update_effective_affinity(irq_get_irq_data(virq + i), + cpumask_of(cpu)); ret = irq_set_irq_type(virq + i, IRQ_TYPE_EDGE_RISING); if (ret) goto error; From patchwork Thu Sep 12 09:30:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801807 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC00F1A0BDA; Thu, 12 Sep 2024 09:31:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133468; cv=none; b=fnb8PbU/RZubHCZbcYGTkK0vDu7mXo/RXunpGJvR86Kuu6PvfsBMnLToEB0B4scx/f63WPdEI0qJsM6jdafXchBAfj0V3lj5wBS9/HUDqYxKzHOHxHLTwNm8hBNtPttxbUdoh39Qfb5Ev/qDzkmi52f3ghcNE9ZS29vvYenMiN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133468; c=relaxed/simple; bh=KyjNq/4JWqFWorsEqD2UphoVO/zOCfofXlli3znYEhs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RQlQ/CQuga1DAKelGjph8e/yr6O+cGgyEyhIFjyMVZgAI+pBxVk8Sw+wY9xoCPl+ddvyFHzWFkH0kPhKPAwtVHOyOSEmcRc+UpPTJNfRFUq19EYwvwhJFfGXi9QyVDmmQR8Fx2K1sCzysvalwQWpecCzvzsNc0pS3+1QiWN7ZsI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TdWFrpiC; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TdWFrpiC" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-42cb5b3c57eso6572695e9.2; Thu, 12 Sep 2024 02:31:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133465; x=1726738265; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F4c0vwBBm2GJGPzUzcwaUmJw/5o8B/sn9ihY/AAAuUE=; b=TdWFrpiC40y2mEVCdbNUJ2C7nmwkhORQdv3wd3/9t7yqFQl/v2D47aSiWGLrBBKj5K GasFQm6BKj+2iRWYBY60LfSGahFlYJku9qRBUe6eVm551l9eDO2Kth53MElMOsy0Ua8A dXHh62yhT9KpbHnsxNKZegTG3aCIBTUvNOuMpcQF/mNHChud1MZIXWsH2+uCmMukcesy 3aexF06tWT3nkv2Wia0G7PWiJZbTJGjFkRE+GL8mYf7lBT40C6fvQEJnOti3u8k/A7LA mMjS0fEQfOGs2cWM0TrBsW8Jjm3ztLKj/pBK7c0kaVpgKfM0wkt/kUAZYIoczIz4fEbB wKjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133465; x=1726738265; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F4c0vwBBm2GJGPzUzcwaUmJw/5o8B/sn9ihY/AAAuUE=; b=uuCLxz3z3NWlydtw9QK/m2ptZGkPcR7f1Gw57VYXwCDUe4v1etm4Pn4qS29tOnSMsw k46H9go/YFtSWfNbiYcumbeAWeDYE+vRH+9V185MBT32Kuy+QSMyZ7UlUhHFzaI8pYjx aywGmNTMFABY0X4IFVVcldTUUKj4BYqEOQ6C6ccWkt9uj+MHJV+u2TepVxQLwFWbr0EB cmMfPzrRQxk0tHBOJN4UPwDZ9mnGGO4DlYOPvPcxW6MFLFK/9LGTelJihkjZNZ5nqQR9 BKok0mRUYdIdhvWD79QCOMCUdUfPzfSjk8dmME7qphgCj8X69FJn76mLdMTKf2xbJ77X fvKA== X-Forwarded-Encrypted: i=1; AJvYcCVQNAh4TNXaoxllog/eQcTTzmjVk1KxO0T7v/ULuYSrbbGZTZ0yt+L90T+9qq1J6reZB7ndK+A00tPcHRY=@vger.kernel.org, AJvYcCX0nb/io019T2Mrz8j1AV13jUWrq7JC/U4Q0Pf2DLbYxFSN6QL5PPGHN1KcuQQ7RDBJtWDxd97tmHaEeg==@vger.kernel.org X-Gm-Message-State: AOJu0YwivrD9Z1a5MnZLsANEP87+w+X3zf8DUzpZ/h968IhHeLZgaBO6 U6rwe2wR96Kk82T/8TLDXW4YzVbex50QjWBO1oppknPWe0XWi6aT X-Google-Smtp-Source: AGHT+IETxzB6XSkf03aqTR6MKpcJw3H5lcYHSpTvlgNQRQGSeEtIMbH3IuLg90T5YLsmvSfxPCGrkA== X-Received: by 2002:a05:600c:190d:b0:42c:ae4e:a990 with SMTP id 5b1f17b1804b1-42cdb5928f1mr14721225e9.35.1726133464911; Thu, 12 Sep 2024 02:31:04 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.31.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:31:04 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 5/9] clocksource: mips-gic-timer: Always use cluster 0 counter as clocksource Date: Thu, 12 Sep 2024 11:30:47 +0200 Message-Id: <20240912093051.452172-6-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton In a multi-cluster MIPS system, there are multiple GICs - one in each cluster - each of which has its independent counter. The counters in each GIC are not synchronized in any way, so they can drift relative to one another through the lifetime of the system. This is problematic for a clock source which ought to be global. Avoid problems by always accessing cluster 0's counter, using cross-cluster register access. This adds overhead so it is applied only on multi-cluster systems. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- drivers/clocksource/mips-gic-timer.c | 39 +++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c index 110347707ff9..7907b740497a 100644 --- a/drivers/clocksource/mips-gic-timer.c +++ b/drivers/clocksource/mips-gic-timer.c @@ -166,6 +166,37 @@ static u64 gic_hpt_read(struct clocksource *cs) return gic_read_count(); } +static u64 gic_hpt_read_multicluster(struct clocksource *cs) +{ + unsigned int hi, hi2, lo; + u64 count; + + mips_cm_lock_other(0, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + if (mips_cm_is64) { + count = read_gic_redir_counter(); + goto out; + } + + hi = read_gic_redir_counter_32h(); + while (true) { + lo = read_gic_redir_counter_32l(); + + /* If hi didn't change then lo didn't wrap & we're done */ + hi2 = read_gic_redir_counter_32h(); + if (hi2 == hi) + break; + + /* Otherwise, repeat with the latest hi value */ + hi = hi2; + } + + count = (((u64)hi) << 32) + lo; +out: + mips_cm_unlock_other(); + return count; +} + static struct clocksource gic_clocksource = { .name = "GIC", .read = gic_hpt_read, @@ -203,6 +234,11 @@ static int __init __gic_clocksource_init(void) gic_clocksource.rating = 200; gic_clocksource.rating += clamp(gic_frequency / 10000000, 0, 99); + if (mips_cps_multicluster_cpus()) { + gic_clocksource.read = &gic_hpt_read_multicluster; + gic_clocksource.vdso_clock_mode = VDSO_CLOCKMODE_NONE; + } + ret = clocksource_register_hz(&gic_clocksource, gic_frequency); if (ret < 0) pr_warn("Unable to register clocksource\n"); @@ -261,7 +297,8 @@ static int __init gic_clocksource_of_init(struct device_node *node) * stable CPU frequency or on the platforms with CM3 and CPU frequency * change performed by the CPC core clocks divider. */ - if (mips_cm_revision() >= CM_REV_CM3 || !IS_ENABLED(CONFIG_CPU_FREQ)) { + if ((mips_cm_revision() >= CM_REV_CM3 || !IS_ENABLED(CONFIG_CPU_FREQ)) && + !mips_cps_multicluster_cpus()) { sched_clock_register(mips_cm_is64 ? gic_read_count_64 : gic_read_count_2x32, gic_count_width, gic_frequency); From patchwork Thu Sep 12 09:30:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801808 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A43C1A0BF0; Thu, 12 Sep 2024 09:31:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133470; cv=none; b=ar8ZREvu6UnwexjhscJ+uL6oUpKdtBQrkrn5GYHVWmiEOhqWnnm3nBATrpAqsySvB6JNLiY4MigUASm2D7a/r5cApDhHMkd/O8xnFVNwUG24XSkQDMQ6HcYz2o9iNV7mx3/BUpBaxmdyR0lB0ZtllGYpBOR0N3htnxte5xgdQLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133470; c=relaxed/simple; bh=V5rAjTu+pU2fpi74fQ5x5suB+2X7eoYvm+lLjy+3z2o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=MHCkOAG/FDYVWyZ12S1xVKz1/z365pxv5Bjb3dlgFPool5bTkH+cY4SCPloGtnqH5hSLMUzY+TVeFytkOC9nyBUsGZ/lSo0UDjxVkwgSk+tE/aMJ0eeqi7JW02qJ6/oG5b9hcYsXPRsTEHvoNK/kSEHx+Pb+7gtGmb+WjyZGm80= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SoMJ9zrh; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SoMJ9zrh" Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-42cb5b3c57eso6573115e9.2; Thu, 12 Sep 2024 02:31:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133467; x=1726738267; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1TY2C2Lxi2lZ2EXUmAkUJgOoy7I3aUdzu6U8J4ynR6M=; b=SoMJ9zrh7s2GkZt96BmtxihxH8fSEu8QhoIWR9HnpHP5olvketDLezWTo9avjpm2Kd iFwxfhSQ5dINYMlftI2Ez0d6N36RkPIzNMnsrS1BYcuDHXjjkMzjKfJRpTFGuSirdzw8 CjxpNIfccyVZkfuz1l/D9DzpSm2yrfLnvmjV0/vrwO7Ar6h9Q8+vD/egKM2oyFtef3rm IjBLTGSxRI+if1C34u5StD3vL1QzfPVY2+3A62H2jWS8E4x7pz86eQKqUgD/HpPanPaD 6UP7+8yRtdxemMMtu9n0OELeRFHbbqDv6bHrPbothpzFMbep3VJCqkb49ca+//0U2jgt 38oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133467; x=1726738267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1TY2C2Lxi2lZ2EXUmAkUJgOoy7I3aUdzu6U8J4ynR6M=; b=HqYWBYZFU2ZfghpCwYeK1n9JRNr+q0vPvweOpXgNDHiH8iB5NUWXb2jJ4cJApT5/wR 5ACNwNhnM00xHwZs4njVh8YAYWr7lILuyI7+bJ1tQMXNOa0Qt78LHRIy0Em4UTIgW0pm Kv7ScpmxhtrxcNAljXbIWG9TsBXSvmeydl6+yg9FERPSyWl95kxsKlpVhjYXtwP6IV+F 9Hq3484D+LsTSLz3yXf2SLrGN1NjC3T1x7C3kpeiekaSn81BUliG4XdqSpKnY8dHHkMs nkBVyCsVKjG+QTx6HV4VRGju53zcwdiX8cCbn4M6iGSqj642BIkEtYhUwA9FIje8B0k+ vAOg== X-Forwarded-Encrypted: i=1; AJvYcCVMxwqloJcGtpaMlNOGqkDXtsoVfOjK5akz74Sq9dpeMPgtr4Op/X56QfbTouhgjoFLY5RDa6ssZxTBHg==@vger.kernel.org, AJvYcCVdp9z5WoLCN/r6R/o7Xxp/Lr2aRDhEH0/U1jv+WO4TczUhqSD1H1Nsf0bXI3O8JAbSPowR4M5QMB6M++A=@vger.kernel.org X-Gm-Message-State: AOJu0Yxiqmdf2iFGrS6gwQU5H/dShjj9mF1iyhzgv7ol/QSJmAOegbpt A7i6cZ0Mb8kPmHq/8Bv+LCqaUicsLYrsTLN3V174hpGxu9ldgkZ3 X-Google-Smtp-Source: AGHT+IFcfu6FeILuRdzD5gxxJGCrEZBwvLP8LJ8bhXKkKveZtbZBTGuRwuN3Pp5UbaIt6IZlxErncA== X-Received: by 2002:a05:600c:3505:b0:42c:a8d5:2df5 with SMTP id 5b1f17b1804b1-42cdb586ee0mr16540935e9.24.1726133466823; Thu, 12 Sep 2024 02:31:06 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.31.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:31:06 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 6/9] clocksource: mips-gic-timer: Enable counter when CPUs start Date: Thu, 12 Sep 2024 11:30:48 +0200 Message-Id: <20240912093051.452172-7-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton In multi-cluster MIPS I6500 systems there is a GIC in each cluster, each with its own counter. When a cluster powers up the counter will be stopped, with the COUNTSTOP bit set in the GIC_CONFIG register. In single cluster systems, it has been fine to clear COUNTSTOP once in gic_clocksource_of_init() to start the counter. In multi-cluster systems, this will only have started the counter in the boot cluster, and any CPUs in other clusters will find their counter stopped which will break the GIC clock_event_device. Resolve this by having CPUs clear the COUNTSTOP bit when they come online, using the existing gic_starting_cpu() CPU hotplug callback. This will allow CPUs in secondary clusters to ensure that the cluster's GIC counter is running as expected. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Reviewed-by: Philippe Mathieu-Daudé Tested-by: Serge Semin --- drivers/clocksource/mips-gic-timer.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c index 7907b740497a..abb685a080a5 100644 --- a/drivers/clocksource/mips-gic-timer.c +++ b/drivers/clocksource/mips-gic-timer.c @@ -115,6 +115,9 @@ static void gic_update_frequency(void *data) static int gic_starting_cpu(unsigned int cpu) { + /* Ensure the GIC counter is running */ + clear_gic_config(GIC_CONFIG_COUNTSTOP); + gic_clockevent_cpu_init(cpu, this_cpu_ptr(&gic_clockevent_device)); return 0; } @@ -288,9 +291,6 @@ static int __init gic_clocksource_of_init(struct device_node *node) pr_warn("Unable to register clock notifier\n"); } - /* And finally start the counter */ - clear_gic_config(GIC_CONFIG_COUNTSTOP); - /* * It's safe to use the MIPS GIC timer as a sched clock source only if * its ticks are stable, which is true on either the platforms with From patchwork Thu Sep 12 09:30:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801809 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3C331A2629; Thu, 12 Sep 2024 09:31:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133472; cv=none; b=ShuY8pKle0Q79XiKGE9r1GCWWbCWfwPGskoKUi3lSx07VNC9XJOe2gpk6mguuiBl3CCPhKa2v+lpuovcUn3MqwNo3A3tNzBzTprHKXIn1JFHuwt+zgg2UxcKZD5yRlPDJuvh9Qvsx/Pks8CWP37N+rENNxVKa2rEs2A6CmkA1bE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133472; c=relaxed/simple; bh=QKto5A9MIMK/IEg/kWynx+UpUoTjf00hu+AW07gvz90=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=boi/sgICyDnEdOGUyjyKEgPoV+gt61xM1PX5xVAXK7145YPvhb9Tk8rhm6B8CZZ+rqkyhOyPikEsQ2PjKhtrv6wIXI8POTFKF5Gh413GjyYkMQXPPHLq6g2957GKbPmyYDwmpikXAXrfuD9XF6lMpFxrZvVr/WydYlYk6jev1Ro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dHxvXhFz; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dHxvXhFz" Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-42cb1e623d1so6861195e9.0; Thu, 12 Sep 2024 02:31:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133469; x=1726738269; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5Wui8yC11QzS3zuL6au0E0B/rgxgmkQnWbsEcia4kwc=; b=dHxvXhFzPyOPhHn7XgXThDBU/osrJRzY3uMW/+aJrDY/iwqtuGX/2U+BdCt1WvfMpD LNLsS5WUsj4rl4Zsz34dW3GX0cYQxHfnMmkqHMK+cJuWTBLY8ua8W3teoMBRcCn+yQ+F ZHjlcBuDc+IISIiwobG8Kfyejk50TU2fmXR6/2xkV3Gq2gS8eXPCVba59SrG2OOesG2q 16559qxyFylDUDRqiOeWpwuLrpPuBpy/49u4nmCTVUo5ZNxx5jdBxPi/57sfv7Udd0Mq 1OIUv8f3lv2KeQX7Q72NzPcRjkEeVPauejCrF9bXNhhTneroFBe17U0jyGWyBUg/lT8V rgzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133469; x=1726738269; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5Wui8yC11QzS3zuL6au0E0B/rgxgmkQnWbsEcia4kwc=; b=OZx/0dVkqH2DWYFiJsLHpIlkAXegCkXjm1KvV7DU7KhoY3yl8c26tTtJFagAjeg838 CuKIYEYmAoQocw8QBUmBGlIRb1ndA8yHVa9Wg/kDxRfM0zbJBc4W+Qri1v8J9uosX+xe +QVpuMBMwmnrxktZcisaQKXet+WAtqfTY33FhxpbLwYEYbehzC5LWQotLz2rBxJiH/D/ Ns90EKkPp3iRSMqpPz5Od789jHseDRo2uRYpba5f5+WcsFv2efuHR2yvKGlPvF5Ww1DX DkguEQVCbgxXwUldNHQKywXS8ByG6TQIPe8nnDtZ+TsJtMtPAZQY6q/oDXgAVWT4QlZD 22Kg== X-Forwarded-Encrypted: i=1; AJvYcCV+IqRCRhzaUUJsvnV64daexcAg7YIPf/gfjsqlRus5U6OLSk99Dq/6C+ohA2PcLTwFFgpSDLD4bzFLnZE=@vger.kernel.org, AJvYcCVvNlqJJ/OzFsGdYiNVrsPd0+7CchS0U9E7Udg9uJYhO0NkA9PoTDUKrHjnvz8mClO7CeahZgLI3VNW+A==@vger.kernel.org X-Gm-Message-State: AOJu0YzV+nppZbNDtw6mz7oRHZkslVmxS0Oe65wN9M0018bhud2wsHlk xrUVNTJPaognuFIqP2GtkIxKqSsimWojKf5hAdtW0NkauayzX4ow X-Google-Smtp-Source: AGHT+IGp1wkVvQWdjaHcKTwqKUkmNIxMIKB8Y7zndoWBL1GCFcS3YugVMp+fB/uTdIsF04nJrJ0l5A== X-Received: by 2002:a05:600c:1c29:b0:42c:bd27:4be4 with SMTP id 5b1f17b1804b1-42cdb531e7emr17811645e9.8.1726133469021; Thu, 12 Sep 2024 02:31:09 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.31.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:31:08 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 7/9] MIPS: pm-cps: Use per-CPU variables as per-CPU, not per-core Date: Thu, 12 Sep 2024 11:30:49 +0200 Message-Id: <20240912093051.452172-8-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton The pm-cps code has up until now used per-CPU variables indexed by core, rather than CPU number, in order to share data amongst sibling CPUs (ie. VPs/threads in a core). This works fine for single cluster systems, but with multi-cluster systems a core number is no longer unique in the system, leading to sharing between CPUs that are not actually siblings. Avoid this issue by using per-CPU variables as they are more generally used - ie. access them using CPU numbers rather than core numbers. Sharing between siblings is then accomplished by: - Assigning the same pointer to entries for each sibling CPU for the nc_asm_enter & ready_count variables, which allow this by virtue of being per-CPU pointers. - Indexing by the first CPU set in a CPUs cpu_sibling_map in the case of pm_barrier, for which we can't use the previous approach because the per-CPU variable is not a pointer. Signed-off-by: Paul Burton Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/kernel/pm-cps.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index d09ca77e624d..9369a8dc385e 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -57,10 +57,7 @@ static DEFINE_PER_CPU_ALIGNED(u32*, ready_count); /* Indicates online CPUs coupled with the current CPU */ static DEFINE_PER_CPU_ALIGNED(cpumask_t, online_coupled); -/* - * Used to synchronize entry to deep idle states. Actually per-core rather - * than per-CPU. - */ +/* Used to synchronize entry to deep idle states */ static DEFINE_PER_CPU_ALIGNED(atomic_t, pm_barrier); /* Saved CPU state across the CPS_PM_POWER_GATED state */ @@ -112,9 +109,10 @@ int cps_pm_enter_state(enum cps_pm_state state) cps_nc_entry_fn entry; struct core_boot_config *core_cfg; struct vpe_boot_config *vpe_cfg; + atomic_t *barrier; /* Check that there is an entry function for this state */ - entry = per_cpu(nc_asm_enter, core)[state]; + entry = per_cpu(nc_asm_enter, cpu)[state]; if (!entry) return -EINVAL; @@ -150,7 +148,7 @@ int cps_pm_enter_state(enum cps_pm_state state) smp_mb__after_atomic(); /* Create a non-coherent mapping of the core ready_count */ - core_ready_count = per_cpu(ready_count, core); + core_ready_count = per_cpu(ready_count, cpu); nc_addr = kmap_noncoherent(virt_to_page(core_ready_count), (unsigned long)core_ready_count); nc_addr += ((unsigned long)core_ready_count & ~PAGE_MASK); @@ -158,7 +156,8 @@ int cps_pm_enter_state(enum cps_pm_state state) /* Ensure ready_count is zero-initialised before the assembly runs */ WRITE_ONCE(*nc_core_ready_count, 0); - coupled_barrier(&per_cpu(pm_barrier, core), online); + barrier = &per_cpu(pm_barrier, cpumask_first(&cpu_sibling_map[cpu])); + coupled_barrier(barrier, online); /* Run the generated entry code */ left = entry(online, nc_core_ready_count); @@ -629,12 +628,14 @@ static void *cps_gen_entry_code(unsigned cpu, enum cps_pm_state state) static int cps_pm_online_cpu(unsigned int cpu) { - enum cps_pm_state state; - unsigned core = cpu_core(&cpu_data[cpu]); + unsigned int sibling, core; void *entry_fn, *core_rc; + enum cps_pm_state state; + + core = cpu_core(&cpu_data[cpu]); for (state = CPS_PM_NC_WAIT; state < CPS_PM_STATE_COUNT; state++) { - if (per_cpu(nc_asm_enter, core)[state]) + if (per_cpu(nc_asm_enter, cpu)[state]) continue; if (!test_bit(state, state_support)) continue; @@ -646,16 +647,19 @@ static int cps_pm_online_cpu(unsigned int cpu) clear_bit(state, state_support); } - per_cpu(nc_asm_enter, core)[state] = entry_fn; + for_each_cpu(sibling, &cpu_sibling_map[cpu]) + per_cpu(nc_asm_enter, sibling)[state] = entry_fn; } - if (!per_cpu(ready_count, core)) { + if (!per_cpu(ready_count, cpu)) { core_rc = kmalloc(sizeof(u32), GFP_KERNEL); if (!core_rc) { pr_err("Failed allocate core %u ready_count\n", core); return -ENOMEM; } - per_cpu(ready_count, core) = core_rc; + + for_each_cpu(sibling, &cpu_sibling_map[cpu]) + per_cpu(ready_count, sibling) = core_rc; } return 0; From patchwork Thu Sep 12 09:30:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801810 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF5D01A2650; Thu, 12 Sep 2024 09:31:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133474; cv=none; b=W9gNKi/txA2OLp21l9Ysx1QtEtpkSrGMyB/XschyVsc6unaK6sP070REEqIUy2EnNoTzjznzt7wEZibdrdNBBsffszpf0gR6oboDVl5houhr/uWrgKppIqvHWEcHe1AV70QAezX5PgZPMQENNAr34dcTw7YD5+8tA/aVRU8ll5E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133474; c=relaxed/simple; bh=iEpl4bRLYAAHUaWD31rzMEtaXYAiMCWoJzhMYBOfe3A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=a56vtbUVYc+8TJUOnlsKBPgxtRgxuSwnZPb5Hcs+CHxfQAtkDZz6SfnDnLEsxkSes70RNp3XVwwHcPuSn6HpVQDgcjMHMNAFVLqCumC0keFiyZpQxtu4AX7KqP9WkFw6a3BMtMtodTj0ZiMMjhbsMb1cQtIllo1EIgjeGw3i5PU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IbG7oeNM; arc=none smtp.client-ip=209.85.128.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IbG7oeNM" Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-42ca4e0299eso5498455e9.2; Thu, 12 Sep 2024 02:31:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133471; x=1726738271; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HFSiDVa5A8xiC3Vbc7Dsg+Rdaig6Ej2jHPtoNOjMC84=; b=IbG7oeNME+byEFGYZni7ReTRNq4HFefhrSq6DydH/y4xFG2M2rspnqD9zS1LQivguV ISa3gE86jzddOiXEKSQs/UNi+jAF1LSgCjRf9bSY05/ymbT0wdPTWoZZWTliZ5WXtKjZ ZgBal8pq3wibzKtQH5d37uB9xqINy9b4KMB3ZTSyDnBZ+M9/I/hYhE0JaQRSajSNQK2g wY06U3jNXq+WB4AE13eig3tGWL93bc9sE33VoP6ZpEAhSfq/t8Pefhnp0JE92ge5xHqe pVYDavbAo8sXCwrurQ8BVqKasXlgz1BCG8r3Xycg+p3BJxKFmCRORxTPmN0OebJFDaHZ gNEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133471; x=1726738271; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HFSiDVa5A8xiC3Vbc7Dsg+Rdaig6Ej2jHPtoNOjMC84=; b=Le2LLE1gVCQlIqqhsU3tQYNJA2ccq2AHQJQ+1q8tE5wfCeMbe+sWlbW6TAUGTBX4MT qD6I/8DMDVDb0t4U9jvWRpqE2bPvyrZpDpj1pX2kIeItBsYv20WA1v5NTvXvJoNwk3+F scBWmBfbPAaxuWT7MYslRPYl2kEBreJlMirCJ2lmgulR/dor/r5Q0lL17uxdKVbaRJih ODudkxakYOMr2+wbgEvOGD0RTpjz7MVpaRthFS5GwE8Twcfdjq6Q0fOh+nZKezNv+KVg G2PGQKOq4Agi0vc6r/eoIEzz0nlU0SmTJIBUV0LF6kL3cAjypgUOLPWirEgTJOwrNA4Q xYiA== X-Forwarded-Encrypted: i=1; AJvYcCUvnZz4socJ6SzJXuy4eOAoW+oyfZ/kLdBpQlZTAARO4Ty1uDM04QrF+brtiRSt/CZ84X5NW6ex4wc1Rg==@vger.kernel.org, AJvYcCWPD6l/wBQPuE4wqp+FWMhkVEZ/svFHi1LruyEXQuOG5ct9V+V5Z7msMefUSMOXNhvQgSojBHueseg/WPQ=@vger.kernel.org X-Gm-Message-State: AOJu0Ywa3elTmdX3cwrvKfPwB2eytiow1T8RgyWmGb3ygSMqCgo8mvfz xtig7ECjrOI01Se610lFiTyvAqY6g1SxLWhB9TRyyZxhjbx6cD54 X-Google-Smtp-Source: AGHT+IHGuNuUY7yMXI11BDkP0q3VzvlK7W/t/gPVV20GhxBqHiMuEjvmUHAgQViKsMlFsXmrw02ySA== X-Received: by 2002:a05:600c:4fc2:b0:42b:af1c:66e with SMTP id 5b1f17b1804b1-42cdb5299e8mr17325085e9.9.1726133471016; Thu, 12 Sep 2024 02:31:11 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.31.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:31:10 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 8/9] MIPS: CPS: Introduce struct cluster_boot_config Date: Thu, 12 Sep 2024 11:30:50 +0200 Message-Id: <20240912093051.452172-9-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton In preparation for supporting multi-cluster systems, introduce a struct cluster_boot_config as an extra layer in the boot configuration maintained by the MIPS Coherent Processing System (CPS) SMP implementation. For now only one struct cluster_boot_config will be allocated & we'll simply defererence its core_config field to find the struct core_boot_config array which can be used to boot as usual. Signed-off-by: Paul Burton Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/include/asm/smp-cps.h | 6 ++- arch/mips/kernel/asm-offsets.c | 3 ++ arch/mips/kernel/cps-vec.S | 19 ++++++-- arch/mips/kernel/pm-cps.c | 5 +- arch/mips/kernel/smp-cps.c | 82 +++++++++++++++++++++------------ 5 files changed, 81 insertions(+), 34 deletions(-) diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cps.h index ab94e50f62b8..a629e948a6fd 100644 --- a/arch/mips/include/asm/smp-cps.h +++ b/arch/mips/include/asm/smp-cps.h @@ -22,7 +22,11 @@ struct core_boot_config { struct vpe_boot_config *vpe_config; }; -extern struct core_boot_config *mips_cps_core_bootcfg; +struct cluster_boot_config { + struct core_boot_config *core_config; +}; + +extern struct cluster_boot_config *mips_cps_cluster_bootcfg; extern void mips_cps_core_boot(int cca, void __iomem *gcr_base); extern void mips_cps_core_init(void); diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c index cb1045ebab06..b29944160b28 100644 --- a/arch/mips/kernel/asm-offsets.c +++ b/arch/mips/kernel/asm-offsets.c @@ -404,6 +404,9 @@ void output_cps_defines(void) { COMMENT(" MIPS CPS offsets. "); + OFFSET(CLUSTERBOOTCFG_CORECONFIG, cluster_boot_config, core_config); + DEFINE(CLUSTERBOOTCFG_SIZE, sizeof(struct cluster_boot_config)); + OFFSET(COREBOOTCFG_VPEMASK, core_boot_config, vpe_mask); OFFSET(COREBOOTCFG_VPECONFIG, core_boot_config, vpe_config); DEFINE(COREBOOTCFG_SIZE, sizeof(struct core_boot_config)); diff --git a/arch/mips/kernel/cps-vec.S b/arch/mips/kernel/cps-vec.S index f876309130ad..2ae7034a3d5c 100644 --- a/arch/mips/kernel/cps-vec.S +++ b/arch/mips/kernel/cps-vec.S @@ -19,6 +19,10 @@ #define GCR_CPC_BASE_OFS 0x0088 #define GCR_CL_COHERENCE_OFS 0x2008 #define GCR_CL_ID_OFS 0x2028 +#define CM3_GCR_Cx_ID_CLUSTER_SHF 8 +#define CM3_GCR_Cx_ID_CLUSTER_MSK (0xff << 8) +#define CM3_GCR_Cx_ID_CORENUM_SHF 0 +#define CM3_GCR_Cx_ID_CORENUM_MSK (0xff << 0) #define CPC_CL_VC_STOP_OFS 0x2020 #define CPC_CL_VC_RUN_OFS 0x2028 @@ -271,12 +275,21 @@ LEAF(mips_cps_core_init) */ LEAF(mips_cps_get_bootcfg) /* Calculate a pointer to this cores struct core_boot_config */ + PTR_LA v0, mips_cps_cluster_bootcfg + PTR_L v0, 0(v0) lw t0, GCR_CL_ID_OFS(s1) +#ifdef CONFIG_CPU_MIPSR6 + ext t1, t0, CM3_GCR_Cx_ID_CLUSTER_SHF, 8 + li t2, CLUSTERBOOTCFG_SIZE + mul t1, t1, t2 + PTR_ADDU \ + v0, v0, t1 +#endif + PTR_L v0, CLUSTERBOOTCFG_CORECONFIG(v0) + andi t0, t0, CM3_GCR_Cx_ID_CORENUM_MSK li t1, COREBOOTCFG_SIZE mul t0, t0, t1 - PTR_LA t1, mips_cps_core_bootcfg - PTR_L t1, 0(t1) - PTR_ADDU v0, t0, t1 + PTR_ADDU v0, v0, t0 /* Calculate this VPEs ID. If the core doesn't support MT use 0 */ li t9, 0 diff --git a/arch/mips/kernel/pm-cps.c b/arch/mips/kernel/pm-cps.c index 9369a8dc385e..3de0e05e0511 100644 --- a/arch/mips/kernel/pm-cps.c +++ b/arch/mips/kernel/pm-cps.c @@ -101,12 +101,14 @@ static void coupled_barrier(atomic_t *a, unsigned online) int cps_pm_enter_state(enum cps_pm_state state) { unsigned cpu = smp_processor_id(); + unsigned int cluster = cpu_cluster(¤t_cpu_data); unsigned core = cpu_core(¤t_cpu_data); unsigned online, left; cpumask_t *coupled_mask = this_cpu_ptr(&online_coupled); u32 *core_ready_count, *nc_core_ready_count; void *nc_addr; cps_nc_entry_fn entry; + struct cluster_boot_config *cluster_cfg; struct core_boot_config *core_cfg; struct vpe_boot_config *vpe_cfg; atomic_t *barrier; @@ -136,7 +138,8 @@ int cps_pm_enter_state(enum cps_pm_state state) if (!mips_cps_smp_in_use()) return -EINVAL; - core_cfg = &mips_cps_core_bootcfg[core]; + cluster_cfg = &mips_cps_cluster_bootcfg[cluster]; + core_cfg = &cluster_cfg->core_config[core]; vpe_cfg = &core_cfg->vpe_config[cpu_vpe_id(¤t_cpu_data)]; vpe_cfg->pc = (unsigned long)mips_cps_pm_restore; vpe_cfg->gp = (unsigned long)current_thread_info(); diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c index 395622c37325..f71e2bb58318 100644 --- a/arch/mips/kernel/smp-cps.c +++ b/arch/mips/kernel/smp-cps.c @@ -40,7 +40,7 @@ static DECLARE_BITMAP(core_power, NR_CPUS); static uint32_t core_entry_reg; static phys_addr_t cps_vec_pa; -struct core_boot_config *mips_cps_core_bootcfg; +struct cluster_boot_config *mips_cps_cluster_bootcfg; static unsigned __init core_vpe_count(unsigned int cluster, unsigned core) { @@ -212,8 +212,10 @@ static void __init cps_smp_setup(void) static void __init cps_prepare_cpus(unsigned int max_cpus) { - unsigned ncores, core_vpes, c, cca; + unsigned int nclusters, ncores, core_vpes, c, cl, cca; bool cca_unsuitable, cores_limited; + struct cluster_boot_config *cluster_bootcfg; + struct core_boot_config *core_bootcfg; mips_mt_set_cpuoptions(); @@ -255,40 +257,54 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) setup_cps_vecs(); - /* Allocate core boot configuration structs */ - ncores = mips_cps_numcores(0); - mips_cps_core_bootcfg = kcalloc(ncores, sizeof(*mips_cps_core_bootcfg), - GFP_KERNEL); - if (!mips_cps_core_bootcfg) { - pr_err("Failed to allocate boot config for %u cores\n", ncores); - goto err_out; - } + /* Allocate cluster boot configuration structs */ + nclusters = mips_cps_numclusters(); + mips_cps_cluster_bootcfg = kcalloc(nclusters, + sizeof(*mips_cps_cluster_bootcfg), + GFP_KERNEL); - /* Allocate VPE boot configuration structs */ - for (c = 0; c < ncores; c++) { - core_vpes = core_vpe_count(0, c); - mips_cps_core_bootcfg[c].vpe_config = kcalloc(core_vpes, - sizeof(*mips_cps_core_bootcfg[c].vpe_config), - GFP_KERNEL); - if (!mips_cps_core_bootcfg[c].vpe_config) { - pr_err("Failed to allocate %u VPE boot configs\n", - core_vpes); + for (cl = 0; cl < nclusters; cl++) { + /* Allocate core boot configuration structs */ + ncores = mips_cps_numcores(cl); + core_bootcfg = kcalloc(ncores, sizeof(*core_bootcfg), + GFP_KERNEL); + if (!core_bootcfg) goto err_out; + mips_cps_cluster_bootcfg[cl].core_config = core_bootcfg; + + /* Allocate VPE boot configuration structs */ + for (c = 0; c < ncores; c++) { + core_vpes = core_vpe_count(cl, c); + core_bootcfg[c].vpe_config = kcalloc(core_vpes, + sizeof(*core_bootcfg[c].vpe_config), + GFP_KERNEL); + if (!core_bootcfg[c].vpe_config) + goto err_out; } } /* Mark this CPU as booted */ - atomic_set(&mips_cps_core_bootcfg[cpu_core(¤t_cpu_data)].vpe_mask, - 1 << cpu_vpe_id(¤t_cpu_data)); + cl = cpu_cluster(¤t_cpu_data); + c = cpu_core(¤t_cpu_data); + cluster_bootcfg = &mips_cps_cluster_bootcfg[cl]; + core_bootcfg = &cluster_bootcfg->core_config[c]; + atomic_set(&core_bootcfg->vpe_mask, 1 << cpu_vpe_id(¤t_cpu_data)); return; err_out: /* Clean up allocations */ - if (mips_cps_core_bootcfg) { - for (c = 0; c < ncores; c++) - kfree(mips_cps_core_bootcfg[c].vpe_config); - kfree(mips_cps_core_bootcfg); - mips_cps_core_bootcfg = NULL; + if (mips_cps_cluster_bootcfg) { + for (cl = 0; cl < nclusters; cl++) { + cluster_bootcfg = &mips_cps_cluster_bootcfg[cl]; + ncores = mips_cps_numcores(cl); + for (c = 0; c < ncores; c++) { + core_bootcfg = &cluster_bootcfg->core_config[c]; + kfree(core_bootcfg->vpe_config); + } + kfree(mips_cps_cluster_bootcfg[c].core_config); + } + kfree(mips_cps_cluster_bootcfg); + mips_cps_cluster_bootcfg = NULL; } /* Effectively disable SMP by declaring CPUs not present */ @@ -376,17 +392,23 @@ static void boot_core(unsigned int core, unsigned int vpe_id) static void remote_vpe_boot(void *dummy) { + unsigned int cluster = cpu_cluster(¤t_cpu_data); unsigned core = cpu_core(¤t_cpu_data); - struct core_boot_config *core_cfg = &mips_cps_core_bootcfg[core]; + struct cluster_boot_config *cluster_cfg = + &mips_cps_cluster_bootcfg[cluster]; + struct core_boot_config *core_cfg = &cluster_cfg->core_config[core]; mips_cps_boot_vpes(core_cfg, cpu_vpe_id(¤t_cpu_data)); } static int cps_boot_secondary(int cpu, struct task_struct *idle) { + unsigned int cluster = cpu_cluster(&cpu_data[cpu]); unsigned core = cpu_core(&cpu_data[cpu]); unsigned vpe_id = cpu_vpe_id(&cpu_data[cpu]); - struct core_boot_config *core_cfg = &mips_cps_core_bootcfg[core]; + struct cluster_boot_config *cluster_cfg = + &mips_cps_cluster_bootcfg[cluster]; + struct core_boot_config *core_cfg = &cluster_cfg->core_config[core]; struct vpe_boot_config *vpe_cfg = &core_cfg->vpe_config[vpe_id]; unsigned int remote; int err; @@ -544,12 +566,14 @@ static void cps_kexec_nonboot_cpu(void) static int cps_cpu_disable(void) { unsigned cpu = smp_processor_id(); + struct cluster_boot_config *cluster_cfg; struct core_boot_config *core_cfg; if (!cps_pm_support_state(CPS_PM_POWER_GATED)) return -EINVAL; - core_cfg = &mips_cps_core_bootcfg[cpu_core(¤t_cpu_data)]; + cluster_cfg = &mips_cps_cluster_bootcfg[cpu_cluster(¤t_cpu_data)]; + core_cfg = &cluster_cfg->core_config[cpu_core(¤t_cpu_data)]; atomic_sub(1 << cpu_vpe_id(¤t_cpu_data), &core_cfg->vpe_mask); smp_mb__after_atomic(); set_cpu_online(cpu, false); From patchwork Thu Sep 12 09:30:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aleksandar Rikalo X-Patchwork-Id: 13801811 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D5881A2C01; Thu, 12 Sep 2024 09:31:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133477; cv=none; b=P0dk9HfzOKHLY8nd3fmhRKY2cB5466ZSoSxHwPvYzjBY3JraVoAHYAuR5Ewi2406rNTPDTj5QYcOg5P2C8Dye/BheUJHCYcN5V7/CUUaO/3bVprLblAUQwDqL3rMg71CPMgz5h9jRAS+0WOXNYk0unejW8gUS6arOBm5zSG6zss= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726133477; c=relaxed/simple; bh=C1UWvLBD18ktmGVl3uHYn1SGIWdmhfk3zyDVlPq5Qdo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JaqdghAZQ3WnlA3ZtXlUawFiSIELcCO4qm8VKv11801tqkqNhFJBEhmxI0mz9rfe56bOfz/a8Z8/pWFlvn4IlSDI02aF+/Y0dgHpe5Ej8sa48IXQW75XcyFwWWx5KwskPCilzs9SsBALhTHp1vLCL0smK8tFSF0pexfhK+JoM8M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MZC0apCY; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MZC0apCY" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-374c5bab490so1161554f8f.1; Thu, 12 Sep 2024 02:31:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726133474; x=1726738274; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GG/8+pJMCIxCghz+y+ZE8tT8LiRET7/nFHMIKLEQQWY=; b=MZC0apCY9mCXQEx5yTfnpbO7ofDNzt5sWTRi66P2NIBnO7S2e1y5073bqNnCvYw3Rp ojCQ8zKAvR2UZ4yEpSPShjevMPGEYKOACG7u7cg4WW+UlTFLmKLM54QsIXAxuw84k9LW tkLbJbylxkGNflaoYvLyiXK379Pp0Kd4sDjH2xdDCkxmSV0J1GWL/4mQHVNznQcOdiM3 QWooFnZ0Tj7RkfwAF9CcGtM4rpIbdUHJh6C112c2ZeG901UQTxyVn8KqGtL2YXl91MV9 e4xgprQAgGFgI0iovvnMtrLZoHji/cK7sSy/MX+7MWWiJ02GTiAjR3piZYqn6Lik5pZm hRHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726133474; x=1726738274; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GG/8+pJMCIxCghz+y+ZE8tT8LiRET7/nFHMIKLEQQWY=; b=EHMpH9A8IF2Glc629NmGFXk+HUxc+A5HABYBcfJyWct7BM0QhlzmryH+GpihTOezvY 8wWzMilr2J275mPrYsdw2o0ZUwQy8gDruMpbS6WeGkImmzGLSeNrLgiiqRE6Sww0sABm ZdssUfBDppfo8WJlLK2Jk+jVj0uQC4phrXGYmPGujopvkwDV597+yM+0F8Lh5diJougn 825vCMncPrspGdDUrdCyIef0zU5WfwwQyEV3WW2iwD4mKosjl6lrLGqWqVSQeJdzw9+C ToD+F7lL5lHeHosvb+zWdgzkpRWkcQbbJTDUdE6vYkAoKJXmzqZeNBVPF8NwyzTl5IVj sRBQ== X-Forwarded-Encrypted: i=1; AJvYcCWo8Hu2sseD0OHePK6Js1tlUMHZK22RBfrffKEY7cgTBhGGWMfKLz+FgjCztgICAaq2Anke6X+ak7/1w5g=@vger.kernel.org, AJvYcCXiOjeZB9mv1zdp0OaAQ7y1G4jp+YsQyfSS4ALFpf49SkgXHCwMlDbuF0VfgDjFvfIx3liYEeXBmmMDLA==@vger.kernel.org X-Gm-Message-State: AOJu0YxEdO95fQ3Fp8NpEfSKe1EbCQuJ8EUGVr1dIpZBgUs0tslWLkax 8FA/rGS8pUfNkeW+BoTh1v4vTcHEXM4lnBNd4obOk010NZFyErho X-Google-Smtp-Source: AGHT+IGtQuR6BNzzY1GeihTF2MV/FDLWHPyy5gHE0TEo6yycTBuLUcMsxsnsNReLBSpGqKSHITYKGw== X-Received: by 2002:a05:6000:bc7:b0:374:b6e4:16a7 with SMTP id ffacd0b85a97d-378a8a0b864mr5064262f8f.8.1726133473642; Thu, 12 Sep 2024 02:31:13 -0700 (PDT) Received: from localhost.localdomain ([212.200.182.192]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42cc137556esm76688225e9.1.2024.09.12.02.31.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Sep 2024 02:31:13 -0700 (PDT) From: Aleksandar Rikalo To: Thomas Bogendoerfer Cc: Aleksandar Rikalo , Chao-ying Fu , Daniel Lezcano , Geert Uytterhoeven , Greg Ungerer , Hauke Mehrtens , Ilya Lipnitskiy , Jiaxun Yang , linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, Marc Zyngier , Paul Burton , Peter Zijlstra , Serge Semin , Thomas Gleixner , Tiezhu Yang Subject: [PATCH v6 9/9] MIPS: CPS: Boot CPUs in secondary clusters Date: Thu, 12 Sep 2024 11:30:51 +0200 Message-Id: <20240912093051.452172-10-arikalo@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240912093051.452172-1-arikalo@gmail.com> References: <20240912093051.452172-1-arikalo@gmail.com> Precedence: bulk X-Mailing-List: linux-mips@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Paul Burton Probe for & boot CPUs (cores & VPs) in secondary clusters (ie. not the cluster that began booting Linux) when they are present in systems with CM 3.5 or higher. Signed-off-by: Paul Burton Signed-off-by: Chao-ying Fu Signed-off-by: Dragan Mladjenovic Signed-off-by: Aleksandar Rikalo Tested-by: Serge Semin --- arch/mips/include/asm/mips-cm.h | 18 +++ arch/mips/include/asm/smp-cps.h | 1 + arch/mips/kernel/mips-cm.c | 4 +- arch/mips/kernel/smp-cps.c | 205 ++++++++++++++++++++++++++++---- 4 files changed, 207 insertions(+), 21 deletions(-) diff --git a/arch/mips/include/asm/mips-cm.h b/arch/mips/include/asm/mips-cm.h index 1e782275850a..4d47163647dd 100644 --- a/arch/mips/include/asm/mips-cm.h +++ b/arch/mips/include/asm/mips-cm.h @@ -255,6 +255,12 @@ GCR_ACCESSOR_RW(32, 0x130, l2_config) GCR_ACCESSOR_RO(32, 0x150, sys_config2) #define CM_GCR_SYS_CONFIG2_MAXVPW GENMASK(3, 0) +/* GCR_L2-RAM_CONFIG - Configuration & status of L2 cache RAMs */ +GCR_ACCESSOR_RW(64, 0x240, l2_ram_config) +#define CM_GCR_L2_RAM_CONFIG_PRESENT BIT(31) +#define CM_GCR_L2_RAM_CONFIG_HCI_DONE BIT(30) +#define CM_GCR_L2_RAM_CONFIG_HCI_SUPPORTED BIT(29) + /* GCR_L2_PFT_CONTROL - Controls hardware L2 prefetching */ GCR_ACCESSOR_RW(32, 0x300, l2_pft_control) #define CM_GCR_L2_PFT_CONTROL_PAGEMASK GENMASK(31, 12) @@ -266,6 +272,18 @@ GCR_ACCESSOR_RW(32, 0x308, l2_pft_control_b) #define CM_GCR_L2_PFT_CONTROL_B_CEN BIT(8) #define CM_GCR_L2_PFT_CONTROL_B_PORTID GENMASK(7, 0) +/* GCR_L2_TAG_ADDR - Access addresses in L2 cache tags */ +GCR_ACCESSOR_RW(64, 0x600, l2_tag_addr) + +/* GCR_L2_TAG_STATE - Access L2 cache tag state */ +GCR_ACCESSOR_RW(64, 0x608, l2_tag_state) + +/* GCR_L2_DATA - Access data in L2 cache lines */ +GCR_ACCESSOR_RW(64, 0x610, l2_data) + +/* GCR_L2_ECC - Access ECC information from L2 cache lines */ +GCR_ACCESSOR_RW(64, 0x618, l2_ecc) + /* GCR_L2SM_COP - L2 cache op state machine control */ GCR_ACCESSOR_RW(32, 0x620, l2sm_cop) #define CM_GCR_L2SM_COP_PRESENT BIT(31) diff --git a/arch/mips/include/asm/smp-cps.h b/arch/mips/include/asm/smp-cps.h index a629e948a6fd..10d3ebd890cb 100644 --- a/arch/mips/include/asm/smp-cps.h +++ b/arch/mips/include/asm/smp-cps.h @@ -23,6 +23,7 @@ struct core_boot_config { }; struct cluster_boot_config { + unsigned long *core_power; struct core_boot_config *core_config; }; diff --git a/arch/mips/kernel/mips-cm.c b/arch/mips/kernel/mips-cm.c index 3eb2cfb893e1..9854bc2b6895 100644 --- a/arch/mips/kernel/mips-cm.c +++ b/arch/mips/kernel/mips-cm.c @@ -308,7 +308,9 @@ void mips_cm_lock_other(unsigned int cluster, unsigned int core, FIELD_PREP(CM3_GCR_Cx_OTHER_VP, vp); if (cm_rev >= CM_REV_CM3_5) { - val |= CM_GCR_Cx_OTHER_CLUSTER_EN; + if (cluster != cpu_cluster(¤t_cpu_data)) + val |= CM_GCR_Cx_OTHER_CLUSTER_EN; + val |= CM_GCR_Cx_OTHER_GIC_EN; val |= FIELD_PREP(CM_GCR_Cx_OTHER_CLUSTER, cluster); val |= FIELD_PREP(CM_GCR_Cx_OTHER_BLOCK, block); } else { diff --git a/arch/mips/kernel/smp-cps.c b/arch/mips/kernel/smp-cps.c index f71e2bb58318..4f344c890a23 100644 --- a/arch/mips/kernel/smp-cps.c +++ b/arch/mips/kernel/smp-cps.c @@ -36,12 +36,56 @@ enum label_id { UASM_L_LA(_not_nmi) -static DECLARE_BITMAP(core_power, NR_CPUS); static uint32_t core_entry_reg; static phys_addr_t cps_vec_pa; struct cluster_boot_config *mips_cps_cluster_bootcfg; +static void power_up_other_cluster(unsigned int cluster) +{ + u32 stat, seq_state; + unsigned int timeout; + + mips_cm_lock_other(cluster, CM_GCR_Cx_OTHER_CORE_CM, 0, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); + stat = read_cpc_co_stat_conf(); + mips_cm_unlock_other(); + + seq_state = stat & CPC_Cx_STAT_CONF_SEQSTATE; + seq_state >>= __ffs(CPC_Cx_STAT_CONF_SEQSTATE); + if (seq_state == CPC_Cx_STAT_CONF_SEQSTATE_U5) + return; + + /* Set endianness & power up the CM */ + mips_cm_lock_other(cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + write_cpc_redir_sys_config(IS_ENABLED(CONFIG_CPU_BIG_ENDIAN)); + write_cpc_redir_pwrup_ctl(1); + mips_cm_unlock_other(); + + /* Wait for the CM to start up */ + timeout = 1000; + mips_cm_lock_other(cluster, CM_GCR_Cx_OTHER_CORE_CM, 0, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); + while (1) { + stat = read_cpc_co_stat_conf(); + seq_state = stat & CPC_Cx_STAT_CONF_SEQSTATE; + seq_state >>= __ffs(CPC_Cx_STAT_CONF_SEQSTATE); + if (seq_state == CPC_Cx_STAT_CONF_SEQSTATE_U5) + break; + + if (timeout) { + mdelay(1); + timeout--; + } else { + pr_warn("Waiting for cluster %u CM to power up... STAT_CONF=0x%x\n", + cluster, stat); + mdelay(1000); + } + } + + mips_cm_unlock_other(); +} + static unsigned __init core_vpe_count(unsigned int cluster, unsigned core) { return min(smp_max_threads, mips_cps_numvps(cluster, core)); @@ -152,6 +196,9 @@ static void __init cps_smp_setup(void) pr_cont(","); pr_cont("{"); + if (mips_cm_revision() >= CM_REV_CM3_5) + power_up_other_cluster(cl); + ncores = mips_cps_numcores(cl); for (c = 0; c < ncores; c++) { core_vpes = core_vpe_count(cl, c); @@ -179,8 +226,8 @@ static void __init cps_smp_setup(void) /* Indicate present CPUs (CPU being synonymous with VPE) */ for (v = 0; v < min_t(unsigned, nvpes, NR_CPUS); v++) { - set_cpu_possible(v, cpu_cluster(&cpu_data[v]) == 0); - set_cpu_present(v, cpu_cluster(&cpu_data[v]) == 0); + set_cpu_possible(v, true); + set_cpu_present(v, true); __cpu_number_map[v] = v; __cpu_logical_map[v] = v; } @@ -188,9 +235,6 @@ static void __init cps_smp_setup(void) /* Set a coherent default CCA (CWB) */ change_c0_config(CONF_CM_CMASK, 0x5); - /* Core 0 is powered up (we're running on it) */ - bitmap_set(core_power, 0, 1); - /* Initialise core 0 */ mips_cps_core_init(); @@ -272,6 +316,10 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) goto err_out; mips_cps_cluster_bootcfg[cl].core_config = core_bootcfg; + mips_cps_cluster_bootcfg[cl].core_power = + kcalloc(BITS_TO_LONGS(ncores), sizeof(unsigned long), + GFP_KERNEL); + /* Allocate VPE boot configuration structs */ for (c = 0; c < ncores; c++) { core_vpes = core_vpe_count(cl, c); @@ -283,11 +331,12 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) } } - /* Mark this CPU as booted */ + /* Mark this CPU as powered up & booted */ cl = cpu_cluster(¤t_cpu_data); c = cpu_core(¤t_cpu_data); cluster_bootcfg = &mips_cps_cluster_bootcfg[cl]; core_bootcfg = &cluster_bootcfg->core_config[c]; + bitmap_set(cluster_bootcfg->core_power, cpu_core(¤t_cpu_data), 1); atomic_set(&core_bootcfg->vpe_mask, 1 << cpu_vpe_id(¤t_cpu_data)); return; @@ -315,13 +364,118 @@ static void __init cps_prepare_cpus(unsigned int max_cpus) } } -static void boot_core(unsigned int core, unsigned int vpe_id) +static void init_cluster_l2(void) { - u32 stat, seq_state; - unsigned timeout; + u32 l2_cfg, l2sm_cop, result; + + while (1) { + l2_cfg = read_gcr_redir_l2_ram_config(); + + /* If HCI is not supported, use the state machine below */ + if (!(l2_cfg & CM_GCR_L2_RAM_CONFIG_PRESENT)) + break; + if (!(l2_cfg & CM_GCR_L2_RAM_CONFIG_HCI_SUPPORTED)) + break; + + /* If the HCI_DONE bit is set, we're finished */ + if (l2_cfg & CM_GCR_L2_RAM_CONFIG_HCI_DONE) + return; + } + + l2sm_cop = read_gcr_redir_l2sm_cop(); + if (WARN(!(l2sm_cop & CM_GCR_L2SM_COP_PRESENT), + "L2 init not supported on this system yet")) + return; + + /* Clear L2 tag registers */ + write_gcr_redir_l2_tag_state(0); + write_gcr_redir_l2_ecc(0); + + /* Ensure the L2 tag writes complete before the state machine starts */ + mb(); + + /* Wait for the L2 state machine to be idle */ + do { + l2sm_cop = read_gcr_redir_l2sm_cop(); + } while (l2sm_cop & CM_GCR_L2SM_COP_RUNNING); + + /* Start a store tag operation */ + l2sm_cop = CM_GCR_L2SM_COP_TYPE_IDX_STORETAG; + l2sm_cop <<= __ffs(CM_GCR_L2SM_COP_TYPE); + l2sm_cop |= CM_GCR_L2SM_COP_CMD_START; + write_gcr_redir_l2sm_cop(l2sm_cop); + + /* Ensure the state machine starts before we poll for completion */ + mb(); + + /* Wait for the operation to be complete */ + do { + l2sm_cop = read_gcr_redir_l2sm_cop(); + result = l2sm_cop & CM_GCR_L2SM_COP_RESULT; + result >>= __ffs(CM_GCR_L2SM_COP_RESULT); + } while (!result); + + WARN(result != CM_GCR_L2SM_COP_RESULT_DONE_OK, + "L2 state machine failed cache init with error %u\n", result); +} + +static void boot_core(unsigned int cluster, unsigned int core, + unsigned int vpe_id) +{ + struct cluster_boot_config *cluster_cfg; + u32 access, stat, seq_state; + unsigned int timeout, ncores; + + cluster_cfg = &mips_cps_cluster_bootcfg[cluster]; + ncores = mips_cps_numcores(cluster); + + if ((cluster != cpu_cluster(¤t_cpu_data)) && + bitmap_empty(cluster_cfg->core_power, ncores)) { + power_up_other_cluster(cluster); + + mips_cm_lock_other(cluster, core, 0, + CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + /* Ensure cluster GCRs are where we expect */ + write_gcr_redir_base(read_gcr_base()); + write_gcr_redir_cpc_base(read_gcr_cpc_base()); + write_gcr_redir_gic_base(read_gcr_gic_base()); + + init_cluster_l2(); + + /* Mirror L2 configuration */ + write_gcr_redir_l2_only_sync_base(read_gcr_l2_only_sync_base()); + write_gcr_redir_l2_pft_control(read_gcr_l2_pft_control()); + write_gcr_redir_l2_pft_control_b(read_gcr_l2_pft_control_b()); + + /* Mirror ECC/parity setup */ + write_gcr_redir_err_control(read_gcr_err_control()); + + /* Set BEV base */ + write_gcr_redir_bev_base(core_entry_reg); + + mips_cm_unlock_other(); + } + + if (cluster != cpu_cluster(¤t_cpu_data)) { + mips_cm_lock_other(cluster, core, 0, + CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + + /* Ensure the core can access the GCRs */ + access = read_gcr_redir_access(); + access |= BIT(core); + write_gcr_redir_access(access); + + mips_cm_unlock_other(); + } else { + /* Ensure the core can access the GCRs */ + access = read_gcr_access(); + access |= BIT(core); + write_gcr_access(access); + } /* Select the appropriate core */ - mips_cm_lock_other(0, core, 0, CM_GCR_Cx_OTHER_BLOCK_LOCAL); + mips_cm_lock_other(cluster, core, 0, CM_GCR_Cx_OTHER_BLOCK_LOCAL); /* Set its reset vector */ write_gcr_co_reset_base(core_entry_reg); @@ -387,7 +541,17 @@ static void boot_core(unsigned int core, unsigned int vpe_id) mips_cm_unlock_other(); /* The core is now powered up */ - bitmap_set(core_power, core, 1); + bitmap_set(cluster_cfg->core_power, core, 1); + + /* + * Restore CM_PWRUP=0 so that the CM can power down if all the cores in + * the cluster do (eg. if they're all removed via hotplug. + */ + if (mips_cm_revision() >= CM_REV_CM3_5) { + mips_cm_lock_other(cluster, 0, 0, CM_GCR_Cx_OTHER_BLOCK_GLOBAL); + write_cpc_redir_pwrup_ctl(0); + mips_cm_unlock_other(); + } } static void remote_vpe_boot(void *dummy) @@ -413,10 +577,6 @@ static int cps_boot_secondary(int cpu, struct task_struct *idle) unsigned int remote; int err; - /* We don't yet support booting CPUs in other clusters */ - if (cpu_cluster(&cpu_data[cpu]) != cpu_cluster(&raw_current_cpu_data)) - return -ENOSYS; - vpe_cfg->pc = (unsigned long)&smp_bootstrap; vpe_cfg->sp = __KSTK_TOS(idle); vpe_cfg->gp = (unsigned long)task_thread_info(idle); @@ -425,14 +585,15 @@ static int cps_boot_secondary(int cpu, struct task_struct *idle) preempt_disable(); - if (!test_bit(core, core_power)) { + if (!test_bit(core, cluster_cfg->core_power)) { /* Boot a VPE on a powered down core */ - boot_core(core, vpe_id); + boot_core(cluster, core, vpe_id); goto out; } if (cpu_has_vp) { - mips_cm_lock_other(0, core, vpe_id, CM_GCR_Cx_OTHER_BLOCK_LOCAL); + mips_cm_lock_other(cluster, core, vpe_id, + CM_GCR_Cx_OTHER_BLOCK_LOCAL); write_gcr_co_reset_base(core_entry_reg); mips_cm_unlock_other(); } @@ -639,11 +800,15 @@ static void cps_cpu_die(unsigned int cpu) { } static void cps_cleanup_dead_cpu(unsigned cpu) { + unsigned int cluster = cpu_cluster(&cpu_data[cpu]); unsigned core = cpu_core(&cpu_data[cpu]); unsigned int vpe_id = cpu_vpe_id(&cpu_data[cpu]); ktime_t fail_time; unsigned stat; int err; + struct cluster_boot_config *cluster_cfg; + + cluster_cfg = &mips_cps_cluster_bootcfg[cluster]; /* * Now wait for the CPU to actually offline. Without doing this that @@ -695,7 +860,7 @@ static void cps_cleanup_dead_cpu(unsigned cpu) } while (1); /* Indicate the core is powered off */ - bitmap_clear(core_power, core, 1); + bitmap_clear(cluster_cfg->core_power, core, 1); } else if (cpu_has_mipsmt) { /* * Have a CPU with access to the offlined CPUs registers wait