From patchwork Tue Feb 4 07:53:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 13958753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EFE94C0218F for ; Tue, 4 Feb 2025 07:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DxWY7Ul6uBA+JZibfqvkkI14zABpBAm/jf7bUpt/zrA=; b=a00FhwP9W+0ZRf kCn5Awk8CPFSDcKY+QElRamarDHdexg4Op+1uMhew/MkSEdoVbXgC2zG1v2BTzh0F8nMhgSzzHR7u Sf//oai4qVkF5GPbXAPjW08vXrodP5A2PcGdP7fp9umwy1SH+xKr6ohn435uUKdsjRA1C6VDsi/XY BwW7xYiineNxJCDzmpNFdMs/pKGwNI5Njt4V8uQMwjsPBgmnYAUd/FMv/waEc6bTwCBp5RuYoh4ul 55qTiflQsiifle1VMzIBjNYAp6w5c0m/qMLMfLUFgFFND/o+IU7c+2yEZyNmjeHsLeqgAIcDxEz9M DE8JaYU3GBWQVfmUPL+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tfDnF-0000000HT9p-0xor; Tue, 04 Feb 2025 07:56:25 +0000 Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tfDlS-0000000HSdC-17Qs for linux-riscv@lists.infradead.org; Tue, 04 Feb 2025 07:54:35 +0000 Received: by mail-qk1-x730.google.com with SMTP id af79cd13be357-7b6e9586b82so465611885a.1 for ; Mon, 03 Feb 2025 23:54:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1738655673; x=1739260473; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CBfvQ3kWrJIo4ZX2fO574K2xNLHLI8xt5fD3UcZCgGE=; b=ZewnfPTku3rCxGEHONcgHgVdlOXBApzQmxEgCdc7X2pN4L+zzCTmgJ6A6DfxN66cnM xll4xX31GJj0G27vYQs/GWseBlJanFSR+d32AKE+hll0cAXTt0vDRzo1kpmWspSyslhA RTRvBJdeS0BNJ7nkpBvPhDak+dD4IGLRuZB8BjodsPkaj89vKG6V1+HjpRe1K29/bCMJ qpWVKM6Hw9eoaN3eNeHFwrD50SmwakTuCtMxCcijayu7BrVVVCJjhwSmYMH8pbfwTr2m dZz0uXa1rriyNloe7dVBMRkrPpk2uDh3oYb08KpCDo27TPIz68/Bijy6bVa9iqJFHHCu SflA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738655673; x=1739260473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CBfvQ3kWrJIo4ZX2fO574K2xNLHLI8xt5fD3UcZCgGE=; b=Ys0ubQzFDK/XfpQnk+jjnCy3ZR+I62k1dU19lPdW6iAXbUYNW5+KS20bDKH9z4LQyc oY4jHaBOydxPwFsLPJMMpEjAmvM7dTIqwOMt1XClWfp08Thh41xuHUu5AJmtYe+rQIxv VHMOBLhwci50kV/7dZ5T6fVorC2v51yr4n4idOapiCPAzphQewLmsCeFFh+NsrbENIxU 5OxSElnQSzVVhi83rWG48Ha7nR8ztAVMBs8Nuewfb1LKB7bL/84SFKJcMjsjtHniJrCC QizLefV+n1q6cJO0O8zOYpmxcJs7V+YujYjWyWNM7NIRlsKPQ+0efKwQvczGzAnPnVsO Fe5w== X-Forwarded-Encrypted: i=1; AJvYcCVL38/1lxjvPGihJul9xx2B2APTUyM55zFDtNohw+kJw2dIo/tWaFbxZmvM47Lc3R2KHcVgpp7MF5TfbQ==@lists.infradead.org X-Gm-Message-State: AOJu0YywEChUS7uKn/7/+cgrb3zHVblOHrTDJSL7VTYf/V87cvMclfYs 8zudENHY0jneRfsweGQ+pKGB8mSxsAX8umzSCoNtmaX+7vBTLeQD/tpufzGzMOQ= X-Gm-Gg: ASbGncs8kKiBsLOiisMPVuWpU0wQYxZUrZtC40jaGZKZiCiqgCCiGWNgxoLZVkPQVaL cSaMl/0JJOK4DyUJ4r9jvTaiMVs+BG8Xs/ib6JyK1pKYjWAi0A5CIQghOC/vDBr4fCkTDtz1Cmf gU1WSvnwghR93ljPXaK+/gyodcRooFFApaZlgS0L51pAwzr+3R6shkA6O5P0D0xI0q07YhMXtc+ Oaxeht5RDYJpuB1jfDhXq2qeR3u3KqIlp6ZDSCqoJ1mJrq5m5g0UsjIMNSpasaosXItGWj3K9Pb CmhL4c32PA3lQSCJgl6pFYlX0ixHyw7R01QZWD/iGYXrgN+J6bYwZ+g= X-Google-Smtp-Source: AGHT+IE1kHVojceFVzcf2jbW9XOHhUZiDggCrSxIzTiwYNG5RUoAlilnWj5uEbIRStQ4m23HHT45hg== X-Received: by 2002:a05:620a:4093:b0:7b6:d1e1:a22e with SMTP id af79cd13be357-7bffcd43a1cmr3525668485a.29.1738655673185; Mon, 03 Feb 2025 23:54:33 -0800 (PST) Received: from anup-ubuntu-vm.localdomain ([103.97.166.196]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7c00a8d05ddsm613373185a.39.2025.02.03.23.54.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Feb 2025 23:54:32 -0800 (PST) From: Anup Patel To: Thomas Gleixner Subject: [PATCH v3 01/10] irqchip/riscv-imsic: Handle non-atomic MSI updates for device Date: Tue, 4 Feb 2025 13:23:56 +0530 Message-ID: <20250204075405.824721-2-apatel@ventanamicro.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250204075405.824721-1-apatel@ventanamicro.com> References: <20250204075405.824721-1-apatel@ventanamicro.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250203_235434_308263_D2749E7D X-CRM114-Status: GOOD ( 20.74 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anup Patel , Andrew Lunn , imx@lists.linux.dev, Marc Zyngier , Sascha Hauer , Atish Patra , linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Palmer Dabbelt , Pengutronix Kernel Team , Paul Walmsley , Anup Patel , Andrew Jones , Shawn Guo , Gregory Clement , linux-arm-kernel@lists.infradead.org, Sebastian Hesselbarth Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Device having non-atomic MSI update might see an intermediate state when changing target IMSIC vector from one CPU to another. To handle such intermediate device state, update MSI address and MSI data through separate MSI writes to the device. Fixes: 027e125acdba ("irqchip/riscv-imsic: Add device MSI domain support for platform devices") Suggested-by: Thomas Gleixner Signed-off-by: Anup Patel --- drivers/irqchip/irq-riscv-imsic-early.c | 8 +++- drivers/irqchip/irq-riscv-imsic-platform.c | 27 ++++++++++++ drivers/irqchip/irq-riscv-imsic-state.c | 50 ++++++++++++++-------- drivers/irqchip/irq-riscv-imsic-state.h | 2 +- 4 files changed, 68 insertions(+), 19 deletions(-) diff --git a/drivers/irqchip/irq-riscv-imsic-early.c b/drivers/irqchip/irq-riscv-imsic-early.c index c5c2e6929a2f..73a93ce8668f 100644 --- a/drivers/irqchip/irq-riscv-imsic-early.c +++ b/drivers/irqchip/irq-riscv-imsic-early.c @@ -77,6 +77,12 @@ static void imsic_handle_irq(struct irq_desc *desc) struct imsic_vector *vec; unsigned long local_id; + /* + * First process pending IMSIC vector enable, disable and movement + * on the current CPU. + */ + imsic_local_sync_all(false); + chained_irq_enter(chip, desc); while ((local_id = csr_swap(CSR_TOPEI, 0))) { @@ -120,7 +126,7 @@ static int imsic_starting_cpu(unsigned int cpu) * Interrupts identities might have been enabled/disabled while * this CPU was not running so sync-up local enable/disable state. */ - imsic_local_sync_all(); + imsic_local_sync_all(true); /* Enable local interrupt delivery */ imsic_local_delivery(true); diff --git a/drivers/irqchip/irq-riscv-imsic-platform.c b/drivers/irqchip/irq-riscv-imsic-platform.c index c708780e8760..b44eb0b3990b 100644 --- a/drivers/irqchip/irq-riscv-imsic-platform.c +++ b/drivers/irqchip/irq-riscv-imsic-platform.c @@ -97,6 +97,7 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask { struct imsic_vector *old_vec, *new_vec; struct irq_data *pd = d->parent_data; + struct imsic_vector tmp_vec; old_vec = irq_data_get_irq_chip_data(pd); if (WARN_ON(!old_vec)) @@ -115,6 +116,32 @@ static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask if (!new_vec) return -ENOSPC; + /* + * Device having non-atomic MSI update might see an intermediate + * state when changing target IMSIC vector from one CPU to another. + * + * To avoid losing interrupt to some intermediate state, do the + * following (just like x86 APIC): + * + * 1) First write a temporary IMSIC vector to the device which + * has MSI address same as the old IMSIC vector but MSI data + * matches the new IMSIC vector. + * + * 2) Next write the new IMSIC vector to the device. + * + * Based on the above, the __imsic_local_sync() must check both + * old MSI data and new MSI data on the old CPU for pending + */ + + if (new_vec->local_id != old_vec->local_id) { + /* Setup temporary vector */ + tmp_vec.cpu = old_vec->cpu; + tmp_vec.local_id = new_vec->local_id; + + /* Point device to the temporary vector */ + imsic_msi_update_msg(d, &tmp_vec); + } + /* Point device to the new vector */ imsic_msi_update_msg(d, new_vec); diff --git a/drivers/irqchip/irq-riscv-imsic-state.c b/drivers/irqchip/irq-riscv-imsic-state.c index b97e6cd89ed7..a8645084bd8f 100644 --- a/drivers/irqchip/irq-riscv-imsic-state.c +++ b/drivers/irqchip/irq-riscv-imsic-state.c @@ -126,21 +126,21 @@ void __imsic_eix_update(unsigned long base_id, unsigned long num_id, bool pend, static void __imsic_local_sync(struct imsic_local_priv *lpriv) { - struct imsic_local_config *mlocal; - struct imsic_vector *vec, *mvec; + struct imsic_local_config *tlocal, *mlocal; + struct imsic_vector *vec, *tvec, *mvec; int i; lockdep_assert_held(&lpriv->lock); for_each_set_bit(i, lpriv->dirty_bitmap, imsic->global.nr_ids + 1) { if (!i || i == IMSIC_IPI_ID) - goto skip; + continue; vec = &lpriv->vectors[i]; if (READ_ONCE(vec->enable)) - __imsic_id_set_enable(i); + __imsic_id_set_enable(vec->local_id); else - __imsic_id_clear_enable(i); + __imsic_id_clear_enable(vec->local_id); /* * If the ID was being moved to a new ID on some other CPU @@ -151,26 +151,47 @@ static void __imsic_local_sync(struct imsic_local_priv *lpriv) mvec = READ_ONCE(vec->move); WRITE_ONCE(vec->move, NULL); if (mvec && mvec != vec) { - if (__imsic_id_read_clear_pending(i)) { + /* + * Device having non-atomic MSI update might see an + * intermediate state so check both old ID and new ID + * for pending interrupts. + * + * For details, refer imsic_irq_set_affinity(). + */ + + tvec = vec->local_id == mvec->local_id ? + NULL : &lpriv->vectors[mvec->local_id]; + if (tvec && __imsic_id_read_clear_pending(tvec->local_id)) { + /* Retrigger temporary vector if it was already in-use */ + if (READ_ONCE(tvec->enable)) { + tlocal = per_cpu_ptr(imsic->global.local, tvec->cpu); + writel_relaxed(tvec->local_id, tlocal->msi_va); + } + mlocal = per_cpu_ptr(imsic->global.local, mvec->cpu); writel_relaxed(mvec->local_id, mlocal->msi_va); } - imsic_vector_free(&lpriv->vectors[i]); + if (__imsic_id_read_clear_pending(vec->local_id)) { + mlocal = per_cpu_ptr(imsic->global.local, mvec->cpu); + writel_relaxed(mvec->local_id, mlocal->msi_va); + } + + imsic_vector_free(vec); } -skip: - bitmap_clear(lpriv->dirty_bitmap, i, 1); + bitmap_clear(lpriv->dirty_bitmap, vec->local_id, 1); } } -void imsic_local_sync_all(void) +void imsic_local_sync_all(bool force_all) { struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv); unsigned long flags; raw_spin_lock_irqsave(&lpriv->lock, flags); - bitmap_fill(lpriv->dirty_bitmap, imsic->global.nr_ids + 1); + if (force_all) + bitmap_fill(lpriv->dirty_bitmap, imsic->global.nr_ids + 1); __imsic_local_sync(lpriv); raw_spin_unlock_irqrestore(&lpriv->lock, flags); } @@ -190,12 +211,7 @@ void imsic_local_delivery(bool enable) #ifdef CONFIG_SMP static void imsic_local_timer_callback(struct timer_list *timer) { - struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv); - unsigned long flags; - - raw_spin_lock_irqsave(&lpriv->lock, flags); - __imsic_local_sync(lpriv); - raw_spin_unlock_irqrestore(&lpriv->lock, flags); + imsic_local_sync_all(false); } static void __imsic_remote_sync(struct imsic_local_priv *lpriv, unsigned int cpu) diff --git a/drivers/irqchip/irq-riscv-imsic-state.h b/drivers/irqchip/irq-riscv-imsic-state.h index 391e44280827..8fae6c99b019 100644 --- a/drivers/irqchip/irq-riscv-imsic-state.h +++ b/drivers/irqchip/irq-riscv-imsic-state.h @@ -74,7 +74,7 @@ static inline void __imsic_id_clear_enable(unsigned long id) __imsic_eix_update(id, 1, false, false); } -void imsic_local_sync_all(void); +void imsic_local_sync_all(bool force_all); void imsic_local_delivery(bool enable); void imsic_vector_mask(struct imsic_vector *vec);