From patchwork Mon Jun 10 14:20:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13692117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B68AC27C65 for ; Mon, 10 Jun 2024 14:21:10 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.737297.1143529 (Exim 4.92) (envelope-from ) id 1sGftG-00009Z-RR; Mon, 10 Jun 2024 14:20:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 737297.1143529; Mon, 10 Jun 2024 14:20:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftG-00009S-On; Mon, 10 Jun 2024 14:20:54 +0000 Received: by outflank-mailman (input) for mailman id 737297; Mon, 10 Jun 2024 14:20:52 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftE-00007Q-Oc for xen-devel@lists.xenproject.org; Mon, 10 Jun 2024 14:20:52 +0000 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [2607:f8b0:4864:20::733]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a35acadf-2734-11ef-90a2-e314d9c70b13; Mon, 10 Jun 2024 16:20:51 +0200 (CEST) Received: by mail-qk1-x733.google.com with SMTP id af79cd13be357-79767180a15so45375185a.1 for ; Mon, 10 Jun 2024 07:20:51 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id af79cd13be357-795758dfe3asm127231685a.105.2024.06.10.07.20.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:20:48 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a35acadf-2734-11ef-90a2-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718029249; x=1718634049; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EmeqCABNjDotan3Y72svsvPEn7gJhsjuiG6QHhCgPvQ=; b=YvJfA12OB3sFdIw1RRotS2fex80GJdQh/gtzshnOXTHsaqPhGu3KpYWccUbMOo06RZ PF5rKb5RGDdC4R61n3IL2MZ+CbMoEhWqU1tUmeDp0WKXudkN05o5AcNRKxLy2XKkIUgu NDEPXfdUUCg3Wp/fH5KSBhWYt6PY8AtfsLxbk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718029249; x=1718634049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EmeqCABNjDotan3Y72svsvPEn7gJhsjuiG6QHhCgPvQ=; b=iLMYJJPxJjepAG7OTjl31Q8V0IYwXt4W4nHBKbiTGytoZa//K4C8OFquGbEkJRK9WV TILdQTZdBdSWyb3IDSzaMm2dHF/JYJGS+1MohQNrgGab2Ouy+VyTRzu3CiccYH3tGAgo ovvtfemPIWurJnaQS6iqMF1tF88pxP6+WXVXAI3sHk/3ILl5wH51wj8AEZqOGWx8DkM/ mZRcnMku3Oc+6OI29P8AgBFusJiF/yu0tTck/yukhra39XrK+0Eyk4Ht5npAJaTfBLqC bHprzqWIY9DEer3qz0u8D2JIeQ0FaN7AqjauHL9ra+4mPKHz3HAXB6YU66Cq2Cwn4XxP EeyA== X-Gm-Message-State: AOJu0Yz6RmjOt/FSJ1cWLL+8QEc5IOn5MdHDiUe33UzynUu4dqc6udbn RiV2/zXd6Un9+TGqmbBt6db7GeIfCOCm9/wiE2WVcfjnz94Y8tkw6ILsOtqy7OdVtKK6z3wU6MM 5 X-Google-Smtp-Source: AGHT+IFVJ6qqqsqLC04Mqokc1ZPW7ysMrqYU9cIPAJV1KObi0//GhIoewHRe9csIUHpaXS5nGBBDKA== X-Received: by 2002:a05:620a:3913:b0:795:f319:e4ae with SMTP id af79cd13be357-795f319f448mr418663485a.26.1718029248808; Mon, 10 Jun 2024 07:20:48 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH v2 1/7] x86/smp: do not use shorthand IPI destinations in CPU hot{,un}plug contexts Date: Mon, 10 Jun 2024 16:20:37 +0200 Message-ID: <20240610142043.11924-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610142043.11924-1-roger.pau@citrix.com> References: <20240610142043.11924-1-roger.pau@citrix.com> MIME-Version: 1.0 Due to the current rwlock logic, if the CPU calling get_cpu_maps() does so from a cpu_hotplug_{begin,done}() region the function will still return success, because a CPU taking the rwlock in read mode after having taken it in write mode is allowed. Such behavior however defeats the purpose of get_cpu_maps(), as it should always return false when called with a CPU hot{,un}plug operation is in progress. Otherwise the logic in send_IPI_mask() is wrong, as it could decide to use the shorthand even when a CPU operation is in progress. Introduce a new helper to detect whether the current caller is between a cpu_hotplug_{begin,done}() region and use it in send_IPI_mask() to restrict shorthand usage. Fixes: 5500d265a2a8 ('x86/smp: use APIC ALLBUT destination shorthand when possible') Signed-off-by: Roger Pau Monné Reviewed-by: Jan Beulich --- Changes since v1: - Modify send_IPI_mask() to detect CPU hotplug context. --- xen/arch/x86/smp.c | 2 +- xen/common/cpu.c | 5 +++++ xen/include/xen/cpu.h | 10 ++++++++++ xen/include/xen/rwlock.h | 2 ++ 4 files changed, 18 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c index 7443ad20335e..04c6a0572319 100644 --- a/xen/arch/x86/smp.c +++ b/xen/arch/x86/smp.c @@ -88,7 +88,7 @@ void send_IPI_mask(const cpumask_t *mask, int vector) * the system have been accounted for. */ if ( system_state > SYS_STATE_smp_boot && - !unaccounted_cpus && !disabled_cpus && + !unaccounted_cpus && !disabled_cpus && !cpu_in_hotplug_context() && /* NB: get_cpu_maps lock requires enabled interrupts. */ local_irq_is_enabled() && (cpus_locked = get_cpu_maps()) && (park_offline_cpus || diff --git a/xen/common/cpu.c b/xen/common/cpu.c index 8709db4d2957..6e35b114c080 100644 --- a/xen/common/cpu.c +++ b/xen/common/cpu.c @@ -68,6 +68,11 @@ void cpu_hotplug_done(void) write_unlock(&cpu_add_remove_lock); } +bool cpu_in_hotplug_context(void) +{ + return rw_is_write_locked_by_me(&cpu_add_remove_lock); +} + static NOTIFIER_HEAD(cpu_chain); void __init register_cpu_notifier(struct notifier_block *nb) diff --git a/xen/include/xen/cpu.h b/xen/include/xen/cpu.h index e1d4eb59675c..6bf578675008 100644 --- a/xen/include/xen/cpu.h +++ b/xen/include/xen/cpu.h @@ -13,6 +13,16 @@ void put_cpu_maps(void); void cpu_hotplug_begin(void); void cpu_hotplug_done(void); +/* + * Returns true when the caller CPU is between a cpu_hotplug_{begin,done}() + * region. + * + * This is required to safely identify hotplug contexts, as get_cpu_maps() + * would otherwise succeed because a caller holding the lock in write mode is + * allowed to acquire the same lock in read mode. + */ +bool cpu_in_hotplug_context(void); + /* Receive notification of CPU hotplug events. */ void register_cpu_notifier(struct notifier_block *nb); diff --git a/xen/include/xen/rwlock.h b/xen/include/xen/rwlock.h index a2e98cad343e..4e7802821859 100644 --- a/xen/include/xen/rwlock.h +++ b/xen/include/xen/rwlock.h @@ -316,6 +316,8 @@ static always_inline void write_lock_irq(rwlock_t *l) #define rw_is_locked(l) _rw_is_locked(l) #define rw_is_write_locked(l) _rw_is_write_locked(l) +#define rw_is_write_locked_by_me(l) \ + lock_evaluate_nospec(_is_write_locked_by_me(atomic_read(&(l)->cnts))) typedef struct percpu_rwlock percpu_rwlock_t; From patchwork Mon Jun 10 14:20:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13692114 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D43DC27C5E for ; Mon, 10 Jun 2024 14:21:06 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.737298.1143537 (Exim 4.92) (envelope-from ) id 1sGftH-0000D9-6p; Mon, 10 Jun 2024 14:20:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 737298.1143537; Mon, 10 Jun 2024 14:20:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftG-0000CU-Vc; Mon, 10 Jun 2024 14:20:54 +0000 Received: by outflank-mailman (input) for mailman id 737298; Mon, 10 Jun 2024 14:20:53 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftF-00007Q-DN for xen-devel@lists.xenproject.org; Mon, 10 Jun 2024 14:20:53 +0000 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [2607:f8b0:4864:20::1135]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a4542401-2734-11ef-90a2-e314d9c70b13; Mon, 10 Jun 2024 16:20:52 +0200 (CEST) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-62cf4d32dceso296927b3.1 for ; Mon, 10 Jun 2024 07:20:52 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b07b4b999csm12151576d6.63.2024.06.10.07.20.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:20:50 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a4542401-2734-11ef-90a2-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718029251; x=1718634051; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DO8Ho59BP8Cja4wQL980lT7rQ+BqnpHm8XHoOOnAnmY=; b=BgIP2REMw6cxRnkppT294rIeiTMVH57PORLjt6+oxx5ictq+LSLyFEr+g8rlq+MvME RsMOIYregntOZMHwtjgMIBq24z7otUeL9K+PDWOOPNX+U3Z68Y4cRfcMkhLImCAW5INw 1zmFxCC0KcrOVtsFmsu+FPeFwJNdS2DKnhR/w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718029251; x=1718634051; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DO8Ho59BP8Cja4wQL980lT7rQ+BqnpHm8XHoOOnAnmY=; b=NEyOjF/nOa78oNfx+amOXQak2lWnKH+eD+IL9AI/3uKm/Xwwom7pTCgMVZw222QUD9 zQ6xhZVKhhN1iW4V3ujl3Kyp6+4wGotq3gHpndZsTe5ahDEmP+c+6/etsiuMysyoTFR+ g8QyXF/3kH1g1JcQHwPoo7Dq0BVlm53kc7nJcF1qwPeA/3kZhYKHw5Tgt7uQMxbPx2CQ XsyZPXDoBvs2lgPSyvKV3FMeyW7hUOX6nANC+jAAGtAdNfbveqf6frK+15kLRTPj8GXn 4ykHYMzuYSqm0vocs+wkwY8xNdgGFSU1mKwdP2Gf9rjydYApTYMNyFquynV66XhlNMpw TGgw== X-Gm-Message-State: AOJu0YypvDd6RBsaHZ5aw7u9pJwOqQKTHndjbtwPKYLI4hRkjfNchcmk fnIcJ7pBETNJmnA4I9DG4HdELbqmAqWvIHZOYDqUoaf5dq/Zd5JzerLVf3IjcPQJeceqbyE7rV7 u X-Google-Smtp-Source: AGHT+IEoMYN3SXv7692z9fWNkS/rCmIyG+AjwRjuRyQB6tQ2K4ivPZvs/F+EjKLE4LDTVnXU8Z9psw== X-Received: by 2002:a0d:cb0a:0:b0:61e:a3a:2538 with SMTP id 00721157ae682-62cd55d157emr87669307b3.18.1718029251068; Mon, 10 Jun 2024 07:20:51 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v2 2/7] x86/irq: describe how the interrupt CPU movement works Date: Mon, 10 Jun 2024 16:20:38 +0200 Message-ID: <20240610142043.11924-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610142043.11924-1-roger.pau@citrix.com> References: <20240610142043.11924-1-roger.pau@citrix.com> MIME-Version: 1.0 The logic to move interrupts across CPUs is complex, attempt to provide a comment that describes the expected behavior so users of the interrupt system have more context about the usage of the arch_irq_desc structure fields. Signed-off-by: Roger Pau Monné Reviewed-by: Jan Beulich --- Changes since v1: - Mention the logic involved in IRQ_MOVE_PENDING and the reduction of old_cpu_mask. --- xen/arch/x86/include/asm/irq.h | 38 ++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h index 413994d2133b..94f634ce10a1 100644 --- a/xen/arch/x86/include/asm/irq.h +++ b/xen/arch/x86/include/asm/irq.h @@ -28,6 +28,44 @@ typedef struct { struct irq_desc; +/* + * Xen logic for moving interrupts around CPUs allows manipulating interrupts + * that target remote CPUs. The logic to move an interrupt from CPU(s) is as + * follows: + * + * 1. irq_set_affinity() is called with the new destination mask, such mask is + * copied into pending_mask and IRQ_MOVE_PENDING is set in status to notice + * an affinity change has been requested. + * 2. An interrupt acked with the IRQ_MOVE_PENDING will trigger the logic to + * migrate it to a destination in pending_mask as long as the mask contains + * any online CPUs. + * 3. cpu_mask and vector is copied to old_cpu_mask and old_vector. + * 4. New cpu_mask and vector are set, vector is setup at the new destination. + * 5. move_in_progress is set. + * 6. Interrupt source is updated to target new CPU and vector. + * 7. Interrupts arriving at old_cpu_mask are processed normally. + * 8. When the first interrupt is delivered at the new destination (cpu_mask) as + * part of acking the interrupt the cleanup of the old destination(s) is + * engaged. move_in_progress is cleared and old_cpu_mask is + * reduced to the online CPUs. If the result is empty the old vector is + * released. Otherwise move_cleanup_count is set to the weight of online + * CPUs in old_cpu_mask and IRQ_MOVE_CLEANUP_VECTOR is sent to them. + * 9. When receiving IRQ_MOVE_CLEANUP_VECTOR CPUs in old_cpu_mask clean the + * vector entry and decrease the count in move_cleanup_count. The CPU that + * sets move_cleanup_count to 0 releases the vector. + * + * Note that when interrupt movement (either move_in_progress or + * move_cleanup_count set) is in progress it's not possible to move the + * interrupt to yet a different CPU. + * + * Interrupt movements done by fixup_irqs() skip setting IRQ_MOVE_PENDING and + * pending_mask as the movement must be performed right away, and so start + * directly from step 3. + * + * By keeping the vector in the old CPU(s) configured until the interrupt is + * acked on the new destination Xen allows draining any pending interrupts at + * the old destinations. + */ struct arch_irq_desc { s16 vector; /* vector itself is only 8 bits, */ s16 old_vector; /* but we use -1 for unassigned */ From patchwork Mon Jun 10 14:20:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13692116 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6A46C27C5E for ; Mon, 10 Jun 2024 14:21:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.737299.1143550 (Exim 4.92) (envelope-from ) id 1sGftJ-0000fL-B5; Mon, 10 Jun 2024 14:20:57 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 737299.1143550; Mon, 10 Jun 2024 14:20:57 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftJ-0000fB-7e; Mon, 10 Jun 2024 14:20:57 +0000 Received: by outflank-mailman (input) for mailman id 737299; Mon, 10 Jun 2024 14:20:56 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftI-0008MI-BQ for xen-devel@lists.xenproject.org; Mon, 10 Jun 2024 14:20:56 +0000 Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [2607:f8b0:4864:20::82e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a585f964-2734-11ef-b4bb-af5377834399; Mon, 10 Jun 2024 16:20:54 +0200 (CEST) Received: by mail-qt1-x82e.google.com with SMTP id d75a77b69052e-43fdb114e07so19620621cf.2 for ; Mon, 10 Jun 2024 07:20:54 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-440c9fed96asm10638341cf.17.2024.06.10.07.20.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:20:52 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a585f964-2734-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718029253; x=1718634053; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xj/7AKm3tnuoONAQVj+xWnm7kpsb26VRcfyAbHMA6OE=; b=Cs7+gK0Blpb8nX19OWjwm8ImsRVYDnuiGp2kKz7Qiexg0LoHwLKk8Fb4nOougrMOSw VWVk80ITKKhF7m5xCTqdTk+6RYr4+e+GG7hGRy1dJhhmtUoSHkMzLCSAUQS1RFJvRbx3 nnw3BHpES43AeJFt5hr4/2K3jgn+AwV6ko3mc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718029253; x=1718634053; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xj/7AKm3tnuoONAQVj+xWnm7kpsb26VRcfyAbHMA6OE=; b=H58slk8sEYcoxJYNpS8KGFsNgVn9eyjCXyboyALtoN/USdVR+Aeqo5roXNlNubkvKw 0PlzhtdXOpcpKv52pAcPYDnmVuuFvd0bR9/L2C93kY7jADjqolhIZlUZNtwDSEyNzEii 65wB6cDtAtE+OsNc2PW8hELTOgOUBGzf6XrHozSud4fwgmiaJz6qPhfXGjhSGSkBB418 oufiG5KaXepdMyGhBfOdRlJPWuVr1ZgTWAY9rlvSBksKz9JSnv14lnxmQEReIYF+20NX EebvEFHIXZjQyWfvWQ+VcusKMkgcLsttLLrnOpqZLcUXRPMZbYbXjooMd8i4LTwM6HIU dteA== X-Gm-Message-State: AOJu0YyAok5BY+t47TCMHTL6lEiUgyVzYGZIBDPK1sVzmrDEuHDXYQqK 4FzDnCKoAzQlk/4Wh87eRl29trDeJNlI2JIRljqQab70NigHcmhoOO1DgFdWSeEOo21oyvKtUXG b X-Google-Smtp-Source: AGHT+IE7KOuxwbv+yqrmisaABSEWMWauOW82L28C76Zoz4VFVCV+s9AHsB515/AQkXeHu+w19TE3Rw== X-Received: by 2002:ac8:5703:0:b0:440:a22f:ac61 with SMTP id d75a77b69052e-440a22fadf0mr44014571cf.32.1718029253243; Mon, 10 Jun 2024 07:20:53 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v2 3/7] x86/irq: limit interrupt movement done by fixup_irqs() Date: Mon, 10 Jun 2024 16:20:39 +0200 Message-ID: <20240610142043.11924-4-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610142043.11924-1-roger.pau@citrix.com> References: <20240610142043.11924-1-roger.pau@citrix.com> MIME-Version: 1.0 The current check used in fixup_irqs() to decide whether to move around interrupts is based on the affinity mask, but such mask can have all bits set, and hence is unlikely to be a subset of the input mask. For example if an interrupt has an affinity mask of all 1s, any input to fixup_irqs() that's not an all set CPU mask would cause that interrupt to be shuffled around unconditionally. What fixup_irqs() care about is evacuating interrupts from CPUs not set on the input CPU mask, and for that purpose it should check whether the interrupt is assigned to a CPU not present in the input mask. Assume that ->arch.cpu_mask is a subset of the ->affinity mask, and keep the current logic that resets the ->affinity mask if the interrupt has to be shuffled around. Doing the affinity movement based on ->arch.cpu_mask requires removing the special handling to ->arch.cpu_mask done for high priority vectors, otherwise the adjustment done to cpu_mask makes them always skip the CPU interrupt movement. While there also adjust the comment as to the purpose of fixup_irqs(). Signed-off-by: Roger Pau Monné Reviewed-by: Jan Beulich --- Changes since v1: - Adjust handling of high priority vectors. Changes since v0 (outside the series): - Do not AND ->arch.cpu_mask with cpu_online_map. - Keep using cpumask_subset(). - Integrate into bigger series. --- xen/arch/x86/include/asm/irq.h | 2 +- xen/arch/x86/irq.c | 21 +++++++++++---------- 2 files changed, 12 insertions(+), 11 deletions(-) diff --git a/xen/arch/x86/include/asm/irq.h b/xen/arch/x86/include/asm/irq.h index 94f634ce10a1..5f445299be61 100644 --- a/xen/arch/x86/include/asm/irq.h +++ b/xen/arch/x86/include/asm/irq.h @@ -168,7 +168,7 @@ void free_domain_pirqs(struct domain *d); int map_domain_emuirq_pirq(struct domain *d, int pirq, int emuirq); int unmap_domain_pirq_emuirq(struct domain *d, int pirq); -/* Reset irq affinities to match the given CPU mask. */ +/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */ void fixup_irqs(const cpumask_t *mask, bool verbose); void fixup_eoi(void); diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index d101ffeaf9f3..263e502bc0f6 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2516,7 +2516,7 @@ static int __init cf_check setup_dump_irqs(void) } __initcall(setup_dump_irqs); -/* Reset irq affinities to match the given CPU mask. */ +/* Evacuate interrupts assigned to CPUs not present in the input CPU mask. */ void fixup_irqs(const cpumask_t *mask, bool verbose) { unsigned int irq; @@ -2540,19 +2540,15 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) vector = irq_to_vector(irq); if ( vector >= FIRST_HIPRIORITY_VECTOR && - vector <= LAST_HIPRIORITY_VECTOR ) + vector <= LAST_HIPRIORITY_VECTOR && + desc->handler == &no_irq_type ) { - cpumask_and(desc->arch.cpu_mask, desc->arch.cpu_mask, mask); - /* * This can in particular happen when parking secondary threads * during boot and when the serial console wants to use a PCI IRQ. */ - if ( desc->handler == &no_irq_type ) - { - spin_unlock(&desc->lock); - continue; - } + spin_unlock(&desc->lock); + continue; } if ( desc->arch.move_cleanup_count ) @@ -2573,7 +2569,12 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) affinity); } - if ( !desc->action || cpumask_subset(desc->affinity, mask) ) + /* + * Avoid shuffling the interrupt around as long as current target CPUs + * are a subset of the input mask. What fixup_irqs() cares about is + * evacuating interrupts from CPUs not in the input mask. + */ + if ( !desc->action || cpumask_subset(desc->arch.cpu_mask, mask) ) { spin_unlock(&desc->lock); continue; From patchwork Mon Jun 10 14:20:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13692119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 504F1C27C6E for ; Mon, 10 Jun 2024 14:21:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.737300.1143560 (Exim 4.92) (envelope-from ) id 1sGftL-0000x5-HD; Mon, 10 Jun 2024 14:20:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 737300.1143560; Mon, 10 Jun 2024 14:20:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftL-0000wy-E5; Mon, 10 Jun 2024 14:20:59 +0000 Received: by outflank-mailman (input) for mailman id 737300; Mon, 10 Jun 2024 14:20:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftK-0008MI-O1 for xen-devel@lists.xenproject.org; Mon, 10 Jun 2024 14:20:58 +0000 Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [2607:f8b0:4864:20::734]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a6f335bc-2734-11ef-b4bb-af5377834399; Mon, 10 Jun 2024 16:20:57 +0200 (CEST) Received: by mail-qk1-x734.google.com with SMTP id af79cd13be357-7952b60b4d7so300652285a.1 for ; Mon, 10 Jun 2024 07:20:57 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b0822399ccsm6739426d6.123.2024.06.10.07.20.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:20:55 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a6f335bc-2734-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718029255; x=1718634055; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GGTwxJouy4hdHEkmRULV0M0hn4a97amWG2LKmAA4/LI=; b=gdPTr4yNpyzE5fZUnwiRORL5A1efOheLMHHR0UTvaLk9Ph0eCTVj/Z/WqhFkfubKrD aluOFtmfm0P9hP7rvGy//Rwqq4pjVK/S8F1SKTwWf9ZEhUV2zpBNyA4VaXJTYHID1CJB sXzOu56ykRSeU4qSAmYNbV8gPvS/mGUc1mxvU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718029255; x=1718634055; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GGTwxJouy4hdHEkmRULV0M0hn4a97amWG2LKmAA4/LI=; b=ndv6p356hElMnpJE/XKF9pC53yvbGjBEc/0tL2ch23rybEc3Sk+i50DrUZCExHwOtI 17OK9OKuSrG4XvgttLcsl6UTyqWyD5KVyQtFzme6XOr5UmVkXfduK8ryPfxzrLj4aS9k DeaXfW0U0SKtFmLhjVoCV+uUpArX9OiNAcZuYjMWxx3s7L6xro5brSiTLAmNx6xBQZM4 joKUgJSH31zrz94HFlzXguRTavbkXVcxl8fpz3E7hHvidOla7mPssT/9iGxD+TmWqkkl O/qDv30UU4DNFUV+bgp0a6nMDVQm+bk3wOR2wc6oT3B5hKEjGjR/kdtP9AntbO+1HfT6 srVQ== X-Gm-Message-State: AOJu0Yx9wtC9F5aWJKcgH0Lx2RD0wnjvBax8q7/kGmFVdjAqZHw8VVfj HgYCf7uBl6adFnbt68H88XT6IXWq3uxte6hFpFWj4yx6fbwEyh6LbCwrrmOGgprvmVp75f2cH0C D X-Google-Smtp-Source: AGHT+IFxyBZcjImT23c50YxZ3XhRQ3nOFbdcVSIf3HtTe5DOm8honhk2yNmjD3TfewFRcmiHX1JqGA== X-Received: by 2002:a05:6214:4520:b0:6af:2344:5811 with SMTP id 6a1803df08f44-6b059f93d58mr116646196d6.55.1718029255616; Mon, 10 Jun 2024 07:20:55 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v2 4/7] x86/irq: restrict CPU movement in set_desc_affinity() Date: Mon, 10 Jun 2024 16:20:40 +0200 Message-ID: <20240610142043.11924-5-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610142043.11924-1-roger.pau@citrix.com> References: <20240610142043.11924-1-roger.pau@citrix.com> MIME-Version: 1.0 If external interrupts are using logical mode it's possible to have an overlap between the current ->arch.cpu_mask and the provided mask (or TARGET_CPUS). If that's the case avoid assigning a new vector and just move the interrupt to a member of ->arch.cpu_mask that overlaps with the provided mask and is online. While there also add an extra assert to ensure the mask containing the possible destinations is not empty before calling cpu_mask_to_apicid(), as at that point having an empty mask is not expected. Signed-off-by: Roger Pau Monné --- xen/arch/x86/irq.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index 263e502bc0f6..306e7756112f 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -837,19 +837,38 @@ void cf_check irq_complete_move(struct irq_desc *desc) unsigned int set_desc_affinity(struct irq_desc *desc, const cpumask_t *mask) { - int ret; - unsigned long flags; cpumask_t dest_mask; if ( mask && !cpumask_intersects(mask, &cpu_online_map) ) return BAD_APICID; - spin_lock_irqsave(&vector_lock, flags); - ret = _assign_irq_vector(desc, mask ?: TARGET_CPUS); - spin_unlock_irqrestore(&vector_lock, flags); + /* + * mask input set can contain CPUs that are not online. To decide whether + * the interrupt needs to be migrated restrict the input mask to the CPUs + * that are online. + */ + if ( mask ) + cpumask_and(&dest_mask, mask, &cpu_online_map); + else + cpumask_copy(&dest_mask, TARGET_CPUS); - if ( ret < 0 ) - return BAD_APICID; + /* + * Only move the interrupt if there are no CPUs left in ->arch.cpu_mask + * that can handle it, otherwise just shuffle it around ->arch.cpu_mask + * to an available destination. + */ + if ( !cpumask_intersects(desc->arch.cpu_mask, &dest_mask) ) + { + int ret; + unsigned long flags; + + spin_lock_irqsave(&vector_lock, flags); + ret = _assign_irq_vector(desc, mask ?: TARGET_CPUS); + spin_unlock_irqrestore(&vector_lock, flags); + + if ( ret < 0 ) + return BAD_APICID; + } if ( mask ) { @@ -862,6 +881,7 @@ unsigned int set_desc_affinity(struct irq_desc *desc, const cpumask_t *mask) cpumask_copy(&dest_mask, desc->arch.cpu_mask); } cpumask_and(&dest_mask, &dest_mask, &cpu_online_map); + ASSERT(!cpumask_empty(&dest_mask)); return cpu_mask_to_apicid(&dest_mask); } From patchwork Mon Jun 10 14:20:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13692118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E66A2C27C55 for ; Mon, 10 Jun 2024 14:21:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.737301.1143570 (Exim 4.92) (envelope-from ) id 1sGftN-0001Ec-SF; Mon, 10 Jun 2024 14:21:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 737301.1143570; Mon, 10 Jun 2024 14:21:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftN-0001EL-Of; Mon, 10 Jun 2024 14:21:01 +0000 Received: by outflank-mailman (input) for mailman id 737301; Mon, 10 Jun 2024 14:21:00 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftL-00007Q-Vr for xen-devel@lists.xenproject.org; Mon, 10 Jun 2024 14:20:59 +0000 Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [2607:f8b0:4864:20::72c]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a841e0bc-2734-11ef-90a2-e314d9c70b13; Mon, 10 Jun 2024 16:20:59 +0200 (CEST) Received: by mail-qk1-x72c.google.com with SMTP id af79cd13be357-7955f3d4516so106311385a.1 for ; Mon, 10 Jun 2024 07:20:59 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-440387964afsm36806921cf.0.2024.06.10.07.20.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:20:57 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a841e0bc-2734-11ef-90a2-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718029258; x=1718634058; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/CHpMOxsvIdkAbF+8l0n1eVm0ADzVdTBMt6vKxN7VAA=; b=hqCGMqSu/r5P9AKQyqMjcgxzD1ZcE+SqoGU2JRvcDeHtdieRa34+QZGZEzUaXHJdmb RPUxW9gVZW0NxdXFMkios4liiz+m2VeteqKrKmmi3QlhlLG/tGpV2l5H0YI1ZiwqFRWw JEghK7IpJLndxVp2OaIAaN8HZrGXfN4tBw9aY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718029258; x=1718634058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/CHpMOxsvIdkAbF+8l0n1eVm0ADzVdTBMt6vKxN7VAA=; b=cNQiqj6rqdC0Nqyr9NSJIK/8FXOE0bU3yNbHFPt5zY0JQcWSG4R+XszN3zaOj1Z/qf 21g3dyFEJuCZuSEYYRlgdZqUwiO5YWxm+QopwNByKGIRIlcLQpzitiE9aq6D3XozjiZ2 6dSHQkZk5PPZ3mPHtj+VhVvNNBqjivKuSACbgOtycixIQGMuJtmIC5iifE7Vbezf8GjQ gn87TQvIzzLhLWcLZqgnsLs7unaqEydOgzqhM+AyoycQeQ7Ap3eeSXRvo34ewlLx55ml 0S1Zhq6GMGZEXd23eq3HYPCoLC/BMM7xqzBsyiBwS4hNKOHthVZNdBP9CQ0XD4b2KQfX ugJw== X-Gm-Message-State: AOJu0YxQbLQ6ZyqF1X+0mnGWUVt1PLcq7XanajlANbyDV1db4OeT5YBN vc7R+EiaJee11t6C6J3UE5sb5cRAyyOtgd1QprEZHaQlQDaD3wtaejXoy9isPIbKoqQ6XfDELHd q X-Google-Smtp-Source: AGHT+IEoORcnXRvpb01P4p2Pp35n1REoevTWgtTP+Wf8+LLKFZGHm5W9a3McbUnGGXRgS9uXGpQhRQ== X-Received: by 2002:a05:620a:4149:b0:795:5c48:4271 with SMTP id af79cd13be357-7955c4843c3mr781619185a.25.1718029257749; Mon, 10 Jun 2024 07:20:57 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v2 5/7] x86/irq: deal with old_cpu_mask for interrupts in movement in fixup_irqs() Date: Mon, 10 Jun 2024 16:20:41 +0200 Message-ID: <20240610142043.11924-6-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610142043.11924-1-roger.pau@citrix.com> References: <20240610142043.11924-1-roger.pau@citrix.com> MIME-Version: 1.0 Given the current logic it's possible for ->arch.old_cpu_mask to get out of sync: if a CPU set in old_cpu_mask is offlined and then onlined again without old_cpu_mask having been updated the data in the mask will no longer be accurate, as when brought back online the CPU will no longer have old_vector configured to handle the old interrupt source. If there's an interrupt movement in progress, and the to be offlined CPU (which is the call context) is in the old_cpu_mask clear it and update the mask, so it doesn't contain stale data. Signed-off-by: Roger Pau Monné Reviewed-by: Jan Beulich --- xen/arch/x86/irq.c | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index 306e7756112f..f07e09b63b53 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2546,7 +2546,7 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) for ( irq = 0; irq < nr_irqs; irq++ ) { bool break_affinity = false, set_affinity = true; - unsigned int vector; + unsigned int vector, cpu = smp_processor_id(); cpumask_t *affinity = this_cpu(scratch_cpumask); if ( irq == 2 ) @@ -2589,6 +2589,28 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) affinity); } + if ( desc->arch.move_in_progress && + !cpumask_test_cpu(cpu, &cpu_online_map) && + cpumask_test_cpu(cpu, desc->arch.old_cpu_mask) ) + { + /* + * This CPU is going offline, remove it from ->arch.old_cpu_mask + * and possibly release the old vector if the old mask becomes + * empty. + * + * Note cleaning ->arch.old_cpu_mask is required if the CPU is + * brought offline and then online again, as when re-onlined the + * per-cpu vector table will no longer have ->arch.old_vector + * setup, and hence ->arch.old_cpu_mask would be stale. + */ + cpumask_clear_cpu(cpu, desc->arch.old_cpu_mask); + if ( cpumask_empty(desc->arch.old_cpu_mask) ) + { + desc->arch.move_in_progress = 0; + release_old_vec(desc); + } + } + /* * Avoid shuffling the interrupt around as long as current target CPUs * are a subset of the input mask. What fixup_irqs() cares about is From patchwork Mon Jun 10 14:20:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13692120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63E77C27C5E for ; Mon, 10 Jun 2024 14:21:14 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.737302.1143580 (Exim 4.92) (envelope-from ) id 1sGftR-0001ZF-6R; Mon, 10 Jun 2024 14:21:05 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 737302.1143580; Mon, 10 Jun 2024 14:21:05 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftR-0001Yz-1z; Mon, 10 Jun 2024 14:21:05 +0000 Received: by outflank-mailman (input) for mailman id 737302; Mon, 10 Jun 2024 14:21:03 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftP-0008MI-DJ for xen-devel@lists.xenproject.org; Mon, 10 Jun 2024 14:21:03 +0000 Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [2607:f8b0:4864:20::833]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a9a999f6-2734-11ef-b4bb-af5377834399; Mon, 10 Jun 2024 16:21:01 +0200 (CEST) Received: by mail-qt1-x833.google.com with SMTP id d75a77b69052e-43ffbc0927fso25295631cf.3 for ; Mon, 10 Jun 2024 07:21:01 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-44038a68e55sm36473281cf.24.2024.06.10.07.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:20:59 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a9a999f6-2734-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718029260; x=1718634060; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4+NbzKdNCPo1s6Mdl+9AvKyg4EH4oF+mfwLbxFaB8IA=; b=PxS/mHzupQbbfPj2V9dYqIojoNX3KQAPgxwMocNksyqibNCKhoc+Ouqo89sbvY8Yf8 +Fp9L5/aCWEvLkXKet74jKqZOlpdc4NtQeAsY5VSANgsd8SAbZha6ytZEdHXIdAx7EG4 ZmzOVMrBiY2NvS4EFLQVy3+toFJS44erp+c5M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718029260; x=1718634060; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4+NbzKdNCPo1s6Mdl+9AvKyg4EH4oF+mfwLbxFaB8IA=; b=F6FjbupZ1v/Ka8nIat/V07JWb5dXYDyo8BpPbzt/xzPc0bGG0oVIRluTHxnfjM7NBc Yup0NFAZ7HUDi+RH3aXc1JCvg4up8gexTPw2WrB65xZXrZGljMjaUKIIfVM70lTg/qR+ BWD2dSr43FD5cX7Tpsc2gtkxg/PBTEtIPjGvaxy4hJ1juR4J6uAobqA0ObGeZQtxFD93 6bHJPrRISht0qt7OzJnEZDi7If0q/eCvwz6Em9mPSmPc4CTs0o4iFr4RHOjkbCQT/cHQ 5sjIzvlLVVA+YAnyZueh4fUtl3FcI1j9nWNJivzeL73kGTHqARzbi8ay+PuPAtBBodrM nGjA== X-Gm-Message-State: AOJu0Yyo3XtjejCA+Mef3AqKgFgU1Uud02Mf15nJgSi3SfJrnYAvcUKO 6THJeONcXkvlC2X0UnlsZVeFdDaR58ZouuIFcZ7qzGhuxuEk4tb9uLKG3iWjKPurQ8wUktwYR4/ s X-Google-Smtp-Source: AGHT+IGoVswSKHWjkGlfxiCqCrFXbRF14HzayUnmcbGoCXniQP/N9W0gX6JoNsdwjyxFEKSB1kvHow== X-Received: by 2002:a05:622a:13c8:b0:43a:e730:3a23 with SMTP id d75a77b69052e-44041c31ad9mr113957351cf.3.1718029259897; Mon, 10 Jun 2024 07:20:59 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v2 6/7] x86/irq: handle moving interrupts in _assign_irq_vector() Date: Mon, 10 Jun 2024 16:20:42 +0200 Message-ID: <20240610142043.11924-7-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610142043.11924-1-roger.pau@citrix.com> References: <20240610142043.11924-1-roger.pau@citrix.com> MIME-Version: 1.0 Currently there's logic in fixup_irqs() that attempts to prevent _assign_irq_vector() from failing, as fixup_irqs() is required to evacuate all interrupts from the CPUs not present in the input mask. The current logic in fixup_irqs() is incomplete, as it doesn't deal with interrupts that have move_cleanup_count > 0 and a non-empty ->arch.old_cpu_mask field. Instead of attempting to fixup the interrupt descriptor in fixup_irqs() so that _assign_irq_vector() cannot fail, introduce logic in _assign_irq_vector() to deal with interrupts that have either move_{in_progress,cleanup_count} set and no remaining online CPUs in ->arch.cpu_mask. If _assign_irq_vector() is requested to move an interrupt in the state described above, first attempt to see if ->arch.old_cpu_mask contains any valid CPUs that could be used as fallback, and if that's the case do move the interrupt back to the previous destination. Note this is easier because the vector hasn't been released yet, so there's no need to allocate and setup a new vector on the destination. Due to the logic in fixup_irqs() that clears offline CPUs from ->arch.old_cpu_mask (and releases the old vector if the mask becomes empty) it shouldn't be possible to get into _assign_irq_vector() with ->arch.move_{in_progress,cleanup_count} set but no online CPUs in ->arch.old_cpu_mask. However if ->arch.move_{in_progress,cleanup_count} is set and the interrupt has also changed affinity, it's possible the members of ->arch.old_cpu_mask are no longer part of the affinity set, move the interrupt to a different CPU part of the provided mask and keep the current ->arch.old_{cpu_mask,vector} for the pending interrupt movement to be completed. Signed-off-by: Roger Pau Monné --- Changes since v1: - Further refine the logic in _assign_irq_vector(). --- xen/arch/x86/irq.c | 87 ++++++++++++++++++++++++++++++---------------- 1 file changed, 58 insertions(+), 29 deletions(-) diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index f07e09b63b53..54eabd23995c 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -544,7 +544,53 @@ static int _assign_irq_vector(struct irq_desc *desc, const cpumask_t *mask) } if ( desc->arch.move_in_progress || desc->arch.move_cleanup_count ) - return -EAGAIN; + { + /* + * If the current destination is online refuse to shuffle. Retry after + * the in-progress movement has finished. + */ + if ( cpumask_intersects(desc->arch.cpu_mask, &cpu_online_map) ) + return -EAGAIN; + + /* + * Due to the logic in fixup_irqs() that clears offlined CPUs from + * ->arch.old_cpu_mask it shouldn't be possible to get here with + * ->arch.move_{in_progress,cleanup_count} set and no online CPUs in + * ->arch.old_cpu_mask. + */ + ASSERT(valid_irq_vector(desc->arch.old_vector)); + ASSERT(cpumask_intersects(desc->arch.old_cpu_mask, &cpu_online_map)); + + if ( cpumask_intersects(desc->arch.old_cpu_mask, mask) ) + { + /* + * Fallback to the old destination if moving is in progress and the + * current destination is to be offlined. This is only possible if + * the CPUs in old_cpu_mask intersect with the affinity mask passed + * in the 'mask' parameter. + */ + desc->arch.vector = desc->arch.old_vector; + cpumask_and(desc->arch.cpu_mask, desc->arch.old_cpu_mask, mask); + + /* Undo any possibly done cleanup. */ + for_each_cpu(cpu, desc->arch.cpu_mask) + per_cpu(vector_irq, cpu)[desc->arch.vector] = irq; + + /* Cancel the pending move. */ + desc->arch.old_vector = IRQ_VECTOR_UNASSIGNED; + cpumask_clear(desc->arch.old_cpu_mask); + desc->arch.move_in_progress = 0; + desc->arch.move_cleanup_count = 0; + + return 0; + } + + /* + * There's an interrupt movement in progress but the destination(s) in + * ->arch.old_cpu_mask are not suitable given the passed 'mask', go + * through the full logic to find a new vector in a suitable CPU. + */ + } err = -ENOSPC; @@ -600,7 +646,17 @@ next: current_vector = vector; current_offset = offset; - if ( valid_irq_vector(old_vector) ) + if ( desc->arch.move_in_progress || desc->arch.move_cleanup_count ) + { + ASSERT(!cpumask_intersects(desc->arch.cpu_mask, &cpu_online_map)); + /* + * Special case when evacuating an interrupt from a CPU to be + * offlined and the interrupt was already in the process of being + * moved. Leave ->arch.old_{vector,cpu_mask} as-is and just + * replace ->arch.{cpu_mask,vector} with the new destination. + */ + } + else if ( valid_irq_vector(old_vector) ) { cpumask_and(desc->arch.old_cpu_mask, desc->arch.cpu_mask, &cpu_online_map); @@ -2622,33 +2678,6 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) continue; } - /* - * In order for the affinity adjustment below to be successful, we - * need _assign_irq_vector() to succeed. This in particular means - * clearing desc->arch.move_in_progress if this would otherwise - * prevent the function from succeeding. Since there's no way for the - * flag to get cleared anymore when there's no possible destination - * left (the only possibility then would be the IRQs enabled window - * after this loop), there's then also no race with us doing it here. - * - * Therefore the logic here and there need to remain in sync. - */ - if ( desc->arch.move_in_progress && - !cpumask_intersects(mask, desc->arch.cpu_mask) ) - { - unsigned int cpu; - - cpumask_and(affinity, desc->arch.old_cpu_mask, &cpu_online_map); - - spin_lock(&vector_lock); - for_each_cpu(cpu, affinity) - per_cpu(vector_irq, cpu)[desc->arch.old_vector] = ~irq; - spin_unlock(&vector_lock); - - release_old_vec(desc); - desc->arch.move_in_progress = 0; - } - if ( !cpumask_intersects(mask, desc->affinity) ) { break_affinity = true; From patchwork Mon Jun 10 14:20:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 13692121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B0DB6C27C55 for ; Mon, 10 Jun 2024 14:21:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.737303.1143590 (Exim 4.92) (envelope-from ) id 1sGftT-0001tT-DS; Mon, 10 Jun 2024 14:21:07 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 737303.1143590; Mon, 10 Jun 2024 14:21:07 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftT-0001tB-9v; Mon, 10 Jun 2024 14:21:07 +0000 Received: by outflank-mailman (input) for mailman id 737303; Mon, 10 Jun 2024 14:21:06 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sGftS-0008MI-L0 for xen-devel@lists.xenproject.org; Mon, 10 Jun 2024 14:21:06 +0000 Received: from mail-oa1-x2d.google.com (mail-oa1-x2d.google.com [2001:4860:4864:20::2d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id aaf2755e-2734-11ef-b4bb-af5377834399; Mon, 10 Jun 2024 16:21:03 +0200 (CEST) Received: by mail-oa1-x2d.google.com with SMTP id 586e51a60fabf-254e42df409so198619fac.0 for ; Mon, 10 Jun 2024 07:21:03 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id af79cd13be357-795ca413ed3sm109519685a.43.2024.06.10.07.21.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jun 2024 07:21:01 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aaf2755e-2734-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718029262; x=1718634062; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1/0Nmqg5/SRyeHMyQYQ756u1OolFE3kJadfERqjt0Rk=; b=Gc70o86TfMk3tM1/TdasBJbrSAS2n6QgRTQCEdHAoC4SwabvYQMj2YIRxRSVRwF8lk o+LOWyx+390iEwQXUi6rpZvKp6Jk1QMr0I0V+AVh1uwCIliYz3HC7HH56cacAU6HebGN ssGWUvD6OX1Qf+G9KDXEXHoJqsBUgBAXxjUWQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718029262; x=1718634062; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1/0Nmqg5/SRyeHMyQYQ756u1OolFE3kJadfERqjt0Rk=; b=aCTg/1FCquiIsW9t3q8krF0s7WjXERVFSUeMoooFsuF5RwbK26h15EbZ0h2cVAnWt9 pYllin5nCPIPXQSjUEKezdH/XR0k7tBlKK7rd017YQMOMrs88UBVDMjTuJamjYpJLGYH us/+/gNBZHnLPsUi17JjiQssFZrBliP5n1M2oprdSfzFN4VDA7Fw/zxs5wUBTW1q+dMP F3BES/yQ0nRrEB9sCSrSCTNHaqEh4qZgirV3YmHZgzZzzBQRRuD2rlTLNsdRmQ25E1LS NE/s6buzFMBqynwRo5B2y9CTnM/dS6VcLNOPFVdWcRh90AGeXLulQNQMVsdOkoyX++/q stRA== X-Gm-Message-State: AOJu0Yz62cchlDNVb4OaTFYo3MG3olqgwBdcbNt7ccmyHTJ0HqPP43ym gber1Z1gDuyomNDtrYCNcblr9Yfw+j+OIYViaf35kFLLmbx+WnwcDAU9PCrK8F17FHdPfYS1u/e X X-Google-Smtp-Source: AGHT+IHogHOuVFsgIfOA2XNMT/LsSuo3pBEUpsSOSwxa/IxpdguZPEPVX8JY1QIHhekoZfHCLyzG6w== X-Received: by 2002:a05:6870:70a9:b0:250:7032:5633 with SMTP id 586e51a60fabf-2546441a0a4mr10990106fac.14.1718029262133; Mon, 10 Jun 2024 07:21:02 -0700 (PDT) From: Roger Pau Monne To: xen-devel@lists.xenproject.org Cc: Roger Pau Monne , Jan Beulich , Andrew Cooper Subject: [PATCH v2 7/7] x86/irq: forward pending interrupts to new destination in fixup_irqs() Date: Mon, 10 Jun 2024 16:20:43 +0200 Message-ID: <20240610142043.11924-8-roger.pau@citrix.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240610142043.11924-1-roger.pau@citrix.com> References: <20240610142043.11924-1-roger.pau@citrix.com> MIME-Version: 1.0 fixup_irqs() is used to evacuate interrupts from to be offlined CPUs. Given the CPU is to become offline, the normal migration logic used by Xen where the vector in the previous target(s) is left configured until the interrupt is received on the new destination is not suitable. Instead attempt to do as much as possible in order to prevent loosing interrupts. If fixup_irqs() is called from the CPU to be offlined (as is currently the case) attempt to forward pending vectors when interrupts that target the current CPU are migrated to a different destination. Additionally, for interrupts that have already been moved from the current CPU prior to the call to fixup_irqs() but that haven't been delivered to the new destination (iow: interrupts with move_in_progress set and the current CPU set in ->arch.old_cpu_mask) also check whether the previous vector is pending and forward it to the new destination. Signed-off-by: Roger Pau Monné --- Changes since v1: - Rename to apic_irr_read(). --- xen/arch/x86/include/asm/apic.h | 5 +++++ xen/arch/x86/irq.c | 37 ++++++++++++++++++++++++++++++++- 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/include/asm/apic.h b/xen/arch/x86/include/asm/apic.h index d1cb001fb4ab..7bd66dc6e151 100644 --- a/xen/arch/x86/include/asm/apic.h +++ b/xen/arch/x86/include/asm/apic.h @@ -132,6 +132,11 @@ static inline bool apic_isr_read(uint8_t vector) (vector & 0x1f)) & 1; } +static inline bool apic_irr_read(unsigned int vector) +{ + return apic_read(APIC_IRR + (vector / 32 * 0x10)) & (1U << (vector % 32)); +} + static inline u32 get_apic_id(void) { u32 id = apic_read(APIC_ID); diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c index 54eabd23995c..ed262fb55f4a 100644 --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2601,7 +2601,7 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) for ( irq = 0; irq < nr_irqs; irq++ ) { - bool break_affinity = false, set_affinity = true; + bool break_affinity = false, set_affinity = true, check_irr = false; unsigned int vector, cpu = smp_processor_id(); cpumask_t *affinity = this_cpu(scratch_cpumask); @@ -2649,6 +2649,25 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) !cpumask_test_cpu(cpu, &cpu_online_map) && cpumask_test_cpu(cpu, desc->arch.old_cpu_mask) ) { + /* + * This to be offlined CPU was the target of an interrupt that's + * been moved, and the new destination target hasn't yet + * acknowledged any interrupt from it. + * + * We know the interrupt is configured to target the new CPU at + * this point, so we can check IRR for any pending vectors and + * forward them to the new destination. + * + * Note the difference between move_in_progress or + * move_cleanup_count being set. For the later we know the new + * destination has already acked at least one interrupt from this + * source, and hence there's no need to forward any stale + * interrupts. + */ + if ( apic_irr_read(desc->arch.old_vector) ) + send_IPI_mask(cpumask_of(cpumask_any(desc->arch.cpu_mask)), + desc->arch.vector); + /* * This CPU is going offline, remove it from ->arch.old_cpu_mask * and possibly release the old vector if the old mask becomes @@ -2689,11 +2708,27 @@ void fixup_irqs(const cpumask_t *mask, bool verbose) if ( desc->handler->disable ) desc->handler->disable(desc); + /* + * If the current CPU is going offline and is (one of) the target(s) of + * the interrupt signal to check whether there are any pending vectors + * to be handled in the local APIC after the interrupt has been moved. + */ + if ( !cpu_online(cpu) && cpumask_test_cpu(cpu, desc->arch.cpu_mask) ) + check_irr = true; + if ( desc->handler->set_affinity ) desc->handler->set_affinity(desc, affinity); else if ( !(warned++) ) set_affinity = false; + if ( check_irr && apic_irr_read(vector) ) + /* + * Forward pending interrupt to the new destination, this CPU is + * going offline and otherwise the interrupt would be lost. + */ + send_IPI_mask(cpumask_of(cpumask_any(desc->arch.cpu_mask)), + desc->arch.vector); + if ( desc->handler->enable ) desc->handler->enable(desc);