From patchwork Mon Apr 29 11:22:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921683 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E83AF1515 for ; Mon, 29 Apr 2019 11:25:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D632B27C2D for ; Mon, 29 Apr 2019 11:25:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C75B5282E8; Mon, 29 Apr 2019 11:25:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 52E3E27C2D for ; Mon, 29 Apr 2019 11:25:09 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4NF-0007mM-4q; Mon, 29 Apr 2019 11:23:05 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4ND-0007mH-IA for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:23:03 +0000 X-Inumbo-ID: 25474eec-6a71-11e9-9fd0-0721d75f5957 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 25474eec-6a71-11e9-9fd0-0721d75f5957; Mon, 29 Apr 2019 11:23:00 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:22:59 -0600 Message-Id: <5CC6DE8F0200007800229E9A@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:22:55 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH RFC 1/9] x86/IRQ: deal with move-in-progress state in fixup_irqs() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP The flag being set may prevent affinity changes, as these often imply assignment of a new vector. When there's no possible destination left for the IRQ, the clearing of the flag needs to happen right from fixup_irqs(). Additionally _assign_irq_vector() needs to avoid setting the flag when there's no online CPU left in what gets put into ->arch.old_cpu_mask. The old vector can be released right away in this case. Also extend the log message about broken affinity to include the new affinity as well, allowing to notice issues with affinity changes not actually having taken place. Swap the if/else-if order there at the same time to reduce the amount of conditions checked. At the same time replace two open coded instances of the new helper function. Signed-off-by: Jan Beulich --- RFC: I've seen the new ASSERT() in irq_move_cleanup_interrupt() trigger. I'm pretty sure that this assertion triggering means something else is wrong, and has been even prior to this change (adding the assertion without any of the other changes here should be valid in my understanding). --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -242,6 +242,20 @@ void destroy_irq(unsigned int irq) xfree(action); } +static void release_old_vec(struct irq_desc *desc) +{ + unsigned int vector = desc->arch.old_vector; + + desc->arch.old_vector = IRQ_VECTOR_UNASSIGNED; + cpumask_clear(desc->arch.old_cpu_mask); + + if ( desc->arch.used_vectors ) + { + ASSERT(test_bit(vector, desc->arch.used_vectors)); + clear_bit(vector, desc->arch.used_vectors); + } +} + static void __clear_irq_vector(int irq) { int cpu, vector, old_vector; @@ -285,14 +299,7 @@ static void __clear_irq_vector(int irq) per_cpu(vector_irq, cpu)[old_vector] = ~irq; } - desc->arch.old_vector = IRQ_VECTOR_UNASSIGNED; - cpumask_clear(desc->arch.old_cpu_mask); - - if ( desc->arch.used_vectors ) - { - ASSERT(test_bit(old_vector, desc->arch.used_vectors)); - clear_bit(old_vector, desc->arch.used_vectors); - } + release_old_vec(desc); desc->arch.move_in_progress = 0; } @@ -517,12 +524,21 @@ next: /* Found one! */ current_vector = vector; current_offset = offset; - if (old_vector > 0) { - desc->arch.move_in_progress = 1; - cpumask_copy(desc->arch.old_cpu_mask, desc->arch.cpu_mask); + + if ( old_vector > 0 ) + { + cpumask_and(desc->arch.old_cpu_mask, desc->arch.cpu_mask, + &cpu_online_map); desc->arch.old_vector = desc->arch.vector; + if ( !cpumask_empty(desc->arch.old_cpu_mask) ) + desc->arch.move_in_progress = 1; + else + /* This can happen while offlining a CPU. */ + release_old_vec(desc); } + trace_irq_mask(TRC_HW_IRQ_ASSIGN_VECTOR, irq, vector, &tmp_mask); + for_each_cpu(new_cpu, &tmp_mask) per_cpu(vector_irq, new_cpu)[vector] = irq; desc->arch.vector = vector; @@ -691,14 +707,8 @@ void irq_move_cleanup_interrupt(struct c if ( desc->arch.move_cleanup_count == 0 ) { - desc->arch.old_vector = IRQ_VECTOR_UNASSIGNED; - cpumask_clear(desc->arch.old_cpu_mask); - - if ( desc->arch.used_vectors ) - { - ASSERT(test_bit(vector, desc->arch.used_vectors)); - clear_bit(vector, desc->arch.used_vectors); - } + ASSERT(vector == desc->arch.old_vector); + release_old_vec(desc); } unlock: spin_unlock(&desc->lock); @@ -2391,6 +2401,24 @@ void fixup_irqs(const cpumask_t *mask, b continue; } + /* + * In order for the affinity adjustment below to be successful, we + * need __assign_irq_vector() to succeed. This in particular means + * clearing desc->arch.move_in_progress if this would otherwise + * prevent the function from succeeding. Since there's no way for the + * flag to get cleared anymore when there's no possible destination + * left (the only possibility then would be the IRQs enabled window + * after this loop), there's then also no race with us doing it here. + * + * Therefore the logic here and there need to remain in sync. + */ + if ( desc->arch.move_in_progress && + !cpumask_intersects(mask, desc->arch.cpu_mask) ) + { + release_old_vec(desc); + desc->arch.move_in_progress = 0; + } + cpumask_and(&affinity, &affinity, mask); if ( cpumask_empty(&affinity) ) { @@ -2409,15 +2437,18 @@ void fixup_irqs(const cpumask_t *mask, b if ( desc->handler->enable ) desc->handler->enable(desc); + cpumask_copy(&affinity, desc->affinity); + spin_unlock(&desc->lock); if ( !verbose ) continue; - if ( break_affinity && set_affinity ) - printk("Broke affinity for irq %i\n", irq); - else if ( !set_affinity ) - printk("Cannot set affinity for irq %i\n", irq); + if ( !set_affinity ) + printk("Cannot set affinity for IRQ%u\n", irq); + else if ( break_affinity ) + printk("Broke affinity for IRQ%u, new: %*pb\n", + irq, nr_cpu_ids, &affinity); } /* That doesn't seem sufficient. Give it 1ms. */ From patchwork Mon Apr 29 11:23:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D02541515 for ; Mon, 29 Apr 2019 11:24:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAE032871E for ; Mon, 29 Apr 2019 11:24:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC383287E7; Mon, 29 Apr 2019 11:24:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 491B22871E for ; Mon, 29 Apr 2019 11:24:44 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Nb-0007o9-Ej; Mon, 29 Apr 2019 11:23:27 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Nb-0007o2-0G for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:23:27 +0000 X-Inumbo-ID: 340a8c9a-6a71-11e9-843c-bc764e045a96 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 340a8c9a-6a71-11e9-843c-bc764e045a96; Mon, 29 Apr 2019 11:23:25 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:23:24 -0600 Message-Id: <5CC6DEA80200007800229E9D@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:23:20 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 2/9] x86/IRQ: deal with move cleanup count state in fixup_irqs() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP The cleanup IPI may get sent immediately before a CPU gets removed from the online map. In such a case the IPI would get handled on the CPU being offlined no earlier than in the interrupts disabled window after fixup_irqs()' main loop. This is too late, however, because a possible affinity change may incur the need for vector assignment, which will fail when the IRQ's move cleanup count is still non-zero. To fix this - record the set of CPUs the cleanup IPIs gets actually sent to alongside setting their count, - adjust the count in fixup_irqs(), accounting for all CPUs that the cleanup IPI was sent to, but that are no longer online, - bail early from the cleanup IPI handler when the CPU is no longer online, to prevent double accounting. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- TBD: The proper recording of the IPI destinations actually makes the move_cleanup_count field redundant. Do we want to drop it, at the price of a few more CPU-mask operations? --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -658,6 +658,9 @@ void irq_move_cleanup_interrupt(struct c ack_APIC_irq(); me = smp_processor_id(); + if ( !cpu_online(me) ) + return; + for ( vector = FIRST_DYNAMIC_VECTOR; vector <= LAST_HIPRIORITY_VECTOR; vector++) { @@ -717,11 +720,14 @@ unlock: static void send_cleanup_vector(struct irq_desc *desc) { - cpumask_t cleanup_mask; + cpumask_and(desc->arch.old_cpu_mask, desc->arch.old_cpu_mask, + &cpu_online_map); + desc->arch.move_cleanup_count = cpumask_weight(desc->arch.old_cpu_mask); - cpumask_and(&cleanup_mask, desc->arch.old_cpu_mask, &cpu_online_map); - desc->arch.move_cleanup_count = cpumask_weight(&cleanup_mask); - send_IPI_mask(&cleanup_mask, IRQ_MOVE_CLEANUP_VECTOR); + if ( desc->arch.move_cleanup_count ) + send_IPI_mask(desc->arch.old_cpu_mask, IRQ_MOVE_CLEANUP_VECTOR); + else + release_old_vec(desc); desc->arch.move_in_progress = 0; } @@ -2394,6 +2400,16 @@ void fixup_irqs(const cpumask_t *mask, b vector <= LAST_HIPRIORITY_VECTOR ) cpumask_and(desc->arch.cpu_mask, desc->arch.cpu_mask, mask); + if ( desc->arch.move_cleanup_count ) + { + /* The cleanup IPI may have got sent while we were still online. */ + cpumask_andnot(&affinity, desc->arch.old_cpu_mask, + &cpu_online_map); + desc->arch.move_cleanup_count -= cpumask_weight(&affinity); + if ( !desc->arch.move_cleanup_count ) + release_old_vec(desc); + } + cpumask_copy(&affinity, desc->affinity); if ( !desc->action || cpumask_subset(&affinity, mask) ) { From patchwork Mon Apr 29 11:23:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921685 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D3B114D5 for ; Mon, 29 Apr 2019 11:25:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EF576285CC for ; Mon, 29 Apr 2019 11:25:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E34392871E; Mon, 29 Apr 2019 11:25:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 792BB285CC for ; Mon, 29 Apr 2019 11:25:33 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4O7-0007rz-OK; Mon, 29 Apr 2019 11:23:59 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4O6-0007rk-Jm for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:23:58 +0000 X-Inumbo-ID: 46048172-6a71-11e9-a0ff-5f3e81f5e566 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 46048172-6a71-11e9-a0ff-5f3e81f5e566; Mon, 29 Apr 2019 11:23:55 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:23:54 -0600 Message-Id: <5CC6DEC50200007800229EA0@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:23:49 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 3/9] x86/IRQ: improve dump_irqs() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Don't log a stray trailing comma. Shorten a few fields. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2318,7 +2318,7 @@ static void dump_irqs(unsigned char key) spin_lock_irqsave(&desc->lock, flags); - printk(" IRQ:%4d affinity:%*pb vec:%02x type=%-15s status=%08x ", + printk(" IRQ:%4d aff:%*pb vec:%02x %-15s status=%03x ", irq, nr_cpu_ids, cpumask_bits(desc->affinity), desc->arch.vector, desc->handler->typename, desc->status); @@ -2329,23 +2329,21 @@ static void dump_irqs(unsigned char key) { action = (irq_guest_action_t *)desc->action; - printk("in-flight=%d domain-list=", action->in_flight); + printk("in-flight=%d%c", + action->in_flight, action->nr_guests ? ' ' : '\n'); - for ( i = 0; i < action->nr_guests; i++ ) + for ( i = 0; i < action->nr_guests; ) { - d = action->guest[i]; + d = action->guest[i++]; pirq = domain_irq_to_pirq(d, irq); info = pirq_info(d, pirq); - printk("%u:%3d(%c%c%c)", + printk("d%d:%3d(%c%c%c)%c", d->domain_id, pirq, evtchn_port_is_pending(d, info->evtchn) ? 'P' : '-', evtchn_port_is_masked(d, info->evtchn) ? 'M' : '-', - (info->masked ? 'M' : '-')); - if ( i != action->nr_guests ) - printk(","); + info->masked ? 'M' : '-', + i < action->nr_guests ? ',' : '\n'); } - - printk("\n"); } else if ( desc->action ) printk("%ps()\n", desc->action->handler); From patchwork Mon Apr 29 11:24:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921687 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C88B91515 for ; Mon, 29 Apr 2019 11:26:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5F5F2871E for ; Mon, 29 Apr 2019 11:26:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A9FE5287E7; Mon, 29 Apr 2019 11:26:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4B58C2871E for ; Mon, 29 Apr 2019 11:26:12 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Os-0007xx-36; Mon, 29 Apr 2019 11:24:46 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Oq-0007xe-FS for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:24:44 +0000 X-Inumbo-ID: 62bac5e2-6a71-11e9-843c-bc764e045a96 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 62bac5e2-6a71-11e9-843c-bc764e045a96; Mon, 29 Apr 2019 11:24:43 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:24:43 -0600 Message-Id: <5CC6DEF70200007800229EA3@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:24:39 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 4/9] x86/IRQ: desc->affinity should strictly represent the requested value X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP desc->arch.cpu_mask reflects the actual set of target CPUs. Don't ever fiddle with desc->affinity itself, except to store caller requested values. This renders both set_native_irq_info() uses (which weren't using proper locking anyway) redundant - drop the function altogether. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -1042,7 +1042,6 @@ static void __init setup_IO_APIC_irqs(vo SET_DEST(entry, logical, cpu_mask_to_apicid(TARGET_CPUS)); spin_lock_irqsave(&ioapic_lock, flags); __ioapic_write_entry(apic, pin, 0, entry); - set_native_irq_info(irq, TARGET_CPUS); spin_unlock_irqrestore(&ioapic_lock, flags); } } @@ -2251,7 +2250,6 @@ int io_apic_set_pci_routing (int ioapic, spin_lock_irqsave(&ioapic_lock, flags); __ioapic_write_entry(ioapic, pin, 0, entry); - set_native_irq_info(irq, TARGET_CPUS); spin_unlock(&ioapic_lock); spin_lock(&desc->lock); --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -572,11 +572,16 @@ int assign_irq_vector(int irq, const cpu spin_lock_irqsave(&vector_lock, flags); ret = __assign_irq_vector(irq, desc, mask ?: TARGET_CPUS); - if (!ret) { + if ( !ret ) + { ret = desc->arch.vector; - cpumask_copy(desc->affinity, desc->arch.cpu_mask); + if ( mask ) + cpumask_copy(desc->affinity, mask); + else + cpumask_setall(desc->affinity); } spin_unlock_irqrestore(&vector_lock, flags); + return ret; } @@ -2318,9 +2323,10 @@ static void dump_irqs(unsigned char key) spin_lock_irqsave(&desc->lock, flags); - printk(" IRQ:%4d aff:%*pb vec:%02x %-15s status=%03x ", - irq, nr_cpu_ids, cpumask_bits(desc->affinity), desc->arch.vector, - desc->handler->typename, desc->status); + printk(" IRQ:%4d aff:%*pb/%*pb vec:%02x %-15s status=%03x ", + irq, nr_cpu_ids, cpumask_bits(desc->affinity), + nr_cpu_ids, cpumask_bits(desc->arch.cpu_mask), + desc->arch.vector, desc->handler->typename, desc->status); if ( ssid ) printk("Z=%-25s ", ssid); @@ -2408,8 +2414,7 @@ void fixup_irqs(const cpumask_t *mask, b release_old_vec(desc); } - cpumask_copy(&affinity, desc->affinity); - if ( !desc->action || cpumask_subset(&affinity, mask) ) + if ( !desc->action || cpumask_subset(desc->affinity, mask) ) { spin_unlock(&desc->lock); continue; @@ -2433,12 +2438,13 @@ void fixup_irqs(const cpumask_t *mask, b desc->arch.move_in_progress = 0; } - cpumask_and(&affinity, &affinity, mask); - if ( cpumask_empty(&affinity) ) + if ( !cpumask_intersects(mask, desc->affinity) ) { break_affinity = true; - cpumask_copy(&affinity, mask); + cpumask_setall(&affinity); } + else + cpumask_copy(&affinity, desc->affinity); if ( desc->handler->disable ) desc->handler->disable(desc); --- a/xen/include/xen/irq.h +++ b/xen/include/xen/irq.h @@ -162,11 +162,6 @@ extern irq_desc_t *domain_spin_lock_irq_ extern irq_desc_t *pirq_spin_lock_irq_desc( const struct pirq *, unsigned long *pflags); -static inline void set_native_irq_info(unsigned int irq, const cpumask_t *mask) -{ - cpumask_copy(irq_to_desc(irq)->affinity, mask); -} - unsigned int set_desc_affinity(struct irq_desc *, const cpumask_t *); #ifndef arch_hwdom_irqs From patchwork Mon Apr 29 11:25:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921689 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0D2D1515 for ; Mon, 29 Apr 2019 11:26:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE1972871E for ; Mon, 29 Apr 2019 11:26:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A272F287E7; Mon, 29 Apr 2019 11:26:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 238CC2871E for ; Mon, 29 Apr 2019 11:26:55 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4PI-00084Q-Hx; Mon, 29 Apr 2019 11:25:12 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4PG-000845-Sn for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:25:10 +0000 X-Inumbo-ID: 71bb1362-6a71-11e9-8ca8-4f52f2f08acb Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 71bb1362-6a71-11e9-8ca8-4f52f2f08acb; Mon, 29 Apr 2019 11:25:09 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:25:08 -0600 Message-Id: <5CC6DF0F0200007800229EBC@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:25:03 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 5/9] x86/IRQ: fix locking around vector management X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP All of __{assign,bind,clear}_irq_vector() manipulate struct irq_desc fields, and hence ought to be called with the descriptor lock held in addition to vector_lock. This is currently the case for only set_desc_affinity() and destroy_irq(), which also clarifies what the nesting behavior between the locks has to be. Reflect the new expectation by having these functions all take a descriptor as parameter instead of an interrupt number. Drop one of the two leading underscores from all three functions at the same time. There's one case left where descriptors get manipulated with just vector_lock held: setup_vector_irq() assumes its caller to acquire vector_lock, and hence can't itself acquire the descriptor locks (wrong lock order). I don't currently see how to address this. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -27,6 +27,7 @@ #include static int parse_irq_vector_map_param(const char *s); +static void _clear_irq_vector(struct irq_desc *desc); /* opt_noirqbalance: If true, software IRQ balancing/affinity is disabled. */ bool __read_mostly opt_noirqbalance; @@ -112,13 +113,12 @@ static void trace_irq_mask(u32 event, in trace_var(event, 1, sizeof(d), &d); } -static int __init __bind_irq_vector(int irq, int vector, const cpumask_t *cpu_mask) +static int __init _bind_irq_vector(struct irq_desc *desc, int vector, + const cpumask_t *cpu_mask) { cpumask_t online_mask; int cpu; - struct irq_desc *desc = irq_to_desc(irq); - BUG_ON((unsigned)irq >= nr_irqs); BUG_ON((unsigned)vector >= NR_VECTORS); cpumask_and(&online_mask, cpu_mask, &cpu_online_map); @@ -129,9 +129,9 @@ static int __init __bind_irq_vector(int return 0; if ( desc->arch.vector != IRQ_VECTOR_UNASSIGNED ) return -EBUSY; - trace_irq_mask(TRC_HW_IRQ_BIND_VECTOR, irq, vector, &online_mask); + trace_irq_mask(TRC_HW_IRQ_BIND_VECTOR, desc->irq, vector, &online_mask); for_each_cpu(cpu, &online_mask) - per_cpu(vector_irq, cpu)[vector] = irq; + per_cpu(vector_irq, cpu)[vector] = desc->irq; desc->arch.vector = vector; cpumask_copy(desc->arch.cpu_mask, &online_mask); if ( desc->arch.used_vectors ) @@ -145,12 +145,18 @@ static int __init __bind_irq_vector(int int __init bind_irq_vector(int irq, int vector, const cpumask_t *cpu_mask) { + struct irq_desc *desc = irq_to_desc(irq); unsigned long flags; int ret; - spin_lock_irqsave(&vector_lock, flags); - ret = __bind_irq_vector(irq, vector, cpu_mask); - spin_unlock_irqrestore(&vector_lock, flags); + BUG_ON((unsigned)irq >= nr_irqs); + + spin_lock_irqsave(&desc->lock, flags); + spin_lock(&vector_lock); + ret = _bind_irq_vector(desc, vector, cpu_mask); + spin_unlock(&vector_lock); + spin_unlock_irqrestore(&desc->lock, flags); + return ret; } @@ -235,7 +241,9 @@ void destroy_irq(unsigned int irq) spin_lock_irqsave(&desc->lock, flags); desc->handler = &no_irq_type; - clear_irq_vector(irq); + spin_lock(&vector_lock); + _clear_irq_vector(desc); + spin_unlock(&vector_lock); desc->arch.used_vectors = NULL; spin_unlock_irqrestore(&desc->lock, flags); @@ -256,11 +264,11 @@ static void release_old_vec(struct irq_d } } -static void __clear_irq_vector(int irq) +static void _clear_irq_vector(struct irq_desc *desc) { - int cpu, vector, old_vector; + unsigned int cpu; + int vector, old_vector, irq = desc->irq; cpumask_t tmp_mask; - struct irq_desc *desc = irq_to_desc(irq); BUG_ON(!desc->arch.vector); @@ -306,11 +314,14 @@ static void __clear_irq_vector(int irq) void clear_irq_vector(int irq) { + struct irq_desc *desc = irq_to_desc(irq); unsigned long flags; - spin_lock_irqsave(&vector_lock, flags); - __clear_irq_vector(irq); - spin_unlock_irqrestore(&vector_lock, flags); + spin_lock_irqsave(&desc->lock, flags); + spin_lock(&vector_lock); + _clear_irq_vector(desc); + spin_unlock(&vector_lock); + spin_unlock_irqrestore(&desc->lock, flags); } int irq_to_vector(int irq) @@ -445,8 +456,7 @@ static vmask_t *irq_get_used_vector_mask return ret; } -static int __assign_irq_vector( - int irq, struct irq_desc *desc, const cpumask_t *mask) +static int _assign_irq_vector(struct irq_desc *desc, const cpumask_t *mask) { /* * NOTE! The local APIC isn't very good at handling @@ -460,7 +470,8 @@ static int __assign_irq_vector( * 0x80, because int 0x80 is hm, kind of importantish. ;) */ static int current_vector = FIRST_DYNAMIC_VECTOR, current_offset = 0; - int cpu, err, old_vector; + unsigned int cpu; + int err, old_vector, irq = desc->irq; cpumask_t tmp_mask; vmask_t *irq_used_vectors = NULL; @@ -570,8 +581,12 @@ int assign_irq_vector(int irq, const cpu BUG_ON(irq >= nr_irqs || irq <0); - spin_lock_irqsave(&vector_lock, flags); - ret = __assign_irq_vector(irq, desc, mask ?: TARGET_CPUS); + spin_lock_irqsave(&desc->lock, flags); + + spin_lock(&vector_lock); + ret = _assign_irq_vector(desc, mask ?: TARGET_CPUS); + spin_unlock(&vector_lock); + if ( !ret ) { ret = desc->arch.vector; @@ -580,7 +595,8 @@ int assign_irq_vector(int irq, const cpu else cpumask_setall(desc->affinity); } - spin_unlock_irqrestore(&vector_lock, flags); + + spin_unlock_irqrestore(&desc->lock, flags); return ret; } @@ -754,7 +770,6 @@ void irq_complete_move(struct irq_desc * unsigned int set_desc_affinity(struct irq_desc *desc, const cpumask_t *mask) { - unsigned int irq; int ret; unsigned long flags; cpumask_t dest_mask; @@ -762,10 +777,8 @@ unsigned int set_desc_affinity(struct ir if (!cpumask_intersects(mask, &cpu_online_map)) return BAD_APICID; - irq = desc->irq; - spin_lock_irqsave(&vector_lock, flags); - ret = __assign_irq_vector(irq, desc, mask); + ret = _assign_irq_vector(desc, mask); spin_unlock_irqrestore(&vector_lock, flags); if (ret < 0) From patchwork Mon Apr 29 11:25:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921691 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A1BB14D5 for ; Mon, 29 Apr 2019 11:27:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E9F9F2873D for ; Mon, 29 Apr 2019 11:27:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D8DBF2871E; Mon, 29 Apr 2019 11:27:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 832EB2871E for ; Mon, 29 Apr 2019 11:27:19 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Q3-0008CI-ST; Mon, 29 Apr 2019 11:25:59 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Q3-0008C9-2d for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:25:59 +0000 X-Inumbo-ID: 8f31a60a-6a71-11e9-843c-bc764e045a96 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 8f31a60a-6a71-11e9-843c-bc764e045a96; Mon, 29 Apr 2019 11:25:58 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:25:57 -0600 Message-Id: <5CC6DF3D0200007800229EBF@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:25:49 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 6/9] x86/IRQ: reduce unused space in struct arch_irq_desc X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- a/xen/include/asm-x86/irq.h +++ b/xen/include/asm-x86/irq.h @@ -35,8 +35,8 @@ struct arch_irq_desc { cpumask_var_t cpu_mask; cpumask_var_t old_cpu_mask; cpumask_var_t pending_mask; - unsigned move_cleanup_count; vmask_t *used_vectors; + unsigned move_cleanup_count; u8 move_in_progress : 1; s8 used; }; From patchwork Mon Apr 29 11:26:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921693 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A13351515 for ; Mon, 29 Apr 2019 11:27:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F0532871E for ; Mon, 29 Apr 2019 11:27:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 82266287F3; Mon, 29 Apr 2019 11:27:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 382AA2871E for ; Mon, 29 Apr 2019 11:27:59 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4QM-0008G1-5b; Mon, 29 Apr 2019 11:26:18 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4QL-0008Fm-8s for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:26:17 +0000 X-Inumbo-ID: 9986e5c4-6a71-11e9-9615-530d5c1f1a41 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 9986e5c4-6a71-11e9-9615-530d5c1f1a41; Mon, 29 Apr 2019 11:26:15 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:26:14 -0600 Message-Id: <5CC6DF520200007800229EC2@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:26:10 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 7/9] x86/IRQ: drop redundant cpumask_empty() from move_masked_irq() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP The subsequent cpumask_intersects() covers the "empty" case quite fine. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -638,9 +638,6 @@ void move_masked_irq(struct irq_desc *de desc->status &= ~IRQ_MOVE_PENDING; - if (unlikely(cpumask_empty(pending_mask))) - return; - if (!desc->handler->set_affinity) return; From patchwork Mon Apr 29 11:26:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921695 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7E2A1515 for ; Mon, 29 Apr 2019 11:28:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 975EF2871E for ; Mon, 29 Apr 2019 11:28:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8BFA1287F3; Mon, 29 Apr 2019 11:28:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B20F72871E for ; Mon, 29 Apr 2019 11:28:12 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Qq-0008Mw-Gt; Mon, 29 Apr 2019 11:26:48 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4Qp-0008Ml-8U for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:26:47 +0000 X-Inumbo-ID: ac0e6450-6a71-11e9-843c-bc764e045a96 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id ac0e6450-6a71-11e9-843c-bc764e045a96; Mon, 29 Apr 2019 11:26:46 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:26:46 -0600 Message-Id: <5CC6DF710200007800229EC5@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:26:41 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 8/9] x86/IRQ: make fixup_irqs() skip unconnected internally used interrupts X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Since the "Cannot set affinity ..." warning is a one time one, avoid triggering it already at boot time when parking secondary threads and the serial console uses a (still unconnected at that time) PCI IRQ. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- a/xen/arch/x86/irq.c +++ b/xen/arch/x86/irq.c @@ -2412,8 +2412,20 @@ void fixup_irqs(const cpumask_t *mask, b vector = irq_to_vector(irq); if ( vector >= FIRST_HIPRIORITY_VECTOR && vector <= LAST_HIPRIORITY_VECTOR ) + { cpumask_and(desc->arch.cpu_mask, desc->arch.cpu_mask, mask); + /* + * This can in particular happen when parking secondary threads + * during boot and when the serial console wants to use a PCI IRQ. + */ + if ( desc->handler == &no_irq_type ) + { + spin_unlock(&desc->lock); + continue; + } + } + if ( desc->arch.move_cleanup_count ) { /* The cleanup IPI may have got sent while we were still online. */ From patchwork Mon Apr 29 11:27:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 10921697 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FAE414D5 for ; Mon, 29 Apr 2019 11:28:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F2112873D for ; Mon, 29 Apr 2019 11:28:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 80C04287F9; Mon, 29 Apr 2019 11:28:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2A5272873D for ; Mon, 29 Apr 2019 11:28:51 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4RC-0008Uu-R9; Mon, 29 Apr 2019 11:27:10 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hL4RB-0008Tk-0X for xen-devel@lists.xenproject.org; Mon, 29 Apr 2019 11:27:09 +0000 X-Inumbo-ID: b8b0421a-6a71-11e9-a71e-cf8cc49792e7 Received: from prv1-mh.provo.novell.com (unknown [137.65.248.33]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id b8b0421a-6a71-11e9-a71e-cf8cc49792e7; Mon, 29 Apr 2019 11:27:08 +0000 (UTC) Received: from INET-PRV1-MTA by prv1-mh.provo.novell.com with Novell_GroupWise; Mon, 29 Apr 2019 05:27:07 -0600 Message-Id: <5CC6DF880200007800229EC8@prv1-mh.provo.novell.com> X-Mailer: Novell GroupWise Internet Agent 18.1.0 Date: Mon, 29 Apr 2019 05:27:04 -0600 From: "Jan Beulich" To: "xen-devel" References: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> In-Reply-To: <5CC6DD090200007800229E80@prv1-mh.provo.novell.com> Mime-Version: 1.0 Content-Disposition: inline Subject: [Xen-devel] [PATCH 9/9] x86/IO-APIC: drop an unused variable from setup_IO_APIC_irqs() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Must be a left-over from earlier days. Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -984,8 +984,6 @@ static void __init setup_IO_APIC_irqs(vo for (apic = 0; apic < nr_ioapics; apic++) { for (pin = 0; pin < nr_ioapic_entries[apic]; pin++) { - struct irq_desc *desc; - /* * add it to the IO-APIC irq-routing table: */ @@ -1038,7 +1036,6 @@ static void __init setup_IO_APIC_irqs(vo if (platform_legacy_irq(irq)) disable_8259A_irq(irq_to_desc(irq)); - desc = irq_to_desc(irq); SET_DEST(entry, logical, cpu_mask_to_apicid(TARGET_CPUS)); spin_lock_irqsave(&ioapic_lock, flags); __ioapic_write_entry(apic, pin, 0, entry);