| Message ID | 1461326731-24270-1-git-send-email-ross.lagerwall@citrix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 04/22/2016 08:05 AM, Ross Lagerwall wrote: > 1fb3a8b2cfb2 ("xen/spinlock: Fix locking path engaging too soon under > PVHVM.") moved the initalization of the kicker interrupt until after > native_cpu_up() is called. However, when using qspinlocks, a CPU may try > to kick another CPU that is spinning (because it has not yet initialized > its kicker interrupt), resulting in the following bug during boot: > > [ 0.384000] kernel BUG at /build/linux-Ay7j_C/linux-4.4.0/drivers/xen/events/events_base.c:1210! > [ 0.384000] invalid opcode: 0000 [#1] SMP > [ 0.384000] Modules linked in: > [ 0.384000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.0-21-generic #37-Ubuntu > [ 0.384000] Hardware name: Xen HVM domU, BIOS 4.6.1-xs124820 04/20/2016 > [ 0.384000] task: ffff88003d758000 ti: ffff88003d760000 task.ti: ffff88003d760000 > [ 0.384000] RIP: 0010:[<ffffffff814c97c9>] [<ffffffff814c97c9>] xen_send_IPI_one+0x59/0x60 > [ 0.384000] RSP: 0000:ffff88003d763d30 EFLAGS: 00010086 > [ 0.384000] RAX: ffff88003da522fc RBX: 00000005528d31c0 RCX: 0000000000000001 > [ 0.384000] RDX: ffff88003da57840 RSI: 0000000000000003 RDI: 00000000ffffffff > [ 0.384000] RBP: ffff88003d763d30 R08: 0000000000000100 R09: ffff88003f7c7900 > [ 0.384000] R10: ffff88003da4a080 R11: ffff88003da4a060 R12: 000000000000a90d > [ 0.384000] R13: 000000055285a88a R14: 0000000000000001 R15: ffffffff820d3db0 > [ 0.384000] FS: 0000000000000000(0000) GS:ffff88003da00000(0000) knlGS:0000000000000000 > [ 0.384000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b > [ 0.384000] CR2: ffff880002200000 CR3: 0000000001e0a000 CR4: 00000000000006f0 > [ 0.384000] Stack: > [ 0.384000] ffff88003d763d40 ffffffff8102be9e ffff88003d763de0 ffffffff810cabc2 > [ 0.384000] ffffffff810ca6d1 ffff88003da4a060 ffff88003da4a080 0000000000000001 > [ 0.384000] ffff88003da4a080 ffffffff820d3db0 0000000000080000 0000000500000000 > [ 0.384000] Call Trace: > [ 0.384000] [<ffffffff8102be9e>] xen_qlock_kick+0xe/0x10 > [ 0.384000] [<ffffffff810cabc2>] __pv_queued_spin_unlock+0xb2/0xf0 > [ 0.384000] [<ffffffff810ca6d1>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20 > [ 0.384000] [<ffffffff81052936>] ? check_tsc_warp+0x76/0x150 > [ 0.384000] [<ffffffff81052aa6>] check_tsc_sync_source+0x96/0x160 > [ 0.384000] [<ffffffff81051e28>] native_cpu_up+0x3d8/0x9f0 > [ 0.384000] [<ffffffff8102b315>] xen_hvm_cpu_up+0x35/0x80 > [ 0.384000] [<ffffffff8108198c>] _cpu_up+0x13c/0x180 > [ 0.384000] [<ffffffff81081a4a>] cpu_up+0x7a/0xa0 > [ 0.384000] [<ffffffff81f80dfc>] smp_init+0x7f/0x81 > [ 0.384000] [<ffffffff81f5a121>] kernel_init_freeable+0xef/0x212 > [ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80 > [ 0.384000] [<ffffffff81817f3e>] kernel_init+0xe/0xe0 > [ 0.384000] [<ffffffff8182488f>] ret_from_fork+0x3f/0x70 > [ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80 > > To fix this, only send the kick if the target CPU's interrupt has been > initialized. This check isn't racy, because the target is waiting for > the spinlock, so it won't have initialized the interrupt in the > meantime. > > Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com> > --- > arch/x86/xen/spinlock.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/arch/x86/xen/spinlock.c b/arch/x86/xen/spinlock.c > index 9e2ba5c..f42e78d 100644 > --- a/arch/x86/xen/spinlock.c > +++ b/arch/x86/xen/spinlock.c > @@ -27,6 +27,12 @@ static bool xen_pvspin = true; > > static void xen_qlock_kick(int cpu) > { > + int irq = per_cpu(lock_kicker_irq, cpu); > + > + /* Don't kick if the target's kicker interrupt is not initialized. */ > + if (irq == -1) > + return; > + > xen_send_IPI_one(cpu, XEN_SPIN_UNLOCK_VECTOR); > } Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
diff --git a/arch/x86/xen/spinlock.c b/arch/x86/xen/spinlock.c index 9e2ba5c..f42e78d 100644 --- a/arch/x86/xen/spinlock.c +++ b/arch/x86/xen/spinlock.c @@ -27,6 +27,12 @@ static bool xen_pvspin = true; static void xen_qlock_kick(int cpu) { + int irq = per_cpu(lock_kicker_irq, cpu); + + /* Don't kick if the target's kicker interrupt is not initialized. */ + if (irq == -1) + return; + xen_send_IPI_one(cpu, XEN_SPIN_UNLOCK_VECTOR); }
1fb3a8b2cfb2 ("xen/spinlock: Fix locking path engaging too soon under PVHVM.") moved the initalization of the kicker interrupt until after native_cpu_up() is called. However, when using qspinlocks, a CPU may try to kick another CPU that is spinning (because it has not yet initialized its kicker interrupt), resulting in the following bug during boot: [ 0.384000] kernel BUG at /build/linux-Ay7j_C/linux-4.4.0/drivers/xen/events/events_base.c:1210! [ 0.384000] invalid opcode: 0000 [#1] SMP [ 0.384000] Modules linked in: [ 0.384000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.0-21-generic #37-Ubuntu [ 0.384000] Hardware name: Xen HVM domU, BIOS 4.6.1-xs124820 04/20/2016 [ 0.384000] task: ffff88003d758000 ti: ffff88003d760000 task.ti: ffff88003d760000 [ 0.384000] RIP: 0010:[<ffffffff814c97c9>] [<ffffffff814c97c9>] xen_send_IPI_one+0x59/0x60 [ 0.384000] RSP: 0000:ffff88003d763d30 EFLAGS: 00010086 [ 0.384000] RAX: ffff88003da522fc RBX: 00000005528d31c0 RCX: 0000000000000001 [ 0.384000] RDX: ffff88003da57840 RSI: 0000000000000003 RDI: 00000000ffffffff [ 0.384000] RBP: ffff88003d763d30 R08: 0000000000000100 R09: ffff88003f7c7900 [ 0.384000] R10: ffff88003da4a080 R11: ffff88003da4a060 R12: 000000000000a90d [ 0.384000] R13: 000000055285a88a R14: 0000000000000001 R15: ffffffff820d3db0 [ 0.384000] FS: 0000000000000000(0000) GS:ffff88003da00000(0000) knlGS:0000000000000000 [ 0.384000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.384000] CR2: ffff880002200000 CR3: 0000000001e0a000 CR4: 00000000000006f0 [ 0.384000] Stack: [ 0.384000] ffff88003d763d40 ffffffff8102be9e ffff88003d763de0 ffffffff810cabc2 [ 0.384000] ffffffff810ca6d1 ffff88003da4a060 ffff88003da4a080 0000000000000001 [ 0.384000] ffff88003da4a080 ffffffff820d3db0 0000000000080000 0000000500000000 [ 0.384000] Call Trace: [ 0.384000] [<ffffffff8102be9e>] xen_qlock_kick+0xe/0x10 [ 0.384000] [<ffffffff810cabc2>] __pv_queued_spin_unlock+0xb2/0xf0 [ 0.384000] [<ffffffff810ca6d1>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20 [ 0.384000] [<ffffffff81052936>] ? check_tsc_warp+0x76/0x150 [ 0.384000] [<ffffffff81052aa6>] check_tsc_sync_source+0x96/0x160 [ 0.384000] [<ffffffff81051e28>] native_cpu_up+0x3d8/0x9f0 [ 0.384000] [<ffffffff8102b315>] xen_hvm_cpu_up+0x35/0x80 [ 0.384000] [<ffffffff8108198c>] _cpu_up+0x13c/0x180 [ 0.384000] [<ffffffff81081a4a>] cpu_up+0x7a/0xa0 [ 0.384000] [<ffffffff81f80dfc>] smp_init+0x7f/0x81 [ 0.384000] [<ffffffff81f5a121>] kernel_init_freeable+0xef/0x212 [ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80 [ 0.384000] [<ffffffff81817f3e>] kernel_init+0xe/0xe0 [ 0.384000] [<ffffffff8182488f>] ret_from_fork+0x3f/0x70 [ 0.384000] [<ffffffff81817f30>] ? rest_init+0x80/0x80 To fix this, only send the kick if the target CPU's interrupt has been initialized. This check isn't racy, because the target is waiting for the spinlock, so it won't have initialized the interrupt in the meantime. Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com> --- arch/x86/xen/spinlock.c | 6 ++++++ 1 file changed, 6 insertions(+)