| Message ID | b0bd0d34-548d-b461-7397-9ab92fb4e6fc@de.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 10/30/2017 02:48 PM, Christian Borntraeger wrote: > > > On 10/30/2017 02:44 PM, Pierre Morel wrote: >> On 30/10/2017 13:44, Christian Borntraeger wrote: >>> >>> >>> On 10/30/2017 01:42 PM, Cornelia Huck wrote: >>>> On Mon, 30 Oct 2017 09:28:09 +0100 >>>> Christian Borntraeger <borntraeger@de.ibm.com> wrote: >>>> >>>>> Now I thought about that for a while and I start to think that we cannot implement ais >>>>> in QEMU and cover all cases. >>>>> One aspect was certainly passthrough (like you handled in patch 4). >>>>> Another aspect is that some interrupts might be injected from the kernel - even for >>>>> emulated devices. e.g. virtio-pci together with vhost-net, will inject interrupts via >>>>> the set_irq callback. I think disabling irqfd for these cases is not a good idea. >>>> >>>> Is there still a fallback for irqfd emulation? >>> >>> it might disable dataplane or other things. (it once did). So I think we should not >>> go down this path. >>> >>>> >>>>> >>>>> So what about adding a new KVM capability (for 4.14), fixup the other things in >>>>> QEMU and then bind it to the new capability? >>>> >>>> For 4.15, surely? >>>> >>>> Probably the only way we can make this work correctly... >>>> >> >> I may have not understand. >> >> Why do we need a new capability, we already have the KVM_CAP_S390_AIS capability? > > To mark a kernel that supports AIS+migration without having to instantiate a flic device. > It's about libvirt probing AFAIU. Right now, to tell if we have migration for AIS we need a flic device instantiated in the kernel (we need the fd to tell if the flic attributes are supported by the kernel). > >> The PCI device has a netdev property pointing to a netdev, if this netdev sets the vhost property, can't we test this to know if we can realize this device or not? >> >> Using virtio-pci instead of virtio-ccw is not the first choice for S390. The use case I see for S390 using virtio-pci is as a fallback in case for the migration of a PCI device the target host does not support AIS or do not have VFIO device and one do not want to modify the guest. > > This was just one example. Having the interrupt controller in the kernel, implementing AIS in > qemu is very prone to break something that we have forgotten about. > I tend to agree with Christian. While doing emulation AIS in QEMU for scenarios where all adapter interrupts (subject to AIS) are guaranteed to go through QEMU is possible, it is also bound to introduce a whole lot of added complexity. Frankly, I doubt the gain outweighs the pain in this case. Regards, Halil
On Mon, 30 Oct 2017 14:48:23 +0100 Christian Borntraeger <borntraeger@de.ibm.com> wrote: > FWIW, I am testing a guest patch that enables zPCI without AIS. Its as simple as > > > diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c > index 7b30af5..9b24836 100644 > --- a/arch/s390/pci/pci.c > +++ b/arch/s390/pci/pci.c > @@ -953,7 +953,7 @@ static int __init pci_base_init(void) > if (!s390_pci_probe) > return 0; > > - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) > + if (!test_facility(69) || !test_facility(71)) > return 0; > > rc = zpci_debug_init(); > diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c > index ea34086..61f8c82 100644 > --- a/arch/s390/pci/pci_insn.c > +++ b/arch/s390/pci/pci_insn.c > @@ -7,6 +7,7 @@ > #include <linux/export.h> > #include <linux/errno.h> > #include <linux/delay.h> > +#include <asm/facility.h> > #include <asm/pci_insn.h> > #include <asm/pci_debug.h> > #include <asm/processor.h> > @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) > /* Set Interruption Controls */ > void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) > { > + if (!test_facility(72)) > + return; > asm volatile ( > " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" > : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused)); > Sounds good. Presumably this makes the adapter interrupt handling work as for virtio (and qdio)? Is there any ais-less pci hardware out in the wild?
On 10/30/2017 05:59 PM, Cornelia Huck wrote: > On Mon, 30 Oct 2017 14:48:23 +0100 > Christian Borntraeger <borntraeger@de.ibm.com> wrote: > > >> FWIW, I am testing a guest patch that enables zPCI without AIS. Its as simple as >> >> >> diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c >> index 7b30af5..9b24836 100644 >> --- a/arch/s390/pci/pci.c >> +++ b/arch/s390/pci/pci.c >> @@ -953,7 +953,7 @@ static int __init pci_base_init(void) >> if (!s390_pci_probe) >> return 0; >> >> - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) >> + if (!test_facility(69) || !test_facility(71)) >> return 0; >> >> rc = zpci_debug_init(); >> diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c >> index ea34086..61f8c82 100644 >> --- a/arch/s390/pci/pci_insn.c >> +++ b/arch/s390/pci/pci_insn.c >> @@ -7,6 +7,7 @@ >> #include <linux/export.h> >> #include <linux/errno.h> >> #include <linux/delay.h> >> +#include <asm/facility.h> >> #include <asm/pci_insn.h> >> #include <asm/pci_debug.h> >> #include <asm/processor.h> >> @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) >> /* Set Interruption Controls */ >> void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) >> { >> + if (!test_facility(72)) >> + return; >> asm volatile ( >> " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" >> : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused)); >> > > Sounds good. Presumably this makes the adapter interrupt handling work > as for virtio (and qdio)? Is there any ais-less pci hardware out in the > wild? > ais is z specific, not PCI specific. So PCI cards should not care as far as I can tell.
On 30/10/2017 18:08, Christian Borntraeger wrote: > > On 10/30/2017 05:59 PM, Cornelia Huck wrote: >> On Mon, 30 Oct 2017 14:48:23 +0100 >> Christian Borntraeger <borntraeger@de.ibm.com> wrote: >> >> >>> FWIW, I am testing a guest patch that enables zPCI without AIS. Its as simple as >>> >>> >>> diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c >>> index 7b30af5..9b24836 100644 >>> --- a/arch/s390/pci/pci.c >>> +++ b/arch/s390/pci/pci.c >>> @@ -953,7 +953,7 @@ static int __init pci_base_init(void) >>> if (!s390_pci_probe) >>> return 0; >>> >>> - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) >>> + if (!test_facility(69) || !test_facility(71)) >>> return 0; >>> >>> rc = zpci_debug_init(); >>> diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c >>> index ea34086..61f8c82 100644 >>> --- a/arch/s390/pci/pci_insn.c >>> +++ b/arch/s390/pci/pci_insn.c >>> @@ -7,6 +7,7 @@ >>> #include <linux/export.h> >>> #include <linux/errno.h> >>> #include <linux/delay.h> >>> +#include <asm/facility.h> >>> #include <asm/pci_insn.h> >>> #include <asm/pci_debug.h> >>> #include <asm/processor.h> >>> @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) >>> /* Set Interruption Controls */ >>> void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) >>> { >>> + if (!test_facility(72)) >>> + return; >>> asm volatile ( >>> " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" >>> : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused)); >>> >> >> Sounds good. Presumably this makes the adapter interrupt handling work >> as for virtio (and qdio)? Is there any ais-less pci hardware out in the >> wild? >> > > ais is z specific, not PCI specific. So PCI cards should not care as far as I can tell. > > I bet Conny meant PCI hardware globaly, including zPCI adapters. I bet none but I will better ask Sebastian or Gerald if they know.
On 10/30/2017 06:38 PM, Pierre Morel wrote: > On 30/10/2017 18:08, Christian Borntraeger wrote: >> >> On 10/30/2017 05:59 PM, Cornelia Huck wrote: >>> On Mon, 30 Oct 2017 14:48:23 +0100 >>> Christian Borntraeger <borntraeger@de.ibm.com> wrote: >>> >>> >>>> FWIW, I am testing a guest patch that enables zPCI without AIS. Its as simple as >>>> >>>> >>>> diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c >>>> index 7b30af5..9b24836 100644 >>>> --- a/arch/s390/pci/pci.c >>>> +++ b/arch/s390/pci/pci.c >>>> @@ -953,7 +953,7 @@ static int __init pci_base_init(void) >>>> if (!s390_pci_probe) >>>> return 0; >>>> - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) >>>> + if (!test_facility(69) || !test_facility(71)) >>>> return 0; >>>> rc = zpci_debug_init(); >>>> diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c >>>> index ea34086..61f8c82 100644 >>>> --- a/arch/s390/pci/pci_insn.c >>>> +++ b/arch/s390/pci/pci_insn.c >>>> @@ -7,6 +7,7 @@ >>>> #include <linux/export.h> >>>> #include <linux/errno.h> >>>> #include <linux/delay.h> >>>> +#include <asm/facility.h> >>>> #include <asm/pci_insn.h> >>>> #include <asm/pci_debug.h> >>>> #include <asm/processor.h> >>>> @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) >>>> /* Set Interruption Controls */ >>>> void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) >>>> { >>>> + if (!test_facility(72)) >>>> + return; >>>> asm volatile ( >>>> " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" >>>> : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused)); >>>> >>> >>> Sounds good. Presumably this makes the adapter interrupt handling work >>> as for virtio (and qdio)? Is there any ais-less pci hardware out in the >>> wild? @Connie: Do you have something in mind, or is curiosity the only reason for asking? >>> >> >> ais is z specific, not PCI specific. So PCI cards should not care as far as I can tell. >> >> > > I bet Conny meant PCI hardware globaly, including zPCI adapters. > I bet none but I will better ask Sebastian or Gerald if they know. > I side with Christian: it is a machine thing and not a card thing. So the question is if there is a z machine which supports zpci but does not support AIS. My guess is that such a machine was never produced. Halil
On Mon, 30 Oct 2017 19:58:15 +0100 Halil Pasic <pasic@linux.vnet.ibm.com> wrote: > On 10/30/2017 06:38 PM, Pierre Morel wrote: > > On 30/10/2017 18:08, Christian Borntraeger wrote: > >> > >> On 10/30/2017 05:59 PM, Cornelia Huck wrote: > >>> On Mon, 30 Oct 2017 14:48:23 +0100 > >>> Christian Borntraeger <borntraeger@de.ibm.com> wrote: > >>> > >>> > >>>> FWIW, I am testing a guest patch that enables zPCI without AIS. Its as simple as > >>>> > >>>> > >>>> diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c > >>>> index 7b30af5..9b24836 100644 > >>>> --- a/arch/s390/pci/pci.c > >>>> +++ b/arch/s390/pci/pci.c > >>>> @@ -953,7 +953,7 @@ static int __init pci_base_init(void) > >>>> if (!s390_pci_probe) > >>>> return 0; > >>>> - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) > >>>> + if (!test_facility(69) || !test_facility(71)) > >>>> return 0; > >>>> rc = zpci_debug_init(); > >>>> diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c > >>>> index ea34086..61f8c82 100644 > >>>> --- a/arch/s390/pci/pci_insn.c > >>>> +++ b/arch/s390/pci/pci_insn.c > >>>> @@ -7,6 +7,7 @@ > >>>> #include <linux/export.h> > >>>> #include <linux/errno.h> > >>>> #include <linux/delay.h> > >>>> +#include <asm/facility.h> > >>>> #include <asm/pci_insn.h> > >>>> #include <asm/pci_debug.h> > >>>> #include <asm/processor.h> > >>>> @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) > >>>> /* Set Interruption Controls */ > >>>> void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) > >>>> { > >>>> + if (!test_facility(72)) > >>>> + return; > >>>> asm volatile ( > >>>> " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" > >>>> : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused)); > >>>> > >>> > >>> Sounds good. Presumably this makes the adapter interrupt handling work > >>> as for virtio (and qdio)? Is there any ais-less pci hardware out in the > >>> wild? > > @Connie: > Do you have something in mind, or is curiosity the only reason for asking? I just want to make sure we don't miss anything. > > >>> > >> > >> ais is z specific, not PCI specific. So PCI cards should not care as far as I can tell. > >> > >> > > > > I bet Conny meant PCI hardware globaly, including zPCI adapters. > > I bet none but I will better ask Sebastian or Gerald if they know. > > > > I side with Christian: it is a machine thing and not a card thing. So > the question is if there is a z machine which supports zpci but does > not support AIS. My guess is that such a machine was never produced. That's exactly what I meant, and I would not expect such a machine to exist either.
On 11/06/2017 09:52 AM, Cornelia Huck wrote: > On Mon, 30 Oct 2017 19:58:15 +0100 > Halil Pasic <pasic@linux.vnet.ibm.com> wrote: > >> On 10/30/2017 06:38 PM, Pierre Morel wrote: >>> On 30/10/2017 18:08, Christian Borntraeger wrote: >>>> >>>> On 10/30/2017 05:59 PM, Cornelia Huck wrote: >>>>> On Mon, 30 Oct 2017 14:48:23 +0100 >>>>> Christian Borntraeger <borntraeger@de.ibm.com> wrote: >>>>> >>>>> >>>>>> FWIW, I am testing a guest patch that enables zPCI without AIS. Its as simple as >>>>>> >>>>>> >>>>>> diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c >>>>>> index 7b30af5..9b24836 100644 >>>>>> --- a/arch/s390/pci/pci.c >>>>>> +++ b/arch/s390/pci/pci.c >>>>>> @@ -953,7 +953,7 @@ static int __init pci_base_init(void) >>>>>> if (!s390_pci_probe) >>>>>> return 0; >>>>>> - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) >>>>>> + if (!test_facility(69) || !test_facility(71)) >>>>>> return 0; >>>>>> rc = zpci_debug_init(); >>>>>> diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c >>>>>> index ea34086..61f8c82 100644 >>>>>> --- a/arch/s390/pci/pci_insn.c >>>>>> +++ b/arch/s390/pci/pci_insn.c >>>>>> @@ -7,6 +7,7 @@ >>>>>> #include <linux/export.h> >>>>>> #include <linux/errno.h> >>>>>> #include <linux/delay.h> >>>>>> +#include <asm/facility.h> >>>>>> #include <asm/pci_insn.h> >>>>>> #include <asm/pci_debug.h> >>>>>> #include <asm/processor.h> >>>>>> @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) >>>>>> /* Set Interruption Controls */ >>>>>> void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) >>>>>> { >>>>>> + if (!test_facility(72)) >>>>>> + return; >>>>>> asm volatile ( >>>>>> " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" >>>>>> : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused)); >>>>>> >>>>> >>>>> Sounds good. Presumably this makes the adapter interrupt handling work >>>>> as for virtio (and qdio)? Is there any ais-less pci hardware out in the >>>>> wild? >> >> @Connie: >> Do you have something in mind, or is curiosity the only reason for asking? > > I just want to make sure we don't miss anything. > >> >>>>> >>>> >>>> ais is z specific, not PCI specific. So PCI cards should not care as far as I can tell. >>>> >>>> >>> >>> I bet Conny meant PCI hardware globaly, including zPCI adapters. >>> I bet none but I will better ask Sebastian or Gerald if they know. >>> >> >> I side with Christian: it is a machine thing and not a card thing. So >> the question is if there is a z machine which supports zpci but does >> not support AIS. My guess is that such a machine was never produced. > > That's exactly what I meant, and I would not expect such a machine to > exist either. That machine is called qemu 2.10 :-p FWIW, I talked with Sebastian about that and he is fine with such a change. He required some changes and I will work on such a patch.
On 06/11/2017 09:54, Christian Borntraeger wrote: > > > On 11/06/2017 09:52 AM, Cornelia Huck wrote: >> On Mon, 30 Oct 2017 19:58:15 +0100 >> Halil Pasic <pasic@linux.vnet.ibm.com> wrote: >> >>> On 10/30/2017 06:38 PM, Pierre Morel wrote: >>>> On 30/10/2017 18:08, Christian Borntraeger wrote: >>>>> >>>>> On 10/30/2017 05:59 PM, Cornelia Huck wrote: >>>>>> On Mon, 30 Oct 2017 14:48:23 +0100 >>>>>> Christian Borntraeger <borntraeger@de.ibm.com> wrote: >>>>>> >>>>>> >>>>>>> FWIW, I am testing a guest patch that enables zPCI without AIS. Its as simple as >>>>>>> >>>>>>> >>>>>>> diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c >>>>>>> index 7b30af5..9b24836 100644 >>>>>>> --- a/arch/s390/pci/pci.c >>>>>>> +++ b/arch/s390/pci/pci.c >>>>>>> @@ -953,7 +953,7 @@ static int __init pci_base_init(void) >>>>>>> if (!s390_pci_probe) >>>>>>> return 0; >>>>>>> - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) >>>>>>> + if (!test_facility(69) || !test_facility(71)) >>>>>>> return 0; >>>>>>> rc = zpci_debug_init(); >>>>>>> diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c >>>>>>> index ea34086..61f8c82 100644 >>>>>>> --- a/arch/s390/pci/pci_insn.c >>>>>>> +++ b/arch/s390/pci/pci_insn.c >>>>>>> @@ -7,6 +7,7 @@ >>>>>>> #include <linux/export.h> >>>>>>> #include <linux/errno.h> >>>>>>> #include <linux/delay.h> >>>>>>> +#include <asm/facility.h> >>>>>>> #include <asm/pci_insn.h> >>>>>>> #include <asm/pci_debug.h> >>>>>>> #include <asm/processor.h> >>>>>>> @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) >>>>>>> /* Set Interruption Controls */ >>>>>>> void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) >>>>>>> { >>>>>>> + if (!test_facility(72)) >>>>>>> + return; >>>>>>> asm volatile ( >>>>>>> " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" >>>>>>> : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused)); >>>>>>> >>>>>> >>>>>> Sounds good. Presumably this makes the adapter interrupt handling work >>>>>> as for virtio (and qdio)? Is there any ais-less pci hardware out in the >>>>>> wild? >>> >>> @Connie: >>> Do you have something in mind, or is curiosity the only reason for asking? >> >> I just want to make sure we don't miss anything. >> >>> >>>>>> >>>>> >>>>> ais is z specific, not PCI specific. So PCI cards should not care as far as I can tell. >>>>> >>>>> >>>> >>>> I bet Conny meant PCI hardware globaly, including zPCI adapters. >>>> I bet none but I will better ask Sebastian or Gerald if they know. >>>> >>> >>> I side with Christian: it is a machine thing and not a card thing. So >>> the question is if there is a z machine which supports zpci but does >>> not support AIS. My guess is that such a machine was never produced. >> >> That's exactly what I meant, and I would not expect such a machine to >> exist either. > > That machine is called qemu 2.10 :-p > > > > FWIW, I talked with Sebastian about that and he is fine with such a change. He > required some changes and I will work on such a patch. > > My analyze is that providing AIS to the guest can work, with the precaution to handle spurious interrupts, until we use GISA for PCI. Then when we introduce GISA for PCI, we will need to think again about providing AIS to the guest.
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 7b30af5..9b24836 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -953,7 +953,7 @@ static int __init pci_base_init(void) if (!s390_pci_probe) return 0; - if (!test_facility(69) || !test_facility(71) || !test_facility(72)) + if (!test_facility(69) || !test_facility(71)) return 0; rc = zpci_debug_init(); diff --git a/arch/s390/pci/pci_insn.c b/arch/s390/pci/pci_insn.c index ea34086..61f8c82 100644 --- a/arch/s390/pci/pci_insn.c +++ b/arch/s390/pci/pci_insn.c @@ -7,6 +7,7 @@ #include <linux/export.h> #include <linux/errno.h> #include <linux/delay.h> +#include <asm/facility.h> #include <asm/pci_insn.h> #include <asm/pci_debug.h> #include <asm/processor.h> @@ -93,6 +94,8 @@ int zpci_refresh_trans(u64 fn, u64 addr, u64 range) /* Set Interruption Controls */ void zpci_set_irq_ctrl(u16 ctl, char *unused, u8 isc) { + if (!test_facility(72)) + return; asm volatile ( " .insn rsy,0xeb00000000d1,%[ctl],%[isc],%[u]\n" : : [ctl] "d" (ctl), [isc] "d" (isc << 27), [u] "Q" (*unused));