| Message ID | 20240320135720.68894-1-roger.pau@citrix.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs | expand |
On 20/03/2024 1:57 pm, Roger Pau Monne wrote: > There's no reason to force HVM guests to have a valid vcpu_info area when > initializing a vCPU, as the vCPU can also be brought online using the local > APIC, and on that path there's no requirement for vcpu_info to be setup ahead > of the bring up. Note an HVM vCPU can operate normally without making use of > vcpu_info. > > Restrict the check against dummy_vcpu_info to only apply to PV guests. > > Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Thanks for looking into this. But, do we actually need to force this on PV either? The only interesting user of dummy_vcpu_info now is vcpu_info_populate() which can cope with any arbitrary vCPU. I have a suspicion that we can (now) delete these two checks, delete the dummy_vcpu_info object, and use a regular NULL pointer in vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger pieces of PV-absurdity from Xen. I can entirely believe that this is only safe to do with Jan's physical registration changes in place. ~Andrew
On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote: > On 20/03/2024 1:57 pm, Roger Pau Monne wrote: > > There's no reason to force HVM guests to have a valid vcpu_info area when > > initializing a vCPU, as the vCPU can also be brought online using the local > > APIC, and on that path there's no requirement for vcpu_info to be setup ahead > > of the bring up. Note an HVM vCPU can operate normally without making use of > > vcpu_info. > > > > Restrict the check against dummy_vcpu_info to only apply to PV guests. > > > > Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') > > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> > > Thanks for looking into this. But, do we actually need to force this on > PV either? Possibly, having now taken a look at the users it does seem they could cope with unpopulated vcpu_info_area. Part of my understanding was that this was some kind of courtesy to PV guests in order to prevent starting them without a vcpu_info, which strictly speaking is not mandatory, but otherwise the guest vCPU won't be able to receive interrupts, not even IPIs. > The only interesting user of dummy_vcpu_info now is vcpu_info_populate() > which can cope with any arbitrary vCPU. > > I have a suspicion that we can (now) delete these two checks, delete the > dummy_vcpu_info object, and use a regular NULL pointer in > vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger > pieces of PV-absurdity from Xen. I was expecting this to be something we can backport. OTOH removing the check completely (or even getting rid of dummy_vcpu_info) is not something that we would want to backport. I would rather do the removal of dummy_vcpu_info as followup work. > I can entirely believe that this is only safe to do with Jan's physical > registration changes in place. Quite possible. Thanks, Roger.
On 20/03/2024 2:26 pm, Roger Pau Monné wrote: > On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote: >> On 20/03/2024 1:57 pm, Roger Pau Monne wrote: >>> There's no reason to force HVM guests to have a valid vcpu_info area when >>> initializing a vCPU, as the vCPU can also be brought online using the local >>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >>> of the bring up. Note an HVM vCPU can operate normally without making use of >>> vcpu_info. >>> >>> Restrict the check against dummy_vcpu_info to only apply to PV guests. >>> >>> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') >>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> >> Thanks for looking into this. But, do we actually need to force this on >> PV either? > Possibly, having now taken a look at the users it does seem they could > cope with unpopulated vcpu_info_area. > > Part of my understanding was that this was some kind of courtesy to PV > guests in order to prevent starting them without a vcpu_info, which > strictly speaking is not mandatory, but otherwise the guest vCPU won't > be able to receive interrupts, not even IPIs. That's more of a stick than a carrot... "you must set up the area of a CPU before you can bring it online". Except as we've seen, HVM has been fine all along without it. >> The only interesting user of dummy_vcpu_info now is vcpu_info_populate() >> which can cope with any arbitrary vCPU. >> >> I have a suspicion that we can (now) delete these two checks, delete the >> dummy_vcpu_info object, and use a regular NULL pointer in >> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger >> pieces of PV-absurdity from Xen. > I was expecting this to be something we can backport. OTOH removing > the check completely (or even getting rid of dummy_vcpu_info) is not > something that we would want to backport. > > I would rather do the removal of dummy_vcpu_info as followup work. I was worried about ARM with your patch, because it's spelt HVM there, rather than PV. However, I'd forgotten that ARM's do_vcpu_op() filters ops down to just VCPUOP_register_{vcpu_info,runstate_memory_area} so VCPUOP_initialise isn't reachable. Therefore, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> It also means ARM can't use any of the PHYS registration yet... ~Andrew
On 20.03.2024 15:06, Andrew Cooper wrote: > On 20/03/2024 1:57 pm, Roger Pau Monne wrote: >> There's no reason to force HVM guests to have a valid vcpu_info area when >> initializing a vCPU, as the vCPU can also be brought online using the local >> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >> of the bring up. Note an HVM vCPU can operate normally without making use of >> vcpu_info. >> >> Restrict the check against dummy_vcpu_info to only apply to PV guests. >> >> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') >> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> > > Thanks for looking into this. But, do we actually need to force this on > PV either? > > The only interesting user of dummy_vcpu_info now is vcpu_info_populate() > which can cope with any arbitrary vCPU. Why would that be the only interesting user? If we were to ... > I have a suspicion that we can (now) delete these two checks, delete the > dummy_vcpu_info object, and use a regular NULL pointer in > vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger > pieces of PV-absurdity from Xen. ... use NULL pointers instead, we'd need to audit all Xen's vcpu_info accesses for properly avoiding to de-reference that NULL. The dummy struct instance was really meant as a safety belt (besides also acting as a sentinel). > I can entirely believe that this is only safe to do with Jan's physical > registration changes in place. Which nobody uses just yet, afaik. Jan
On 20.03.2024 14:57, Roger Pau Monne wrote: > There's no reason to force HVM guests to have a valid vcpu_info area when > initializing a vCPU, as the vCPU can also be brought online using the local > APIC, and on that path there's no requirement for vcpu_info to be setup ahead > of the bring up. Note an HVM vCPU can operate normally without making use of > vcpu_info. While I'd agree if you started with "There's no real need to force ...", I still think there is a reason: If one wants to use paravirt interfaces (i.e. hypercalls), they would better do so consistently. After all there's also no need to use VCPUOP_initialise, yet you're not disabling its use. As said in reply to Andrew's reply, besides acting as a sentinel that structure instance also acts as a sink for Xen accesses to a vCPU's vcpu_info. By removing the check, you switch that from being a safeguard to being something that actually has to be expected to be accessed. Unless you've audited all uses to prove that no such access exists. Jan
On 20/03/2024 3:09 pm, Jan Beulich wrote: > On 20.03.2024 14:57, Roger Pau Monne wrote: >> There's no reason to force HVM guests to have a valid vcpu_info area when >> initializing a vCPU, as the vCPU can also be brought online using the local >> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >> of the bring up. Note an HVM vCPU can operate normally without making use of >> vcpu_info. > While I'd agree if you started with "There's no real need to force ...", I > still think there is a reason: If one wants to use paravirt interfaces (i.e. > hypercalls), they would better do so consistently. After all there's also > no need to use VCPUOP_initialise, yet you're not disabling its use. I firmly disagree. There are good reasons to use VCPUOP_initialise over INIT-SIPI-SIPI (like avoiding 16bit mode - in the case we want it here, to fix APIC_ID enumeration in a way that doesn't involve putting more complexity into HVMLoader), and forcing us to set up a useless structure before we can boot vCPU number 32 is just wrong. It was dumb to design a hypercall like this in the first place for PV guests, and it definitely isn't OK to keep guests broken because of it. ~Andrew
On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote: > On 20.03.2024 14:57, Roger Pau Monne wrote: > > There's no reason to force HVM guests to have a valid vcpu_info area when > > initializing a vCPU, as the vCPU can also be brought online using the local > > APIC, and on that path there's no requirement for vcpu_info to be setup ahead > > of the bring up. Note an HVM vCPU can operate normally without making use of > > vcpu_info. > > While I'd agree if you started with "There's no real need to force ...", I > still think there is a reason: If one wants to use paravirt interfaces (i.e. > hypercalls), they would better do so consistently. After all there's also > no need to use VCPUOP_initialise, yet you're not disabling its use. > > As said in reply to Andrew's reply, besides acting as a sentinel that > structure instance also acts as a sink for Xen accesses to a vCPU's > vcpu_info. By removing the check, you switch that from being a safeguard to > being something that actually has to be expected to be accessed. Unless > you've audited all uses to prove that no such access exists. I'm kind of lost in this last paragraph, how is that different than what currently happens when an HVM vCPU >= 32 is brought up using the lapic and has no vpcu_info mapped? Also, from a quick look it seems like sites do check whether vcpu_info == dummy_vcpu_info, otherwise we would already be in trouble. Thanks, Roger.
On 20.03.2024 16:20, Andrew Cooper wrote: > On 20/03/2024 3:09 pm, Jan Beulich wrote: >> On 20.03.2024 14:57, Roger Pau Monne wrote: >>> There's no reason to force HVM guests to have a valid vcpu_info area when >>> initializing a vCPU, as the vCPU can also be brought online using the local >>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >>> of the bring up. Note an HVM vCPU can operate normally without making use of >>> vcpu_info. >> While I'd agree if you started with "There's no real need to force ...", I >> still think there is a reason: If one wants to use paravirt interfaces (i.e. >> hypercalls), they would better do so consistently. After all there's also >> no need to use VCPUOP_initialise, yet you're not disabling its use. > > I firmly disagree. > > There are good reasons to use VCPUOP_initialise over INIT-SIPI-SIPI > (like avoiding 16bit mode - in the case we want it here, to fix APIC_ID > enumeration in a way that doesn't involve putting more complexity into > HVMLoader), and forcing us to set up a useless structure before we can > boot vCPU number 32 is just wrong. Just to mention it: I can certainly accept this as one possible valid viewpoint, and my looking at it differently is not an objection to the patch. It's just that the other aspect mentioned needs sorting (perhaps by just extending the patch description). > It was dumb to design a hypercall like this in the first place for PV > guests, and it definitely isn't OK to keep guests broken because of it. And again just to mention it: Originally, with a vCPU limit of 32, all vCPU-s would reliably have had vcpu_info (by way of that being embedded in shared_info). When raising the limit, the goal was to not chance overlooking any vcpu_info access in Xen. Hence why, instead of putting a NULL pointer there (or perhaps some non-canonical sentinel), the dummy approach was chosen. It then seemed quite desirable to prevent vCPU-s coming online without them first being detached from that dummy structure. And I think this firmly needs to continue to hold for PV. Jan
On 20.03.2024 17:29, Roger Pau Monné wrote: > On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote: >> On 20.03.2024 14:57, Roger Pau Monne wrote: >>> There's no reason to force HVM guests to have a valid vcpu_info area when >>> initializing a vCPU, as the vCPU can also be brought online using the local >>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >>> of the bring up. Note an HVM vCPU can operate normally without making use of >>> vcpu_info. >> >> While I'd agree if you started with "There's no real need to force ...", I >> still think there is a reason: If one wants to use paravirt interfaces (i.e. >> hypercalls), they would better do so consistently. After all there's also >> no need to use VCPUOP_initialise, yet you're not disabling its use. >> >> As said in reply to Andrew's reply, besides acting as a sentinel that >> structure instance also acts as a sink for Xen accesses to a vCPU's >> vcpu_info. By removing the check, you switch that from being a safeguard to >> being something that actually has to be expected to be accessed. Unless >> you've audited all uses to prove that no such access exists. > > I'm kind of lost in this last paragraph, how is that different than > what currently happens when an HVM vCPU >= 32 is brought up using the > lapic and has no vpcu_info mapped? I think this aspect was simply missed back at the time. And I think it wants mentioning explicitly to justify the change. As said in reply to Andrew, I don't think the dummy structure can be got rid of. Nor can the checks here be (easily) removed altogether, i.e. your change cannot (easily) be extended to PV as well. Even conditional removal of the structure in !PV builds would first require all vcpu_info accesses to gain a suitable conditional. Which may be undesirable, as some of these may be deemed fast paths. > Also, from a quick look it seems like sites do check whether vcpu_info > == dummy_vcpu_info, otherwise we would already be in trouble. Such checks exist in code managing vcpu_info, but not - afaics - in places actually accessing it. Jan
On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote: > On 20.03.2024 17:29, Roger Pau Monné wrote: > > On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote: > >> On 20.03.2024 14:57, Roger Pau Monne wrote: > >>> There's no reason to force HVM guests to have a valid vcpu_info area when > >>> initializing a vCPU, as the vCPU can also be brought online using the local > >>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead > >>> of the bring up. Note an HVM vCPU can operate normally without making use of > >>> vcpu_info. > >> > >> While I'd agree if you started with "There's no real need to force ...", I > >> still think there is a reason: If one wants to use paravirt interfaces (i.e. > >> hypercalls), they would better do so consistently. After all there's also > >> no need to use VCPUOP_initialise, yet you're not disabling its use. > >> > >> As said in reply to Andrew's reply, besides acting as a sentinel that > >> structure instance also acts as a sink for Xen accesses to a vCPU's > >> vcpu_info. By removing the check, you switch that from being a safeguard to > >> being something that actually has to be expected to be accessed. Unless > >> you've audited all uses to prove that no such access exists. > > > > I'm kind of lost in this last paragraph, how is that different than > > what currently happens when an HVM vCPU >= 32 is brought up using the > > lapic and has no vpcu_info mapped? > > I think this aspect was simply missed back at the time. And I think it > wants mentioning explicitly to justify the change. OK, I can add to the commit message: "Note an HVM vCPU can operate normally without making use of vcpu_info, and in fact does so when brought up from the local APIC." > As said in reply to Andrew, I don't think the dummy structure can be got > rid of. Nor can the checks here be (easily) removed altogether, i.e. your > change cannot (easily) be extended to PV as well. Even conditional removal > of the structure in !PV builds would first require all vcpu_info accesses > to gain a suitable conditional. Which may be undesirable, as some of these > may be deemed fast paths. I didn't intended to do this here, as replied to Andrew. If we want to get rid of the check for PV also it needs to be done in a different patch, and with a different justification and analysis. > > Also, from a quick look it seems like sites do check whether vcpu_info > > == dummy_vcpu_info, otherwise we would already be in trouble. > > Such checks exist in code managing vcpu_info, but not - afaics - in places > actually accessing it. Quite possibly, I didn't look that close TBH, since my intention was not to remove dummy_vcpu_info. I've noticed however that __update_vcpu_system_time() checks for v->vcpu_info_area.map == NULL, which is fine, but shouldn't it also check for v->vcpu_info_area.map == &dummy_vcpu_info, as it's pointless to update the vcpu system time if pointing to the dummy_vcpu_info? Thanks, Roger.
On 21.03.2024 10:10, Roger Pau Monné wrote: > On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote: >> On 20.03.2024 17:29, Roger Pau Monné wrote: >>> On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote: >>>> On 20.03.2024 14:57, Roger Pau Monne wrote: >>>>> There's no reason to force HVM guests to have a valid vcpu_info area when >>>>> initializing a vCPU, as the vCPU can also be brought online using the local >>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >>>>> of the bring up. Note an HVM vCPU can operate normally without making use of >>>>> vcpu_info. >>>> >>>> While I'd agree if you started with "There's no real need to force ...", I >>>> still think there is a reason: If one wants to use paravirt interfaces (i.e. >>>> hypercalls), they would better do so consistently. After all there's also >>>> no need to use VCPUOP_initialise, yet you're not disabling its use. >>>> >>>> As said in reply to Andrew's reply, besides acting as a sentinel that >>>> structure instance also acts as a sink for Xen accesses to a vCPU's >>>> vcpu_info. By removing the check, you switch that from being a safeguard to >>>> being something that actually has to be expected to be accessed. Unless >>>> you've audited all uses to prove that no such access exists. >>> >>> I'm kind of lost in this last paragraph, how is that different than >>> what currently happens when an HVM vCPU >= 32 is brought up using the >>> lapic and has no vpcu_info mapped? >> >> I think this aspect was simply missed back at the time. And I think it >> wants mentioning explicitly to justify the change. > > OK, I can add to the commit message: > > "Note an HVM vCPU can operate normally without making use of > vcpu_info, and in fact does so when brought up from the local APIC." I'd be fine adding this (or having this added) while committing. >> As said in reply to Andrew, I don't think the dummy structure can be got >> rid of. Nor can the checks here be (easily) removed altogether, i.e. your >> change cannot (easily) be extended to PV as well. Even conditional removal >> of the structure in !PV builds would first require all vcpu_info accesses >> to gain a suitable conditional. Which may be undesirable, as some of these >> may be deemed fast paths. > > I didn't intended to do this here, as replied to Andrew. If we want > to get rid of the check for PV also it needs to be done in a different > patch, and with a different justification and analysis. > >>> Also, from a quick look it seems like sites do check whether vcpu_info >>> == dummy_vcpu_info, otherwise we would already be in trouble. >> >> Such checks exist in code managing vcpu_info, but not - afaics - in places >> actually accessing it. > > Quite possibly, I didn't look that close TBH, since my intention was > not to remove dummy_vcpu_info. I've noticed however that > __update_vcpu_system_time() checks for v->vcpu_info_area.map == NULL, > which is fine, but shouldn't it also check for v->vcpu_info_area.map > == &dummy_vcpu_info, as it's pointless to update the vcpu system time > if pointing to the dummy_vcpu_info? The check is there to guard against NULL deref. As said, the aspect of a vCPU being brought up the "native" way yet then still using its vCPU info was, by mistake, neglected earlier on. So yes, such a check could be added here, but it isn't strictly necessary as long as we don't avoid accessing the dummy structure uniformly everywhere (which, as said, I'm not sure we want to do). Jan
On Thu, Mar 21, 2024 at 10:17:25AM +0100, Jan Beulich wrote: > On 21.03.2024 10:10, Roger Pau Monné wrote: > > On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote: > >> On 20.03.2024 17:29, Roger Pau Monné wrote: > >>> On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote: > >>>> On 20.03.2024 14:57, Roger Pau Monne wrote: > >>>>> There's no reason to force HVM guests to have a valid vcpu_info area when > >>>>> initializing a vCPU, as the vCPU can also be brought online using the local > >>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead > >>>>> of the bring up. Note an HVM vCPU can operate normally without making use of > >>>>> vcpu_info. > >>>> > >>>> While I'd agree if you started with "There's no real need to force ...", I > >>>> still think there is a reason: If one wants to use paravirt interfaces (i.e. > >>>> hypercalls), they would better do so consistently. After all there's also > >>>> no need to use VCPUOP_initialise, yet you're not disabling its use. > >>>> > >>>> As said in reply to Andrew's reply, besides acting as a sentinel that > >>>> structure instance also acts as a sink for Xen accesses to a vCPU's > >>>> vcpu_info. By removing the check, you switch that from being a safeguard to > >>>> being something that actually has to be expected to be accessed. Unless > >>>> you've audited all uses to prove that no such access exists. > >>> > >>> I'm kind of lost in this last paragraph, how is that different than > >>> what currently happens when an HVM vCPU >= 32 is brought up using the > >>> lapic and has no vpcu_info mapped? > >> > >> I think this aspect was simply missed back at the time. And I think it > >> wants mentioning explicitly to justify the change. > > > > OK, I can add to the commit message: > > > > "Note an HVM vCPU can operate normally without making use of > > vcpu_info, and in fact does so when brought up from the local APIC." > > I'd be fine adding this (or having this added) while committing. > > >> As said in reply to Andrew, I don't think the dummy structure can be got > >> rid of. Nor can the checks here be (easily) removed altogether, i.e. your > >> change cannot (easily) be extended to PV as well. Even conditional removal > >> of the structure in !PV builds would first require all vcpu_info accesses > >> to gain a suitable conditional. Which may be undesirable, as some of these > >> may be deemed fast paths. > > > > I didn't intended to do this here, as replied to Andrew. If we want > > to get rid of the check for PV also it needs to be done in a different > > patch, and with a different justification and analysis. > > > >>> Also, from a quick look it seems like sites do check whether vcpu_info > >>> == dummy_vcpu_info, otherwise we would already be in trouble. > >> > >> Such checks exist in code managing vcpu_info, but not - afaics - in places > >> actually accessing it. Is there anything else that needs adjusting then, or are you happy to pick this up and adjust the commit message? > > Quite possibly, I didn't look that close TBH, since my intention was > > not to remove dummy_vcpu_info. I've noticed however that > > __update_vcpu_system_time() checks for v->vcpu_info_area.map == NULL, > > which is fine, but shouldn't it also check for v->vcpu_info_area.map > > == &dummy_vcpu_info, as it's pointless to update the vcpu system time > > if pointing to the dummy_vcpu_info? > > The check is there to guard against NULL deref. As said, the aspect of a > vCPU being brought up the "native" way yet then still using its vCPU info > was, by mistake, neglected earlier on. So yes, such a check could be > added here, but it isn't strictly necessary as long as we don't avoid > accessing the dummy structure uniformly everywhere (which, as said, I'm > not sure we want to do). I could add such a check in a separate patch, my main concern with this is not correctness, it's just that we waste cycles updating the contents of dummy_vcpu_info which is useless. Thanks, Roger.
On 21.03.2024 10:57, Roger Pau Monné wrote: > On Thu, Mar 21, 2024 at 10:17:25AM +0100, Jan Beulich wrote: >> On 21.03.2024 10:10, Roger Pau Monné wrote: >>> On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote: >>>> On 20.03.2024 17:29, Roger Pau Monné wrote: >>>>> On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote: >>>>>> On 20.03.2024 14:57, Roger Pau Monne wrote: >>>>>>> There's no reason to force HVM guests to have a valid vcpu_info area when >>>>>>> initializing a vCPU, as the vCPU can also be brought online using the local >>>>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >>>>>>> of the bring up. Note an HVM vCPU can operate normally without making use of >>>>>>> vcpu_info. >>>>>> >>>>>> While I'd agree if you started with "There's no real need to force ...", I >>>>>> still think there is a reason: If one wants to use paravirt interfaces (i.e. >>>>>> hypercalls), they would better do so consistently. After all there's also >>>>>> no need to use VCPUOP_initialise, yet you're not disabling its use. >>>>>> >>>>>> As said in reply to Andrew's reply, besides acting as a sentinel that >>>>>> structure instance also acts as a sink for Xen accesses to a vCPU's >>>>>> vcpu_info. By removing the check, you switch that from being a safeguard to >>>>>> being something that actually has to be expected to be accessed. Unless >>>>>> you've audited all uses to prove that no such access exists. >>>>> >>>>> I'm kind of lost in this last paragraph, how is that different than >>>>> what currently happens when an HVM vCPU >= 32 is brought up using the >>>>> lapic and has no vpcu_info mapped? >>>> >>>> I think this aspect was simply missed back at the time. And I think it >>>> wants mentioning explicitly to justify the change. >>> >>> OK, I can add to the commit message: >>> >>> "Note an HVM vCPU can operate normally without making use of >>> vcpu_info, and in fact does so when brought up from the local APIC." >> >> I'd be fine adding this (or having this added) while committing. >> >>>> As said in reply to Andrew, I don't think the dummy structure can be got >>>> rid of. Nor can the checks here be (easily) removed altogether, i.e. your >>>> change cannot (easily) be extended to PV as well. Even conditional removal >>>> of the structure in !PV builds would first require all vcpu_info accesses >>>> to gain a suitable conditional. Which may be undesirable, as some of these >>>> may be deemed fast paths. >>> >>> I didn't intended to do this here, as replied to Andrew. If we want >>> to get rid of the check for PV also it needs to be done in a different >>> patch, and with a different justification and analysis. >>> >>>>> Also, from a quick look it seems like sites do check whether vcpu_info >>>>> == dummy_vcpu_info, otherwise we would already be in trouble. >>>> >>>> Such checks exist in code managing vcpu_info, but not - afaics - in places >>>> actually accessing it. > > Is there anything else that needs adjusting then, or are you happy to > pick this up and adjust the commit message? I'll pick this up and adjust (unless Andrew beats me). Jan
On 20.03.2024 14:57, Roger Pau Monne wrote: > There's no reason to force HVM guests to have a valid vcpu_info area when > initializing a vCPU, as the vCPU can also be brought online using the local > APIC, and on that path there's no requirement for vcpu_info to be setup ahead > of the bring up. Note an HVM vCPU can operate normally without making use of > vcpu_info. > > Restrict the check against dummy_vcpu_info to only apply to PV guests. > > Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> Despite the Fixes: tag I have to admit I'm uncertain of backporting here. HVM guests need to be prepared anyway that they may be denied use of this sub-op without first setting up vCPU info. Jan
Hi Andrew, On 20/03/2024 14:39, Andrew Cooper wrote: > On 20/03/2024 2:26 pm, Roger Pau Monné wrote: >> On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote: >>> On 20/03/2024 1:57 pm, Roger Pau Monne wrote: >>>> There's no reason to force HVM guests to have a valid vcpu_info area when >>>> initializing a vCPU, as the vCPU can also be brought online using the local >>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >>>> of the bring up. Note an HVM vCPU can operate normally without making use of >>>> vcpu_info. >>>> >>>> Restrict the check against dummy_vcpu_info to only apply to PV guests. >>>> >>>> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') >>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> >>> Thanks for looking into this. But, do we actually need to force this on >>> PV either? >> Possibly, having now taken a look at the users it does seem they could >> cope with unpopulated vcpu_info_area. >> >> Part of my understanding was that this was some kind of courtesy to PV >> guests in order to prevent starting them without a vcpu_info, which >> strictly speaking is not mandatory, but otherwise the guest vCPU won't >> be able to receive interrupts, not even IPIs. > > That's more of a stick than a carrot... "you must set up the area of a > CPU before you can bring it online". Except as we've seen, HVM has been > fine all along without it. >>> The only interesting user of dummy_vcpu_info now is vcpu_info_populate() >>> which can cope with any arbitrary vCPU. >>> >>> I have a suspicion that we can (now) delete these two checks, delete the >>> dummy_vcpu_info object, and use a regular NULL pointer in >>> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger >>> pieces of PV-absurdity from Xen. >> I was expecting this to be something we can backport. OTOH removing >> the check completely (or even getting rid of dummy_vcpu_info) is not >> something that we would want to backport. >> >> I would rather do the removal of dummy_vcpu_info as followup work. > > I was worried about ARM with your patch, because it's spelt HVM there, > rather than PV. > > However, I'd forgotten that ARM's do_vcpu_op() filters ops down to just > VCPUOP_register_{vcpu_info,runstate_memory_area} so VCPUOP_initialise > isn't reachable. > > Therefore, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> > > It also means ARM can't use any of the PHYS registration yet... Whoops. I don't think this was intended. Jan, I don't seem to find any use in Linux. Do you have any patches you could share? I would like to give a try on Arm before sending a patch? Cheers,
On 26.03.2024 23:08, Julien Grall wrote: > Hi Andrew, > > On 20/03/2024 14:39, Andrew Cooper wrote: >> On 20/03/2024 2:26 pm, Roger Pau Monné wrote: >>> On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote: >>>> On 20/03/2024 1:57 pm, Roger Pau Monne wrote: >>>>> There's no reason to force HVM guests to have a valid vcpu_info area when >>>>> initializing a vCPU, as the vCPU can also be brought online using the local >>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead >>>>> of the bring up. Note an HVM vCPU can operate normally without making use of >>>>> vcpu_info. >>>>> >>>>> Restrict the check against dummy_vcpu_info to only apply to PV guests. >>>>> >>>>> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') >>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> >>>> Thanks for looking into this. But, do we actually need to force this on >>>> PV either? >>> Possibly, having now taken a look at the users it does seem they could >>> cope with unpopulated vcpu_info_area. >>> >>> Part of my understanding was that this was some kind of courtesy to PV >>> guests in order to prevent starting them without a vcpu_info, which >>> strictly speaking is not mandatory, but otherwise the guest vCPU won't >>> be able to receive interrupts, not even IPIs. >> >> That's more of a stick than a carrot... "you must set up the area of a >> CPU before you can bring it online". Except as we've seen, HVM has been >> fine all along without it. >>>> The only interesting user of dummy_vcpu_info now is vcpu_info_populate() >>>> which can cope with any arbitrary vCPU. >>>> >>>> I have a suspicion that we can (now) delete these two checks, delete the >>>> dummy_vcpu_info object, and use a regular NULL pointer in >>>> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger >>>> pieces of PV-absurdity from Xen. >>> I was expecting this to be something we can backport. OTOH removing >>> the check completely (or even getting rid of dummy_vcpu_info) is not >>> something that we would want to backport. >>> >>> I would rather do the removal of dummy_vcpu_info as followup work. >> >> I was worried about ARM with your patch, because it's spelt HVM there, >> rather than PV. >> >> However, I'd forgotten that ARM's do_vcpu_op() filters ops down to just >> VCPUOP_register_{vcpu_info,runstate_memory_area} so VCPUOP_initialise >> isn't reachable. >> >> Therefore, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com> >> >> It also means ARM can't use any of the PHYS registration yet... > > Whoops. I don't think this was intended. Jan, I don't seem to find any > use in Linux. Do you have any patches you could share? No, I don't. I did all development with hacked up XTF tests, and I was expecting Linux folks to be looking into making use of the new subops. Jan > I would like to > give a try on Arm before sending a patch? > > Cheers, >
diff --git a/xen/common/compat/domain.c b/xen/common/compat/domain.c index 7ff238cc2656..6b4afc823217 100644 --- a/xen/common/compat/domain.c +++ b/xen/common/compat/domain.c @@ -49,7 +49,7 @@ int compat_common_vcpu_op(int cmd, struct vcpu *v, { case VCPUOP_initialise: { - if ( v->vcpu_info_area.map == &dummy_vcpu_info ) + if ( is_pv_domain(d) && v->vcpu_info_area.map == &dummy_vcpu_info ) return -EINVAL; #ifdef CONFIG_HVM diff --git a/xen/common/domain.c b/xen/common/domain.c index f6f557499660..d956dc09eca0 100644 --- a/xen/common/domain.c +++ b/xen/common/domain.c @@ -1817,7 +1817,7 @@ long common_vcpu_op(int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg) switch ( cmd ) { case VCPUOP_initialise: - if ( v->vcpu_info_area.map == &dummy_vcpu_info ) + if ( is_pv_domain(d) && v->vcpu_info_area.map == &dummy_vcpu_info ) return -EINVAL; rc = arch_initialise_vcpu(v, arg);
There's no reason to force HVM guests to have a valid vcpu_info area when initializing a vCPU, as the vCPU can also be brought online using the local APIC, and on that path there's no requirement for vcpu_info to be setup ahead of the bring up. Note an HVM vCPU can operate normally without making use of vcpu_info. Restrict the check against dummy_vcpu_info to only apply to PV guests. Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs') Signed-off-by: Roger Pau Monné <roger.pau@citrix.com> --- xen/common/compat/domain.c | 2 +- xen/common/domain.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)