mbox series

[v2,00/13] x86: more or less log-dirty related improvements

Message ID 0bebfe8c-6897-dc8b-7fe0-9127d4996eb8@suse.com (mailing list archive)
Headers show
Series x86: more or less log-dirty related improvements | expand

Message

Jan Beulich July 5, 2021, 3:09 p.m. UTC
... or so I hope. This series continues the attempt to deal with
the ovmf change putting the shared info page at a very high address
(which is now planned to get reverted there, but the general
problem doesn't go away by them doing so). There are further issues
with truncated value, which are being dealt with here. But there
are also not directly related changes, when I simply spotted things
that aren't very likely to be right the way they are. And then
there are also adjustments to the underlying hypervisor
implementation, with the goal of making the returned data more
useful to the consumers.

With these changes in place, a 1Gb guest which has "inflated"
itself by putting a page right below the 16Tb boundary migrates
successfully, albeit the process takes from some 20 minutes to over
half an hour on my test system.

In v2, besides integrating 2 patches that were previously sent,
there's one new patch and otherwise review feedback addressed
(albeit there wasn't any for a number of patches).

01: libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl
02: libxc: split xc_logdirty_control() from xc_shadow_control()
03: libxenguest: deal with log-dirty op stats overflow
04: libxenguest: short-circuit "all-dirty" handling
05: libxenguest: avoid allocating unused deferred-pages bitmap
06: libxenguest: complete loops in xc_map_domain_meminfo()
07: libxenguest: guard against overflow from too large p2m when checkpointing
08: libxenguest: fix off-by-1 in colo-secondary-bitmap merging
09: libxenguest: restrict PV guest size
10: libxc: simplify HYPERCALL_BUFFER()
11: x86/paging: supply more useful log-dirty page count
12: x86/mm: update log-dirty bitmap when manipulating P2M
13: SUPPORT.md: write down restriction of 32-bit tool stacks

Jan

Comments

Jan Beulich July 19, 2021, 7:46 a.m. UTC | #1
On 05.07.2021 17:09, Jan Beulich wrote:
> ... or so I hope. This series continues the attempt to deal with
> the ovmf change putting the shared info page at a very high address
> (which is now planned to get reverted there, but the general
> problem doesn't go away by them doing so). There are further issues
> with truncated value, which are being dealt with here. But there
> are also not directly related changes, when I simply spotted things
> that aren't very likely to be right the way they are. And then
> there are also adjustments to the underlying hypervisor
> implementation, with the goal of making the returned data more
> useful to the consumers.
> 
> With these changes in place, a 1Gb guest which has "inflated"
> itself by putting a page right below the 16Tb boundary migrates
> successfully, albeit the process takes from some 20 minutes to over
> half an hour on my test system.
> 
> In v2, besides integrating 2 patches that were previously sent,
> there's one new patch and otherwise review feedback addressed
> (albeit there wasn't any for a number of patches).
> 
> 01: libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl

while I did get an R-b from Anthony on this one, but ...

> 02: libxc: split xc_logdirty_control() from xc_shadow_control()
> 03: libxenguest: deal with log-dirty op stats overflow
> 04: libxenguest: short-circuit "all-dirty" handling
> 05: libxenguest: avoid allocating unused deferred-pages bitmap
> 06: libxenguest: complete loops in xc_map_domain_meminfo()
> 07: libxenguest: guard against overflow from too large p2m when checkpointing
> 08: libxenguest: fix off-by-1 in colo-secondary-bitmap merging
> 09: libxenguest: restrict PV guest size
> 10: libxc: simplify HYPERCALL_BUFFER()
> 11: x86/paging: supply more useful log-dirty page count
> 12: x86/mm: update log-dirty bitmap when manipulating P2M

... all of these are still in needed of suitable acks (patches 8
and 10 have an R-b though, and are independent of earlier parts of
this series). Patches 3 and 5 have objections pending by Andrew,
which I did reply to verbally without it having become clear
whether these replies were addressing the concerns, or what exactly
the misunderstanding on either side is (and hence which, if any,
changes I should make).

Thanks, Jan
Jan Beulich Aug. 13, 2021, 9:24 a.m. UTC | #2
On 19.07.2021 09:46, Jan Beulich wrote:
> On 05.07.2021 17:09, Jan Beulich wrote:
>> ... or so I hope. This series continues the attempt to deal with
>> the ovmf change putting the shared info page at a very high address
>> (which is now planned to get reverted there, but the general
>> problem doesn't go away by them doing so). There are further issues
>> with truncated value, which are being dealt with here. But there
>> are also not directly related changes, when I simply spotted things
>> that aren't very likely to be right the way they are. And then
>> there are also adjustments to the underlying hypervisor
>> implementation, with the goal of making the returned data more
>> useful to the consumers.
>>
>> With these changes in place, a 1Gb guest which has "inflated"
>> itself by putting a page right below the 16Tb boundary migrates
>> successfully, albeit the process takes from some 20 minutes to over
>> half an hour on my test system.
>>
>> In v2, besides integrating 2 patches that were previously sent,
>> there's one new patch and otherwise review feedback addressed
>> (albeit there wasn't any for a number of patches).
>>
>> 01: libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl
> 
> while I did get an R-b from Anthony on this one, but ...
> 
>> 02: libxc: split xc_logdirty_control() from xc_shadow_control()
>> 03: libxenguest: deal with log-dirty op stats overflow
>> 04: libxenguest: short-circuit "all-dirty" handling
>> 05: libxenguest: avoid allocating unused deferred-pages bitmap
>> 06: libxenguest: complete loops in xc_map_domain_meminfo()
>> 07: libxenguest: guard against overflow from too large p2m when checkpointing
>> 08: libxenguest: fix off-by-1 in colo-secondary-bitmap merging
>> 09: libxenguest: restrict PV guest size
>> 10: libxc: simplify HYPERCALL_BUFFER()
>> 11: x86/paging: supply more useful log-dirty page count
>> 12: x86/mm: update log-dirty bitmap when manipulating P2M
> 
> ... all of these are still in needed of suitable acks (patches 8
> and 10 have an R-b though, and are independent of earlier parts of
> this series).

Since I've not had any feedback since the ping, I'm intending to
commit the patches that have acks / R-b by Andrew (which actually
also includes patch 6), on the basis that informally (iirc
mentioned on irc more than once, in other but similar contexts)
this was indicated to be okay by the maintainers.

For all other patches here I'd like to renew the ping. I have to
admit that I'm unclear about what else I can do to progress this
series.

Jan

> Patches 3 and 5 have objections pending by Andrew,
> which I did reply to verbally without it having become clear
> whether these replies were addressing the concerns, or what exactly
> the misunderstanding on either side is (and hence which, if any,
> changes I should make).
> 
> Thanks, Jan
Jan Beulich Aug. 20, 2021, 7:20 a.m. UTC | #3
On 19.07.2021 09:46, Jan Beulich wrote:
> On 05.07.2021 17:09, Jan Beulich wrote:
>> ... or so I hope. This series continues the attempt to deal with
>> the ovmf change putting the shared info page at a very high address
>> (which is now planned to get reverted there, but the general
>> problem doesn't go away by them doing so). There are further issues
>> with truncated value, which are being dealt with here. But there
>> are also not directly related changes, when I simply spotted things
>> that aren't very likely to be right the way they are. And then
>> there are also adjustments to the underlying hypervisor
>> implementation, with the goal of making the returned data more
>> useful to the consumers.
>>
>> With these changes in place, a 1Gb guest which has "inflated"
>> itself by putting a page right below the 16Tb boundary migrates
>> successfully, albeit the process takes from some 20 minutes to over
>> half an hour on my test system.
>>
>> In v2, besides integrating 2 patches that were previously sent,
>> there's one new patch and otherwise review feedback addressed
>> (albeit there wasn't any for a number of patches).
>>
>> 01: libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl
> 
> while I did get an R-b from Anthony on this one, but ...
> 
>> 02: libxc: split xc_logdirty_control() from xc_shadow_control()
>> 03: libxenguest: deal with log-dirty op stats overflow
>> 04: libxenguest: short-circuit "all-dirty" handling
>> 05: libxenguest: avoid allocating unused deferred-pages bitmap

with Jürgen's R-b for 2, 4, and 5, may I ask for a maintainer ack on
at least patch 2? While I did address Andrew's comments on v1 of 4
and 5 (verbally), he didn't indicate back whether he was okay with
my replies, so some judgement may need applying there when deciding
whether to also give an ack there. Thanks.

Patch 3 remains controversial as it seems, unfortunately.

Jan