diff mbox series

[06/12] docs/migration Specify migration v3 and STATIC_DATA_END

Message ID 20191224151932.6304-7-andrew.cooper3@citrix.com (mailing list archive)
State Superseded
Headers show
Series Support CPUID/MSR data in migration streams | expand

Commit Message

Andrew Cooper Dec. 24, 2019, 3:19 p.m. UTC
Migration data can be split into two parts - that which is invariant of
guest execution, and that which is not.  Separate these two with the
STATIC_DATA_END record.

The short term, we want to move the x86 CPU Policy data into the stream.
In the longer term, we want to provisionally send the static data only
to the destination as a more robust compatibility check.  In both cases,
we will want a callback into the higher level toolstack.

Mandate the presence of the STATIC_DATA_END record, and declare this v3,
along with instructions for how to compatibly interpret a v2 stream.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 docs/specs/libxc-migration-stream.pandoc | 39 +++++++++++++++++++++++++++++---
 tools/libxc/xc_sr_common.c               |  1 +
 tools/libxc/xc_sr_stream_format.h        |  1 +
 tools/python/xen/migration/libxc.py      |  2 ++
 4 files changed, 40 insertions(+), 3 deletions(-)

Comments

Jan Beulich Jan. 3, 2020, 2:44 p.m. UTC | #1
On 24.12.2019 16:19, Andrew Cooper wrote:
> Migration data can be split into two parts - that which is invariant of
> guest execution, and that which is not.  Separate these two with the
> STATIC_DATA_END record.
> 
> The short term, we want to move the x86 CPU Policy data into the stream.
> In the longer term, we want to provisionally send the static data only
> to the destination as a more robust compatibility check.  In both cases,
> we will want a callback into the higher level toolstack.
> 
> Mandate the presence of the STATIC_DATA_END record, and declare this v3,
> along with instructions for how to compatibly interpret a v2 stream.

What doesn't become clear (to me) from all of the above is why this
record is needed (wanted), and hence why it is to be mandatory.
After all ...

> @@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like:
>  HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect
>  the validity of architectural state in the context.
>  
> +Compatibility with older versions
> +=================================
> +
> +v3 compat with v2
> +-----------------
> +
> +A v3 stream is compatible with a v2 stream, but mandates the presense of a
> +STATIC_DATA_END record ahead of any memory/register content.  This is to ease
> +the introduction of new static configuration records over time.
> +
> +A v3-compatible reciever interpreting a v2 stream should infer the position of
> +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV
> +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END
> +had been sent.

... you already imply a v3 receiver can deal with its absence.

Jan
Andrew Cooper Jan. 9, 2020, 2:54 p.m. UTC | #2
On 03/01/2020 14:44, Jan Beulich wrote:
> On 24.12.2019 16:19, Andrew Cooper wrote:
>> Migration data can be split into two parts - that which is invariant of
>> guest execution, and that which is not.  Separate these two with the
>> STATIC_DATA_END record.
>>
>> The short term, we want to move the x86 CPU Policy data into the stream.
>> In the longer term, we want to provisionally send the static data only
>> to the destination as a more robust compatibility check.  In both cases,
>> we will want a callback into the higher level toolstack.
>>
>> Mandate the presence of the STATIC_DATA_END record, and declare this v3,
>> along with instructions for how to compatibly interpret a v2 stream.
> What doesn't become clear (to me) from all of the above is why this
> record is needed (wanted), and hence why it is to be mandatory.
> After all ...
>
>> @@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like:
>>  HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect
>>  the validity of architectural state in the context.
>>  
>> +Compatibility with older versions
>> +=================================
>> +
>> +v3 compat with v2
>> +-----------------
>> +
>> +A v3 stream is compatible with a v2 stream, but mandates the presense of a
>> +STATIC_DATA_END record ahead of any memory/register content.  This is to ease
>> +the introduction of new static configuration records over time.
>> +
>> +A v3-compatible reciever interpreting a v2 stream should infer the position of
>> +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV
>> +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END
>> +had been sent.
> ... you already imply a v3 receiver can deal with its absence.

It provides an easier way to reason about the stream, as new features
get added, and this is helpful when debugging.

Most importantly however, it offers a clean way to deprecate/remove
support for obsolete senders.

~Andrew
Ian Jackson Jan. 14, 2020, 4:07 p.m. UTC | #3
Andrew Cooper writes ("[PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END"):
> Migration data can be split into two parts - that which is invariant of
> guest execution, and that which is not.  Separate these two with the
> STATIC_DATA_END record.
> 
> The short term, we want to move the x86 CPU Policy data into the stream.
> In the longer term, we want to provisionally send the static data only
> to the destination as a more robust compatibility check.  In both cases,
> we will want a callback into the higher level toolstack.
> 
> Mandate the presence of the STATIC_DATA_END record, and declare this v3,
> along with instructions for how to compatibly interpret a v2 stream.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
diff mbox series

Patch

diff --git a/docs/specs/libxc-migration-stream.pandoc b/docs/specs/libxc-migration-stream.pandoc
index a7a8a08936..22ff306e0b 100644
--- a/docs/specs/libxc-migration-stream.pandoc
+++ b/docs/specs/libxc-migration-stream.pandoc
@@ -127,7 +127,7 @@  marker      0xFFFFFFFFFFFFFFFF.
 
 id          0x58454E46 ("XENF" in ASCII).
 
-version     0x00000002.  The version of this specification.
+version     0x00000003.  The version of this specification.
 
 options     bit 0: Endianness.  0 = little-endian, 1 = big-endian.
 
@@ -620,6 +620,21 @@  The count of pfns is: record->length/sizeof(uint64_t).
 
 \clearpage
 
+STATIC_DATA_END
+---------------
+
+A static data end record marks the end of the static state.  I.e. state which
+is invariant of guest execution.
+
+
+     0     1     2     3     4     5     6     7 octet
+    +-------------------------------------------------+
+
+The end record contains no fields; its body_length is 0.
+
+\clearpage
+
+
 Layout
 ======
 
@@ -639,7 +654,9 @@  A typical save record for an x86 PV guest image would look like:
 
 * Image header
 * Domain header
-* X86_PV_INFO record
+* Static data records:
+    * X86_PV_INFO record
+    * STATIC_DATA_END
 * X86_PV_P2M_FRAMES record
 * Many PAGE_DATA records
 * X86_TSC_INFO
@@ -667,6 +684,8 @@  A typical save record for an x86 HVM guest image would look like:
 
 * Image header
 * Domain header
+* Static data records:
+    * STATIC_DATA_END
 * Many PAGE_DATA records
 * X86_TSC_INFO
 * HVM_PARAMS
@@ -675,9 +694,23 @@  A typical save record for an x86 HVM guest image would look like:
 HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect
 the validity of architectural state in the context.
 
+Compatibility with older versions
+=================================
+
+v3 compat with v2
+-----------------
+
+A v3 stream is compatible with a v2 stream, but mandates the presense of a
+STATIC_DATA_END record ahead of any memory/register content.  This is to ease
+the introduction of new static configuration records over time.
+
+A v3-compatible reciever interpreting a v2 stream should infer the position of
+STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV
+guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END
+had been sent.
 
 Legacy Images (x86 only)
-========================
+------------------------
 
 Restoring legacy images from older tools shall be handled by
 translating the legacy format image into this new format.
diff --git a/tools/libxc/xc_sr_common.c b/tools/libxc/xc_sr_common.c
index dd9a11b4b5..7f22cf0365 100644
--- a/tools/libxc/xc_sr_common.c
+++ b/tools/libxc/xc_sr_common.c
@@ -36,6 +36,7 @@  static const char *const mandatory_rec_types[] =
     [REC_TYPE_VERIFY]                       = "Verify",
     [REC_TYPE_CHECKPOINT]                   = "Checkpoint",
     [REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST]    = "Checkpoint dirty pfn list",
+    [REC_TYPE_STATIC_DATA_END]              = "Static data end",
 };
 
 const char *rec_type_to_str(uint32_t type)
diff --git a/tools/libxc/xc_sr_stream_format.h b/tools/libxc/xc_sr_stream_format.h
index ae7c0de393..81c9765b0a 100644
--- a/tools/libxc/xc_sr_stream_format.h
+++ b/tools/libxc/xc_sr_stream_format.h
@@ -73,6 +73,7 @@  struct xc_sr_rhdr
 #define REC_TYPE_VERIFY                     0x0000000dU
 #define REC_TYPE_CHECKPOINT                 0x0000000eU
 #define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST  0x0000000fU
+#define REC_TYPE_STATIC_DATA_END            0x00000010U
 
 #define REC_TYPE_OPTIONAL             0x80000000U
 
diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index 63b3558029..d0c4f3527d 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -56,6 +56,7 @@ 
 REC_TYPE_verify                     = 0x0000000d
 REC_TYPE_checkpoint                 = 0x0000000e
 REC_TYPE_checkpoint_dirty_pfn_list  = 0x0000000f
+REC_TYPE_static_data_end            = 0x00000010
 
 rec_type_to_str = {
     REC_TYPE_end                        : "End",
@@ -74,6 +75,7 @@ 
     REC_TYPE_verify                     : "Verify",
     REC_TYPE_checkpoint                 : "Checkpoint",
     REC_TYPE_checkpoint_dirty_pfn_list  : "Checkpoint dirty pfn list",
+    REC_TYPE_static_data_end            : "Static data end",
 }
 
 # page_data