[RESEND,VIRTIO,GPU,v3,1/1] virtio-gpu: Add new feature flag VIRTIO_GPU_F_FREEZE_S3

Commit Message

Chen, Jiqian Sept. 11, 2023, 10:04 a.m. UTC

When we suspend/resume guest on Xen, the display can't come back.
This is because when guest suspended, it called into Qemu. Then
Qemu destroyed all resources which is used for display. So that
guest's display can't come back to the time when it was suspended.

To solve above problem, I added a new mechanism that when guest is
suspending, it will notify Qemu, and then Qemu will not destroy
resourcesi which are created by using commands
VIRTIO_GPU_CMD_RESOURCE_CREATE_*.

Due to that mechanism needs cooperation between guest and host,
I need to add a new feature flag, so that guest and host can
negotiate whenever freeze_S3 is supported or not.

Signed-off-by: Jiqian Chen <[e-mail]>
---
 device-types/gpu/description.tex | 42 ++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

Comments

Mikhail Golubev-Ciuchea Sept. 11, 2023, 4:33 p.m. UTC | #1

Hi Jiqian,

Thanks for the proposal.

Some time ago I was working on the same issue with suspending the gpu device
(on arm).  Additionally, I had troubles with virtio-video device as well, see
https://lore.kernel.org/lkml/[e-mail]/T/ for
details.

In your case, the
VIRTIO_GPU_FREEZE_MODE_FREEZE_S3/VIRTIO_GPU_FREEZE_MODE_UNFREEZE do influence
how reset is being handled by Qemu, is this correct?  Since multiple devices
can benefit from the same mechanism, would it be possible to: a) have a more
generic, non ACPI-based name, b) make the feature generic, applicable to other
devices as well?


Best wishes,
Mikhail Golubev-Ciuchea



--

Mikhail Golubev-Ciuchea


OpenSynergy GmbH

Rotherstr. 20, 10245 Berlin

Telefon: [phone] - 903

EMail:   [e-mail]

www.opensynergy.com

Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B

Geschäftsführer/Managing Director: Regis Adjamah

Chen, Jiqian Sept. 12, 2023, 10:41 a.m. UTC | #2

Hi Mikhail Golubev-Ciuchea,

On 2023/9/12 00:33, Mikhail Golubev-Ciuchea wrote:
> Hi Jiqian,
> 
> Thanks for the proposal.
> 
> Some time ago I was working on the same issue with suspending the gpu device
> (on arm).  Additionally, I had troubles with virtio-video device as well, see
> https://lore.kernel.org/lkml/[e-mail]/T/ for
> details.
> 
> In your case, the
> VIRTIO_GPU_FREEZE_MODE_FREEZE_S3/VIRTIO_GPU_FREEZE_MODE_UNFREEZE do influence
> how reset is being handled by Qemu, is this correct?  Since multiple devices
Yes, it can affect the reset behavior of Qemu.

> can benefit from the same mechanism, would it be possible to: a) have a more
> generic, non ACPI-based name, b) make the feature generic, applicable to other
> devices as well?
It seems your opinion is the same as Parav Pandit's, both hope this feature can be more generic. 
If that's the case, its role and scope of influence will become quite large, and I'm not sure if that's appropriate.
I will try to modify the code to improve the level of this feature, if I can make it work, I will send a new series of patch.
And during this period, I am still looking forward to receiving more suggestions and guidance from everyone. Thank you!

> 
> 
> Best wishes,
> Mikhail Golubev-Ciuchea
> 
> 
> 
> --
> 
> Mikhail Golubev-Ciuchea
> 
> 
> OpenSynergy GmbH
> 
> Rotherstr. 20, 10245 Berlin
> 
> Telefon: [phone] - 903
> 
> EMail:   [e-mail]
> 
> www.opensynergy.com
> 
> Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
> 
> Geschäftsführer/Managing Director: Regis Adjamah
> 
> 
> ________________________________________
> From: [e-mail] <[e-mail]> on behalf of Jiqian Chen <[e-mail]>
> Sent: Monday, September 11, 2023 12:04 PM
> To: Gerd Hoffmann; Marc-André Lureau; Robert Beckett; [e-mail]; [e-mail]
> Cc: [e-mail]; [e-mail]; Stefano Stabellini; Roger Pau Monné; Alex Deucher; Christian Koenig; Stewart Hildebrand; Xenia Ragiadakou; Honglei Huang; Julia Zhang; Huang Rui; Jiqian Chen
> Subject: [virtio-dev] [RESEND VIRTIO GPU PATCH v3 1/1] virtio-gpu: Add new feature flag VIRTIO_GPU_F_FREEZE_S3
> 
> When we suspend/resume guest on Xen, the display can't come back.
> This is because when guest suspended, it called into Qemu. Then
> Qemu destroyed all resources which is used for display. So that
> guest's display can't come back to the time when it was suspended.
> 
> To solve above problem, I added a new mechanism that when guest is
> suspending, it will notify Qemu, and then Qemu will not destroy
> resourcesi which are created by using commands
> VIRTIO_GPU_CMD_RESOURCE_CREATE_*.
> 
> Due to that mechanism needs cooperation between guest and host,
> I need to add a new feature flag, so that guest and host can
> negotiate whenever freeze_S3 is supported or not.
> 
> Signed-off-by: Jiqian Chen <[e-mail]>
> ---
>  device-types/gpu/description.tex | 42 ++++++++++++++++++++++++++++++++
>  1 file changed, 42 insertions(+)
> 
> diff --git a/device-types/gpu/description.tex b/device-types/gpu/description.tex
> index 4435248..1a137e7 100644
> --- a/device-types/gpu/description.tex
> +++ b/device-types/gpu/description.tex
> @@ -37,6 +37,8 @@ \subsection{Feature bits}\label{sec:Device Types / GPU Device / Feature bits}
>    resources is supported.
>  \item[VIRTIO_GPU_F_CONTEXT_INIT (4)] multiple context types and
>    synchronization timelines supported.  Requires VIRTIO_GPU_F_VIRGL.
> +\item[VIRTIO_GPU_F_FREEZE_S3 (5)] freezing virtio-gpu and keeping resources
> +  alive is supported.
>  \end{description}
> 
>  \subsection{Device configuration layout}\label{sec:Device Types / GPU Device / Device configuration layout}
> @@ -228,6 +230,9 @@ \subsubsection{Device Operation: Request header}\label{sec:Device Types / GPU De
>          VIRTIO_GPU_CMD_UPDATE_CURSOR = 0x0300,
>          VIRTIO_GPU_CMD_MOVE_CURSOR,
> 
> +        /* freeze mode */
> +        VIRTIO_GPU_CMD_SET_FREEZE_MODE = 0x0400,
> +
>          /* success responses */
>          VIRTIO_GPU_RESP_OK_NODATA = 0x1100,
>          VIRTIO_GPU_RESP_OK_DISPLAY_INFO,
> @@ -838,6 +843,43 @@ \subsubsection{Device Operation: cursorq}\label{sec:Device Types / GPU Device /
> 
>  \end{description}
> 
> +\subsubsection{Device Operation: freeze_mode}\label{sec:Device Types / GPU Device / Device Operation / Device Operation: freeze_mode}
> +
> +\begin{lstlisting}
> +typedef enum {
> +  VIRTIO_GPU_FREEZE_MODE_UNFREEZE = 0,
> +  VIRTIO_GPU_FREEZE_MODE_FREEZE_S3 = 3,
> +} virtio_gpu_freeze_mode_t;
> +
> +struct virtio_gpu_set_freeze_mode {
> +  struct virtio_gpu_ctrl_hdr hdr;
> +  virtio_gpu_freeze_mode_t freeze_mode;
> +};
> +\end{lstlisting}
> +
> +\begin{description}
> +
> +\item[VIRTIO_GPU_CMD_SET_FREEZE_MODE]
> +Notify freeze mode through controlq.
> +Request data is \field{struct virtio_gpu_set_freeze_mode}.
> +Response type is VIRTIO_GPU_RESP_OK_NODATA.
> +
> +This is added for S3 function in guest with virtio-gpu. When guest does
> +S3, let it notify QEMU that virtio-gpu is in what freeze mode in
> +\field{freeze_mode}. VIRTIO_GPU_FREEZE_MODE_FREEZE_S3 means guest is
> +doing S3 and virtio-gpu will be freezed, VIRTIO_GPU_FREEZE_MODE_UNFREEZE
> +means virtio-gpu can be used as usual. When virtio-gpu is freezed, QEMU
> +will not destroy resources which are created by using commands
> +VIRTIO_GPU_CMD_RESOURCE_CREATE_*, so that guest can use those resources
> +to resume display.
> +
> +Note: this change is not enough to solve the problems of S4 function.
> +QEMU may lose resources after hibernation. It needs more research and
> +development. If S4 is supported in the future, it may need another
> +feature flag here.
> +
> +\end{description}
> +
>  \subsection{VGA Compatibility}\label{sec:Device Types / GPU Device / VGA Compatibility}
> 
>  Applies to Virtio Over PCI only.  The GPU device can come with and
> --
> 2.34.1
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [e-mail]
> For additional commands, e-mail: [e-mail]
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [e-mail]
> For additional commands, e-mail: [e-mail]
>

Chen, Jiqian Sept. 19, 2023, 12:04 p.m. UTC | #3

Hi Mikhail Golubev-Ciuchea,

I have improved this implementation to the virtio pci level. Could you please try my patches and see if them are useful in your scene?
My new series:
V5 of Qemu patch:
https://lore.kernel.org/qemu-devel/[e-mail]/T/#t
V5 of kernel patch:
https://lore.kernel.org/lkml/[e-mail]/T/#t
V5 of virtio-spec patch:
https://lists.oasis-open.org/archives/virtio-comment/202309/msg00245.html


On 2023/9/12 00:33, Mikhail Golubev-Ciuchea wrote:
> Hi Jiqian,
> 
> Thanks for the proposal.
> 
> Some time ago I was working on the same issue with suspending the gpu device
> (on arm).  Additionally, I had troubles with virtio-video device as well, see
> https://lore.kernel.org/lkml/[e-mail]/T/ for
> details.
> 
> In your case, the
> VIRTIO_GPU_FREEZE_MODE_FREEZE_S3/VIRTIO_GPU_FREEZE_MODE_UNFREEZE do influence
> how reset is being handled by Qemu, is this correct?  Since multiple devices
> can benefit from the same mechanism, would it be possible to: a) have a more
> generic, non ACPI-based name, b) make the feature generic, applicable to other
> devices as well?
> 
> 
> Best wishes,
> Mikhail Golubev-Ciuchea
> 
> 
> 
> --
> 
> Mikhail Golubev-Ciuchea
> 
> 
> OpenSynergy GmbH
> 
> Rotherstr. 20, 10245 Berlin
> 
> Telefon: [phone] - 903
> 
> EMail:   [e-mail]
> 
> www.opensynergy.com
> 
> Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
> 
> Geschäftsführer/Managing Director: Regis Adjamah
> 
> 
> ________________________________________
> From: [e-mail] <[e-mail]> on behalf of Jiqian Chen <[e-mail]>
> Sent: Monday, September 11, 2023 12:04 PM
> To: Gerd Hoffmann; Marc-André Lureau; Robert Beckett; [e-mail]; [e-mail]
> Cc: [e-mail]; [e-mail]; Stefano Stabellini; Roger Pau Monné; Alex Deucher; Christian Koenig; Stewart Hildebrand; Xenia Ragiadakou; Honglei Huang; Julia Zhang; Huang Rui; Jiqian Chen
> Subject: [virtio-dev] [RESEND VIRTIO GPU PATCH v3 1/1] virtio-gpu: Add new feature flag VIRTIO_GPU_F_FREEZE_S3
> 
> When we suspend/resume guest on Xen, the display can't come back.
> This is because when guest suspended, it called into Qemu. Then
> Qemu destroyed all resources which is used for display. So that
> guest's display can't come back to the time when it was suspended.
> 
> To solve above problem, I added a new mechanism that when guest is
> suspending, it will notify Qemu, and then Qemu will not destroy
> resourcesi which are created by using commands
> VIRTIO_GPU_CMD_RESOURCE_CREATE_*.
> 
> Due to that mechanism needs cooperation between guest and host,
> I need to add a new feature flag, so that guest and host can
> negotiate whenever freeze_S3 is supported or not.
> 
> Signed-off-by: Jiqian Chen <[e-mail]>
> ---
>  device-types/gpu/description.tex | 42 ++++++++++++++++++++++++++++++++
>  1 file changed, 42 insertions(+)
> 
> diff --git a/device-types/gpu/description.tex b/device-types/gpu/description.tex
> index 4435248..1a137e7 100644
> --- a/device-types/gpu/description.tex
> +++ b/device-types/gpu/description.tex
> @@ -37,6 +37,8 @@ \subsection{Feature bits}\label{sec:Device Types / GPU Device / Feature bits}
>    resources is supported.
>  \item[VIRTIO_GPU_F_CONTEXT_INIT (4)] multiple context types and
>    synchronization timelines supported.  Requires VIRTIO_GPU_F_VIRGL.
> +\item[VIRTIO_GPU_F_FREEZE_S3 (5)] freezing virtio-gpu and keeping resources
> +  alive is supported.
>  \end{description}
> 
>  \subsection{Device configuration layout}\label{sec:Device Types / GPU Device / Device configuration layout}
> @@ -228,6 +230,9 @@ \subsubsection{Device Operation: Request header}\label{sec:Device Types / GPU De
>          VIRTIO_GPU_CMD_UPDATE_CURSOR = 0x0300,
>          VIRTIO_GPU_CMD_MOVE_CURSOR,
> 
> +        /* freeze mode */
> +        VIRTIO_GPU_CMD_SET_FREEZE_MODE = 0x0400,
> +
>          /* success responses */
>          VIRTIO_GPU_RESP_OK_NODATA = 0x1100,
>          VIRTIO_GPU_RESP_OK_DISPLAY_INFO,
> @@ -838,6 +843,43 @@ \subsubsection{Device Operation: cursorq}\label{sec:Device Types / GPU Device /
> 
>  \end{description}
> 
> +\subsubsection{Device Operation: freeze_mode}\label{sec:Device Types / GPU Device / Device Operation / Device Operation: freeze_mode}
> +
> +\begin{lstlisting}
> +typedef enum {
> +  VIRTIO_GPU_FREEZE_MODE_UNFREEZE = 0,
> +  VIRTIO_GPU_FREEZE_MODE_FREEZE_S3 = 3,
> +} virtio_gpu_freeze_mode_t;
> +
> +struct virtio_gpu_set_freeze_mode {
> +  struct virtio_gpu_ctrl_hdr hdr;
> +  virtio_gpu_freeze_mode_t freeze_mode;
> +};
> +\end{lstlisting}
> +
> +\begin{description}
> +
> +\item[VIRTIO_GPU_CMD_SET_FREEZE_MODE]
> +Notify freeze mode through controlq.
> +Request data is \field{struct virtio_gpu_set_freeze_mode}.
> +Response type is VIRTIO_GPU_RESP_OK_NODATA.
> +
> +This is added for S3 function in guest with virtio-gpu. When guest does
> +S3, let it notify QEMU that virtio-gpu is in what freeze mode in
> +\field{freeze_mode}. VIRTIO_GPU_FREEZE_MODE_FREEZE_S3 means guest is
> +doing S3 and virtio-gpu will be freezed, VIRTIO_GPU_FREEZE_MODE_UNFREEZE
> +means virtio-gpu can be used as usual. When virtio-gpu is freezed, QEMU
> +will not destroy resources which are created by using commands
> +VIRTIO_GPU_CMD_RESOURCE_CREATE_*, so that guest can use those resources
> +to resume display.
> +
> +Note: this change is not enough to solve the problems of S4 function.
> +QEMU may lose resources after hibernation. It needs more research and
> +development. If S4 is supported in the future, it may need another
> +feature flag here.
> +
> +\end{description}
> +
>  \subsection{VGA Compatibility}\label{sec:Device Types / GPU Device / VGA Compatibility}
> 
>  Applies to Virtio Over PCI only.  The GPU device can come with and
> --
> 2.34.1
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [e-mail]
> For additional commands, e-mail: [e-mail]
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [e-mail]
> For additional commands, e-mail: [e-mail]
>

Patch

diff --git a/device-types/gpu/description.tex b/device-types/gpu/description.tex
index 4435248..1a137e7 100644
--- a/device-types/gpu/description.tex
+++ b/device-types/gpu/description.tex
@@ -37,6 +37,8 @@  \subsection{Feature bits}\label{sec:Device Types / GPU Device / Feature bits}
   resources is supported.
 \item[VIRTIO_GPU_F_CONTEXT_INIT (4)] multiple context types and
   synchronization timelines supported.  Requires VIRTIO_GPU_F_VIRGL.
+\item[VIRTIO_GPU_F_FREEZE_S3 (5)] freezing virtio-gpu and keeping resources
+  alive is supported.
 \end{description}
 
 \subsection{Device configuration layout}\label{sec:Device Types / GPU Device / Device configuration layout}
@@ -228,6 +230,9 @@  \subsubsection{Device Operation: Request header}\label{sec:Device Types / GPU De
         VIRTIO_GPU_CMD_UPDATE_CURSOR = 0x0300,
         VIRTIO_GPU_CMD_MOVE_CURSOR,
 
+        /* freeze mode */
+        VIRTIO_GPU_CMD_SET_FREEZE_MODE = 0x0400,
+
         /* success responses */
         VIRTIO_GPU_RESP_OK_NODATA = 0x1100,
         VIRTIO_GPU_RESP_OK_DISPLAY_INFO,
@@ -838,6 +843,43 @@  \subsubsection{Device Operation: cursorq}\label{sec:Device Types / GPU Device /
 
 \end{description}
 
+\subsubsection{Device Operation: freeze_mode}\label{sec:Device Types / GPU Device / Device Operation / Device Operation: freeze_mode}
+
+\begin{lstlisting}
+typedef enum {
+  VIRTIO_GPU_FREEZE_MODE_UNFREEZE = 0,
+  VIRTIO_GPU_FREEZE_MODE_FREEZE_S3 = 3,
+} virtio_gpu_freeze_mode_t;
+
+struct virtio_gpu_set_freeze_mode {
+  struct virtio_gpu_ctrl_hdr hdr;
+  virtio_gpu_freeze_mode_t freeze_mode;
+};
+\end{lstlisting}
+
+\begin{description}
+
+\item[VIRTIO_GPU_CMD_SET_FREEZE_MODE]
+Notify freeze mode through controlq.
+Request data is \field{struct virtio_gpu_set_freeze_mode}.
+Response type is VIRTIO_GPU_RESP_OK_NODATA.
+
+This is added for S3 function in guest with virtio-gpu. When guest does
+S3, let it notify QEMU that virtio-gpu is in what freeze mode in
+\field{freeze_mode}. VIRTIO_GPU_FREEZE_MODE_FREEZE_S3 means guest is
+doing S3 and virtio-gpu will be freezed, VIRTIO_GPU_FREEZE_MODE_UNFREEZE
+means virtio-gpu can be used as usual. When virtio-gpu is freezed, QEMU
+will not destroy resources which are created by using commands
+VIRTIO_GPU_CMD_RESOURCE_CREATE_*, so that guest can use those resources
+to resume display.
+
+Note: this change is not enough to solve the problems of S4 function.
+QEMU may lose resources after hibernation. It needs more research and
+development. If S4 is supported in the future, it may need another
+feature flag here.
+
+\end{description}
+
 \subsection{VGA Compatibility}\label{sec:Device Types / GPU Device / VGA Compatibility}
 
 Applies to Virtio Over PCI only.  The GPU device can come with and