diff mbox series

[18/23] ui/vnc-enc-hextile: Use definitions to avoid dynamic stack allocation

Message ID 20210505211047.1496765-19-philmd@redhat.com (mailing list archive)
State New, archived
Headers show
Series misc: Remove variable-length arrays on the stack | expand

Commit Message

Philippe Mathieu-Daudé May 5, 2021, 9:10 p.m. UTC
We know 'pf.bytes_per_pixel' will be at most 'VNC_SERVER_FB_BYTES'
(which is actually 4 bytes for 32bpp). Instead of having the compiler
use variable-length array, use this 'small' maximum length and
autofree to allocate the buffer on the heap.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 ui/vnc-enc-hextile-template.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Comments

Richard Henderson May 7, 2021, 4:46 p.m. UTC | #1
On 5/5/21 2:10 PM, Philippe Mathieu-Daudé wrote:
> We know 'pf.bytes_per_pixel' will be at most 'VNC_SERVER_FB_BYTES'
> (which is actually 4 bytes for 32bpp). Instead of having the compiler
> use variable-length array, use this 'small' maximum length and
> autofree to allocate the buffer on the heap.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@redhat.com>
> ---
>   ui/vnc-enc-hextile-template.h | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)

Wait, you know the value is small (max 4), and you don't want to allocate a 
constant 1k on the stack?


r~
diff mbox series

Patch

diff --git a/ui/vnc-enc-hextile-template.h b/ui/vnc-enc-hextile-template.h
index 0c56262afff..85e67bd9d88 100644
--- a/ui/vnc-enc-hextile-template.h
+++ b/ui/vnc-enc-hextile-template.h
@@ -25,10 +25,11 @@  static void CONCAT(send_hextile_tile_, NAME)(VncState *vs,
     int bg_count = 0;
     int fg_count = 0;
     int flags = 0;
-    uint8_t data[(vs->client_pf.bytes_per_pixel + 2) * 16 * 16];
+    g_autofree uint8_t *data = g_malloc((VNC_SERVER_FB_BYTES + 2) * 16 * 16);
     int n_data = 0;
     int n_subtiles = 0;
 
+    assert(vs->client_pf.bytes_per_pixel <= VNC_SERVER_FB_BYTES);
     for (j = 0; j < h; j++) {
         for (i = 0; i < w; i++) {
             switch (n_colors) {