From patchwork Wed Nov 8 18:05:36 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Denis V. Lunev\" via" X-Patchwork-Id: 10049561 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8912060381 for ; Wed, 8 Nov 2017 22:18:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F8212A523 for ; Wed, 8 Nov 2017 22:18:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 644A42A5B8; Wed, 8 Nov 2017 22:18:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DC8072A5BD for ; Wed, 8 Nov 2017 22:18:08 +0000 (UTC) Received: from localhost ([::1]:34159 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eCYfe-00062z-5R for patchwork-qemu-devel@patchwork.kernel.org; Wed, 08 Nov 2017 17:18:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51473) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eCUkH-0002LO-4c for qemu-devel@nongnu.org; Wed, 08 Nov 2017 13:06:53 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eCUjU-0000UK-7Y for qemu-devel@nongnu.org; Wed, 08 Nov 2017 13:06:37 -0500 Received: from mail-io0-x244.google.com ([2607:f8b0:4001:c06::244]:51471) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eCUjT-0000TH-Ua for qemu-devel@nongnu.org; Wed, 08 Nov 2017 13:05:48 -0500 Received: by mail-io0-x244.google.com with SMTP id b186so6893033iof.8 for ; Wed, 08 Nov 2017 10:05:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=tObpzAYavIylGHXVRfjtQMKSfwzaN45qZos+a281Y6E=; b=FrA1Y8PddcaPhAdng9m0Oof1kOg066GdNaoX6u/EGcztWzOrh9thJUVek6VDEBFHmi 6QdEnSnxVIGV1TTUOl0mw4ifFQLf/+ipoEnbR8G0a/d2tLb4SiWLR61mfoG9lqGpEC1K TKm1W1KEIqHFtGhF+fulMpYtnUlh5JY5xxJpKS5Tmw+erqcbHeSbNBSmlSt9Yg4r/Ax1 xDCR1im75DIKDPvjUSjjrWNkF2q0nI7ceGHy8DOkauLn9Wsb1X7r67+jVX6C6sneJGR7 sLN+v4Gpwonjbp+SpkdXtjRz3qaVnM9rVtpGiEMahtOmtJiFR3EzwTAIydr1lo1pkHs5 Avew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=tObpzAYavIylGHXVRfjtQMKSfwzaN45qZos+a281Y6E=; b=j+KHLrq+noYGikFQvB0x3EPEX/l5mWpKGYkRcF1/3Dmhrj7nwMTmugBlfQ2hXB1Mw2 xgP0zsbYgB6txPRkUdtbQhjX7UsJmFaj74HL1XX5dTriznZa7h2tU9KG2+YnwPB9OqDj uSkQT9VSPCJo93yl9DDGfNh/bJhz5Kvi/sp5jAm/tKAi6FnJczfWN0NhLURv0Dpo4Cyi LT9daRuNmKqd55imu1GZyHbh7DS2dgMCqLMEOyoMdCu/sk3N98denvTkr8xoBxgnWn5Y mZRVV6FrFDsNaq99B8C8YWs9TLQBLpam6MZ97C85YoAFZlxqFcXfaAOoEfjNo/uR8Sy9 PQ2g== X-Gm-Message-State: AJaThX6qjvGYcy2RGKkaXOMYs+A2vZZkFSenNcxj/8LgPKVAGn9uXwwH vYWkVTuq8yL2kpeZZEuXlyUVPA== X-Google-Smtp-Source: ABhQp+QlMC1arrUwVzEMyAXkf1Fu1WdDhPDBxy1e6xxz4BZh39XGo1v9ZYRvFdD6b4HyVy7leCWImQ== X-Received: by 10.107.131.85 with SMTP id f82mr1719664iod.215.1510164346098; Wed, 08 Nov 2017 10:05:46 -0800 (PST) Received: from paxos.mtv.corp.google.com ([2620:0:1000:1611:7567:9b12:9a7b:9aa0]) by smtp.gmail.com with ESMTPSA id p17sm2098295iod.40.2017.11.08.10.05.44 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 08 Nov 2017 10:05:45 -0800 (PST) To: qemu-devel@nongnu.org, kraxel@redhat.com Date: Wed, 8 Nov 2017 10:05:36 -0800 Message-Id: <20171108180536.38781-1-lepton@google.com> X-Mailer: git-send-email 2.15.0.403.gc27cc4dac6-goog X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4001:c06::244 X-Mailman-Approved-At: Wed, 08 Nov 2017 17:17:00 -0500 Subject: [Qemu-devel] [PATCH] virtio-gpu: fix bug in host memory calculation. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tao Wu via Qemu-devel From: "Denis V. Lunev\" via" Reply-To: Tao Wu Cc: marcandre.lureau@redhat.com, Tao Wu , liqiang6-s@360.cn, airlied@redhat.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The old code treats bits as bytes when calculating host memory usage. Change it to be consistent with allocation logic in pixman library. Signed-off-by: Tao Wu Reviewed-by: Marc-André Lureau --- hw/display/virtio-gpu.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c index 43bbe09ea0..428786f291 100644 --- a/hw/display/virtio-gpu.c +++ b/hw/display/virtio-gpu.c @@ -322,6 +322,15 @@ static pixman_format_code_t get_pixman_format(uint32_t virtio_gpu_format) } } +static uint32_t calc_image_hostmem(pixman_format_code_t pformat, + uint32_t width, uint32_t height) +{ + /* copied from pixman/pixman-bits-image.c, skip integer overflow check. */ + int bpp = PIXMAN_FORMAT_BPP(pformat); + int stride = ((width * bpp + 0x1f) >> 5) * sizeof(uint32_t); + return height * stride; +} + static void virtio_gpu_resource_create_2d(VirtIOGPU *g, struct virtio_gpu_ctrl_command *cmd) { @@ -366,7 +375,7 @@ static void virtio_gpu_resource_create_2d(VirtIOGPU *g, return; } - res->hostmem = PIXMAN_FORMAT_BPP(pformat) * c2d.width * c2d.height; + res->hostmem = calc_image_hostmem(pformat, c2d.width, c2d.height); if (res->hostmem + g->hostmem < g->conf.max_hostmem) { res->image = pixman_image_create_bits(pformat, c2d.width, @@ -1087,7 +1096,7 @@ static int virtio_gpu_load(QEMUFile *f, void *opaque, size_t size, return -EINVAL; } - res->hostmem = PIXMAN_FORMAT_BPP(pformat) * res->width * res->height; + res->hostmem = calc_image_hostmem(pformat, res->width, res->height); res->addrs = g_new(uint64_t, res->iov_cnt); res->iov = g_new(struct iovec, res->iov_cnt);