From patchwork Sat Jan 4 08:00:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B9F60E77199 for ; Sat, 4 Jan 2025 08:01:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz69-0003YN-Mn; Sat, 04 Jan 2025 03:01:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz65-0003Wl-DZ for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:25 -0500 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz64-0006eU-1Q for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:25 -0500 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-219f8263ae0so136955335ad.0 for ; Sat, 04 Jan 2025 00:01:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977683; x=1736582483; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=pf5SZ6e3cD61gqtSQU/hVD3w+JbOpHtrUnhXJppalog=; b=PdlIknEujYSbYgGRr9gpokQ5w2eGwqMlXGau7fXI2x8W15GjelWWzpTHHVmUB4hVfh 7YW3irRpc7a2xrNjXzCBFS+iPSFH17xlzGjr7JcuVK9DhTJZUA0XJ+HnZbqw0Eq+Tg20 n2m6jbIp5P4zFOi5rHWU8BW7TkN1Fd7sKqQcyf8ZmIRrNhDhSLERO/2vS9ggHUeVmXGR qb4USu3fIpjWNM5B6oxL2IL+Q8Xv0eJ3/VDvppucfwpmsuSNo8TGUxqf8ukFHFMzJ+6k dzwTDVDV0YqFqdcEncRpAa6BjQYWUXKfAEdW6NGNP9ZPYyr/5kuDL+o4V1BmbAO0J8nq DWkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977683; x=1736582483; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pf5SZ6e3cD61gqtSQU/hVD3w+JbOpHtrUnhXJppalog=; b=wKndTP3tjVhB/1b/hwDazvxIAL+9drVqo2s8jIndbBkUeJ88EMd5QhSJW6iEYY9Omd jF7Vjow7akEEd4h1nUbZrUhVRQ28GM/jcdRAyPTDWiQAVunbZvWwEW7GNy8/B9WRq49M 1QXY44PntCmE0yojuSo1EcebnaWcjXie4Y1DdYtTqss6J2exXkcFZraDczMN2byd7c7g vnF53lDWohf32fNCGEPd/Ysx3jvavawW+XmeSw2IDHFQqgDP7ZdEYSjOW3i/u4/I/+hp HdOsGpHts2A8nu9rR8HrPj/xf2jgKuPdOMFlluEUs2KckArp6rb5IvOVOzz8CMLUKP89 2nwg== X-Gm-Message-State: AOJu0YzsjrNQigPHu97PyTCthuNyl7uGz7oTzwPC+Px5z7WIWMQxNPPq 8vgeFeoMoDQ9zjeyvRmDi0d4YZQoaDb2kFXnR53oj09Hl19gRKVY7qKpkdqQcoU= X-Gm-Gg: ASbGncu7TjrQ7bO6L627YqSV/LQgb47eeCmGy/2DABCxszHHt/mQmweJcBFprJ6rQNb 0CSuLuus6Hw4Xid1SqOliTKaZEwwT0U/N8joAbvOz7weuxEISjjo4Mo+9XroiDb7k0S4dvAHPDj RnBBebbi+7BpCidll2oYWiCvPp7XELHAdxCQHUGfKKj3NEMsap84CXlQkDavY65OR2tCIukEVFs 3Fdpxs5Kvr2Kh9718URxICRHmVp31f9qUJPCY3/0NOS48elEcGvAsePIwmN X-Google-Smtp-Source: AGHT+IGsAWzPzd75/veVnKmlNmphUQNMqY6BodbR1KsWAbqUjje6GVNQxcv/Tq4d/BpEujrc0H6VtQ== X-Received: by 2002:a17:902:c40b:b0:206:9a3f:15e5 with SMTP id d9443c01a7336-219e6ebb6e2mr710534215ad.32.1735977682796; Sat, 04 Jan 2025 00:01:22 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-219dc9f5211sm255346465ad.188.2025.01.04.00.01.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:22 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:00:54 +0900 Subject: [PATCH v8 1/9] hw/pci: Do not add ROM BAR for SR-IOV VF MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-1-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A SR-IOV VF cannot have a ROM BAR. Co-developed-by: Yui Washizu Signed-off-by: Akihiko Odaki --- hw/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 8f160c0c0d8a..c4fa32dc987a 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2432,6 +2432,14 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, return; } + if (pci_is_vf(pdev)) { + if (pdev->rom_bar > 0) { + error_setg(errp, "ROM BAR cannot be enabled for SR-IOV VF"); + } + + return; + } + if (load_file || pdev->romsize == UINT32_MAX) { path = qemu_find_file(QEMU_FILE_TYPE_BIOS, pdev->romfile); if (path == NULL) { From patchwork Sat Jan 4 08:00:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8085CE77197 for ; Sat, 4 Jan 2025 08:02:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz6G-0003am-Vi; Sat, 04 Jan 2025 03:01:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6F-0003aE-JA for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:35 -0500 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz69-0006eq-JD for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:35 -0500 Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-21628b3fe7dso176418715ad.3 for ; Sat, 04 Jan 2025 00:01:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977688; x=1736582488; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=yuQhf5BHhf8srsz41f2lJ2BM75ujVVRrcc9sOO738qc=; b=YokNk015F4degeUc0zm9lx4y9g1lZbKXb7DmknR6QnOrKKpg7nUTKOuy0sZ3nLwxgb NmtRrGpQzg5npt5LcD/rMgQzu0O1diyDKnsVZm9YeSSVX8f9lfGFdjkkn07PYhZNb6NM rTYYGbI8MokuZ99rjjN4fIbQDfJ9lnTPDeGsYBaruP9zY6g4x+Gq0oqQHEau0GZQtW7C 7mlUCw3emBHuAxou7CU3B3ARX/dk92di9SHK09pljTX4ILUgEWWoP+4a94j+D8ffqSSl s/bSRTYQ6mwknx8JAFJ7RMNJVcnhsnVDJ3odvIJuYmL9O+dqclb5WONnQxA6lXPzqZpW XzHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977688; x=1736582488; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yuQhf5BHhf8srsz41f2lJ2BM75ujVVRrcc9sOO738qc=; b=f4bWPHhWxDw96sIsCZfCnBOxurQTvfZt1IAYLUfufeiMPq/hb+O6INSoos9OH58D7X KkoPoq8HZ1yP5aMb37Z8s7LBmeVUPjEcWiO9vVBme83I+YdIj8tvNaFBO0NfbO1beui/ kY8yng/c6eNXCpz68rvMixLnW2fQ4/NhlS01Ghvbgiw2IZkWxTuwnUl0MWu/xlUkousg b6GS7jSO6Hxv3xLz0ccd/RhZTIZSk0RejQmXzsZMNPeXkWwvA0kYmvUxjRFcu1coNYh2 EdPeNxF9VknTf8Lo6pJkSfb1PF1RzRCdz/sFqa4wTqdpUjOqt4jhuAwwA+MsYEXJ9bfS U0pA== X-Gm-Message-State: AOJu0Yym3Wifqc/f97BOyyfQ0VBV9o0GK0Hp+27q4RiXPYpuu4HdjIDy +CffMs64yZBQVci776JwakEluSAZfuDDy7tBw/2ED/mBFrJkx9YIfGnIIPs3qjA= X-Gm-Gg: ASbGncsVPA5P1q6yb0GT0Dw+oYNmpvjCNJS8iXgDu59kxNoioDFsM+a48catSh1GUF3 s5QtrY5PoNHPg8dx3G5tfj+M27htNFSSCiGPZUBilJlB/198Mx8u9G+w036tuxd5hR+61OUSPsx D/4Kxmd3CsyWS4MkpCOeUn9W7m+KgeKLbNWPlGp82scUgXYo7M+a5Kh8gSX7bRztdL1Ks4p+26m laphUVjH3holW2rdqq5RsgqMcSCzV0nZEzkwCR4XxShFOWvYS93015FfIii X-Google-Smtp-Source: AGHT+IFNQh9GAy/axAg5p8Rn/JzAAhQKp7IWKta0a52Lq6QMfYGSpm+9dCZctLjTCC8Szz3UUoxRXQ== X-Received: by 2002:a17:902:c951:b0:216:5e6e:68b4 with SMTP id d9443c01a7336-219e6f10987mr764309565ad.46.1735977688151; Sat, 04 Jan 2025 00:01:28 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-842b31ef0c0sm21460545a12.28.2025.01.04.00.01.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:27 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:00:55 +0900 Subject: [PATCH v8 2/9] hw/pci: Fix SR-IOV VF number calculation MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-2-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x633.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org pci_config_get_bar_addr() had a division by vf_stride. vf_stride needs to be non-zero when there are multiple VFs, but the specification does not prohibit to make it zero when there is only one VF. Do not perform the division for the first VF to avoid division by zero. Signed-off-by: Akihiko Odaki --- hw/pci/pci.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index c4fa32dc987a..ba53302ecc5f 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -1510,7 +1510,11 @@ static pcibus_t pci_config_get_bar_addr(PCIDevice *d, int reg, pci_get_word(pf->config + sriov_cap + PCI_SRIOV_VF_OFFSET); uint16_t vf_stride = pci_get_word(pf->config + sriov_cap + PCI_SRIOV_VF_STRIDE); - uint32_t vf_num = (d->devfn - (pf->devfn + vf_offset)) / vf_stride; + uint32_t vf_num = d->devfn - (pf->devfn + vf_offset); + + if (vf_num) { + vf_num /= vf_stride; + } if (type & PCI_BASE_ADDRESS_MEM_TYPE_64) { new_addr = pci_get_quad(pf->config + bar); From patchwork Sat Jan 4 08:00:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AB385E77188 for ; Sat, 4 Jan 2025 08:02:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz6G-0003ah-Ig; Sat, 04 Jan 2025 03:01:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6F-0003aH-OV for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:35 -0500 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz6E-0006fe-63 for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:35 -0500 Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2eed82ca5b4so18120247a91.2 for ; Sat, 04 Jan 2025 00:01:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977693; x=1736582493; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=oTbvY0b5lrRZQshDH0+J48N1NJEpTN8UFElTBKrhUcs=; b=rl+4k3LnU+Rvp6mHB1ULnmK+0fWQtUcJs3PpozabOgIcrlp3yF7a5ojBN7na4dapmU fgjVtOGDF9RM6beqWY7UxJ69IyYKOpZjNkfrG/C07AXGDogPMUcjPbTFcXIthCs3cINQ OLD5zmg0gT9ltDKIdtIUE9nvc5RWw3JWe5P4Fnetc/UN4I8ZEn9Vyv3tncyAc1u6/4nW o5Uz8aFO3Vvsza2vDO7xgS89FFzdnsxTpSMTRzEtDG173+zpwXWajW9X58K3Nn99KhF4 exvw4aBmYUTfucqy+6Oh/f7tYn9SC6nBiQtpsxF06qch7bd5p+2moD8ulPHBm4AynxOI rfHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977693; x=1736582493; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oTbvY0b5lrRZQshDH0+J48N1NJEpTN8UFElTBKrhUcs=; b=LN/G1YDf6ib73/2lLVUPSVf7umg5RMth8kWqiuJsP1S2WzWOXBqzNADecS/9nCAY/B XD/2C6wutCseMZcbyPtK7PZHPclWGOF/e6cx4teiY2Z2cNm4eY1x3DimCxcKx/shQfsr GEScNqzIuCqnZwu9Fxn+hx2NbvHnduiv4YjjrsRS6gJS95Mcglrc9d8zQk1vcYpjkHIf IRp9RT8iLVtTNgJecximkoCFyfTdoAuq5bYRbmx1rwz+4I4zDigCc9PwBsza2EcSOJ5N ffTcemVXiCAi2BZVnzGw02R0Zqx3/L2BAAFa99MDE/2gNV4RvNcuWSLgm0nd1sUMnStC 4RjA== X-Gm-Message-State: AOJu0YxBe5RzIwKQrE8Rk1fJTXxw04Sh5nvg7CMhiqimXSMt9I0NB+de XXFhPkklJB5HcVQR26Mq5n78j6BPSyqBwunb2ieYDExdwP4LWdA2EJesS4LcRJo= X-Gm-Gg: ASbGncvTX82R3iyBmfXtW4VRHG1Ge7LDelHwHtfKMEKyOZqD+kZfI8AqXg7ASuGr8S/ 0JJo0a9utK9ZTphs7Z5S2FK9Zrcfu98a72oUb+b+mWxnjSodUWKnbl2Az54UIC9B24ARo07R0yV Jvdxp+Ux33cdcwUgoRjN/LJTIuuf2OfI8dTzXW4MT8hBtmgSuOWNlHYauelYXszprgPYZop4uOz FJIs7BJFjcKk+v1jzROZc7uuTycUKgyXde4m/6aVgYLUaLyH+7Fa3thjvx1 X-Google-Smtp-Source: AGHT+IGYhht4fS5uvygWsYjit6SAtOcgp0RXCGcdpU+Oy+lSBg44NM7nxdYMVucHmaBjs3L+SqS6xA== X-Received: by 2002:a05:6a00:410d:b0:727:3c37:d5fb with SMTP id d2e1a72fcca58-72abde8461bmr69433018b3a.16.1735977692940; Sat, 04 Jan 2025 00:01:32 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72aad816305sm27327642b3a.31.2025.01.04.00.01.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:32 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:00:56 +0900 Subject: [PATCH v8 3/9] pcie_sriov: Ensure PF and VF are mutually exclusive MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-3-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=akihiko.odaki@daynix.com; helo=mail-pj1-x1031.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A device cannot be a SR-IOV PF and a VF at the same time. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 1eb4358256de..109b2ebcccba 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -42,6 +42,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (pci_is_vf(dev)) { + error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); + return false; + } + if (total_vfs && (uint32_t)devfn + (uint32_t)(total_vfs - 1) * vf_stride >= PCI_DEVFN_MAX) { error_setg(errp, "VF addr overflows"); From patchwork Sat Jan 4 08:00:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A173FE77188 for ; Sat, 4 Jan 2025 08:03:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz6N-0003cG-6T; Sat, 04 Jan 2025 03:01:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6L-0003bv-GO for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:41 -0500 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz6J-0006gU-T5 for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:41 -0500 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-2164b1f05caso178248725ad.3 for ; Sat, 04 Jan 2025 00:01:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977698; x=1736582498; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=C4a4ZwwnrOLQXIqjgUnW4cDOfa379c3kqq/nLZmFVbU=; b=Tjxvc6VcR5CY1IIqTMQI6fl1yt+iRwwUllS/PMqMEvy48DsEr3rAGg04nrIKW038PH +ZrSmGHRaTKqsjWmQQhFXX3vDFhj9WeHSKE2czs7fsBDEff+yenFAn7bkvEiyDGSs186 zJOZjT+gySSG6OyIoDdWCNl7ZYsZAAAufcr7OWxKdv4cxqfwjLdqyrf7ksWYhKiG+lmK Y2qRZwz0XfICwKCG2njUrcAy31osWfvkGJYDmiHuwrQqi1AQ9rb3DGCfFBwOuT0FfEmy qZ/+ofaleE5S5lRrZ4wY1E+DiMxj0sZb27BqpuCPEZ0wrmgmkUiF8atb8QKLU50QsKp0 CkXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977698; x=1736582498; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C4a4ZwwnrOLQXIqjgUnW4cDOfa379c3kqq/nLZmFVbU=; b=CNMaQxYndI3p4YECsn7jcBmaPZOXAXC0AGgyEca7DzshQ7YNxZIxnZCBv1/jel83nT CLWVzjhsC1F9pYwV7IvOlcJhbGgx87v7uYPpJr8Ll38kCdKOO84V1BVNgbeJdCJpQU5/ fw1N8wXyRNF2rCGYvMR6fIsSzaFsyvXl2pO+7CK/ImDUOXlB0YYvknkHLEHECaDCm8pT EI9j8IdW/MYG5kTk9/uQx6bgaFyTA7cahRyp8SUXhBcBqTeHMBBBH4xNKmaD5mjx2WiQ d1U8RJSe34mPXapMB0/e2yDrcnb053K9AZvVO39NQXxCAvdowtx/gacn8Rkh7oTPTF8j oJgA== X-Gm-Message-State: AOJu0YwpqTnsoisQJyXSKMRHRwx0bJRAtyQupg75LWVEu96ToaJLRWEL hVtjl0pfzyGdWTyqLBQaHz7RWJfzI1Qm/uZGVMfp1RvRCrfTzW2+1jYCDbZjd2A= X-Gm-Gg: ASbGncu3oro+dSxSq9clux5zce0Nd8gBuv8ZqnQtNrEJiYGkRCi1DKkPleYfSSyIYJj 0YOnu2eu8/CvcV3smVZk7VCYd4Tq9zKgFHVhS27X1OFFUPLMJsHiv4PwV1YcUyLKmXPglJ+WzzR LGuPa+SMvam3l7HbMNUoYX6PZccWbORfdNhSrFdj9D04KmOgRN9dTsU6RgJ1s4MaDSM20LUJyKi cfLIFsNEpjyG/OEsy+HkQ4ZKzqfCcriXRH5wyu5hJZizicQHGZfZ2Dz1IXR X-Google-Smtp-Source: AGHT+IEfqUdidRKTk5Q6pQg1nqejmWKPyU9JGFXi9ZyeGK6qKQ/b8g6PCNBZ0E/7tbQtV9t2UyRqcQ== X-Received: by 2002:a17:903:2344:b0:212:996:3536 with SMTP id d9443c01a7336-219e6e8c3b8mr702853145ad.10.1735977698666; Sat, 04 Jan 2025 00:01:38 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-219dc9f68a6sm255365025ad.212.2025.01.04.00.01.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:37 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:00:57 +0900 Subject: [PATCH v8 4/9] pcie_sriov: Check PCI Express for SR-IOV PF MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-4-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62f.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org SR-IOV requires PCI Express. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 109b2ebcccba..a5b546abe8bb 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -42,6 +42,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV PF"); + return false; + } + if (pci_is_vf(dev)) { error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); return false; From patchwork Sat Jan 4 08:00:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B72CAE77197 for ; Sat, 4 Jan 2025 08:02:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz6o-0003hs-TE; Sat, 04 Jan 2025 03:02:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6S-0003dM-LY for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:49 -0500 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz6P-0006gx-JV for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:48 -0500 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-215770613dbso134104275ad.2 for ; Sat, 04 Jan 2025 00:01:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977704; x=1736582504; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=S3hhGs5D8bLiF/iQvK0+cNG3Gcgrbjo4DRCWzXYb98g=; b=dVZ9lbhj/cjWzEQH1YObF7wHuyvkHjN5mem+Ht/2JXggLDCkeTE5wigiQ0nPrPaJbD dMhMPMRHVobPcVcHzTzgEBwcfOQ70uquBN3vMeC9GScJ2zxEmX+S8BEFciWCVYoW4Q5t YoLotSjtUe6oQbqV1vVVPy98Lf1BBaB9v3Dmr1BjAWPStPOOgRqZTdSYSylDpGwC8+Uq n4NSV204G9LGTVFUKOKd94+3exGiUMCHsLJTqrcbKI0/hLPEt6Z9s8POiknhRmiMffdC 1ThAI4cYJxYFNcDkLm2aRzkBfsWwgYG8r9b/pWhlvwcVr47Lr7IwmiqJD+B9dsGSu80Z MZQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977704; x=1736582504; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S3hhGs5D8bLiF/iQvK0+cNG3Gcgrbjo4DRCWzXYb98g=; b=Aj4VOabJXLP60vEjZnhtJg+NEcQaveh2Xj1RCNsEhM6o8lGTFZFXQG1p6z+T9ORNio ts+A7xGZ55CdjqkU023uxP9dafgE3HmeX2CDe87UJYmrm5ZYJ+a4kyBKL/47GXfyVhag roXbivnklMo1KrstXi8euKzvDi9TSqRXfFL0lzzyDMvHfCvol69rZw/nLSxGy5ezk25p JUUr2NBRQ32cFdlWtr/cdqY/46Dh0R8LwLJzPvgBBbunTSpeXPkr6GgRkzlS+rpbqzZk vydWtUD9WtRJIXQxXCjOgYw/lNC1PK7OgM2LiC++ek/bc/zTTWPuC52I/EqWYdMt2PHS VB+Q== X-Gm-Message-State: AOJu0YwClxDz1wmX8kguvAXdD7bK62owhmi3DdZs6w9ucwy09YLDl2I/ Omhlys7Eo7HYWi209smVmgNqTOAjZSJ2krTjT/FUVMShAEYK2Vxs+5GhmSIzX0w= X-Gm-Gg: ASbGncvYHaa+q/tVqjnoiaYAWrL7SFmyujr/2Vjk45tdIDBSG0yldU8A0TZqtjNJI7x Xbdn14GMPHkhoEJVs7MYK38amhHnsdBcRE82rbfaP3vfb+2X5f4nDAsXP1Jpx+PzlBZL6AK3XWk gYKxmhc2rH51ahK1+k1EzROteNb6ICsZTwSYIBx+cU7yrUfg0p6m9IA1KQ2f3Cs1pgkrNsTujbm gVfQwz486+jiOGYGWV5ZxtyPSFFzW6Pgj1juvxTAtdEXOCHUpXjMR+3BYoO X-Google-Smtp-Source: AGHT+IFCXfe0rzAiC8WqsF+h3UpU94gecuAhsTQ1Quo3fc62d9USqyKK3o1zO2hiZgVj6KvTly2L3A== X-Received: by 2002:a17:902:ef08:b0:20b:6d82:acb with SMTP id d9443c01a7336-219e6e9f521mr703284305ad.23.1735977704083; Sat, 04 Jan 2025 00:01:44 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-219dc970a4esm255516395ad.104.2025.01.04.00.01.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:43 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:00:58 +0900 Subject: [PATCH v8 5/9] pcie_sriov: Allow user to create SR-IOV device MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-5-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x630.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A user can create a SR-IOV device by specifying the PF with the sriov-pf property of the VFs. The VFs must be added before the PF. A user-creatable VF must have PCIDeviceClass::sriov_vf_user_creatable set. Such a VF cannot refer to the PF because it is created before the PF. A PF that user-creatable VFs can be attached calls pcie_sriov_pf_init_from_user_created_vfs() during realization and pcie_sriov_pf_exit() when exiting. Signed-off-by: Akihiko Odaki --- include/hw/pci/pci_device.h | 6 +- include/hw/pci/pcie_sriov.h | 18 +++ hw/pci/pci.c | 62 ++++++---- hw/pci/pcie_sriov.c | 278 +++++++++++++++++++++++++++++++++++--------- 4 files changed, 286 insertions(+), 78 deletions(-) diff --git a/include/hw/pci/pci_device.h b/include/hw/pci/pci_device.h index 613f78aebf62..7af09b3202e3 100644 --- a/include/hw/pci/pci_device.h +++ b/include/hw/pci/pci_device.h @@ -38,6 +38,8 @@ struct PCIDeviceClass { uint16_t subsystem_id; /* only for header type = 0 */ const char *romfile; /* rom bar */ + + bool sriov_vf_user_creatable; }; enum PCIReqIDType { @@ -174,6 +176,8 @@ struct PCIDevice { * realizing the device. */ uint32_t max_bounce_buffer_size; + + char *sriov_pf; }; static inline int pci_intx(PCIDevice *pci_dev) @@ -206,7 +210,7 @@ static inline int pci_is_express_downstream_port(const PCIDevice *d) static inline int pci_is_vf(const PCIDevice *d) { - return d->exp.sriov_vf.pf != NULL; + return d->sriov_pf || d->exp.sriov_vf.pf != NULL; } static inline uint32_t pci_config_size(const PCIDevice *d) diff --git a/include/hw/pci/pcie_sriov.h b/include/hw/pci/pcie_sriov.h index c5d2d318d330..f75b8f22ee92 100644 --- a/include/hw/pci/pcie_sriov.h +++ b/include/hw/pci/pcie_sriov.h @@ -18,6 +18,7 @@ typedef struct PCIESriovPF { uint8_t vf_bar_type[PCI_NUM_REGIONS]; /* Store type for each VF bar */ PCIDevice **vf; /* Pointer to an array of num_vfs VF devices */ + bool vf_user_created; /* If VFs are created by user */ } PCIESriovPF; typedef struct PCIESriovVF { @@ -40,6 +41,23 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory); +/** + * pcie_sriov_pf_init_from_user_created_vfs() - Initialize PF with user-created + * VFs. + * @dev: A PCIe device being realized. + * @offset: The offset of the SR-IOV capability. + * @errp: pointer to Error*, to store an error if it happens. + * + * Return: The size of added capability. 0 if the user did not create VFs. + * -1 if failed. + */ +int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, + uint16_t offset, + Error **errp); + +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp); +void pcie_sriov_unregister_device(PCIDevice *dev); + /* * Default (minimal) page size support values * as required by the SR/IOV standard: diff --git a/hw/pci/pci.c b/hw/pci/pci.c index ba53302ecc5f..7e9dbab949ce 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -101,6 +101,7 @@ static const Property pci_props[] = { QEMU_PCIE_ARI_NEXTFN_1_BITNR, false), DEFINE_PROP_SIZE32("x-max-bounce-buffer-size", PCIDevice, max_bounce_buffer_size, DEFAULT_MAX_BOUNCE_BUFFER_SIZE), + DEFINE_PROP_STRING("sriov-pf", PCIDevice, sriov_pf), DEFINE_PROP_BIT("x-pcie-ext-tag", PCIDevice, cap_present, QEMU_PCIE_EXT_TAG_BITNR, true), { .name = "busnr", .info = &prop_pci_busnr }, @@ -1029,13 +1030,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) dev->config[PCI_HEADER_TYPE] |= PCI_HEADER_TYPE_MULTI_FUNCTION; } - /* - * With SR/IOV and ARI, a device at function 0 need not be a multifunction - * device, as it may just be a VF that ended up with function 0 in - * the legacy PCI interpretation. Avoid failing in such cases: - */ - if (pci_is_vf(dev) && - dev->exp.sriov_vf.pf->cap_present & QEMU_PCI_CAP_MULTIFUNCTION) { + /* SR/IOV is not handled here. */ + if (pci_is_vf(dev)) { return; } @@ -1068,7 +1064,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) } /* function 0 indicates single function, so function > 0 must be NULL */ for (func = 1; func < PCI_FUNC_MAX; ++func) { - if (bus->devices[PCI_DEVFN(slot, func)]) { + PCIDevice *device = bus->devices[PCI_DEVFN(slot, func)]; + if (device && !pci_is_vf(device)) { error_setg(errp, "PCI: %x.0 indicates single function, " "but %x.%x is already populated.", slot, slot, func); @@ -1356,6 +1353,7 @@ static void pci_qdev_unrealize(DeviceState *dev) pci_unregister_io_regions(pci_dev); pci_del_option_rom(pci_dev); + pcie_sriov_unregister_device(pci_dev); if (pc->exit) { pc->exit(pci_dev); @@ -1387,7 +1385,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, pcibus_t size = memory_region_size(memory); uint8_t hdr_type; - assert(!pci_is_vf(pci_dev)); /* VFs must use pcie_sriov_vf_register_bar */ assert(region_num >= 0); assert(region_num < PCI_NUM_REGIONS); assert(is_power_of_2(size)); @@ -1398,7 +1395,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, assert(hdr_type != PCI_HEADER_TYPE_BRIDGE || region_num < 2); r = &pci_dev->io_regions[region_num]; - r->addr = PCI_BAR_UNMAPPED; r->size = size; r->type = type; r->memory = memory; @@ -1406,22 +1402,35 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, ? pci_get_bus(pci_dev)->address_space_io : pci_get_bus(pci_dev)->address_space_mem; - wmask = ~(size - 1); - if (region_num == PCI_ROM_SLOT) { - /* ROM enable bit is writable */ - wmask |= PCI_ROM_ADDRESS_ENABLE; - } - - addr = pci_bar(pci_dev, region_num); - pci_set_long(pci_dev->config + addr, type); + if (pci_is_vf(pci_dev)) { + PCIDevice *pf = pci_dev->exp.sriov_vf.pf; + assert(!pf || type == pf->exp.sriov_pf.vf_bar_type[region_num]); - if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && - r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { - pci_set_quad(pci_dev->wmask + addr, wmask); - pci_set_quad(pci_dev->cmask + addr, ~0ULL); + r->addr = pci_bar_address(pci_dev, region_num, r->type, r->size); + if (r->addr != PCI_BAR_UNMAPPED) { + memory_region_add_subregion_overlap(r->address_space, + r->addr, r->memory, 1); + } } else { - pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); - pci_set_long(pci_dev->cmask + addr, 0xffffffff); + r->addr = PCI_BAR_UNMAPPED; + + wmask = ~(size - 1); + if (region_num == PCI_ROM_SLOT) { + /* ROM enable bit is writable */ + wmask |= PCI_ROM_ADDRESS_ENABLE; + } + + addr = pci_bar(pci_dev, region_num); + pci_set_long(pci_dev->config + addr, type); + + if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && + r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { + pci_set_quad(pci_dev->wmask + addr, wmask); + pci_set_quad(pci_dev->cmask + addr, ~0ULL); + } else { + pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); + pci_set_long(pci_dev->cmask + addr, 0xffffffff); + } } } @@ -2182,6 +2191,11 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp) } } + if (!pcie_sriov_register_device(pci_dev, errp)) { + pci_qdev_unrealize(DEVICE(pci_dev)); + return; + } + /* * A PCIe Downstream Port that do not have ARI Forwarding enabled must * associate only Device 0 with the device attached to the bus diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index a5b546abe8bb..08f707e847fd 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -15,11 +15,12 @@ #include "hw/pci/pcie.h" #include "hw/pci/pci_bus.h" #include "hw/qdev-properties.h" -#include "qemu/error-report.h" #include "qemu/range.h" #include "qapi/error.h" #include "trace.h" +static GHashTable *pfs; + static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) { for (uint16_t i = 0; i < total_vfs; i++) { @@ -31,13 +32,43 @@ static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) dev->exp.sriov_pf.vf = NULL; } -bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, - const char *vfname, uint16_t vf_dev_id, - uint16_t init_vfs, uint16_t total_vfs, - uint16_t vf_offset, uint16_t vf_stride, - Error **errp) +static void register_vfs(PCIDevice *dev) +{ + uint16_t num_vfs; + uint16_t i; + uint16_t sriov_cap = dev->exp.sriov_cap; + + assert(sriov_cap > 0); + num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); + + trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn), num_vfs); + for (i = 0; i < num_vfs; i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + } + + pci_set_word(dev->wmask + sriov_cap + PCI_SRIOV_NUM_VF, 0); +} + +static void unregister_vfs(PCIDevice *dev) +{ + uint8_t *cfg = dev->config + dev->exp.sriov_cap; + uint16_t i; + + trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn)); + for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + } + + pci_set_word(dev->wmask + dev->exp.sriov_cap + PCI_SRIOV_NUM_VF, 0xffff); +} + +static bool pcie_sriov_pf_init_common(PCIDevice *dev, uint16_t offset, + uint16_t vf_dev_id, uint16_t init_vfs, + uint16_t total_vfs, uint16_t vf_offset, + uint16_t vf_stride, Error **errp) { - BusState *bus = qdev_get_parent_bus(&dev->qdev); int32_t devfn = dev->devfn + vf_offset; uint8_t *cfg = dev->config + offset; uint8_t *wmask; @@ -94,6 +125,28 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, qdev_prop_set_bit(&dev->qdev, "multifunction", true); + return true; +} + +bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, + const char *vfname, uint16_t vf_dev_id, + uint16_t init_vfs, uint16_t total_vfs, + uint16_t vf_offset, uint16_t vf_stride, + Error **errp) +{ + BusState *bus = qdev_get_parent_bus(&dev->qdev); + int32_t devfn = dev->devfn + vf_offset; + + if (pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, init_vfs, + total_vfs, vf_offset, vf_stride, errp)) { + return false; + } + dev->exp.sriov_pf.vf = g_new(PCIDevice *, total_vfs); for (uint16_t i = 0; i < total_vfs; i++) { @@ -123,7 +176,22 @@ void pcie_sriov_pf_exit(PCIDevice *dev) { uint8_t *cfg = dev->config + dev->exp.sriov_cap; - unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + if (dev->exp.sriov_pf.vf_user_created) { + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t total_vfs = pci_get_word(dev->config + PCI_SRIOV_TOTAL_VF); + uint16_t vf_dev_id = pci_get_word(dev->config + PCI_SRIOV_VF_DID); + + unregister_vfs(dev); + + for (uint16_t i = 0; i < total_vfs; i++) { + dev->exp.sriov_pf.vf[i]->exp.sriov_vf.pf = NULL; + + pci_config_set_vendor_id(dev->exp.sriov_pf.vf[i]->config, ven_id); + pci_config_set_device_id(dev->exp.sriov_pf.vf[i]->config, vf_dev_id); + } + } else { + unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + } } void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, @@ -156,69 +224,173 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory) { - PCIIORegion *r; - PCIBus *bus = pci_get_bus(dev); uint8_t type; - pcibus_t size = memory_region_size(memory); - assert(pci_is_vf(dev)); /* PFs must use pci_register_bar */ - assert(region_num >= 0); - assert(region_num < PCI_NUM_REGIONS); + assert(dev->exp.sriov_vf.pf); type = dev->exp.sriov_vf.pf->exp.sriov_pf.vf_bar_type[region_num]; - if (!is_power_of_2(size)) { - error_report("%s: PCI region size must be a power" - " of two - type=0x%x, size=0x%"FMT_PCIBUS, - __func__, type, size); - exit(1); - } + return pci_register_bar(dev, region_num, type, memory); +} - r = &dev->io_regions[region_num]; - r->memory = memory; - r->address_space = - type & PCI_BASE_ADDRESS_SPACE_IO - ? bus->address_space_io - : bus->address_space_mem; - r->size = size; - r->type = type; - - r->addr = pci_bar_address(dev, region_num, r->type, r->size); - if (r->addr != PCI_BAR_UNMAPPED) { - memory_region_add_subregion_overlap(r->address_space, - r->addr, r->memory, 1); - } +static gint compare_vf_devfns(gconstpointer a, gconstpointer b) +{ + return (*(PCIDevice **)a)->devfn - (*(PCIDevice **)b)->devfn; } -static void register_vfs(PCIDevice *dev) +int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, + uint16_t offset, + Error **errp) { - uint16_t num_vfs; + GPtrArray *pf; + PCIDevice **vfs; + BusState *bus = qdev_get_parent_bus(DEVICE(dev)); + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t vf_dev_id; + uint16_t vf_offset; + uint16_t vf_stride; uint16_t i; - uint16_t sriov_cap = dev->exp.sriov_cap; - assert(sriov_cap > 0); - num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); + if (!pfs || !dev->qdev.id) { + return 0; + } - trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn), num_vfs); - for (i = 0; i < num_vfs; i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + pf = g_hash_table_lookup(pfs, dev->qdev.id); + if (!pf) { + return 0; } - pci_set_word(dev->wmask + sriov_cap + PCI_SRIOV_NUM_VF, 0); + if (pf->len > UINT16_MAX) { + error_setg(errp, "too many VFs"); + return -1; + } + + g_ptr_array_sort(pf, compare_vf_devfns); + vfs = (void *)pf->pdata; + + if (vfs[0]->devfn <= dev->devfn) { + error_setg(errp, "a VF function number is less than the PF function number"); + return -1; + } + + vf_dev_id = pci_get_word(vfs[0]->config + PCI_DEVICE_ID); + vf_offset = vfs[0]->devfn - dev->devfn; + vf_stride = pf->len < 2 ? 0 : vfs[1]->devfn - vfs[0]->devfn; + + for (i = 0; i < pf->len; i++) { + if (bus != qdev_get_parent_bus(&vfs[i]->qdev)) { + error_setg(errp, "SR-IOV VF parent bus mismatches with PF"); + return -1; + } + + if (ven_id != pci_get_word(vfs[i]->config + PCI_VENDOR_ID)) { + error_setg(errp, "SR-IOV VF vendor ID mismatches with PF"); + return -1; + } + + if (vf_dev_id != pci_get_word(vfs[i]->config + PCI_DEVICE_ID)) { + error_setg(errp, "inconsistent SR-IOV VF device IDs"); + return -1; + } + + for (size_t j = 0; j < PCI_NUM_REGIONS; j++) { + if (vfs[i]->io_regions[j].size != vfs[0]->io_regions[j].size || + vfs[i]->io_regions[j].type != vfs[0]->io_regions[j].type) { + error_setg(errp, "inconsistent SR-IOV BARs"); + return -1; + } + } + + if (vfs[i]->devfn - vfs[0]->devfn != vf_stride * i) { + error_setg(errp, "inconsistent SR-IOV stride"); + return -1; + } + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, pf->len, + pf->len, vf_offset, vf_stride, errp)) { + return -1; + } + + for (i = 0; i < pf->len; i++) { + vfs[i]->exp.sriov_vf.pf = dev; + vfs[i]->exp.sriov_vf.vf_number = i; + + /* set vid/did according to sr/iov spec - they are not used */ + pci_config_set_vendor_id(vfs[i]->config, 0xffff); + pci_config_set_device_id(vfs[i]->config, 0xffff); + } + + dev->exp.sriov_pf.vf = vfs; + dev->exp.sriov_pf.vf_user_created = true; + + for (i = 0; i < PCI_NUM_REGIONS; i++) { + PCIIORegion *region = &vfs[0]->io_regions[i]; + + if (region->size) { + pcie_sriov_pf_init_vf_bar(dev, i, region->type, region->size); + } + } + + return PCI_EXT_CAP_SRIOV_SIZEOF; } -static void unregister_vfs(PCIDevice *dev) +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp) { - uint8_t *cfg = dev->config + dev->exp.sriov_cap; - uint16_t i; + if (!dev->exp.sriov_pf.vf && dev->qdev.id && + pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } - trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn)); - for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + if (dev->sriov_pf) { + PCIDevice *pci_pf; + GPtrArray *pf; + + if (!PCI_DEVICE_GET_CLASS(dev)->sriov_vf_user_creatable) { + error_setg(errp, "user cannot create SR-IOV VF with this device type"); + return false; + } + + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV VF"); + return false; + } + + if (!pci_qdev_find_device(dev->sriov_pf, &pci_pf)) { + error_setg(errp, "PCI device specified as SR-IOV PF already exists"); + return false; + } + + if (!pfs) { + pfs = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, NULL); + } + + pf = g_hash_table_lookup(pfs, dev->sriov_pf); + if (!pf) { + pf = g_ptr_array_new(); + g_hash_table_insert(pfs, g_strdup(dev->sriov_pf), pf); + } + + g_ptr_array_add(pf, dev); } - pci_set_word(dev->wmask + dev->exp.sriov_cap + PCI_SRIOV_NUM_VF, 0xffff); + return true; +} + +void pcie_sriov_unregister_device(PCIDevice *dev) +{ + if (dev->sriov_pf && pfs) { + GPtrArray *pf = g_hash_table_lookup(pfs, dev->sriov_pf); + + if (pf) { + g_ptr_array_remove_fast(pf, dev); + + if (!pf->len) { + g_hash_table_remove(pfs, dev->sriov_pf); + g_ptr_array_free(pf, FALSE); + } + } + } } void pcie_sriov_config_write(PCIDevice *dev, uint32_t address, @@ -314,7 +486,7 @@ void pcie_sriov_pf_add_sup_pgsize(PCIDevice *dev, uint16_t opt_sup_pgsize) uint16_t pcie_sriov_vf_number(PCIDevice *dev) { - assert(pci_is_vf(dev)); + assert(dev->exp.sriov_vf.pf); return dev->exp.sriov_vf.vf_number; } From patchwork Sat Jan 4 08:00:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926160 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 40B4BE77188 for ; Sat, 4 Jan 2025 08:03:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz7N-0004fk-1W; Sat, 04 Jan 2025 03:02:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6W-0003gn-8G for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:53 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz6U-0006hf-Jf for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:51 -0500 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-2164b1f05caso178250265ad.3 for ; Sat, 04 Jan 2025 00:01:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977709; x=1736582509; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ssDQc8psUCNDRLn18K6TEiijD4IORPJi7OPvf7+BFzQ=; b=FSoXsES2QDl1BpdQPc70+LzQ+9nt96zmcxlXY0cLrVtjHy+GhgsBWMvPQJi2ARRjM9 JmI41nZJUUbSKT+oMS1wEJ65/2lN7E5bR4E2rOOanpc0fJSUbCv0/tIDGi+qKGZfs0o6 Fs5UOzKG+IH2HKfwEqn6x79dD8+iEQvu8zA9Z+ub6PgMKYH57rqR7dISLIXdQKVFZjdb 9MUxsxhbwiCNOuBIri1AEViGgCm2KzGGOlos3kj5WWK+ZhdfZJmQU9hoeIy9bHPxNXwQ jyEErAokaQvEUcVQq8/erW6n5deCEqyI7X6wr02Wvp9AJbJ2T/y281ZTw1zf1boPFk9+ v+zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977709; x=1736582509; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ssDQc8psUCNDRLn18K6TEiijD4IORPJi7OPvf7+BFzQ=; b=ihUFoDXLE3YKpqP30LsHXgU4eZHMEnq4qNi/2VK7W3Y8xLYoTunMMFGSG6jWlEc9Ow 2+htpO/UUACjnMcAGf8xwg2EKgs6EhsLKHQbQNNZeLv8b1S2MMsa0haOVwFXrcVAwa8n gfHD4Ea9PM4ANufC6tIWrcRm2KqIsyby3k/UIXSzRTuBCvTMrme/S+sL3L4lt7FHpsIk j8bh6AffVx+iRLgFQsWMcPJya4qkh9SM4OUDXNfY05TnMjXxqG4BoVykDyIA3Sy07VJW exhCWLLgunbfHqkE+H4o+rVhLbYF1hRG8KVBeP3uvidIsQtSFMdbw9EgNYmfO05QxmM3 BAWw== X-Gm-Message-State: AOJu0Yx88zGGF7198sxfuQKfFUuKO5YpJLCOh9ElrwwW/1kqp2mFEtaY buqUUzmAjFJS0qE6UrpfPUKr/i19ucoeuZbUoZeC9eS4fdGoA1jf4PCV02L0LnM= X-Gm-Gg: ASbGnctvd3eIEeKq4eMU4N/PTRm9kYmYhaW2WxpO96R5lSTn7qPdCmUzo/wPul6vBlC 3CiQKhBhyW3AIPsUFfe/i/8O3xKDjWjtPsPzMJSNMiEYhG5uH2jQadMNweCjgb6E4Q9MX5LLFDA CENzPMGHpz4+iEYS186pPI54bAUeCS1KcJdCMydRAbqDGNIP5RpQD0dH3V0wo0goYTXYWA8qaZb JWX5omtNucSsmgZPJOEsqYAAMqQGOwd8oXqX+vzEBHJbeW4AZPw3as3gkH8 X-Google-Smtp-Source: AGHT+IGrkshHhjCc4F9VOMMdoGuPFumzPj9vIMx+ZtCuNkrhjFoXjtm0za9AtzVF9UDd9IZwKXG6bQ== X-Received: by 2002:a05:6a00:1947:b0:725:456e:76e with SMTP id d2e1a72fcca58-72abdd8caf2mr75773205b3a.6.1735977709444; Sat, 04 Jan 2025 00:01:49 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72aad8fb7dbsm27356326b3a.133.2025.01.04.00.01.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:49 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:00:59 +0900 Subject: [PATCH v8 6/9] virtio-pci: Implement SR-IOV PF MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-6-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allow user to attach SR-IOV VF to a virtio-pci PF. Signed-off-by: Akihiko Odaki --- include/hw/virtio/virtio-pci.h | 1 + hw/virtio/virtio-pci.c | 20 +++++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h index 971c5fabd444..b473274834e9 100644 --- a/include/hw/virtio/virtio-pci.h +++ b/include/hw/virtio/virtio-pci.h @@ -155,6 +155,7 @@ struct VirtIOPCIProxy { uint32_t modern_io_bar_idx; uint32_t modern_mem_bar_idx; int config_cap; + uint16_t last_pcie_cap_offset; uint32_t flags; bool disable_modern; bool ignore_backend_features; diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index c773a9130c7e..5ea8e7f6d0a8 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1962,6 +1962,7 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) uint8_t *config; uint32_t size; VirtIODevice *vdev = virtio_bus_get_device(bus); + int16_t res; /* * Virtio capabilities present without @@ -2109,6 +2110,14 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) pci_register_bar(&proxy->pci_dev, proxy->legacy_io_bar_idx, PCI_BASE_ADDRESS_SPACE_IO, &proxy->bar); } + + res = pcie_sriov_pf_init_from_user_created_vfs(&proxy->pci_dev, + proxy->last_pcie_cap_offset, + errp); + if (res > 0) { + proxy->last_pcie_cap_offset += res; + virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + } } static void virtio_pci_device_unplugged(DeviceState *d) @@ -2199,7 +2208,7 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) if (pcie_port && pci_is_express(pci_dev)) { int pos; - uint16_t last_pcie_cap_offset = PCI_CONFIG_SPACE_SIZE; + proxy->last_pcie_cap_offset = PCI_CONFIG_SPACE_SIZE; pos = pcie_endpoint_cap_init(pci_dev, 0); assert(pos > 0); @@ -2219,9 +2228,9 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) pci_set_word(pci_dev->config + pos + PCI_PM_PMC, 0x3); if (proxy->flags & VIRTIO_PCI_FLAG_AER) { - pcie_aer_init(pci_dev, PCI_ERR_VER, last_pcie_cap_offset, + pcie_aer_init(pci_dev, PCI_ERR_VER, proxy->last_pcie_cap_offset, PCI_ERR_SIZEOF, NULL); - last_pcie_cap_offset += PCI_ERR_SIZEOF; + proxy->last_pcie_cap_offset += PCI_ERR_SIZEOF; } if (proxy->flags & VIRTIO_PCI_FLAG_INIT_DEVERR) { @@ -2246,9 +2255,9 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) } if (proxy->flags & VIRTIO_PCI_FLAG_ATS) { - pcie_ats_init(pci_dev, last_pcie_cap_offset, + pcie_ats_init(pci_dev, proxy->last_pcie_cap_offset, proxy->flags & VIRTIO_PCI_FLAG_ATS_PAGE_ALIGNED); - last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF; + proxy->last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF; } if (proxy->flags & VIRTIO_PCI_FLAG_INIT_FLR) { @@ -2276,6 +2285,7 @@ static void virtio_pci_exit(PCIDevice *pci_dev) !pci_bus_is_root(pci_get_bus(pci_dev)); bool modern_pio = proxy->flags & VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY; + pcie_sriov_pf_exit(&proxy->pci_dev); msix_uninit_exclusive_bar(pci_dev); if (proxy->flags & VIRTIO_PCI_FLAG_AER && pcie_port && pci_is_express(pci_dev)) { From patchwork Sat Jan 4 08:01:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 34888E77188 for ; Sat, 4 Jan 2025 08:03:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz6t-000422-AL; Sat, 04 Jan 2025 03:02:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6a-0003ks-QV for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:58 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz6Z-0006i2-Fm for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:01:56 -0500 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-21661be2c2dso165240065ad.1 for ; Sat, 04 Jan 2025 00:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977714; x=1736582514; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=J0JQM/aP0W17LDTb+wBV4wTDP/u13ijT1gRctKp4C3w=; b=v14/OGRA5ZbQJolrmycXl0PVRIuKbAXSOkJpNg7uw8xX45Sxavqah35AQ6Ui3ZPBxc y6s/9q1+mPm6y1UhbmC9GhBrnGqoLFFK7LhQyQt/rnmGGXLEOSg18dyX1S4AECy1RJrV jHezQf8zIpOHi7VnrGXCaE2YebTCcQNK555GVi+S3bZq8FJK+Z+bs6rVahBQaE2zlcjg 1pX0tP3eagKFE8EYoqsSyu31s8yc2xeuqrQnL3mKxBbg0BifQLfvwj7jEHEMlmyNe3SM hZZUgx5lNyOPJvdJUs6u2NrzkdfXM+a2OAV8yXPKQgALx3dyLizKou8ZQfc5cf3K8m7G dMeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977714; x=1736582514; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J0JQM/aP0W17LDTb+wBV4wTDP/u13ijT1gRctKp4C3w=; b=vL+TsVw9rMmy8kOPZhHr1fmvHpXl7sHhBqncqm64orvrc9lS4L6IUrggDRgVrI0FUN aHdYa+yMtAvRATPX6CM0ZcR1klJYm1JpP43TqN0R4Ji+hFx+Leh9Rfn6msy+oYstKLu/ 63/DdlH24Pq0GgntFetHoNKHHSmgGixDWoFteOm2KEUYBEa30N6tVbBNwNAPOcuwJwSg KjTLN0rYt/al0mN0QdrcbvEC28rb1vOeZsqQEO2qmwvYLs0YoCvFYQ6j7XG6/GkAMq0K NTVSssbb8lqoaSiwf8vD6j+Fp9rBBzcaHQP2Y+YbbJCTNU5Uba2+TiQz/zkz6nRKkjTZ m5ZA== X-Gm-Message-State: AOJu0Yx68m0LiM3YVwsNd9szj6BdKHSdkD9zkGav1zQqdCrYphvJIbFY nSDHS9tAPSedViW4U/9A5hQs31xJf5Q8DNoaK2LPHw8Ke42xZdiTn42JWtj4BLg= X-Gm-Gg: ASbGncsCZ8HaAVPjXVV8utMQyXvxX/yyWAJ/MHBsrmAxFCdFnjhaRRsqGsimHYPUAJZ RBGA31HE5GNssdwx4vBpF99ughjMrbyo059q60fS0D1oAhr0f/4n72KMvpnrgj8AsK1fO+yFc3w mzbPMhqgHCNO59AugzVtQbfne2JX5BAUy2xBrvLC/qBYb/HtcxBLeB2Sw/hCZ4h1qdDVm7DDQXV XOFqjERkraRlQvtgel/aLadmmb1R5UHtRWMFK698U9eu0XHV5aRW+eIxSZu X-Google-Smtp-Source: AGHT+IGhZtYVn3vNwV0TngVRHeMJ7ixq5+jl0fpOHdJ+oA4yH35WmqVBPx3HJYpZRVRvzBHA+NiFNw== X-Received: by 2002:a05:6a21:8dc2:b0:1e1:ccf3:a72 with SMTP id adf61e73a8af0-1e5e07ffd40mr78064894637.28.1735977714265; Sat, 04 Jan 2025 00:01:54 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72aad836bd9sm27512968b3a.78.2025.01.04.00.01.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:53 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:01:00 +0900 Subject: [PATCH v8 7/9] virtio-net: Implement SR-IOV VF MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-7-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A virtio-net device can be added as a SR-IOV VF to another virtio-pci device that will be the PF. Signed-off-by: Akihiko Odaki --- hw/virtio/virtio-net-pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/virtio/virtio-net-pci.c b/hw/virtio/virtio-net-pci.c index e18953ad674b..430b727ea3f4 100644 --- a/hw/virtio/virtio-net-pci.c +++ b/hw/virtio/virtio-net-pci.c @@ -74,6 +74,7 @@ static void virtio_net_pci_class_init(ObjectClass *klass, void *data) k->device_id = PCI_DEVICE_ID_VIRTIO_NET; k->revision = VIRTIO_PCI_ABI_VERSION; k->class_id = PCI_CLASS_NETWORK_ETHERNET; + k->sriov_vf_user_creatable = true; set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); device_class_set_props(dc, virtio_net_properties); vpciklass->realize = virtio_net_pci_realize; From patchwork Sat Jan 4 08:01:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926162 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 528EEE77199 for ; Sat, 4 Jan 2025 08:03:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz7E-0004NP-Tq; Sat, 04 Jan 2025 03:02:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6j-0003ot-S2 for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:02:10 -0500 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz6h-0006ix-NE for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:02:05 -0500 Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-2166022c5caso159893335ad.2 for ; Sat, 04 Jan 2025 00:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977719; x=1736582519; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=urRn/q5yC0SUy3lJXp0X0hOtR2WXLfh8pzdTuyPYr0Q=; b=S0uO3pFTsjQRIHOeZjWJhC1IVkRVqqEJ2kBZVY4wP/S8FqRQN1pHsGWsl4tWFsAnl2 Crcyl2i7+3sBiRty78dBzd/iINCLYoi5tgV9i6v5Pj6AY7Xgf80pgHmKYaIR/myWtk1S of46MN5k1tRk0Cp9RdFQwl7EiFlmdN1bdO8cNGHoVnDN2NMUvLS7PGxTDLd5bny4lDcw dZVMRz0LnqDMFqtNoWUGF04bGUHgKJ3vvNGIkPup9emXVIAS2LsOZHeMyDN8u3R1tckU kI5gQmvjOIpBg3Ib9+ruCo+4xpkvpSfW1DCeKy2bvixyxxJWQ5BIKXVJ5IGzFfB5nDhh 4RXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977719; x=1736582519; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=urRn/q5yC0SUy3lJXp0X0hOtR2WXLfh8pzdTuyPYr0Q=; b=rM11cY6dkzGCr0iEFz3UktI+mzxexMS9rEqpzAXRsCy1Qk1pbaCm69TIeOl9GnldEk cwvjvBzjrd2k56Jihl3vofPPwWg5izgF1Fzxj9AoMGByEXIi+UvRZ/ipJKsR9KYa+/zT Wd8EJpD9LKLq4UsWpkk070FCF/fPFN7Vd7a0Y/QUQBOz7o7jBAh622gsKaoCYVUwQEqd mEiYQ2DEgdh7Audk0CA4X7iTKp7yyyLCldN8ven8zE9fIG/2DiIAd9BF1ewoAL2ZAzGm InYZ3eo/LgQMRlfiGQKxmadbFU861LSNexEZ8bC066S7JuMjuc5FjL+JT4VPCA+/Ho0B R6VQ== X-Gm-Message-State: AOJu0YwXp0zB4x/p85SmX8nyc5NpgUoUaPy3cug8KNd0T4HdGValEBX/ uozIsFa5wGutMizkd+bbJ8BoA5E6jf3KkhO44FbnpF7OgQsGKPxojtNYeY0kW/A= X-Gm-Gg: ASbGncvkdetduHOJF6euvFBG+cmFyNakqvc+PhD18s+X4jwGado8zRf/lXQOfoTLIPX VVR0x3fs1dXSPlxHUIZjjIVFdywZNbVmIQKyBZOauOK80XdTBmwpu9DNpHUBlLnZPtqJoa/Jg/D YXk1ujbcsZHUpkxQGSs4gANelTLq9Th+2xxfkreKJc4kxK83heVUuJR2G5aiBXjA+jweJvwAVOl oEoq1ocXRDTYokP6BNYE3+q/OukWh3aToyC1tXtYv+F9uKJ3LzZJSjJEozK X-Google-Smtp-Source: AGHT+IFzqH3FwxnV0+2gacVSJBofHG086EFQkXM1PAAyZWgyzEmRMm8ot1fnJK6bWc3WYY1YQsj0Og== X-Received: by 2002:a17:902:db05:b0:216:725c:a11a with SMTP id d9443c01a7336-219e6e8c9ccmr655314325ad.10.1735977718995; Sat, 04 Jan 2025 00:01:58 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-219dc9cddf2sm256588875ad.166.2025.01.04.00.01.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:01:58 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:01:01 +0900 Subject: [PATCH v8 8/9] docs: Document composable SR-IOV device MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-8-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x636.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Akihiko Odaki --- MAINTAINERS | 1 + docs/system/index.rst | 1 + docs/system/sriov.rst | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 38a290e9c2ce..b0821a17c4fa 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2006,6 +2006,7 @@ F: hw/pci-bridge/* F: qapi/pci.json F: docs/pci* F: docs/specs/*pci* +F: docs/system/sriov.rst PCIE DOE M: Huai-Cheng Kuo diff --git a/docs/system/index.rst b/docs/system/index.rst index c21065e51932..718e9d3c56bb 100644 --- a/docs/system/index.rst +++ b/docs/system/index.rst @@ -39,3 +39,4 @@ or Hypervisor.Framework. multi-process confidential-guest-support vm-templating + sriov diff --git a/docs/system/sriov.rst b/docs/system/sriov.rst new file mode 100644 index 000000000000..a851a66a4b8b --- /dev/null +++ b/docs/system/sriov.rst @@ -0,0 +1,36 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +Compsable SR-IOV device +======================= + +SR-IOV (Single Root I/O Virtualization) is an optional extended capability of a +PCI Express device. It allows a single physical function (PF) to appear as +multiple virtual functions (VFs) for the main purpose of eliminating software +overhead in I/O from virtual machines. + +There are devices with predefined SR-IOV configurations, but it is also possible +to compose an SR-IOV device yourself. Composing an SR-IOV device is currently +only supported by virtio-net-pci. + +Users can configure an SR-IOV-capable virtio-net device by adding +virtio-net-pci functions to a bus. Below is a command line example: + +.. code-block:: shell + + -netdev user,id=n -netdev user,id=o + -netdev user,id=p -netdev user,id=q + -device pcie-root-port,id=b + -device virtio-net-pci,bus=b,addr=0x0.0x3,netdev=q,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x2,netdev=p,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x1,netdev=o,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x0,netdev=n,id=f + +The VFs specify the paired PF with ``sriov-pf`` property. The PF must be +added after all VFs. It is the user's responsibility to ensure that VFs have +function numbers larger than one of the PF, and that the function numbers +have a consistent stride. + +You may also need to perform additional steps to activate the SR-IOV feature on +your guest. For Linux, refer to [1]_. + +.. [1] https://docs.kernel.org/PCI/pci-iov-howto.html From patchwork Sat Jan 4 08:01:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13926157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A8D87E77188 for ; Sat, 4 Jan 2025 08:02:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tTz73-00049T-Gn; Sat, 04 Jan 2025 03:02:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tTz6m-0003s4-4Q for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:02:10 -0500 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tTz6j-0006kO-Oc for qemu-devel@nongnu.org; Sat, 04 Jan 2025 03:02:07 -0500 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2ef714374c0so15571200a91.0 for ; Sat, 04 Jan 2025 00:02:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1735977724; x=1736582524; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ysOqYR5lQLZYXNjeXy0RSiaXesPdODAKqoYiZJpO200=; b=0RQq/heAPXtGxnNiltWQFoZXtkXfkFBWu7kkt4mkMsc70+CsE62hZaBUtHrkM2dJ+N dmJLKGgIZnOdm0vigbwjrEvKsu/pR3KPtFDjItnXr6fLtoENTQTsW5rvZQlU5/Cjj3lS frhc49jdIpktBb0i0UrJRcoBXoPLdPvJCVOwDGnC2kC+cyTts6eZWP4xZXYPw0gk2wzn ch18X2yIW/LHSAHfkRd/IVrpiM7AEJJAAwTgKF9IuYlGl2QKkMU80y4F3p9yR0xRmduV wFlGceVRal57sF2F4SyUvwbxVSfKzKAQrX+kTl7oW9nG24DIyFhLp2cD0EtSoYVH4KRA NCtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735977724; x=1736582524; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ysOqYR5lQLZYXNjeXy0RSiaXesPdODAKqoYiZJpO200=; b=Ow9Sorjcwo5kb10QS6PD+vHVBHBFWdSHduX84QgohUQQ/RfL6cDSzS5CiEast8teyA fbfdYnFrHcAk3qWpo/SEmyYh1M1/BdCRAWqPjlmzFvK52hzjbBvhurNmNBe8hMixuNb7 Dk+CLV7UbmPCtJlebwDNIgJcpmPVD3uNrBiolkX5d+pRjcQmysyPCgM1eoo4V2T3IKT6 znx/BHtvirO8tGavFTWnNO6HFRATQ89uDCl91Dzafq/cu82R6SC0fLQ7W44Tzc3E5plu ykblGh0uyY2istCGRBUGZU7NaHhHpxehjK2597C/WU/+xAGo6XfUzlghn5Sjnv2ZWPGC Hbow== X-Gm-Message-State: AOJu0YxL0NOK73SwCNwe1CutgPZcATtBd5ob/9BGmGeBAUMlpxefnEQv DcApEqcYH/B1LhELKGBB7CiXqiCVbgBrKX4sCAX837/ZVx28D5QN0NR9OH/p7M0= X-Gm-Gg: ASbGncuXjUVLBd9jPqrg8+zUBz2JBttVK3f3nwkTTDPmkXVWD1oX703emkuPTUXlZyl HaJt7OWH5mUf7+p54WMZf17bRChETOeQVIw5jR7Twsqrr6BEedhrDGW6P4cpJEBcLHZejwELzb3 oOerTOPdjJ4XjeuZ9L3MhciVjF5UjUd1gXQ0XZ+YRKsuAioHHR9/NfPKJWnuKEauE15YyI4ev5e MESuAXzaqrcYHzd55bu0Wthqd7P+0z3YJZ/UB1nzwIxHp2ADnkGTDIkTDQp X-Google-Smtp-Source: AGHT+IF129YrYoArS1xJ3aeUSbjZ57zkZJGR7G5Y9unzaUQaIUsizP2Qt6vV1oDp7Y6cLUvOqoAXtw== X-Received: by 2002:a17:90b:5150:b0:2f2:a974:fc11 with SMTP id 98e67ed59e1d1-2f4438e4576mr81496209a91.17.1735977724333; Sat, 04 Jan 2025 00:02:04 -0800 (PST) Received: from localhost ([157.82.207.107]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-219dc972251sm257158295ad.96.2025.01.04.00.02.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 04 Jan 2025 00:02:04 -0800 (PST) From: Akihiko Odaki Date: Sat, 04 Jan 2025 17:01:02 +0900 Subject: [PATCH v8 9/9] pcie_sriov: Make a PCI device with user-created VF ARI-capable MIME-Version: 1.0 Message-Id: <20250104-sriov-v8-9-56144cfdc7d9@daynix.com> References: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> In-Reply-To: <20250104-sriov-v8-0-56144cfdc7d9@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, devel@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=akihiko.odaki@daynix.com; helo=mail-pj1-x1030.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Akihiko Odaki --- docs/system/sriov.rst | 3 ++- include/hw/pci/pcie_sriov.h | 7 +++++-- hw/pci/pcie_sriov.c | 8 +++++++- hw/virtio/virtio-pci.c | 16 ++++++++++------ 4 files changed, 24 insertions(+), 10 deletions(-) diff --git a/docs/system/sriov.rst b/docs/system/sriov.rst index a851a66a4b8b..d12178f3c319 100644 --- a/docs/system/sriov.rst +++ b/docs/system/sriov.rst @@ -28,7 +28,8 @@ virtio-net-pci functions to a bus. Below is a command line example: The VFs specify the paired PF with ``sriov-pf`` property. The PF must be added after all VFs. It is the user's responsibility to ensure that VFs have function numbers larger than one of the PF, and that the function numbers -have a consistent stride. +have a consistent stride. Both the PF and VFs are ARI-capable so you can have +255 VFs at maximum. You may also need to perform additional steps to activate the SR-IOV feature on your guest. For Linux, refer to [1]_. diff --git a/include/hw/pci/pcie_sriov.h b/include/hw/pci/pcie_sriov.h index f75b8f22ee92..aeaa38cf3456 100644 --- a/include/hw/pci/pcie_sriov.h +++ b/include/hw/pci/pcie_sriov.h @@ -43,12 +43,15 @@ void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, /** * pcie_sriov_pf_init_from_user_created_vfs() - Initialize PF with user-created - * VFs. + * VFs, adding ARI to PF * @dev: A PCIe device being realized. * @offset: The offset of the SR-IOV capability. * @errp: pointer to Error*, to store an error if it happens. * - * Return: The size of added capability. 0 if the user did not create VFs. + * Initializes a PF with user-created VFs, adding the ARI extended capability to + * the PF. The VFs should call pcie_ari_init() to form an ARI device. + * + * Return: The size of added capabilities. 0 if the user did not create VFs. * -1 if failed. */ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 08f707e847fd..3ad18744f4a8 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -245,6 +245,7 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, PCIDevice **vfs; BusState *bus = qdev_get_parent_bus(DEVICE(dev)); uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t size = PCI_EXT_CAP_SRIOV_SIZEOF; uint16_t vf_dev_id; uint16_t vf_offset; uint16_t vf_stride; @@ -311,6 +312,11 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, return -1; } + if (!pcie_find_capability(dev, PCI_EXT_CAP_ID_ARI)) { + pcie_ari_init(dev, offset + size); + size += PCI_ARI_SIZEOF; + } + for (i = 0; i < pf->len; i++) { vfs[i]->exp.sriov_vf.pf = dev; vfs[i]->exp.sriov_vf.vf_number = i; @@ -331,7 +337,7 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, } } - return PCI_EXT_CAP_SRIOV_SIZEOF; + return size; } bool pcie_sriov_register_device(PCIDevice *dev, Error **errp) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 5ea8e7f6d0a8..e3ac5543195a 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -2111,12 +2111,16 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) PCI_BASE_ADDRESS_SPACE_IO, &proxy->bar); } - res = pcie_sriov_pf_init_from_user_created_vfs(&proxy->pci_dev, - proxy->last_pcie_cap_offset, - errp); - if (res > 0) { - proxy->last_pcie_cap_offset += res; - virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + if (pci_is_vf(&proxy->pci_dev)) { + pcie_ari_init(&proxy->pci_dev, proxy->last_pcie_cap_offset); + proxy->last_pcie_cap_offset += PCI_ARI_SIZEOF; + } else { + res = pcie_sriov_pf_init_from_user_created_vfs( + &proxy->pci_dev, proxy->last_pcie_cap_offset, errp); + if (res > 0) { + proxy->last_pcie_cap_offset += res; + virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + } } }