From patchwork Fri Aug 2 06:49:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A02F0C52D71 for ; Fri, 2 Aug 2024 06:52:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7P-0000gw-Hd; Fri, 02 Aug 2024 02:50:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7O-0000cT-DU for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:26 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7N-0000ux-0X for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:26 -0400 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1fd9e6189d5so63245215ad.3 for ; Thu, 01 Aug 2024 23:50:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581424; x=1723186224; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=p/xbyP1Am7mZb37ldbOaJNJuDRPmUAjsJzjuyi2y2fE=; b=FrwFPgl66Z6pS1y5+q3LNUYtWsGaoUYHAktUklxR0edxdcpV4b09AYsBnvqbUFSwFc apC/pkEjfsPZhabYm8ao+AEfpImUqgB79dOM7fpRbA3xtiijsAuTIm0Hth2uSHOO2ath ZP2i3D/Mc8XSvSXoTneDP55fmnZGplm+iMTT9OL+z2HvpUtdkm2hS8kXFmWbhN8OYkkz GU10jNWeN5CMEZiMbc3pdnUxeVGL5Nmg/kRKfe16P6hIco67uPG7e4ds9nYowyhhkmEq P1RTo7/yLxA+KmANkrLs3c+HqhnoKsMznzPVNfQxlUDwuWqbEn5+hSIxyTWe19T0cm3A iE3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581424; x=1723186224; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p/xbyP1Am7mZb37ldbOaJNJuDRPmUAjsJzjuyi2y2fE=; b=l9rH1z8pGgxLDBoUu4DJshYU+BJwB6W0Uu81KBamPsxg0ByAUpM7zu7pJ2O0+wwGMR 3Gwe5kp32kscdZfhOSU+F0/5GBeyiWMKu/XFUVH0iFDKvVf5ndOn3rjJrAMdJuImC9iE DHO4bUA+YF+cwI0h852rsHIcQmyi8jKMnxHcq9oUMkVDMgbpEdTvVlF9kWBFM0H2pMPq PJI9IinTNvW5oHdU5PtVazsu2QWTI67FxdNjgHZdGOsJP5qXjxKTZ6AJdL5rqwMQAAkt CF3QQDHOYk490+ckU5/KI2bsKXpVA8gPmxSxpbTDmKitBVdCbyvYnziVlvhvCjd57z2K kyWA== X-Gm-Message-State: AOJu0Yzl7djOFfv3FptDDAnfP6+JaJX2aTwTo159QKfgYBFTwW+ifRXB 8JrzO0KPHwZ24rfGtQ/wpEuOig5yENZ0pdAtFpqxtzs/VekH4S2yEVV3zhxGJw8= X-Google-Smtp-Source: AGHT+IF4K/OW5vEQiPX+KgI0NvYOBhps/u8u1mGGK7gKTtlJPt1SaQCaqpv/gG9LvXmg6hnZ+opmDQ== X-Received: by 2002:a17:902:ecd2:b0:1fd:64d2:62da with SMTP id d9443c01a7336-1ff572c4e13mr37040385ad.29.1722581423720; Thu, 01 Aug 2024 23:50:23 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-1ff58f29f96sm9849195ad.1.2024.08.01.23.50.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:23 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:45 +0900 Subject: [PATCH for-9.2 v6 1/9] hw/pci: Do not add ROM BAR for SR-IOV VF MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-1-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::635; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x635.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A SR-IOV VF cannot have a ROM BAR. Co-developed-by: Yui Washizu Signed-off-by: Akihiko Odaki --- hw/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index d2eaf0c51dde..60b1747d60e6 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2359,6 +2359,14 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, return; } + if (pci_is_vf(pdev)) { + if (pdev->rom_bar > 0) { + error_setg(errp, "ROM BAR cannot be enabled for SR-IOV VF"); + } + + return; + } + if (load_file || pdev->romsize == UINT32_MAX) { path = qemu_find_file(QEMU_FILE_TYPE_BIOS, pdev->romfile); if (path == NULL) { From patchwork Fri Aug 2 06:49:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93B47C52D70 for ; Fri, 2 Aug 2024 06:51:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7V-000151-4V; Fri, 02 Aug 2024 02:50:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7T-0000wt-30 for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:31 -0400 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7R-0000va-KG for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:30 -0400 Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-7b80afeb099so112230a12.2 for ; Thu, 01 Aug 2024 23:50:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581428; x=1723186228; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=jdptavMuPUq6FleoJPG4kUJogZ48lw2HQ6y5CWLpBPo=; b=KMw/ovx6msiTuF9Jfs6pZ34wt4s3Uc7b1k6ZCtHpiIOtTZ2OBW0z/P86MmLZGLjEWr r/X6F6/qMR89vaPfIRT15Q6t4Q8EzDwcGGHXunLq3JrC7Vtod3zxQxXZpEqB9ojD2e4i B+9JeMpLWtmlzfsl0hfN/gsh0uwVcJmlrSdaJV9xG6VmCaQYN2c8NPrmQrfsrE4Zg+IT XWr4P6p2+5e/AaUfTNLxvsutI9baN2BLMJrdLTDpj+pEAB2lM71Qu5HNFrKD4rnkHVjA DzscAWwAT+GSniqHM9VDvCo83KfImKypLE9JgEd5wQr3f0iuWEHeKYucFp+a87SnDpr0 Rolw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581428; x=1723186228; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jdptavMuPUq6FleoJPG4kUJogZ48lw2HQ6y5CWLpBPo=; b=NxBhMUfUlbWVmIbfyVKbQEgNS1f3lX9nq81U2csjnM6qInPKf5Xw9YhrTQiqKvXmfK nDOC/8kzKR8qAfDlw171OwE7JO1MiH1gPxTMBM7NZw119ZFMGoaHSj9W8t6jXfV17Ro4 8d0Lo8IC42908DED3EkU3+Oo7ObKYD47Zj3/NLT/S8SLvDM5XmNNRd7Cx1oXkNAZj1R9 h1rRfKjzLyXCxvL3wAtpRl9kSi9nIta5GVMfJhj9nws5ScYo4AafN9C4zzseYw7BywPj 8ZnO8FMdq/34BhcCEifLBRMvo3PRVi9k97SO9shCMrroHFjZ9IQqYxx8LUp/lQYunbpM Yf2A== X-Gm-Message-State: AOJu0Yz6KsCU9S5iflktzXr7i+rbUVx4NOy3KSMXEtQffZv0WPR28fCS bKAMhpOSIE30Zdok8/8xvYmp1irHm5BGJHcIftPSF8bMYSac6WCIJnbj10l8M4A= X-Google-Smtp-Source: AGHT+IHwTazeFrpe4MmMRbjQs8vkZe2vjW3S59VADjiRKuC2w5u1vhuDS4fWJ01+pLWG+m61RkX90Q== X-Received: by 2002:a05:6a20:158a:b0:1c4:8da5:21a4 with SMTP id adf61e73a8af0-1c6995edd71mr3661359637.41.1722581428416; Thu, 01 Aug 2024 23:50:28 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-1ff59294e92sm9705605ad.240.2024.08.01.23.50.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:28 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:46 +0900 Subject: [PATCH for-9.2 v6 2/9] hw/pci: Fix SR-IOV VF number calculation MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-2-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::536; envelope-from=akihiko.odaki@daynix.com; helo=mail-pg1-x536.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org pci_config_get_bar_addr() had a division by vf_stride. vf_stride needs to be non-zero when there are multiple VFs, but the specification does not prohibit to make it zero when there is only one VF. Do not perform the division for the first VF to avoid division by zero. Signed-off-by: Akihiko Odaki --- hw/pci/pci.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 60b1747d60e6..0956fe5eb444 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -1437,7 +1437,11 @@ static pcibus_t pci_config_get_bar_addr(PCIDevice *d, int reg, pci_get_word(pf->config + sriov_cap + PCI_SRIOV_VF_OFFSET); uint16_t vf_stride = pci_get_word(pf->config + sriov_cap + PCI_SRIOV_VF_STRIDE); - uint32_t vf_num = (d->devfn - (pf->devfn + vf_offset)) / vf_stride; + uint32_t vf_num = d->devfn - (pf->devfn + vf_offset); + + if (vf_num) { + vf_num /= vf_stride; + } if (type & PCI_BASE_ADDRESS_MEM_TYPE_64) { new_addr = pci_get_quad(pf->config + bar); From patchwork Fri Aug 2 06:49:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C91CFC52D70 for ; Fri, 2 Aug 2024 06:51:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7a-0001Q3-7l; Fri, 02 Aug 2024 02:50:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7Y-0001Ha-5J for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:36 -0400 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7W-0000wb-Lf for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:35 -0400 Received: by mail-pg1-x535.google.com with SMTP id 41be03b00d2f7-6e7e23b42c3so4998127a12.1 for ; Thu, 01 Aug 2024 23:50:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581433; x=1723186233; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ZZGRu8v6oxuF32W/xrA/69BjX55/YFUQg7PXZsmlE7o=; b=bwQhuOwtdn8cs7TWkvhG7XUHPvhXg5Fop+3Imn0OTePQBH8Vn74GjoIC8HgSCN9Ee1 oWMb+GjiWmUKqhwxwVbIfbbIx08l4jqA46em7scmE/7o6ELYVYASuTRCPaOIj5wgD8UZ qs7Y3sZq2in73kjTuJV5EzGcAW4SNobmjhewfq+VhzBQqDqXiq3DwBzZyU9ufO14CXwb z/TdVFTQdo3wIrAtP+L6RSucybC4pnJ+mM6W4DCpSpBF72HcbrWFJ0CGq9LuypIA4yvo +ppOzFjv7UALEedynQ9eecLun3M6CE7r6KYCyif18JjkADlq/EID1WS+QyacK8fRDPHw ejaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581433; x=1723186233; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZZGRu8v6oxuF32W/xrA/69BjX55/YFUQg7PXZsmlE7o=; b=PWxSls3jOms5GhqIHONgGQNtaR3m+oYza4jT5q4SLAAf2Nd7h0HiBmCi1s6x7VOPkB duh/tCEuBC46VjRo0KFRHjzwotEVswzOpqnEyM9sfmbvoL1s7kaMqk5DmlK+tcdA1a57 olW9ILj/4mo7mH1MoShhKYahGuDiXG4TqQYcJOMpY3s5va1SbKhs+deefJjV5dnHR9vC dyA+6FIblA4mQMAtrU0+1ca24zGi7pLSEonXpFQ1imcktfSgPvItIN+2T7vouF/a3/Ic bvRnrd+Dlnul+A0u8eZWzJ1citBkDycYifWN6MfCsa0Z0YobFvZb7mM6yqurlNL02BZf CuNg== X-Gm-Message-State: AOJu0YySFbSQ9PMwkCTGlyiTO4J7MX8rqTPLNnvt0rK+dTWNSaFcYrA3 ZBpSrSwKGyfY7Vvsm82eHukETOIoprD81jPSqcX+KejVrVR5CJMEp5BhQKVmTZU= X-Google-Smtp-Source: AGHT+IGwVSyhB8pGmmvcsq34mHa6UqBFBHAfw36HYcu5TpzyDqYgSyynLNp5G3/5WfJuDonlIjlRUQ== X-Received: by 2002:a05:6a21:b85:b0:1c4:a30a:c819 with SMTP id adf61e73a8af0-1c699559e30mr2937915637.21.1722581433243; Thu, 01 Aug 2024 23:50:33 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-1ff58f5a473sm9934605ad.106.2024.08.01.23.50.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:32 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:47 +0900 Subject: [PATCH for-9.2 v6 3/9] pcie_sriov: Ensure PF and VF are mutually exclusive MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-3-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::535; envelope-from=akihiko.odaki@daynix.com; helo=mail-pg1-x535.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A device cannot be a SR-IOV PF and a VF at the same time. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 56523ab4e833..ec8fc0757b92 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -42,6 +42,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (pci_is_vf(dev)) { + error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); + return false; + } + if (total_vfs) { uint16_t ari_cap = pcie_find_capability(dev, PCI_EXT_CAP_ID_ARI); uint16_t first_vf_devfn = dev->devfn + vf_offset; From patchwork Fri Aug 2 06:49:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6380EC52D71 for ; Fri, 2 Aug 2024 06:51:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7d-0001fn-RA; Fri, 02 Aug 2024 02:50:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7c-0001Zi-JJ for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:40 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7b-0000x1-5M for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:40 -0400 Received: by mail-pg1-x52a.google.com with SMTP id 41be03b00d2f7-7a2123e9ad5so1988952a12.1 for ; Thu, 01 Aug 2024 23:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581438; x=1723186238; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=eIrCsb1ntTQZ4u2QiCIc/D2bj4QGskaeiB3kw7Zorrg=; b=wrIJkb77fV18QDf5DtFAoE8uznHpY3dOyWm12Zkx2c455DTeSzvj9f+A1eEIyDQCzC gsmSC5DAznkvbIP1GaQh2s5/UHkCW0GTVv7DxvJHKBnNuhns0AEjiCkVBjowL/fvv2Jf C0wqwBgk45YARvEe6pijBx1RTwI0vs9LkRrx2eOzTTWOjpb+8oFQdUZL/dYXVqny1N9W m0qivEpKS7HWAjKccQNHVs5vAYsaZjX4LLwlXYLR2t7VJYy5h9/KY3WmMQpDHD+rrdjE TlVM4Xw95+qLti5yqEcim2+eTc9M997Y079BEQVAWGijSbUXVgT6Mp6PGoTfbCAu01Dk hwYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581438; x=1723186238; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eIrCsb1ntTQZ4u2QiCIc/D2bj4QGskaeiB3kw7Zorrg=; b=UNdEKUE9Q22ZBnZzF7Aq+rAz5dVeYooCNvSxyqXMvobImoskm+Sx+KGoj/Heq82YBL LCBGyTexztylhc7YubFDetSJ25KprqKVqtmpbe8XmWhEINiv9IXXHGyEgAIigflaPneT Kq7VadFDlsAYu73LAc7L+LPnCGkNIm6lUlhiqrkhNy3SBLaeIyyZlgU3vT4GVypF/rXA tiLpkgr6XskMAX9bKNgm1dkAvmAHcy5hE9vmVvygzG5B/m5YjMNNFf0EyDIZFTq9uTnn OvBXlTZG51unG9p1CHjadT8XFWqNk5wkfQuGR+SWwA97kHUEBEzgfg8IqL0mkZ007nkP MAHw== X-Gm-Message-State: AOJu0Yw1SJFQ1p0mL0T1V1mvkxyF20ZgfawSYCJOPlpTczrmUGEeS9Ci MhmwdKtPtuifmDEUoJfxxeRz21VIP17bbo5EgKS+USGDrATPJ4Et6/Cpr+JzcPA= X-Google-Smtp-Source: AGHT+IEft9ZxxwTiurDIjItlQ1rzd+ViC1S7qUUBS0dXQB1N27yx6aJgikarHd3qTbPwcMKLTK6/mw== X-Received: by 2002:a05:6a21:32a4:b0:1c0:f09c:1b98 with SMTP id adf61e73a8af0-1c69434da17mr6864668637.16.1722581437947; Thu, 01 Aug 2024 23:50:37 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-7b830f296d4sm99471a12.68.2024.08.01.23.50.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:37 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:48 +0900 Subject: [PATCH for-9.2 v6 4/9] pcie_sriov: Check PCI Express for SR-IOV PF MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-4-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::52a; envelope-from=akihiko.odaki@daynix.com; helo=mail-pg1-x52a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org SR-IOV requires PCI Express. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index ec8fc0757b92..3af0cc7d560a 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -42,6 +42,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV PF"); + return false; + } + if (pci_is_vf(dev)) { error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); return false; From patchwork Fri Aug 2 06:49:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8EB5FC3DA4A for ; Fri, 2 Aug 2024 06:51:32 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7l-0002HG-4R; Fri, 02 Aug 2024 02:50:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7j-00024H-55 for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:47 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7g-0000xy-4x for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:46 -0400 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2cf78366187so5230825a91.3 for ; Thu, 01 Aug 2024 23:50:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581443; x=1723186243; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=SdNXSqDwQlmYBRuHK4p4XhRInWoKK2Iyoi2jVfBMA9Q=; b=FPJ6rH6nuIEV5nhphEsLvbWI900CSBMI9+vEOAIItJ5dXJBLtfqjjRdjCfR/cf2ZWb SHl2peRUpZtttwKVp2RjtYR0Ipgoae1U04Fmq6NDP3n2V/xCBit83qJkee+nOG96gLk9 8bsKjHueMs7YHgB8iz9qWCCErKH5XOnHwrSk9ajV+azqK2U9cN88VaWxkWjW5t5orXUG QIuh4oYAIQYYBitZMUolgG7q5yJ/RUSjTTsWl4dI97TrbWnfUnPsC8OUlA0uESfdpPJp rm8pNH6DN33y1o58ussRqmd9j4RiGLCMoR8nC94Y8vXmH9S2nwnNdWbQ+TEWqtqmtB5R NA5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581443; x=1723186243; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SdNXSqDwQlmYBRuHK4p4XhRInWoKK2Iyoi2jVfBMA9Q=; b=R5KjPHuC7NTeg9jNRBd0L3UEZJouIMVfbD47J8VcRnsWhCJv42qvDZwzJIow1MD/xc BU7CX16xZ/dAvLtvxmkCjGP47h5bMxbZxjEqCz4qxr8UZJ4THucDlcTLLzgPiYwBYXak mzKJWogcA5ds8BckoIDM0wlD8x2Ti/KZexgbxV6ZHQHau4NX47qvc25LeFNe63WT0nIN Ot1r98KGiIXKo4VHVcPJWxm2wnJVdrjIBQP3tTyMlhf1OiiYdTfPNd3rwrZSw+52LwQt rFGt0OwuMoirz8oqtaevILqu/h3F5eoP2JWCW+WWPNhoYDAw5m2pOEhSwDkKJtJipVtt 4w0A== X-Gm-Message-State: AOJu0Yz6+nCE6rkmEZfJn069DHTDRQAimijWqzIIJIVNwmJeT7VJe4xc 5QLP+MdGSq0ut+6PpP42TnjWNxAhw6eSWI0d6QxrcK8XJpayI1vA/NfP3jBXWT4= X-Google-Smtp-Source: AGHT+IEre6OOJmmay6OHUGWj7ZQ1MKGHDqu3okImyaT7UuwgizXNTV/Q+mQ7kTy2qKmp1TkL6Fl7sg== X-Received: by 2002:a17:902:da82:b0:1fd:73e6:83d1 with SMTP id d9443c01a7336-1ff57274ab9mr42443255ad.21.1722581442706; Thu, 01 Aug 2024 23:50:42 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-1ff58f5c5f6sm9855435ad.111.2024.08.01.23.50.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:42 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:49 +0900 Subject: [PATCH for-9.2 v6 5/9] pcie_sriov: Allow user to create SR-IOV device MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-5-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::1029; envelope-from=akihiko.odaki@daynix.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A user can create a SR-IOV device by specifying the PF with the sriov-pf property of the VFs. The VFs must be added before the PF. A user-creatable VF must have PCIDeviceClass::sriov_vf_user_creatable set. Such a VF cannot refer to the PF because it is created before the PF. A PF that user-creatable VFs can be attached calls pcie_sriov_pf_init_from_user_created_vfs() during realization and pcie_sriov_pf_exit() when exiting. Signed-off-by: Akihiko Odaki --- include/hw/pci/pci_device.h | 6 +- include/hw/pci/pcie_sriov.h | 18 +++ hw/pci/pci.c | 62 ++++++---- hw/pci/pcie_sriov.c | 288 +++++++++++++++++++++++++++++++++++--------- 4 files changed, 291 insertions(+), 83 deletions(-) diff --git a/include/hw/pci/pci_device.h b/include/hw/pci/pci_device.h index 8fa845beee5e..1d31099dd4dc 100644 --- a/include/hw/pci/pci_device.h +++ b/include/hw/pci/pci_device.h @@ -38,6 +38,8 @@ struct PCIDeviceClass { uint16_t subsystem_id; /* only for header type = 0 */ const char *romfile; /* rom bar */ + + bool sriov_vf_user_creatable; }; enum PCIReqIDType { @@ -167,6 +169,8 @@ struct PCIDevice { /* ID of standby device in net_failover pair */ char *failover_pair_id; uint32_t acpi_index; + + char *sriov_pf; }; static inline int pci_intx(PCIDevice *pci_dev) @@ -199,7 +203,7 @@ static inline int pci_is_express_downstream_port(const PCIDevice *d) static inline int pci_is_vf(const PCIDevice *d) { - return d->exp.sriov_vf.pf != NULL; + return d->sriov_pf || d->exp.sriov_vf.pf != NULL; } static inline uint32_t pci_config_size(const PCIDevice *d) diff --git a/include/hw/pci/pcie_sriov.h b/include/hw/pci/pcie_sriov.h index c5d2d318d330..f75b8f22ee92 100644 --- a/include/hw/pci/pcie_sriov.h +++ b/include/hw/pci/pcie_sriov.h @@ -18,6 +18,7 @@ typedef struct PCIESriovPF { uint8_t vf_bar_type[PCI_NUM_REGIONS]; /* Store type for each VF bar */ PCIDevice **vf; /* Pointer to an array of num_vfs VF devices */ + bool vf_user_created; /* If VFs are created by user */ } PCIESriovPF; typedef struct PCIESriovVF { @@ -40,6 +41,23 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory); +/** + * pcie_sriov_pf_init_from_user_created_vfs() - Initialize PF with user-created + * VFs. + * @dev: A PCIe device being realized. + * @offset: The offset of the SR-IOV capability. + * @errp: pointer to Error*, to store an error if it happens. + * + * Return: The size of added capability. 0 if the user did not create VFs. + * -1 if failed. + */ +int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, + uint16_t offset, + Error **errp); + +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp); +void pcie_sriov_unregister_device(PCIDevice *dev); + /* * Default (minimal) page size support values * as required by the SR/IOV standard: diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 0956fe5eb444..e693f5b1e044 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -85,6 +85,7 @@ static Property pci_props[] = { QEMU_PCIE_ERR_UNC_MASK_BITNR, true), DEFINE_PROP_BIT("x-pcie-ari-nextfn-1", PCIDevice, cap_present, QEMU_PCIE_ARI_NEXTFN_1_BITNR, false), + DEFINE_PROP_STRING("sriov-pf", PCIDevice, sriov_pf), DEFINE_PROP_END_OF_LIST() }; @@ -959,13 +960,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) dev->config[PCI_HEADER_TYPE] |= PCI_HEADER_TYPE_MULTI_FUNCTION; } - /* - * With SR/IOV and ARI, a device at function 0 need not be a multifunction - * device, as it may just be a VF that ended up with function 0 in - * the legacy PCI interpretation. Avoid failing in such cases: - */ - if (pci_is_vf(dev) && - dev->exp.sriov_vf.pf->cap_present & QEMU_PCI_CAP_MULTIFUNCTION) { + /* SR/IOV is not handled here. */ + if (pci_is_vf(dev)) { return; } @@ -998,7 +994,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) } /* function 0 indicates single function, so function > 0 must be NULL */ for (func = 1; func < PCI_FUNC_MAX; ++func) { - if (bus->devices[PCI_DEVFN(slot, func)]) { + PCIDevice *device = bus->devices[PCI_DEVFN(slot, func)]; + if (device && !pci_is_vf(device)) { error_setg(errp, "PCI: %x.0 indicates single function, " "but %x.%x is already populated.", slot, slot, func); @@ -1283,6 +1280,7 @@ static void pci_qdev_unrealize(DeviceState *dev) pci_unregister_io_regions(pci_dev); pci_del_option_rom(pci_dev); + pcie_sriov_unregister_device(pci_dev); if (pc->exit) { pc->exit(pci_dev); @@ -1314,7 +1312,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, pcibus_t size = memory_region_size(memory); uint8_t hdr_type; - assert(!pci_is_vf(pci_dev)); /* VFs must use pcie_sriov_vf_register_bar */ assert(region_num >= 0); assert(region_num < PCI_NUM_REGIONS); assert(is_power_of_2(size)); @@ -1325,7 +1322,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, assert(hdr_type != PCI_HEADER_TYPE_BRIDGE || region_num < 2); r = &pci_dev->io_regions[region_num]; - r->addr = PCI_BAR_UNMAPPED; r->size = size; r->type = type; r->memory = memory; @@ -1333,22 +1329,35 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, ? pci_get_bus(pci_dev)->address_space_io : pci_get_bus(pci_dev)->address_space_mem; - wmask = ~(size - 1); - if (region_num == PCI_ROM_SLOT) { - /* ROM enable bit is writable */ - wmask |= PCI_ROM_ADDRESS_ENABLE; - } - - addr = pci_bar(pci_dev, region_num); - pci_set_long(pci_dev->config + addr, type); + if (pci_is_vf(pci_dev)) { + PCIDevice *pf = pci_dev->exp.sriov_vf.pf; + assert(!pf || type == pf->exp.sriov_pf.vf_bar_type[region_num]); - if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && - r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { - pci_set_quad(pci_dev->wmask + addr, wmask); - pci_set_quad(pci_dev->cmask + addr, ~0ULL); + r->addr = pci_bar_address(pci_dev, region_num, r->type, r->size); + if (r->addr != PCI_BAR_UNMAPPED) { + memory_region_add_subregion_overlap(r->address_space, + r->addr, r->memory, 1); + } } else { - pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); - pci_set_long(pci_dev->cmask + addr, 0xffffffff); + r->addr = PCI_BAR_UNMAPPED; + + wmask = ~(size - 1); + if (region_num == PCI_ROM_SLOT) { + /* ROM enable bit is writable */ + wmask |= PCI_ROM_ADDRESS_ENABLE; + } + + addr = pci_bar(pci_dev, region_num); + pci_set_long(pci_dev->config + addr, type); + + if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && + r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { + pci_set_quad(pci_dev->wmask + addr, wmask); + pci_set_quad(pci_dev->cmask + addr, ~0ULL); + } else { + pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); + pci_set_long(pci_dev->cmask + addr, 0xffffffff); + } } } @@ -2109,6 +2118,11 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp) } } + if (!pcie_sriov_register_device(pci_dev, errp)) { + pci_qdev_unrealize(DEVICE(pci_dev)); + return; + } + /* * A PCIe Downstream Port that do not have ARI Forwarding enabled must * associate only Device 0 with the device attached to the bus diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 3af0cc7d560a..0c875e61fe96 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -20,6 +20,8 @@ #include "qapi/error.h" #include "trace.h" +static GHashTable *pfs; + static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) { for (uint16_t i = 0; i < total_vfs; i++) { @@ -31,14 +33,49 @@ static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) dev->exp.sriov_pf.vf = NULL; } -bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, - const char *vfname, uint16_t vf_dev_id, - uint16_t init_vfs, uint16_t total_vfs, - uint16_t vf_offset, uint16_t vf_stride, - Error **errp) +static void clear_ctrl_vfe(PCIDevice *dev) +{ + uint8_t *ctrl = dev->config + dev->exp.sriov_cap + PCI_SRIOV_CTRL; + pci_set_word(ctrl, pci_get_word(ctrl) & ~PCI_SRIOV_CTRL_VFE); +} + +static void register_vfs(PCIDevice *dev) +{ + uint16_t num_vfs; + uint16_t i; + uint16_t sriov_cap = dev->exp.sriov_cap; + + assert(sriov_cap > 0); + num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); + if (num_vfs > pci_get_word(dev->config + sriov_cap + PCI_SRIOV_TOTAL_VF)) { + clear_ctrl_vfe(dev); + return; + } + + trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn), num_vfs); + for (i = 0; i < num_vfs; i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + } +} + +static void unregister_vfs(PCIDevice *dev) +{ + uint16_t i; + uint8_t *cfg = dev->config + dev->exp.sriov_cap; + + trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn)); + for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + } +} + +static bool pcie_sriov_pf_init_common(PCIDevice *dev, uint16_t offset, + uint16_t vf_dev_id, uint16_t init_vfs, + uint16_t total_vfs, uint16_t vf_offset, + uint16_t vf_stride, Error **errp) { - BusState *bus = qdev_get_parent_bus(&dev->qdev); - int32_t devfn = dev->devfn + vf_offset; uint8_t *cfg = dev->config + offset; uint8_t *wmask; @@ -100,6 +137,28 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, qdev_prop_set_bit(&dev->qdev, "multifunction", true); + return true; +} + +bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, + const char *vfname, uint16_t vf_dev_id, + uint16_t init_vfs, uint16_t total_vfs, + uint16_t vf_offset, uint16_t vf_stride, + Error **errp) +{ + BusState *bus = qdev_get_parent_bus(&dev->qdev); + int32_t devfn = dev->devfn + vf_offset; + + if (pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, init_vfs, + total_vfs, vf_offset, vf_stride, errp)) { + return false; + } + dev->exp.sriov_pf.vf = g_new(PCIDevice *, total_vfs); for (uint16_t i = 0; i < total_vfs; i++) { @@ -129,7 +188,22 @@ void pcie_sriov_pf_exit(PCIDevice *dev) { uint8_t *cfg = dev->config + dev->exp.sriov_cap; - unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + if (dev->exp.sriov_pf.vf_user_created) { + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t total_vfs = pci_get_word(dev->config + PCI_SRIOV_TOTAL_VF); + uint16_t vf_dev_id = pci_get_word(dev->config + PCI_SRIOV_VF_DID); + + unregister_vfs(dev); + + for (uint16_t i = 0; i < total_vfs; i++) { + dev->exp.sriov_pf.vf[i]->exp.sriov_vf.pf = NULL; + + pci_config_set_vendor_id(dev->exp.sriov_pf.vf[i]->config, ven_id); + pci_config_set_device_id(dev->exp.sriov_pf.vf[i]->config, vf_dev_id); + } + } else { + unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + } } void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, @@ -162,74 +236,172 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory) { - PCIIORegion *r; - PCIBus *bus = pci_get_bus(dev); uint8_t type; - pcibus_t size = memory_region_size(memory); - assert(pci_is_vf(dev)); /* PFs must use pci_register_bar */ - assert(region_num >= 0); - assert(region_num < PCI_NUM_REGIONS); + assert(dev->exp.sriov_vf.pf); type = dev->exp.sriov_vf.pf->exp.sriov_pf.vf_bar_type[region_num]; - if (!is_power_of_2(size)) { - error_report("%s: PCI region size must be a power" - " of two - type=0x%x, size=0x%"FMT_PCIBUS, - __func__, type, size); - exit(1); - } - - r = &dev->io_regions[region_num]; - r->memory = memory; - r->address_space = - type & PCI_BASE_ADDRESS_SPACE_IO - ? bus->address_space_io - : bus->address_space_mem; - r->size = size; - r->type = type; - - r->addr = pci_bar_address(dev, region_num, r->type, r->size); - if (r->addr != PCI_BAR_UNMAPPED) { - memory_region_add_subregion_overlap(r->address_space, - r->addr, r->memory, 1); - } + return pci_register_bar(dev, region_num, type, memory); } -static void clear_ctrl_vfe(PCIDevice *dev) +static gint compare_vf_devfns(gconstpointer a, gconstpointer b) { - uint8_t *ctrl = dev->config + dev->exp.sriov_cap + PCI_SRIOV_CTRL; - pci_set_word(ctrl, pci_get_word(ctrl) & ~PCI_SRIOV_CTRL_VFE); + return (*(PCIDevice **)a)->devfn - (*(PCIDevice **)b)->devfn; } -static void register_vfs(PCIDevice *dev) +int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, + uint16_t offset, + Error **errp) { - uint16_t num_vfs; + GPtrArray *pf; + PCIDevice **vfs; + BusState *bus = qdev_get_parent_bus(DEVICE(dev)); + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t vf_dev_id; + uint16_t vf_offset; + uint16_t vf_stride; uint16_t i; - uint16_t sriov_cap = dev->exp.sriov_cap; - assert(sriov_cap > 0); - num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); - if (num_vfs > pci_get_word(dev->config + sriov_cap + PCI_SRIOV_TOTAL_VF)) { - clear_ctrl_vfe(dev); - return; + if (!pfs || !dev->qdev.id) { + return 0; } - trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn), num_vfs); - for (i = 0; i < num_vfs; i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + pf = g_hash_table_lookup(pfs, dev->qdev.id); + if (!pf) { + return 0; + } + + if (pf->len > UINT16_MAX) { + error_setg(errp, "too many VFs"); + return -1; + } + + g_ptr_array_sort(pf, compare_vf_devfns); + vfs = (void *)pf->pdata; + + if (vfs[0]->devfn <= dev->devfn) { + error_setg(errp, "a VF function number is less than the PF function number"); + return -1; + } + + vf_dev_id = pci_get_word(vfs[0]->config + PCI_DEVICE_ID); + vf_offset = vfs[0]->devfn - dev->devfn; + vf_stride = pf->len < 2 ? 0 : vfs[1]->devfn - vfs[0]->devfn; + + for (i = 0; i < pf->len; i++) { + if (bus != qdev_get_parent_bus(&vfs[i]->qdev)) { + error_setg(errp, "SR-IOV VF parent bus mismatches with PF"); + return -1; + } + + if (ven_id != pci_get_word(vfs[i]->config + PCI_VENDOR_ID)) { + error_setg(errp, "SR-IOV VF vendor ID mismatches with PF"); + return -1; + } + + if (vf_dev_id != pci_get_word(vfs[i]->config + PCI_DEVICE_ID)) { + error_setg(errp, "inconsistent SR-IOV VF device IDs"); + return -1; + } + + for (size_t j = 0; j < PCI_NUM_REGIONS; j++) { + if (vfs[i]->io_regions[j].size != vfs[0]->io_regions[j].size || + vfs[i]->io_regions[j].type != vfs[0]->io_regions[j].type) { + error_setg(errp, "inconsistent SR-IOV BARs"); + return -1; + } + } + + if (vfs[i]->devfn - vfs[0]->devfn != vf_stride * i) { + error_setg(errp, "inconsistent SR-IOV stride"); + return -1; + } + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, pf->len, + pf->len, vf_offset, vf_stride, errp)) { + return -1; + } + + for (i = 0; i < pf->len; i++) { + vfs[i]->exp.sriov_vf.pf = dev; + vfs[i]->exp.sriov_vf.vf_number = i; + + /* set vid/did according to sr/iov spec - they are not used */ + pci_config_set_vendor_id(vfs[i]->config, 0xffff); + pci_config_set_device_id(vfs[i]->config, 0xffff); } + + dev->exp.sriov_pf.vf = vfs; + dev->exp.sriov_pf.vf_user_created = true; + + for (i = 0; i < PCI_NUM_REGIONS; i++) { + PCIIORegion *region = &vfs[0]->io_regions[i]; + + if (region->size) { + pcie_sriov_pf_init_vf_bar(dev, i, region->type, region->size); + } + } + + return PCI_EXT_CAP_SRIOV_SIZEOF; } -static void unregister_vfs(PCIDevice *dev) +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp) { - uint16_t i; - uint8_t *cfg = dev->config + dev->exp.sriov_cap; + if (!dev->exp.sriov_pf.vf && dev->qdev.id && + pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } - trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn)); - for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + if (dev->sriov_pf) { + PCIDevice *pci_pf; + GPtrArray *pf; + + if (!PCI_DEVICE_GET_CLASS(dev)->sriov_vf_user_creatable) { + error_setg(errp, "user cannot create SR-IOV VF with this device type"); + return false; + } + + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV VF"); + return false; + } + + if (!pci_qdev_find_device(dev->sriov_pf, &pci_pf)) { + error_setg(errp, "PCI device specified as SR-IOV PF already exists"); + return false; + } + + if (!pfs) { + pfs = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, NULL); + } + + pf = g_hash_table_lookup(pfs, dev->sriov_pf); + if (!pf) { + pf = g_ptr_array_new(); + g_hash_table_insert(pfs, g_strdup(dev->sriov_pf), pf); + } + + g_ptr_array_add(pf, dev); + } + + return true; +} + +void pcie_sriov_unregister_device(PCIDevice *dev) +{ + if (dev->sriov_pf && pfs) { + GPtrArray *pf = g_hash_table_lookup(pfs, dev->sriov_pf); + + if (pf) { + g_ptr_array_remove_fast(pf, dev); + + if (!pf->len) { + g_hash_table_remove(pfs, dev->sriov_pf); + g_ptr_array_free(pf, FALSE); + } + } } } @@ -316,7 +488,7 @@ void pcie_sriov_pf_add_sup_pgsize(PCIDevice *dev, uint16_t opt_sup_pgsize) uint16_t pcie_sriov_vf_number(PCIDevice *dev) { - assert(pci_is_vf(dev)); + assert(dev->exp.sriov_vf.pf); return dev->exp.sriov_vf.vf_number; } From patchwork Fri Aug 2 06:49:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B8F0C3DA4A for ; Fri, 2 Aug 2024 06:51:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7m-0002XJ-Qs; Fri, 02 Aug 2024 02:50:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7m-0002SE-5k for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:50 -0400 Received: from mail-ot1-x32e.google.com ([2607:f8b0:4864:20::32e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7k-0000yZ-Jh for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:49 -0400 Received: by mail-ot1-x32e.google.com with SMTP id 46e09a7af769-7093f3a1af9so4710254a34.1 for ; Thu, 01 Aug 2024 23:50:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581447; x=1723186247; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=vdR/tAAVHxSpgFFmSFp64O9usJu3eSZIZ8gMew9CW0c=; b=0h7dFTTNis2O+lfqEepJpFIsFwEwsTz+tF4Ix2sjJ8Cg5xQFHW2gUZrEDFdbcBmCnE BvSwEyuDrgCUHqiOC0AkmLRTQkHNJVxgenE3kgQVUCjABxjzgMTH4QtrvCM25yJCzcjT tl8ftvm0+H1/7H/G3uId6TJbVunih9HJR+CmFRSpuwyvH75qXLeM1PmBWeBu4V3rQV4a 2yxmLFk7HffkOEGFaCpupE2yXpYfCj9oB0f+Y5vr8/7DvoWl0QFnXTvWMQhhr175OkVx 4j7ujR187Epb9KrZupNAQPhORXP5vGNd2nVRLpFvkPfpP8HrbYxegHrckd0v50Wd+MTH HNnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581447; x=1723186247; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vdR/tAAVHxSpgFFmSFp64O9usJu3eSZIZ8gMew9CW0c=; b=FbKmuD3ru2EBlgyzr28oHp5tX+z4IPukMwM0j+3VuOfn+LWCEnejJ+tYu+rSt429M6 YNtiTeesGhFouEIX+hpMrcWsqAyRPl8Cb9gGlj9ogzgufWRht7Z5nxO1Xt8Ts+Awqyf0 yxrA16920PlUY6FQtvVhOknFH9qhO4v47qaH989r3JBrlQYG+XcD7bY7OzyGNANQaUZG C7gSn0By1xvPEoU9lZA5+1nZvpntoVMVdtUCLCXAIiR4Y9qX0wXpj+snPBd5visu+3wE q3m9F22m23T+yYgwi+Z0ESyI0kxmpMBzuLOeeCO7tynPqDZeKWapzNbcvx0/Y77gTHpb VVzA== X-Gm-Message-State: AOJu0YwnirE5WekMW75VU+7XyLYrWsLvquXUWyHWuTTmjtz5YW3skE08 2+VtDqBaBJ2mtKnySMgcRRi1dEsll7iYqo9T594DbDe9eg8agI0rKl8fUygYonno1XAWR7pmy+u dRQ8= X-Google-Smtp-Source: AGHT+IHWJRIR2S0OGEJAuKHTV4V4yE/ZHczC8RdJi2EhDyDONld+xoKUfr/+8PJgd340dhIa0xKjHA== X-Received: by 2002:a05:6830:6987:b0:709:3ba1:fc88 with SMTP id 46e09a7af769-709b9981fd2mr3083580a34.33.1722581447352; Thu, 01 Aug 2024 23:50:47 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7106ed14f38sm775291b3a.184.2024.08.01.23.50.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:47 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:50 +0900 Subject: [PATCH for-9.2 v6 6/9] virtio-pci: Implement SR-IOV PF MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-6-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::32e; envelope-from=akihiko.odaki@daynix.com; helo=mail-ot1-x32e.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allow user to attach SR-IOV VF to a virtio-pci PF. Signed-off-by: Akihiko Odaki --- include/hw/virtio/virtio-pci.h | 1 + hw/virtio/virtio-pci.c | 20 +++++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/include/hw/virtio/virtio-pci.h b/include/hw/virtio/virtio-pci.h index 9e67ba38c748..34539f2f6722 100644 --- a/include/hw/virtio/virtio-pci.h +++ b/include/hw/virtio/virtio-pci.h @@ -152,6 +152,7 @@ struct VirtIOPCIProxy { uint32_t modern_io_bar_idx; uint32_t modern_mem_bar_idx; int config_cap; + uint16_t last_pcie_cap_offset; uint32_t flags; bool disable_modern; bool ignore_backend_features; diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 9534730bba19..0c8fcc5627d5 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1955,6 +1955,7 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) uint8_t *config; uint32_t size; VirtIODevice *vdev = virtio_bus_get_device(bus); + int16_t res; /* * Virtio capabilities present without @@ -2100,6 +2101,14 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) pci_register_bar(&proxy->pci_dev, proxy->legacy_io_bar_idx, PCI_BASE_ADDRESS_SPACE_IO, &proxy->bar); } + + res = pcie_sriov_pf_init_from_user_created_vfs(&proxy->pci_dev, + proxy->last_pcie_cap_offset, + errp); + if (res > 0) { + proxy->last_pcie_cap_offset += res; + virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + } } static void virtio_pci_device_unplugged(DeviceState *d) @@ -2187,7 +2196,7 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) if (pcie_port && pci_is_express(pci_dev)) { int pos; - uint16_t last_pcie_cap_offset = PCI_CONFIG_SPACE_SIZE; + proxy->last_pcie_cap_offset = PCI_CONFIG_SPACE_SIZE; pos = pcie_endpoint_cap_init(pci_dev, 0); assert(pos > 0); @@ -2207,9 +2216,9 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) pci_set_word(pci_dev->config + pos + PCI_PM_PMC, 0x3); if (proxy->flags & VIRTIO_PCI_FLAG_AER) { - pcie_aer_init(pci_dev, PCI_ERR_VER, last_pcie_cap_offset, + pcie_aer_init(pci_dev, PCI_ERR_VER, proxy->last_pcie_cap_offset, PCI_ERR_SIZEOF, NULL); - last_pcie_cap_offset += PCI_ERR_SIZEOF; + proxy->last_pcie_cap_offset += PCI_ERR_SIZEOF; } if (proxy->flags & VIRTIO_PCI_FLAG_INIT_DEVERR) { @@ -2234,9 +2243,9 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp) } if (proxy->flags & VIRTIO_PCI_FLAG_ATS) { - pcie_ats_init(pci_dev, last_pcie_cap_offset, + pcie_ats_init(pci_dev, proxy->last_pcie_cap_offset, proxy->flags & VIRTIO_PCI_FLAG_ATS_PAGE_ALIGNED); - last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF; + proxy->last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF; } if (proxy->flags & VIRTIO_PCI_FLAG_INIT_FLR) { @@ -2263,6 +2272,7 @@ static void virtio_pci_exit(PCIDevice *pci_dev) bool pcie_port = pci_bus_is_express(pci_get_bus(pci_dev)) && !pci_bus_is_root(pci_get_bus(pci_dev)); + pcie_sriov_pf_exit(&proxy->pci_dev); msix_uninit_exclusive_bar(pci_dev); if (proxy->flags & VIRTIO_PCI_FLAG_AER && pcie_port && pci_is_express(pci_dev)) { From patchwork Fri Aug 2 06:49:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 639D0C52D6D for ; Fri, 2 Aug 2024 06:51:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7s-00039m-Kc; Fri, 02 Aug 2024 02:50:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7q-0002zk-WD for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:55 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7p-0000zH-EX for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:54 -0400 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-70d1d6369acso2176447b3a.0 for ; Thu, 01 Aug 2024 23:50:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581452; x=1723186252; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=NlrY553tddcDoCWOe56+tRoCgkm3F8OvMmopjLbpLLI=; b=gjPPdih9PgHz+NkrQv4qScLsGHlPh+rPp4A33K9N9FaM8ziq3TZ5alQlNhnlzlIg/1 60LiJ0PVLk3DEd5JXkJ/JuJOCJseTGV4aCFiqCG5QdRsTE79xLs/wIrSMpVLjuEjLUU6 hzdgqYg6CejlLSQMv3w4hYWtFGWztYpaG63IeiYqwsg1eshFYKM/o3Wqr08UQSXl8sxp vmwEmv080hKWHg+AvXvqfTXP0008Kjw1DxaHWe6qARqnFAd6XxlC9bumyn7BknEX2rjQ JnwOU00gaQe4v0jvve3Pv49Oxl+5/A92ojBbf76VNfnr7vDLvEHuC2TkPSr0wB73ErgF JXSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581452; x=1723186252; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NlrY553tddcDoCWOe56+tRoCgkm3F8OvMmopjLbpLLI=; b=gpfhJfaR34Dt+Z1vBODfmKYTFiUzHPj0x/USKwyzH1UR/2rkIJGojz78cwt8AeImgw pH4fi49c6Yo7z+gmb9KIkYt1bELLYZRrebPsqrpcTqmOThApFiHa23fhOvKELl1HjioN i4qXPGfNxQoLsO2qIqG7mNNTBSxYR7ryBDMelBc+wtPpMHh4Z5gHAk9X0vsMTMP3w2rB ewr6lnvEtOzVU3fjEF27GC886VgdW027mM+dtXRuPARK8JtH6t3r78KQulviVibPkekV 4pW2yCCVPi2Dmc3UKway0gvva9lishVtWxMeXPqJWhdFw0tKXG3E4gD0I/vYuUHxaN+Q NySw== X-Gm-Message-State: AOJu0Yxg7lB1wQA5UJfQJwHcuQlYJat1fx7zajdKAlYeUph3tdaux8oS 6dEo7vvwq7ihluobwgQD7p2R9uzQa5s9QVZGmx7CY5kUMJBe74U68oLS3gnWAl+Vhqum+t/oKPx 6SBA= X-Google-Smtp-Source: AGHT+IHdgQU+Sh3GVRQA4Zt3b8Wt5BKHZHuODJcqt8s1p8QsHOAbIo0fV8g98rfCMhWdTl6hKB49Uw== X-Received: by 2002:a05:6a21:9983:b0:1c4:6ac5:be17 with SMTP id adf61e73a8af0-1c6943c8608mr6097700637.21.1722581452014; Thu, 01 Aug 2024 23:50:52 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id 41be03b00d2f7-7b763945c05sm648875a12.49.2024.08.01.23.50.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:51 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:51 +0900 Subject: [PATCH for-9.2 v6 7/9] virtio-net: Implement SR-IOV VF MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-7-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::436; envelope-from=akihiko.odaki@daynix.com; helo=mail-pf1-x436.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A virtio-net device can be added as a SR-IOV VF to another virtio-pci device that will be the PF. Signed-off-by: Akihiko Odaki --- hw/virtio/virtio-net-pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/virtio/virtio-net-pci.c b/hw/virtio/virtio-net-pci.c index e03543a70a75..dba4987d6e04 100644 --- a/hw/virtio/virtio-net-pci.c +++ b/hw/virtio/virtio-net-pci.c @@ -75,6 +75,7 @@ static void virtio_net_pci_class_init(ObjectClass *klass, void *data) k->device_id = PCI_DEVICE_ID_VIRTIO_NET; k->revision = VIRTIO_PCI_ABI_VERSION; k->class_id = PCI_CLASS_NETWORK_ETHERNET; + k->sriov_vf_user_creatable = true; set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); device_class_set_props(dc, virtio_net_properties); vpciklass->realize = virtio_net_pci_realize; From patchwork Fri Aug 2 06:49:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18609C52D71 for ; Fri, 2 Aug 2024 06:51:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm7w-0003SX-Rg; Fri, 02 Aug 2024 02:51:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm7v-0003My-JS for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:59 -0400 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7t-0000zv-St for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:50:59 -0400 Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-2cb4c4de4cbso5285055a91.1 for ; Thu, 01 Aug 2024 23:50:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581456; x=1723186256; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=C5Qkc2xHJDIrJ32DpraIz8uFeHLkPAtAdY7nT6vwWJE=; b=AYUeW5JQttS1UCC4MrTrlWb78TURWhHrD1cN291+9hrfSZQgldxT4UL+xq/s0GibiD 7sLh6a445InCci+9NmUFcvjel2Z47JmIYcFnxM4Vb74xBWcvqCa8nOumE4UaTkXFY1tv DNrTVMPEVbxT32Usmlvc/X3lHcKvKAH1Wf8j4uAlw+y00wh0nfgbmNpNyWRTmXMTGgxn VdZI4k/obwWY3sltQ5K07BWcJhsKa81lfkX7OAnOOVeGnd//D+zHbdqN7kX+UtZLKcmv sUfFvMzWAl77l3qZ5snbjYLtu3LMoLAeX8KTnEG6SQRm1qj7ZTPKTZUdZ57qj0a0eFNR BYIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581456; x=1723186256; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C5Qkc2xHJDIrJ32DpraIz8uFeHLkPAtAdY7nT6vwWJE=; b=EATFdwlWHBGs/+L44NriEeOiok5o32t4CJFzh82cNOt3vwzfu/hhjD77xtYSUd0rN6 gq4FDET3VCGJaifHN4MYbwGz3b869EP5mmgzsXAibAvdr6RBD8HIq+VLkQ70+lraSATp jpHRWElo7kh/42hckoh17ZEkE3a8HV4Rb30GhXWkvCou1W+/+MrAQCLeeIdrQq3VSO9O acpe+LLQXyUXjn4Z+Jog4Zn+2GI7f9rw6OT2z4OW37yrDK9kwDspMTsQvHKaFdV2EoYI l/MfPXwqDGajj7ufwS/E05+MC9/RJFELiSFzeRK+VvGxI3TAesVFnp76ZdIdqqF4/S3b MYHw== X-Gm-Message-State: AOJu0Yy6SG2mdULsfrRs2Y3cyrhL5tR/7clkpxCdfs2fn9whoj4d4Gs4 t+Ig85qGS5KFZusWPJ6l5TR0ZIwm65x7pPaETMVkbkeHPbpKtqiIoC+4tXdvm4g= X-Google-Smtp-Source: AGHT+IHbP57HzdQgdOoAT09Ovx6+WonHPvuUhuDJfxkQknuIax702CuZfvqS0a4NczNyMxxYXhJA2Q== X-Received: by 2002:a17:90a:e006:b0:2cb:6419:5b53 with SMTP id 98e67ed59e1d1-2cff9475f78mr3375451a91.24.1722581456480; Thu, 01 Aug 2024 23:50:56 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2cffb38d3fcsm1018670a91.50.2024.08.01.23.50.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:50:56 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:52 +0900 Subject: [PATCH for-9.2 v6 8/9] docs: Document composable SR-IOV device MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-8-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::102c; envelope-from=akihiko.odaki@daynix.com; helo=mail-pj1-x102c.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Akihiko Odaki --- MAINTAINERS | 1 + docs/system/index.rst | 1 + docs/system/sriov.rst | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index e34c2bd4cda2..72b3c6736088 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2011,6 +2011,7 @@ F: hw/pci-bridge/* F: qapi/pci.json F: docs/pci* F: docs/specs/*pci* +F: docs/system/sriov.rst PCIE DOE M: Huai-Cheng Kuo diff --git a/docs/system/index.rst b/docs/system/index.rst index c21065e51932..718e9d3c56bb 100644 --- a/docs/system/index.rst +++ b/docs/system/index.rst @@ -39,3 +39,4 @@ or Hypervisor.Framework. multi-process confidential-guest-support vm-templating + sriov diff --git a/docs/system/sriov.rst b/docs/system/sriov.rst new file mode 100644 index 000000000000..a851a66a4b8b --- /dev/null +++ b/docs/system/sriov.rst @@ -0,0 +1,36 @@ +.. SPDX-License-Identifier: GPL-2.0-or-later + +Compsable SR-IOV device +======================= + +SR-IOV (Single Root I/O Virtualization) is an optional extended capability of a +PCI Express device. It allows a single physical function (PF) to appear as +multiple virtual functions (VFs) for the main purpose of eliminating software +overhead in I/O from virtual machines. + +There are devices with predefined SR-IOV configurations, but it is also possible +to compose an SR-IOV device yourself. Composing an SR-IOV device is currently +only supported by virtio-net-pci. + +Users can configure an SR-IOV-capable virtio-net device by adding +virtio-net-pci functions to a bus. Below is a command line example: + +.. code-block:: shell + + -netdev user,id=n -netdev user,id=o + -netdev user,id=p -netdev user,id=q + -device pcie-root-port,id=b + -device virtio-net-pci,bus=b,addr=0x0.0x3,netdev=q,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x2,netdev=p,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x1,netdev=o,sriov-pf=f + -device virtio-net-pci,bus=b,addr=0x0.0x0,netdev=n,id=f + +The VFs specify the paired PF with ``sriov-pf`` property. The PF must be +added after all VFs. It is the user's responsibility to ensure that VFs have +function numbers larger than one of the PF, and that the function numbers +have a consistent stride. + +You may also need to perform additional steps to activate the SR-IOV feature on +your guest. For Linux, refer to [1]_. + +.. [1] https://docs.kernel.org/PCI/pci-iov-howto.html From patchwork Fri Aug 2 06:49:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13751157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6585BC3DA4A for ; Fri, 2 Aug 2024 06:52:06 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sZm83-00041s-Hz; Fri, 02 Aug 2024 02:51:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sZm81-0003o3-GM for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:51:05 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sZm7y-00010d-M5 for qemu-devel@nongnu.org; Fri, 02 Aug 2024 02:51:05 -0400 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1fc6ee64512so60482365ad.0 for ; Thu, 01 Aug 2024 23:51:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1722581461; x=1723186261; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=04oTSlM6/qEgQv6cXgurvQT4fTrVXkJlhpZElpuiYcs=; b=JmMbPOYoSKfoX3boinkABZmvXaxue1Xz/eR48iPkx9BOn4gyqENvM4dZjLlfupchSi EhxU/Prmn4cJIl3aQnQv/0g1LKryJSz/TB2qnwkECMcGPpHpF3NXSKN8wKEOduCnjYVZ 1rjJTG99Q0DY1aor2wPxci65Io4tjRZXoF/ax0kxLqsBt/osqq8+i0SfX4Hdb800wHqa yWchjXe2axFHKEFfOPPM9fKSqtg63SGZT2TCcePMh5doWtL1s+iGeqjb12ja1HUdescc GoA0/2l3IytfUcAhwtMfgA/mFwJWjPFxTtxf4NvybMw6eYZ9aYXsK2b90QGf3Tw2G+Wu B+uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722581461; x=1723186261; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=04oTSlM6/qEgQv6cXgurvQT4fTrVXkJlhpZElpuiYcs=; b=MA5MZ1PCWY5SW5Lriz5IMFckqWJeHR47j59GpZiEzruX03VHgDg6YzHIKconNKXqLc 83m/cZYZgwfXuDNpt9+F/GR6ymV17BZ1ZXyozU+8Cz30Yr6Lj0WvUsUdlz/9Kdk1UQmi GVB2YfgYFzOrXBxlM1yczZj44YNzLGAnX2GKVR2s09qo8kGpgshDQvv82vyWtA4VCsQL sfkxzqfJHTCyTpyOtjPbL6e0dC73WJ/gbKLk068FiUG7VlHaSULB849wa6Ca5FFdrzwY AlV/pFqR8jUtlrLaJpAnZAkOYk702qgZtzUGUQ+j3bp3+R+qTTMS2FcNmMcuRWA4+avo tAsA== X-Gm-Message-State: AOJu0YxDG7iVIwJgRKN9w2NE5mdzM2wZ8+tK12ZllHtNIdJO6Vtf0KcZ UiAtzxh3HGAy46ZY/fR4ewcLcTaf452wrAOE/+I/0ejEMFzOr0zykgCQoQ2TScw= X-Google-Smtp-Source: AGHT+IExEk7nXbgT7/gjlTCKUgUTTzyPjiURhgMzHPaOUZaB8djl4uttlE++4fo+dgJNilA2hAoEUw== X-Received: by 2002:a17:902:e892:b0:1fc:327a:1f48 with SMTP id d9443c01a7336-1ff57258160mr28754585ad.12.1722581461245; Thu, 01 Aug 2024 23:51:01 -0700 (PDT) Received: from localhost ([157.82.201.15]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-1ff592b5434sm9682025ad.298.2024.08.01.23.50.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 01 Aug 2024 23:51:00 -0700 (PDT) From: Akihiko Odaki Date: Fri, 02 Aug 2024 15:49:53 +0900 Subject: [PATCH for-9.2 v6 9/9] pcie_sriov: Make a PCI device with user-created VF ARI-capable MIME-Version: 1.0 Message-Id: <20240802-sriov-v6-9-0c8ff49c4276@daynix.com> References: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> In-Reply-To: <20240802-sriov-v6-0-0c8ff49c4276@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.14-dev-fd6e3 Received-SPF: none client-ip=2607:f8b0:4864:20::631; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x631.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Akihiko Odaki --- docs/system/sriov.rst | 3 ++- include/hw/pci/pcie_sriov.h | 7 +++++-- hw/pci/pcie_sriov.c | 12 ++++++++---- hw/virtio/virtio-pci.c | 16 ++++++++++------ 4 files changed, 25 insertions(+), 13 deletions(-) diff --git a/docs/system/sriov.rst b/docs/system/sriov.rst index a851a66a4b8b..d12178f3c319 100644 --- a/docs/system/sriov.rst +++ b/docs/system/sriov.rst @@ -28,7 +28,8 @@ virtio-net-pci functions to a bus. Below is a command line example: The VFs specify the paired PF with ``sriov-pf`` property. The PF must be added after all VFs. It is the user's responsibility to ensure that VFs have function numbers larger than one of the PF, and that the function numbers -have a consistent stride. +have a consistent stride. Both the PF and VFs are ARI-capable so you can have +255 VFs at maximum. You may also need to perform additional steps to activate the SR-IOV feature on your guest. For Linux, refer to [1]_. diff --git a/include/hw/pci/pcie_sriov.h b/include/hw/pci/pcie_sriov.h index f75b8f22ee92..aeaa38cf3456 100644 --- a/include/hw/pci/pcie_sriov.h +++ b/include/hw/pci/pcie_sriov.h @@ -43,12 +43,15 @@ void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, /** * pcie_sriov_pf_init_from_user_created_vfs() - Initialize PF with user-created - * VFs. + * VFs, adding ARI to PF * @dev: A PCIe device being realized. * @offset: The offset of the SR-IOV capability. * @errp: pointer to Error*, to store an error if it happens. * - * Return: The size of added capability. 0 if the user did not create VFs. + * Initializes a PF with user-created VFs, adding the ARI extended capability to + * the PF. The VFs should call pcie_ari_init() to form an ARI device. + * + * Return: The size of added capabilities. 0 if the user did not create VFs. * -1 if failed. */ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 0c875e61fe96..64c655db29c4 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -90,12 +90,10 @@ static bool pcie_sriov_pf_init_common(PCIDevice *dev, uint16_t offset, } if (total_vfs) { - uint16_t ari_cap = pcie_find_capability(dev, PCI_EXT_CAP_ID_ARI); uint16_t first_vf_devfn = dev->devfn + vf_offset; uint16_t last_vf_devfn = first_vf_devfn + vf_stride * (total_vfs - 1); - if ((!ari_cap && PCI_SLOT(dev->devfn) != PCI_SLOT(last_vf_devfn)) || - last_vf_devfn >= PCI_DEVFN_MAX) { + if (last_vf_devfn >= PCI_DEVFN_MAX) { error_setg(errp, "VF function number overflows"); return false; } @@ -257,6 +255,7 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, PCIDevice **vfs; BusState *bus = qdev_get_parent_bus(DEVICE(dev)); uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t size = PCI_EXT_CAP_SRIOV_SIZEOF; uint16_t vf_dev_id; uint16_t vf_offset; uint16_t vf_stride; @@ -323,6 +322,11 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, return -1; } + if (!pcie_find_capability(dev, PCI_EXT_CAP_ID_ARI)) { + pcie_ari_init(dev, offset + size); + size += PCI_ARI_SIZEOF; + } + for (i = 0; i < pf->len; i++) { vfs[i]->exp.sriov_vf.pf = dev; vfs[i]->exp.sriov_vf.vf_number = i; @@ -343,7 +347,7 @@ int16_t pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, } } - return PCI_EXT_CAP_SRIOV_SIZEOF; + return size; } bool pcie_sriov_register_device(PCIDevice *dev, Error **errp) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 0c8fcc5627d5..b19e2983ee22 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -2102,12 +2102,16 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) PCI_BASE_ADDRESS_SPACE_IO, &proxy->bar); } - res = pcie_sriov_pf_init_from_user_created_vfs(&proxy->pci_dev, - proxy->last_pcie_cap_offset, - errp); - if (res > 0) { - proxy->last_pcie_cap_offset += res; - virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + if (pci_is_vf(&proxy->pci_dev)) { + pcie_ari_init(&proxy->pci_dev, proxy->last_pcie_cap_offset); + proxy->last_pcie_cap_offset += PCI_ARI_SIZEOF; + } else { + res = pcie_sriov_pf_init_from_user_created_vfs( + &proxy->pci_dev, proxy->last_pcie_cap_offset, errp); + if (res > 0) { + proxy->last_pcie_cap_offset += res; + virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + } } }