From patchwork Tue Mar 5 08:58:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13581868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 685CBC54E41 for ; Tue, 5 Mar 2024 08:59:34 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhQdg-0003tJ-CY; Tue, 05 Mar 2024 03:59:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhQdd-0003rv-TH for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:05 -0500 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhQda-0003eX-Th for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:05 -0500 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1dbd32cff0bso45842135ad.0 for ; Tue, 05 Mar 2024 00:59:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1709629141; x=1710233941; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=41HOhmfFzvzHUWDzT1tLGC+0kqqS1QTmSsIo/CJC9aw=; b=gTtgVVLc2iwUZWKKFnuz0bydQOTsdACWA69S2teEQ5tVNyqGR0rBK0c1cXRb5goASK 0nikpL8K7RZQF8ikX5Bthkq2aItDA6ZRecS0SehUycJXDc0E+fBbLCqnLjE/qGuJvaNz ZZDEOIzPHHyDuuyQOxMX7Nlcj9rI0EUNVQwBV9BSqnK2yMzLKv/8277VRGGPCUIivLG6 HAoqBmEQBoZHNGn4Y6HNweaHnBLQ04x6afAMtBTNU9/0dAMSppWhGHjJwT4wFf7na35Z ieNtW2JnprStcfPHEBebSSSRJEhlkz8mj7wIo2urVKwqBCZqkK1hAruHk1nVfKUkJ5Sf UK0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709629141; x=1710233941; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=41HOhmfFzvzHUWDzT1tLGC+0kqqS1QTmSsIo/CJC9aw=; b=MUnvKIkrFtw8qTl7MxddLoI96LY9bbnZlv2XktVJLZuPWKRp6BglJ1/vN6z9Qv2Y4D TwH5RP7F0QfjxJ43GX+Hz9MfZVzsFV33BcVC52lnrVClHFHbyVcqpd7NKXx+dY8qVB8p S4WPr2cDb2sFUGogrAtwE7fhVGm0B6DL7Mzn3jmsVO8+zgPMSgvj57xlQScjORtn8/lE yVI4z67pqFpIsUvGA4kh3qk3FXY4ZgYflI6zsuITkss5kGyrXiYrlrnTb/h5ry5M6VCr 2fd+I+YetirNbgnICyv8mcqp7+sk4HkYXSEtE9B6CUWBa64sP9xlxq7j9oyHsr3OL8Ib bsSw== X-Gm-Message-State: AOJu0Yy0OxWD+7mq9+c/O0sB5i6pDrbByeWzXgRUGnUylx8ZfAAsPNVt nVQBWr9IdP5vVJRJvlY7rsx+/fIvycNpXz3MG6yVGq5hGbzWJrJ0oZAZAYoZwdk= X-Google-Smtp-Source: AGHT+IGR2YzRjLxs4vKHJWgiJis2qudqTHhJ8QiUBQ3YaGqJTP++Zq1W3IpdBMbJ8yHv3NEAwjgnhA== X-Received: by 2002:a17:902:eb8f:b0:1db:ab9c:d33d with SMTP id q15-20020a170902eb8f00b001dbab9cd33dmr1561733plg.9.1709629141222; Tue, 05 Mar 2024 00:59:01 -0800 (PST) Received: from localhost ([157.82.203.206]) by smtp.gmail.com with UTF8SMTPSA id u11-20020a170903124b00b001dc01efaec2sm9959404plh.168.2024.03.05.00.58.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Mar 2024 00:59:00 -0800 (PST) From: Akihiko Odaki Date: Tue, 05 Mar 2024 17:58:48 +0900 Subject: [PATCH RFC v3 1/6] hw/pci: Do not add ROM BAR for SR-IOV VF MIME-Version: 1.0 Message-Id: <20240305-sriov-v3-1-abdb75770372@daynix.com> References: <20240305-sriov-v3-0-abdb75770372@daynix.com> In-Reply-To: <20240305-sriov-v3-0-abdb75770372@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, 20240228-reuse-v8-0-282660281e60@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.12.3 Received-SPF: none client-ip=2607:f8b0:4864:20::631; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x631.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A SR-IOV VF cannot have a ROM BAR. Co-developed-by: Yui Washizu Signed-off-by: Akihiko Odaki --- hw/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index cb5ac46e9f27..201ff64e11cc 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2359,6 +2359,14 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, return; } + if (pci_is_vf(pdev)) { + if (pdev->rom_bar != UINT32_MAX) { + error_setg(errp, "ROM BAR cannot be enabled for SR-IOV VF"); + } + + return; + } + if (load_file || pdev->romsize == UINT32_MAX) { path = qemu_find_file(QEMU_FILE_TYPE_BIOS, pdev->romfile); if (path == NULL) { From patchwork Tue Mar 5 08:58:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13581871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1A7DC54E41 for ; Tue, 5 Mar 2024 09:01:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhQdj-0003vB-4v; Tue, 05 Mar 2024 03:59:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhQdg-0003tr-WA for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:09 -0500 Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhQdf-0003gm-Gb for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:08 -0500 Received: by mail-oi1-x229.google.com with SMTP id 5614622812f47-3c1ea5e54d3so1208279b6e.0 for ; Tue, 05 Mar 2024 00:59:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1709629146; x=1710233946; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=cIQOQVYqY0DEobqSlWtvEDM6sg4WeMXNv2XNh/AB3Us=; b=rlc+2hnse5zsfJQ69VSNIyp8t0NgMiBUK/0gVH0JmlmlvTgHTkNHwwV1obuwbvlHmW O29QVl0FQWcQBbRhGSFhRBaB1VPRfY+4ZGzBuNUkEp8OZJ8um4cJpxVpESVopOeEkgMV cgFATlkie/eK3W52uQp1Fc4c88s5HYYwA/aJa21RBVH49XtW7J8nwKAAfKwVdeHLHTkB 65wxjBXfUaZU1sHHV72P84pKVWr+d3463mMrLVwEEazwRkGlcsZoV3HM0BaDMmXOcgUg +B/Qxm7jTczta7bJZbB9thuv+5t9tTT2Vj+jVciZ77+LospJjXeJ7XMtot9ND0IKYxqO s2kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709629146; x=1710233946; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cIQOQVYqY0DEobqSlWtvEDM6sg4WeMXNv2XNh/AB3Us=; b=EsC6yiowJFjlN1Oa6LNxy3qrS/G+WLi2VIeZ9I3sntBCDygobN+mPfBKD9C3E86xnK 5wcvpsz4UKn4z6AIfUafrX3c5DufwClOWaPk1/AJHPaWmPDkRvn6qr/+7CrlS7u25/1p P8cldVAs+MxSU56UZZtMxBGSXksO24G5+qLSyGvy2Kf+sn8dFg3Whj0zvFRSW2X7VjHb yOCbrO73Mg0FkifCxctbroJbPJYKm8nnf4wHjQq6Kon558ZAth+IVctF3kdIp/mXFM1R in5Cbq4aJUQPlq2OKlUdg33TUwUk1BlWzgpMfZYUtmnVK4EYp81Pd84l9q50LkliNyWN tCZA== X-Gm-Message-State: AOJu0YxtKcywdaf0JAeCAOaXE3tuLjub3HhA82slKc1VX4i57xV0cQsG 5KyLIvckuLLpAbAwmMU9q/51T5DYpEfbx4pRxOFuaCbsBcaxKUZTItD4r2KcLe4= X-Google-Smtp-Source: AGHT+IHaEIXxVTn9V25tFDYy8qk2ZpPmEsmn/nkR3cjZFqkRYTgQbxZsAqb9CliQHK1PYJDM9gRTwQ== X-Received: by 2002:a05:6808:2186:b0:3c1:e699:4cc0 with SMTP id be6-20020a056808218600b003c1e6994cc0mr1595758oib.7.1709629146527; Tue, 05 Mar 2024 00:59:06 -0800 (PST) Received: from localhost ([157.82.203.206]) by smtp.gmail.com with UTF8SMTPSA id q12-20020a63504c000000b005df58c83e89sm8716599pgl.84.2024.03.05.00.59.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Mar 2024 00:59:06 -0800 (PST) From: Akihiko Odaki Date: Tue, 05 Mar 2024 17:58:49 +0900 Subject: [PATCH RFC v3 2/6] pcie_sriov: Ensure PF and VF are mutually exclusive MIME-Version: 1.0 Message-Id: <20240305-sriov-v3-2-abdb75770372@daynix.com> References: <20240305-sriov-v3-0-abdb75770372@daynix.com> In-Reply-To: <20240305-sriov-v3-0-abdb75770372@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, 20240228-reuse-v8-0-282660281e60@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.12.3 Received-SPF: none client-ip=2607:f8b0:4864:20::229; envelope-from=akihiko.odaki@daynix.com; helo=mail-oi1-x229.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A device cannot be a SR-IOV PF and a VF at the same time. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index 09a53ed30027..aac12e70f122 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -41,6 +41,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (pci_is_vf(dev)) { + error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); + return false; + } + pcie_add_capability(dev, PCI_EXT_CAP_ID_SRIOV, 1, offset, PCI_EXT_CAP_SRIOV_SIZEOF); dev->exp.sriov_cap = offset; From patchwork Tue Mar 5 08:58:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13581872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CC682C54798 for ; Tue, 5 Mar 2024 09:01:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhQdn-0003wH-P8; Tue, 05 Mar 2024 03:59:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhQdm-0003vg-N8 for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:14 -0500 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhQdl-0003hJ-8b for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:14 -0500 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1dcd0431f00so33538405ad.3 for ; Tue, 05 Mar 2024 00:59:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1709629152; x=1710233952; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=uGqy4+tpzaC83CNC1aw1LjIzpM4137yW+W5QGCQBPRk=; b=j127QpBNBCgfYeC3nsMHZWRkVw4DptAbdkJGov/QLP0YlLEewzqCmwlwcssrPb4TK+ /x6lF9eb63xd8YPUzUACLs30p4i3NvqLpAV7KXAc205FcVeumpKcOyD7f7mtYmqImbwW kFLocOvPRa1Pc6ioStBzYk2Hj41mJgEHGdaS6jSixEYNFYg50IF/EzlMcvw+qm3Q8lf+ ya2xboT7oLApmdZqAAc7AfpRzGjvN0xdHMkNt6lqYHRlyIhQAxpgcO7pErzkDbjyEX5u 8ftzX6AvWN+eFzUytcKxlrHhXNbdRkmo8SQRBx+tR2y7k8aWgz++jsY81CcaITxKiI7S LGFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709629152; x=1710233952; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uGqy4+tpzaC83CNC1aw1LjIzpM4137yW+W5QGCQBPRk=; b=tkMsJh91CBlPPQbvlpS+ifR6EFW8zmhg+XfoUAXD0lyn44iQLdRG9FUSXpIbqzq8Ln P0uSf9OeteY/HBmfl+UT+DIYrdt7YkitjIp/lG9jgE4AKEjiqGczafap7BV0cKXXLLFf cKlBnV56Vw3XIAsPDAoJsirGDRessPwQJQFPK/Xt+bzSjF19wOVnAsO9pnfvrWgfb0uk gz5pPuw5aJle+zO5KFM1r1diXufxJ7c7C7zGDEFHADfajVCovvR08nLBzAm50u/PO40v +1K8RJApubKhfV4U6yOzlusu3pOodCgwN9jCjtd4jq4IssZBXif5VTFmh4lZboBq4ZnU BHtw== X-Gm-Message-State: AOJu0Yzg915dVZ+BJ6u2iiPjWwf0FAuELX1FXagGlkFYMTDOrYpZNzn5 NYUZMJEqp0UJFiLMXaO1aJp+A8Aopr/YijDBeDE6zRUu30RKofjHhP+l0/luRjA= X-Google-Smtp-Source: AGHT+IHYu3yhFha2jQeP0Ol9JLt8XPNY/+sA3Utq973I+kYxbtDCsEVPsBK+sIycXUYU3g9KV1KM/A== X-Received: by 2002:a17:902:cf0c:b0:1dc:cba3:f909 with SMTP id i12-20020a170902cf0c00b001dccba3f909mr1327907plg.54.1709629151901; Tue, 05 Mar 2024 00:59:11 -0800 (PST) Received: from localhost ([157.82.203.206]) by smtp.gmail.com with UTF8SMTPSA id q3-20020a170902a3c300b001dbcb39dd7dsm10091529plb.125.2024.03.05.00.59.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Mar 2024 00:59:11 -0800 (PST) From: Akihiko Odaki Date: Tue, 05 Mar 2024 17:58:50 +0900 Subject: [PATCH RFC v3 3/6] pcie_sriov: Check PCI Express for SR-IOV PF MIME-Version: 1.0 Message-Id: <20240305-sriov-v3-3-abdb75770372@daynix.com> References: <20240305-sriov-v3-0-abdb75770372@daynix.com> In-Reply-To: <20240305-sriov-v3-0-abdb75770372@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, 20240228-reuse-v8-0-282660281e60@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.12.3 Received-SPF: none client-ip=2607:f8b0:4864:20::62e; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org SR-IOV requires PCI Express. Signed-off-by: Akihiko Odaki --- hw/pci/pcie_sriov.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index aac12e70f122..c449ddd0ac39 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -41,6 +41,11 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, uint8_t *cfg = dev->config + offset; uint8_t *wmask; + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV PF"); + return false; + } + if (pci_is_vf(dev)) { error_setg(errp, "a device cannot be a SR-IOV PF and a VF at the same time"); return false; From patchwork Tue Mar 5 08:58:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13581874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79E96C54E41 for ; Tue, 5 Mar 2024 09:01:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhQdy-0003xm-Vs; Tue, 05 Mar 2024 03:59:27 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhQdu-0003xH-1u for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:22 -0500 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhQdr-0003he-Ez for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:21 -0500 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-5e152c757a5so3796589a12.2 for ; Tue, 05 Mar 2024 00:59:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1709629157; x=1710233957; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=j1eQPmD/cclnNs/ltzs+A6VvqcjsXDrfmpDDpfoLHOg=; b=xq+ObCV4bfzWPAhHbGo3Gjo08A7Iqa2AV5Uyg+VLcWFWSYU+TQCKrn4lflXiQ1HOe0 VwpSeFHI/hN3vTRfxuHzThwagnexG3rOHUIW2pRZt0FJbMXzdHlxGs6Osg/cuQ1oQ4dA ufFzP2T1tW8OWUfznF3eTQ6hiILgty0rKfX0tKWObCuECMbivAFDFB7P8FtfVapLk77U NZTVT4bruUBHH0iEWzyhmPrqn06yO4qQquzv/1Ed5zv5p32kFAuUlUkWUY27KysqADHU GpoBeyZ6ybpv4W0Dtt0rCPy7qaDyF4d0qw3WqJbNEeN13QJDWBBOGx0JS2jh7R5y0FSG q1bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709629157; x=1710233957; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j1eQPmD/cclnNs/ltzs+A6VvqcjsXDrfmpDDpfoLHOg=; b=bcqSmbssEeitzzoPFtx8SU2z5h9K2f1dS6nihgGMGkaoMjj74/YKjRL9vc0H16ClKh ywp9WkX7byg3Z0nVF3LXPgPssxMDPXiFWXVQmwQuij1SiMtJc+CUquutm1npJJtvXtD0 EiED4T1q2IyMtLzTeah1SZMuxMDQJ7yxhNHUEAgqf5cqZ/FdX4rEPrOmXKgPpyjB3Z7M gaD5HS5kDacoL5zdcyH2R+AGHfMjyRSd7lPuYPoPeiwLeNJ8wVHF6nwjDJ+gzmFrJv6E mKH/7QrYcB9Usm5udRwz0N2Rf/iLmFddFKfIgyIVwa945geq5mTFI/DtVUlDSdSC/D4q BT+Q== X-Gm-Message-State: AOJu0YyTTjW3kteu3g1uvcTgHF4PFHC2I+aGDZhszT2J8dXKX8PD6V2h 3a9aO3mPjeyQwMxbCi4t9sYNkYEAVE6Fn1RzcqYdKd5AVA9usUH54GvSGcoTAj4= X-Google-Smtp-Source: AGHT+IGt2dmyj/ohIQu59YQYCU/rsKzk2NXrPP8Fy3rt4mrV2efjMM5ippzwmJi2qrfKsGUBr0iCAQ== X-Received: by 2002:a05:6a20:c315:b0:19e:b9ab:189e with SMTP id dk21-20020a056a20c31500b0019eb9ab189emr926520pzb.60.1709629156909; Tue, 05 Mar 2024 00:59:16 -0800 (PST) Received: from localhost ([157.82.203.206]) by smtp.gmail.com with UTF8SMTPSA id c5-20020a170902d48500b001db693d89fdsm9984324plg.179.2024.03.05.00.59.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Mar 2024 00:59:16 -0800 (PST) From: Akihiko Odaki Date: Tue, 05 Mar 2024 17:58:51 +0900 Subject: [PATCH RFC v3 4/6] pcie_sriov: Allow user to create SR-IOV device MIME-Version: 1.0 Message-Id: <20240305-sriov-v3-4-abdb75770372@daynix.com> References: <20240305-sriov-v3-0-abdb75770372@daynix.com> In-Reply-To: <20240305-sriov-v3-0-abdb75770372@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, 20240228-reuse-v8-0-282660281e60@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.12.3 Received-SPF: none client-ip=2607:f8b0:4864:20::531; envelope-from=akihiko.odaki@daynix.com; helo=mail-pg1-x531.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A user can create a SR-IOV device by specifying the PF with the sriov-pf property of the VFs. The VFs must be added before the PF. A user-creatable VF must have PCIDeviceClass::sriov_vf_user_creatable set. Such a VF cannot refer to the PF because it is created before the PF. A PF that user-creatable VFs can be attached calls pcie_sriov_pf_init_from_user_created_vfs() during realization and pcie_sriov_pf_exit() when exiting. Signed-off-by: Akihiko Odaki --- include/hw/pci/pci_device.h | 6 +- include/hw/pci/pcie_sriov.h | 19 +++ hw/pci/pci.c | 62 ++++++---- hw/pci/pcie_sriov.c | 289 +++++++++++++++++++++++++++++++++++--------- 4 files changed, 293 insertions(+), 83 deletions(-) diff --git a/include/hw/pci/pci_device.h b/include/hw/pci/pci_device.h index 6be0f989ebe0..eefd9d9a7b5a 100644 --- a/include/hw/pci/pci_device.h +++ b/include/hw/pci/pci_device.h @@ -37,6 +37,8 @@ struct PCIDeviceClass { uint16_t subsystem_id; /* only for header type = 0 */ const char *romfile; /* rom bar */ + + bool sriov_vf_user_creatable; }; enum PCIReqIDType { @@ -160,6 +162,8 @@ struct PCIDevice { /* ID of standby device in net_failover pair */ char *failover_pair_id; uint32_t acpi_index; + + char *sriov_pf; }; static inline int pci_intx(PCIDevice *pci_dev) @@ -192,7 +196,7 @@ static inline int pci_is_express_downstream_port(const PCIDevice *d) static inline int pci_is_vf(const PCIDevice *d) { - return d->exp.sriov_vf.pf != NULL; + return d->sriov_pf || d->exp.sriov_vf.pf != NULL; } static inline uint32_t pci_config_size(const PCIDevice *d) diff --git a/include/hw/pci/pcie_sriov.h b/include/hw/pci/pcie_sriov.h index d576a8c6be19..20626b5605c9 100644 --- a/include/hw/pci/pcie_sriov.h +++ b/include/hw/pci/pcie_sriov.h @@ -18,6 +18,7 @@ struct PCIESriovPF { uint8_t vf_bar_type[PCI_NUM_REGIONS]; /* Store type for each VF bar */ PCIDevice **vf; /* Pointer to an array of num_vfs VF devices */ + bool vf_user_created; /* If VFs are created by user */ }; struct PCIESriovVF { @@ -40,6 +41,24 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory); +/** + * pcie_sriov_pf_init_from_user_created_vfs() - Initialize PF with user-created + * VFs. + * @dev: A PCIe device being realized. + * @offset: The offset of the SR-IOV capability. + * @errp: pointer to Error*, to store an error if it happens. + * + * Return: + * * true - @dev is initialized as a PCIe SR-IOV PF. + * * false - @dev is not initialized because there is no SR-IOV VFs or an error + * occurred. + */ +bool pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, uint16_t offset, + Error **errp); + +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp); +void pcie_sriov_unregister_device(PCIDevice *dev); + /* * Default (minimal) page size support values * as required by the SR/IOV standard: diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 201ff64e11cc..40fa234bd6c8 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -85,6 +85,7 @@ static Property pci_props[] = { QEMU_PCIE_ERR_UNC_MASK_BITNR, true), DEFINE_PROP_BIT("x-pcie-ari-nextfn-1", PCIDevice, cap_present, QEMU_PCIE_ARI_NEXTFN_1_BITNR, false), + DEFINE_PROP_STRING("sriov-pf", PCIDevice, sriov_pf), DEFINE_PROP_END_OF_LIST() }; @@ -959,13 +960,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) dev->config[PCI_HEADER_TYPE] |= PCI_HEADER_TYPE_MULTI_FUNCTION; } - /* - * With SR/IOV and ARI, a device at function 0 need not be a multifunction - * device, as it may just be a VF that ended up with function 0 in - * the legacy PCI interpretation. Avoid failing in such cases: - */ - if (pci_is_vf(dev) && - dev->exp.sriov_vf.pf->cap_present & QEMU_PCI_CAP_MULTIFUNCTION) { + /* SR/IOV is not handled here. */ + if (pci_is_vf(dev)) { return; } @@ -998,7 +994,8 @@ static void pci_init_multifunction(PCIBus *bus, PCIDevice *dev, Error **errp) } /* function 0 indicates single function, so function > 0 must be NULL */ for (func = 1; func < PCI_FUNC_MAX; ++func) { - if (bus->devices[PCI_DEVFN(slot, func)]) { + PCIDevice *device = bus->devices[PCI_DEVFN(slot, func)]; + if (device && !pci_is_vf(device)) { error_setg(errp, "PCI: %x.0 indicates single function, " "but %x.%x is already populated.", slot, slot, func); @@ -1283,6 +1280,7 @@ static void pci_qdev_unrealize(DeviceState *dev) pci_unregister_io_regions(pci_dev); pci_del_option_rom(pci_dev); + pcie_sriov_unregister_device(pci_dev); if (pc->exit) { pc->exit(pci_dev); @@ -1314,7 +1312,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, pcibus_t size = memory_region_size(memory); uint8_t hdr_type; - assert(!pci_is_vf(pci_dev)); /* VFs must use pcie_sriov_vf_register_bar */ assert(region_num >= 0); assert(region_num < PCI_NUM_REGIONS); assert(is_power_of_2(size)); @@ -1325,7 +1322,6 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, assert(hdr_type != PCI_HEADER_TYPE_BRIDGE || region_num < 2); r = &pci_dev->io_regions[region_num]; - r->addr = PCI_BAR_UNMAPPED; r->size = size; r->type = type; r->memory = memory; @@ -1333,22 +1329,35 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num, ? pci_get_bus(pci_dev)->address_space_io : pci_get_bus(pci_dev)->address_space_mem; - wmask = ~(size - 1); - if (region_num == PCI_ROM_SLOT) { - /* ROM enable bit is writable */ - wmask |= PCI_ROM_ADDRESS_ENABLE; - } - - addr = pci_bar(pci_dev, region_num); - pci_set_long(pci_dev->config + addr, type); + if (pci_is_vf(pci_dev)) { + PCIDevice *pf = pci_dev->exp.sriov_vf.pf; + assert(!pf || type == pf->exp.sriov_pf.vf_bar_type[region_num]); - if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && - r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { - pci_set_quad(pci_dev->wmask + addr, wmask); - pci_set_quad(pci_dev->cmask + addr, ~0ULL); + r->addr = pci_bar_address(pci_dev, region_num, r->type, r->size); + if (r->addr != PCI_BAR_UNMAPPED) { + memory_region_add_subregion_overlap(r->address_space, + r->addr, r->memory, 1); + } } else { - pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); - pci_set_long(pci_dev->cmask + addr, 0xffffffff); + r->addr = PCI_BAR_UNMAPPED; + + wmask = ~(size - 1); + if (region_num == PCI_ROM_SLOT) { + /* ROM enable bit is writable */ + wmask |= PCI_ROM_ADDRESS_ENABLE; + } + + addr = pci_bar(pci_dev, region_num); + pci_set_long(pci_dev->config + addr, type); + + if (!(r->type & PCI_BASE_ADDRESS_SPACE_IO) && + r->type & PCI_BASE_ADDRESS_MEM_TYPE_64) { + pci_set_quad(pci_dev->wmask + addr, wmask); + pci_set_quad(pci_dev->cmask + addr, ~0ULL); + } else { + pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff); + pci_set_long(pci_dev->cmask + addr, 0xffffffff); + } } } @@ -2105,6 +2114,11 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp) } } + if (!pcie_sriov_register_device(pci_dev, errp)) { + pci_qdev_unrealize(DEVICE(pci_dev)); + return; + } + /* * A PCIe Downstream Port that do not have ARI Forwarding enabled must * associate only Device 0 with the device attached to the bus diff --git a/hw/pci/pcie_sriov.c b/hw/pci/pcie_sriov.c index c449ddd0ac39..2693667e7870 100644 --- a/hw/pci/pcie_sriov.c +++ b/hw/pci/pcie_sriov.c @@ -17,8 +17,11 @@ #include "hw/qdev-properties.h" #include "qemu/error-report.h" #include "qemu/range.h" +#include "qapi/error.h" #include "trace.h" +static GHashTable *pfs; + static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) { for (uint16_t i = 0; i < total_vfs; i++) { @@ -30,14 +33,49 @@ static void unparent_vfs(PCIDevice *dev, uint16_t total_vfs) dev->exp.sriov_pf.vf = NULL; } -bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, - const char *vfname, uint16_t vf_dev_id, - uint16_t init_vfs, uint16_t total_vfs, - uint16_t vf_offset, uint16_t vf_stride, - Error **errp) +static void clear_ctrl_vfe(PCIDevice *dev) +{ + uint8_t *ctrl = dev->config + dev->exp.sriov_cap + PCI_SRIOV_CTRL; + pci_set_word(ctrl, pci_get_word(ctrl) & ~PCI_SRIOV_CTRL_VFE); +} + +static void register_vfs(PCIDevice *dev) +{ + uint16_t num_vfs; + uint16_t i; + uint16_t sriov_cap = dev->exp.sriov_cap; + + assert(sriov_cap > 0); + num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); + if (num_vfs > pci_get_word(dev->config + sriov_cap + PCI_SRIOV_TOTAL_VF)) { + clear_ctrl_vfe(dev); + return; + } + + trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn), num_vfs); + for (i = 0; i < num_vfs; i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + } +} + +static void unregister_vfs(PCIDevice *dev) +{ + uint16_t i; + uint8_t *cfg = dev->config + dev->exp.sriov_cap; + + trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn)); + for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { + pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + } +} + +static bool pcie_sriov_pf_init_common(PCIDevice *dev, uint16_t offset, + uint16_t vf_dev_id, uint16_t init_vfs, + uint16_t total_vfs, uint16_t vf_offset, + uint16_t vf_stride, Error **errp) { - BusState *bus = qdev_get_parent_bus(&dev->qdev); - int32_t devfn = dev->devfn + vf_offset; uint8_t *cfg = dev->config + offset; uint8_t *wmask; @@ -87,6 +125,28 @@ bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, qdev_prop_set_bit(&dev->qdev, "multifunction", true); + return true; +} + +bool pcie_sriov_pf_init(PCIDevice *dev, uint16_t offset, + const char *vfname, uint16_t vf_dev_id, + uint16_t init_vfs, uint16_t total_vfs, + uint16_t vf_offset, uint16_t vf_stride, + Error **errp) +{ + BusState *bus = qdev_get_parent_bus(&dev->qdev); + int32_t devfn = dev->devfn + vf_offset; + + if (pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, init_vfs, + total_vfs, vf_offset, vf_stride, errp)) { + return false; + } + dev->exp.sriov_pf.vf = g_new(PCIDevice *, total_vfs); for (uint16_t i = 0; i < total_vfs; i++) { @@ -116,7 +176,22 @@ void pcie_sriov_pf_exit(PCIDevice *dev) { uint8_t *cfg = dev->config + dev->exp.sriov_cap; - unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + if (dev->exp.sriov_pf.vf_user_created) { + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t total_vfs = pci_get_word(dev->config + PCI_SRIOV_TOTAL_VF); + uint16_t vf_dev_id = pci_get_word(dev->config + PCI_SRIOV_VF_DID); + + unregister_vfs(dev); + + for (uint16_t i = 0; i < total_vfs; i++) { + dev->exp.sriov_pf.vf[i]->exp.sriov_vf.pf = NULL; + + pci_config_set_vendor_id(dev->exp.sriov_pf.vf[i]->config, ven_id); + pci_config_set_device_id(dev->exp.sriov_pf.vf[i]->config, vf_dev_id); + } + } else { + unparent_vfs(dev, pci_get_word(cfg + PCI_SRIOV_TOTAL_VF)); + } } void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, @@ -149,74 +224,172 @@ void pcie_sriov_pf_init_vf_bar(PCIDevice *dev, int region_num, void pcie_sriov_vf_register_bar(PCIDevice *dev, int region_num, MemoryRegion *memory) { - PCIIORegion *r; - PCIBus *bus = pci_get_bus(dev); uint8_t type; - pcibus_t size = memory_region_size(memory); - assert(pci_is_vf(dev)); /* PFs must use pci_register_bar */ - assert(region_num >= 0); - assert(region_num < PCI_NUM_REGIONS); + assert(dev->exp.sriov_vf.pf); type = dev->exp.sriov_vf.pf->exp.sriov_pf.vf_bar_type[region_num]; - if (!is_power_of_2(size)) { - error_report("%s: PCI region size must be a power" - " of two - type=0x%x, size=0x%"FMT_PCIBUS, - __func__, type, size); - exit(1); - } - - r = &dev->io_regions[region_num]; - r->memory = memory; - r->address_space = - type & PCI_BASE_ADDRESS_SPACE_IO - ? bus->address_space_io - : bus->address_space_mem; - r->size = size; - r->type = type; - - r->addr = pci_bar_address(dev, region_num, r->type, r->size); - if (r->addr != PCI_BAR_UNMAPPED) { - memory_region_add_subregion_overlap(r->address_space, - r->addr, r->memory, 1); - } + return pci_register_bar(dev, region_num, type, memory); } -static void clear_ctrl_vfe(PCIDevice *dev) +static gint compare_vf_devfns(gconstpointer a, gconstpointer b) { - uint8_t *ctrl = dev->config + dev->exp.sriov_cap + PCI_SRIOV_CTRL; - pci_set_word(ctrl, pci_get_word(ctrl) & ~PCI_SRIOV_CTRL_VFE); + return (*(PCIDevice **)a)->devfn - (*(PCIDevice **)b)->devfn; } -static void register_vfs(PCIDevice *dev) +bool pcie_sriov_pf_init_from_user_created_vfs(PCIDevice *dev, uint16_t offset, + Error **errp) { - uint16_t num_vfs; + GPtrArray *pf; + PCIDevice **vfs; + BusState *bus = qdev_get_parent_bus(DEVICE(dev)); + uint16_t ven_id = pci_get_word(dev->config + PCI_VENDOR_ID); + uint16_t vf_dev_id; + uint16_t vf_offset; + uint16_t vf_stride; uint16_t i; - uint16_t sriov_cap = dev->exp.sriov_cap; - assert(sriov_cap > 0); - num_vfs = pci_get_word(dev->config + sriov_cap + PCI_SRIOV_NUM_VF); - if (num_vfs > pci_get_word(dev->config + sriov_cap + PCI_SRIOV_TOTAL_VF)) { - clear_ctrl_vfe(dev); - return; + if (!pfs || !dev->qdev.id) { + return false; } - trace_sriov_register_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn), num_vfs); - for (i = 0; i < num_vfs; i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], true); + pf = g_hash_table_lookup(pfs, dev->qdev.id); + if (!pf) { + return false; + } + + if (pf->len > UINT16_MAX) { + error_setg(errp, "too many VFs"); + return false; + } + + g_ptr_array_sort(pf, compare_vf_devfns); + vfs = (void *)pf->pdata; + + if (vfs[0]->devfn <= dev->devfn) { + error_setg(errp, "a VF function number is less than the PF function number"); + return false; } + + vf_dev_id = pci_get_word(vfs[0]->config + PCI_DEVICE_ID); + vf_offset = vfs[0]->devfn - dev->devfn; + vf_stride = pf->len < 2 ? 0 : vfs[1]->devfn - vfs[0]->devfn; + + for (i = 0; i < pf->len; i++) { + if (bus != qdev_get_parent_bus(&vfs[i]->qdev)) { + error_setg(errp, "SR-IOV VF parent bus mismatches with PF"); + return false; + } + + if (ven_id != pci_get_word(vfs[i]->config + PCI_VENDOR_ID)) { + error_setg(errp, "SR-IOV VF vendor ID mismatches with PF"); + return false; + } + + if (vf_dev_id != pci_get_word(vfs[i]->config + PCI_DEVICE_ID)) { + error_setg(errp, "inconsistent SR-IOV VF device IDs"); + return false; + } + + for (size_t j = 0; j < PCI_NUM_REGIONS; j++) { + if (vfs[i]->io_regions[j].size != vfs[0]->io_regions[j].size || + vfs[i]->io_regions[j].type != vfs[0]->io_regions[j].type) { + error_setg(errp, "inconsistent SR-IOV BARs"); + return false; + } + } + + if (vfs[i]->devfn - vfs[0]->devfn != vf_stride * i) { + error_setg(errp, "inconsistent SR-IOV stride"); + return false; + } + } + + if (!pcie_sriov_pf_init_common(dev, offset, vf_dev_id, pf->len, + pf->len, vf_offset, vf_stride, errp)) { + return false; + } + + for (i = 0; i < pf->len; i++) { + vfs[i]->exp.sriov_vf.pf = dev; + vfs[i]->exp.sriov_vf.vf_number = i; + + /* set vid/did according to sr/iov spec - they are not used */ + pci_config_set_vendor_id(vfs[i]->config, 0xffff); + pci_config_set_device_id(vfs[i]->config, 0xffff); + } + + dev->exp.sriov_pf.vf = vfs; + dev->exp.sriov_pf.vf_user_created = true; + + for (i = 0; i < PCI_NUM_REGIONS; i++) { + uint8_t type = vfs[0]->io_regions[i].type; + pcibus_t size = vfs[0]->io_regions[i].size; + + if (size) { + pcie_sriov_pf_init_vf_bar(dev, i, type, size); + } + } + + return true; } -static void unregister_vfs(PCIDevice *dev) +bool pcie_sriov_register_device(PCIDevice *dev, Error **errp) { - uint16_t i; - uint8_t *cfg = dev->config + dev->exp.sriov_cap; + if (!dev->exp.sriov_pf.vf && dev->qdev.id && + pfs && g_hash_table_contains(pfs, dev->qdev.id)) { + error_setg(errp, "attaching user-created SR-IOV VF unsupported"); + return false; + } - trace_sriov_unregister_vfs(dev->name, PCI_SLOT(dev->devfn), - PCI_FUNC(dev->devfn)); - for (i = 0; i < pci_get_word(cfg + PCI_SRIOV_TOTAL_VF); i++) { - pci_set_enabled(dev->exp.sriov_pf.vf[i], false); + if (dev->sriov_pf) { + PCIDevice *pci_pf; + GPtrArray *pf; + + if (!PCI_DEVICE_GET_CLASS(dev)->sriov_vf_user_creatable) { + error_setg(errp, "user cannot create SR-IOV VF with this device type"); + return false; + } + + if (!pci_is_express(dev)) { + error_setg(errp, "PCI Express is required for SR-IOV VF"); + return false; + } + + if (!pci_qdev_find_device(dev->sriov_pf, &pci_pf)) { + error_setg(errp, "PCI device specified as SR-IOV PF already exists"); + return false; + } + + if (!pfs) { + pfs = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, NULL); + } + + pf = g_hash_table_lookup(pfs, dev->sriov_pf); + if (!pf) { + pf = g_ptr_array_new(); + g_hash_table_insert(pfs, g_strdup(dev->sriov_pf), pf); + } + + g_ptr_array_add(pf, dev); + } + + return true; +} + +void pcie_sriov_unregister_device(PCIDevice *dev) +{ + if (dev->sriov_pf && pfs) { + GPtrArray *pf = g_hash_table_lookup(pfs, dev->qdev.id); + + if (pf) { + g_ptr_array_remove_fast(pf, dev); + + if (!pf->len) { + g_hash_table_remove(pfs, dev->qdev.id); + g_ptr_array_free(pf, FALSE); + } + } } } @@ -303,7 +476,7 @@ void pcie_sriov_pf_add_sup_pgsize(PCIDevice *dev, uint16_t opt_sup_pgsize) uint16_t pcie_sriov_vf_number(PCIDevice *dev) { - assert(pci_is_vf(dev)); + assert(dev->exp.sriov_vf.pf); return dev->exp.sriov_vf.vf_number; } From patchwork Tue Mar 5 08:58:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13581870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F756C54E55 for ; Tue, 5 Mar 2024 09:00:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhQe7-00043u-8x; Tue, 05 Mar 2024 03:59:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhQdx-0003xv-Td for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:26 -0500 Received: from mail-oi1-x22f.google.com ([2607:f8b0:4864:20::22f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhQdw-0003i7-8t for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:25 -0500 Received: by mail-oi1-x22f.google.com with SMTP id 5614622812f47-3c1f55ba3ecso738529b6e.3 for ; Tue, 05 Mar 2024 00:59:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1709629162; x=1710233962; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=9/PxcCc+L0baSbSLqEG7FKAVe7dX0Ps3Ftgof5mfrJs=; b=sUpZxPPnAf6OtD+zAvanZLg8lCNccscTITyKzGfKQnD5vZacn55yN+BZs2sa9p/5Db fNgS+gigW+vZvxu/AHh6mahq6HJ9ScGIjD39Oyu9JgjigiUrTv1E1qR2P1MWi58h2MMI LSgigaV0FBuLNEjp2HP8F2EHCFpwOELXINXTzeDP/unSt3aaLvvfMzd5Ap+MbzYel0KQ WfHCIkGMq1oUB3JKeMDnuLmNl9TbIMRbXXi/6ugK14pLelqOwgsABH5lv8di/tK0X1f9 3dJozvAfEP6W063OrRI31mZduBO4ml5UzCiMjsjM8zhboBZ3wCZr3gOIEt+bF80qe9sY H8Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709629162; x=1710233962; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9/PxcCc+L0baSbSLqEG7FKAVe7dX0Ps3Ftgof5mfrJs=; b=Mpm/FPiL5YK/8+ybAE9GJp8xYbWDyjuNHghG+3bCv98Td0HxaemLsz3/UNV7u4fUdJ CTNR6WqkRyVgqPrfGVzTZ3n85nONkQxPXB+4HVvzszH3FHQ1aDWBuxpDTOczpVUS7V6l Mo3ySvR4gPLANLkVyj9CFPe8kz6ekwVuDJu8Mh8HCLHNoGn54BFYZcU9qycOcEnrbYqi mPXrOzW06FE+BGGOXPVzXOaHVY4KS2IqBcAL0Nus6FWfgSW1RamLvuOgKPn4aTYlFJ7W wEL7csl51rv+MosAQMbI0P1FWMlpHkoH+JuKiprtdzOX/P81BLZgAqMS+LW0jc6T+HDQ MxUw== X-Gm-Message-State: AOJu0YzSDlf2eunl7T9yFZY2N8NEtqu9YXNrr60uDOXiBQz+jRDsAWoz UHyujvBlrEVpJ30LMBe/b+vD5cR54hFZUc3HH0DMq9cv10KLHeTVB2ghEc2Nvao= X-Google-Smtp-Source: AGHT+IHNoMHyxDp+6hezykFHxjdP1Dj4Ao25i1zldbKekttOwMNAckNvfxWys5IfhxF4QNPavee6/g== X-Received: by 2002:a05:6808:1524:b0:3c1:ed50:7c2b with SMTP id u36-20020a056808152400b003c1ed507c2bmr1404124oiw.47.1709629162351; Tue, 05 Mar 2024 00:59:22 -0800 (PST) Received: from localhost ([157.82.203.206]) by smtp.gmail.com with UTF8SMTPSA id a30-20020a631a5e000000b005d30550f954sm8652972pgm.31.2024.03.05.00.59.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Mar 2024 00:59:21 -0800 (PST) From: Akihiko Odaki Date: Tue, 05 Mar 2024 17:58:52 +0900 Subject: [PATCH RFC v3 5/6] virtio-pci: Implement SR-IOV PF MIME-Version: 1.0 Message-Id: <20240305-sriov-v3-5-abdb75770372@daynix.com> References: <20240305-sriov-v3-0-abdb75770372@daynix.com> In-Reply-To: <20240305-sriov-v3-0-abdb75770372@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, 20240228-reuse-v8-0-282660281e60@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.12.3 Received-SPF: none client-ip=2607:f8b0:4864:20::22f; envelope-from=akihiko.odaki@daynix.com; helo=mail-oi1-x22f.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Allow user to attach SR-IOV VF to a virtio-pci PF. Signed-off-by: Akihiko Odaki --- hw/virtio/virtio-pci.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 1a7039fb0c68..f6a2dbb3b5e2 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -2075,6 +2075,12 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp) pci_register_bar(&proxy->pci_dev, proxy->legacy_io_bar_idx, PCI_BASE_ADDRESS_SPACE_IO, &proxy->bar); } + + if (pcie_sriov_pf_init_from_user_created_vfs(&proxy->pci_dev, + PCI_CONFIG_SPACE_SIZE, + errp)) { + virtio_add_feature(&vdev->host_features, VIRTIO_F_SR_IOV); + } } static void virtio_pci_device_unplugged(DeviceState *d) @@ -2083,6 +2089,7 @@ static void virtio_pci_device_unplugged(DeviceState *d) bool modern = virtio_pci_modern(proxy); bool modern_pio = proxy->flags & VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY; + pcie_sriov_pf_exit(&proxy->pci_dev); virtio_pci_stop_ioeventfd(proxy); if (modern) { From patchwork Tue Mar 5 08:58:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akihiko Odaki X-Patchwork-Id: 13581873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2351DC54798 for ; Tue, 5 Mar 2024 09:01:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhQeA-0004IT-6J; Tue, 05 Mar 2024 03:59:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhQe1-000434-T9 for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:30 -0500 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhQe0-0003ic-E1 for qemu-devel@nongnu.org; Tue, 05 Mar 2024 03:59:29 -0500 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1dc75972f25so44871415ad.1 for ; Tue, 05 Mar 2024 00:59:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20230601.gappssmtp.com; s=20230601; t=1709629167; x=1710233967; darn=nongnu.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=agEvzBa4RHbrJsI2JBzinS8CmNut8NaPKnAYNO+Q2DM=; b=UAbW0h6FOVu17lww1MdOmJQzERVEg8J1bySy3cF+day+0KDDN1Ojr4IwOdQuH0niXR s/G5xrJYVPI7eljUvPUaaMgYMnvzt2yZur/mqP0SW8sgfpgJlgTsiMafT/1EuMA6Fahl rfOkV0XunLlKJObQvw/6OP7RMDrxAGw+dopnsRkxDaFw4PkedM/wTdK9m2urh4Cjy9CC mctUNrMrKwS0FzsPAwLXf0oQaYX7fVSI8N2YUlmMrysFKb+dTz4EI0JlZJh4y6Sawqob 02kuOKml0FDhdZvaZSlW+DDJsdaqAJx14ZEeMqgCDXKnT0VsaB4ztEyAJ+4DurZwJDLv N2OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709629167; x=1710233967; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=agEvzBa4RHbrJsI2JBzinS8CmNut8NaPKnAYNO+Q2DM=; b=I5jM5xnhED1nPdHwOPACTGXMPyMRisxVSpB3ei7ReDCTGMEEasIvu96lb16i8Mq3Bp H0GPRJpNl/kA+SJ9Kt24K32gGdDyj68hKw/IxnoLuNqZuUUWgsgDQ7YFTLdCJkRSDDNB 628atUF65z6PCBTZwTaQL05AA5WgBY0DHqS/EV8aQnfj8ZAI/7QE7n5vfYboFYI2ahD3 1Qmfe8XrQVo4yu+IDveDJ2wL9NsZx+bKwo+i25s1X0cpCuvfdWbtuD1y223IHuaIlpQE 6F0AUmqYmQpnxd2n61cL1M0/pVP+aqh4bSGcMKVS2bUAl30OxrFOD5mHkBaXxHqfF3RQ dZlg== X-Gm-Message-State: AOJu0YyVv/sHsejNBUI/ZM9MWJlJqFc/RNdxte8T1I+QJHrw46S3lVE4 N07bxgIQD9F6zoatAYwpt9WYXM8aNS4Y5A17CJOS9yZ/ZGxmG6SYwQIzuo8cbug= X-Google-Smtp-Source: AGHT+IGxiMWtBmC2IwLho96So1A+U3w93tecQCB5aKQ9TL6sI1iiHX2jwOWCgSQsn4EdbdV1bTG3Zg== X-Received: by 2002:a17:902:a987:b0:1db:fc02:f96e with SMTP id bh7-20020a170902a98700b001dbfc02f96emr961011plb.24.1709629167148; Tue, 05 Mar 2024 00:59:27 -0800 (PST) Received: from localhost ([157.82.203.206]) by smtp.gmail.com with UTF8SMTPSA id l12-20020a170903120c00b001db66f3748fsm9934013plh.182.2024.03.05.00.59.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Mar 2024 00:59:26 -0800 (PST) From: Akihiko Odaki Date: Tue, 05 Mar 2024 17:58:53 +0900 Subject: [PATCH RFC v3 6/6] virtio-net: Implement SR-IOV VF MIME-Version: 1.0 Message-Id: <20240305-sriov-v3-6-abdb75770372@daynix.com> References: <20240305-sriov-v3-0-abdb75770372@daynix.com> In-Reply-To: <20240305-sriov-v3-0-abdb75770372@daynix.com> To: "Michael S. Tsirkin" , Marcel Apfelbaum , Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_Goa?= =?utf-8?q?ter?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Jason Wang , Sriram Yagnaraman , Keith Busch , Klaus Jensen Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, 20240228-reuse-v8-0-282660281e60@daynix.com, Yui Washizu , Akihiko Odaki X-Mailer: b4 0.12.3 Received-SPF: none client-ip=2607:f8b0:4864:20::62c; envelope-from=akihiko.odaki@daynix.com; helo=mail-pl1-x62c.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org A virtio-net device can be added as a SR-IOV VF to another virtio-pci device that will be the PF. Signed-off-by: Akihiko Odaki --- hw/virtio/virtio-net-pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/virtio/virtio-net-pci.c b/hw/virtio/virtio-net-pci.c index e03543a70a75..dba4987d6e04 100644 --- a/hw/virtio/virtio-net-pci.c +++ b/hw/virtio/virtio-net-pci.c @@ -75,6 +75,7 @@ static void virtio_net_pci_class_init(ObjectClass *klass, void *data) k->device_id = PCI_DEVICE_ID_VIRTIO_NET; k->revision = VIRTIO_PCI_ABI_VERSION; k->class_id = PCI_CLASS_NETWORK_ETHERNET; + k->sriov_vf_user_creatable = true; set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); device_class_set_props(dc, virtio_net_properties); vpciklass->realize = virtio_net_pci_realize;