From patchwork Tue Apr 12 22:43:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tanjore Suresh X-Patchwork-Id: 12811275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEC6BC433EF for ; Tue, 12 Apr 2022 23:22:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229966AbiDLXZM (ORCPT ); Tue, 12 Apr 2022 19:25:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229783AbiDLXY3 (ORCPT ); Tue, 12 Apr 2022 19:24:29 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 196897CDCF for ; Tue, 12 Apr 2022 15:43:56 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id z15-20020a25bb0f000000b00613388c7d99so276367ybg.8 for ; Tue, 12 Apr 2022 15:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=AMvt99Ex3LWgVMSklCg5XNF3Fu8kmLTfCLQjqVndY6w=; b=RazfDvaD5aj8XVU4guL9lnFicByqFFYKMKgI9ETIiiTeEWVsXQiUmC4r9d+KA7NlGz SMpX/oUt8RqMU0SPyZhXdqUQDx8LQQUHQ6iD3GKz8/G7FOVI0743NRajKeuQyq4Oxk4W eFksHyrtSZwa3mmURayx+lMlxJJVuTmYVWEbV46ZNxN8Mx+pyDt8lkTgCjOVXGBWGHCm yHyM2NQPx3xwEvLMhtxV4Jxn4NYyIPvlfVHF1PJ8ypg4N4dqW624dI5Z07Dr/nvsczuy CXbA2dXFHKpoD+vUdHIKSREFJjDiBl/oozVwp9T6RVdmqMsulQamDI0WhVn3V2vcxJvy OjsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=AMvt99Ex3LWgVMSklCg5XNF3Fu8kmLTfCLQjqVndY6w=; b=2z5JJES7Ylfpq7K/szxVQ0sZIGfMBhQOkInf2pKRPv+uwn5Ppx7uZvI2z7qHk76xSk zMfkcUilOAlJ+GonULE4U/mN7yBC0jqRsUxqW0RHCs8qguy+TGLN2G7IVnoIMmnlt8OG 67lqZV50v6GEjn8XjDKvWjAeqXZyvhBUsNhdfjc200z74z2qK9GKm+cWm72RPmsf19QB AZzscyxZSxRG1I9uGWKnCxoacXlw/CB4S96sw614GI9mS1K54DIrRNaFuk5PBDizFzma J1/q15/mFVq64804gPug0HgoJ/TMf/l9UE4YKZfOR9n9+MISKfGNPfSkZCXBYvt6+fX+ 7+TQ== X-Gm-Message-State: AOAM531/So7dOs0rqf6mNaFnVFZS4foiDWPQLMhAIjYVWjbPQmlqtqPP PWGZdQrEwHC3vjADjcfQQIceQVU4I3yIwyk= X-Google-Smtp-Source: ABdhPJxCd0bNO7tLDwtDjUMz0mlBl45zHQLJLdIutO0p8cCHIr5/4Sb9YUXg6f/qhNMrRWACrr9iOa0pZYfPhJM= X-Received: from tansuresh.svl.corp.google.com ([2620:15c:2c5:13:8573:aa64:c3e8:ebc]) (user=tansuresh job=sendgmr) by 2002:a25:d34f:0:b0:641:3dcc:c2de with SMTP id e76-20020a25d34f000000b006413dccc2demr11259325ybf.547.1649803435313; Tue, 12 Apr 2022 15:43:55 -0700 (PDT) Date: Tue, 12 Apr 2022 15:43:45 -0700 Message-Id: <20220412224348.1038613-1-tansuresh@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.36.0.rc0.470.gd361397f0d-goog Subject: [PATCH v2 0/3] Asynchronous shutdown interface and example implementation From: Tanjore Suresh To: Greg Kroah-Hartman , "Rafael J . Wysocki" , Christoph Hellwig , Sagi Grimberg , Bjorn Helgaas Cc: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, Tanjore Suresh Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Problem: Some of our machines are configured with many NVMe devices and are validated for strict shutdown time requirements. Each NVMe device plugged into the system, typicaly takes about 4.5 secs to shutdown. A system with 16 such NVMe devices will takes approximately 80 secs to shutdown and go through reboot. The current shutdown APIs as defined at bus level is defined to be synchronous. Therefore, more devices are in the system the greater the time it takes to shutdown. This shutdown time significantly contributes the machine reboot time. Solution: This patch set proposes an asynchronous shutdown interface at bus level, modifies the core driver, device shutdown routine to exploit the new interface while maintaining backward compatibility with synchronous implementation already existing (Patch 1 of 3) and exploits new interface to enable all PCI-E based devices to use asynchronous interface semantics if necessary (Patch 2 of 3). The implementation at PCI-E level also works in a backward compatible way, to allow exiting device implementation to work with current synchronous semantics. Only show cases an example implementation for NVMe device to exploit this asynchronous shutdown interface. (Patch 3 of 3). Changelog: v2: - Replaced the shutdown_pre & shutdown_post entry point names with the recommended names (async_shutdown_start and asynch_shutdown_end). - Comment about ordering requirements between bridge shutdown versus leaf/endpoint shutdown was agreed to be different when calling async_shutdown_start and async_shutdown_end. Now this implements the same order of calling both start and end entry points. Tanjore Suresh (3): driver core: Support asynchronous driver shutdown PCI: Support asynchronous shutdown nvme: Add async shutdown support drivers/base/core.c | 38 +++++++++++++++++- drivers/nvme/host/core.c | 28 +++++++++---- drivers/nvme/host/nvme.h | 8 ++++ drivers/nvme/host/pci.c | 80 ++++++++++++++++++++++++-------------- drivers/pci/pci-driver.c | 20 ++++++++-- include/linux/device/bus.h | 12 ++++++ include/linux/pci.h | 4 ++ 7 files changed, 149 insertions(+), 41 deletions(-)