From patchwork Tue Dec 3 22:33:14 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rajat Jain X-Patchwork-Id: 3279151 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 239B89F377 for ; Tue, 3 Dec 2013 22:33:24 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 01A822047C for ; Tue, 3 Dec 2013 22:33:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A4FE220465 for ; Tue, 3 Dec 2013 22:33:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755310Ab3LCWdU (ORCPT ); Tue, 3 Dec 2013 17:33:20 -0500 Received: from mail-pb0-f53.google.com ([209.85.160.53]:51923 "EHLO mail-pb0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755117Ab3LCWdR (ORCPT ); Tue, 3 Dec 2013 17:33:17 -0500 Received: by mail-pb0-f53.google.com with SMTP id ma3so22101078pbc.12 for ; Tue, 03 Dec 2013 14:33:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :content-type:content-transfer-encoding; bh=YL+K8ixxC5HAAKLZ8OigNu1rknGNuCasYmMWFA40C8c=; b=kcm+5FKMffTHIhyX2Qryt9p8/Ymlg8fsnXMX+xUb5Xp/Q0noFfbr0N8BfrMkk9D2/E s7n0oD31WmYrYREvUq2ywqnaQPyhxQSOfiItBAAXkVYC88NwKmuZR2AwO3s2gWZkIW4P FWKyaEE5pifl0Ag+gRdNMLifx3AEgW9qrL9F5BP2UM+Lgl4uJ1xc7gZNpy0qPWtVjkEJ p2La/N3PcA3UfT9xiXu9I2oAg8jwjZN31vMF2QqbPPUNHagxy87CZy3a+AgZXCQdwPun X1p8rYJ1gJUO3jEO3PzOjp0nB4t/Yz3gci4DQzmOdWWx4EBslCQ8hv0GVeTTcl09lu6L gA4g== X-Received: by 10.66.6.40 with SMTP id x8mr76200127pax.97.1386109996761; Tue, 03 Dec 2013 14:33:16 -0800 (PST) Received: from [192.168.211.135] ([66.129.239.12]) by mx.google.com with ESMTPSA id vk17sm150818222pab.5.2013.12.03.14.33.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 03 Dec 2013 14:33:16 -0800 (PST) Message-ID: <529E5C2A.1060905@gmail.com> Date: Tue, 03 Dec 2013 14:33:14 -0800 From: Rajat Jain User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130803 Thunderbird/17.0.8 MIME-Version: 1.0 To: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Bjorn Helgaas , Kenji Kaneshige , Alex Williamson , Yijing Wang , Paul Bolle CC: Rajat Jain , Rajat Jain , Rajat Jain , Guenter Roeck , Guenter Roeck Subject: [PATCH v2 3/4] pciehp: Ensure very fast hotplug events are also processed. Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Today, this is how all the hotplug and unplug events work: Hotplug / Removal needs to be done => Set slot->state (protected by slot->lock) to either POWERON_STATE (for enabling) or POWEROFF_STATE (for disabling). => Submit the work item for pciehp_power_thread() to slot->wq. Problem: There is a problem if the hotplug events can happen fast enough that they do not give SW enough time to add or remove the new devices. => Assume: Event for unplug comes (e.g. surprise removal). But before the pciehp_power_thread() work item was executed, the card was replaced by another card, causing surprise hotplug event. => What goes wrong: => The hot-removal event sets slot->state to POWEROFF_STATE, and schedules the pciehp_power_thread(). => The hot-add event sets slot->state to POWERON_STATE, and schedules the pciehp_power_thread(). => Now the pciehp_power_thread() is scheduled twice, and on both occasions it will find POWERON_STATE and will try to add the devices on the slot, and will fail complaining that the devices already exist. => Why this is a problem: If the device was replaced between the hot removal and hot-add, then we should unload the old driver and reload the new one. This does not happen today. The kernel or the driver is not even aware that the device was replaced. The problem is that the pciehp_power_thread() only looks at the slot->state which would only contain the *latest* state - not the actual event (add / remove) that was the intent of the IRQ handler who submitted the work. What this patch does: => Hotplug events pass on an actual request (for addition or removal) to pciehp_power_thread() which is local to that work item submission. => pciehp_power_thread() does not need to look at slote->state and hence no locks needed in that. => Essentially this results in all the hotplug and unplug events "replayed" by pciehp_power_thread(). Signed-off-by: Rajat Jain Signed-off-by: Guenter Roeck --- v2: Same as v1, dropped the "RFC" from subject, added Guenter's signature drivers/pci/hotplug/pciehp_ctrl.c | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/drivers/pci/hotplug/pciehp_ctrl.c b/drivers/pci/hotplug/pciehp_ctrl.c index 4c2544c..c2cd02a 100644 --- a/drivers/pci/hotplug/pciehp_ctrl.c +++ b/drivers/pci/hotplug/pciehp_ctrl.c @@ -297,6 +297,9 @@ static int remove_board(struct slot *p_slot) struct power_work_info { struct slot *p_slot; struct work_struct work; + unsigned int req; +#define DISABLE_REQ 0 +#define ENABLE_REQ 1 }; /** @@ -312,10 +315,8 @@ static void pciehp_power_thread(struct work_struct *work) container_of(work, struct power_work_info, work); struct slot *p_slot = info->p_slot; - mutex_lock(&p_slot->lock); - switch (p_slot->state) { - case POWEROFF_STATE: - mutex_unlock(&p_slot->lock); + switch (info->req) { + case DISABLE_REQ: ctrl_dbg(p_slot->ctrl, "Disabling domain:bus:device=%04x:%02x:00\n", pci_domain_nr(p_slot->ctrl->pcie->port->subordinate), @@ -323,18 +324,22 @@ static void pciehp_power_thread(struct work_struct *work) pciehp_disable_slot(p_slot); mutex_lock(&p_slot->lock); p_slot->state = STATIC_STATE; - break; - case POWERON_STATE: mutex_unlock(&p_slot->lock); + break; + case ENABLE_REQ: + ctrl_dbg(p_slot->ctrl, + "Enabling domain:bus:device=%04x:%02x:00\n", + pci_domain_nr(p_slot->ctrl->pcie->port->subordinate), + p_slot->ctrl->pcie->port->subordinate->number); if (pciehp_enable_slot(p_slot) && PWR_LED(p_slot->ctrl)) pciehp_green_led_off(p_slot); mutex_lock(&p_slot->lock); p_slot->state = STATIC_STATE; + mutex_unlock(&p_slot->lock); break; default: break; } - mutex_unlock(&p_slot->lock); kfree(info); } @@ -357,9 +362,11 @@ void pciehp_queue_pushbutton_work(struct work_struct *work) switch (p_slot->state) { case BLINKINGOFF_STATE: p_slot->state = POWEROFF_STATE; + info->req = DISABLE_REQ; break; case BLINKINGON_STATE: p_slot->state = POWERON_STATE; + info->req = ENABLE_REQ; break; default: kfree(info); @@ -455,10 +462,13 @@ static void handle_surprise_event(struct slot *p_slot) INIT_WORK(&info->work, pciehp_power_thread); pciehp_get_adapter_status(p_slot, &getstatus); - if (!getstatus) + if (!getstatus) { p_slot->state = POWEROFF_STATE; - else + info->req = DISABLE_REQ; + } else { p_slot->state = POWERON_STATE; + info->req = ENABLE_REQ; + } queue_work(p_slot->wq, &info->work); } @@ -487,6 +497,7 @@ static void handle_link_up_event(struct slot *p_slot) /* Fall through */ case STATIC_STATE: p_slot->state = POWERON_STATE; + info->req = ENABLE_REQ; queue_work(p_slot->wq, &info->work); break; case POWERON_STATE: @@ -500,6 +511,7 @@ static void handle_link_up_event(struct slot *p_slot) "Link Up event queued on slot(%s): currently getting powered off\n", slot_name(p_slot)); p_slot->state = POWERON_STATE; + info->req = ENABLE_REQ; queue_work(p_slot->wq, &info->work); break; default: @@ -534,6 +546,7 @@ static void handle_link_down_event(struct slot *p_slot) /* Fall through */ case STATIC_STATE: p_slot->state = POWEROFF_STATE; + info->req = DISABLE_REQ; queue_work(p_slot->wq, &info->work); break; case POWEROFF_STATE: @@ -547,6 +560,7 @@ static void handle_link_down_event(struct slot *p_slot) "Link Down event queued on slot(%s): currently getting powered on\n", slot_name(p_slot)); p_slot->state = POWEROFF_STATE; + info->req = DISABLE_REQ; queue_work(p_slot->wq, &info->work); break; default: