From patchwork Mon Mar 6 11:25:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 9605777 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 306F76046A for ; Mon, 6 Mar 2017 11:32:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E66952711E for ; Mon, 6 Mar 2017 11:32:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DB4D627317; Mon, 6 Mar 2017 11:32:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A8EC23B23 for ; Mon, 6 Mar 2017 11:32:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753593AbdCFLbX (ORCPT ); Mon, 6 Mar 2017 06:31:23 -0500 Received: from mail-wm0-f65.google.com ([74.125.82.65]:34007 "EHLO mail-wm0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752945AbdCFL0S (ORCPT ); Mon, 6 Mar 2017 06:26:18 -0500 Received: by mail-wm0-f65.google.com with SMTP id u132so5159482wmg.1 for ; Mon, 06 Mar 2017 03:25:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=pmWEqATlXQkoH7DiwzrRZRV35huVHYCG9dsFzBfn4F0=; b=URkvfafVoQF7wzIqcYL1ireIFg92YtVnNENjJ83s4mFRGCZ62oy7uZMWEzi5nd26hG hb/IQ/ujZUEws7yWbXTeO2VTVMDMZKSQ6GjSJadQbo+yq/lES9wV+/4Sv9nNp8VeSg+A iXc8a+157mJFp97gZqm2Xy/c3y4UQiioE+c4W1lrAhrFr1NvGMMnR6pCbxljS80zHRqk 9pHqf/oU3q5HhOudWOFy7jooSdmiQlnov0zObFwTXPir1R+JQA7uxBXEQIbAUKRg+HIG yyusHTsSGrF8RL++Hl3o0PapK0jB3sqQZXf5MMVvujTOjOyHPgi3auXeGD8j6/RhWe7p hVaQ== X-Gm-Message-State: AMke39mhxIwHekJDpBfqYYhGO62lxfVZktA3gNdLsuig4C4FXUzjrWAqI7mbU7oUEuUceA== X-Received: by 10.28.203.197 with SMTP id b188mr14043239wmg.110.1488799542100; Mon, 06 Mar 2017 03:25:42 -0800 (PST) Received: from [192.168.64.116] (bzq-82-81-101-184.red.bezeqint.net. [82.81.101.184]) by smtp.gmail.com with ESMTPSA id 82sm3138743wmg.0.2017.03.06.03.25.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Mar 2017 03:25:41 -0800 (PST) Subject: Re: kernull NULL pointer observed on initiator side after 'nvmetcli clear' on target side To: Yi Zhang , linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org References: <1053522223.31446389.1488721184925.JavaMail.zimbra@redhat.com> Cc: hch@lst.de From: Sagi Grimberg Message-ID: <644fc4ab-df6b-a337-1431-bad881ef56ee@grimberg.me> Date: Mon, 6 Mar 2017 13:25:39 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <1053522223.31446389.1488721184925.JavaMail.zimbra@redhat.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP > Hi experts > > If I offline one CPU on initiator side and nvmetcli clear on target side, it will cause kernel NULL pointer on initiator side, could you help check it, thanks > > Steps to reproduce: > 1. setup nvmet target with null-blk device: > #modprobe nvmet > #modprobe nvmet-rdma > #modprobe null_blk nr_devices=1 > #nvmetcli restore rdma.json > > 2. connect the target on initiator side and offline one cpu: > #modprobe nvme-rdma > #nvme connect-all -t rdma -a 172.31.2.3 -s 1023 > #echo 0 > /sys/devices/system/cpu/cpu1/online > > 3. nvmetcli clear on target side > #nvmetcli clear > > Kernel log: > > [ 125.039340] nvme nvme0: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 172.31.2.3:1023 > [ 125.160587] nvme nvme0: creating 16 I/O queues. > [ 125.602244] nvme nvme0: new ctrl: NQN "testnqn", addr 172.31.2.3:1023 > [ 140.930343] Broke affinity for irq 16 > [ 140.950295] Broke affinity for irq 28 > [ 140.969957] Broke affinity for irq 70 > [ 140.986584] Broke affinity for irq 90 > [ 141.003160] Broke affinity for irq 93 > [ 141.019779] Broke affinity for irq 97 > [ 141.036341] Broke affinity for irq 100 > [ 141.053782] Broke affinity for irq 104 > [ 141.072860] smpboot: CPU 1 is now offline > [ 154.768104] nvme nvme0: reconnecting in 10 seconds > [ 165.349689] BUG: unable to handle kernel NULL pointer dereference at (null) > [ 165.387783] IP: blk_mq_reinit_tagset+0x35/0x80 Looks like blk_mq_reinit_tagset is not aware that tags can go away with cpu hotplug... Does this fix your issue: --- -- -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index e48bc2c72615..9d97bfc4d465 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -295,6 +295,9 @@ int blk_mq_reinit_tagset(struct blk_mq_tag_set *set) for (i = 0; i < set->nr_hw_queues; i++) { struct blk_mq_tags *tags = set->tags[i]; + if (!tags) + continue; + for (j = 0; j < tags->nr_tags; j++) { if (!tags->static_rqs[j]) continue;