From patchwork Thu Mar 23 11:24:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 9640877 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 590A9602CA for ; Thu, 23 Mar 2017 11:25:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40E4C26E96 for ; Thu, 23 Mar 2017 11:25:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 35ACD28438; Thu, 23 Mar 2017 11:25:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCACC26E96 for ; Thu, 23 Mar 2017 11:25:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932712AbdCWLZB (ORCPT ); Thu, 23 Mar 2017 07:25:01 -0400 Received: from frisell.zx2c4.com ([192.95.5.64]:37275 "EHLO frisell.zx2c4.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751601AbdCWLZA (ORCPT ); Thu, 23 Mar 2017 07:25:00 -0400 Received: by frisell.zx2c4.com (ZX2C4 Mail Server) with ESMTP id 706426c2; Thu, 23 Mar 2017 11:20:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=zx2c4.com; h=from:to:cc :subject:date:message-id:in-reply-to:references; s=mail; bh=oKqm zO7oiPTBtQfAAu6kHdWo/YE=; b=1jmm02U+wPQfMUiJSyAvXRrfDDBZPrL1KS6b Fio5l1QxTEU6xcbfPXeTmrXnnNpNH8w0bgYDtEUXx2NctrsYal8ap5KAfguCu14b 54t/4JHrhmZTQxDIvckeeE6m+gQ/fcy/KhhXDJ0dn7RE8Shmsh0byz+4gNtV0X/d /UjEp/hfAj+feYWSOdjAzk6bRMKUhIlxeln3Zm4UVQPQz4JCehII2AkN5Jgf2Rts AX1fzkOqVf+PzimSwtYKd8WFKPkFatKFZmvQlOeuBNYq60gzPG69a0OLt3s8M+nQ dmDl8gbO0jBHrBdTwQnMsxszL9tLzEiIvK/7Quheq2N3tBRzYQ== Received: by frisell.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id dfc07fed (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Thu, 23 Mar 2017 11:20:26 +0000 (UTC) From: "Jason A. Donenfeld" To: Steffen Klassert , Linux Crypto Mailing List , LKML , Netdev Cc: "Jason A. Donenfeld" Subject: [PATCH] padata: avoid race in reordering Date: Thu, 23 Mar 2017 12:24:43 +0100 Message-Id: <20170323112443.30843-1-Jason@zx2c4.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: References: Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Under extremely heavy uses of padata, crashes occur, and with list debugging turned on, this happens instead: [87487.298728] WARNING: CPU: 1 PID: 882 at lib/list_debug.c:33 __list_add+0xae/0x130 [87487.301868] list_add corruption. prev->next should be next (ffffb17abfc043d0), but was ffff8dba70872c80. (prev=ffff8dba70872b00). [87487.339011] [] dump_stack+0x68/0xa3 [87487.342198] [] ? console_unlock+0x281/0x6d0 [87487.345364] [] __warn+0xff/0x140 [87487.348513] [] warn_slowpath_fmt+0x4a/0x50 [87487.351659] [] __list_add+0xae/0x130 [87487.354772] [] ? _raw_spin_lock+0x64/0x70 [87487.357915] [] padata_reorder+0x1e6/0x420 [87487.361084] [] padata_do_serial+0xa5/0x120 padata_reorder calls list_add_tail with the list to which its adding locked, which seems correct: spin_lock(&squeue->serial.lock); list_add_tail(&padata->list, &squeue->serial.list); spin_unlock(&squeue->serial.lock); This therefore leaves only place where such inconsistency could occur: if padata->list is added at the same time on two different threads. This pdata pointer comes from the function call to padata_get_next(pd), which has in it the following block: next_queue = per_cpu_ptr(pd->pqueue, cpu); padata = NULL; reorder = &next_queue->reorder; if (!list_empty(&reorder->list)) { padata = list_entry(reorder->list.next, struct padata_priv, list); spin_lock(&reorder->lock); list_del_init(&padata->list); atomic_dec(&pd->reorder_objects); spin_unlock(&reorder->lock); pd->processed++; goto out; } out: return padata; I strongly suspect that the problem here is that two threads can race on reorder list. Even though the deletion is locked, call to list_entry is not locked, which means it's feasible that two threads pick up the same padata object and subsequently call list_add_tail on them at the same time. The fix is thus be hoist that lock outside of that block. Signed-off-by: Jason A. Donenfeld Acked-by: Steffen Klassert --- kernel/padata.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/padata.c b/kernel/padata.c index 05316c9f32da..3202aa17492c 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -186,19 +186,20 @@ static struct padata_priv *padata_get_next(struct parallel_data *pd) reorder = &next_queue->reorder; + spin_lock(&reorder->lock); if (!list_empty(&reorder->list)) { padata = list_entry(reorder->list.next, struct padata_priv, list); - spin_lock(&reorder->lock); list_del_init(&padata->list); atomic_dec(&pd->reorder_objects); - spin_unlock(&reorder->lock); pd->processed++; + spin_unlock(&reorder->lock); goto out; } + spin_unlock(&reorder->lock); if (__this_cpu_read(pd->pqueue->cpu_index) == next_queue->cpu_index) { padata = ERR_PTR(-ENODATA);