From patchwork Fri Sep 8 18:57:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mathias Krause X-Patchwork-Id: 9944895 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D790960224 for ; Fri, 8 Sep 2017 18:57:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D53B828779 for ; Fri, 8 Sep 2017 18:57:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA56F2877B; Fri, 8 Sep 2017 18:57:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 51B692877F for ; Fri, 8 Sep 2017 18:57:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756760AbdIHS51 (ORCPT ); Fri, 8 Sep 2017 14:57:27 -0400 Received: from mail-wm0-f66.google.com ([74.125.82.66]:37325 "EHLO mail-wm0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756802AbdIHS5Z (ORCPT ); Fri, 8 Sep 2017 14:57:25 -0400 Received: by mail-wm0-f66.google.com with SMTP id f4so2291419wmh.4 for ; Fri, 08 Sep 2017 11:57:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=blv9dcUvUgrHojhsD00iA6GZ2Z9Cmuypxi5AW0blxho=; b=lxoY4W/NSNBxCpfLBGdlWqYZ0iQi8qyn0qSgZkUfkKeNDUVJfymYqP3QCs2YtUL1Y9 DhK06K63qNjuGI2PQ0D4vlrOlPEuBmbO8OQOI4Vf3TtSGjMtT/V3CVWuIP1i/hD1CW02 PGMkukGDzSGeKhc6qZQLGSstFdwH23FsZ+RaGNfl921Jm7Szjw/HNVhCX+tFhuJgSEtB 6sC6t5MhldCEXsTUh5T9SptZQyk0BtYMqaYzX1oDCo1ID8NDi9BXjjxjBo8yZ4ZXp5B6 xOJdg47EQb8+d64cdVdW11wtxoJ4Lxp9W2MgR2ZUb5Z8pcuFIrUyFSD06R6WlodFuwgM ZuCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=blv9dcUvUgrHojhsD00iA6GZ2Z9Cmuypxi5AW0blxho=; b=MeVvfVB0cRIUbox253Dq2vqmTVBfEzjyRjWrzkKbTv/kNZLbYDl75vtzYwW2R+vX98 4a3gACHWw6ifY9FdckTa7C0u/L0gulDC6tDv7ZGXEW4I/Bo72bTezk5znOc0M5fbsr2L R7PKK8fO23ISMUOynYG1SaNq/6ZqHe35UHX3LxoSdHOCh0c1826a5Ki9C65toeqsvXdW GO46iNY3eHTlQQpCT95NJcX7m9WIbR9SW3E6IWqJMWFo7Lj+gqG6IaH5joYgEizOn+to ot1fzU+TASgD/5LsXACZj8oBr5whyCEABXIFlb7NTcz8QN5dXRJF6DVJBW7zUuSiPUrF XuoA== X-Gm-Message-State: AHPjjUj22mfkfiSHVZ/7eERfpUA1P59+3JSitCp9qb5tiv0fPGa9hk8i DCBWeaFSdS6//s+VTtu9Tls= X-Google-Smtp-Source: AOwi7QDRA8aw1YwddBdMpCgq8zMLAIyrzuRGCtSDexlQLg4xdvgG7vGu/O/JewbvPqXPLuaiNZ2CjQ== X-Received: by 10.28.63.131 with SMTP id m125mr2424312wma.19.1504897044786; Fri, 08 Sep 2017 11:57:24 -0700 (PDT) Received: from jig.fritz.box (ip-90-186-2-4.web.vodafone.de. [90.186.2.4]) by smtp.gmail.com with ESMTPSA id a69sm1458126wma.44.2017.09.08.11.57.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Sep 2017 11:57:24 -0700 (PDT) From: Mathias Krause To: Steffen Klassert , Herbert Xu Cc: linux-crypto@vger.kernel.org, Mathias Krause Subject: [PATCH 3/3] padata: ensure padata_do_serial() runs on the correct CPU Date: Fri, 8 Sep 2017 20:57:11 +0200 Message-Id: <1504897031-27523-4-git-send-email-minipli@googlemail.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1504897031-27523-1-git-send-email-minipli@googlemail.com> References: <1504897031-27523-1-git-send-email-minipli@googlemail.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If the algorithm we're parallelizing is asynchronous we might change CPUs between padata_do_parallel() and padata_do_serial(). However, we don't expect this to happen as we need to enqueue the padata object into the per-cpu reorder queue we took it from, i.e. the same-cpu's parallel queue. Ensure we're not switching CPUs for a given padata object by tracking the CPU within the padata object. If the serial callback gets called on the wrong CPU, defer invoking padata_reorder() via a kernel worker on the CPU we're expected to run on. Signed-off-by: Mathias Krause --- include/linux/padata.h | 2 ++ kernel/padata.c | 20 +++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/include/linux/padata.h b/include/linux/padata.h index 5c0175bbc179..5d13d25da2c8 100644 --- a/include/linux/padata.h +++ b/include/linux/padata.h @@ -37,6 +37,7 @@ * @list: List entry, to attach to the padata lists. * @pd: Pointer to the internal control structure. * @cb_cpu: Callback cpu for serializatioon. + * @cpu: Cpu for parallelization. * @seq_nr: Sequence number of the parallelized data object. * @info: Used to pass information from the parallel to the serial function. * @parallel: Parallel execution function. @@ -46,6 +47,7 @@ struct padata_priv { struct list_head list; struct parallel_data *pd; int cb_cpu; + int cpu; int info; void (*parallel)(struct padata_priv *padata); void (*serial)(struct padata_priv *padata); diff --git a/kernel/padata.c b/kernel/padata.c index b4066147bce4..f262c9a4e70a 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -131,6 +131,7 @@ int padata_do_parallel(struct padata_instance *pinst, padata->cb_cpu = cb_cpu; target_cpu = padata_cpu_hash(pd); + padata->cpu = target_cpu; queue = per_cpu_ptr(pd->pqueue, target_cpu); spin_lock(&queue->parallel.lock); @@ -363,10 +364,21 @@ void padata_do_serial(struct padata_priv *padata) int cpu; struct padata_parallel_queue *pqueue; struct parallel_data *pd; + int reorder_via_wq = 0; pd = padata->pd; cpu = get_cpu(); + + /* We need to run on the same CPU padata_do_parallel(.., padata, ..) + * was called on -- or, at least, enqueue the padata object into the + * correct per-cpu queue. + */ + if (cpu != padata->cpu) { + reorder_via_wq = 1; + cpu = padata->cpu; + } + pqueue = per_cpu_ptr(pd->pqueue, cpu); spin_lock(&pqueue->reorder.lock); @@ -376,7 +388,13 @@ void padata_do_serial(struct padata_priv *padata) put_cpu(); - padata_reorder(pd); + /* If we're running on the wrong CPU, call padata_reorder() via a + * kernel worker. + */ + if (reorder_via_wq) + queue_work_on(cpu, pd->pinst->wq, &pqueue->reorder_work); + else + padata_reorder(pd); } EXPORT_SYMBOL(padata_do_serial);