From patchwork Thu Oct 5 16:39:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julia Cartwright X-Patchwork-Id: 9987559 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0248B602B8 for ; Thu, 5 Oct 2017 16:40:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E771A28CFF for ; Thu, 5 Oct 2017 16:40:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC25128CFA; Thu, 5 Oct 2017 16:40:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8199828D04 for ; Thu, 5 Oct 2017 16:40:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751409AbdJEQkV (ORCPT ); Thu, 5 Oct 2017 12:40:21 -0400 Received: from mx0a-00010702.pphosted.com ([148.163.156.75]:53934 "EHLO mx0b-00010702.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751144AbdJEQkS (ORCPT ); Thu, 5 Oct 2017 12:40:18 -0400 Received: from pps.filterd (m0098780.ppops.net [127.0.0.1]) by mx0a-00010702.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v95GagQL018209; Thu, 5 Oct 2017 11:39:35 -0500 Received: from ni.com (skprod2.natinst.com [130.164.80.23]) by mx0a-00010702.pphosted.com with ESMTP id 2dcnky5kx3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Oct 2017 11:39:35 -0500 Received: from us-aus-exhub1.ni.corp.natinst.com (us-aus-exhub1.ni.corp.natinst.com [130.164.68.41]) by us-aus-skprod2.natinst.com (8.16.0.21/8.16.0.21) with ESMTPS id v95GdYjf010772 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 5 Oct 2017 11:39:34 -0500 Received: from us-aus-exhub1.ni.corp.natinst.com (130.164.68.41) by us-aus-exhub1.ni.corp.natinst.com (130.164.68.41) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Thu, 5 Oct 2017 11:39:34 -0500 Received: from jcartwri.amer.corp.natinst.com (130.164.49.7) by us-aus-exhub1.ni.corp.natinst.com (130.164.68.41) with Microsoft SMTP Server id 15.0.1156.6 via Frontend Transport; Thu, 5 Oct 2017 11:39:34 -0500 Received: by jcartwri.amer.corp.natinst.com (Postfix, from userid 1000) id F2A78302D70; Thu, 5 Oct 2017 11:39:33 -0500 (CDT) Date: Thu, 5 Oct 2017 11:39:33 -0500 From: Julia Cartwright To: Thomas Gleixner CC: Steven Rostedt , Arnaldo Carvalho de Melo , , , , Arnaldo Carvalho de Melo , Clark Williams , Dean Luick , Dennis Dalessandro , Doug Ledford , Kaike Wan , Leon Romanovsky , , Peter Zijlstra , Sebastian Andrzej Siewior , Sebastian Sanchez Subject: Re: [PATCH 1/2] IB/hfi1: Use preempt_{dis,en}able_nort() Message-ID: <20171005163933.GJ647@jcartwri.amer.corp.natinst.com> References: <20171003154920.31566-1-acme@kernel.org> <20171003154920.31566-2-acme@kernel.org> <20171005141744.GC21185@jcartwri.amer.corp.natinst.com> <20171005153759.GG647@jcartwri.amer.corp.natinst.com> <20171005115539.7daafb66@gandalf.local.home> <20171005160535.GI647@jcartwri.amer.corp.natinst.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.0 (2017-09-02) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-10-05_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=30 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=30 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1710050229 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Oct 05, 2017 at 06:16:23PM +0200, Thomas Gleixner wrote: > On Thu, 5 Oct 2017, Julia Cartwright wrote: > > > On Thu, Oct 05, 2017 at 11:55:39AM -0400, Steven Rostedt wrote: > > > On Thu, 5 Oct 2017 10:37:59 -0500 > > > Julia Cartwright wrote: > > > > > > > On Thu, Oct 05, 2017 at 05:27:30PM +0200, Thomas Gleixner wrote: > > > > > On Thu, 5 Oct 2017, Julia Cartwright wrote: > > > > > > On Tue, Oct 03, 2017 at 12:49:19PM -0300, Arnaldo Carvalho de Melo wrote: > > > > > > > - preempt_disable(); > > > > > > > + preempt_disable_nort(); > > > > > > > this_cpu_inc(*sc->buffers_allocated); > > > > > > > > > > > > Have you tried this on RT w/ CONFIG_DEBUG_PREEMPT? I believe that the > > > > > > this_cpu_* operations perform a preemption check, which we'd trip. > > > > > > > > > > Good point. Changing this to migrate_disable() would do the trick. > > > > > > > > Wouldn't we still trip the preempt check even with migration disabled? > > > > In another thread I asked the same question: should the preemption > > > > checks here be converted to migration-checks in RT? > > > > > > Is it a "preemption check"? > > > > Sorry if I was unclear, more precisely: the this_cpu_* family of > > accessors, w/ CONFIG_DEBUG_PREEMPT currently spits out a warning when > > the caller is invoked in a context where preemption is enabled. > > > > The check is shared w/ the smp_processor_id() check, as implemented in > > lib/smp_processor_id.c. It effectively boils down to a check of > > preempt_count() and irqs_disabled(). > > Except that on RT that check cares about the migrate disable state. You can > invoke this_cpu_* and smp_processor_id() in preemptible/interruptible > context because of: > > if (cpumask_equal(current->cpus_ptr, cpumask_of(this_cpu))) > goto out; > > That's true even on mainline. > > But you are right that this check needs some update because > migrate_disable() does not immediately update the allowed cpumask IIRC. Actually, I think it does: migrate_disable() -> p = current; ... migrate_disable_update_cpus_allowed(p) -> p->cpus_ptr = cpumask_of(smp_processor_id()) ... Perhaps it's worth a simple comment update, below. Julia -- 8< -- Subject: [PATCH] kernel: sched: document smp_processor_id/this_cpu safety in migration-disabled context On RT, users of smp_processor_id() and this_cpu_*() per-cpu accessors are considered safe if the caller executes with migration disabled. On !RT, preempt_disable() is sufficient to make this guarantee, however on RT, the lesser migrate_disable() is sufficient. It is not entirely obvious which check in check_preemption_disabled() makes this work, so document it. Signed-off-by: Julia Cartwright --- lib/smp_processor_id.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/smp_processor_id.c b/lib/smp_processor_id.c index de3b2d925473..c8091d9eb1b4 100644 --- a/lib/smp_processor_id.c +++ b/lib/smp_processor_id.c @@ -20,7 +20,11 @@ notrace static unsigned int check_preemption_disabled(const char *what1, /* * Kernel threads bound to a single CPU can safely use - * smp_processor_id(): + * smp_processor_id(). + * + * In addition, threads which are currently executing within + * a migration disabled region can safely use smp_processor_id() and + * this_cpu accessors. */ if (cpumask_equal(current->cpus_ptr, cpumask_of(this_cpu))) goto out;