From patchwork Mon Mar 6 20:08:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 13162299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3E61C61DA4 for ; Mon, 6 Mar 2023 20:10:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230202AbjCFUKS (ORCPT ); Mon, 6 Mar 2023 15:10:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230222AbjCFUKQ (ORCPT ); Mon, 6 Mar 2023 15:10:16 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 926D94C6DD for ; Mon, 6 Mar 2023 12:09:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678133370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Grvm3WHWtWaUdwJ9Qfi2o/ztDJyGIgGXieKrU2ipubo=; b=U5NHjzv2APehYE7g/x0ZJ8mXQLxVjD0muk6iGgAY7Nrook/Zjpg0ikgdIxfS3fhMi6QKXb AC6YD9GnrvZ1umpG9wz0Cey5Ih+sQStTuoBoVgw2uMgYPyTXMp+Dt04dIo+unA20wJ4STG POuhVoJD+RmRFtrcnTE4aUaXIJbSwUo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-125-DbWfXIfcNhCArXncUlTaqg-1; Mon, 06 Mar 2023 15:09:27 -0500 X-MC-Unique: DbWfXIfcNhCArXncUlTaqg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6617885A588; Mon, 6 Mar 2023 20:09:26 +0000 (UTC) Received: from llong.com (dhcp-17-153.bos.redhat.com [10.18.17.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22F58400DFA1; Mon, 6 Mar 2023 20:09:26 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Will Deacon , Peter Zijlstra , Waiman Long Subject: [PATCH 1/5] cgroup/cpuset: Skip task update if hotplug doesn't affect current cpuset Date: Mon, 6 Mar 2023 15:08:45 -0500 Message-Id: <20230306200849.376804-2-longman@redhat.com> In-Reply-To: <20230306200849.376804-1-longman@redhat.com> References: <20230306200849.376804-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org If a hotplug event doesn't affect the current cpuset, there is no point to call hotplug_update_tasks() or hotplug_update_tasks_legacy(). So just skip it. Signed-off-by: Waiman Long Reviewed-by: Michal Koutný --- kernel/cgroup/cpuset.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 636f1c682ac0..a801abad3bac 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -3508,6 +3508,8 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp) update_tasks: cpus_updated = !cpumask_equal(&new_cpus, cs->effective_cpus); mems_updated = !nodes_equal(new_mems, cs->effective_mems); + if (!cpus_updated && !mems_updated) + goto unlock; /* Hotplug doesn't affect this cpuset */ if (mems_updated) check_insane_mems_config(&new_mems); @@ -3519,6 +3521,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp) hotplug_update_tasks_legacy(cs, &new_cpus, &new_mems, cpus_updated, mems_updated); +unlock: percpu_up_write(&cpuset_rwsem); } From patchwork Mon Mar 6 20:08:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 13162298 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3667CC64EC4 for ; Mon, 6 Mar 2023 20:10:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230314AbjCFUKR (ORCPT ); Mon, 6 Mar 2023 15:10:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230202AbjCFUKQ (ORCPT ); Mon, 6 Mar 2023 15:10:16 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C214C497F3 for ; Mon, 6 Mar 2023 12:09:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678133368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XMpGzFxDVrhKLpAq2mb4oo3DA00+S6OlDRBjGlenDeM=; b=HB6Yo4iRo4xY9Lh/ZKtrxfmY3hVZPCBdeV9clOdmW5pg0o0KopD8zJJ0AHkZCJhrr3T5NR qGd1M2pqsXadddu+DLiPPGbhgyNS8iwuzGgoWkXghs03AHV3rAeN4fm2ic5ebasI0R9zJk g46qOO6nEUQ0cTsE+1rzqDpIkRibPXw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-672-qTnL-CUCNN6x3dhM75SdCw-1; Mon, 06 Mar 2023 15:09:27 -0500 X-MC-Unique: qTnL-CUCNN6x3dhM75SdCw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1C51857A89; Mon, 6 Mar 2023 20:09:26 +0000 (UTC) Received: from llong.com (dhcp-17-153.bos.redhat.com [10.18.17.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6DA934010E7B; Mon, 6 Mar 2023 20:09:26 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Will Deacon , Peter Zijlstra , Waiman Long Subject: [PATCH 2/5] cgroup/cpuset: Include offline CPUs when tasks' cpumasks in top_cpuset are updated Date: Mon, 6 Mar 2023 15:08:46 -0500 Message-Id: <20230306200849.376804-3-longman@redhat.com> In-Reply-To: <20230306200849.376804-1-longman@redhat.com> References: <20230306200849.376804-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Similar to commit 3fb906e7fabb ("group/cpuset: Don't filter offline CPUs in cpuset_cpus_allowed() for top cpuset tasks"), the whole set of possible CPUs including offline ones should be used for setting cpumasks for tasks in the top cpuset when a cpuset partition is modified. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index a801abad3bac..bbf57dcb2f68 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -1209,7 +1209,8 @@ void rebuild_sched_domains(void) * * Iterate through each task of @cs updating its cpus_allowed to the * effective cpuset's. As this function is called with cpuset_rwsem held, - * cpuset membership stays stable. + * cpuset membership stays stable. For top_cpuset, task_cpu_possible_mask() + * is used instead of effective_cpus. */ static void update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus) { @@ -1219,15 +1220,18 @@ static void update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus) css_task_iter_start(&cs->css, 0, &it); while ((task = css_task_iter_next(&it))) { - /* - * Percpu kthreads in top_cpuset are ignored - */ - if (top_cs && (task->flags & PF_KTHREAD) && - kthread_is_per_cpu(task)) - continue; + const struct cpumask *possible_mask = task_cpu_possible_mask(task); - cpumask_and(new_cpus, cs->effective_cpus, - task_cpu_possible_mask(task)); + if (top_cs) { + /* + * Percpu kthreads in top_cpuset are ignored + */ + if ((task->flags & PF_KTHREAD) && kthread_is_per_cpu(task)) + continue; + cpumask_andnot(new_cpus, possible_mask, cs->subparts_cpus); + } else { + cpumask_and(new_cpus, cs->effective_cpus, possible_mask); + } set_cpus_allowed_ptr(task, new_cpus); } css_task_iter_end(&it); From patchwork Mon Mar 6 20:08:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 13162303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 031F8C64EC4 for ; Mon, 6 Mar 2023 20:12:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230011AbjCFUMB (ORCPT ); Mon, 6 Mar 2023 15:12:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51256 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229983AbjCFULt (ORCPT ); Mon, 6 Mar 2023 15:11:49 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C06AC4DE37 for ; Mon, 6 Mar 2023 12:09:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678133371; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZPOLCRRq9h+M6y4H0byc+i6v9jsYVGqazZOEyY69QjU=; b=Y5+9M7ELitWzmrDc/TN2dbrXqgDpq4Yv1uTBgAKxB1l1ETvqX3GgYzUNy//reKguDenYLX OdLJS1SLiY8lnKYjcu6q3q9kWkScISCDfB+8iZSZKfVoXcAACV2Jf4GImRjeWLlC/g6s1N A+55NtkINZeGOulCBcSlnFSz5P8UCBo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-626-WJXboG_qNv6uRFmfnQGtTw-1; Mon, 06 Mar 2023 15:09:27 -0500 X-MC-Unique: WJXboG_qNv6uRFmfnQGtTw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 07FEE3C025B2; Mon, 6 Mar 2023 20:09:27 +0000 (UTC) Received: from llong.com (dhcp-17-153.bos.redhat.com [10.18.17.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id B89C940C83B6; Mon, 6 Mar 2023 20:09:26 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Will Deacon , Peter Zijlstra , Waiman Long Subject: [PATCH 3/5] cgroup/cpuset: Find another usable CPU if none found in current cpuset Date: Mon, 6 Mar 2023 15:08:47 -0500 Message-Id: <20230306200849.376804-4-longman@redhat.com> In-Reply-To: <20230306200849.376804-1-longman@redhat.com> References: <20230306200849.376804-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org On a system with asymmetric CPUs, a restricted task is one that can run only a selected subset of available CPUs. When a CPU goes offline or when "cpuset.cpus" is changed, it is possible that a restricted task may not have any runnable CPUs left in the current cpuset even if there is still some CPUs in effective_cpus. In this case, the restricted task cannot be run at all. There are several ways we may be able to handle this situation. Treating it like empty effective_cpus is probably too disruptive and is unfair to the normal tasks. So it is better to have some special handling for these restricted tasks. One possibility is to move the restricted tasks up the cpuset hierarchy, but it is tricky to do it right. Another solution is to assign other usable CPUs to these tasks. This patch implements the later alternative by finding one usable CPU by walking up the cpuset hierarchy and printing an informational message to let the users know that these restricted tasks are running in a cpuset with no usable CPU. Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 56 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 55 insertions(+), 1 deletion(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index bbf57dcb2f68..aa8225daf1d3 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -1202,6 +1202,38 @@ void rebuild_sched_domains(void) cpus_read_unlock(); } +/* + * Find a usable effective (online) CPU up the cpuset hierarchy and return it. + */ +static int find_usable_cpu(struct cpuset *cs, struct cpumask *new_cpus, + const struct cpumask *possible_mask) +{ + struct cpuset *parent; + unsigned long flags; + int cpu; + + /* + * When offlining cpu, some effective_cpus may not be up to date. + * So check cpu_online_mask to be sure. + */ + parent = parent_cs(cs); + while (parent && + (!cpumask_and(new_cpus, parent->effective_cpus, possible_mask) || + !cpumask_and(new_cpus, new_cpus, cpu_online_mask))) + parent = parent_cs(cs); + + /* Fall back to all possible online cpus, if necessary */ + if (!parent) + cpumask_and(new_cpus, possible_mask, cpu_online_mask); + + /* cpumask_any_distribute() has to be called with preemption disabled */ + local_irq_save(flags); + cpu = cpumask_any_distribute(new_cpus); + local_irq_restore(flags); + + return cpu; +} + /** * update_tasks_cpumask - Update the cpumasks of tasks in the cpuset. * @cs: the cpuset in which each task's cpus_allowed mask needs to be changed @@ -1218,6 +1250,7 @@ static void update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus) struct task_struct *task; bool top_cs = cs == &top_cpuset; + percpu_rwsem_assert_held(&cpuset_rwsem); css_task_iter_start(&cs->css, 0, &it); while ((task = css_task_iter_next(&it))) { const struct cpumask *possible_mask = task_cpu_possible_mask(task); @@ -1232,7 +1265,28 @@ static void update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus) } else { cpumask_and(new_cpus, cs->effective_cpus, possible_mask); } - set_cpus_allowed_ptr(task, new_cpus); + /* + * On systems with assymetric CPUs, it is possible that + * cpumask will become empty or set_cpus_allowed_ptr() will + * return an error even if we still have CPUs in + * effective_cpus. In this case, we find a usable CPU walking + * up the cpuset hierarchy and use that for this particular + * task with an informational message about the change in the + * hope that the users will adjust "cpuset.cpus" accordingly. + */ + if (cpumask_empty(new_cpus) || + set_cpus_allowed_ptr(task, new_cpus)) { + char name[80]; + int cpu; + + cpu = find_usable_cpu(cs, new_cpus, possible_mask); + cpumask_clear(new_cpus); + cpumask_set_cpu(cpu, new_cpus); + WARN_ON_ONCE(set_cpus_allowed_ptr(task, new_cpus)); + cgroup_name(cs->css.cgroup, name, sizeof(name)); + pr_info("cpuset: Restricted task %s(%d) in cpuset %s is forced to run on outside CPU %d\n", + task->comm, task->pid, name, cpu); + } } css_task_iter_end(&it); } From patchwork Mon Mar 6 20:08:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 13162302 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33003C61DA4 for ; Mon, 6 Mar 2023 20:12:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229813AbjCFUMA (ORCPT ); Mon, 6 Mar 2023 15:12:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230394AbjCFULs (ORCPT ); Mon, 6 Mar 2023 15:11:48 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C40A34E5FC for ; Mon, 6 Mar 2023 12:09:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678133373; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8DOL1Xn5BRvEVzQbK0JT1QOBJzRVq//5ngfgMzTUxGY=; b=X7dbByQHviXSfvCs8M/M64K2RV/eMj2xJk30EzAF71WZ5pW5Tm4IbCSPPyV5aGShkfVLdv mezmloEf36pqiC0r4Re3A+xPvzUpu3jYId+20gUx6o492ztvx0WSyyyhKctl+OUxYyFdpI Mz/JukujCQgTUELpCSnAjNbGR5ln2eM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-319-8Nd4UDtXP3uOUYxmG6HB6A-1; Mon, 06 Mar 2023 15:09:28 -0500 X-MC-Unique: 8Nd4UDtXP3uOUYxmG6HB6A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5301D885625; Mon, 6 Mar 2023 20:09:27 +0000 (UTC) Received: from llong.com (dhcp-17-153.bos.redhat.com [10.18.17.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0FE9E400DFA1; Mon, 6 Mar 2023 20:09:27 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Will Deacon , Peter Zijlstra , Waiman Long Subject: [PATCH 4/5] cgroup/cpuset: Add CONFIG_DEBUG_CPUSETS config for cpuset testing Date: Mon, 6 Mar 2023 15:08:48 -0500 Message-Id: <20230306200849.376804-5-longman@redhat.com> In-Reply-To: <20230306200849.376804-1-longman@redhat.com> References: <20230306200849.376804-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Since commit 431c69fac05b ("cpuset: Honour task_cpu_possible_mask() in guarantee_online_cpus()"), task_cpu_possible_mask() is used within the cpuset code. However, it is hard to find a arm64 system that can actually makes task_cpu_possible_mask() return different cpu mask. As a result, it is hard to exercise the correctness of the code that handle exception cases due to task_cpu_possible_mask(). To help in exercising those code paths, we need a way to force task_cpu_possible_mask() to return a different cpu mask. This patch adds a new CONFIG_DEBUG_CPUSETS config option to enable some debug code to do just that. The idea is to create a debugfs file "debug_cpu_possible_mask" that holds the cpumask to be returned by task_cpu_possible_mask() when a task with name started with the special prefix "cstest" is used as the input argument. Userspace testing code is then able to exercise the different code that is affected by task_cpu_possible_mask(). Signed-off-by: Waiman Long --- init/Kconfig | 5 +++ kernel/cgroup/cpuset.c | 76 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 81 insertions(+) diff --git a/init/Kconfig b/init/Kconfig index 18f0bf50c468..2abaa830aff0 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1140,6 +1140,11 @@ config PROC_PID_CPUSET depends on CPUSETS default y +config DEBUG_CPUSETS + bool "Enable cpuset debugging" + depends on CPUSETS && DEBUG_FS + default n + config CGROUP_DEVICE bool "Device controller" help diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index aa8225daf1d3..45051ebb6606 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -220,6 +220,29 @@ static inline bool is_prs_invalid(int prs_state) return prs_state < 0; } +#ifdef CONFIG_DEBUG_CPUSETS +static struct cpumask debug_cpu_possible_mask; + +/* + * Debugging code for testing code involving task_cpu_possible_mask() + */ +static inline const struct cpumask * +__task_cpu_possible_mask(struct task_struct *p) +{ + const struct cpumask *mask = task_cpu_possible_mask(p); + + if (mask != cpu_possible_mask) + return mask; + else if (!strncmp(p->comm, "cstest", 6)) + return &debug_cpu_possible_mask; + else + return cpu_possible_mask; +} + +#undef task_cpu_possible_mask +#define task_cpu_possible_mask(p) __task_cpu_possible_mask(p) +#endif /* CONFIG_DEBUG_CPUSETS */ + /* * Temporary cpumasks for working with partitions that are passed among * functions to avoid memory allocation in inner functions. @@ -4139,3 +4162,56 @@ void cpuset_task_status_allowed(struct seq_file *m, struct task_struct *task) seq_printf(m, "Mems_allowed_list:\t%*pbl\n", nodemask_pr_args(&task->mems_allowed)); } + +#ifdef CONFIG_DEBUG_CPUSETS +#include + +/* + * Add a debugfs file "debug_cpu_possible_mask" that allows user to set + * a debug mask for testing. + */ +static ssize_t read_debug_mask(struct file *file, char __user *user_buf, + size_t count, loff_t *ppos) +{ + char buf[80]; + int len; + + len = snprintf(buf, sizeof(buf) - 1, "%*pbl\n", + cpumask_pr_args(&debug_cpu_possible_mask)); + return simple_read_from_buffer(user_buf, count, ppos, buf, len); +} + +static ssize_t write_debug_mask(struct file *file, const char __user *user_buf, + size_t count, loff_t *ppos) +{ + unsigned int len; + char buf[80]; + int retval = 0; + + len = min(count, sizeof(buf) - 1); + if (copy_from_user(buf, user_buf, len)) + return -EFAULT; + + if (!*buf) + cpumask_clear(&debug_cpu_possible_mask); + else + retval = cpulist_parse(buf, &debug_cpu_possible_mask); + + return (retval < 0) ? retval : count; +} + +static const struct file_operations fops_debug_mask = { + .read = read_debug_mask, + .write = write_debug_mask, + .llseek = default_llseek, +}; + +static int __init create_debug_cpu_possible_mask(void) +{ + cpumask_copy(&debug_cpu_possible_mask, cpu_possible_mask); + debugfs_create_file("debug_cpu_possible_mask", 0600, NULL, NULL, + &fops_debug_mask); + return 0; +} +late_initcall(create_debug_cpu_possible_mask); +#endif /* CONFIG_DEBUG_CPUSETS */ From patchwork Mon Mar 6 20:08:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 13162301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E4B9C64EC4 for ; Mon, 6 Mar 2023 20:11:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229591AbjCFULP (ORCPT ); Mon, 6 Mar 2023 15:11:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230405AbjCFULD (ORCPT ); Mon, 6 Mar 2023 15:11:03 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 924C04B83A for ; Mon, 6 Mar 2023 12:09:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678133370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XQ/fbLiMTxYvcy4Xv+isuWXHLsrugKjziame8MLcKpQ=; b=KX1urgzxFrUEwyTJjbWqDRa+f7t3AYJfbrLIdvTLeoANczZ+yHsy0r1MK0tvckZsEOLBTH KH2fZTeLA2WxbgfLIYn+ojzDNUTD3Cxggl1m3TUxxP+f7Ar25/dW1Qs4DvL/uAL114DwuO qQrHwtykS29l6ghvy/WqlkOImzXGVCA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-317-5KLDa69LNIygZSqWcf9rbw-1; Mon, 06 Mar 2023 15:09:28 -0500 X-MC-Unique: 5KLDa69LNIygZSqWcf9rbw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9DEC93C20EE5; Mon, 6 Mar 2023 20:09:27 +0000 (UTC) Received: from llong.com (dhcp-17-153.bos.redhat.com [10.18.17.153]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5ABB940B40E4; Mon, 6 Mar 2023 20:09:27 +0000 (UTC) From: Waiman Long To: Tejun Heo , Zefan Li , Johannes Weiner , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Will Deacon , Peter Zijlstra , Waiman Long Subject: [PATCH 5/5] cgroup/cpuset: Minor updates to test_cpuset_prs.sh Date: Mon, 6 Mar 2023 15:08:49 -0500 Message-Id: <20230306200849.376804-6-longman@redhat.com> In-Reply-To: <20230306200849.376804-1-longman@redhat.com> References: <20230306200849.376804-1-longman@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org This patch makes the following minor updates to the cpuset partition testing script test_cpuset_prs.sh. - Remove online_cpus function call as it will be called anyway on exit in cleanup. - Make the enabling of sched/verbose debugfs flag conditional on the "-v" verbose option and set DELAY_FACTOR to 2 in this case as cpuset partition operations are likely to be slowed down by enabling that. Signed-off-by: Waiman Long Reviewed-by: Kamalesh Babulal --- .../selftests/cgroup/test_cpuset_prs.sh | 25 +++++++++++-------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/testing/selftests/cgroup/test_cpuset_prs.sh index 75c100de90ff..2b5215cc599f 100755 --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh @@ -15,13 +15,6 @@ skip_test() { [[ $(id -u) -eq 0 ]] || skip_test "Test must be run as root!" -# Set sched verbose flag, if available -if [[ -d /sys/kernel/debug/sched ]] -then - # Used to restore the original setting during cleanup - SCHED_DEBUG=$(cat /sys/kernel/debug/sched/verbose) - echo Y > /sys/kernel/debug/sched/verbose -fi # Get wait_inotify location WAIT_INOTIFY=$(cd $(dirname $0); pwd)/wait_inotify @@ -37,10 +30,14 @@ CPUS=$(lscpu | grep "^CPU(s):" | sed -e "s/.*:[[:space:]]*//") PROG=$1 VERBOSE= DELAY_FACTOR=1 +SCHED_DEBUG= while [[ "$1" = -* ]] do case "$1" in -v) VERBOSE=1 + # Enable sched/verbose can slow thing down + [[ $DELAY_FACTOR -eq 1 ]] && + DELAY_FACTOR=2 break ;; -d) DELAY_FACTOR=$2 @@ -54,6 +51,14 @@ do shift done +# Set sched verbose flag if available when "-v" option is specified +if [[ -n "$VERBOSE" && -d /sys/kernel/debug/sched ]] +then + # Used to restore the original setting during cleanup + SCHED_DEBUG=$(cat /sys/kernel/debug/sched/verbose) + echo Y > /sys/kernel/debug/sched/verbose +fi + cd $CGROUP2 echo +cpuset > cgroup.subtree_control [[ -d test ]] || mkdir test @@ -65,7 +70,8 @@ cleanup() rmdir A1/A2/A3 A1/A2 A1 B1 > /dev/null 2>&1 cd .. rmdir test > /dev/null 2>&1 - echo "$SCHED_DEBUG" > /sys/kernel/debug/sched/verbose + [[ -n "$SCHED_DEBUG" ]] && + echo "$SCHED_DEBUG" > /sys/kernel/debug/sched/verbose } # Pause in ms @@ -571,7 +577,6 @@ run_state_test() echo "Test $TEST[$I] failed result check!" eval echo \"\${$TEST[$I]}\" dump_states - online_cpus exit 1 } @@ -582,7 +587,6 @@ run_state_test() eval echo \"\${$TEST[$I]}\" echo dump_states - online_cpus exit 1 } } @@ -594,7 +598,6 @@ run_state_test() eval echo \"\${$TEST[$I]}\" echo dump_states - online_cpus exit 1 } }