From patchwork Wed Jul 3 12:59:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11029571 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 537F518E8 for ; Wed, 3 Jul 2019 13:01:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F75B286BD for ; Wed, 3 Jul 2019 13:01:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 33925287CC; Wed, 3 Jul 2019 13:01:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 866722899F for ; Wed, 3 Jul 2019 13:01:57 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiesA-0004Hx-FL; Wed, 03 Jul 2019 13:00:30 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hies9-0004Hs-IO for xen-devel@lists.xenproject.org; Wed, 03 Jul 2019 13:00:29 +0000 X-Inumbo-ID: 86731e47-9d92-11e9-8980-bc764e045a96 Received: from m4a0040g.houston.softwaregrp.com (unknown [15.124.2.86]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 86731e47-9d92-11e9-8980-bc764e045a96; Wed, 03 Jul 2019 13:00:28 +0000 (UTC) Received: FROM m4a0040g.houston.softwaregrp.com (15.120.17.146) BY m4a0040g.houston.softwaregrp.com WITH ESMTP; Wed, 3 Jul 2019 13:00:05 +0000 Received: from M4W0335.microfocus.com (2002:f78:1193::f78:1193) by M4W0334.microfocus.com (2002:f78:1192::f78:1192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Wed, 3 Jul 2019 12:59:38 +0000 Received: from NAM01-BN3-obe.outbound.protection.outlook.com (15.124.8.13) by M4W0335.microfocus.com (15.120.17.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Wed, 3 Jul 2019 12:59:38 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com (10.255.139.95) by BY5PR18MB3204.namprd18.prod.outlook.com (10.255.137.97) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2032.17; Wed, 3 Jul 2019 12:59:36 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc]) by BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc%3]) with mapi id 15.20.2052.010; Wed, 3 Jul 2019 12:59:36 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH v2 1/5] x86/cpuidle: switch to uniform meaning of "max_cstate=" Thread-Index: AQHVMZ8qQ6Jc8o6GrEKsOZ2R6D7BYQ== Date: Wed, 3 Jul 2019 12:59:36 +0000 Message-ID: <0f8bb5dd-718c-7226-db4c-b57ee7089735@suse.com> References: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> In-Reply-To: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB6PR01CA0034.eurprd01.prod.exchangelabs.com (2603:10a6:6:45::47) To BY5PR18MB3394.namprd18.prod.outlook.com (2603:10b6:a03:194::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 574600c8-f354-4d27-229c-08d6ffb64cf8 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BY5PR18MB3204; x-ms-traffictypediagnostic: BY5PR18MB3204: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1388; x-forefront-prvs: 00872B689F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(366004)(39860400002)(376002)(136003)(346002)(396003)(54094003)(199004)(189003)(2906002)(66446008)(64756008)(66946007)(14454004)(66476007)(66556008)(53936002)(73956011)(71190400001)(71200400001)(6486002)(5640700003)(72206003)(476003)(66066001)(305945005)(6436002)(31696002)(2616005)(86362001)(4326008)(5660300002)(316002)(54906003)(6512007)(7736002)(36756003)(31686004)(2351001)(25786009)(256004)(8936002)(14444005)(81166006)(386003)(26005)(478600001)(3846002)(186003)(6116002)(68736007)(52116002)(8676002)(76176011)(11346002)(81156014)(6916009)(102836004)(80792005)(486006)(6506007)(2501003)(446003)(99286004); DIR:OUT; SFP:1102; SCL:1; SRVR:BY5PR18MB3204; H:BY5PR18MB3394.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: aa07txHGTrzQNydFSpyKrUU/AvcAMyoTt4V18WBXNR7oRRF5+r+3i1ecI4Ln5QynMDSDN9rmvnwsKSByqryu1WPuMzgRZNdH1fn1enR18+tNxTUJPKr4pFV6rzxJHxpCZg+biPn+5ENdt6BRI/DR+XF2+4QHOdgE0d6vR5lgMX0sYSulLIShLdy0/vdC3vfPoUE66yh8BRioraZ/fuVq/CXMlUzJZmDt4PFJpyurZNNjYRbYXSeAhaQcjnJFovGb7ZYempJPPPxofeiIyz8PPxNaxk+k59eHLxmgLZlWuGpWHan8bjdmXqvcOo9yQ30MWnamUJdj6h7OX64k4+T4YAlbLlu65VSSfsW+sRuxFfoVirS9/6jItQGkt/bkbSbdloOxnBhAcMWAGtkM1GxBbNGnSetPnfwzq98HV3cEEaQ= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 574600c8-f354-4d27-229c-08d6ffb64cf8 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jul 2019 12:59:36.2413 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR18MB3204 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH v2 1/5] x86/cpuidle: switch to uniform meaning of "max_cstate=" X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Ian Jackson , Wei Liu , =?utf-8?q?Roger?= =?utf-8?q?_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP While the MWAIT idle driver already takes it to mean an actual C state, the ACPI idle driver so far used it as a list index. The list index, however, is an implementation detail of Xen and affected by firmware settings (i.e. not necessarily uniform for a particular system). While touching this code also avoid invoking menu_get_trace_data() when tracing is not active. For consistency do this also for the MWAIT driver. Note that I'm intentionally not adding any sorting logic to set_cx(): Before and after this patch we assume entries to arrive in order, so this would be an orthogonal change. Take the opportunity and add minimal documentation for the command line option. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau MonnĂ© Acked-by: Andrew Cooper --- v2: Adjust xenpm output wording a little. Explicitly log "unlimited". --- TBD: I wonder if we really need struct acpi_processor_cx's idx field anymore. It's used in a number of (questionable) places ... --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1376,6 +1376,8 @@ This option is ignored in **pv-shim** mo ### max_cstate (x86) > `= ` +Specify the deepest C-state CPUs are permitted to be placed in. + ### max_gsi_irqs (x86) > `= ` --- a/tools/misc/xenpm.c +++ b/tools/misc/xenpm.c @@ -64,7 +64,7 @@ void show_help(void) " set-sched-smt enable|disable enable/disable scheduler smt power saving\n" " set-vcpu-migration-delay set scheduler vcpu migration delay in us\n" " get-vcpu-migration-delay get scheduler vcpu migration delay\n" - " set-max-cstate set the C-State limitation ( >= 0)\n" + " set-max-cstate |'unlimited' set the C-State limitation ( >= 0)\n" " start [seconds] start collect Cx/Px statistics,\n" " output after CTRL-C or SIGINT or several seconds.\n" " enable-turbo-mode [cpuid] enable Turbo Mode for processors that support it.\n" @@ -194,7 +194,11 @@ static int show_max_cstate(xc_interface if ( (ret = xc_get_cpuidle_max_cstate(xc_handle, &value)) ) return ret; - printf("Max possible C-state: C%d\n\n", value); + if ( value < XEN_SYSCTL_CX_UNLIMITED ) + printf("Max possible C-state: C%"PRIu32"\n\n", value); + else + printf("All C-states allowed\n\n"); + return 0; } @@ -1117,18 +1121,24 @@ void get_vcpu_migration_delay_func(int a void set_max_cstate_func(int argc, char *argv[]) { int value; + char buf[12]; - if ( argc != 1 || sscanf(argv[0], "%d", &value) != 1 || value < 0 ) + if ( argc != 1 || + (sscanf(argv[0], "%d", &value) == 1 + ? value < 0 + : (value = XEN_SYSCTL_CX_UNLIMITED, strcmp(argv[0], "unlimited"))) ) { - fprintf(stderr, "Missing or invalid argument(s)\n"); + fprintf(stderr, "Missing, excess, or invalid argument(s)\n"); exit(EINVAL); } + snprintf(buf, ARRAY_SIZE(buf), "C%d", value); + if ( !xc_set_cpuidle_max_cstate(xc_handle, (uint32_t)value) ) - printf("set max_cstate to C%d succeeded\n", value); + printf("max C-state set to %s\n", value >= 0 ? buf : argv[0]); else - fprintf(stderr, "set max_cstate to C%d failed (%d - %s)\n", - value, errno, strerror(errno)); + fprintf(stderr, "Failed to set max C-state to %s (%d - %s)\n", + value >= 0 ? buf : argv[0], errno, strerror(errno)); } void enable_turbo_mode(int argc, char *argv[]) --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -103,7 +103,7 @@ bool lapic_timer_init(void) } void (*__read_mostly pm_idle_save)(void); -unsigned int max_cstate __read_mostly = ACPI_PROCESSOR_MAX_POWER - 1; +unsigned int max_cstate __read_mostly = UINT_MAX; integer_param("max_cstate", max_cstate); static bool __read_mostly local_apic_timer_c2_ok; boolean_param("lapic_timer_c2_ok", local_apic_timer_c2_ok); @@ -344,7 +344,10 @@ static void dump_cx(unsigned char key) unsigned int cpu; printk("'%c' pressed -> printing ACPI Cx structures\n", key); - printk("max cstate: C%u\n", max_cstate); + if ( max_cstate < UINT_MAX ) + printk("max state: C%u\n", max_cstate); + else + printk("max state: unlimited\n"); for_each_present_cpu ( cpu ) { struct acpi_processor_power *power = processor_powers[cpu]; @@ -582,13 +585,19 @@ static void acpi_processor_idle(void) if ( max_cstate > 0 && power && !sched_has_urgent_vcpu() && (next_state = cpuidle_current_governor->select(power)) > 0 ) { - cx = &power->states[next_state]; - if ( cx->type == ACPI_STATE_C3 && power->flags.bm_check && - acpi_idle_bm_check() ) - cx = power->safe_state; - if ( cx->idx > max_cstate ) - cx = &power->states[max_cstate]; - menu_get_trace_data(&exp, &pred); + do { + cx = &power->states[next_state]; + } while ( cx->type > max_cstate && --next_state ); + if ( next_state ) + { + if ( cx->type == ACPI_STATE_C3 && power->flags.bm_check && + acpi_idle_bm_check() ) + cx = power->safe_state; + if ( tb_init_done ) + menu_get_trace_data(&exp, &pred); + } + else + cx = NULL; } if ( !cx ) { @@ -1396,12 +1405,12 @@ int pmstat_reset_cx_stat(uint32_t cpuid) void cpuidle_disable_deep_cstate(void) { - if ( max_cstate > 1 ) + if ( max_cstate > ACPI_STATE_C1 ) { if ( local_apic_timer_c2_ok ) - max_cstate = 2; + max_cstate = ACPI_STATE_C2; else - max_cstate = 1; + max_cstate = ACPI_STATE_C1; } hpet_disable_legacy_broadcast(); @@ -1409,7 +1418,8 @@ void cpuidle_disable_deep_cstate(void) bool cpuidle_using_deep_cstate(void) { - return xen_cpuidle && max_cstate > (local_apic_timer_c2_ok ? 2 : 1); + return xen_cpuidle && max_cstate > (local_apic_timer_c2_ok ? ACPI_STATE_C2 + : ACPI_STATE_C1); } static int cpu_callback( --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -731,7 +731,8 @@ static void mwait_idle(void) } while (cx->type > max_cstate && --next_state); if (!next_state) cx = NULL; - menu_get_trace_data(&exp, &pred); + else if (tb_init_done) + menu_get_trace_data(&exp, &pred); } if (!cx) { if (pm_idle_save) --- a/xen/arch/x86/time.c +++ b/xen/arch/x86/time.c @@ -2268,7 +2268,7 @@ static void dump_softtsc(unsigned char k else if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC ) ) { printk("TSC has constant rate, "); - if (max_cstate <= 2 && tsc_max_warp == 0) + if ( max_cstate <= ACPI_STATE_C2 && tsc_max_warp == 0 ) printk("no deep Cstates, passed warp test, deemed reliable, "); else printk("deep Cstates possible, so not reliable, "); --- a/xen/drivers/acpi/pmstat.c +++ b/xen/drivers/acpi/pmstat.c @@ -451,6 +451,7 @@ int do_pm_op(struct xen_sysctl_pm_op *op case XEN_SYSCTL_pm_op_get_max_cstate: { + BUILD_BUG_ON(XEN_SYSCTL_CX_UNLIMITED != UINT_MAX); op->u.get_max_cstate = acpi_get_cstate_limit(); break; } --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -370,6 +370,7 @@ struct xen_sysctl_pm_op { struct xen_set_cpufreq_para set_para; uint64_aligned_t get_avgfreq; uint32_t set_sched_opt_smt; +#define XEN_SYSCTL_CX_UNLIMITED 0xffffffff uint32_t get_max_cstate; uint32_t set_max_cstate; } u; From patchwork Wed Jul 3 13:00:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11029573 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 00BC9138B for ; Wed, 3 Jul 2019 13:04:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4E2F289DD for ; Wed, 3 Jul 2019 13:04:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D8D24289F7; Wed, 3 Jul 2019 13:04:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 79102289DD for ; Wed, 3 Jul 2019 13:04:33 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hieuC-0004Pq-UZ; Wed, 03 Jul 2019 13:02:36 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hieuB-0004Pj-NA for xen-devel@lists.xenproject.org; Wed, 03 Jul 2019 13:02:35 +0000 X-Inumbo-ID: d2b08ba7-9d92-11e9-8980-bc764e045a96 Received: from m9a0001g.houston.softwaregrp.com (unknown [15.124.64.66]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id d2b08ba7-9d92-11e9-8980-bc764e045a96; Wed, 03 Jul 2019 13:02:34 +0000 (UTC) Received: FROM m9a0001g.houston.softwaregrp.com (15.121.0.191) BY m9a0001g.houston.softwaregrp.com WITH ESMTP; Wed, 3 Jul 2019 13:02:33 +0000 Received: from M4W0334.microfocus.com (2002:f78:1192::f78:1192) by M9W0068.microfocus.com (2002:f79:bf::f79:bf) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Wed, 3 Jul 2019 13:00:14 +0000 Received: from NAM04-SN1-obe.outbound.protection.outlook.com (15.124.8.11) by M4W0334.microfocus.com (15.120.17.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Wed, 3 Jul 2019 13:00:12 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com (10.255.139.95) by BY5PR18MB3282.namprd18.prod.outlook.com (10.255.138.79) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2032.20; Wed, 3 Jul 2019 13:00:11 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc]) by BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc%3]) with mapi id 15.20.2052.010; Wed, 3 Jul 2019 13:00:11 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH v2 2/5] x86/cpuidle: really use C1 for "urgent" CPUs Thread-Index: AQHVMZ8/Ypvn1rOb/0ed9fyvGVPQHQ== Date: Wed, 3 Jul 2019 13:00:10 +0000 Message-ID: References: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> In-Reply-To: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB7PR08CA0011.eurprd08.prod.outlook.com (2603:10a6:5:16::24) To BY5PR18MB3394.namprd18.prod.outlook.com (2603:10b6:a03:194::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8f7a6c57-1d91-4897-3d12-08d6ffb661ae x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BY5PR18MB3282; x-ms-traffictypediagnostic: BY5PR18MB3282: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-forefront-prvs: 00872B689F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(39860400002)(136003)(376002)(396003)(346002)(366004)(189003)(199004)(6436002)(6506007)(4326008)(66946007)(316002)(5640700003)(66476007)(54906003)(72206003)(36756003)(5660300002)(66066001)(53936002)(486006)(478600001)(6512007)(6916009)(6486002)(2906002)(25786009)(68736007)(31686004)(52116002)(2616005)(11346002)(14454004)(305945005)(64756008)(3846002)(73956011)(80792005)(476003)(66446008)(66556008)(26005)(256004)(71190400001)(71200400001)(2501003)(446003)(7736002)(102836004)(186003)(31696002)(6116002)(99286004)(8936002)(76176011)(81156014)(86362001)(2351001)(81166006)(8676002)(386003); DIR:OUT; SFP:1102; SCL:1; SRVR:BY5PR18MB3282; H:BY5PR18MB3394.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 3mh4tKKXKRpajz29Q1oBMeOSmNI0SlkiljN3q2Wa7efPRm4FHNM/aak5X676pWceeuRMyDbjY3OQINEkp1dRKSxOLWENdaS7tMaHXTjqIwC+/C5xBZZMmlshkusxe++TuetkEhhfpy2VUo5r5lZO7LIdCgcsf9VGuunjWK8KuZX8xfq9D15kjNr9rAMWvi7VCpIwfYLwWAVcUqHKoNashEPbPzU5rOTI3SwfljQt3/fYgP7//Lk0OmvEf1+9Mq21RDTjK/pah+SpZWeCIpy7646Xdt+ItbREknLHHnisisNOP+lcnR/xk6Sh8xIIeeI88UM98dDmiFwBgpQa2umM/ttq6YjR03TLDWkJv5yxWaBCVuj2lSmwqpcnEXiatgicIPul2zfxFt8RRhNEdXdfUxphvDFKxHZ89Hj4wIxv7Ow= Content-ID: <71BF2B014811CF42A44EC51FF9168F39@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 8f7a6c57-1d91-4897-3d12-08d6ffb661ae X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jul 2019 13:00:10.9772 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR18MB3282 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH v2 2/5] x86/cpuidle: really use C1 for "urgent" CPUs X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP For one on recent AMD CPUs entering C1 (if available at all) requires use of MWAIT, while HLT (i.e. default_idle()) would put the processor into as deep as CC6. And then even on other vendors' CPUs we should avoid entering default_idle() when the intended state can be reached by using the active idle driver's facilities. Signed-off-by: Jan Beulich Reviewed-by: Andrew Cooper --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -582,12 +582,15 @@ static void acpi_processor_idle(void) u32 exp = 0, pred = 0; u32 irq_traced[4] = { 0 }; - if ( max_cstate > 0 && power && !sched_has_urgent_vcpu() && + if ( max_cstate > 0 && power && (next_state = cpuidle_current_governor->select(power)) > 0 ) { + unsigned int max_state = sched_has_urgent_vcpu() ? ACPI_STATE_C1 + : max_cstate; + do { cx = &power->states[next_state]; - } while ( cx->type > max_cstate && --next_state ); + } while ( cx->type > max_state && --next_state ); if ( next_state ) { if ( cx->type == ACPI_STATE_C3 && power->flags.bm_check && --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -724,11 +724,14 @@ static void mwait_idle(void) u64 before, after; u32 exp = 0, pred = 0, irq_traced[4] = { 0 }; - if (max_cstate > 0 && power && !sched_has_urgent_vcpu() && + if (max_cstate > 0 && power && (next_state = cpuidle_current_governor->select(power)) > 0) { + unsigned int max_state = sched_has_urgent_vcpu() ? ACPI_STATE_C1 + : max_cstate; + do { cx = &power->states[next_state]; - } while (cx->type > max_cstate && --next_state); + } while (cx->type > max_state && --next_state); if (!next_state) cx = NULL; else if (tb_init_done) From patchwork Wed Jul 3 13:01:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11029579 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58C2B14F6 for ; Wed, 3 Jul 2019 13:08:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4893C2872E for ; Wed, 3 Jul 2019 13:08:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3CBE928682; Wed, 3 Jul 2019 13:08:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B00AB285E1 for ; Wed, 3 Jul 2019 13:08:06 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiexl-0004f9-1p; Wed, 03 Jul 2019 13:06:17 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiexj-0004ew-UB for xen-devel@lists.xenproject.org; Wed, 03 Jul 2019 13:06:15 +0000 X-Inumbo-ID: 548f691e-9d93-11e9-9437-771ce69894bc Received: from m9a0001g.houston.softwaregrp.com (unknown [15.124.64.66]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 548f691e-9d93-11e9-9437-771ce69894bc; Wed, 03 Jul 2019 13:06:12 +0000 (UTC) Received: FROM m9a0001g.houston.softwaregrp.com (15.121.0.190) BY m9a0001g.houston.softwaregrp.com WITH ESMTP; Wed, 3 Jul 2019 13:06:11 +0000 Received: from M4W0334.microfocus.com (2002:f78:1192::f78:1192) by M9W0067.microfocus.com (2002:f79:be::f79:be) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Wed, 3 Jul 2019 13:01:49 +0000 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (15.124.8.12) by M4W0334.microfocus.com (15.120.17.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Wed, 3 Jul 2019 13:01:49 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com (10.255.139.95) by BY5PR18MB3393.namprd18.prod.outlook.com (10.255.136.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2032.20; Wed, 3 Jul 2019 13:01:48 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc]) by BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc%3]) with mapi id 15.20.2052.010; Wed, 3 Jul 2019 13:01:48 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH v2 3/5] x86/AMD: make C-state handling independent of Dom0 Thread-Index: AQHVMZ95Pg5KTi9KKk6xsZ3jj2BciA== Date: Wed, 3 Jul 2019 13:01:48 +0000 Message-ID: <7e5fba10-b957-81ed-a1d8-d4c7447f0d51@suse.com> References: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> In-Reply-To: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB6P191CA0022.EURP191.PROD.OUTLOOK.COM (2603:10a6:6:28::32) To BY5PR18MB3394.namprd18.prod.outlook.com (2603:10b6:a03:194::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5bca7c7d-8d7b-4591-dc1d-08d6ffb69bc3 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BY5PR18MB3393; x-ms-traffictypediagnostic: BY5PR18MB3393: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5797; x-forefront-prvs: 00872B689F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(396003)(136003)(376002)(366004)(346002)(39860400002)(189003)(199004)(6512007)(316002)(476003)(36756003)(71190400001)(76176011)(54906003)(53936002)(4326008)(2501003)(80792005)(446003)(11346002)(31686004)(14444005)(256004)(26005)(5660300002)(99286004)(72206003)(2616005)(71200400001)(14454004)(486006)(186003)(305945005)(52116002)(68736007)(25786009)(2906002)(64756008)(73956011)(3846002)(66946007)(102836004)(66476007)(66066001)(6486002)(8676002)(81166006)(6916009)(386003)(81156014)(66556008)(6506007)(478600001)(31696002)(6436002)(66446008)(8936002)(7736002)(5640700003)(2351001)(86362001)(6116002); DIR:OUT; SFP:1102; SCL:1; SRVR:BY5PR18MB3393; H:BY5PR18MB3394.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: /klbCjr5MVx+2x9rCTR6Fwd03Y7zJ5HF9ztumj1Vf90uhaD1PpjExgzBsB5OFFlhoEO7SKwkvJCuaga3an6+Q+6U5h5iK5A6pabWgdMChBtt75LqBXugpY4Ybp/ftBMf1xElGdj0zq5hOuLVWmPrC0rFFy1aVxUpkITHDUXyNFEdrocR8c4ArXU/JkAN8nQHvae512D8ruBfgl/sD3m4UpKZvfI6RS9+3B5s9WW19ao2cW/Rll9qti2devN7Y0SYJs7hCUwiWTm7OnCaEYttz2dtA06Hi/p9+omKzWzL1969rtuhO/a36qccQe6IVBj6IZnuJvVQ7uUrYthOTvrOSlwGyb+t4Yafcidc9juBA+sQtCrPXUKGj/oOH224XR81NOAaBTBhyznbPuTaDtzoonE+WrgHR6YNKchEuZ7qSv0= Content-ID: <67CBFD6AE5019A419456A35A66782C37@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 5bca7c7d-8d7b-4591-dc1d-08d6ffb69bc3 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jul 2019 13:01:48.3917 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR18MB3393 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH v2 3/5] x86/AMD: make C-state handling independent of Dom0 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Brian Woods , Wei Liu , =?utf-8?q?Roger_Pa?= =?utf-8?q?u_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP At least for more recent CPUs, following what BKDG / PPR suggest for the BIOS to surface via ACPI we can make ourselves independent of Dom0 uploading respective data. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau MonnĂ© Acked-by: Andrew Cooper --- v2: Handle Hygon Fam18. Set local_apic_timer_c2_ok (for Fam17 and Hygon Fam18 only for now). --- TBD: We may want to verify that HLT indeed is configured to enter CC6. TBD: Brian's series specifies .target_residency as 1000 for CC6; may want to do so here as well. Question then is whether this value is also suitable for the older families. TBD: I guess we could extend this to families older then Fam15 as well. --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -110,6 +110,8 @@ boolean_param("lapic_timer_c2_ok", local struct acpi_processor_power *__read_mostly processor_powers[NR_CPUS]; +static int8_t __read_mostly vendor_override; + struct hw_residencies { uint64_t mc0; @@ -1214,6 +1216,9 @@ long set_cx_pminfo(uint32_t acpi_id, str if ( pm_idle_save && pm_idle != acpi_processor_idle ) return 0; + if ( vendor_override > 0 ) + return 0; + print_cx_pminfo(acpi_id, power); cpu_id = get_cpu_id(acpi_id); @@ -1286,6 +1291,103 @@ long set_cx_pminfo(uint32_t acpi_id, str return 0; } +static void amd_cpuidle_init(struct acpi_processor_power *power) +{ + unsigned int i, nr = 0; + const struct cpuinfo_x86 *c = ¤t_cpu_data; + const unsigned int ecx_req = CPUID5_ECX_EXTENSIONS_SUPPORTED | + CPUID5_ECX_INTERRUPT_BREAK; + const struct acpi_processor_cx *cx = NULL; + static const struct acpi_processor_cx fam17[] = { + { + .type = ACPI_STATE_C1, + .entry_method = ACPI_CSTATE_EM_FFH, + .address = 0, + .latency = 1, + }, + { + .type = ACPI_STATE_C2, + .entry_method = ACPI_CSTATE_EM_HALT, + .latency = 400, + }, + }; + + if ( pm_idle_save && pm_idle != acpi_processor_idle ) + return; + + if ( vendor_override < 0 ) + return; + + switch ( c->x86 ) + { + case 0x18: + if ( boot_cpu_data.x86_vendor != X86_VENDOR_HYGON ) + { + default: + vendor_override = -1; + return; + } + /* fall through */ + case 0x17: + if ( cpu_has_monitor && c->cpuid_level >= CPUID_MWAIT_LEAF && + (cpuid_ecx(CPUID_MWAIT_LEAF) & ecx_req) == ecx_req ) + { + cx = fam17; + nr = ARRAY_SIZE(fam17); + local_apic_timer_c2_ok = true; + break; + } + /* fall through */ + case 0x15: + case 0x16: + cx = &fam17[1]; + nr = ARRAY_SIZE(fam17) - 1; + break; + } + + power->flags.has_cst = true; + + for ( i = 0; i < nr; ++i ) + { + if ( cx[i].type > max_cstate ) + break; + power->states[i + 1] = cx[i]; + power->states[i + 1].idx = i + 1; + power->states[i + 1].target_residency = cx[i].latency * latency_factor; + } + + if ( i ) + { + power->count = i + 1; + power->safe_state = &power->states[i]; + + if ( !vendor_override ) + { + if ( !boot_cpu_has(X86_FEATURE_ARAT) ) + hpet_broadcast_init(); + + if ( !lapic_timer_init() ) + { + vendor_override = -1; + cpuidle_init_cpu(power->cpu); + return; + } + + if ( !pm_idle_save ) + { + pm_idle_save = pm_idle; + pm_idle = acpi_processor_idle; + } + + dead_idle = acpi_dead_idle; + + vendor_override = 1; + } + } + else + vendor_override = -1; +} + uint32_t pmstat_get_cx_nr(uint32_t cpuid) { return processor_powers[cpuid] ? processor_powers[cpuid]->count : 0; @@ -1432,8 +1534,8 @@ static int cpu_callback( int rc = 0; /* - * Only hook on CPU_UP_PREPARE because a dead cpu may utilize the info - * to enter deep C-state. + * Only hook on CPU_UP_PREPARE / CPU_ONLINE because a dead cpu may utilize + * the info to enter deep C-state. */ switch ( action ) { @@ -1442,6 +1544,13 @@ static int cpu_callback( if ( !rc && cpuidle_current_governor->enable ) rc = cpuidle_current_governor->enable(processor_powers[cpu]); break; + + case CPU_ONLINE: + if ( (boot_cpu_data.x86_vendor & + (X86_VENDOR_AMD | X86_VENDOR_HYGON)) && + processor_powers[cpu] ) + amd_cpuidle_init(processor_powers[cpu]); + break; } return !rc ? NOTIFY_DONE : notifier_from_errno(rc); From patchwork Wed Jul 3 13:03:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11029581 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B832138B for ; Wed, 3 Jul 2019 13:08:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C21828678 for ; Wed, 3 Jul 2019 13:08:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 806E9286E0; Wed, 3 Jul 2019 13:08:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0337928678 for ; Wed, 3 Jul 2019 13:08:20 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hieyR-0004lF-Cn; Wed, 03 Jul 2019 13:06:59 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hieyQ-0004kD-CA for xen-devel@lists.xenproject.org; Wed, 03 Jul 2019 13:06:58 +0000 X-Inumbo-ID: 6f23fc58-9d93-11e9-8980-bc764e045a96 Received: from m9a0001g.houston.softwaregrp.com (unknown [15.124.64.66]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 6f23fc58-9d93-11e9-8980-bc764e045a96; Wed, 03 Jul 2019 13:06:56 +0000 (UTC) Received: FROM m9a0001g.houston.softwaregrp.com (15.121.0.190) BY m9a0001g.houston.softwaregrp.com WITH ESMTP; Wed, 3 Jul 2019 13:06:55 +0000 Received: from M9W0068.microfocus.com (2002:f79:bf::f79:bf) by M9W0067.microfocus.com (2002:f79:be::f79:be) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Wed, 3 Jul 2019 13:03:03 +0000 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (15.124.72.13) by M9W0068.microfocus.com (15.121.0.191) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Wed, 3 Jul 2019 13:03:03 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com (10.255.139.95) by BY5PR18MB3218.namprd18.prod.outlook.com (10.255.137.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2052.18; Wed, 3 Jul 2019 13:03:02 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc]) by BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc%3]) with mapi id 15.20.2052.010; Wed, 3 Jul 2019 13:03:02 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH v2 4/5] x86: allow limiting the max C-state sub-state Thread-Index: AQHVMZ+l48br4ffcsk28PZ/9fMg8ZA== Date: Wed, 3 Jul 2019 13:03:02 +0000 Message-ID: References: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> In-Reply-To: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB6P192CA0014.EURP192.PROD.OUTLOOK.COM (2603:10a6:4:b8::24) To BY5PR18MB3394.namprd18.prod.outlook.com (2603:10b6:a03:194::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0bd5c976-5457-4eea-d6e0-08d6ffb6c7f8 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BY5PR18MB3218; x-ms-traffictypediagnostic: BY5PR18MB3218: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:901; x-forefront-prvs: 00872B689F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(396003)(366004)(376002)(39860400002)(136003)(346002)(199004)(189003)(6486002)(31686004)(11346002)(476003)(2616005)(25786009)(4326008)(486006)(446003)(6506007)(386003)(6436002)(6916009)(26005)(14454004)(102836004)(7736002)(186003)(305945005)(99286004)(36756003)(64756008)(66556008)(66476007)(66446008)(2351001)(8936002)(8676002)(53936002)(81166006)(81156014)(2906002)(73956011)(66946007)(256004)(5660300002)(76176011)(5640700003)(31696002)(3846002)(6116002)(14444005)(54906003)(52116002)(6512007)(80792005)(86362001)(72206003)(66066001)(68736007)(2501003)(478600001)(316002)(71190400001)(71200400001); DIR:OUT; SFP:1102; SCL:1; SRVR:BY5PR18MB3218; H:BY5PR18MB3394.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 5h6fTWZ6HMn5Fx5ixfo4tsWJQBjyMdtKkwcYQ8mnQ8axOe0+10maClnuEZ/qWjsRvsNh0VYyzQh+7oZwRSDX8U9FwrirkswA2sIYgy9aL/ZHBMjEgTdhx5TiSVAkXb7oNPPvcl22b6gDgvfhQ75lUZb0ZIBjlMGEGkSyONE7gqhYYlUf/UpZwDco9I+nkg8PdtVMR53bvgGGLuJv79YMdzP1V2548H4n2mC3iXQAEt682Ryt32ZZ1xuhSmQs7kvz4Jlw+ecQmbuKh3ichhTn5EX209jI9NVy2ABzWm1rWeNliS/0pFfPFVs/iaik1+T2chtcyk2Pn0S1TgvQPjh5UzShJWmVnaBIA+AnSbbSml2yONUmu3mVcf4gRDe31BRkWw5zwF9aZ/s/N1QJcxA66KXBjXHFKMHapUuAHRnOkog= Content-ID: <55F5C9EDFC66A64387679472D5DFBB20@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 0bd5c976-5457-4eea-d6e0-08d6ffb6c7f8 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jul 2019 13:03:02.5747 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR18MB3218 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH v2 4/5] x86: allow limiting the max C-state sub-state X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Ross Lagerwall Allow limiting the max C-state sub-state by appending to the max_cstate command-line parameter. E.g. max_cstate=1,0 The limit only applies to the highest legal C-state. For example: max_cstate = 1, max_csubstate = 0 ==> C0, C1 okay, but not C1E max_cstate = 1, max_csubstate = 1 ==> C0, C1 and C1E okay, but not C2 max_cstate = 2, max_csubstate = 0 ==> C0, C1, C1E, C2 okay, but not C3 max_cstate = 2, max_csubstate = 1 ==> C0, C1, C1E, C2 okay, but not C3 Signed-off-by: Ross Lagerwall Signed-off-by: Jan Beulich Acked-by: Andrew Cooper , subject to the --- v2: Explicitly log "unlimited". Pass NULL in the 2nd simple_strtoul() invocation. --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -1374,9 +1374,11 @@ Specify the maximum number of CPUs that This option is ignored in **pv-shim** mode. ### max_cstate (x86) -> `= ` +> `= [,]` -Specify the deepest C-state CPUs are permitted to be placed in. +Specify the deepest C-state CPUs are permitted to be placed in, and +optionally the maximum sub C-state to be used used. The latter only applies +to the highest permitted C-state. ### max_gsi_irqs (x86) > `= ` --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -104,7 +104,17 @@ bool lapic_timer_init(void) void (*__read_mostly pm_idle_save)(void); unsigned int max_cstate __read_mostly = UINT_MAX; -integer_param("max_cstate", max_cstate); +unsigned int max_csubstate __read_mostly = UINT_MAX; + +static int __init parse_cstate(const char *s) +{ + max_cstate = simple_strtoul(s, &s, 0); + if ( *s == ',' ) + max_csubstate = simple_strtoul(s + 1, NULL, 0); + return 0; +} +custom_param("max_cstate", parse_cstate); + static bool __read_mostly local_apic_timer_c2_ok; boolean_param("lapic_timer_c2_ok", local_apic_timer_c2_ok); @@ -347,7 +357,13 @@ static void dump_cx(unsigned char key) printk("'%c' pressed -> printing ACPI Cx structures\n", key); if ( max_cstate < UINT_MAX ) + { printk("max state: C%u\n", max_cstate); + if ( max_csubstate < UINT_MAX ) + printk("max sub-state: %u\n", max_csubstate); + else + printk("max sub-state: unlimited\n"); + } else printk("max state: unlimited\n"); for_each_present_cpu ( cpu ) @@ -592,7 +608,13 @@ static void acpi_processor_idle(void) do { cx = &power->states[next_state]; - } while ( cx->type > max_state && --next_state ); + } while ( (cx->type > max_state || + cx->entry_method == ACPI_CSTATE_EM_NONE || + (cx->entry_method == ACPI_CSTATE_EM_FFH && + cx->type == max_cstate && + (cx->address & MWAIT_SUBSTATE_MASK) > max_csubstate)) && + --next_state ); + cx = &power->states[next_state]; if ( next_state ) { if ( cx->type == ACPI_STATE_C3 && power->flags.bm_check && --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -731,7 +731,9 @@ static void mwait_idle(void) do { cx = &power->states[next_state]; - } while (cx->type > max_state && --next_state); + } while ((cx->type > max_state || (cx->type == max_cstate && + MWAIT_HINT2SUBSTATE(cx->address) > max_csubstate)) && + --next_state); if (!next_state) cx = NULL; else if (tb_init_done) --- a/xen/include/xen/acpi.h +++ b/xen/include/xen/acpi.h @@ -141,13 +141,21 @@ void acpi_unregister_gsi (u32 gsi); #ifdef CONFIG_ACPI_CSTATE /* - * Set highest legal C-state - * 0: C0 okay, but not C1 - * 1: C1 okay, but not C2 - * 2: C2 okay, but not C3 etc. + * max_cstate sets the highest legal C-state. + * max_cstate = 0: C0 okay, but not C1 + * max_cstate = 1: C1 okay, but not C2 + * max_cstate = 2: C2 okay, but not C3 etc. + + * max_csubstate sets the highest legal C-state sub-state. Only applies to the + * highest legal C-state. + * max_cstate = 1, max_csubstate = 0 ==> C0, C1 okay, but not C1E + * max_cstate = 1, max_csubstate = 1 ==> C0, C1 and C1E okay, but not C2 + * max_cstate = 2, max_csubstate = 0 ==> C0, C1, C1E, C2 okay, but not C3 + * max_cstate = 2, max_csubstate = 1 ==> C0, C1, C1E, C2 okay, but not C3 */ extern unsigned int max_cstate; +extern unsigned int max_csubstate; static inline unsigned int acpi_get_cstate_limit(void) { From patchwork Wed Jul 3 13:04:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11029577 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C5AF14F6 for ; Wed, 3 Jul 2019 13:07:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C37428628 for ; Wed, 3 Jul 2019 13:07:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2092928682; Wed, 3 Jul 2019 13:07:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 077EB28628 for ; Wed, 3 Jul 2019 13:07:35 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiexf-0004dY-IT; Wed, 03 Jul 2019 13:06:11 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hiexe-0004dR-EM for xen-devel@lists.xenproject.org; Wed, 03 Jul 2019 13:06:10 +0000 X-Inumbo-ID: 509c19e2-9d93-11e9-b814-236c25e6328f Received: from m4a0039g.houston.softwaregrp.com (unknown [15.124.2.85]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 509c19e2-9d93-11e9-b814-236c25e6328f; Wed, 03 Jul 2019 13:06:07 +0000 (UTC) Received: FROM m4a0039g.houston.softwaregrp.com (15.120.17.147) BY m4a0039g.houston.softwaregrp.com WITH ESMTP; Wed, 3 Jul 2019 13:02:28 +0000 Received: from M4W0334.microfocus.com (2002:f78:1192::f78:1192) by M4W0335.microfocus.com (2002:f78:1193::f78:1193) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Wed, 3 Jul 2019 13:04:15 +0000 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (15.124.8.12) by M4W0334.microfocus.com (15.120.17.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Wed, 3 Jul 2019 13:04:15 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com (10.255.139.95) by BY5PR18MB3393.namprd18.prod.outlook.com (10.255.136.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2032.20; Wed, 3 Jul 2019 13:04:14 +0000 Received: from BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc]) by BY5PR18MB3394.namprd18.prod.outlook.com ([fe80::2005:4b02:1d60:d1bc%3]) with mapi id 15.20.2052.010; Wed, 3 Jul 2019 13:04:14 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH v2 5/5] tools/libxc: allow controlling the max C-state sub-state Thread-Index: AQHVMZ/Q+p945gYsm0arEFP/PvEB7w== Date: Wed, 3 Jul 2019 13:04:13 +0000 Message-ID: <16586069-2940-bf5b-aed1-6c9e150c70b5@suse.com> References: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> In-Reply-To: <0dbf8520-89c7-753e-c10a-13399aec9cd5@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB6PR0501CA0021.eurprd05.prod.outlook.com (2603:10a6:4:8f::31) To BY5PR18MB3394.namprd18.prod.outlook.com (2603:10b6:a03:194::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e4271b32-a4e3-407d-f5c0-08d6ffb6f289 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:BY5PR18MB3393; x-ms-traffictypediagnostic: BY5PR18MB3393: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:551; x-forefront-prvs: 00872B689F x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(396003)(136003)(376002)(366004)(346002)(39860400002)(189003)(199004)(6512007)(316002)(476003)(36756003)(71190400001)(76176011)(54906003)(53936002)(4326008)(2501003)(80792005)(446003)(11346002)(31686004)(14444005)(256004)(26005)(5660300002)(99286004)(72206003)(2616005)(71200400001)(14454004)(486006)(186003)(305945005)(52116002)(68736007)(25786009)(2906002)(64756008)(73956011)(3846002)(66946007)(102836004)(66476007)(66066001)(6486002)(8676002)(81166006)(6916009)(386003)(81156014)(66556008)(6506007)(478600001)(31696002)(6436002)(66446008)(8936002)(7736002)(5640700003)(2351001)(86362001)(6116002)(309714004); DIR:OUT; SFP:1102; SCL:1; SRVR:BY5PR18MB3393; H:BY5PR18MB3394.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: NbYGxe6R2jnBHB7zHVsXA+S2btxR3SjYb4eItlR+wHk7ftae0E//29XXldMMKrKlQCrVesNt1/VmT4JNU/l4h+XmWwsEZjEJ5w2Fgl6wHyUprqwFn8GIcuWooN39SN3fprfH2yBpEpftzNCQxYdTozRBg+vDJRG6f2QrlvXGtUScU3JEu9HQFcBp5t5a4WmyfrQB0AJJUnXn20EBt/un7uy7VaV1PbXNRHsn/ZKO0galdVfrWxvig5P+D2hqYUlo16ogfm5HJMXW8xN7LGA0p1UdFdO4Is5iYutm3ofHCyXxN/jVUYD4ldN7146ZgpDS8xOL+5AdIdba4TUny92rpBh6Hw3r8pkNvKlLAW0MUpvz+EdZfnnAreqo4fw43ud0DAyHzDd/Qegf+2bTNMBNez6RDk8Y2kvFtE0qCp9rxdI= Content-ID: <36174B3B9213774289BE393BE0E65B81@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: e4271b32-a4e3-407d-f5c0-08d6ffb6f289 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jul 2019 13:04:13.9850 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR18MB3393 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH v2 5/5] tools/libxc: allow controlling the max C-state sub-state X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Ian Jackson , Wei Liu , =?utf-8?q?Roger?= =?utf-8?q?_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Ross Lagerwall Signed-off-by: Ross Lagerwall Make handling in do_pm_op() more homogeneous: Before interpreting op->cpuid as such, handle all operations not acting on a particular CPU. Also expose the setting via xenpm. Signed-off-by: Jan Beulich Acked-by: Wei Liu Acked-by: Andrew Cooper , subject to fixing --- v2: Adjust xenpm's usage message. Also adjust its output wording a little. --- a/tools/libxc/xc_pm.c +++ b/tools/libxc/xc_pm.c @@ -367,7 +367,7 @@ int xc_set_sched_opt_smt(xc_interface *x return rc; } -int xc_get_cpuidle_max_cstate(xc_interface *xch, uint32_t *value) +static int get_max_cstate(xc_interface *xch, uint32_t *value, uint32_t type) { int rc; DECLARE_SYSCTL; @@ -379,7 +379,7 @@ int xc_get_cpuidle_max_cstate(xc_interfa } sysctl.cmd = XEN_SYSCTL_pm_op; sysctl.u.pm_op.cmd = XEN_SYSCTL_pm_op_get_max_cstate; - sysctl.u.pm_op.cpuid = 0; + sysctl.u.pm_op.cpuid = type; sysctl.u.pm_op.u.get_max_cstate = 0; rc = do_sysctl(xch, &sysctl); *value = sysctl.u.pm_op.u.get_max_cstate; @@ -387,7 +387,17 @@ int xc_get_cpuidle_max_cstate(xc_interfa return rc; } -int xc_set_cpuidle_max_cstate(xc_interface *xch, uint32_t value) +int xc_get_cpuidle_max_cstate(xc_interface *xch, uint32_t *value) +{ + return get_max_cstate(xch, value, 0); +} + +int xc_get_cpuidle_max_csubstate(xc_interface *xch, uint32_t *value) +{ + return get_max_cstate(xch, value, 1); +} + +static int set_max_cstate(xc_interface *xch, uint32_t value, uint32_t type) { DECLARE_SYSCTL; @@ -398,12 +408,22 @@ int xc_set_cpuidle_max_cstate(xc_interfa } sysctl.cmd = XEN_SYSCTL_pm_op; sysctl.u.pm_op.cmd = XEN_SYSCTL_pm_op_set_max_cstate; - sysctl.u.pm_op.cpuid = 0; + sysctl.u.pm_op.cpuid = type; sysctl.u.pm_op.u.set_max_cstate = value; return do_sysctl(xch, &sysctl); } +int xc_set_cpuidle_max_cstate(xc_interface *xch, uint32_t value) +{ + return set_max_cstate(xch, value, 0); +} + +int xc_set_cpuidle_max_csubstate(xc_interface *xch, uint32_t value) +{ + return set_max_cstate(xch, value, 1); +} + int xc_enable_turbo(xc_interface *xch, int cpuid) { DECLARE_SYSCTL; --- a/tools/libxc/include/xenctrl.h +++ b/tools/libxc/include/xenctrl.h @@ -1906,6 +1906,9 @@ int xc_set_sched_opt_smt(xc_interface *x int xc_get_cpuidle_max_cstate(xc_interface *xch, uint32_t *value); int xc_set_cpuidle_max_cstate(xc_interface *xch, uint32_t value); +int xc_get_cpuidle_max_csubstate(xc_interface *xch, uint32_t *value); +int xc_set_cpuidle_max_csubstate(xc_interface *xch, uint32_t value); + int xc_enable_turbo(xc_interface *xch, int cpuid); int xc_disable_turbo(xc_interface *xch, int cpuid); --- a/tools/misc/xenpm.c +++ b/tools/misc/xenpm.c @@ -64,7 +64,9 @@ void show_help(void) " set-sched-smt enable|disable enable/disable scheduler smt power saving\n" " set-vcpu-migration-delay set scheduler vcpu migration delay in us\n" " get-vcpu-migration-delay get scheduler vcpu migration delay\n" - " set-max-cstate |'unlimited' set the C-State limitation ( >= 0)\n" + " set-max-cstate |'unlimited'[,|'unlimited']\n" + " set the C-State limitation ( >= 0) and\n" + " optionally the C-sub-state limitation ( >= 0)\n" " start [seconds] start collect Cx/Px statistics,\n" " output after CTRL-C or SIGINT or several seconds.\n" " enable-turbo-mode [cpuid] enable Turbo Mode for processors that support it.\n" @@ -195,7 +197,15 @@ static int show_max_cstate(xc_interface return ret; if ( value < XEN_SYSCTL_CX_UNLIMITED ) - printf("Max possible C-state: C%"PRIu32"\n\n", value); + { + printf("Max possible C-state: C%"PRIu32"\n", value); + if ( (ret = xc_get_cpuidle_max_csubstate(xc_handle, &value)) ) + return ret; + if ( value < XEN_SYSCTL_CX_UNLIMITED ) + printf("Max possible substate: %"PRIu32"\n\n", value); + else + puts(""); + } else printf("All C-states allowed\n\n"); @@ -1120,13 +1130,17 @@ void get_vcpu_migration_delay_func(int a void set_max_cstate_func(int argc, char *argv[]) { - int value; + int value, subval = XEN_SYSCTL_CX_UNLIMITED; char buf[12]; - if ( argc != 1 || + if ( argc < 1 || argc > 2 || (sscanf(argv[0], "%d", &value) == 1 ? value < 0 - : (value = XEN_SYSCTL_CX_UNLIMITED, strcmp(argv[0], "unlimited"))) ) + : (value = XEN_SYSCTL_CX_UNLIMITED, strcmp(argv[0], "unlimited"))) || + (argc == 2 && + (sscanf(argv[1], "%d", &subval) == 1 + ? subval < 0 + : (subval = XEN_SYSCTL_CX_UNLIMITED, strcmp(argv[1], "unlimited")))) ) { fprintf(stderr, "Missing, excess, or invalid argument(s)\n"); exit(EINVAL); @@ -1137,8 +1151,23 @@ void set_max_cstate_func(int argc, char if ( !xc_set_cpuidle_max_cstate(xc_handle, (uint32_t)value) ) printf("max C-state set to %s\n", value >= 0 ? buf : argv[0]); else + { fprintf(stderr, "Failed to set max C-state to %s (%d - %s)\n", value >= 0 ? buf : argv[0], errno, strerror(errno)); + return; + } + + if ( value != XEN_SYSCTL_CX_UNLIMITED ) + { + snprintf(buf, ARRAY_SIZE(buf), "%d", subval); + + if ( !xc_set_cpuidle_max_csubstate(xc_handle, (uint32_t)subval) ) + printf("max C-substate set to %s succeeded\n", + subval >= 0 ? buf : "unlimited"); + else + fprintf(stderr, "Failed to set max C-substate to %s (%d - %s)\n", + subval >= 0 ? buf : "unlimited", errno, strerror(errno)); + } } void enable_turbo_mode(int argc, char *argv[]) --- a/xen/drivers/acpi/pmstat.c +++ b/xen/drivers/acpi/pmstat.c @@ -398,7 +398,40 @@ int do_pm_op(struct xen_sysctl_pm_op *op int ret = 0; const struct processor_pminfo *pmpt; - if ( !op || op->cpuid >= nr_cpu_ids || !cpu_online(op->cpuid) ) + switch ( op->cmd ) + { + case XEN_SYSCTL_pm_op_set_sched_opt_smt: + { + uint32_t saved_value = sched_smt_power_savings; + + if ( op->cpuid != 0 ) + return -EINVAL; + sched_smt_power_savings = !!op->u.set_sched_opt_smt; + op->u.set_sched_opt_smt = saved_value; + return 0; + } + + case XEN_SYSCTL_pm_op_get_max_cstate: + BUILD_BUG_ON(XEN_SYSCTL_CX_UNLIMITED != UINT_MAX); + if ( op->cpuid == 0 ) + op->u.get_max_cstate = acpi_get_cstate_limit(); + else if ( op->cpuid == 1 ) + op->u.get_max_cstate = acpi_get_csubstate_limit(); + else + ret = -EINVAL; + return ret; + + case XEN_SYSCTL_pm_op_set_max_cstate: + if ( op->cpuid == 0 ) + acpi_set_cstate_limit(op->u.set_max_cstate); + else if ( op->cpuid == 1 ) + acpi_set_csubstate_limit(op->u.set_max_cstate); + else + ret = -EINVAL; + return ret; + } + + if ( op->cpuid >= nr_cpu_ids || !cpu_online(op->cpuid) ) return -EINVAL; pmpt = processor_pminfo[op->cpuid]; @@ -438,30 +471,6 @@ int do_pm_op(struct xen_sysctl_pm_op *op break; } - case XEN_SYSCTL_pm_op_set_sched_opt_smt: - { - uint32_t saved_value; - - saved_value = sched_smt_power_savings; - sched_smt_power_savings = !!op->u.set_sched_opt_smt; - op->u.set_sched_opt_smt = saved_value; - - break; - } - - case XEN_SYSCTL_pm_op_get_max_cstate: - { - BUILD_BUG_ON(XEN_SYSCTL_CX_UNLIMITED != UINT_MAX); - op->u.get_max_cstate = acpi_get_cstate_limit(); - break; - } - - case XEN_SYSCTL_pm_op_set_max_cstate: - { - acpi_set_cstate_limit(op->u.set_max_cstate); - break; - } - case XEN_SYSCTL_pm_op_enable_turbo: { ret = cpufreq_update_turbo(op->cpuid, CPUFREQ_TURBO_ENABLED); --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -350,7 +350,11 @@ struct xen_sysctl_pm_op { /* set/reset scheduler power saving option */ #define XEN_SYSCTL_pm_op_set_sched_opt_smt 0x21 - /* cpuidle max_cstate access command */ + /* + * cpuidle max C-state and max C-sub-state access command: + * Set cpuid to 0 for max C-state. + * Set cpuid to 1 for max C-sub-state. + */ #define XEN_SYSCTL_pm_op_get_max_cstate 0x22 #define XEN_SYSCTL_pm_op_set_max_cstate 0x23 --- a/xen/include/xen/acpi.h +++ b/xen/include/xen/acpi.h @@ -166,9 +166,22 @@ static inline void acpi_set_cstate_limit max_cstate = new_limit; return; } + +static inline unsigned int acpi_get_csubstate_limit(void) +{ + return max_csubstate; +} + +static inline void acpi_set_csubstate_limit(unsigned int new_limit) +{ + max_csubstate = new_limit; +} + #else static inline unsigned int acpi_get_cstate_limit(void) { return 0; } static inline void acpi_set_cstate_limit(unsigned int new_limit) { return; } +static inline unsigned int acpi_get_csubstate_limit(void) { return 0; } +static inline void acpi_set_csubstate_limit(unsigned int new_limit) { return; } #endif #ifdef XEN_GUEST_HANDLE_PARAM