From patchwork Thu May 23 10:54:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Kagan X-Patchwork-Id: 10957415 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6368B924 for ; Thu, 23 May 2019 10:58:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49B9B28468 for ; Thu, 23 May 2019 10:58:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3E42928471; Thu, 23 May 2019 10:58:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.0 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1C9C028468 for ; Thu, 23 May 2019 10:58:11 +0000 (UTC) Received: from localhost ([127.0.0.1]:33642 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTlQJ-0008GI-A7 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 23 May 2019 06:58:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57292) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hTlN7-0005FQ-Ja for qemu-devel@nongnu.org; Thu, 23 May 2019 06:54:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hTlN6-0004sd-3y for qemu-devel@nongnu.org; Thu, 23 May 2019 06:54:53 -0400 Received: from mail-eopbgr80108.outbound.protection.outlook.com ([40.107.8.108]:34195 helo=EUR04-VI1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hTlN5-0004qs-CN for qemu-devel@nongnu.org; Thu, 23 May 2019 06:54:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EDyQPjCSBOpacGlKqNWOmQ/h/gpLlI3lmI0MqsAgJBo=; b=Hmj3nhEpz80NFYcjp32b1FfxUyIurrCiAEGiWpw/3POj84uG/Phgkh5pe3VICOHxQJCE9XJlwefGPB/fj/yXffZDzo2P8Emc07XPS85qs6SwyZ1GnWT5XUcDgM2U0j+6pQbcaVCeDnz6UjqRWFxb0tq+SJAA66zOeyKfhPYwuZQ= Received: from DBBPR08MB4854.eurprd08.prod.outlook.com (20.179.46.205) by DBBPR08MB4760.eurprd08.prod.outlook.com (10.255.79.215) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1878.24; Thu, 23 May 2019 10:54:48 +0000 Received: from DBBPR08MB4854.eurprd08.prod.outlook.com ([fe80::4d46:2c95:d6e:ab3f]) by DBBPR08MB4854.eurprd08.prod.outlook.com ([fe80::4d46:2c95:d6e:ab3f%7]) with mapi id 15.20.1900.010; Thu, 23 May 2019 10:54:48 +0000 From: Roman Kagan To: Paolo Bonzini , "qemu-devel@nongnu.org" Thread-Topic: [RFC PATCH 0/2] establish nesting rule of BQL vs cpu-exclusive Thread-Index: AQHVEVXwLFxTyUznOk6fV4juQbW3Hw== Date: Thu, 23 May 2019 10:54:47 +0000 Message-ID: <20190523105440.27045-1-rkagan@virtuozzo.com> Accept-Language: en-US, ru-RU Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [185.231.240.5] x-clientproxiedby: HE1PR0901CA0058.eurprd09.prod.outlook.com (2603:10a6:3:45::26) To DBBPR08MB4854.eurprd08.prod.outlook.com (2603:10a6:10:da::13) authentication-results: spf=none (sender IP is ) smtp.mailfrom=rkagan@virtuozzo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.21.0 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9b63f334-e4ee-4264-62be-08d6df6d1288 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:DBBPR08MB4760; x-ms-traffictypediagnostic: DBBPR08MB4760: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3276; x-forefront-prvs: 00462943DE x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39840400004)(346002)(376002)(366004)(136003)(396003)(189003)(199004)(256004)(14444005)(486006)(52116002)(2616005)(66066001)(68736007)(3846002)(6486002)(64756008)(6116002)(71190400001)(66946007)(66556008)(73956011)(186003)(71200400001)(110136005)(305945005)(66446008)(66476007)(99286004)(476003)(53936002)(6436002)(25786009)(6512007)(478600001)(316002)(2906002)(8936002)(26005)(81166006)(81156014)(8676002)(7736002)(36756003)(50226002)(86362001)(5660300002)(2501003)(102836004)(6506007)(14454004)(386003)(1076003); DIR:OUT; SFP:1102; SCL:1; SRVR:DBBPR08MB4760; H:DBBPR08MB4854.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: w8KxG5nJubNciR66L9d9EB8oJb598GmrYm8GqwlK3B4K80DiytuNFDoNQQABUKPNnX9QJvD57kS2fshwzTPAj4Az82UhFi+qAh3pcTJBuSaMlwtVSWLXNccoQRCBk2yfqBeSE9yBAAx8mDyRsDezHzxB2k8MSi0C/2kzNJWiTsMP4DYfRS+KOeXFPjRMqF2h/PpO+TG5BuBL30KH7LEkYt3Z/hQrYtKZm/WnSfwbdHRCRwV2zGYSnHqXgVBMYnV9/hDslEWxBIfnMQ4UY6+NKZkJ0ZQxUwPPRbQedU9+2hEaHQXegJrke3EM7VnyX6aUmkAx3llmMwc4S3EkipTfDPoKZ8e0mq11AauOD6Kqb7deMaCHXL4a89tjNG4jJ1X4dOadQm7puuWvGca/Cto+CPsqHrNHMoNHplsVKryPG/o= MIME-Version: 1.0 X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b63f334-e4ee-4264-62be-08d6df6d1288 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 May 2019 10:54:47.9208 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4760 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.8.108 Subject: [Qemu-devel] [RFC PATCH 0/2] establish nesting rule of BQL vs cpu-exclusive X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP I came across the following AB-BA deadlock: vCPU thread main thread ----------- ----------- async_safe_run_on_cpu(self, async_synic_update) ... [cpu hot-add] process_queued_cpu_work() qemu_mutex_unlock_iothread() [grab BQL] start_exclusive() cpu_list_add() async_synic_update() finish_safe_work() qemu_mutex_lock_iothread() cpu_exec_start() ATM async_synic_update seems to be the only async safe work item that grabs BQL. However it isn't quite obvious that it shouldn't; in the past there were more examples of this (e.g. memory_region_do_invalidate_mmio_ptr). It looks like the problem is generally in the lack of the nesting rule for cpu-exclusive sections against BQL, so I thought I would try to address that. This patchset is my feeble attempt at this; I'm not sure I fully comprehend all the consequences (rather, I'm sure I don't) hence RFC. Roman Kagan (2): cpus-common: nuke finish_safe_work cpus-common: assert BQL nesting within cpu-exclusive sections cpus-common.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)