From patchwork Thu Apr 5 17:17:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 10325089 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 333B360467 for ; Thu, 5 Apr 2018 17:22:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 208A4292F6 for ; Thu, 5 Apr 2018 17:22:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 150B829311; Thu, 5 Apr 2018 17:22:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED,DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A991629303 for ; Thu, 5 Apr 2018 17:22:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=U3/ZnB0GkthaN5BrFSETTOigzhEbdHqvSzZVgUdtQIs=; b=txifQUzC6Ur77c oLS3gqaGWrOMPHWe1FGKHz2dynkHYoYGQMoueLFIont8/9R97kUASoTarq6I/KTMZLQiFiB2zDshL ETbd/qlafU2mRNco3QtJuM7+xkCg0ca/iflWu3+l4qMjdvnI3o1xzEUXdN84E9BTequdhsG4j5+ZO 27AZGRpbnfOaKFH/EnQgp/EzlgQ5dMHx2adVBjY7FpxRb9jBU5j8ihUnDF+wifT5baHfdKFlRV4ue tsg1+cH8lm46EYyTLjeh5SNWTH5lspRHDI/azBi9XdRl3yLoDdJAoqCEm7jQelw/fg/p0dMoj2h1W 8hrTqLW5jOL0AkT57+Fg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1f48a7-0001bc-2S; Thu, 05 Apr 2018 17:21:51 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1f48Xo-0007dj-Uo for linux-arm-kernel@bombadil.infradead.org; Thu, 05 Apr 2018 17:19:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Content-Type:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=XEQz6emAgwntPRfKIbHfyMCuekS15EPUov8us/EwudA=; b=RejVvYnleYMX+q/wBP1tMlvxN Vu1eruskAt/sp3Jip/32iOCzCTiJcUE8wTn5/Kl51mpz/4ADlrfaQQreBqC03/55TbN+ar8truknz On+hVInhhF77ZyMb7PUpIhuHpX3kqsKKdAIM/8AgmDxD7GMPYEhiCnEZv826R+1nUKCyaBUSjOCkC GzDGqXlXu/vlbbT/cJV70smLmAzZwSIcBpiLThulHh1guRkgJa1LqDvotTy5vd4aa9nqIrbH8ch6/ FA5dJ3hfU9qdyotfwlNdvdRGGVqJeF7Y/iP/h2x5sGRF+E1FAqJGmwq2SUYMAWzMECDdNNBx6WdNj P+qrByeYQ==; Received: from mail-sn1nam02on061e.outbound.protection.outlook.com ([2a01:111:f400:fe44::61e] helo=NAM02-SN1-obe.outbound.protection.outlook.com) by merlin.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1f48Xl-0002He-RX for linux-arm-kernel@lists.infradead.org; Thu, 05 Apr 2018 17:19:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=XEQz6emAgwntPRfKIbHfyMCuekS15EPUov8us/EwudA=; b=X1B7/cREZenmL3517mSZRD8FpfGN2CHYmjYI9/HhNespGU639aX7lfT7Qnm+IAU3kKK+J0AfXmHJp6rW8nQ5wVF4acKad0yDouB20ZCMK1fhjdeB+vP53zLKmtSKFVbEaJBnR9QkM8WiGu2GnHkNyBcGdwjmgEyNCasNEwTwagk= Received: from localhost (50.233.148.156) by BN6PR07MB2900.namprd07.prod.outlook.com (10.173.28.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.653.12; Thu, 5 Apr 2018 17:19:06 +0000 From: Yury Norov To: "Paul E. McKenney" , Mark Rutland , Will Deacon , Chris Metcalf , Christopher Lameter , Russell King - ARM Linux , Steven Rostedt , Mathieu Desnoyers , Catalin Marinas , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Alexey Klimov Subject: [PATCH 3/5] arm64: early ISB at exit from extended quiescent state Date: Thu, 5 Apr 2018 20:17:58 +0300 Message-Id: <20180405171800.5648-4-ynorov@caviumnetworks.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180405171800.5648-1-ynorov@caviumnetworks.com> References: <20180405171800.5648-1-ynorov@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: DB6P190CA0023.EURP190.PROD.OUTLOOK.COM (10.175.240.36) To BN6PR07MB2900.namprd07.prod.outlook.com (10.173.28.146) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2d291463-4721-4742-9652-08d59b195715 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:BN6PR07MB2900; X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB2900; 3:veM/WOD1D4bdKT+lpqIQWJPdcb2mOjO0m18KNq/4Z9eXQ9NOI60K1jqpUg21hF5RCd7OuHgv+6LxWERczWXYj9bNKwh5A2AO5gGZOXzk3OK7ChzLtRYMO8NEpTPrlvoQ/1d1x7VfepzNALK/JiIh7VF/iyUap4mIKEleDaSGCoM97V4PtPCAJVCwD8x17/ruIncO7YP2WmArf1oWvKiZAgORRj0kjJcCgoVjype/jCuX4sKGxRO2rLXVF0cQvdjc; 25:cbaXrNKpplzidXfnqZv14s/C6CbYMwYqM8QqyhAVODQzQdM7m5LIcgg168pMNwqXBcEemPJgtBiafSm0zM61gVTXZe6IXM02vz39IxnBFKqIw1R4Wh8ZlJ2lreicaqCKiXXxd1b5kId14LWZ3DtzTRx1c8l8PAwe8kcq+uLQKnGqp4fyfXM/iHawFE6Df4o/bahiPw74A9thrlTvOzaUVKz55GG8pG8mHVtaKibmP1hb5Xa6b67PRqMEYI7DkYMGszivyu6X1lef6DnYHdTr7pzWu7ksauiDqQpyxAevzOFXH0HsW7SRFTPB2xHXQtnsHYEzdyfcRN9lqMQrJM38Sw==; 31:3rWyqhSO2uAfJ8AS4WRIT7xvCTAuUv8mQu/mhQGOK00wNLQPjY4epKCyXBUN47Yv7/pB42DSVZ3RHv39f2yxfvp/Eckv16d0DxbKzQepMyzlQ1X4cehIxcRS0GUNHyWyoCi7nmT6lGhwoqm9RUIAX7mwaklx1RkVs+bgbEn9dYETtLUUGRjFGzYVB/pciWfvvIJH2SAAuIbL7l/Xq9/oiWbmaeKk2upUMBOeCrXcqVo= X-MS-TrafficTypeDiagnostic: BN6PR07MB2900: X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB2900; 20:br48vpiaQdD3m4G+Rzq0MoLHr1rG9WJp1xqk1w9THGTO0aHSc7Ka0rmkwb5ixk+wGBuIkDyp0y6zo4no7Mmj5tnwp6ODGeKaI2WHz4yDN+RkQN8tu2Z4lIC+MQdZiyC95DJxYJb6Ye6aisYsPvShBaOkuHqurLLwuwNAHnlAFkmVAF5mvz0v7pgEitkkw7qaJ8qME/kNcNyJ5wPXflZC7VyvoKF9DdT2ZdcirhBPSiZ+9xMFu0BrqaBs7QXPbeEhMxjOWMlFtaLqHMqasBDCRoX1cSV0K795VkqT4uAUE3OH/HdB2yTXXpxUdKJWK8Gmx5y77Kzf0u6xBGqgfq1eT67Shs9qNNxpkDGjXW1W6OGV5yjxh0mH8enYmHGVdNXjOZg3BwdNLQE1oCn/BlOi649sdNsI5yRyZLVkGmAf7CDSUfiMF9ThTtaKNlPHGvpxXg+xwhJplxRXIASA3FAHTeHOWtmRVrFA/DvOgHw8TVVi0/kjowvtfeE2CIA9tyArARQ9sDcYd2zd7N2lV3/jRQ7+SP01egfyIoi3Ia0AtGQMYgTJ4O//E1rX3xd2HgWnU/KlFmlD+9UxCGcwwE4TYjM92ztUXCpPN9tPlk2qrDE=; 4:yvW+1+ufUo6kbU6Xzdi4vVJG4s6wE6KiwVJCHkXQyLg/K35K/grNH7Y2nHwLaoMLmqb3Ewfoa30eO4z82P9jWphoGZdWsxq20nAKYMLUs1JtVWO4z8a60QShVpYufHyFtWUFpnkWnv/H+YdoEODoisuecbSEmof5VRexQPwlrW/qaLiR9xrXW5zCniYxqJ/+EIvQ+8UyCiVCyBd1oGRUYBUNfKAn0zI9Wtf8u3/LxBN349UMPdJHPw4JP8Y/feLstv12oV9q8YQXf5PZ6h+bVbsOmTJZKNHu0LuUxfevzq1lkx9SFLPc5tfekb+0FJno X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(180628864354917); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(10201501046)(3231221)(944501327)(52105095)(6041310)(20161123564045)(20161123558120)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(6072148)(201708071742011); SRVR:BN6PR07MB2900; BCL:0; PCL:0; RULEID:; SRVR:BN6PR07MB2900; X-Forefront-PRVS: 06339BAE63 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(39380400002)(346002)(376002)(39860400002)(366004)(396003)(189003)(199004)(478600001)(1076002)(11346002)(47776003)(39060400002)(50466002)(48376002)(53936002)(105586002)(3846002)(25786009)(476003)(6116002)(66066001)(956004)(2616005)(386003)(42882007)(186003)(16526019)(68736007)(6346003)(26005)(4326008)(486006)(6486002)(2906002)(8936002)(106356001)(7416002)(36756003)(50226002)(81156014)(81166006)(6666003)(97736004)(446003)(5660300001)(8676002)(110136005)(16586007)(76506005)(305945005)(316002)(52116002)(76176011)(72206003)(51416003)(59450400001)(7736002)(6496006)(575784001)(921003)(1121003); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR07MB2900; H:localhost; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Yuri.Norov@cavium.com; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN6PR07MB2900; 23:CWwq2Hq+fK9pdiVYBSdvM8wy5EtqJjpaznRrkmnuc?= =?us-ascii?Q?ydAvdZ5Ri49FjBQX9mBOYXZ5teSih4jgx63Py+OhQIrn/NXKJXb2pC1bkS1f?= =?us-ascii?Q?VSRTB9LovIGcWqBIfzHA3B9qG38fJz4JUWwwU6qX8AFZIM+POarXr3vvCeLc?= =?us-ascii?Q?sVuBpTC1NF9Y/O1MPGLvxkeHcXv7pVodVRq3PY0vPd3FkJaRGp3gOZJU8hq7?= =?us-ascii?Q?fh6TUeidbGQ7xfVypeGcJttuUF5urf7EWXDxaRxUq50E9mRNpse8FRZp4Ewp?= =?us-ascii?Q?o+bHLehWr52C/QMOGp0jV2IYyhpYAB0at+poIfVg+tt8sWj20m+29zlnH6e2?= =?us-ascii?Q?U9OBiCwCTy1HgZnnZ1P4E4Qy9M1WFaFJD5WoFQm5lswlrMWJjDLuIorD2VzD?= =?us-ascii?Q?AyinSPyQkLnUb9iLSZOSkGdxLIDTb5ijX9o7fMlPRqlso8Lc3rpIZ/FPpW3a?= =?us-ascii?Q?8E7F0wphTQu5U4S3WIaEfUa5vaWKV3+yBZUJknu4tV4QdvZxwFm4Ceg6E69i?= =?us-ascii?Q?12ONXxix1ub5nkTZe9amBMQT3DCIBJ627GXQiDpXZtCySGpGbLyQ/Vg0xU+6?= =?us-ascii?Q?POjdZMTF+E0VON3pkIHKUyITmLKkUHrnNmskxkqczKlB3vrZBsnu2MOnU9KI?= =?us-ascii?Q?8Cn7LRtgHbBfywtBAbFjYtuq1ftV3efuWHszAXtu0Ojv9DdxUQtpd/nw1LoS?= =?us-ascii?Q?eoplebJU4gbgrHkJR2W9pUXnHS1gj3HktcWsTdLpibIXCqeJ5CK5xRa522WD?= =?us-ascii?Q?ZB+mubBFrI70jcLdAnIdeBSICM614RMKR8JZpwTuv3XfDjeP0wWTuBYNLaTI?= =?us-ascii?Q?yLYsECWeHd0nCCK1rITS22+XRYYclRk2AOz4u9N98dkZzGSe+qk/qTojbzoJ?= =?us-ascii?Q?BXxPyLuz5DWiu1jwwrawxfrl8FK8Ylx0D66GAa9Pf5wNq+PCeh4vzGZ0d6M0?= =?us-ascii?Q?5l48BN61/NdshQBcjNOcf9V+ryzhC2coz8/ToW+F0ytDAQ1H0VrZXnBQR+zh?= =?us-ascii?Q?JvFGQA5ohouGLcuxywIT05Ds/YPtUDnq6OhxmOmtnh9pPWa0H0+b6+B9x6Qv?= =?us-ascii?Q?f31xxvaR0yQ3URJh6dmtilF35YT2eKuA6i7zkKpnT8/AgKFntGopEDtQ0MWb?= =?us-ascii?Q?x/8KC+J4tzyYUd3IVd0B6D36sJ78AlbQT15ZvU0j26+OjvGNVDcIY1A2AcsH?= =?us-ascii?Q?pzuUVJxD6YY64pi+VpeNcIQjYoZ4os59YvXM3dWzaHvIFFsWiFV72xEo27WD?= =?us-ascii?Q?rdwnuKGQGSYuc/d2wSRNrnAKd+V3NDk8sdV4gofp+uh0PwnGQ1FYF7BiYaIi?= =?us-ascii?Q?zQiKGa9BY8y+jjgNsZfHcAWOwxBdwRpd5z8pYf3jk5R2tHHY8JYCx45OsUe2?= =?us-ascii?Q?3eZUHYAnrUzJTsKHH/XE+I+R0z/7ys5jc6v/wA92vwtMjul?= X-Microsoft-Antispam-Message-Info: GoepqHY4TTbpqY72OcF3alIUOmElOIjqhI7eM0iiWOWKQh46Y2V7bbVsV6KUYK7DavbwMx5SUd+XQCaK97OMxqCgaq3qncFmB7OrIF/TV+Z8VW3YQfhAUv6NDbZOROUVIEmqVtZVraBD6VtJysZ5Am4trFxrKh/ePEoxWUceXND/JVgRha669/EZ0Ilvv7do X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB2900; 6:k47GBhyTEWGN29FUBS/Q+2ypbpXjEJ1Fhb7dGQay3WTPudCivBn9B4ACDLXFGcCs321Jcf3XPCohJ4mrbYSOoT/Z3xTx5kQHsXh1T6ThXJAGAnddPUVX9CRs3Jq//yLR8lT/WIjSNMNnVMkyrQrXhk+Zne99TVW5mZhapLM/MBvuZmEIye/v0/MbDNsilTok3WfTwY2lSaRZVePyizETA7h99HZpRoL/vx3DuBl/0x7qYSrto59Tq5BSi3kCv2J7Ob96leNH9VVhsvdP1ShijPDMvtYPanqivtQ4bKRgrojL8v3LeGURtE9H7/DA1s2qG/SF6K2GwLQI+GuT48LTq8PxvEqO8hI7SAfcvvGPWS761sBh1zOQzBDbVoq02sSJzCOFyEC89CIVm3tCT+9nzpuLyFWo9ft3epcSWOIRxLFyJ0Qmtd3KUA7u/oUQvazeMvT/iZTaFL3IXzf1NHKT/g==; 5:iUP2Pj4HSZoQZea9fmOQMcDwLjYovcrzzHTmuxRUgD1Oh25XccZnoFJoKAuz03eJ4hp2miWoRAoqaLRzmYUNoUPZx+a+V7RYrKAHQK9KMMSRTwUPpnuC3RsYC7ftkCScHhieVD0bwvg8vu3PRzbBEEwpsE0ushxogVd87oHEmbY=; 24:yMJAELcZdezB4AmxolxjEM0mOBBz/oT/iZeqqhU0mw9rFUs4RKnftvTMpYiJEGy6yNeNJo4RPm7TKgDbW4AZxxTuEEKlo/RIjCmid4pEyrg= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB2900; 7:y10FXTdwQnhlsYuyXXRUdIIihdAAMB8HfL0n8kxPR4YVJyMrqc6AY/3OQxO+FHxsB8kjjmwCQd/WdaK2oUR8n5VsMeJh7ZQ+RYmPJ8MXg1yo0pO8gdhP7HyXIf0qBDMYfQbmt4MPsyk+iD0UNYMn3D089OzUAtGfmOffZYOwHFLi2/2swkbiXPagBJ4xChiPiskp3stILUuWA/YieYCqjsILwQdwC3+LLrh1v+XrDlYyDXy1BFDtl8COjSbDTfKh X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Apr 2018 17:19:06.5170 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2d291463-4721-4742-9652-08d59b195715 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR07MB2900 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180405_131926_113366_ADFE4F41 X-CRM114-Status: GOOD ( 15.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-kernel@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-mm@kvack.org, Yury Norov , linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP This series enables delaying of kernel memory synchronization for CPUs running in extended quiescent state (EQS) till the exit of that state. ARM64 uses IPI mechanism to notify all cores in SMP system that kernel text is changed; and IPI handler calls isb() to synchronize. If we don't deliver IPI to EQS CPUs anymore, we should add ISB early in EQS exit path. There are 2 such paths. One starts in do_idle() loop, and other in el0_svc entry. For do_idle(), isb() is added in arch_cpu_idle_exit() hook. And for SVC handler, isb is called in el0_svc_naked. Suggested-by: Will Deacon Signed-off-by: Yury Norov --- arch/arm64/kernel/entry.S | 16 +++++++++++++++- arch/arm64/kernel/process.c | 7 +++++++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index c8d9ec363ddd..b1e1c19b4432 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -48,7 +48,7 @@ .endm .macro el0_svc_restore_syscall_args -#if defined(CONFIG_CONTEXT_TRACKING) +#if !defined(CONFIG_TINY_RCU) || defined(CONFIG_CONTEXT_TRACKING) restore_syscall_args #endif .endm @@ -483,6 +483,19 @@ __bad_stack: ASM_BUG() .endm +/* + * If CPU is in extended quiescent state we need isb to ensure that + * possible change of kernel text is visible by the core. + */ + .macro isb_if_eqs +#ifndef CONFIG_TINY_RCU + bl rcu_is_watching + cbnz x0, 1f + isb // pairs with aarch64_insn_patch_text +1: +#endif + .endm + el0_sync_invalid: inv_entry 0, BAD_SYNC ENDPROC(el0_sync_invalid) @@ -949,6 +962,7 @@ alternative_else_nop_endif el0_svc_naked: // compat entry point stp x0, xscno, [sp, #S_ORIG_X0] // save the original x0 and syscall number + isb_if_eqs enable_daif ct_user_exit el0_svc_restore_syscall_args diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index f08a2ed9db0d..74cad496b07b 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -88,6 +88,13 @@ void arch_cpu_idle(void) trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, smp_processor_id()); } +void arch_cpu_idle_exit(void) +{ + /* Pairs with aarch64_insn_patch_text() for EQS CPUs. */ + if (!rcu_is_watching()) + isb(); +} + #ifdef CONFIG_HOTPLUG_CPU void arch_cpu_idle_dead(void) {