From patchwork Wed Nov 22 09:28:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huang Shijie X-Patchwork-Id: 13464323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C324C61D9D for ; Wed, 22 Nov 2023 09:30:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PMisb67JjY/QRuPlK0eT1NO4BxvRdHfJKbQPJ20HHSI=; b=Q3osVGLBQiX7uy JN8tsNxgP3FlwacUSKQpaulII3xh40sqlGlo+yee7WIxxX1dlj050At74pP/pENyWmVVnPdaTiaM1 ZthI0l1vOzPpXd2GnOa6+Ll+FzIKa0PxQZcrlzpi6WBqb+4ZZHJ12q0Sz7fHn1yfUG1Mod1bbeFcD /CYNaj+MnrSLdIn2SA3iYC3H7om//d3/BuCRzGI8DkMc2W06abycpcWWIBNNUJ8EoI7UkCNl+xp1d SEN/xwjQb2S2s7drlkVa2te1C0VVUzDTwXqPInG83D05+vlzHNr9MKha1x0BnuIi1lLAJfn3nYAuJ mjnkIfECWWyASoEfB/xw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r5jYZ-001Czl-1V; Wed, 22 Nov 2023 09:30:03 +0000 Received: from mail-bn1nam02on20730.outbound.protection.outlook.com ([2a01:111:f400:7eb2::730] helo=NAM02-BN1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r5jYS-001CsA-1g for linux-arm-kernel@lists.infradead.org; Wed, 22 Nov 2023 09:29:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LnHI9ph2qoiZ73ZxRc4pKB49m2ttJbAKfYDTOGDgE+0j69ywZCo7WoSnuD1s94JatM7EU67ZMTujqeY3faowWYEUcy1ST74HQeraB6KIRNosVhpC8VwcU1ju3DJmbmEBUoaOt2WZK4FpwqShIEaixE8m1qsLYmCBhNLx/ahHHE5RtwAeCK/N3PeSAc8xJJzlKhv0iAMGN6gUrYzjYYvRZDXthcPMwp4P+OMg9Mm9emLyZoiQ2BMi/1XOKuDkqE0wTwjJyvCJVnAAH1Ub7LymrLzsOZIerc8lIT2QaQ635Af1f51kpyyWudI3E1l/7NcLshG3NJWkt0Z0Ddirh+VDzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vtkB6JlqAx6Ig7BPg1WUx9jtZreCP+mVpAxMP6FHeHQ=; b=XRq0OZi4zUMtNIqP8YEjhJyPEFXyikK9/PfkV+qMzuAOxRzcsW2RKi66zpbk1Reewaga/lt4U7skqBlQyxkGqil+P1lRl6arVvit3S+FfHWrvhnugP5PhMewesf8c1kuxD4vdTNvaLL7UpDKJ0Wl5hEVK1MFEH+hYem7gy3GdNF7zSPrJWd/Uz3Bd/ghjnlhyA116qUtraalvQqA5CWGDlAXlmCaVt8rJYbRNDvaOY5RLHlaJOxzrK7fS9l1vZj5Qk/1vdRUsrppRvAxrV+E+W8omQV7k0p1pW1iI79QHLu0c780n9AEljHWz/uV0Dcu6VJVdvOUB8Ga58MpwvBcIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vtkB6JlqAx6Ig7BPg1WUx9jtZreCP+mVpAxMP6FHeHQ=; b=cl9wSKf81PxAZ9hd3qtGyp+Q6rsScpnOzePzc+qpQyuGnfXv1JyA6xxp08CoVOgSYE+dG6sVLHUFuiDNNMjT9p5ImdysKB02i+m1EFpRm/rHkK1rjXzGrI06iu1TCB47b7BjTAOWLWGa3r3i3ghsx5+dBY+mVUso8WiXp5A7yYM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from PH0PR01MB7975.prod.exchangelabs.com (2603:10b6:510:26d::15) by CO1PR01MB6760.prod.exchangelabs.com (2603:10b6:303:f2::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7025.18; Wed, 22 Nov 2023 09:29:48 +0000 Received: from PH0PR01MB7975.prod.exchangelabs.com ([fe80::3f45:6905:e017:3b77]) by PH0PR01MB7975.prod.exchangelabs.com ([fe80::3f45:6905:e017:3b77%7]) with mapi id 15.20.7002.027; Wed, 22 Nov 2023 09:29:48 +0000 From: Huang Shijie To: catalin.marinas@arm.com Cc: will@kernel.org, mark.rutland@arm.com, suzuki.poulose@arm.com, broonie@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, anshuman.khandual@arm.com, robh@kernel.org, oliver.upton@linux.dev, maz@kernel.org, patches@amperecomputing.com, Huang Shijie Subject: [PATCH 3/4] arm64: copy_template.S: add loop_for_copy_128_bytes macro Date: Wed, 22 Nov 2023 17:28:54 +0800 Message-Id: <20231122092855.4440-4-shijie@os.amperecomputing.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231122092855.4440-1-shijie@os.amperecomputing.com> References: <20231122092855.4440-1-shijie@os.amperecomputing.com> X-ClientProxiedBy: CH0PR03CA0247.namprd03.prod.outlook.com (2603:10b6:610:e5::12) To PH0PR01MB7975.prod.exchangelabs.com (2603:10b6:510:26d::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PH0PR01MB7975:EE_|CO1PR01MB6760:EE_ X-MS-Office365-Filtering-Correlation-Id: 9de6066c-1b17-4f10-9f1b-08dbeb3d9254 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: m2+YSYKB8XofwUlyfPO/EVNGLciuj4D8+pYStyj5RvPu350rzVw7Yx6V4p042+fNfCf63DcGjgcb0MLd7GGYCNMuopYDyljsWhkU0urJT6rwu0n1q1UUk/yYKohUOwga38qkNqDvL7zh3l4XvrRExvj1VOk6NHwQEHlbqv0xcQTiyKkHJiuuZ8xy6rcb8LEdCj/ySMNDD2FDfsYdLGjGeNVHKFduqcpz4rHxfZ4Q1lkrjCsIzrmEfMA8mLRjWgLCwuZxr54JYPfSHdfEwYgKJlBeJVaTCPujXjEq+L4VxUTaOBWBAGwjR8irlb9vIMggRpk8sCqe3l5yXsoRyf3YYhn8LUU3WqpoGtjBD2zQ+FA/rX9Ap9JVn9W7r7B81f8mvF30L2hEJ6mZ4udc+oBT2K3j6jRdlcQiQNXcOJP8cXbtgloOtagzAVS7ut6lMMkhKq76fOMyn1YMs47s0W3HQGsQnjcaHdg/W530w3tDuYDRfB7u74zU1rKKK8kZQuV+gIw49DbuHdJ0PSV1SvxIqDrfsZatJ4Qd6RcZ5GeZUTHcU7tGhrBQN/sd7JzyDBsergz5jN0uWLG88xmcFyO1K3ir0GjgwzN/P8UmvGDnjhAREAZ1ppRf5y+AFY62muJJ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR01MB7975.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230031)(39850400004)(396003)(136003)(366004)(376002)(346002)(230922051799003)(451199024)(186009)(64100799003)(1800799012)(41300700001)(2906002)(86362001)(7416002)(5660300002)(38350700005)(6512007)(2616005)(1076003)(26005)(478600001)(83380400001)(6486002)(107886003)(6506007)(52116002)(6666004)(38100700002)(66946007)(316002)(4326008)(8676002)(66556008)(8936002)(6916009)(66476007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8MNkqwJJbfzyWUnWwcSMDBgv79lVeVSEYAiANMIPiypHto8alOaf/wdYNqeVZknBlBJJ5YHsZsmyLn22Sxo9nLJkSfxZkKOd+vIOjgnQz1DIGuouEN1ULDxQtLMoM4GvrdGmU7/nTqAnzNHmFhDhU+48pzYcoGv8RCdCgCbY1eZiKdYjewoyg8oOtdsKFI1DtJBZOs6c+9703yPmcUCB0FiDcJ+k9Z0Trb6iU7SEwSlEDOY8pjjZN2Zf5257+xzbzpuSlxbHVGSuXKOBIT+b79vYKs6JDNNZ0xorGtgh8Wm2p2UOQQx2Uz6T2M6+CVgzf2mUH1HkWXd+b4izKCStcCNzTkaNG8S65cBhNYk6FB6Ni9l6lWwpkkhqrFflTfyQcWA4nWMvcvFHTLQIhYrqCFWdPDDJ4WLAKAcMKC6lbDKZhSgVkwL6lqS73dezRJJnvjRFo/d1v0AU6x7CyINKA/cGmaDmS5yKA7GnC79hw+zJIzyGWgoQQqqyf+3+CJ8gwv/PgXFOGGMzRn6uV+dPutczU+OqSXduaS3YE4EnKAmhVSJh0HFhcYMdg8+vwPYeZp6HHIvlCQChbfSSo2UaNsgPgkZWozCA9sRmKwXVCKRQW30blSPuZsmcsVl1Px78mu0D6+3mrv+1ZvhbZ0aUGc65pfgt49LL4vZKgPBFMPm62cgjtJ8LeVO5uhAnwpicg7hOnkiFZVMd9z6Rd43dM4/L4Pz+wSNKpvXiB3YWKzKa6k6765waO8jyaerV9D/oJLuyNmMUUdjgbt4TwKP37PKuYaDSu+0Jdza2ahk0j9cZAsKScpjiVFa/VFhqozVLjMmHNPS0/pCp9RBAk+7z+G5H3ifPBYzFi2nHfSo1WzDfzbAIWmV1aNJFaiXXHgRd1TEaKfpSaY/BCqTRPqC3UkN9dcgjNDXfqAaI8u8dhjc8DC2+n42nPHE8IXNwEkhotJbSMMRlPHJSGXGMZ4DuehBuHZyeUB+6nf3lJtkgAR3han8Kz8hmq6PyME7ckt+jl5DVQS/wyYyvWHFkUUgzeY9PGswHH/FrrZR53lkH8Rk509cp4byNLbf6elCmwtfVzYgce5CAZ8rE+tKKMTUcAZObU8tftz5JseyxDHRA8tsI0FOTcHR3HO5MfQpE8EpMS8q3b1IMMPJCgsN+9+6Ax46cHlvkElcZB4L1OACb2iu9tE8RSsy6bpeRju+QsT1f6jbqzNhVvN8Ci40IgM9WJ58Tgf3/TN+bFGTORBdsgnDo6+85PWn9uEv0XlXaw/UECrgeNC1jQYhhWZndGIVUyp3th4QkuQI1t+rbeEptKkUHfZme8Eok6oxcff58vSyDErh1SLM1t1K/ZrMvTo9moKT6TLvXTSsS3qEDGVW9rF+EWmiEa66LHUd5gQnIqOmh28GjmSDW6nCnhGzK4EY6+KGzwrMQJQsM0b9K1g/QJ37B81UNksiObcg7c3v4PtokJj27m9W6zv9ma3MB87VRo7/7QhYcMyYwcY+nuKtMkWumRgbnebjrqDlPeYkzN3r0w1ZczwCi5BPT07pREKlq5QibgVfv8A1WwZKL7a9IYmigFCtbwh9pUqE5pueNI+LhKJM4usA6RpcALLz9OHbjLw== X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9de6066c-1b17-4f10-9f1b-08dbeb3d9254 X-MS-Exchange-CrossTenant-AuthSource: PH0PR01MB7975.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2023 09:29:48.7083 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1WOACMTPFMsr1ZRP3NMuIddXCHOyDZdS7HnpFUDbsufymSDzIGZjwsSqhKfMLlBK3jb2Kd6g8m+wvB3S1mNoqZnnE9f+IAtmCahJQ4FE1YVkw60qbSxbD9XKLAgqbRsi X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB6760 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231122_012956_561200_C03EB7D8 X-CRM114-Status: GOOD ( 11.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add the loop_for_copy_128_bytes macro, to make the code clean. And make preparation for the next patch. Signed-off-by: Huang Shijie --- arch/arm64/lib/copy_template.S | 58 ++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 27 deletions(-) diff --git a/arch/arm64/lib/copy_template.S b/arch/arm64/lib/copy_template.S index 488df234c49a..79b32569260c 100644 --- a/arch/arm64/lib/copy_template.S +++ b/arch/arm64/lib/copy_template.S @@ -10,6 +10,36 @@ * files/head:/src/aarch64/ */ +.macro loop_for_copy_128_bytes extra_ops + /* pre-get 64 bytes data. */ + ldp1 A_l, A_h, src, #16 + ldp1 B_l, B_h, src, #16 + ldp1 C_l, C_h, src, #16 + ldp1 D_l, D_h, src, #16 +1: + \extra_ops + /* + * interlace the load of next 64 bytes data block with store of the last + * loaded 64 bytes data. + */ + stp1 A_l, A_h, dst, #16 + ldp1 A_l, A_h, src, #16 + stp1 B_l, B_h, dst, #16 + ldp1 B_l, B_h, src, #16 + stp1 C_l, C_h, dst, #16 + ldp1 C_l, C_h, src, #16 + stp1 D_l, D_h, dst, #16 + ldp1 D_l, D_h, src, #16 + subs count, count, #64 + b.ge 1b + stp1 A_l, A_h, dst, #16 + stp1 B_l, B_h, dst, #16 + stp1 C_l, C_h, dst, #16 + stp1 D_l, D_h, dst, #16 + + tst count, #0x3f + b.ne .Ltail63 +.endm /* * Copy a buffer from src to dest (alignment handled by the hardware) @@ -151,31 +181,5 @@ D_h .req x14 */ .p2align L1_CACHE_SHIFT .Lcpy_body_large: - /* pre-get 64 bytes data. */ - ldp1 A_l, A_h, src, #16 - ldp1 B_l, B_h, src, #16 - ldp1 C_l, C_h, src, #16 - ldp1 D_l, D_h, src, #16 -1: - /* - * interlace the load of next 64 bytes data block with store of the last - * loaded 64 bytes data. - */ - stp1 A_l, A_h, dst, #16 - ldp1 A_l, A_h, src, #16 - stp1 B_l, B_h, dst, #16 - ldp1 B_l, B_h, src, #16 - stp1 C_l, C_h, dst, #16 - ldp1 C_l, C_h, src, #16 - stp1 D_l, D_h, dst, #16 - ldp1 D_l, D_h, src, #16 - subs count, count, #64 - b.ge 1b - stp1 A_l, A_h, dst, #16 - stp1 B_l, B_h, dst, #16 - stp1 C_l, C_h, dst, #16 - stp1 D_l, D_h, dst, #16 - - tst count, #0x3f - b.ne .Ltail63 + loop_for_copy_128_bytes .Lexitfunc: