From patchwork Wed Sep 14 14:48:20 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Li X-Patchwork-Id: 9331533 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C2ADE607FD for ; Wed, 14 Sep 2016 13:51:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B246029E27 for ; Wed, 14 Sep 2016 13:51:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A635629E89; Wed, 14 Sep 2016 13:51:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6415A29E27 for ; Wed, 14 Sep 2016 13:51:45 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bkAZZ-0004iw-04; Wed, 14 Sep 2016 13:49:57 +0000 Received: from mail-eopbgr00069.outbound.protection.outlook.com ([40.107.0.69] helo=EUR02-AM5-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bkAY2-0003XC-C9 for linux-arm-kernel@lists.infradead.org; Wed, 14 Sep 2016 13:48:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=GFRuCeGQ5iRHO9IppktIBGjj4WCiie8k/j2rGYpuSO4=; b=YmFER2KSAMoiZxCujVQs/1oBZo/B9RIaRIPpiHbae+1zMFfqghlYAqVrcsY4QEqHxfuhk8Q79KkWwc/UX6ast6DPxgOCY6ebZESuJRg5GyhktUZaFa1r2BAbxWwnVN7lKJ/9uMKJOEiDkBARE9Bd7q2COVpBhZ3CVP8kMnQ+xvE= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=frank.li@nxp.com; Received: from lizhi-Precision-Tower-5810.am.freescale.net (192.88.168.49) by AM3PR04MB258.eurprd04.prod.outlook.com (2a01:111:e400:8817::13) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.619.3; Wed, 14 Sep 2016 13:47:55 +0000 From: Frank Li To: Subject: [PATCH 1/1 v6] ARM: imx: Added perf functionality to mmdc driver Date: Wed, 14 Sep 2016 09:48:20 -0500 Message-ID: <1473864500-24256-1-git-send-email-Frank.Li@nxp.com> X-Mailer: git-send-email 2.5.2 MIME-Version: 1.0 X-Originating-IP: [192.88.168.49] X-ClientProxiedBy: BY1PR14CA0023.namprd14.prod.outlook.com (2a01:111:e400:506f::33) To AM3PR04MB258.eurprd04.prod.outlook.com (2a01:111:e400:8817::13) X-MS-Office365-Filtering-Correlation-Id: de671e43-f934-4880-db5b-08d3dca5bcdd X-Microsoft-Exchange-Diagnostics: 1; AM3PR04MB258; 2:Afqy2A+sxOMWrm5yH/MWrws5mrHMGyNuPC1fTJQV0sx1MWizNe3EIRlVTFEXpsQaLBpLfh0i70BJ9NS+rrRJUA2TnbuWV7WPwj/lMLIWdRTEmHD9o6a1fVaGBnwgTqQvxoMkNF5kMNoOwGJEmzHBM30ICwgzd/yu8SjEk/4tqXzoSh8ubzJlRW0IkuPluc6L; 3:iVeovgdjrbyPeYkI0PwoP9mwd1tbjcrdBGbmqualbS2CUzeJ5sXn7Za2Z+K3crAKy3vx7JUw5XPn610bQG4INSF6R+wKxH9EDyHx7DM4Ql1QSGU8DTPwqlTdcNjaeuHb; 25:aIyV1oRJB8ISv4SCgaDjleX1flnzbp+ic4eNJ8BiOPTD63236KwW48Fa+AG9UIJ73HDW6KUIViOHKxf0r3TMCg9cw98UX3QgObze0FTK/rSc+QltY8XyKYMepD3Ob9Qg4/zr+EdVLnWk/ZlH+to7KpBoqsjxtI4cMKyeVCwJKqZ7+KP4dCLVjweWe2/msHqSBWNYqRdPI+6sd/35X8xLuwCBEPB7lnHJjTRaO1QTjiixIJ+st0BtiifAqWVuaPk7vAZTMlu5fZdAKZNoZjRIjTXDWJd1xwbhioQvpN5d/R92nzuP+IfyAb13OsGMXIpUusGjhGEZRplsZv9fQqOADdESPeiTxj+52AkYqjQArk1O3xpsfKY4WSvXyU3rTh9KGNjuTsl4KKpDnvjA9NOqimrf4lNoAYNCx8XzlZ3AMDg= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM3PR04MB258; X-Microsoft-Exchange-Diagnostics: 1; AM3PR04MB258; 31:W+dITzuCdkHbUTdSfvW02pNfaUAyNkX6fy0LWQtjvJ/zy8uk6f5DVQwtes+VVTMnxSnZ5AUS4aVPGe6yQN+euDuLrBga1DFgQ27mRFkbUgxwt2nEeIqSJeBVmdfRNTX6xWafSUSP7vxNU0HuC8PzLqr0zLXXlgEQhdjoHDNo3CD7grdmDFhP34V5mXaJrS11zpdtpEVz1W/4HPUIYIjHLKHeeuifPTKfHlhsm/BCwL4=; 20:ETKXc+LPei0TCoWLOCCZ1kDCLLC3sr6xjUEwyBghZDOgzmJsDrpmpQUsnpgH5P5bp8MnUYa3skCer9kapMa/J+nJ5WbqCQZgqqa8HICpSQS8QnBQIxdQAcm4WY2i/kEIKo30hptSgcAaA1SK6KBcniczonODNzXC6MpVQutYuyfwAIGRHyTHWK/nE61udg/Q1BRGpx/R87h05x71Qm2wassDC3jjfw87nlLqzoTsryygWTeu+P9EaqKvPnsTV0PUk2LkPl+f1S+7mwax6f1ruj8Uz4ATcmmHR7xEYZII0OSZieB3LMQPGUCtCOkaQ5hQkp0YObtH+2gV0boVLxiqvcpJ40NJU6dSgXu8ciZQs3CPjIzZToM4r3xDLIDX4o2j9moy/qhqrx4pFy1S07Nf1Zwg+p5lE5Xwms4rdP5BBlny6p0ceW4waid7GMocr5lYNZHjSpNTSLp87LaIyQrdqyHeEH9JrrR3XoI1RX+lwSbt5cs3GdyVLcXQhsoQTTSw43wPyDyG+EAXg8+XQfKbhxg0eBWTaieTp4sH0vWlWlyg2X4LKWlGEhxy8G/p1HWfFCeAtKYZXmwzBAWbh8c6qzHqO7LaSjFlfW32ids8Mi8= X-NIPS-smtp-INBOUND: xFSL-EDGE X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(250305191791016)(22074186197030)(185117386973197)(788757137089); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6055026); SRVR:AM3PR04MB258; BCL:0; PCL:0; RULEID:; SRVR:AM3PR04MB258; X-Microsoft-Exchange-Diagnostics: 1; AM3PR04MB258; 4:qA/OVzFdbHe9dDyu2/zgZTRmwxfcYw6mDrjNyeZCe3KBviOsTfSfDyiBlaFTjxOuamKvpeiFXwk9mtNz580XMjMkcTDJYyV4vkdcAgzVzJSgrQ6vVcnn7fburK00UNHVA++OXwJfFPE2rjV22q0ADFzkGtDhfwdbuvispODWfuCcn0Y4W1Uvb7JjMlkmdMGMEyvF/EiXNaHIJR5x3NJQPcaoqggmLggplAx4KcIQvMySivQLsSu34kX3MGHs3yLfoDhBH9gDFOuOWhwrvwT5mMXmENIzquxQWLjR7NCqLJe1KG0+8TJjQVCr9KE4gnmihY3LlmFyKAkio4ejgscdrqIKQ4/EH3TvERZAoyyf3Jx2mDOS0qcgoI7oFbQNdKEJXxeZhcbvuWX+5zqTGx1UEnR0J/7O7tOprQPkZpeFrXfXW7M+a4cV4mDpAR0rsnQYDZL0O18ivaYgwEMhjku2DXKHGb9ELIsGe3sIxAK4JE+vUExwlHeikskobLnW1ju1Fqp5+mmFFIEGnapQLhSHeWrS++Cznw8eeErgRoqcnHw= X-Forefront-PRVS: 006546F32A X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(189002)(7736002)(5660300001)(2351001)(189998001)(15975445007)(92566002)(106356001)(229853001)(50986999)(4326007)(5003940100001)(105586002)(50226002)(50466002)(101416001)(2906002)(48376002)(586003)(19580395003)(305945005)(7846002)(81156014)(81166006)(7416002)(36756003)(86362001)(97736004)(47776003)(68736007)(3846002)(5890100001)(42186005)(6116002)(8676002)(66066001)(19580405001)(110136003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM3PR04MB258; H:lizhi-Precision-Tower-5810.am.freescale.net; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM3PR04MB258; 23:u28eoHUlaC7fA4F8HDCkA2L1g4sKZjyyU9W0Uvcjkr?= =?us-ascii?Q?p+oBeopEaXudGHTwO29eKdFncorOSM7qLrMlckn38hfJHCn8O/lt6YqgFngO?= =?us-ascii?Q?ULQbm9cndKqEdM5tdy9CtVZs8dhahViQRSDdXupOopGj6VNvKp+0VwoXlGJM?= =?us-ascii?Q?hvzYDZT8eQNyHMwAedW+aQXYxNS3Cy1jxLfaYuIiSyDtE4cAY841yXXDchaN?= =?us-ascii?Q?CtCOtXbJHAwQN+KcSwb7aJTgE2n2pxJ9dygRwwB1fqol7rS0n8Ui/JFeTCbE?= =?us-ascii?Q?Jt+rT3RnHC8ad4Im8vQSEfmTYDAIP0aIWNHKS9CHnxFiHI8UMzqQ9q45EW1E?= =?us-ascii?Q?Q6CVoCC6UGRwVRzvdd8U78b2yvDizB8p0girvKd/vO19CkaqcTHoilGG8NUD?= =?us-ascii?Q?YIl1Dbt/d5U94W6xGM1x06z+y5SOSxtFpGx9lCVjmq9AyVeUZ3scTOmNa/Eg?= =?us-ascii?Q?l46P4gjnI81YSR0ZaKyzbccd/fMQrJdASSo8sIew8i9foNhYzvAt9XLR01HO?= =?us-ascii?Q?h1BTaQl0CckmW1OhL6g2deXMUnsvPfCaFWJXW956FqI2Sq8mrFO1Qo2NojH8?= =?us-ascii?Q?yGe6RHWCl87uPCmkngw6t8lU/yUep+oCPrW2Hz1IqguQ/bRthViBVqg0OuYV?= =?us-ascii?Q?91Wk/1fLJjzlB+jcJthVCdCJZadqpa1Ze04Rhsm5Xl3b2f3nhzI57fhIWsca?= =?us-ascii?Q?kqwgtILh751pigacOfnM3wvCE/8DKiQtu6RKPe0KMOddTuFyDm/5zMIfcyu6?= =?us-ascii?Q?sGeUAaBN2DapgZQ0shXkCMGFRikuE4Ujm0zZR7YQRfhnO/bjpfE5C2hAhBSO?= =?us-ascii?Q?r84Iat/AURBWoK/DKDpmYA2SEQh2GoCm9HreLI8Jy6XUveRZL5B9ZVtGTSic?= =?us-ascii?Q?mi8UPzsmjPewabB3cKgpgfMbrnmYr5S6SpDjnva5mzEY0Qqr0V/R90Cx1TTC?= =?us-ascii?Q?1EtUgaMhQk2ZBkH8JIc2ozOsi4zKdPxKjFV0CdcBxCzr0q9WbZUzFJuGtE2T?= =?us-ascii?Q?ASuaai2aie65sw3mkHnSiDacIwaLGeA4fN/5mdxZkAn7bHes6pASM3pF6/7q?= =?us-ascii?Q?oES/MpwKjB4kaU74y+tz7oezed?= X-Microsoft-Exchange-Diagnostics: 1; AM3PR04MB258; 6:r7u+OrUI38FQDy1/Hr0R0muyv+RbffjPec9rUWxgXps8llaiOyCFD3tTH5SVIm77vQ0FL4QMK1eazA3DLkYGQglxzfi6emqBs1CTz5AruZWmIRnXra36MWGRIAsCoJl+19nrfJHxD297UJIoNPregtofBtOgTOhd7rmlOzZbDjKRabalcWs8ToMDl3Jhq5s6xj6snT0CtgX5NDlYC1vlcV9c36XDYcKst1crAsf+NCV3qeqpIW+Zg7oQL3VH+op1qUJRV8GOLfB1u/q23xYXmQtrdzlFZ14wFzGaiBu2Be/8MYOHcNc/LDUtRaEwDLjOD1HMZKkQEPmOhbIUBYcuJw==; 5:O6PRm77cyNe08mQpfog6TXZ0Md1TH2hXNCF84cLCXTLIqPythI7uoI7Uq5BEn6bUHnxtU8egzGczct0HOH3ZnW30bZ0UVd5y3yZEL2dyg35WWiDQALqpmJ6guuKlaOhC/UGZQf5zU2DPg4jesfhWsQ==; 24:FztbllHYc83rXoy2gnvKHXqW2Ve88LCNnzu23C6b8Mv6zPI58Z3RpNw06TFe8OUcJ2R5XvpgS1ES30XuCSLuXrIpN9DCuNpNZ3vWxAZL7TQ=; 7:5OuFKZCJ/JMQPsYR5xgFQdI5vAnOl5k+w9iemFkIk6es025UsvybFJsGu7GW5wflKf4+5CdgFFdhnZk82VuKyekCBKfOeILz3tih3BjaD2yvtVgpp/q7Uzk1/T0kmW2wUjugg8ojhSpjeG712YKXGxa0CDzGXAFtyEVbFt/4+lqBASlFn3rb66HV5ywN+/x30Sk8RnEu/0pWjUz2w7HVM6/6Ve+VY9bvaogfcpyztI2umuWBb3T4BXv+r8fP+rbl SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2016 13:47:55.6949 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM3PR04MB258 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160914_064823_467110_53EB99B5 X-CRM114-Status: GOOD ( 12.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, peterz@infradead.org, frank.li@nxp.com, linux-kernel@vger.kernel.org, acme@kernel.org, Zhengyu Shen , alexander.shishkin@linux.intel.com, mingo@redhat.com, jerry.zhengyu.shen@gmail.com, lznuaa@gmail.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Zhengyu Shen MMDC is a multi-mode DDR controller that supports DDR3/DDR3L x16/x32/x64 and LPDDR2 two channel x16/x32 memory types. MMDC is configurable, high performance, and optimized. MMDC is present on i.MX6 Quad and i.MX6 QuadPlus devices, but this driver only supports i.MX6 Quad at the moment. MMDC provides registers for performance counters which read via this driver to help debug memory throughput and similar issues. $ perf stat -a -e mmdc/busy-cycles/,mmdc/read-accesses/,mmdc/read-bytes/,mmdc/total-cycles/,mmdc/write-accesses/,mmdc/write-bytes/ dd if=/dev/zero of=/dev/null bs=1M count=5000 Performance counter stats for 'dd if=/dev/zero of=/dev/null bs=1M count=5000': 898021787 mmdc/busy-cycles/ 14819600 mmdc/read-accesses/ 471.30 MB mmdc/read-bytes/ 2815419216 mmdc/total-cycles/ 13367354 mmdc/write-accesses/ 427.76 MB mmdc/write-bytes/ 5.334757334 seconds time elapsed Signed-off-by: Zhengyu Shen Signed-off-by: Frank Li --- Changes from v5 to v6 Improve group event error handle Changes from v4 to v5 Remove mmdc_pmu:irq remove static variable cpuhp_mmdc_pmu remove spin_lock check is_sampling_event(event) remove unnecessary cast use hw_perf_event::prev_count Changes from v3 to v4: Tested and fixed crash relating to removing events with perf fuzzer Adjusted formatting Moved all perf event code under CONFIG_PERF_EVENTS Switched cpuhp_setup_state to cpuhp_setup_state_nocalls Changes from v2 to v3: Use WARN_ONCE instead of returning generic error values Replace CPU Notifiers with newer state machine hotplug Added additional checks on event_init for grouping and sampling Remove useless mmdc_enable_profiling function Added comments Moved start index of events from 0x01 to 0x00 Added a counter to pmu_mmdc to only stop hrtimer after all events are finished Replace readl_relaxed and writel_relaxed with readl and writel Removed duplicate update function Used devm_kasprintf when naming mmdcs probed Changes from v1 to v2: Added cpumask and migration handling support to driver Validated event during event_init Added code to properly stop counters Used perf_invalid_context instead of perf_sw_context Added hrtimer to poll for overflow Added better description Added support for multiple mmdcs arch/arm/mach-imx/mmdc.c | 463 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 461 insertions(+), 2 deletions(-) diff --git a/arch/arm/mach-imx/mmdc.c b/arch/arm/mach-imx/mmdc.c index db9621c..bd53b8e 100644 --- a/arch/arm/mach-imx/mmdc.c +++ b/arch/arm/mach-imx/mmdc.c @@ -1,5 +1,5 @@ /* - * Copyright 2011 Freescale Semiconductor, Inc. + * Copyright 2011,2016 Freescale Semiconductor, Inc. * Copyright 2011 Linaro Ltd. * * The code contained herein is licensed under the GNU General Public @@ -10,12 +10,16 @@ * http://www.gnu.org/copyleft/gpl.html */ +#include #include +#include #include #include #include #include #include +#include +#include #include "common.h" @@ -27,8 +31,462 @@ #define BM_MMDC_MDMISC_DDR_TYPE 0x18 #define BP_MMDC_MDMISC_DDR_TYPE 0x3 +#define TOTAL_CYCLES 0x0 +#define BUSY_CYCLES 0x1 +#define READ_ACCESSES 0x2 +#define WRITE_ACCESSES 0x3 +#define READ_BYTES 0x4 +#define WRITE_BYTES 0x5 + +/* Enables, resets, freezes, overflow profiling*/ +#define DBG_DIS 0x0 +#define DBG_EN 0x1 +#define DBG_RST 0x2 +#define PRF_FRZ 0x4 +#define CYC_OVF 0x8 + +#define MMDC_MADPCR0 0x410 +#define MMDC_MADPSR0 0x418 +#define MMDC_MADPSR1 0x41C +#define MMDC_MADPSR2 0x420 +#define MMDC_MADPSR3 0x424 +#define MMDC_MADPSR4 0x428 +#define MMDC_MADPSR5 0x42C + +#define MMDC_NUM_COUNTERS 6 + +#define to_mmdc_pmu(p) container_of(p, struct mmdc_pmu, pmu) + static int ddr_type; +#ifdef CONFIG_PERF_EVENTS + +static DEFINE_IDA(mmdc_ida); + +PMU_EVENT_ATTR_STRING(total-cycles, mmdc_total_cycles, "event=0x00") +PMU_EVENT_ATTR_STRING(busy-cycles, mmdc_busy_cycles, "event=0x01") +PMU_EVENT_ATTR_STRING(read-accesses, mmdc_read_accesses, "event=0x02") +PMU_EVENT_ATTR_STRING(write-accesses, mmdc_write_accesses, "config=0x03") +PMU_EVENT_ATTR_STRING(read-bytes, mmdc_read_bytes, "event=0x04") +PMU_EVENT_ATTR_STRING(read-bytes.unit, mmdc_read_bytes_unit, "MB"); +PMU_EVENT_ATTR_STRING(read-bytes.scale, mmdc_read_bytes_scale, "0.000001"); +PMU_EVENT_ATTR_STRING(write-bytes, mmdc_write_bytes, "event=0x05") +PMU_EVENT_ATTR_STRING(write-bytes.unit, mmdc_write_bytes_unit, "MB"); +PMU_EVENT_ATTR_STRING(write-bytes.scale, mmdc_write_bytes_scale, "0.000001"); + +struct mmdc_pmu { + struct pmu pmu; + void __iomem *mmdc_base; + cpumask_t cpu; + struct hrtimer hrtimer; + unsigned int active_events; + struct device *dev; + struct perf_event *mmdc_events[MMDC_NUM_COUNTERS]; + struct hlist_node node; +}; + +static struct mmdc_pmu *cpuhp_mmdc_pmu; + +/* + * Polling period is set to one second, overflow of total-cycles (the fastest + * increasing counter) takes ten seconds so one second is safe + */ +static unsigned int mmdc_poll_period_us = 1000000; + +module_param_named(pmu_poll_period_us, mmdc_poll_period_us, uint, + S_IRUGO | S_IWUSR); + +static ktime_t mmdc_timer_period(void) +{ + return ns_to_ktime((u64)mmdc_poll_period_us * 1000); +} + +static ssize_t mmdc_cpumask_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct mmdc_pmu *pmu_mmdc = dev_get_drvdata(dev); + + return cpumap_print_to_pagebuf(true, buf, &pmu_mmdc->cpu); +} + +static struct device_attribute mmdc_cpumask_attr = +__ATTR(cpumask, S_IRUGO, mmdc_cpumask_show, NULL); + +static struct attribute *mmdc_cpumask_attrs[] = { + &mmdc_cpumask_attr.attr, + NULL, +}; + +static struct attribute_group mmdc_cpumask_attr_group = { + .attrs = mmdc_cpumask_attrs, +}; + +static struct attribute *mmdc_events_attrs[] = { + &mmdc_total_cycles.attr.attr, + &mmdc_busy_cycles.attr.attr, + &mmdc_read_accesses.attr.attr, + &mmdc_write_accesses.attr.attr, + &mmdc_read_bytes.attr.attr, + &mmdc_read_bytes_unit.attr.attr, + &mmdc_read_bytes_scale.attr.attr, + &mmdc_write_bytes.attr.attr, + &mmdc_write_bytes_unit.attr.attr, + &mmdc_write_bytes_scale.attr.attr, + NULL, +}; + +static struct attribute_group mmdc_events_attr_group = { + .name = "events", + .attrs = mmdc_events_attrs, +}; + +PMU_FORMAT_ATTR(event, "config:0-63"); +static struct attribute *mmdc_format_attrs[] = { + &format_attr_event.attr, + NULL, +}; + +static struct attribute_group mmdc_format_attr_group = { + .name = "format", + .attrs = mmdc_format_attrs, +}; + +static const struct attribute_group *attr_groups[] = { + &mmdc_events_attr_group, + &mmdc_format_attr_group, + &mmdc_cpumask_attr_group, + NULL, +}; + +static u32 mmdc_read_counter(struct mmdc_pmu *pmu_mmdc, int cfg) +{ + void __iomem *mmdc_base, *reg; + + mmdc_base = pmu_mmdc->mmdc_base; + + switch (cfg) { + case TOTAL_CYCLES: + reg = mmdc_base + MMDC_MADPSR0; + break; + case BUSY_CYCLES: + reg = mmdc_base + MMDC_MADPSR1; + break; + case READ_ACCESSES: + reg = mmdc_base + MMDC_MADPSR2; + break; + case WRITE_ACCESSES: + reg = mmdc_base + MMDC_MADPSR3; + break; + case READ_BYTES: + reg = mmdc_base + MMDC_MADPSR4; + break; + case WRITE_BYTES: + reg = mmdc_base + MMDC_MADPSR5; + break; + default: + return WARN_ONCE(1, + "invalid configuration %d for mmdc counter", cfg); + } + return readl(reg); +} + +static int mmdc_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct mmdc_pmu *pmu_mmdc = hlist_entry_safe(node, struct mmdc_pmu, node); + int target; + + if (!cpumask_test_and_clear_cpu(cpu, &pmu_mmdc->cpu)) + return 0; + + target = cpumask_any_but(cpu_online_mask, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu_mmdc->pmu, cpu, target); + cpumask_set_cpu(target, &pmu_mmdc->cpu); + + return 0; +} + +static bool mmdc_pmu_group_is_valid(struct perf_event *event) +{ + struct pmu *pmu = event->pmu; + struct perf_event *leader = event->group_leader; + struct perf_event *sibling; + + int cfg = leader->attr.config; + int counter_mask = 0; + + if (cfg < 0 || cfg >= MMDC_NUM_COUNTERS) + return false; + + if (leader->pmu == pmu) + counter_mask |= 1 << cfg; + else if (!is_software_event(leader)) + return false; + + list_for_each_entry(sibling, &leader->sibling_list, group_entry) { + if (sibling->pmu == pmu) { + cfg = sibling->attr.config; + if (cfg < 0 || cfg >= MMDC_NUM_COUNTERS) + return false; + counter_mask |= 1 << cfg; + } else if (!is_software_event(sibling)) { + return false; + } + } + + if (event == leader) + return true; + + cfg = event->attr.config; + if (cfg < 0 || cfg >= MMDC_NUM_COUNTERS) + return false; + + return !(counter_mask & (1 << cfg)); +} + +static int mmdc_event_init(struct perf_event *event) +{ + struct mmdc_pmu *pmu_mmdc = to_mmdc_pmu(event->pmu); + int cfg = event->attr.config; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + if (is_sampling_event(event) || event->attach_state & PERF_ATTACH_TASK) + return -EOPNOTSUPP; + + if (event->cpu < 0) { + dev_warn(pmu_mmdc->dev, "Can't provide per-task data!\n"); + return -EOPNOTSUPP; + } + + if (event->attr.exclude_user || + event->attr.exclude_kernel || + event->attr.exclude_hv || + event->attr.exclude_idle || + event->attr.exclude_host || + event->attr.exclude_guest || + event->attr.sample_period) + return -EINVAL; + + if (cfg < 0 || cfg >= MMDC_NUM_COUNTERS) + return -EINVAL; + + if (!mmdc_pmu_group_is_valid(event)) + return -EINVAL; + + event->cpu = cpumask_first(&pmu_mmdc->cpu); + return 0; +} + +static void mmdc_event_update(struct perf_event *event) +{ + struct mmdc_pmu *pmu_mmdc = to_mmdc_pmu(event->pmu); + struct hw_perf_event *hwc = &event->hw; + u64 delta, prev_raw_count, new_raw_count; + + do { + prev_raw_count = local64_read(&hwc->prev_count); + new_raw_count = mmdc_read_counter(pmu_mmdc, + event->attr.config); + } while (local64_cmpxchg(&hwc->prev_count, prev_raw_count, + new_raw_count) != prev_raw_count); + + delta = (new_raw_count - prev_raw_count) & 0xFFFFFFFF; + + local64_add(delta, &event->count); +} + +static void mmdc_event_start(struct perf_event *event, int flags) +{ + struct mmdc_pmu *pmu_mmdc = to_mmdc_pmu(event->pmu); + struct hw_perf_event *hwc = &event->hw; + void __iomem *mmdc_base, *reg; + + mmdc_base = pmu_mmdc->mmdc_base; + reg = mmdc_base + MMDC_MADPCR0; + + /* + * hrtimer is required because mmdc does not provide an interrupt so + * polling is necessary + */ + hrtimer_start(&pmu_mmdc->hrtimer, mmdc_timer_period(), + HRTIMER_MODE_REL_PINNED); + + local64_set(&hwc->prev_count, 0); + + writel(DBG_RST, reg); + writel(DBG_EN, reg); +} + +static int mmdc_event_add(struct perf_event *event, int flags) +{ + struct mmdc_pmu *pmu_mmdc = to_mmdc_pmu(event->pmu); + struct hw_perf_event *hwc = &event->hw; + + int cfg = event->attr.config; + + if (WARN_ONCE((cfg < 0 || cfg >= MMDC_NUM_COUNTERS), + "invalid configuration %d for mmdc", cfg)) + return -1; + + if (flags & PERF_EF_START) + mmdc_event_start(event, flags); + + pmu_mmdc->mmdc_events[cfg] = event; + pmu_mmdc->active_events++; + + local64_set(&hwc->prev_count, mmdc_read_counter(pmu_mmdc, cfg)); + + return 0; +} + +static void mmdc_event_stop(struct perf_event *event, int flags) +{ + struct mmdc_pmu *pmu_mmdc = to_mmdc_pmu(event->pmu); + void __iomem *mmdc_base, *reg; + int cfg = (int)event->attr.config; + + mmdc_base = pmu_mmdc->mmdc_base; + reg = mmdc_base + MMDC_MADPCR0; + + if (WARN_ONCE((cfg < 0 || cfg >= MMDC_NUM_COUNTERS), + "invalid configuration %d for mmdc counter", cfg)) + return; + + writel(PRF_FRZ, reg); + mmdc_event_update(event); +} + +static void mmdc_event_del(struct perf_event *event, int flags) +{ + struct mmdc_pmu *pmu_mmdc = to_mmdc_pmu(event->pmu); + int cfg = event->attr.config; + + pmu_mmdc->mmdc_events[cfg] = NULL; + pmu_mmdc->active_events--; + + if (pmu_mmdc->active_events <= 0) + hrtimer_cancel(&pmu_mmdc->hrtimer); + + mmdc_event_stop(event, PERF_EF_UPDATE); +} + +static void mmdc_overflow_handler(struct mmdc_pmu *pmu_mmdc) +{ + int i; + + for (i = 0; i < MMDC_NUM_COUNTERS; i++) { + struct perf_event *event = pmu_mmdc->mmdc_events[i]; + + if (event) + mmdc_event_update(event); + } +} + +static enum hrtimer_restart mmdc_timer_handler(struct hrtimer *hrtimer) +{ + struct mmdc_pmu *pmu_mmdc = container_of(hrtimer, struct mmdc_pmu, + hrtimer); + + mmdc_overflow_handler(pmu_mmdc); + hrtimer_forward_now(hrtimer, mmdc_timer_period()); + + return HRTIMER_RESTART; +} + +static int mmdc_pmu_init(struct mmdc_pmu *pmu_mmdc, + void __iomem *mmdc_base, struct device *dev) +{ + int mmdc_num; + + *pmu_mmdc = (struct mmdc_pmu) { + .pmu = (struct pmu) { + .task_ctx_nr = perf_invalid_context, + .attr_groups = attr_groups, + .event_init = mmdc_event_init, + .add = mmdc_event_add, + .del = mmdc_event_del, + .start = mmdc_event_start, + .stop = mmdc_event_stop, + .read = mmdc_event_update, + }, + .mmdc_base = mmdc_base, + .dev = dev, + .active_events = 0, + }; + + mmdc_num = ida_simple_get(&mmdc_ida, 0, 0, GFP_KERNEL); + + return mmdc_num; +} + +static int imx_mmdc_remove(struct platform_device *pdev) +{ + struct mmdc_pmu *pmu_mmdc = platform_get_drvdata(pdev); + + perf_pmu_unregister(&pmu_mmdc->pmu); + cpuhp_remove_state_nocalls(CPUHP_ONLINE); + cpuhp_mmdc_pmu = NULL; + kfree(pmu_mmdc); + return 0; +} + +static int imx_mmdc_perf_init(struct platform_device *pdev, void __iomem *mmdc_base) +{ + struct mmdc_pmu *pmu_mmdc; + char *name; + int mmdc_num; + int ret; + + pmu_mmdc = kzalloc(sizeof(*pmu_mmdc), GFP_KERNEL); + if (!pmu_mmdc) { + pr_err("failed to allocate PMU device!\n"); + return -ENOMEM; + } + + mmdc_num = mmdc_pmu_init(pmu_mmdc, mmdc_base, &pdev->dev); + if (mmdc_num == 0) + name = "mmdc"; + else + name = devm_kasprintf(&pdev->dev, + GFP_KERNEL, "mmdc%d", mmdc_num); + + hrtimer_init(&pmu_mmdc->hrtimer, CLOCK_MONOTONIC, + HRTIMER_MODE_REL); + pmu_mmdc->hrtimer.function = mmdc_timer_handler; + + cpuhp_state_add_instance_nocalls(CPUHP_ONLINE, + &pmu_mmdc->node); + cpumask_set_cpu(smp_processor_id(), &pmu_mmdc->cpu); + ret = cpuhp_setup_state_multi(CPUHP_AP_NOTIFY_ONLINE, + "MMDC_ONLINE", NULL, + mmdc_pmu_offline_cpu); + if (ret) { + pr_err("cpuhp_setup_state_multi failure\n"); + goto pmu_register_err; + } + + ret = perf_pmu_register(&(pmu_mmdc->pmu), name, -1); + platform_set_drvdata(pdev, pmu_mmdc); + if (ret) + goto pmu_register_err; + return 0; + +pmu_register_err: + pr_warn("MMDC Perf PMU failed (%d), disabled\n", ret); + hrtimer_cancel(&pmu_mmdc->hrtimer); + kfree(pmu_mmdc); + return ret; +} + +#else +#define imx_mmdc_remove NULL +#define imx_mmdc_perf_init(pdev, mmdc_base) 0 +#endif + static int imx_mmdc_probe(struct platform_device *pdev) { struct device_node *np = pdev->dev.of_node; @@ -62,7 +520,7 @@ static int imx_mmdc_probe(struct platform_device *pdev) return -EBUSY; } - return 0; + return imx_mmdc_perf_init(pdev, mmdc_base); } int imx_mmdc_get_ddr_type(void) @@ -81,6 +539,7 @@ static struct platform_driver imx_mmdc_driver = { .of_match_table = imx_mmdc_dt_ids, }, .probe = imx_mmdc_probe, + .remove = imx_mmdc_remove, }; static int __init imx_mmdc_init(void)