From patchwork Tue Mar 10 15:19:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 11429623 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A249138D for ; Tue, 10 Mar 2020 15:20:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DF69021655 for ; Tue, 10 Mar 2020 15:20:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bhReD04x" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727731AbgCJPUS (ORCPT ); Tue, 10 Mar 2020 11:20:18 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:40933 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727361AbgCJPUS (ORCPT ); Tue, 10 Mar 2020 11:20:18 -0400 Received: by mail-wm1-f65.google.com with SMTP id e26so1797973wme.5; Tue, 10 Mar 2020 08:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kOW4sWg6Q7VGPDYbJbyB7HbqHDmbBp1ycDkxRC8YVxA=; b=bhReD04xb47VJuOb863x1yRYJpYpSJLzDmqHmONtE6BqjWWb1xza3YQXQzUVanb+1v VOLgMp4HmmsQUxcv7PS4j3j0OmfQ7BA20kY4jXEDTtSOI5rDkDFFz7hlgBCGAp4na+gg vmph4Iq71vvISazrB8QEYdHb9f2wMNpCH/LgL6YAvu8UIvoRIlGZZML/sZT3MWSYawd3 /febHCnezEAF0RxtgPNBgkHEqvR8p84IecwAgC9RTNDtsrgbXqnC8LUX5e9eiRYqmlyg m/6c5LDGrO97bzB8SFtZk7jdMgGTwLmmbrvkWygBqRRR+haujBWSRajhaZqfcKDY6NkJ tYcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kOW4sWg6Q7VGPDYbJbyB7HbqHDmbBp1ycDkxRC8YVxA=; b=iaB11yXExUee4046emdiIzUZj56CJqfVK/2HrkISGASa9j11iC9FDXpiR/2cENYrkA GtbDEVHTT2Pp55lKuepC1p9lNYtGK75es4eeQzwsB895jc0ISiRSvqSWzT2H/1nFXz+8 HWnj6MufsKvl8efm4A8+L5rQpMgK8ZRXMlBBVQB8szDOAHvt9x5TpuEux36eZ8NYv89x z92Cl3u23F/u048sD9SgyFIlAaL+nbT2Gek7bwdeeHVndVJnv4ih4/tKvZ9SrhG04Qfd fUl/0lVnx6WEXXJJxrFscPp/e9oTrH+IIx4lqYc7d5lj5nZqBkijX6UgIf3eNq5+PIAj B0kw== X-Gm-Message-State: ANhLgQ3yICfLxm5srwI3ij55xAqhhktmqdcmT4ifi6GkLjUcm47musqS iDeAVpBCkbOh+qnnJFRZy+U= X-Google-Smtp-Source: ADFU+vu468Cq8+15bTySdCPrW5DR0zf2iX5vUsNk41jQPts1j/zhomX5PEvvoDppHlQBS0o81G+9wQ== X-Received: by 2002:a1c:964f:: with SMTP id y76mr2589173wmd.62.1583853613555; Tue, 10 Mar 2020 08:20:13 -0700 (PDT) Received: from localhost (pD9E516A9.dip0.t-ipconnect.de. [217.229.22.169]) by smtp.gmail.com with ESMTPSA id o9sm69181610wrw.20.2020.03.10.08.20.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 08:20:11 -0700 (PDT) From: Thierry Reding To: Thierry Reding Cc: Jon Hunter , Dmitry Osipenko , Rob Herring , Mark Rutland , Michael Turquette , Stephen Boyd , Joseph Lo , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 1/8] clk: tegra: Add PLLP_UD and PLLMB_UD for Tegra210 Date: Tue, 10 Mar 2020 16:19:56 +0100 Message-Id: <20200310152003.2945170-2-thierry.reding@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310152003.2945170-1-thierry.reding@gmail.com> References: <20200310152003.2945170-1-thierry.reding@gmail.com> MIME-Version: 1.0 Sender: linux-clk-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org From: Joseph Lo Introduce the low jitter path of PLLP and PLLMB which can be used as EMC clock source. Signed-off-by: Joseph Lo Signed-off-by: Thierry Reding --- drivers/clk/tegra/clk-tegra210.c | 11 +++++++++++ include/dt-bindings/clock/tegra210-car.h | 4 ++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/clk/tegra/clk-tegra210.c b/drivers/clk/tegra/clk-tegra210.c index 45d54ead30bc..f99647b4a71f 100644 --- a/drivers/clk/tegra/clk-tegra210.c +++ b/drivers/clk/tegra/clk-tegra210.c @@ -3161,6 +3161,17 @@ static void __init tegra210_pll_init(void __iomem *clk_base, clk_register_clkdev(clk, "pll_m_ud", NULL); clks[TEGRA210_CLK_PLL_M_UD] = clk; + /* PLLMB_UD */ + clk = clk_register_fixed_factor(NULL, "pll_mb_ud", "pll_mb", + CLK_SET_RATE_PARENT, 1, 1); + clk_register_clkdev(clk, "pll_mb_ud", NULL); + clks[TEGRA210_CLK_PLL_MB_UD] = clk; + + /* PLLP_UD */ + clk = clk_register_fixed_factor(NULL, "pll_p_ud", "pll_p", + 0, 1, 1); + clks[TEGRA210_CLK_PLL_P_UD] = clk; + /* PLLU_VCO */ if (!tegra210_init_pllu()) { clk = clk_register_fixed_rate(NULL, "pll_u_vco", "pll_ref", 0, diff --git a/include/dt-bindings/clock/tegra210-car.h b/include/dt-bindings/clock/tegra210-car.h index 7a8f10b9a66d..5c93b01156d4 100644 --- a/include/dt-bindings/clock/tegra210-car.h +++ b/include/dt-bindings/clock/tegra210-car.h @@ -351,8 +351,8 @@ #define TEGRA210_CLK_PLL_P_OUT_XUSB 317 #define TEGRA210_CLK_XUSB_SSP_SRC 318 #define TEGRA210_CLK_PLL_RE_OUT1 319 -/* 320 */ -/* 321 */ +#define TEGRA210_CLK_PLL_MB_UD 320 +#define TEGRA210_CLK_PLL_P_UD 321 #define TEGRA210_CLK_ISP 322 #define TEGRA210_CLK_PLL_A_OUT_ADSP 323 #define TEGRA210_CLK_PLL_A_OUT0_OUT_ADSP 324 From patchwork Tue Mar 10 15:19:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 11429621 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9813139A for ; Tue, 10 Mar 2020 15:20:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AA14721655 for ; Tue, 10 Mar 2020 15:20:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fvFIf0s8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726637AbgCJPUR (ORCPT ); Tue, 10 Mar 2020 11:20:17 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:36869 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726283AbgCJPUR (ORCPT ); Tue, 10 Mar 2020 11:20:17 -0400 Received: by mail-wr1-f68.google.com with SMTP id 6so16405523wre.4; Tue, 10 Mar 2020 08:20:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ac+20iayhp4+0FnX9pq9ZNATMk94e/yNZak2klt7liY=; b=fvFIf0s8tJ7nMhiwYoqjiydXgG4bC0KQkNRaca18ueKV6HcTx8FQWG2bUr/AoWqzOb liAZ/jX+9/3EwpFXNhNvOiw0lQ8fHtRLpaI2t1+rBynwu1tCErOb4rnRzxaLgJ8/9cXL ns9BNz5uEQiiGMYH0/zII8o1wX1zEeSwBb3NGhvdNXUeGYQFGy028PSKBUJh6YZnQNPx x4oJ/8u/I3nI0mNunh3KMhcRxN+7TNkSQyQH58tnLxozeNU7FwGLNbLOD4y0hjoIUWfe 3Xnn+J4RSYho0qTrwtHmNUThhEzEHS9EwL4/De4YlhadvPVuqMHb++c+WsU3TNqnA/hF BA5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ac+20iayhp4+0FnX9pq9ZNATMk94e/yNZak2klt7liY=; b=bQV4ouMOtr3DEnpNsiuHvtqZ7X8pVpP50a5xb+V0gxw2q+zVBiCCJpCFA4GRvpacHF oYimiS8FStmyJdFnpQndiQNHWR5uuzCfakmQ04IA84b8rSLrRQcNLMcbzy9dJFr+KPEM kz6eAPDnIgVUjA1Wdy5U2n3HoBxTJUf1A0eERYUemuyJD4WkWs1qGyy+V9c555CrU6nl yD5cX9zF9oO8psGE2W5A6G8E84lsFxylf1KlaB+Z7q6hBacqUxlol4NxEFlZeXmmigpo JNPxlu1U/GEk/7KSXnCphmbrDm475RaITQsKv21sHS7KHPdqg2r2P6agrzPnDGc9uXv8 OHWA== X-Gm-Message-State: ANhLgQ2ljYuODQ2vc2GfPJhYjWVMOXgnV88m3nnPai11cmFTaxjCaNoF xUvX3xLV17Ay6pHusAyNsLg= X-Google-Smtp-Source: ADFU+vv+5coWzF5H1z7jNZwrIOJw0ccmtqYOGmc9003dtqOsXVm8IKrBMBmxJaHP+yjocAhoIv6Hug== X-Received: by 2002:adf:8501:: with SMTP id 1mr29813544wrh.56.1583853615306; Tue, 10 Mar 2020 08:20:15 -0700 (PDT) Received: from localhost (pD9E516A9.dip0.t-ipconnect.de. [217.229.22.169]) by smtp.gmail.com with ESMTPSA id e22sm4369921wme.45.2020.03.10.08.20.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 08:20:14 -0700 (PDT) From: Thierry Reding To: Thierry Reding Cc: Jon Hunter , Dmitry Osipenko , Rob Herring , Mark Rutland , Michael Turquette , Stephen Boyd , Joseph Lo , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 2/8] clk: tegra: Export functions for EMC clock scaling Date: Tue, 10 Mar 2020 16:19:57 +0100 Message-Id: <20200310152003.2945170-3-thierry.reding@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310152003.2945170-1-thierry.reding@gmail.com> References: <20200310152003.2945170-1-thierry.reding@gmail.com> MIME-Version: 1.0 Sender: linux-clk-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org From: Joseph Lo Export functions to allow accessing the CAR register required by EMC clock scaling. These functions will be used to access the CAR register as part of the scaling sequence. Signed-off-by: Joseph Lo Signed-off-by: Thierry Reding --- Changes in v5: - remove tegra210_clk_emc_update_setting() which is no longer needed drivers/clk/tegra/clk-tegra210.c | 26 ++++++++++++++++++++++++++ include/linux/clk/tegra.h | 3 +++ 2 files changed, 29 insertions(+) diff --git a/drivers/clk/tegra/clk-tegra210.c b/drivers/clk/tegra/clk-tegra210.c index f99647b4a71f..0a5be781da60 100644 --- a/drivers/clk/tegra/clk-tegra210.c +++ b/drivers/clk/tegra/clk-tegra210.c @@ -37,6 +37,7 @@ #define CLK_SOURCE_LA 0x1f8 #define CLK_SOURCE_SDMMC2 0x154 #define CLK_SOURCE_SDMMC4 0x164 +#define CLK_SOURCE_EMC_DLL 0x664 #define PLLC_BASE 0x80 #define PLLC_OUT 0x84 @@ -227,6 +228,10 @@ #define RST_DFLL_DVCO 0x2f4 #define DVFS_DFLL_RESET_SHIFT 0 +#define CLK_RST_CONTROLLER_CLK_OUT_ENB_X_SET 0x284 +#define CLK_RST_CONTROLLER_CLK_OUT_ENB_X_CLR 0x288 +#define CLK_OUT_ENB_X_CLK_ENB_EMC_DLL BIT(14) + #define CLK_RST_CONTROLLER_RST_DEV_Y_SET 0x2a8 #define CLK_RST_CONTROLLER_RST_DEV_Y_CLR 0x2ac #define CPU_SOFTRST_CTRL 0x380 @@ -555,6 +560,27 @@ void tegra210_set_sata_pll_seq_sw(bool state) } EXPORT_SYMBOL_GPL(tegra210_set_sata_pll_seq_sw); +void tegra210_clk_emc_dll_enable(bool flag) +{ + u32 offset = flag ? CLK_RST_CONTROLLER_CLK_OUT_ENB_X_SET : + CLK_RST_CONTROLLER_CLK_OUT_ENB_X_CLR; + + writel_relaxed(CLK_OUT_ENB_X_CLK_ENB_EMC_DLL, clk_base + offset); +} +EXPORT_SYMBOL_GPL(tegra210_clk_emc_dll_enable); + +void tegra210_clk_emc_dll_update_setting(u32 emc_dll_src_value) +{ + writel_relaxed(emc_dll_src_value, clk_base + CLK_SOURCE_EMC_DLL); +} +EXPORT_SYMBOL_GPL(tegra210_clk_emc_dll_update_setting); + +void tegra210_clk_emc_update_setting(u32 emc_src_value) +{ + writel_relaxed(emc_src_value, clk_base + CLK_SOURCE_EMC); +} +EXPORT_SYMBOL_GPL(tegra210_clk_emc_update_setting); + static void tegra210_generic_mbist_war(struct tegra210_domain_mbist_war *mbist) { u32 val; diff --git a/include/linux/clk/tegra.h b/include/linux/clk/tegra.h index 2b1b35240074..5b0bdb413460 100644 --- a/include/linux/clk/tegra.h +++ b/include/linux/clk/tegra.h @@ -131,6 +131,9 @@ extern void tegra210_set_sata_pll_seq_sw(bool state); extern void tegra210_put_utmipll_in_iddq(void); extern void tegra210_put_utmipll_out_iddq(void); extern int tegra210_clk_handle_mbist_war(unsigned int id); +extern void tegra210_clk_emc_dll_enable(bool flag); +extern void tegra210_clk_emc_dll_update_setting(u32 emc_dll_src_value); +extern void tegra210_clk_emc_update_setting(u32 emc_src_value); struct clk; From patchwork Tue Mar 10 15:19:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 11429625 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15023138D for ; Tue, 10 Mar 2020 15:20:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BB9D621655 for ; Tue, 10 Mar 2020 15:20:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ohDGyV7v" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727751AbgCJPUU (ORCPT ); Tue, 10 Mar 2020 11:20:20 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:47068 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726283AbgCJPUU (ORCPT ); Tue, 10 Mar 2020 11:20:20 -0400 Received: by mail-wr1-f65.google.com with SMTP id n15so16353958wrw.13; Tue, 10 Mar 2020 08:20:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QwENgQCrbdtzWEqXCsJfBerr8q7ubNstw+yo2U/aY1M=; b=ohDGyV7vhg1s6VLyDKmDIjqEbdqA/hubKtwc/jD44iLMapqcmaHyPB8xki5Pi6tYeZ ZDAypD0E5EKeaRcCh1O2pEAyAziIrqH6LSwtatiuUFEGvVc5kJOmvRzh9YD2d4+GVXBn A5Tw+kZAGrLdXDYKf9pBrRdVvJMoKiqBwIHklDQsOVu2MIYwMTK/iPHcYgix5mZdD7pP TTtEHupL5G+o0QvPvC/gVdnpNF7A93UgaqI9yrp8MYOmDacIgZKPG8JmMZIsn9LIOIxs 3XUk52HqMQ596ZxdEwZ0cYIkFSbOWp2cIp6aPMUe69aDiQzd6ffWT1rLCBdKPf6byEFD lSzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QwENgQCrbdtzWEqXCsJfBerr8q7ubNstw+yo2U/aY1M=; b=X/SlksrZ+XKgJU2wnXip9QmCleOsdiOwDE+pMwr3kiz4bivfAYS1xra+SG581aPJy9 Vn6mWFJ+tLg2x3wCXX+n9Ml72BQ8VgzWwrbetJns/A4vivuS5Ht1q+9rOHlJW7qaX6Y0 ZZhjMBWi16Akmi9SuysMUgPJOduGYc7yc+kUe1SOabyan49rcbNWVcF/4kxozwTICq/e C93v9Mo7L/kaKjgcRyawlR0GT/fE+NEPQnaRzkGZUhNGFhDbkLcfA18nBhIlLsYuklgh x6JCowPPLreEORDkVklUofvf1kjn03o/6CmpCCfSVwpMxFDXCDRtow3OczRVIQfB0bp3 Ioig== X-Gm-Message-State: ANhLgQ0fQd+NpP37buCe59rHZ8pgStGpn/VyC3ZtvqcONSnKsi9aK9qL 3Zn6BxQBggJ6R6kBo+TrqNU= X-Google-Smtp-Source: ADFU+vsjIR1U4N7DQb89EUm0tpfhOF5ELvjBWzWgXFAcbO8uOaIsTdl8J5UkbrV8s7E5vxjwUqHCUQ== X-Received: by 2002:adf:ef45:: with SMTP id c5mr28446976wrp.302.1583853617246; Tue, 10 Mar 2020 08:20:17 -0700 (PDT) Received: from localhost (pD9E516A9.dip0.t-ipconnect.de. [217.229.22.169]) by smtp.gmail.com with ESMTPSA id l64sm4569704wmf.30.2020.03.10.08.20.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 08:20:16 -0700 (PDT) From: Thierry Reding To: Thierry Reding Cc: Jon Hunter , Dmitry Osipenko , Rob Herring , Mark Rutland , Michael Turquette , Stephen Boyd , Joseph Lo , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 3/8] clk: tegra: Implement Tegra210 EMC clock Date: Tue, 10 Mar 2020 16:19:58 +0100 Message-Id: <20200310152003.2945170-4-thierry.reding@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310152003.2945170-1-thierry.reding@gmail.com> References: <20200310152003.2945170-1-thierry.reding@gmail.com> MIME-Version: 1.0 Sender: linux-clk-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org From: Joseph Lo The EMC clock needs to carefully coordinate with the EMC controller programming to make sure external memory can be properly clocked. Do so by hooking up the EMC clock with an EMC provider that will specify which rates are supported by the EMC and provide a callback to use for setting the clock rate at the EMC. Based on work by Peter De Schrijver . Signed-off-by: Joseph Lo Signed-off-by: Thierry Reding --- Changes in v5: - major rework and cleanup drivers/clk/tegra/Makefile | 1 + drivers/clk/tegra/clk-tegra210-emc.c | 352 ++++++++++++++ drivers/clk/tegra/clk.h | 3 + drivers/memory/tegra/Kconfig | 10 + drivers/memory/tegra/Makefile | 1 + drivers/memory/tegra/tegra210-emc.c | 671 +++++++++++++++++++++++++++ drivers/memory/tegra/tegra210-emc.h | 156 +++++++ include/linux/clk/tegra.h | 23 + 8 files changed, 1217 insertions(+) create mode 100644 drivers/clk/tegra/clk-tegra210-emc.c create mode 100644 drivers/memory/tegra/tegra210-emc.c create mode 100644 drivers/memory/tegra/tegra210-emc.h diff --git a/drivers/clk/tegra/Makefile b/drivers/clk/tegra/Makefile index 6f6bc2f0aa43..6df344331bf1 100644 --- a/drivers/clk/tegra/Makefile +++ b/drivers/clk/tegra/Makefile @@ -26,5 +26,6 @@ obj-$(CONFIG_TEGRA_CLK_EMC) += clk-tegra124-emc.o obj-$(CONFIG_ARCH_TEGRA_132_SOC) += clk-tegra124.o obj-y += cvb.o obj-$(CONFIG_ARCH_TEGRA_210_SOC) += clk-tegra210.o +obj-$(CONFIG_ARCH_TEGRA_210_SOC) += clk-tegra210-emc.o obj-$(CONFIG_CLK_TEGRA_BPMP) += clk-bpmp.o obj-y += clk-utils.o diff --git a/drivers/clk/tegra/clk-tegra210-emc.c b/drivers/clk/tegra/clk-tegra210-emc.c new file mode 100644 index 000000000000..3eb58c37d6c0 --- /dev/null +++ b/drivers/clk/tegra/clk-tegra210-emc.c @@ -0,0 +1,352 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2015-2020, NVIDIA CORPORATION. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include + +#define CLK_SOURCE_EMC 0x19c +#define CLK_SOURCE_EMC_2X_CLK_SRC_SHIFT 29 +#define CLK_SOURCE_EMC_2X_CLK_SRC_MASK 0x7 +#define CLK_SOURCE_EMC_MC_EMC_SAME_FREQ BIT(16) +#define CLK_SOURCE_EMC_2X_CLK_DIVISOR_SHIFT 0 +#define CLK_SOURCE_EMC_2X_CLK_DIVISOR_MASK 0xff + +#define CLK_SRC_PLLM 0 +#define CLK_SRC_PLLC 1 +#define CLK_SRC_PLLP 2 +#define CLK_SRC_CLK_M 3 +#define CLK_SRC_PLLM_UD 4 +#define CLK_SRC_PLLMB_UD 5 +#define CLK_SRC_PLLMB 6 +#define CLK_SRC_PLLP_UD 7 + +struct tegra210_clk_emc { + struct clk_hw hw; + void __iomem *regs; + + struct tegra210_clk_emc_provider *provider; + + struct clk *parents[8]; +}; + +static inline struct tegra210_clk_emc * +to_tegra210_clk_emc(struct clk_hw *hw) +{ + return container_of(hw, struct tegra210_clk_emc, hw); +} + +static const char *tegra210_clk_emc_parents[] = { + "pll_m", "pll_c", "pll_p", "clk_m", "pll_m_ud", "pll_mb_ud", + "pll_mb", "pll_p_ud", +}; + +static u8 tegra210_clk_emc_get_parent(struct clk_hw *hw) +{ + struct tegra210_clk_emc *emc = to_tegra210_clk_emc(hw); + u32 value; + u8 src; + + value = readl_relaxed(emc->regs + CLK_SOURCE_EMC); + src = (value >> CLK_SOURCE_EMC_2X_CLK_SRC_SHIFT) & + CLK_SOURCE_EMC_2X_CLK_SRC_MASK; + + return src; +} + +static unsigned long tegra210_clk_emc_recalc_rate(struct clk_hw *hw, + unsigned long parent_rate) +{ + struct tegra210_clk_emc *emc = to_tegra210_clk_emc(hw); + u32 value, div; + + value = readl_relaxed(emc->regs + CLK_SOURCE_EMC); + + div = (value >> CLK_SOURCE_EMC_2X_CLK_DIVISOR_SHIFT) & + CLK_SOURCE_EMC_2X_CLK_DIVISOR_MASK; + div += 2; + + return DIV_ROUND_UP(parent_rate * 2, div); +} + +static long tegra210_clk_emc_round_rate(struct clk_hw *hw, unsigned long rate, + unsigned long *prate) +{ + struct tegra210_clk_emc *emc = to_tegra210_clk_emc(hw); + struct tegra210_clk_emc_provider *provider = emc->provider; + unsigned int i; + + if (!provider || !provider->configs || provider->num_configs == 0) + return clk_hw_get_rate(hw); + + for (i = 0; i < provider->num_configs; i++) { + if (provider->configs[i].rate >= rate) + return provider->configs[i].rate; + } + + return provider->configs[i - 1].rate; +} + +static struct clk *tegra210_clk_emc_find_parent(struct tegra210_clk_emc *emc, + u8 index) +{ + struct clk_hw *parent = clk_hw_get_parent_by_index(&emc->hw, index); + const char *name = clk_hw_get_name(parent); + + /* XXX implement cache? */ + + return __clk_lookup(name); +} + +static int tegra210_clk_emc_set_rate(struct clk_hw *hw, unsigned long rate, + unsigned long parent_rate) +{ + struct tegra210_clk_emc *emc = to_tegra210_clk_emc(hw); + struct tegra210_clk_emc_provider *provider = emc->provider; + struct tegra210_clk_emc_config *config; + struct device *dev = provider->dev; + struct clk_hw *old, *new, *parent; + u8 old_idx, new_idx, index; + struct clk *clk; + unsigned int i; + int err; + + if (!provider || !provider->configs || provider->num_configs == 0) + return -EINVAL; + + for (i = 0; i < provider->num_configs; i++) { + if (provider->configs[i].rate >= rate) { + config = &provider->configs[i]; + break; + } + } + + if (i == provider->num_configs) + config = &provider->configs[i - 1]; + + old_idx = tegra210_clk_emc_get_parent(hw); + new_idx = (config->value >> CLK_SOURCE_EMC_2X_CLK_SRC_SHIFT) & + CLK_SOURCE_EMC_2X_CLK_SRC_MASK; + + old = clk_hw_get_parent_by_index(hw, old_idx); + new = clk_hw_get_parent_by_index(hw, new_idx); + + /* if the rate has changed... */ + if (config->parent_rate != clk_hw_get_rate(old)) { + /* ... but the clock source remains the same ... */ + if (new_idx == old_idx) { + /* ... switch to the alternative clock source. */ + switch (new_idx) { + case CLK_SRC_PLLM: + new_idx = CLK_SRC_PLLMB; + break; + + case CLK_SRC_PLLM_UD: + new_idx = CLK_SRC_PLLMB_UD; + break; + + case CLK_SRC_PLLMB_UD: + new_idx = CLK_SRC_PLLM_UD; + break; + + case CLK_SRC_PLLMB: + new_idx = CLK_SRC_PLLM; + break; + } + + /* + * This should never happen because we can't deal with + * it. + */ + if (WARN_ON(new_idx == old_idx)) + return -EINVAL; + + new = clk_hw_get_parent_by_index(hw, new_idx); + } + + index = new_idx; + parent = new; + } else { + index = old_idx; + parent = old; + } + + clk = tegra210_clk_emc_find_parent(emc, index); + if (IS_ERR(clk)) { + err = PTR_ERR(clk); + dev_err(dev, "failed to get parent clock for index %u: %d\n", + index, err); + return err; + } + + /* set the new parent clock to the required rate */ + if (clk_get_rate(clk) != config->parent_rate) { + err = clk_set_rate(clk, config->parent_rate); + if (err < 0) { + dev_err(dev, "failed to set rate %lu Hz for %pC: %d\n", + config->parent_rate, clk, err); + return err; + } + } + + /* enable the new parent clock */ + if (parent != old) { + err = clk_prepare_enable(clk); + if (err < 0) { + dev_err(dev, "failed to enable parent clock %pC: %d\n", + clk, err); + return err; + } + } + + /* update the EMC source configuration to reflect the new parent */ + config->value &= ~(CLK_SOURCE_EMC_2X_CLK_SRC_MASK << + CLK_SOURCE_EMC_2X_CLK_SRC_SHIFT); + config->value |= (index & CLK_SOURCE_EMC_2X_CLK_SRC_MASK) << + CLK_SOURCE_EMC_2X_CLK_SRC_SHIFT; + + /* + * Finally, switch the EMC programming with both old and new parent + * clocks enabled. + */ + err = provider->set_rate(dev, config); + if (err < 0) { + dev_err(dev, "failed to set EMC rate to %lu Hz: %d\n", rate, + err); + + /* + * If we're unable to switch to the new EMC frequency, we no + * longer need the new parent to be enabled. + */ + if (parent != old) + clk_disable_unprepare(clk); + + return err; + } + + /* reparent to new parent clock and disable the old parent clock */ + if (parent != old) { + clk = tegra210_clk_emc_find_parent(emc, old_idx); + if (IS_ERR(clk)) { + err = PTR_ERR(clk); + dev_err(dev, + "failed to get parent clock for index %u: %d\n", + old_idx, err); + return err; + } + + clk_hw_reparent(hw, parent); + clk_disable_unprepare(clk); + } + + return err; +} + +static const struct clk_ops tegra210_clk_emc_ops = { + .get_parent = tegra210_clk_emc_get_parent, + .recalc_rate = tegra210_clk_emc_recalc_rate, + .round_rate = tegra210_clk_emc_round_rate, + .set_rate = tegra210_clk_emc_set_rate, +}; + +struct clk *tegra210_clk_register_emc(struct device_node *np, + void __iomem *regs) +{ + struct tegra210_clk_emc *emc; + struct clk_init_data init; + struct clk *clk; + + emc = kzalloc(sizeof(*emc), GFP_KERNEL); + if (!emc) + return ERR_PTR(-ENOMEM); + + emc->regs = regs; + + init.name = "emc"; + init.ops = &tegra210_clk_emc_ops; + init.flags = CLK_IS_CRITICAL | CLK_GET_RATE_NOCACHE; + init.parent_names = tegra210_clk_emc_parents; + init.num_parents = ARRAY_SIZE(tegra210_clk_emc_parents); + emc->hw.init = &init; + + clk = clk_register(NULL, &emc->hw); + if (IS_ERR(clk)) { + kfree(emc); + return clk; + } + + return clk; +} + +int tegra210_clk_emc_attach(struct clk *clk, + struct tegra210_clk_emc_provider *provider) +{ + struct clk_hw *hw = __clk_get_hw(clk); + struct tegra210_clk_emc *emc = to_tegra210_clk_emc(hw); + struct device *dev = provider->dev; + unsigned int i; + + for (i = 0; i < provider->num_configs; i++) { + struct tegra210_clk_emc_config *config = &provider->configs[i]; + struct clk_hw *parent; + bool same_freq; + u8 src, div; + + src = (config->value >> CLK_SOURCE_EMC_2X_CLK_SRC_SHIFT) & + CLK_SOURCE_EMC_2X_CLK_SRC_MASK; + div = (config->value >> CLK_SOURCE_EMC_2X_CLK_DIVISOR_SHIFT) & + CLK_SOURCE_EMC_2X_CLK_DIVISOR_MASK; + + /* do basic sanity checking on the EMC timings */ + if (div & 0x1) { + dev_err(dev, "invalid odd divider %u for rate %lu Hz\n", + div, config->rate); + return -EINVAL; + } + + same_freq = config->value & CLK_SOURCE_EMC_MC_EMC_SAME_FREQ; + + if (same_freq != config->same_freq) { + dev_err(dev, + "ambiguous EMC to MC ratio for rate %lu Hz\n", + config->rate); + return -EINVAL; + } + + parent = clk_hw_get_parent_by_index(hw, src); + config->parent = src; + + if (src == CLK_SRC_PLLM || src == CLK_SRC_PLLM_UD) { + config->parent_rate = config->rate * (1 + div / 2); + } else { + unsigned long rate = config->rate * (1 + div / 2); + + config->parent_rate = clk_hw_get_rate(parent); + + if (config->parent_rate != rate) { + dev_err(dev, + "rate %lu Hz does not match input\n", + config->rate); + return -EINVAL; + } + } + } + + emc->provider = provider; + + return 0; +} +EXPORT_SYMBOL_GPL(tegra210_clk_emc_attach); + +void tegra210_clk_emc_detach(struct clk *clk) +{ + struct tegra210_clk_emc *emc = to_tegra210_clk_emc(__clk_get_hw(clk)); + + emc->provider = NULL; +} +EXPORT_SYMBOL_GPL(tegra210_clk_emc_detach); diff --git a/drivers/clk/tegra/clk.h b/drivers/clk/tegra/clk.h index 416a6b09f6a3..d196427d2cc0 100644 --- a/drivers/clk/tegra/clk.h +++ b/drivers/clk/tegra/clk.h @@ -908,4 +908,7 @@ void tegra_clk_periph_resume(void); bool tegra20_clk_emc_driver_available(struct clk_hw *emc_hw); struct clk *tegra20_clk_register_emc(void __iomem *ioaddr, bool low_jitter); +struct clk *tegra210_clk_register_emc(struct device_node *np, + void __iomem *regs); + #endif /* TEGRA_CLK_H */ diff --git a/drivers/memory/tegra/Kconfig b/drivers/memory/tegra/Kconfig index fbfbaada61a2..a319f4df6126 100644 --- a/drivers/memory/tegra/Kconfig +++ b/drivers/memory/tegra/Kconfig @@ -36,3 +36,13 @@ config TEGRA124_EMC Tegra124 chips. The EMC controls the external DRAM on the board. This driver is required to change memory timings / clock rate for external memory. + +config TEGRA210_EMC + bool "NVIDIA Tegra210 External Memory Controller driver" + default y + depends on TEGRA_MC && ARCH_TEGRA_210_SOC + help + This driver is for the External Memory Controller (EMC) found on + Tegra210 chips. The EMC controls the external DRAM on the board. + This driver is required to change memory timings / clock rate for + external memory. diff --git a/drivers/memory/tegra/Makefile b/drivers/memory/tegra/Makefile index 529d10bc5650..1c59150ccf58 100644 --- a/drivers/memory/tegra/Makefile +++ b/drivers/memory/tegra/Makefile @@ -13,5 +13,6 @@ obj-$(CONFIG_TEGRA_MC) += tegra-mc.o obj-$(CONFIG_TEGRA20_EMC) += tegra20-emc.o obj-$(CONFIG_TEGRA30_EMC) += tegra30-emc.o obj-$(CONFIG_TEGRA124_EMC) += tegra124-emc.o +obj-$(CONFIG_TEGRA210_EMC) += tegra210-emc.o obj-$(CONFIG_ARCH_TEGRA_186_SOC) += tegra186.o tegra186-emc.o obj-$(CONFIG_ARCH_TEGRA_194_SOC) += tegra186.o tegra186-emc.o diff --git a/drivers/memory/tegra/tegra210-emc.c b/drivers/memory/tegra/tegra210-emc.c new file mode 100644 index 000000000000..80ea14d1e6ce --- /dev/null +++ b/drivers/memory/tegra/tegra210-emc.c @@ -0,0 +1,671 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2015-2019, NVIDIA CORPORATION. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "mc.h" +#include "tegra210-emc.h" + +#define CLK_RST_CONTROLLER_CLK_SOURCE_EMC 0x19c +#define EMC_CLK_EMC_2X_CLK_SRC_SHIFT 29 +#define EMC_CLK_EMC_2X_CLK_SRC_MASK \ + (0x7 << EMC_CLK_EMC_2X_CLK_SRC_SHIFT) +#define EMC_CLK_MC_EMC_SAME_FREQ BIT(16) +#define EMC_CLK_EMC_2X_CLK_DIVISOR_SHIFT 0 +#define EMC_CLK_EMC_2X_CLK_DIVISOR_MASK \ + (0xff << EMC_CLK_EMC_2X_CLK_DIVISOR_SHIFT) + +#define MC_EMEM_ARB_MISC0_EMC_SAME_FREQ BIT(27) + +#define TEGRA_EMC_MAX_FREQS 16 +#define TEGRA210_EMC_SUSPEND_RATE 204000000 + +#define CLK_CHANGE_DELAY 100 +#define TRAINING_TIME 100 + +enum { + TEGRA_EMC_SRC_PLLM, + TEGRA_EMC_SRC_PLLC, + TEGRA_EMC_SRC_PLLP, + TEGRA_EMC_SRC_CLKM, + TEGRA_EMC_SRC_PLLM_UD, + TEGRA_EMC_SRC_PLLMB_UD, + TEGRA_EMC_SRC_PLLMB, + TEGRA_EMC_SRC_PLLP_UD, + TEGRA_EMC_SRC_COUNT, +}; + +struct emc_sel { + struct clk *input; + u32 value; + unsigned long input_rate; + + struct clk *input_b; // second source of PLLM: PLLMB + u32 value_b; + unsigned long input_rate_b; +}; + +static struct emc_sel *emc_clk_sel; +static struct clk *emc_src[TEGRA_EMC_SRC_COUNT]; +static const char *emc_src_names[TEGRA_EMC_SRC_COUNT] = { + [TEGRA_EMC_SRC_PLLM] = "pll_m", + [TEGRA_EMC_SRC_PLLC] = "pll_c", + [TEGRA_EMC_SRC_PLLP] = "pll_p", + [TEGRA_EMC_SRC_CLKM] = "clk_m", + [TEGRA_EMC_SRC_PLLM_UD] = "pll_m_ud", + [TEGRA_EMC_SRC_PLLMB_UD] = "pll_mb_ud", + [TEGRA_EMC_SRC_PLLMB] = "pll_mb", + [TEGRA_EMC_SRC_PLLP_UD] = "pll_p_ud", +}; + +static const struct supported_sequence supported_seqs[] = { + { + 0, + NULL, + NULL, + NULL + } +}; +static const struct supported_sequence *seq = supported_seqs; +static DEFINE_SPINLOCK(emc_access_lock); + +static inline struct tegra_emc *clk_hw_to_emc(struct clk_hw *hw) +{ + return container_of(hw, struct tegra_emc, hw); +} + +u32 emc_readl(struct tegra_emc *emc, unsigned long offset) +{ + return readl_relaxed(emc->emc_base[REG_EMC] + offset); +} + +u32 emc_readl_per_ch(struct tegra_emc *emc, int type, + unsigned long offset) +{ + u32 val = 0; + + switch (type) { + case REG_EMC: + case REG_EMC0: + val = readl_relaxed(emc->emc_base[REG_EMC] + offset); + break; + case REG_EMC1: + val = readl_relaxed(emc->emc_base[REG_EMC1] + offset); + break; + } + + return val; +} + +static inline u32 emc_src_val(u32 val) +{ + return (val & EMC_CLK_EMC_2X_CLK_SRC_MASK) >> + EMC_CLK_EMC_2X_CLK_SRC_SHIFT; +} + +static inline u32 emc_div_val(u32 val) +{ + return (val & EMC_CLK_EMC_2X_CLK_DIVISOR_MASK) >> + EMC_CLK_EMC_2X_CLK_DIVISOR_SHIFT; +} + +static void emc_train_func(struct timer_list *tmr) +{ + unsigned long flags; + struct tegra_emc *emc = from_timer(emc, tmr, training_timer); + + if (!emc->current_timing) + return; + + spin_lock_irqsave(&emc_access_lock, flags); + if (seq->periodic_compensation) + seq->periodic_compensation(emc); + spin_unlock_irqrestore(&emc_access_lock, flags); + + mod_timer(&emc->training_timer, + jiffies + msecs_to_jiffies(emc->timer_period_training)); +} + +static void emc_training_timer_start(struct tegra_emc *emc) +{ + mod_timer(&emc->training_timer, + jiffies + msecs_to_jiffies(emc->timer_period_training)); +} + +static void emc_training_timer_stop(struct tegra_emc *emc) +{ + del_timer(&emc->training_timer); +} + +static void emc_set_clock(struct tegra_emc *emc, u32 clksrc) +{ + seq->set_clock(emc, clksrc); + + if (emc->next_timing->periodic_training) + emc_training_timer_start(emc); + else + emc_training_timer_stop(emc); +} + +static inline unsigned long emc_get_src_clk_rate(void) +{ + int div; + u32 val; + unsigned long rate; + + val = tegra210_clk_emc_get_setting(); + rate = clk_get_rate(emc_src[emc_src_val(val)]); + div = emc_div_val(val); + div += 2; + rate *= 2; + rate += div - 1; + do_div(rate, div); + + return rate; +} + +static int emc_table_lookup(struct tegra_emc *emc, unsigned long rate) +{ + int i; + + for (i = 0; i < emc->emc_table_size; i++) { + if (emc_clk_sel[i].input == NULL) + continue; + + if (emc->emc_table[i].rate == rate) + return i; + } + + return -EINVAL; +} + +static struct clk *emc_predict_parent(struct tegra_emc *emc, + unsigned long rate) +{ + struct clk *old_parent, *new_parent; + unsigned long parent_rate; + int idx; + + idx = emc_table_lookup(emc, rate / 1000); + if (idx < 0) + return ERR_PTR(-EINVAL); + + parent_rate = emc_clk_sel[idx].input_rate * 1000; + new_parent = emc_clk_sel[idx].input; + old_parent = clk_get_parent(emc->emc_clk); + + if (parent_rate == clk_get_rate(old_parent)) + return old_parent; + + if (clk_is_match(new_parent, old_parent)) + new_parent = emc_clk_sel[idx].input_b; + + if (parent_rate != clk_get_rate(new_parent)) + clk_set_rate(new_parent, parent_rate); + + return new_parent; +} + +static int emc_set_rate(struct tegra_emc *emc, unsigned long rate) +{ + int i; + unsigned long flags; + s64 last_change_delay; + struct clk *parent; + + if (emc->emc_suspend) + rate = TEGRA210_EMC_SUSPEND_RATE; + + if (rate == emc->current_timing->rate) + return 0; + + i = emc_table_lookup(emc, rate / 1000); + + if (i < 0) + return i; + + if (rate > 204000000 && !emc->emc_table[i].trained) + return -EINVAL; + + parent = emc_predict_parent(emc, rate); + if (clk_is_match(parent, emc_clk_sel[i].input)) + emc->clk_setting = emc_clk_sel[i].value; + else + emc->clk_setting = emc_clk_sel[i].value_b; + + emc->next_timing = &emc->emc_table[i]; + last_change_delay = ktime_us_delta(ktime_get(), emc->clkchange_time); + if ((last_change_delay >= 0) && + (last_change_delay < emc->clkchange_delay)) + udelay(emc->clkchange_delay - (int)last_change_delay); + + spin_lock_irqsave(&emc_access_lock, flags); + emc_set_clock(emc, emc->clk_setting); + emc->clkchange_time = ktime_get(); + emc->current_timing = &emc->emc_table[i]; + spin_unlock_irqrestore(&emc_access_lock, flags); + + return 0; +} + +#ifdef CONFIG_DEBUG_FS +static int debug_emc_get_rate(void *data, u64 *val) +{ + struct clk *c = data; + + *val = clk_get_rate(c); + + return 0; +} + +static int debug_emc_set_rate(void *data, u64 val) +{ + struct clk *c = data; + + return clk_set_rate(c, val); +} +DEFINE_SIMPLE_ATTRIBUTE(emc_rate_fops, debug_emc_get_rate, + debug_emc_set_rate, "%llu\n"); + +static int tegra_emc_debug_init(struct tegra_emc *emc) +{ + struct dentry *emc_debugfs_root; + + emc_debugfs_root = debugfs_create_dir("tegra_emc", NULL); + if (!emc_debugfs_root) + return -ENOMEM; + + if (!debugfs_create_file("rate", 0644, emc_debugfs_root, emc->emc_clk, + &emc_rate_fops)) + goto err_out; + + return 0; + +err_out: + debugfs_remove_recursive(emc_debugfs_root); + return -ENOMEM; +} +#endif /* CONFIG_DEBUG_FS */ + +static u8 clk_emc_get_parent(struct clk_hw *hw) +{ + struct tegra_emc *emc = clk_hw_to_emc(hw); + + if (!emc->clk_setting) + emc->clk_setting = tegra210_clk_emc_get_setting(); + + return emc_src_val(emc->clk_setting); +} + +static unsigned long clk_emc_recalc_rate(struct clk_hw *hw, + unsigned long parent_rate) +{ + struct tegra_emc *emc = clk_hw_to_emc(hw); + + if (!emc->emc_table_size || !seq) { + u32 emc_setting = tegra210_clk_emc_get_setting(); + + return clk_get_rate(emc_src[emc_src_val(emc_setting)]); + } + + return emc->current_timing->rate * 1000; +} + +static long clk_emc_round_rate(struct clk_hw *hw, unsigned long rate, + unsigned long *prate) +{ + struct tegra_emc *emc = clk_hw_to_emc(hw); + int i; + + if (!emc->emc_table_size || !seq) { + u32 emc_setting = tegra210_clk_emc_get_setting(); + return clk_get_rate(emc_src[emc_src_val(emc_setting)]); + } + + if (emc->emc_suspend) + return TEGRA210_EMC_SUSPEND_RATE; + + rate /= 1000; + + for (i = 0; i < emc->emc_table_size; i++) { + if (emc->emc_table[i].rate >= rate) + return emc->emc_table[i].rate * 1000; + } + + return emc->emc_table[i - 1].rate * 1000; +} + +static int clk_emc_set_rate(struct clk_hw *hw, unsigned long rate, + unsigned long parent_rate) +{ + struct tegra_emc *emc = clk_hw_to_emc(hw); + struct clk *old_parent, *new_parent; + int ret = -EINVAL; + + if (!emc->emc_table_size || !seq) + return ret; + + if (emc->emc_suspend) + rate = TEGRA210_EMC_SUSPEND_RATE; + + old_parent = clk_get_parent(hw->clk); + new_parent = emc_predict_parent(emc, rate); + if (IS_ERR(new_parent)) + goto out; + + if (!clk_is_match(new_parent, old_parent)) + clk_prepare_enable(new_parent); + + ret = emc_set_rate(emc, rate); + if (ret) { + if (new_parent != old_parent) + clk_disable_unprepare(new_parent); + goto out; + } + + if (!clk_is_match(new_parent, old_parent)) { + clk_hw_reparent(hw, __clk_get_hw(new_parent)); + clk_disable_unprepare(old_parent); + } + +out: + return ret; +} + +static const struct clk_ops tegra_clk_emc_ops = { + .get_parent = clk_emc_get_parent, + .recalc_rate = clk_emc_recalc_rate, + .round_rate = clk_emc_round_rate, + .set_rate = clk_emc_set_rate, +}; + +static int find_matching_input(struct emc_table *table, struct emc_sel *sel) +{ + u32 div_value; + u32 src_value; + unsigned long input_rate = 0; + struct clk *input_clk; + + div_value = emc_div_val(table->clk_src_emc); + src_value = emc_src_val(table->clk_src_emc); + + if (div_value & 0x1) { + pr_warn("Tegra EMC: invalid odd divider for EMC rate %u\n", + table->rate); + return -EINVAL; + } + + if (!(table->clk_src_emc & EMC_CLK_MC_EMC_SAME_FREQ) != + !(MC_EMEM_ARB_MISC0_EMC_SAME_FREQ & + table->burst_mc_regs[MC_EMEM_ARB_MISC0_INDEX])) { + pr_warn("Tegra EMC: ambiguous EMC to MC ratio for rate %u\n", + table->rate); + return -EINVAL; + } + + input_clk = emc_src[src_value]; + if (input_clk == emc_src[TEGRA_EMC_SRC_PLLM] + || input_clk == emc_src[TEGRA_EMC_SRC_PLLM_UD]) { + input_rate = table->rate * (1 + div_value / 2); + } else { + input_rate = clk_get_rate(input_clk) / 1000; + if (input_rate != (table->rate * (1 + div_value / 2))) { + pr_warn("Tegra EMC: rate %u doesn't match input\n", + table->rate); + return -EINVAL; + } + } + + sel->input = input_clk; + sel->input_rate = input_rate; + sel->value = table->clk_src_emc; + sel->input_b = input_clk; + sel->input_rate_b = input_rate; + sel->value_b = table->clk_src_emc; + + if (input_clk == emc_src[TEGRA_EMC_SRC_PLLM]) { + sel->input_b = emc_src[TEGRA_EMC_SRC_PLLMB]; + sel->value_b = table->clk_src_emc & + ~EMC_CLK_EMC_2X_CLK_SRC_MASK; + sel->value_b |= TEGRA_EMC_SRC_PLLMB << + EMC_CLK_EMC_2X_CLK_SRC_SHIFT; + } + + if (input_clk == emc_src[TEGRA_EMC_SRC_PLLM_UD]) { + sel->input_b = emc_src[TEGRA_EMC_SRC_PLLMB_UD]; + sel->value_b = table->clk_src_emc & + ~EMC_CLK_EMC_2X_CLK_SRC_MASK; + sel->value_b |= TEGRA_EMC_SRC_PLLMB_UD << + EMC_CLK_EMC_2X_CLK_SRC_SHIFT; + } + + return 0; +} + +static int tegra210_emc_probe(struct platform_device *pdev) +{ + int i; + unsigned long table_rate; + unsigned long current_rate; + struct clk *emc_clk; + struct device_node *np; + struct platform_device *mc; + struct resource res; + struct tegra_emc *emc; + void *table_addr; + + emc_clk = devm_clk_get(&pdev->dev, "emc"); + if (IS_ERR(emc_clk)) + return PTR_ERR(emc_clk); + emc = clk_hw_to_emc(__clk_get_hw(emc_clk)); + + np = of_parse_phandle(pdev->dev.of_node, "nvidia,memory-controller", 0); + if (!np) { + dev_err(&pdev->dev, "could not get memory controller\n"); + return -ENOENT; + } + + mc = of_find_device_by_node(np); + of_node_put(np); + if (!mc) + return -ENOENT; + + emc->mc = platform_get_drvdata(mc); + if (!emc->mc) + return -EPROBE_DEFER; + + emc->emc_base[REG_EMC] = devm_platform_ioremap_resource(pdev, 0); + emc->emc_base[REG_EMC0] = devm_platform_ioremap_resource(pdev, 1); + emc->emc_base[REG_EMC1] = devm_platform_ioremap_resource(pdev, 2); + + for (i = 0; i < TEGRA_EMC_SRC_COUNT; i++) { + if (!IS_ERR(emc_src[i])) + clk_put(emc_src[i]); + + emc_src[i] = devm_clk_get(&pdev->dev, emc_src_names[i]); + if (IS_ERR(emc_src[i])) { + dev_err(&pdev->dev, "Can not find EMC source clock\n"); + return -ENODATA; + } + } + + np = of_parse_phandle(pdev->dev.of_node, "memory-region", 0); + if (!np) { + dev_err(&pdev->dev, "could not find EMC table\n"); + return -ENODATA; + } + + if (!of_device_is_compatible(np, "nvidia,tegra210-emc-table") || + !of_device_is_available(np)) { + dev_err(&pdev->dev, "EMC table is invalid\n"); + return -ENODATA; + } + + of_address_to_resource(np, 0, &res); + table_addr = memremap(res.start, resource_size(&res), MEMREMAP_WB); + of_node_put(np); + if (!table_addr) { + dev_err(&pdev->dev, "could not map EMC table\n"); + return -ENOMEM; + } + emc->emc_table = (struct emc_table *)table_addr; + + for (i = 0; i < TEGRA_EMC_MAX_FREQS; i++) + if (emc->emc_table[i].rev != 0) + emc->emc_table_size++; + else + break; + + /* check the supported sequence */ + while (seq->table_rev) { + if (seq->table_rev == emc->emc_table[0].rev) + break; + seq++; + } + if (!seq->set_clock) { + seq = NULL; + dev_err(&pdev->dev, "Invalid EMC sequence for table Rev. %d\n", + emc->emc_table[0].rev); + return -ENODATA; + } + + emc_clk_sel = devm_kcalloc(&pdev->dev, + emc->emc_table_size, + sizeof(struct emc_sel), + GFP_KERNEL); + + /* calculate the rate from source clock */ + current_rate = emc_get_src_clk_rate() / 1000; + + /* validate the table */ + for (i = 0; i < emc->emc_table_size; i++) { + table_rate = emc->emc_table[i].rate; + if (!table_rate) + continue; + + if (i && ((table_rate <= emc->emc_table[i-1].rate) || + (emc->emc_table[i].min_volt < + emc->emc_table[i-1].min_volt))) + continue; + + if (emc->emc_table[i].rev != emc->emc_table[0].rev) + continue; + + if (find_matching_input(&emc->emc_table[i], &emc_clk_sel[i])) + continue; + + if (table_rate == current_rate) + emc->current_timing = &emc->emc_table[i]; + } + + emc->clk_setting = tegra210_clk_emc_get_setting(); + emc->clkchange_delay = CLK_CHANGE_DELAY; + emc->timer_period_training = TRAINING_TIME; + emc->dev = &pdev->dev; + dev_set_drvdata(emc->dev, emc); + + /* EMC training timer */ + timer_setup(&emc->training_timer, emc_train_func, 0); + +#ifdef CONFIG_DEBUG_FS + tegra_emc_debug_init(emc); +#endif + + return 0; +} + +struct clk *tegra210_clk_register_emc(void) +{ + struct clk_init_data init; + struct clk *clk; + struct tegra_emc *emc; + int i; + + emc = kzalloc(sizeof(*emc), GFP_KERNEL); + if (!emc) + return ERR_PTR(-ENOMEM); + + for (i = 0; i < TEGRA_EMC_SRC_COUNT; i++) + emc_src[i] = clk_get_sys(NULL, emc_src_names[i]); + + init.name = "emc"; + init.ops = &tegra_clk_emc_ops; + init.flags = CLK_IS_CRITICAL | CLK_GET_RATE_NOCACHE; + init.parent_names = emc_src_names; + init.num_parents = ARRAY_SIZE(emc_src_names); + emc->hw.init = &init; + + clk = clk_register(NULL, &emc->hw); + if (IS_ERR(clk)) { + kfree(emc); + return clk; + } + emc->emc_clk = clk; + + return clk; +} +EXPORT_SYMBOL_GPL(tegra210_clk_register_emc); + +#ifdef CONFIG_PM_SLEEP +static int tegra210_emc_suspend(struct device *dev) +{ + struct tegra_emc *emc = dev_get_drvdata(dev); + + emc->emc_suspend = true; + emc->emc_resume_rate = clk_get_rate(emc->emc_clk); + clk_set_rate(emc->emc_clk, TEGRA210_EMC_SUSPEND_RATE); + + pr_debug("%s at rate %lu\n", __func__, clk_get_rate(emc->emc_clk)); + + return 0; +} + +static int tegra210_emc_resume(struct device *dev) +{ + struct tegra_emc *emc = dev_get_drvdata(dev); + + emc->emc_suspend = false; + clk_set_rate(emc->emc_clk, emc->emc_resume_rate); + + pr_debug("%s at rate %lu\n", __func__, clk_get_rate(emc->emc_clk)); + + return 0; +} +#endif + +static const struct dev_pm_ops tegra210_emc_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(tegra210_emc_suspend, tegra210_emc_resume) +}; + +static const struct of_device_id tegra210_emc_of_match[] = { + { .compatible = "nvidia,tegra210-emc", }, + { }, +}; + +static struct platform_driver tegra210_emc_driver = { + .driver = { + .name = "tegra210-emc", + .of_match_table = tegra210_emc_of_match, + .pm = &tegra210_emc_pm_ops, + .suppress_bind_attrs = true, + }, + .probe = tegra210_emc_probe, +}; + +static int __init tegra210_emc_init(void) +{ + return platform_driver_register(&tegra210_emc_driver); +} +subsys_initcall(tegra210_emc_init); diff --git a/drivers/memory/tegra/tegra210-emc.h b/drivers/memory/tegra/tegra210-emc.h new file mode 100644 index 000000000000..029f8afb2d66 --- /dev/null +++ b/drivers/memory/tegra/tegra210-emc.h @@ -0,0 +1,156 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2015-2019, NVIDIA CORPORATION. All rights reserved. + */ + +#ifndef _TEGRA210_EMC_REG_H +#define _TEGRA210_EMC_REG_H + +#include +#include +#include + +#include "mc.h" + +enum burst_mc_regs_list { + MC_EMEM_ARB_MISC0_INDEX = 20, +}; + +enum { + REG_EMC, + REG_EMC0, + REG_EMC1, + REG_TYPE_NUM, +}; + +enum { + C0D0U0, + C0D0U1, + C0D1U0, + C0D1U1, + C1D0U0, + C1D0U1, + C1D1U0, + C1D1U1, + DRAM_CLKTREE_NUM, +}; + +enum { + VREF_REGS_PER_CH_SIZE = 4, + DRAM_TIMINGS_NUM = 5, + BURST_REGS_PER_CH_SIZE = 8, + TRIM_REGS_PER_CH_SIZE = 10, + PTFV_ARRAY_SIZE = 12, + SAVE_RESTORE_MOD_REGS_SIZE = 12, + TRAINING_MOD_REGS_SIZE = 20, + BURST_UP_DOWN_REGS_SIZE = 24, + BURST_MC_REGS_SIZE = 33, + TRIM_REGS_SIZE = 138, + BURST_REGS_SIZE = 221, +}; + +struct emc_table { + u32 rev; + const char dvfs_ver[60]; + u32 rate; + u32 min_volt; + u32 gpu_min_volt; + const char clock_src[32]; + u32 clk_src_emc; + u32 needs_training; + u32 training_pattern; + u32 trained; + + u32 periodic_training; + u32 trained_dram_clktree[DRAM_CLKTREE_NUM]; + u32 current_dram_clktree[DRAM_CLKTREE_NUM]; + u32 run_clocks; + u32 tree_margin; + + u32 num_burst; + u32 num_burst_per_ch; + u32 num_trim; + u32 num_trim_per_ch; + u32 num_mc_regs; + u32 num_up_down; + u32 vref_num; + u32 training_mod_num; + u32 dram_timing_num; + + u32 ptfv_list[PTFV_ARRAY_SIZE]; + + u32 burst_regs[BURST_REGS_SIZE]; + u32 burst_reg_per_ch[BURST_REGS_PER_CH_SIZE]; + u32 shadow_regs_ca_train[BURST_REGS_SIZE]; + u32 shadow_regs_quse_train[BURST_REGS_SIZE]; + u32 shadow_regs_rdwr_train[BURST_REGS_SIZE]; + + u32 trim_regs[TRIM_REGS_SIZE]; + u32 trim_perch_regs[TRIM_REGS_PER_CH_SIZE]; + + u32 vref_perch_regs[VREF_REGS_PER_CH_SIZE]; + + u32 dram_timings[DRAM_TIMINGS_NUM]; + u32 training_mod_regs[TRAINING_MOD_REGS_SIZE]; + u32 save_restore_mod_regs[SAVE_RESTORE_MOD_REGS_SIZE]; + u32 burst_mc_regs[BURST_MC_REGS_SIZE]; + u32 la_scale_regs[BURST_UP_DOWN_REGS_SIZE]; + + u32 min_mrs_wait; + u32 emc_mrw; + u32 emc_mrw2; + u32 emc_mrw3; + u32 emc_mrw4; + u32 emc_mrw9; + u32 emc_mrs; + u32 emc_emrs; + u32 emc_emrs2; + u32 emc_auto_cal_config; + u32 emc_auto_cal_config2; + u32 emc_auto_cal_config3; + u32 emc_auto_cal_config4; + u32 emc_auto_cal_config5; + u32 emc_auto_cal_config6; + u32 emc_auto_cal_config7; + u32 emc_auto_cal_config8; + u32 emc_cfg_2; + u32 emc_sel_dpd_ctrl; + u32 emc_fdpd_ctrl_cmd_no_ramp; + u32 dll_clk_src; + u32 clk_out_enb_x_0_clk_enb_emc_dll; + u32 latency; +}; + +struct tegra_emc { + struct clk_hw hw; + struct clk *emc_clk; + struct device *dev; + struct timer_list training_timer; + + struct tegra_mc *mc; + + void __iomem *emc_base[REG_TYPE_NUM]; + + struct emc_table *current_timing; + struct emc_table *next_timing; + + struct emc_table *emc_table; + unsigned int emc_table_size; + + u32 clk_setting; + ktime_t clkchange_time; + int clkchange_delay; + u32 timer_period_training; + + bool emc_suspend; + unsigned long emc_resume_rate; +}; + +struct supported_sequence { + u8 table_rev; + void (*set_clock)(struct tegra_emc *emc, u32 clksrc); + u32 (*periodic_compensation)(struct tegra_emc *emc); + char *seq_rev; +}; + +#endif diff --git a/include/linux/clk/tegra.h b/include/linux/clk/tegra.h index 5b0bdb413460..2ccb10eab4a8 100644 --- a/include/linux/clk/tegra.h +++ b/include/linux/clk/tegra.h @@ -146,4 +146,27 @@ void tegra20_clk_set_emc_round_callback(tegra20_clk_emc_round_cb *round_cb, void *cb_arg); int tegra20_clk_prepare_emc_mc_same_freq(struct clk *emc_clk, bool same); +struct tegra210_clk_emc_config { + unsigned long rate; + bool same_freq; + u32 value; + + unsigned long parent_rate; + u8 parent; +}; + +struct tegra210_clk_emc_provider { + struct device *dev; + + struct tegra210_clk_emc_config *configs; + unsigned int num_configs; + + int (*set_rate)(struct device *dev, + const struct tegra210_clk_emc_config *config); +}; + +int tegra210_clk_emc_attach(struct clk *clk, + struct tegra210_clk_emc_provider *provider); +void tegra210_clk_emc_detach(struct clk *clk); + #endif /* __LINUX_CLK_TEGRA_H_ */ From patchwork Tue Mar 10 15:19:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 11429627 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A8142139A for ; Tue, 10 Mar 2020 15:20:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8686C20727 for ; Tue, 10 Mar 2020 15:20:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hPJQI53C" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727763AbgCJPUX (ORCPT ); Tue, 10 Mar 2020 11:20:23 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:53572 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726283AbgCJPUX (ORCPT ); Tue, 10 Mar 2020 11:20:23 -0400 Received: by mail-wm1-f68.google.com with SMTP id 25so1848434wmk.3; Tue, 10 Mar 2020 08:20:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=19P5Q8UerRaXg+pM9/nCaLV3h2AyveSWuKf0Gpk16Qc=; b=hPJQI53CXKGUKQ6jU+3Yk4A2f/61DleVLJlpLu7gmcgNeuqRC+h9jHegtHtMiMoirs eSZTsuwwpk2gJWCUXK31t/AYyWoqN3/U0Afs5MVe8GdJTCmORo8fJu+CpULL9yJYF9/m yEj/S1/0ITfGWqQfBD0MfJdTEMqUxcYd61dUCepw/nmTYNBXU/uXX1c4viWMZ1//GJiP f0KFevmKVuxqvTLyn1R+0vRCH9SIDd1gF30LjW3itEhy6mtrRaNIZtzrmlTUk4WLw0W4 XEgY78A/Hv/2fI6lQ8BN7GCAPgnMlloUlRqYu1ctshqIVcTtzrV6NhjhG60JB3KH1GF/ +WLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=19P5Q8UerRaXg+pM9/nCaLV3h2AyveSWuKf0Gpk16Qc=; b=igqjkBQggQsofPvYJ2+H8ndLXKdhwzNM4OgyBMMoxpysQs4XnhWlTCK9QVZn3p+kVK HoApndFoj8VhQfC++M7MHK7U/eV9dxOp04rtPbB6QhRMO1PHzjrDMGjdZL727hsBYjLa e7hKYUinhNcMKIRj+PpNNsxsb+xHVwf9FErei2ntAU6CYmnElvp8i0GInAW/fzyAK/xS Jb5uWuBw+jNiK+uOLCCsqtQhE2YhHxtH3A+Z3W8BNHtQzdfI+WYHXaXobjy7K6zfX8KE tCOz9Ig+VCRmMnmJuVuyFZ03dtaNC6dFMCJt90RQ8lAIWBJvlt4c4TxV+GL7RNisLzoW 6Odw== X-Gm-Message-State: ANhLgQ38fasVIJpf+qFO++iPr4XksGY3wB+VUxkCUwTV+CeZnPjxkouy ZuI3C0X6blOQ5tbXrUdOuj1/m/By X-Google-Smtp-Source: ADFU+vtIkqP9tnxLdbeEmLYZOzuONJ3rU8LjegCTivGOd0PUquObJXoCx5ureVJpw5rPnAo1hMaFzA== X-Received: by 2002:a1c:6a0c:: with SMTP id f12mr2590876wmc.78.1583853619358; Tue, 10 Mar 2020 08:20:19 -0700 (PDT) Received: from localhost (pD9E516A9.dip0.t-ipconnect.de. [217.229.22.169]) by smtp.gmail.com with ESMTPSA id u17sm4230626wmj.47.2020.03.10.08.20.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 08:20:18 -0700 (PDT) From: Thierry Reding To: Thierry Reding Cc: Jon Hunter , Dmitry Osipenko , Rob Herring , Mark Rutland , Michael Turquette , Stephen Boyd , Joseph Lo , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 4/8] dt-bindings: memory: tegra: Add external memory controller binding for Tegra210 Date: Tue, 10 Mar 2020 16:19:59 +0100 Message-Id: <20200310152003.2945170-5-thierry.reding@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310152003.2945170-1-thierry.reding@gmail.com> References: <20200310152003.2945170-1-thierry.reding@gmail.com> MIME-Version: 1.0 Sender: linux-clk-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org From: Joseph Lo Add the binding document for the external memory controller (EMC) which communicates with external LPDDR4 devices. It includes the bindings of the EMC node and a sub-node of EMC table which under the reserved memory node. The EMC table contains the data of the rates that EMC supported. Signed-off-by: Joseph Lo Signed-off-by: Thierry Reding --- Changes in v5: - convert to dt-schema .../nvidia,tegra210-emc.yaml | 83 +++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 Documentation/devicetree/bindings/memory-controllers/nvidia,tegra210-emc.yaml diff --git a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra210-emc.yaml b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra210-emc.yaml new file mode 100644 index 000000000000..caf21c08f9cc --- /dev/null +++ b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra210-emc.yaml @@ -0,0 +1,83 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/memory-controllers/nvidia,tegra210-emc.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: NVIDIA Tegra210 SoC External Memory Controller + +maintainers: + - Thierry Reding + - Jon Hunter + +description: | + The EMC interfaces with the off-chip SDRAM to service the request stream + sent from the memory controller. + +properties: + compatible: + const: nvidia,tegra210-emc + + reg: + maxItems: 3 + + clocks: + items: + - description: external memory clock + + clock-names: + items: + - const: emc + + interrupts: + items: + - description: EMC general interrupt + + memory-region: + $ref: /schemas/types.yaml#/definitions/phandle + description: + phandle to a reserved memory region describing the table of EMC + frequencies trained by the firmware + + nvidia,memory-controller: + $ref: /schemas/types.yaml#/definitions/phandle + description: + phandle of the memory controller node + +required: + - compatible + - reg + - clocks + - clock-names + - nvidia,memory-controller + +additionalProperties: false + +examples: + - | + #include + #include + # + reserved-memory { + #address-cells = <2>; + #size-cells = <2>; + ranges; + + emc_table: emc-table@83400000 { + compatible = "nvidia,tegra210-emc-table"; + reg = <0x0 0x83400000 0x0 0x10000>; + status = "okay"; + }; + }; + + external-memory-controller@7001b000 { + compatible = "nvidia,tegra210-emc"; + reg = <0x0 0x7001b000 0x0 0x1000>, + <0x0 0x7001e000 0x0 0x1000>, + <0x0 0x7001f000 0x0 0x1000>; + clocks = <&tegra_car TEGRA210_CLK_EMC>; + clock-names = "emc"; + interrupts = ; + memory-region = <&emc_table>; + nvidia,memory-controller = <&mc>; + }; From patchwork Tue Mar 10 15:20:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 11429633 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF796139A for ; Tue, 10 Mar 2020 15:20:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4F7DC21927 for ; Tue, 10 Mar 2020 15:20:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lisuPpPy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726605AbgCJPUd (ORCPT ); Tue, 10 Mar 2020 11:20:33 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:36991 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727300AbgCJPUc (ORCPT ); Tue, 10 Mar 2020 11:20:32 -0400 Received: by mail-wm1-f68.google.com with SMTP id a141so1809824wme.2; Tue, 10 Mar 2020 08:20:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3plVQVKSX4nWebGyldnyIT8CqvbozEek2doWoXXZJn0=; b=lisuPpPylA/xRlETTsYpBdJb33wh2BSTUx8FDtrTFvqh7Kkw8YiMTppZnrxUDIUDi3 uQtThNxjhV8h7mbek5BOSDiRzjB71iW4gQ44w9fg4MTfi2Rs098LLk8YZ+Fz8Yx31J8E QMb2i3ffc4t/zHArs1dR0u5q+qqnE73dC9XL5yZEAwZau8MjUX/IdGw6HmvkbhQiQifw di1GePneAoMsaFqXZkk0fwpIo1lBQjpmUTG6sqg0A/ewhD1gEJjOdHCSkYOaY3aNZN+/ YfvCEUrV2cz4HersmHYpGWVCTkeVku/48XeKTWbDYUV9j6tRRIYkvEHfkhoWdMTJ1ztR Ev1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3plVQVKSX4nWebGyldnyIT8CqvbozEek2doWoXXZJn0=; b=Cn/ftP0aAIszwXFL8U2NTF4PNq1v/gNUgF+xL93vTX67Gl7wTAPU5I+79FIPOeEYEn bntcxlK2ZT/n5sqnalI4XAski9BTl5XkEKZ7O1kbJI4AF+ULlKVc6Gs7Bq2Tzud7KuUo N5IMLNha1Cy8zRbFN0gVjhgmTqoWhdaVBqgXoxpHALHyhc06E8RH5sa203onFWrhS8uD WelTaHDvsnqWaYWSJAAtrfNthJT3UmQmB7HqWs7k7ZkgUt4jGRB8XVHzs90hX9/tTvbk Bsfzzw5545BzJCRx4n+btkvw2y7HwudXFUKMAkiMxewjQzwipAWzBU5Pv81CipVo8F+W b6wQ== X-Gm-Message-State: ANhLgQ06/FdMbWfYEFUk1l29KiT+BeU5oR4g9Tf9oBrzvwxB83JwIHKi LESO8yb5EMOovFf59H/hTHs= X-Google-Smtp-Source: ADFU+vtATBILwB/ChhNINp5SqCj1jXXo1geHTbHLIRzCTHoZvWj9xd9Mmz1YWkFWvN2o5IOIjdzvrg== X-Received: by 2002:a1c:f319:: with SMTP id q25mr2595905wmq.142.1583853624270; Tue, 10 Mar 2020 08:20:24 -0700 (PDT) Received: from localhost (pD9E516A9.dip0.t-ipconnect.de. [217.229.22.169]) by smtp.gmail.com with ESMTPSA id u8sm20305457wrn.69.2020.03.10.08.20.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 08:20:22 -0700 (PDT) From: Thierry Reding To: Thierry Reding Cc: Jon Hunter , Dmitry Osipenko , Rob Herring , Mark Rutland , Michael Turquette , Stephen Boyd , Joseph Lo , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 6/8] memory: tegra: Add EMC scaling sequence code for Tegra210 Date: Tue, 10 Mar 2020 16:20:01 +0100 Message-Id: <20200310152003.2945170-7-thierry.reding@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310152003.2945170-1-thierry.reding@gmail.com> References: <20200310152003.2945170-1-thierry.reding@gmail.com> MIME-Version: 1.0 Sender: linux-clk-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org From: Joseph Lo This patch includes the sequence for clock tuning and the dynamic training mechanism for the clock above 800MHz. And historically there have been different sequences to change the EMC clock. The sequence to be used is specified in the EMC table. However, for the currently supported upstreaming platform, only the most recent sequence is used. So only support that in this patch. Based on the work of Peter De Schrijver . Signed-off-by: Joseph Lo Signed-off-by: Thierry Reding --- Changes in v5: - major rework and cleanup drivers/memory/tegra/Makefile | 2 +- drivers/memory/tegra/tegra210-emc-cc-r21021.c | 1782 +++++++++++++++++ drivers/memory/tegra/tegra210-emc.c | 1 + drivers/memory/tegra/tegra210-emc.h | 102 + 4 files changed, 1886 insertions(+), 1 deletion(-) create mode 100644 drivers/memory/tegra/tegra210-emc-cc-r21021.c diff --git a/drivers/memory/tegra/Makefile b/drivers/memory/tegra/Makefile index 1c59150ccf58..8a663bb3b263 100644 --- a/drivers/memory/tegra/Makefile +++ b/drivers/memory/tegra/Makefile @@ -13,6 +13,6 @@ obj-$(CONFIG_TEGRA_MC) += tegra-mc.o obj-$(CONFIG_TEGRA20_EMC) += tegra20-emc.o obj-$(CONFIG_TEGRA30_EMC) += tegra30-emc.o obj-$(CONFIG_TEGRA124_EMC) += tegra124-emc.o -obj-$(CONFIG_TEGRA210_EMC) += tegra210-emc.o +obj-$(CONFIG_TEGRA210_EMC) += tegra210-emc.o tegra210-emc-cc-r21021.o obj-$(CONFIG_ARCH_TEGRA_186_SOC) += tegra186.o tegra186-emc.o obj-$(CONFIG_ARCH_TEGRA_194_SOC) += tegra186.o tegra186-emc.o diff --git a/drivers/memory/tegra/tegra210-emc-cc-r21021.c b/drivers/memory/tegra/tegra210-emc-cc-r21021.c new file mode 100644 index 000000000000..042ac3127c15 --- /dev/null +++ b/drivers/memory/tegra/tegra210-emc-cc-r21021.c @@ -0,0 +1,1782 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2014-2020, NVIDIA CORPORATION. All rights reserved. + */ + +#include +#include +#include +#include +#include + +#include + +#include "mc.h" +#include "tegra210-emc.h" + +/* + * Enable flags for specifying verbosity. + */ +#define INFO (1 << 0) +#define STEPS (1 << 1) +#define SUB_STEPS (1 << 2) +#define PRELOCK (1 << 3) +#define PRELOCK_STEPS (1 << 4) +#define ACTIVE_EN (1 << 5) +#define PRAMP_UP (1 << 6) +#define PRAMP_DN (1 << 7) +#define EMA_WRITES (1 << 10) +#define EMA_UPDATES (1 << 11) +#define PER_TRAIN (1 << 16) +#define CC_PRINT (1 << 17) +#define CCFIFO (1 << 29) +#define REGS (1 << 30) +#define REG_LISTS (1 << 31) + +#define emc_dbg(emc, flags, ...) dev_dbg(emc->dev, __VA_ARGS__) + +#define DVFS_CLOCK_CHANGE_VERSION 21021 +#define EMC_PRELOCK_VERSION 2101 + +enum { + DVFS_SEQUENCE = 1, + WRITE_TRAINING_SEQUENCE = 2, + PERIODIC_TRAINING_SEQUENCE = 3, + DVFS_PT1 = 10, + DVFS_UPDATE = 11, + TRAINING_PT1 = 12, + TRAINING_UPDATE = 13, + PERIODIC_TRAINING_UPDATE = 14 +}; + +/* + * PTFV defines - basically just indexes into the per table PTFV array. + */ +#define PTFV_DQSOSC_MOVAVG_C0D0U0_INDEX 0 +#define PTFV_DQSOSC_MOVAVG_C0D0U1_INDEX 1 +#define PTFV_DQSOSC_MOVAVG_C0D1U0_INDEX 2 +#define PTFV_DQSOSC_MOVAVG_C0D1U1_INDEX 3 +#define PTFV_DQSOSC_MOVAVG_C1D0U0_INDEX 4 +#define PTFV_DQSOSC_MOVAVG_C1D0U1_INDEX 5 +#define PTFV_DQSOSC_MOVAVG_C1D1U0_INDEX 6 +#define PTFV_DQSOSC_MOVAVG_C1D1U1_INDEX 7 +#define PTFV_DVFS_SAMPLES_INDEX 9 +#define PTFV_MOVAVG_WEIGHT_INDEX 10 +#define PTFV_CONFIG_CTRL_INDEX 11 + +#define PTFV_CONFIG_CTRL_USE_PREVIOUS_EMA (1 << 0) + +/* + * Do arithmetic in fixed point. + */ +#define MOVAVG_PRECISION_FACTOR 100 + +/* + * The division portion of the average operation. + */ +#define __AVERAGE_PTFV(dev) \ + ({ next->ptfv_list[PTFV_DQSOSC_MOVAVG_ ## dev ## _INDEX] = \ + next->ptfv_list[PTFV_DQSOSC_MOVAVG_ ## dev ## _INDEX] / \ + next->ptfv_list[PTFV_DVFS_SAMPLES_INDEX]; }) + +/* + * Convert val to fixed point and add it to the temporary average. + */ +#define __INCREMENT_PTFV(dev, val) \ + ({ next->ptfv_list[PTFV_DQSOSC_MOVAVG_ ## dev ## _INDEX] += \ + ((val) * MOVAVG_PRECISION_FACTOR); }) + +/* + * Convert a moving average back to integral form and return the value. + */ +#define __MOVAVG_AC(timing, dev) \ + ((timing)->ptfv_list[PTFV_DQSOSC_MOVAVG_ ## dev ## _INDEX] / \ + MOVAVG_PRECISION_FACTOR) + +/* Weighted update. */ +#define __WEIGHTED_UPDATE_PTFV(dev, nval) \ + do { \ + int w = PTFV_MOVAVG_WEIGHT_INDEX; \ + int dqs = PTFV_DQSOSC_MOVAVG_ ## dev ## _INDEX; \ + \ + next->ptfv_list[dqs] = \ + ((nval * MOVAVG_PRECISION_FACTOR) + \ + (next->ptfv_list[dqs] * \ + next->ptfv_list[w])) / \ + (next->ptfv_list[w] + 1); \ + \ + emc_dbg(emc, EMA_UPDATES, "%s: (s=%lu) EMA: %u\n", \ + __stringify(dev), nval, next->ptfv_list[dqs]); \ + } while (0) + +/* Access a particular average. */ +#define __MOVAVG(timing, dev) \ + ((timing)->ptfv_list[PTFV_DQSOSC_MOVAVG_ ## dev ## _INDEX]) + +static u32 tegra210_emc_mrr_read(struct tegra210_emc *emc, unsigned int chip, + unsigned int address, + unsigned int num_channels) +{ + u32 value, ret = 0; + unsigned int i; + + value = (chip & EMC_MRR_DEV_SEL_MASK) << EMC_MRR_DEV_SEL_SHIFT | + (address & EMC_MRR_MA_MASK) << EMC_MRR_MA_SHIFT; + emc_writel(emc, value, EMC_MRR); + + for (i = 0; i < num_channels; i++) + WARN(tegra210_emc_wait_for_update(emc, i, EMC_EMC_STATUS, + EMC_EMC_STATUS_MRR_DIVLD, 1), + "Timed out waiting for MRR %u (ch=%u)\n", address, i); + + for (i = 0; i < num_channels; i++) { + value = emc_channel_readl(emc, i, EMC_MRR); + value &= EMC_MRR_DATA_MASK; + + ret = (ret << 16) | value; + } + + return ret; +} + +static u32 update_clock_tree_delay(struct tegra210_emc *emc, + unsigned int num_channels, int type) +{ + bool periodic_training_update = type == PERIODIC_TRAINING_UPDATE; + struct tegra210_emc_timing *last = emc->last; + struct tegra210_emc_timing *next = emc->next; + u32 last_timing_rate_mhz = last->rate / 1000; + u32 next_timing_rate_mhz = next->rate / 1000; + bool dvfs_update = type == DVFS_UPDATE; + s32 tdel = 0, tmdel = 0, adel = 0; + bool dvfs_pt1 = type == DVFS_PT1; + unsigned long cval = 0; + u32 temp[2][2], value; + unsigned int i; + + /* + * Dev0 MSB. + */ + if (dvfs_pt1 || periodic_training_update) { + value = tegra210_emc_mrr_read(emc, 2, 19, num_channels); + + for (i = 0; i < num_channels; i++) { + temp[i][0] = (value & 0x00ff) << 8; + temp[i][1] = (value & 0xff00) << 0; + value >>= 16; + } + + /* + * Dev0 LSB. + */ + value = tegra210_emc_mrr_read(emc, 2, 18, num_channels); + + for (i = 0; i < num_channels; i++) { + temp[i][0] |= (value & 0x00ff) >> 0; + temp[i][1] |= (value & 0xff00) >> 8; + value >>= 16; + } + } + + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[0][0]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C0D0U0, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C0D0U0); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C0D0U0, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C0D0U0] - + __MOVAVG_AC(next, C0D0U0); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C0D0U0] = + __MOVAVG_AC(next, C0D0U0); + } + + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[0][1]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C0D0U1, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C0D0U1); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C0D0U1, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C0D0U1] - + __MOVAVG_AC(next, C0D0U1); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + + if (tmdel > adel) + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C0D0U1] = + __MOVAVG_AC(next, C0D0U1); + } + + if (num_channels > 1) { + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[1][0]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C1D0U0, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C1D0U0); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C1D0U0, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C1D0U0] - + __MOVAVG_AC(next, C1D0U0); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + + if (tmdel > adel) + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C1D0U0] = + __MOVAVG_AC(next, C1D0U0); + } + + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[1][1]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C1D0U1, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C1D0U1); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C1D0U1, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C1D0U1] - + __MOVAVG_AC(next, C1D0U1); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + + if (tmdel > adel) + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C1D0U1] = + __MOVAVG_AC(next, C1D0U1); + } + } + + if (emc->num_devices < 2) + goto done; + + /* + * Dev1 MSB. + */ + if (dvfs_pt1 || periodic_training_update) { + value = tegra210_emc_mrr_read(emc, 1, 19, num_channels); + + for (i = 0; i < num_channels; i++) { + temp[i][0] = (value & 0x00ff) << 8; + temp[i][1] = (value & 0xff00) << 0; + value >>= 16; + } + + /* + * Dev1 LSB. + */ + value = tegra210_emc_mrr_read(emc, 2, 18, num_channels); + + for (i = 0; i < num_channels; i++) { + temp[i][0] |= (value & 0x00ff) >> 0; + temp[i][1] |= (value & 0xff00) >> 8; + value >>= 16; + } + } + + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[0][0]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C0D1U0, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C0D1U0); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C0D1U0, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C0D1U0] - + __MOVAVG_AC(next, C0D1U0); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + + if (tmdel > adel) + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C0D1U0] = + __MOVAVG_AC(next, C0D1U0); + } + + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[0][1]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C0D1U1, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C0D1U1); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C0D1U1, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C0D1U1] - + __MOVAVG_AC(next, C0D1U1); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + + if (tmdel > adel) + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C0D1U1] = + __MOVAVG_AC(next, C0D1U1); + } + + if (num_channels > 1) { + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[1][0]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C1D1U0, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C1D1U0); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C1D1U0, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C1D1U0] - + __MOVAVG_AC(next, C1D1U0); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + + if (tmdel > adel) + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C1D1U0] = + __MOVAVG_AC(next, C1D1U0); + } + + if (dvfs_pt1 || periodic_training_update) { + cval = tegra210_emc_actual_osc_clocks(last->run_clocks); + cval *= 1000000; + cval /= last_timing_rate_mhz * 2 * temp[1][1]; + } + + if (dvfs_pt1) + __INCREMENT_PTFV(C1D1U1, cval); + else if (dvfs_update) + __AVERAGE_PTFV(C1D1U1); + else if (periodic_training_update) + __WEIGHTED_UPDATE_PTFV(C1D1U1, cval); + + if (dvfs_update || periodic_training_update) { + tdel = next->current_dram_clktree[C1D1U1] - + __MOVAVG_AC(next, C1D1U1); + tmdel = (tdel < 0) ? -1 * tdel : tdel; + + if (tmdel > adel) + adel = tmdel; + + if (tmdel * 128 * next_timing_rate_mhz / 1000000 > + next->tree_margin) + next->current_dram_clktree[C1D1U1] = + __MOVAVG_AC(next, C1D1U1); + } + } + +done: + return adel; +} + +static u32 periodic_compensation_handler(struct tegra210_emc *emc, u32 type, + unsigned int num_channels, + struct tegra210_emc_timing *last, + struct tegra210_emc_timing *next) +{ +#define __COPY_EMA(nt, lt, dev) \ + ({ __MOVAVG(nt, dev) = __MOVAVG(lt, dev) * \ + (nt)->ptfv_list[PTFV_DVFS_SAMPLES_INDEX]; }) + + u32 i, adel = 0, samples = next->ptfv_list[PTFV_DVFS_SAMPLES_INDEX]; + u32 delay; + + delay = tegra210_emc_actual_osc_clocks(last->run_clocks); + delay *= 1000; + delay = 2 + (delay / last->rate); + + if (!next->periodic_training) + return 0; + + if (type == DVFS_SEQUENCE) { + if (last->periodic_training && + (next->ptfv_list[PTFV_CONFIG_CTRL_INDEX] & + PTFV_CONFIG_CTRL_USE_PREVIOUS_EMA)) { + /* + * If the previous frequency was using periodic + * calibration then we can reuse the previous + * frequencies EMA data. + */ + __COPY_EMA(next, last, C0D0U0); + __COPY_EMA(next, last, C0D0U1); + __COPY_EMA(next, last, C1D0U0); + __COPY_EMA(next, last, C1D0U1); + __COPY_EMA(next, last, C0D1U0); + __COPY_EMA(next, last, C0D1U1); + __COPY_EMA(next, last, C1D1U0); + __COPY_EMA(next, last, C1D1U1); + } else { + /* Reset the EMA.*/ + __MOVAVG(next, C0D0U0) = 0; + __MOVAVG(next, C0D0U1) = 0; + __MOVAVG(next, C1D0U0) = 0; + __MOVAVG(next, C1D0U1) = 0; + __MOVAVG(next, C0D1U0) = 0; + __MOVAVG(next, C0D1U1) = 0; + __MOVAVG(next, C1D1U0) = 0; + __MOVAVG(next, C1D1U1) = 0; + + for (i = 0; i < samples; i++) { + tegra210_emc_start_periodic_compensation(emc); + udelay(delay); + + /* + * Generate next sample of data. + */ + adel = update_clock_tree_delay(emc, + num_channels, + DVFS_PT1); + } + } + + /* + * Seems like it should be part of the + * 'if (last_timing->periodic_training)' conditional + * since is already done for the else clause. + */ + adel = update_clock_tree_delay(emc, num_channels, DVFS_UPDATE); + } + + if (type == PERIODIC_TRAINING_SEQUENCE) { + tegra210_emc_start_periodic_compensation(emc); + udelay(delay); + + adel = update_clock_tree_delay(emc, num_channels, + PERIODIC_TRAINING_UPDATE); + } + + return adel; +} + +static u32 tegra210_emc_r21021_periodic_compensation(struct tegra210_emc *emc) +{ + u32 emc_cfg, emc_cfg_o, emc_cfg_update, del, value; + unsigned int num_channels = 1; + u32 list[] = { + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_0, + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_1, + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_2, + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_3, + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_0, + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_1, + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_2, + EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_3, + EMC_DATA_BRLSHFT_0, + EMC_DATA_BRLSHFT_1 + }; + struct tegra210_emc_timing *last = emc->last; + unsigned int items = ARRAY_SIZE(list), i; + unsigned long delay; + + if (last->periodic_training) { + if (last->burst_regs[EMC_FBIO_CFG7_INDEX] & (1 << 2)) + num_channels = 2; + + emc_dbg(emc, PER_TRAIN, "Periodic training starting\n"); + + value = emc_readl(emc, EMC_DBG); + emc_cfg_o = emc_readl(emc, EMC_CFG); + emc_cfg = emc_cfg_o & ~(EMC_CFG_DYN_SELF_REF | + EMC_CFG_DRAM_ACPD | + EMC_CFG_DRAM_CLKSTOP_PD | + EMC_CFG_DRAM_CLKSTOP_PD); + + + /* + * 1. Power optimizations should be off. + */ + emc_writel(emc, emc_cfg, EMC_CFG); + + /* Does emc_timing_update() for above changes. */ + tegra210_emc_dll_disable(emc, num_channels); + + for (i = 0; i < num_channels; i++) + tegra210_emc_wait_for_update(emc, i, EMC_EMC_STATUS, + EMC_EMC_STATUS_DRAM_IN_POWERDOWN_MASK, + 0); + + for (i = 0; i < num_channels; i++) + tegra210_emc_wait_for_update(emc, i, EMC_EMC_STATUS, + EMC_EMC_STATUS_DRAM_IN_SELF_REFRESH_MASK, + 0); + + emc_cfg_update = value = emc_readl(emc, EMC_CFG_UPDATE); + value &= ~EMC_CFG_UPDATE_UPDATE_DLL_IN_UPDATE_MASK; + value |= (2 << EMC_CFG_UPDATE_UPDATE_DLL_IN_UPDATE_SHIFT); + emc_writel(emc, value, EMC_CFG_UPDATE); + + /* + * 2. osc kick off - this assumes training and dvfs have set + * correct MR23. + */ + tegra210_emc_start_periodic_compensation(emc); + + /* + * 3. Let dram capture its clock tree delays. + */ + delay = tegra210_emc_actual_osc_clocks(last->run_clocks); + delay *= 1000; + delay /= last->rate + 1; + udelay(delay); + + /* + * 4. Check delta wrt previous values (save value if margin + * exceeds what is set in table). + */ + del = periodic_compensation_handler(emc, + PERIODIC_TRAINING_SEQUENCE, + num_channels, last, last); + + /* + * 5. Apply compensation w.r.t. trained values (if clock tree + * has drifted more than the set margin). + */ + if (last->tree_margin < ((del * 128 * (last->rate / 1000)) / 1000000)) { + for (i = 0; i < items; i++) { + value = tegra210_emc_compensate(last, list[i]); + emc_dbg(emc, EMA_WRITES, "0x%08x <= 0x%08x\n", + list[i], value); + emc_writel(emc, value, list[i]); + } + } + + emc_writel(emc, emc_cfg_o, EMC_CFG); + + /* + * 6. Timing update actally applies the new trimmers. + */ + tegra210_emc_timing_update(emc, num_channels); + + /* 6.1. Restore the UPDATE_DLL_IN_UPDATE field. */ + emc_writel(emc, emc_cfg_update, EMC_CFG_UPDATE); + + /* 6.2. Restore the DLL. */ + tegra210_emc_dll_enable(emc, num_channels); + } + + return 0; +} + +/* + * Do the clock change sequence. + */ +static void tegra210_emc_r21021_set_clock(struct tegra210_emc *emc, u32 clksrc) +{ + /* + * This is the timing table for the source frequency. It does _not_ + * necessarily correspond to the actual timing values in the EMC at the + * moment. If the boot BCT differs from the table then this can happen. + * However, we need it for accessing the dram_timings (which are not + * really registers) array for the current frequency. + */ + u32 tmp, cya_allow_ref_cc = 0, ref_b4_sref_en = 0, cya_issue_pc_ref = 0; + struct tegra210_emc_timing *fake, *last = emc->last, *next = emc->next; + u32 bg_regulator_switch_complete_wait_clks, bg_regulator_mode_change; + u32 opt_zcal_en_cc, opt_do_sw_qrst = 1, opt_dvfs_mode, opt_dll_mode; + u32 emc_zcal_wait_cnt_old, emc_zcal_wait_cnt_new, emc_dbg_active; + u32 opt_cc_short_zcal = 1, opt_short_zcal = 1, opt_war_200024907; + u32 tRTM, RP_war, R2P_war, TRPab_war, deltaTWATM, W2P_war, tRPST; + u32 adel = 0, compensate_trimmer_applicable = 0, mrw_req, value; + unsigned long next_timing_rate_mhz = next->rate / 1000, delay; + u32 tZQCAL_lpddr4 = 1000000, zq_wait_long, shared_zq_resistor; + s32 zq_latch_dvfs_wait_time, tZQCAL_lpddr4_fc_adj, nRTP; + u32 tFC_lpddr4 = 1000 * next->dram_timings[T_FC_LPDDR4]; + u32 emc_auto_cal_config, auto_cal_en, mr13_catr_enable; + u32 zq_op, zcal_wait_time_clocks, zcal_wait_time_ps; + u32 emc_cfg, emc_sel_dpd_ctrl, emc_zcal_interval; + int next_push, next_dq_e_ivref, next_dqs_e_ivref; + u32 mr13_flip_fspwr, mr13_flip_fspop, is_lpddr3; + u32 enable_bglp_regulator, enable_bg_regulator; + u32 emc_dbg_o, emc_cfg_pipe_clk_o, emc_pin_o; + u32 ramp_up_wait = 0, ramp_down_wait = 0; + u32 save_restore_clkstop_pd = 1, dll_out; + u32 ref_delay_mult, ref_delay, dram_type; + unsigned int num_channels = 1, i; + static u32 fsp_for_next_freq; + /* In picoseconds. */ + u32 source_clock_period, destination_clock_period; + u32 zqcal_before_cc_cutoff = 2400; + + emc_dbg(emc, INFO, "Running clock change.\n"); + + /* XXX fake == last */ + fake = tegra210_emc_find_timing(emc, last->rate * 1000UL); + fsp_for_next_freq = !fsp_for_next_freq; + + dram_type = emc_readl(emc, EMC_FBIO_CFG5) & + EMC_FBIO_CFG5_DRAM_TYPE_MASK >> + EMC_FBIO_CFG5_DRAM_TYPE_SHIFT; + shared_zq_resistor = last->burst_regs[EMC_ZCAL_WAIT_CNT_INDEX] & + 1 << 31; + if (last->burst_regs[EMC_FBIO_CFG7_INDEX] & (1 << 2)) + num_channels = 2; + opt_zcal_en_cc = (next->burst_regs[EMC_ZCAL_INTERVAL_INDEX] && + !last->burst_regs[EMC_ZCAL_INTERVAL_INDEX]) || + dram_type == DRAM_TYPE_LPDDR4; + opt_dll_mode = (dram_type == DRAM_TYPE_DDR3) ? + tegra210_emc_get_dll_state(next) : 0; + is_lpddr3 = (dram_type == DRAM_TYPE_LPDDR2) && + next->burst_regs[EMC_FBIO_CFG5_INDEX] & + 1 << 25; + opt_war_200024907 = (dram_type == DRAM_TYPE_LPDDR4); + opt_dvfs_mode = MAN_SR; + + emc_readl(emc, EMC_CFG); + emc_auto_cal_config = emc_readl(emc, EMC_AUTO_CAL_CONFIG); + + source_clock_period = 1000000000 / last->rate; + destination_clock_period = 1000000000 / next->rate; + + tZQCAL_lpddr4_fc_adj = (destination_clock_period > + zqcal_before_cc_cutoff) ? + tZQCAL_lpddr4 / destination_clock_period : + (tZQCAL_lpddr4 - tFC_lpddr4) / destination_clock_period; + emc_dbg_o = emc_readl(emc, EMC_DBG); + emc_pin_o = emc_readl(emc, EMC_PIN); + emc_cfg_pipe_clk_o = emc_readl(emc, EMC_CFG_PIPE_CLK); + + emc_cfg = next->burst_regs[EMC_CFG_INDEX]; + emc_cfg &= ~(EMC_CFG_DYN_SELF_REF | EMC_CFG_DRAM_ACPD | + EMC_CFG_DRAM_CLKSTOP_SR | EMC_CFG_DRAM_CLKSTOP_PD); + emc_sel_dpd_ctrl = next->emc_sel_dpd_ctrl; + emc_sel_dpd_ctrl &= ~(EMC_SEL_DPD_CTRL_CLK_SEL_DPD_EN | + EMC_SEL_DPD_CTRL_CA_SEL_DPD_EN | + EMC_SEL_DPD_CTRL_RESET_SEL_DPD_EN | + EMC_SEL_DPD_CTRL_ODT_SEL_DPD_EN | + EMC_SEL_DPD_CTRL_DATA_SEL_DPD_EN); + + emc_dbg(emc, INFO, "Clock change version: %d\n", + DVFS_CLOCK_CHANGE_VERSION); + emc_dbg(emc, INFO, "DRAM type = %d\n", dram_type); + emc_dbg(emc, INFO, "DRAM dev #: %u\n", emc->num_devices); + emc_dbg(emc, INFO, "Next EMC clksrc: 0x%08x\n", clksrc); + emc_dbg(emc, INFO, "DLL clksrc: 0x%08x\n", next->dll_clk_src); + emc_dbg(emc, INFO, "last rate: %u, next rate %u\n", last->rate, + next->rate); + emc_dbg(emc, INFO, "last period: %u, next period: %u\n", + source_clock_period, destination_clock_period); + emc_dbg(emc, INFO, " shared_zq_resistor: %d\n", !!shared_zq_resistor); + emc_dbg(emc, INFO, " num_channels: %u\n", num_channels); + emc_dbg(emc, INFO, " opt_dll_mode: %d\n", opt_dll_mode); + + /* + * Step 1: + * Pre DVFS SW sequence. + */ + emc_dbg(emc, STEPS, "Step 1\n"); + emc_dbg(emc, STEPS, "Step 1.1: Disable DLL temporarily.\n"); + tmp = emc_readl(emc, EMC_CFG_DIG_DLL); + tmp &= ~EMC_CFG_DIG_DLL_CFG_DLL_EN; + emc_writel(emc, tmp, EMC_CFG_DIG_DLL); + + tegra210_emc_timing_update(emc, num_channels); + + for (i = 0; i < num_channels; i++) + tegra210_emc_wait_for_update(emc, i, EMC_CFG_DIG_DLL, + EMC_CFG_DIG_DLL_CFG_DLL_EN, 0); + + emc_dbg(emc, STEPS, "Step 1.2: Disable AUTOCAL temporarily.\n"); + emc_auto_cal_config = next->emc_auto_cal_config; + auto_cal_en = emc_auto_cal_config & EMC_AUTO_CAL_CONFIG_AUTO_CAL_ENABLE; + emc_auto_cal_config &= ~EMC_AUTO_CAL_CONFIG_AUTO_CAL_START; + emc_auto_cal_config |= EMC_AUTO_CAL_CONFIG_AUTO_CAL_MEASURE_STALL; + emc_auto_cal_config |= EMC_AUTO_CAL_CONFIG_AUTO_CAL_UPDATE_STALL; + emc_auto_cal_config |= auto_cal_en; + emc_writel(emc, emc_auto_cal_config, EMC_AUTO_CAL_CONFIG); + emc_readl(emc, EMC_AUTO_CAL_CONFIG); /* Flush write. */ + + emc_dbg(emc, STEPS, "Step 1.3: Disable other power features.\n"); + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + emc_writel(emc, emc_cfg, EMC_CFG); + emc_writel(emc, emc_sel_dpd_ctrl, EMC_SEL_DPD_CTRL); + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + + if (next->periodic_training) { + tegra210_emc_reset_dram_clktree_values(next); + + for (i = 0; i < num_channels; i++) + tegra210_emc_wait_for_update(emc, i, EMC_EMC_STATUS, + EMC_EMC_STATUS_DRAM_IN_POWERDOWN_MASK, + 0); + + for (i = 0; i < num_channels; i++) + tegra210_emc_wait_for_update(emc, i, EMC_EMC_STATUS, + EMC_EMC_STATUS_DRAM_IN_SELF_REFRESH_MASK, + 0); + + tegra210_emc_start_periodic_compensation(emc); + + delay = 1000 * tegra210_emc_actual_osc_clocks(last->run_clocks); + udelay((delay / last->rate) + 2); + + adel = periodic_compensation_handler(emc, DVFS_SEQUENCE, + num_channels, fake, + next); + compensate_trimmer_applicable = + next->periodic_training && + ((adel * 128 * next_timing_rate_mhz) / 1000000) > + next->tree_margin; + } + + emc_writel(emc, EMC_INTSTATUS_CLKCHANGE_COMPLETE, EMC_INTSTATUS); + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + emc_writel(emc, emc_cfg, EMC_CFG); + emc_writel(emc, emc_sel_dpd_ctrl, EMC_SEL_DPD_CTRL); + emc_writel(emc, emc_cfg_pipe_clk_o | EMC_CFG_PIPE_CLK_CLK_ALWAYS_ON, + EMC_CFG_PIPE_CLK); + emc_writel(emc, next->emc_fdpd_ctrl_cmd_no_ramp & + ~EMC_FDPD_CTRL_CMD_NO_RAMP_CMD_DPD_NO_RAMP_ENABLE, + EMC_FDPD_CTRL_CMD_NO_RAMP); + + bg_regulator_mode_change = + ((next->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + EMC_PMACRO_BG_BIAS_CTRL_0_BGLP_E_PWRD) ^ + (last->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + EMC_PMACRO_BG_BIAS_CTRL_0_BGLP_E_PWRD)) || + ((next->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + EMC_PMACRO_BG_BIAS_CTRL_0_BG_E_PWRD) ^ + (last->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + EMC_PMACRO_BG_BIAS_CTRL_0_BG_E_PWRD)); + enable_bglp_regulator = + (next->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + EMC_PMACRO_BG_BIAS_CTRL_0_BGLP_E_PWRD) == 0; + enable_bg_regulator = + (next->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + EMC_PMACRO_BG_BIAS_CTRL_0_BG_E_PWRD) == 0; + + if (bg_regulator_mode_change) { + if (enable_bg_regulator) + emc_writel(emc, last->burst_regs + [EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + ~EMC_PMACRO_BG_BIAS_CTRL_0_BG_E_PWRD, + EMC_PMACRO_BG_BIAS_CTRL_0); + + if (enable_bglp_regulator) + emc_writel(emc, last->burst_regs + [EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + ~EMC_PMACRO_BG_BIAS_CTRL_0_BGLP_E_PWRD, + EMC_PMACRO_BG_BIAS_CTRL_0); + } + + /* Check if we need to turn on VREF generator. */ + if ((((last->burst_regs[EMC_PMACRO_DATA_PAD_TX_CTRL_INDEX] & + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQ_E_IVREF) == 0) && + ((next->burst_regs[EMC_PMACRO_DATA_PAD_TX_CTRL_INDEX] & + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQ_E_IVREF) == 1)) || + (((last->burst_regs[EMC_PMACRO_DATA_PAD_TX_CTRL_INDEX] & + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQS_E_IVREF) == 0) && + ((next->burst_regs[EMC_PMACRO_DATA_PAD_TX_CTRL_INDEX] & + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQS_E_IVREF) != 0))) { + u32 pad_tx_ctrl = + next->burst_regs[EMC_PMACRO_DATA_PAD_TX_CTRL_INDEX]; + u32 last_pad_tx_ctrl = + last->burst_regs[EMC_PMACRO_DATA_PAD_TX_CTRL_INDEX]; + + next_dqs_e_ivref = pad_tx_ctrl & + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQS_E_IVREF; + next_dq_e_ivref = pad_tx_ctrl & + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQ_E_IVREF; + next_push = (last_pad_tx_ctrl & + ~EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQ_E_IVREF & + ~EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQS_E_IVREF) | + next_dq_e_ivref | next_dqs_e_ivref; + emc_writel(emc, next_push, EMC_PMACRO_DATA_PAD_TX_CTRL); + udelay(1); + } else if (bg_regulator_mode_change) { + udelay(1); + } + + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + + /* + * Step 2: + * Prelock the DLL. + */ + emc_dbg(emc, STEPS, "Step 2\n"); + if (next->burst_regs[EMC_CFG_DIG_DLL_INDEX] & + EMC_CFG_DIG_DLL_CFG_DLL_EN) { + emc_dbg(emc, INFO, "Prelock enabled for target frequency.\n"); + dll_out = tegra210_emc_dll_prelock(emc, clksrc); + emc_dbg(emc, INFO, "DLL out: 0x%03x\n", dll_out); + } else { + emc_dbg(emc, INFO, "Disabling DLL for target frequency.\n"); + tegra210_emc_dll_disable(emc, num_channels); + } + + /* + * Step 3: + * Prepare autocal for the clock change. + */ + emc_dbg(emc, STEPS, "Step 3\n"); + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + emc_writel(emc, next->emc_auto_cal_config2, EMC_AUTO_CAL_CONFIG2); + emc_writel(emc, next->emc_auto_cal_config3, EMC_AUTO_CAL_CONFIG3); + emc_writel(emc, next->emc_auto_cal_config4, EMC_AUTO_CAL_CONFIG4); + emc_writel(emc, next->emc_auto_cal_config5, EMC_AUTO_CAL_CONFIG5); + emc_writel(emc, next->emc_auto_cal_config6, EMC_AUTO_CAL_CONFIG6); + emc_writel(emc, next->emc_auto_cal_config7, EMC_AUTO_CAL_CONFIG7); + emc_writel(emc, next->emc_auto_cal_config8, EMC_AUTO_CAL_CONFIG8); + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + + emc_auto_cal_config |= (EMC_AUTO_CAL_CONFIG_AUTO_CAL_COMPUTE_START | + auto_cal_en); + emc_writel(emc, emc_auto_cal_config, EMC_AUTO_CAL_CONFIG); + + /* + * Step 4: + * Update EMC_CFG. (??) + */ + emc_dbg(emc, STEPS, "Step 4\n"); + if (source_clock_period > 50000 && dram_type == DRAM_TYPE_LPDDR4) + ccfifo_writel(emc, 1, EMC_SELF_REF, 0); + else + emc_writel(emc, next->emc_cfg_2, EMC_CFG_2); + + /* + * Step 5: + * Prepare reference variables for ZQCAL regs. + */ + emc_dbg(emc, STEPS, "Step 5\n"); + emc_zcal_interval = 0; + emc_zcal_wait_cnt_old = last->burst_regs[EMC_ZCAL_WAIT_CNT_INDEX]; + emc_zcal_wait_cnt_new = next->burst_regs[EMC_ZCAL_WAIT_CNT_INDEX]; + emc_zcal_wait_cnt_old &= ~EMC_ZCAL_WAIT_CNT_ZCAL_WAIT_CNT_MASK; + emc_zcal_wait_cnt_new &= ~EMC_ZCAL_WAIT_CNT_ZCAL_WAIT_CNT_MASK; + + if (dram_type == DRAM_TYPE_LPDDR4) + zq_wait_long = max((u32)1, + div_o3(1000000, destination_clock_period)); + else if (dram_type == DRAM_TYPE_LPDDR2 || is_lpddr3) + zq_wait_long = max(next->min_mrs_wait, + div_o3(360000, destination_clock_period)) + + 4; + else if (dram_type == DRAM_TYPE_DDR3) + zq_wait_long = max((u32)256, + div_o3(320000, destination_clock_period) + + 2); + else + zq_wait_long = 0; + + /* + * Step 6: + * Training code - removed. + */ + emc_dbg(emc, STEPS, "Step 6\n"); + + /* + * Step 7: + * Program FSP reference registers and send MRWs to new FSPWR. + */ + emc_dbg(emc, STEPS, "Step 7\n"); + emc_dbg(emc, SUB_STEPS, "Step 7.1: Bug 200024907 - Patch RP R2P"); + if (opt_war_200024907) { + nRTP = 16; + if (source_clock_period >= 1000000/1866) /* 535.91 ps */ + nRTP = 14; + if (source_clock_period >= 1000000/1600) /* 625.00 ps */ + nRTP = 12; + if (source_clock_period >= 1000000/1333) /* 750.19 ps */ + nRTP = 10; + if (source_clock_period >= 1000000/1066) /* 938.09 ps */ + nRTP = 8; + + deltaTWATM = max_t(u32, div_o3(7500, source_clock_period), 8); + + /* + * Originally there was a + .5 in the tRPST calculation. + * However since we can't do FP in the kernel and the tRTM + * computation was in a floating point ceiling function, adding + * one to tRTP should be ok. There is no other source of non + * integer values, so the result was always going to be + * something for the form: f_ceil(N + .5) = N + 1; + */ + tRPST = ((last->emc_mrw & 0x80) >> 7); + tRTM = fake->dram_timings[RL] + + div_o3(3600, source_clock_period) + + max_t(u32, div_o3(7500, source_clock_period), 8) + + tRPST + 1 + nRTP; + + emc_dbg(emc, INFO, "tRTM = %u, EMC_RP = %u\n", tRTM, + next->burst_regs[EMC_RP_INDEX]); + + if (last->burst_regs[EMC_RP_INDEX] < tRTM) { + if (tRTM > (last->burst_regs[EMC_R2P_INDEX] + + last->burst_regs[EMC_RP_INDEX])) { + R2P_war = tRTM - + last->burst_regs[EMC_RP_INDEX]; + RP_war = last->burst_regs[EMC_RP_INDEX]; + TRPab_war = last->burst_regs[EMC_TRPAB_INDEX]; + if (R2P_war > 63) { + RP_war = R2P_war + + last->burst_regs[EMC_RP_INDEX] - 63; + if (TRPab_war < RP_war) + TRPab_war = RP_war; + R2P_war = 63; + } + } else { + R2P_war = last->burst_regs[EMC_R2P_INDEX]; + RP_war = last->burst_regs[EMC_RP_INDEX]; + TRPab_war = last->burst_regs[EMC_TRPAB_INDEX]; + } + + if (RP_war < deltaTWATM) { + W2P_war = last->burst_regs[EMC_W2P_INDEX] + + deltaTWATM - RP_war; + if (W2P_war > 63) { + RP_war = RP_war + W2P_war - 63; + if (TRPab_war < RP_war) + TRPab_war = RP_war; + W2P_war = 63; + } + } else { + W2P_war = last->burst_regs[ + EMC_W2P_INDEX]; + } + + if ((last->burst_regs[EMC_W2P_INDEX] ^ W2P_war) || + (last->burst_regs[EMC_R2P_INDEX] ^ R2P_war) || + (last->burst_regs[EMC_RP_INDEX] ^ RP_war) || + (last->burst_regs[EMC_TRPAB_INDEX] ^ TRPab_war)) { + emc_writel(emc, RP_war, EMC_RP); + emc_writel(emc, R2P_war, EMC_R2P); + emc_writel(emc, W2P_war, EMC_W2P); + emc_writel(emc, TRPab_war, EMC_TRPAB); + } + + tegra210_emc_timing_update(emc, 2); + } else { + emc_dbg(emc, INFO, "Skipped WAR\n"); + } + } + + if (!fsp_for_next_freq) { + mr13_flip_fspwr = (next->emc_mrw3 & 0xffffff3f) | 0x80; + mr13_flip_fspop = (next->emc_mrw3 & 0xffffff3f) | 0x00; + } else { + mr13_flip_fspwr = (next->emc_mrw3 & 0xffffff3f) | 0x40; + mr13_flip_fspop = (next->emc_mrw3 & 0xffffff3f) | 0xc0; + } + + mr13_catr_enable = (mr13_flip_fspwr & 0xFFFFFFFE) | 0x01; + if (emc->num_devices > 1) + mr13_catr_enable = (mr13_catr_enable & 0x3fffffff) | 0x80000000; + + if (dram_type == DRAM_TYPE_LPDDR4) { + emc_writel(emc, mr13_flip_fspwr, EMC_MRW3); + emc_writel(emc, next->emc_mrw, EMC_MRW); + emc_writel(emc, next->emc_mrw2, EMC_MRW2); + } + + /* + * Step 8: + * Program the shadow registers. + */ + emc_dbg(emc, STEPS, "Step 8\n"); + emc_dbg(emc, SUB_STEPS, "Writing burst_regs\n"); + + for (i = 0; i < next->num_burst; i++) { + const u16 *offsets = emc->offsets->burst; + u16 offset; + + if (!offsets[i]) + continue; + + value = next->burst_regs[i]; + offset = offsets[i]; + + if (dram_type != DRAM_TYPE_LPDDR4 && + (offset == EMC_MRW6 || offset == EMC_MRW7 || + offset == EMC_MRW8 || offset == EMC_MRW9 || + offset == EMC_MRW10 || offset == EMC_MRW11 || + offset == EMC_MRW12 || offset == EMC_MRW13 || + offset == EMC_MRW14 || offset == EMC_MRW15 || + offset == EMC_TRAINING_CTRL)) + continue; + + /* Pain... And suffering. */ + if (offset == EMC_CFG) { + value &= ~EMC_CFG_DRAM_ACPD; + value &= ~EMC_CFG_DYN_SELF_REF; + if (dram_type == DRAM_TYPE_LPDDR4) { + value &= ~EMC_CFG_DRAM_CLKSTOP_SR; + value &= ~EMC_CFG_DRAM_CLKSTOP_PD; + } + } else if (offset == EMC_MRS_WAIT_CNT && + dram_type == DRAM_TYPE_LPDDR2 && + opt_zcal_en_cc && !opt_cc_short_zcal && + opt_short_zcal) { + value = (value & ~(EMC_MRS_WAIT_CNT_SHORT_WAIT_MASK << + EMC_MRS_WAIT_CNT_SHORT_WAIT_SHIFT)) | + ((zq_wait_long & EMC_MRS_WAIT_CNT_SHORT_WAIT_MASK) << + EMC_MRS_WAIT_CNT_SHORT_WAIT_SHIFT); + } else if (offset == EMC_ZCAL_WAIT_CNT && + dram_type == DRAM_TYPE_DDR3 && opt_zcal_en_cc && + !opt_cc_short_zcal && opt_short_zcal) { + value = (value & ~(EMC_ZCAL_WAIT_CNT_ZCAL_WAIT_CNT_MASK << + EMC_ZCAL_WAIT_CNT_ZCAL_WAIT_CNT_SHIFT)) | + ((zq_wait_long & EMC_ZCAL_WAIT_CNT_ZCAL_WAIT_CNT_MASK) << + EMC_MRS_WAIT_CNT_SHORT_WAIT_SHIFT); + } else if (offset == EMC_ZCAL_INTERVAL && opt_zcal_en_cc) { + value = 0; /* EMC_ZCAL_INTERVAL reset value. */ + } else if (offset == EMC_PMACRO_AUTOCAL_CFG_COMMON) { + value |= EMC_PMACRO_AUTOCAL_CFG_COMMON_E_CAL_BYPASS_DVFS; + } else if (offset == EMC_PMACRO_DATA_PAD_TX_CTRL) { + value &= ~(EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQSP_TX_E_DCC | + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQSN_TX_E_DCC | + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQ_TX_E_DCC | + EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_CMD_TX_E_DCC); + } else if (offset == EMC_PMACRO_CMD_PAD_TX_CTRL) { + value |= EMC_PMACRO_CMD_PAD_TX_CTRL_CMD_DQ_TX_DRVFORCEON; + value &= ~(EMC_PMACRO_CMD_PAD_TX_CTRL_CMD_DQSP_TX_E_DCC | + EMC_PMACRO_CMD_PAD_TX_CTRL_CMD_DQSN_TX_E_DCC | + EMC_PMACRO_CMD_PAD_TX_CTRL_CMD_DQ_TX_E_DCC | + EMC_PMACRO_CMD_PAD_TX_CTRL_CMD_CMD_TX_E_DCC); + } else if (offset == EMC_PMACRO_BRICK_CTRL_RFU1) { + value &= 0xf800f800; + } else if (offset == EMC_PMACRO_COMMON_PAD_TX_CTRL) { + value &= 0xfffffff0; + } + + emc_writel(emc, value, offset); + } + + /* SW addition: do EMC refresh adjustment here. */ + tegra210_emc_set_over_temp_timing(emc, next); + + if (dram_type == DRAM_TYPE_LPDDR4) { + mrw_req = (23 << EMC_MRW_MRW_MA_SHIFT) | + (next->run_clocks & EMC_MRW_MRW_OP_MASK); + emc_writel(emc, mrw_req, EMC_MRW); + } + + /* Per channel burst registers. */ + emc_dbg(emc, SUB_STEPS, "Writing burst_regs_per_ch\n"); + for (i = 0; i < next->num_burst_per_ch; i++) { + const struct tegra210_emc_per_channel_regs *burst = + emc->offsets->burst_per_channel; + + if (!burst[i].offset) + continue; + + if (dram_type != DRAM_TYPE_LPDDR4 && + (burst[i].offset == EMC_MRW6 || + burst[i].offset == EMC_MRW7 || + burst[i].offset == EMC_MRW8 || + burst[i].offset == EMC_MRW9 || + burst[i].offset == EMC_MRW10 || + burst[i].offset == EMC_MRW11 || + burst[i].offset == EMC_MRW12 || + burst[i].offset == EMC_MRW13 || + burst[i].offset == EMC_MRW14 || + burst[i].offset == EMC_MRW15)) + continue; + + /* Filter out second channel if not in DUAL_CHANNEL mode. */ + if (num_channels < 2 && burst[i].bank >= 1) + continue; + + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + next->burst_reg_per_ch[i], burst[i].offset); + emc_channel_writel(emc, burst[i].bank, + next->burst_reg_per_ch[i], + burst[i].offset); + } + + /* Vref regs. */ + emc_dbg(emc, SUB_STEPS, "Writing vref_regs\n"); + for (i = 0; i < next->vref_num; i++) { + const struct tegra210_emc_per_channel_regs *vref = + emc->offsets->vref_per_channel; + + if (!vref[i].offset) + continue; + + if (num_channels < 2 && vref[i].bank >= 1) + continue; + + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + next->vref_perch_regs[i], vref[i].offset); + emc_channel_writel(emc, vref[i].bank, next->vref_perch_regs[i], + vref[i].offset); + } + + /* Trimmers. */ + emc_dbg(emc, SUB_STEPS, "Writing trim_regs\n"); + for (i = 0; i < next->num_trim; i++) { + const u16 *offsets = emc->offsets->trim; + + if (!offsets[i]) + continue; + + if (compensate_trimmer_applicable && + (offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_0 || + offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_1 || + offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_2 || + offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_3 || + offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_0 || + offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_1 || + offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_2 || + offsets[i] == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_3 || + offsets[i] == EMC_DATA_BRLSHFT_0 || + offsets[i] == EMC_DATA_BRLSHFT_1)) { + value = tegra210_emc_compensate(next, offsets[i]); + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + value, offsets[i]); + emc_dbg(emc, EMA_WRITES, "0x%08x <= 0x%08x\n", + (u32)(u64)offsets[i], value); + emc_writel(emc, value, offsets[i]); + } else { + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + next->trim_regs[i], offsets[i]); + emc_writel(emc, next->trim_regs[i], offsets[i]); + } + } + + /* Per channel trimmers. */ + emc_dbg(emc, SUB_STEPS, "Writing trim_regs_per_ch\n"); + for (i = 0; i < next->num_trim_per_ch; i++) { + const struct tegra210_emc_per_channel_regs *trim = + &emc->offsets->trim_per_channel[0]; + unsigned int offset; + + if (!trim[i].offset) + continue; + + if (num_channels < 2 && trim[i].bank >= 1) + continue; + + offset = trim[i].offset; + + if (compensate_trimmer_applicable && + (offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_0 || + offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_1 || + offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_2 || + offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK0_3 || + offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_0 || + offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_1 || + offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_2 || + offset == EMC_PMACRO_OB_DDLL_LONG_DQ_RANK1_3 || + offset == EMC_DATA_BRLSHFT_0 || + offset == EMC_DATA_BRLSHFT_1)) { + value = tegra210_emc_compensate(next, offset); + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + value, offset); + emc_dbg(emc, EMA_WRITES, "0x%08x <= 0x%08x\n", offset, + value); + emc_channel_writel(emc, trim[i].bank, value, offset); + } else { + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + next->trim_perch_regs[i], offset); + emc_channel_writel(emc, trim[i].bank, + next->trim_perch_regs[i], offset); + } + } + + emc_dbg(emc, SUB_STEPS, "Writing burst_mc_regs\n"); + for (i = 0; i < next->num_mc_regs; i++) { + const u16 *offsets = emc->offsets->burst_mc; + u32 *values = next->burst_mc_regs; + + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + values[i], offsets[i]); + mc_writel(emc->mc, values[i], offsets[i]); + } + + /* Registers to be programmed on the faster clock. */ + if (next->rate < last->rate) { + const u16 *la = emc->offsets->la_scale; + + emc_dbg(emc, SUB_STEPS, "Writing la_scale_regs\n"); + for (i = 0; i < next->num_up_down; i++) { + emc_dbg(emc, REG_LISTS, "(%u) 0x%08x => 0x%08x\n", i, + next->la_scale_regs[i], la[i]); + mc_writel(emc->mc, next->la_scale_regs[i], la[i]); + } + } + + /* Flush all the burst register writes. */ + mc_readl(emc->mc, MC_EMEM_ADR_CFG); + + /* + * Step 9: + * LPDDR4 section A. + */ + emc_dbg(emc, STEPS, "Step 9\n"); + if (dram_type == DRAM_TYPE_LPDDR4) { + emc_writel(emc, emc_zcal_interval, EMC_ZCAL_INTERVAL); + emc_writel(emc, emc_zcal_wait_cnt_new, EMC_ZCAL_WAIT_CNT); + + value = emc_dbg_o | (EMC_DBG_WRITE_MUX_ACTIVE | + EMC_DBG_WRITE_ACTIVE_ONLY); + + emc_writel(emc, value, EMC_DBG); + emc_writel(emc, emc_zcal_interval, EMC_ZCAL_INTERVAL); + emc_writel(emc, emc_dbg_o, EMC_DBG); + } + + /* + * Step 10: + * LPDDR4 and DDR3 common section. + */ + emc_dbg(emc, STEPS, "Step 10\n"); + if (opt_dvfs_mode == MAN_SR || dram_type == DRAM_TYPE_LPDDR4) { + if (dram_type == DRAM_TYPE_LPDDR4) + ccfifo_writel(emc, 0x101, EMC_SELF_REF, 0); + else + ccfifo_writel(emc, 0x1, EMC_SELF_REF, 0); + + if (dram_type == DRAM_TYPE_LPDDR4 && + destination_clock_period <= zqcal_before_cc_cutoff) { + ccfifo_writel(emc, mr13_flip_fspwr ^ 0x40, EMC_MRW3, 0); + ccfifo_writel(emc, (next->burst_regs[EMC_MRW6_INDEX] & + 0xFFFF3F3F) | + (last->burst_regs[EMC_MRW6_INDEX] & + 0x0000C0C0), EMC_MRW6, 0); + ccfifo_writel(emc, (next->burst_regs[EMC_MRW14_INDEX] & + 0xFFFF0707) | + (last->burst_regs[EMC_MRW14_INDEX] & + 0x00003838), EMC_MRW14, 0); + + if (emc->num_devices > 1) { + ccfifo_writel(emc, + (next->burst_regs[EMC_MRW7_INDEX] & + 0xFFFF3F3F) | + (last->burst_regs[EMC_MRW7_INDEX] & + 0x0000C0C0), EMC_MRW7, 0); + ccfifo_writel(emc, + (next->burst_regs[EMC_MRW15_INDEX] & + 0xFFFF0707) | + (last->burst_regs[EMC_MRW15_INDEX] & + 0x00003838), EMC_MRW15, 0); + } + + if (opt_zcal_en_cc) { + if (emc->num_devices < 2) + ccfifo_writel(emc, + 2UL << EMC_ZQ_CAL_DEV_SEL_SHIFT + | EMC_ZQ_CAL_ZQ_CAL_CMD, + EMC_ZQ_CAL, 0); + else if (shared_zq_resistor) + ccfifo_writel(emc, + 2UL << EMC_ZQ_CAL_DEV_SEL_SHIFT + | EMC_ZQ_CAL_ZQ_CAL_CMD, + EMC_ZQ_CAL, 0); + else + ccfifo_writel(emc, + EMC_ZQ_CAL_ZQ_CAL_CMD, + EMC_ZQ_CAL, 0); + } + } + } + + if (dram_type == DRAM_TYPE_LPDDR4) { + ccfifo_writel(emc, mr13_flip_fspop | 0x8, EMC_MRW3, + (1000 * fake->dram_timings[T_RP]) / + source_clock_period); + ccfifo_writel(emc, 0, 0, tFC_lpddr4 / source_clock_period); + } + + if (dram_type == DRAM_TYPE_LPDDR4 || opt_dvfs_mode != MAN_SR) { + u32 t = 30 + (cya_allow_ref_cc ? + (4000 * fake->dram_timings[T_RFC]) + + ((1000 * fake->dram_timings[T_RP]) / + source_clock_period) : 0); + + ccfifo_writel(emc, emc_pin_o & ~(EMC_PIN_PIN_CKE_PER_DEV | + EMC_PIN_PIN_CKEB | EMC_PIN_PIN_CKE), + EMC_PIN, t); + } + + ref_delay_mult = 1; + ref_b4_sref_en = 0; + cya_issue_pc_ref = 0; + + ref_delay_mult += ref_b4_sref_en ? 1 : 0; + ref_delay_mult += cya_allow_ref_cc ? 1 : 0; + ref_delay_mult += cya_issue_pc_ref ? 1 : 0; + ref_delay = ref_delay_mult * + ((1000 * fake->dram_timings[T_RP] / + source_clock_period) + + (1000 * fake->dram_timings[T_RFC] / + source_clock_period)) + 20; + + /* + * Step 11: + * Ramp down. + */ + emc_dbg(emc, STEPS, "Step 11\n"); + ccfifo_writel(emc, 0x0, EMC_CFG_SYNC, + dram_type == DRAM_TYPE_LPDDR4 ? 0 : ref_delay); + + emc_dbg_active = emc_dbg_o | (EMC_DBG_WRITE_MUX_ACTIVE | + EMC_DBG_WRITE_ACTIVE_ONLY); + ccfifo_writel(emc, emc_dbg_active, EMC_DBG, 0); + + ramp_down_wait = tegra210_emc_dvfs_power_ramp_down(emc, + source_clock_period, + 0); + + /* + * Step 12: + * And finally - trigger the clock change. + */ + emc_dbg(emc, STEPS, "Step 12\n"); + ccfifo_writel(emc, 1, EMC_STALL_THEN_EXE_AFTER_CLKCHANGE, 0); + emc_dbg_active &= ~EMC_DBG_WRITE_ACTIVE_ONLY; + ccfifo_writel(emc, emc_dbg_active, EMC_DBG, 0); + + /* + * Step 13: + * Ramp up. + */ + emc_dbg(emc, STEPS, "Step 13\n"); + ramp_up_wait = tegra210_emc_dvfs_power_ramp_up(emc, + destination_clock_period, + 0); + ccfifo_writel(emc, emc_dbg_o, EMC_DBG, 0); + + /* + * Step 14: + * Bringup CKE pins. + */ + emc_dbg(emc, STEPS, "Step 14\n"); + if (dram_type == DRAM_TYPE_LPDDR4) { + u32 r = emc_pin_o | EMC_PIN_PIN_CKE; + + if (emc->num_devices > 1) + ccfifo_writel(emc, r | EMC_PIN_PIN_CKEB | + EMC_PIN_PIN_CKE_PER_DEV, EMC_PIN, 0); + else + ccfifo_writel(emc, r & ~(EMC_PIN_PIN_CKEB | + EMC_PIN_PIN_CKE_PER_DEV), EMC_PIN, 0); + } + + /* + * Step 15: (two step 15s ??) + * Calculate zqlatch wait time; has dependency on ramping times. + */ + emc_dbg(emc, STEPS, "Step 15\n"); + + if (destination_clock_period <= zqcal_before_cc_cutoff) { + s32 t = (s32)(ramp_up_wait + ramp_down_wait) / + (s32)destination_clock_period; + zq_latch_dvfs_wait_time = (s32)tZQCAL_lpddr4_fc_adj - t; + } else { + zq_latch_dvfs_wait_time = tZQCAL_lpddr4_fc_adj - + div_o3(1000 * next->dram_timings[T_PDEX], + destination_clock_period); + } + + emc_dbg(emc, INFO, "tZQCAL_lpddr4_fc_adj = %u\n", tZQCAL_lpddr4_fc_adj); + emc_dbg(emc, INFO, "destination_clock_period = %u\n", + destination_clock_period); + emc_dbg(emc, INFO, "next->dram_timings[T_PDEX] = %u\n", + next->dram_timings[T_PDEX]); + emc_dbg(emc, INFO, "zq_latch_dvfs_wait_time = %d\n", + max_t(s32, 0, zq_latch_dvfs_wait_time)); + + if (dram_type == DRAM_TYPE_LPDDR4 && opt_zcal_en_cc) { + if (emc->num_devices < 2) { + if (destination_clock_period > zqcal_before_cc_cutoff) + ccfifo_writel(emc, + 2UL << EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_CAL_CMD, EMC_ZQ_CAL, + div_o3(1000 * + next->dram_timings[T_PDEX], + destination_clock_period)); + + ccfifo_writel(emc, (mr13_flip_fspop & 0xFFFFFFF7) | + 0x0C000000, EMC_MRW3, + div_o3(1000 * + next->dram_timings[T_PDEX], + destination_clock_period)); + ccfifo_writel(emc, 0, EMC_SELF_REF, 0); + ccfifo_writel(emc, 0, EMC_REF, 0); + ccfifo_writel(emc, 2UL << EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_LATCH_CMD, + EMC_ZQ_CAL, + max_t(s32, 0, zq_latch_dvfs_wait_time)); + } else if (shared_zq_resistor) { + if (destination_clock_period > zqcal_before_cc_cutoff) + ccfifo_writel(emc, + 2UL << EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_CAL_CMD, EMC_ZQ_CAL, + div_o3(1000 * + next->dram_timings[T_PDEX], + destination_clock_period)); + + ccfifo_writel(emc, 2UL << EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_LATCH_CMD, EMC_ZQ_CAL, + max_t(s32, 0, zq_latch_dvfs_wait_time) + + div_o3(1000 * + next->dram_timings[T_PDEX], + destination_clock_period)); + ccfifo_writel(emc, 1UL << EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_LATCH_CMD, + EMC_ZQ_CAL, 0); + + ccfifo_writel(emc, (mr13_flip_fspop & 0xfffffff7) | + 0x0c000000, EMC_MRW3, 0); + ccfifo_writel(emc, 0, EMC_SELF_REF, 0); + ccfifo_writel(emc, 0, EMC_REF, 0); + + ccfifo_writel(emc, 1UL << EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_LATCH_CMD, EMC_ZQ_CAL, + tZQCAL_lpddr4 / destination_clock_period); + } else { + if (destination_clock_period > zqcal_before_cc_cutoff) + ccfifo_writel(emc, EMC_ZQ_CAL_ZQ_CAL_CMD, + EMC_ZQ_CAL, + div_o3(1000 * + next->dram_timings[T_PDEX], + destination_clock_period)); + + ccfifo_writel(emc, (mr13_flip_fspop & 0xfffffff7) | + 0x0c000000, EMC_MRW3, + div_o3(1000 * + next->dram_timings[T_PDEX], + destination_clock_period)); + ccfifo_writel(emc, 0, EMC_SELF_REF, 0); + ccfifo_writel(emc, 0, EMC_REF, 0); + + ccfifo_writel(emc, EMC_ZQ_CAL_ZQ_LATCH_CMD, EMC_ZQ_CAL, + max_t(s32, 0, zq_latch_dvfs_wait_time)); + } + } + + /* WAR: delay for zqlatch */ + ccfifo_writel(emc, 0, 0, 10); + + /* + * Step 16: + * LPDDR4 Conditional Training Kickoff. Removed. + */ + + /* + * Step 17: + * MANSR exit self refresh. + */ + emc_dbg(emc, STEPS, "Step 17\n"); + if (opt_dvfs_mode == MAN_SR && dram_type != DRAM_TYPE_LPDDR4) + ccfifo_writel(emc, 0, EMC_SELF_REF, 0); + + /* + * Step 18: + * Send MRWs to LPDDR3/DDR3. + */ + emc_dbg(emc, STEPS, "Step 18\n"); + if (dram_type == DRAM_TYPE_LPDDR2) { + ccfifo_writel(emc, next->emc_mrw2, EMC_MRW2, 0); + ccfifo_writel(emc, next->emc_mrw, EMC_MRW, 0); + if (is_lpddr3) + ccfifo_writel(emc, next->emc_mrw4, EMC_MRW4, 0); + } else if (dram_type == DRAM_TYPE_DDR3) { + if (opt_dll_mode) + ccfifo_writel(emc, next->emc_emrs & + ~EMC_EMRS_USE_EMRS_LONG_CNT, EMC_EMRS, 0); + ccfifo_writel(emc, next->emc_emrs2 & + ~EMC_EMRS2_USE_EMRS2_LONG_CNT, EMC_EMRS2, 0); + ccfifo_writel(emc, next->emc_mrs | + EMC_EMRS_USE_EMRS_LONG_CNT, EMC_MRS, 0); + } + + /* + * Step 19: + * ZQCAL for LPDDR3/DDR3 + */ + emc_dbg(emc, STEPS, "Step 19\n"); + if (opt_zcal_en_cc) { + if (dram_type == DRAM_TYPE_LPDDR2) { + u32 r; + + zq_op = opt_cc_short_zcal ? 0x56 : 0xAB; + zcal_wait_time_ps = opt_cc_short_zcal ? 90000 : 360000; + zcal_wait_time_clocks = div_o3(zcal_wait_time_ps, + destination_clock_period); + r = zcal_wait_time_clocks << + EMC_MRS_WAIT_CNT2_MRS_EXT2_WAIT_CNT_SHIFT | + zcal_wait_time_clocks << + EMC_MRS_WAIT_CNT2_MRS_EXT1_WAIT_CNT_SHIFT; + ccfifo_writel(emc, r, EMC_MRS_WAIT_CNT2, 0); + ccfifo_writel(emc, 2 << EMC_MRW_MRW_DEV_SELECTN_SHIFT | + EMC_MRW_USE_MRW_EXT_CNT | + 10 << EMC_MRW_MRW_MA_SHIFT | + zq_op << EMC_MRW_MRW_OP_SHIFT, + EMC_MRW, 0); + if (emc->num_devices > 1) { + r = 1 << EMC_MRW_MRW_DEV_SELECTN_SHIFT | + EMC_MRW_USE_MRW_EXT_CNT | + 10 << EMC_MRW_MRW_MA_SHIFT | + zq_op << EMC_MRW_MRW_OP_SHIFT; + ccfifo_writel(emc, r, EMC_MRW, 0); + } + } else if (dram_type == DRAM_TYPE_DDR3) { + zq_op = opt_cc_short_zcal ? 0 : EMC_ZQ_CAL_LONG; + ccfifo_writel(emc, zq_op | 2 << + EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_CAL_CMD, EMC_ZQ_CAL, 0); + if (emc->num_devices > 1) + ccfifo_writel(emc, zq_op | + 1 << EMC_ZQ_CAL_DEV_SEL_SHIFT | + EMC_ZQ_CAL_ZQ_CAL_CMD, + EMC_ZQ_CAL, 0); + } + } + + if (bg_regulator_mode_change) { + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + bg_regulator_switch_complete_wait_clks = + ramp_up_wait > 1250000 ? 0 : + (1250000 - ramp_up_wait) / destination_clock_period; + ccfifo_writel(emc, next->burst_regs + [EMC_PMACRO_BG_BIAS_CTRL_0_INDEX], + EMC_PMACRO_BG_BIAS_CTRL_0, + bg_regulator_switch_complete_wait_clks); + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + } + + /* + * Step 20: + * Issue ref and optional QRST. + */ + emc_dbg(emc, STEPS, "Step 20\n"); + if (dram_type != DRAM_TYPE_LPDDR4) + ccfifo_writel(emc, 0, EMC_REF, 0); + + if (opt_do_sw_qrst) { + ccfifo_writel(emc, 1, EMC_ISSUE_QRST, 0); + ccfifo_writel(emc, 0, EMC_ISSUE_QRST, 2); + } + + /* + * Step 21: + * Restore ZCAL and ZCAL interval. + */ + emc_dbg(emc, STEPS, "Step 21\n"); + if (save_restore_clkstop_pd || opt_zcal_en_cc) { + ccfifo_writel(emc, emc_dbg_o | EMC_DBG_WRITE_MUX_ACTIVE, + EMC_DBG, 0); + if (opt_zcal_en_cc && dram_type != DRAM_TYPE_LPDDR4) + ccfifo_writel(emc, next->burst_regs[EMC_ZCAL_INTERVAL_INDEX], + EMC_ZCAL_INTERVAL, 0); + + if (save_restore_clkstop_pd) + ccfifo_writel(emc, next->burst_regs[EMC_CFG_INDEX] & + ~EMC_CFG_DYN_SELF_REF, + EMC_CFG, 0); + ccfifo_writel(emc, emc_dbg_o, EMC_DBG, 0); + } + + /* + * Step 22: + * Restore EMC_CFG_PIPE_CLK. + */ + emc_dbg(emc, STEPS, "Step 22\n"); + ccfifo_writel(emc, emc_cfg_pipe_clk_o, EMC_CFG_PIPE_CLK, 0); + + if (bg_regulator_mode_change) { + if (enable_bg_regulator) + emc_writel(emc, + next->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + ~EMC_PMACRO_BG_BIAS_CTRL_0_BGLP_E_PWRD, + EMC_PMACRO_BG_BIAS_CTRL_0); + else + emc_writel(emc, + next->burst_regs[EMC_PMACRO_BG_BIAS_CTRL_0_INDEX] & + ~EMC_PMACRO_BG_BIAS_CTRL_0_BG_E_PWRD, + EMC_PMACRO_BG_BIAS_CTRL_0); + } + + /* + * Step 23: + */ + emc_dbg(emc, STEPS, "Step 23\n"); + + tmp = emc_readl(emc, EMC_CFG_DIG_DLL); + tmp |= EMC_CFG_DIG_DLL_CFG_DLL_STALL_ALL_TRAFFIC; + tmp &= ~EMC_CFG_DIG_DLL_CFG_DLL_STALL_RW_UNTIL_LOCK; + tmp &= ~EMC_CFG_DIG_DLL_CFG_DLL_STALL_ALL_UNTIL_LOCK; + tmp &= ~EMC_CFG_DIG_DLL_CFG_DLL_EN; + tmp = (tmp & ~EMC_CFG_DIG_DLL_CFG_DLL_MODE_MASK) | + (2 << EMC_CFG_DIG_DLL_CFG_DLL_MODE_SHIFT); + emc_writel(emc, tmp, EMC_CFG_DIG_DLL); + + tegra210_emc_do_clock_change(emc, clksrc); + + /* + * Step 24: + * Save training results. Removed. + */ + + /* + * Step 25: + * Program MC updown registers. + */ + emc_dbg(emc, STEPS, "Step 25\n"); + + if (next->rate > last->rate) { + for (i = 0; i < next->num_up_down; i++) + mc_writel(emc->mc, next->la_scale_regs[i], + emc->offsets->la_scale[i]); + + tegra210_emc_timing_update(emc, num_channels); + } + + /* + * Step 26: + * Restore ZCAL registers. + */ + emc_dbg(emc, STEPS, "Step 26\n"); + if (dram_type == DRAM_TYPE_LPDDR4) { + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + emc_writel(emc, next->burst_regs[EMC_ZCAL_WAIT_CNT_INDEX], + EMC_ZCAL_WAIT_CNT); + emc_writel(emc, next->burst_regs[EMC_ZCAL_INTERVAL_INDEX], + EMC_ZCAL_INTERVAL); + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + } + + if (dram_type != DRAM_TYPE_LPDDR4 && opt_zcal_en_cc && + !opt_short_zcal && opt_cc_short_zcal) { + udelay(2); + + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + if (dram_type == DRAM_TYPE_LPDDR2) + emc_writel(emc, next->burst_regs[EMC_MRS_WAIT_CNT_INDEX], + EMC_MRS_WAIT_CNT); + else if (dram_type == DRAM_TYPE_DDR3) + emc_writel(emc, next->burst_regs[EMC_ZCAL_WAIT_CNT_INDEX], + EMC_ZCAL_WAIT_CNT); + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + } + + /* + * Step 27: + * Restore EMC_CFG, FDPD registers. + */ + emc_dbg(emc, STEPS, "Step 27\n"); + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + emc_writel(emc, next->burst_regs[EMC_CFG_INDEX], EMC_CFG); + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + emc_writel(emc, next->emc_fdpd_ctrl_cmd_no_ramp, + EMC_FDPD_CTRL_CMD_NO_RAMP); + emc_writel(emc, next->emc_sel_dpd_ctrl, EMC_SEL_DPD_CTRL); + + /* + * Step 28: + * Training recover. Removed. + */ + emc_dbg(emc, STEPS, "Step 28\n"); + + tegra210_emc_set_shadow_bypass(emc, ACTIVE); + emc_writel(emc, + next->burst_regs[EMC_PMACRO_AUTOCAL_CFG_COMMON_INDEX], + EMC_PMACRO_AUTOCAL_CFG_COMMON); + tegra210_emc_set_shadow_bypass(emc, ASSEMBLY); + + /* + * Step 29: + * Power fix WAR. + */ + emc_dbg(emc, STEPS, "Step 29\n"); + emc_writel(emc, EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE0 | + EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE1 | + EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE2 | + EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE3 | + EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE4 | + EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE5 | + EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE6 | + EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE7, + EMC_PMACRO_CFG_PM_GLOBAL_0); + emc_writel(emc, EMC_PMACRO_TRAINING_CTRL_0_CH0_TRAINING_E_WRPTR, + EMC_PMACRO_TRAINING_CTRL_0); + emc_writel(emc, EMC_PMACRO_TRAINING_CTRL_1_CH1_TRAINING_E_WRPTR, + EMC_PMACRO_TRAINING_CTRL_1); + emc_writel(emc, 0, EMC_PMACRO_CFG_PM_GLOBAL_0); + + /* + * Step 30: + * Re-enable autocal. + */ + emc_dbg(emc, STEPS, "Step 30: Re-enable DLL and AUTOCAL\n"); + if (next->burst_regs[EMC_CFG_DIG_DLL_INDEX] & EMC_CFG_DIG_DLL_CFG_DLL_EN) { + tmp = emc_readl(emc, EMC_CFG_DIG_DLL); + tmp |= EMC_CFG_DIG_DLL_CFG_DLL_STALL_ALL_TRAFFIC; + tmp |= EMC_CFG_DIG_DLL_CFG_DLL_EN; + tmp &= ~EMC_CFG_DIG_DLL_CFG_DLL_STALL_RW_UNTIL_LOCK; + tmp &= ~EMC_CFG_DIG_DLL_CFG_DLL_STALL_ALL_UNTIL_LOCK; + tmp = (tmp & ~EMC_CFG_DIG_DLL_CFG_DLL_MODE_MASK) | + (2 << EMC_CFG_DIG_DLL_CFG_DLL_MODE_SHIFT); + emc_writel(emc, tmp, EMC_CFG_DIG_DLL); + tegra210_emc_timing_update(emc, num_channels); + } + + emc_auto_cal_config = next->emc_auto_cal_config; + emc_writel(emc, emc_auto_cal_config, EMC_AUTO_CAL_CONFIG); + + /* Done! Yay. */ +} + +const struct tegra210_emc_sequence tegra210_emc_r21021 = { + .revision = 0x7, + .set_clock = tegra210_emc_r21021_set_clock, + .periodic_compensation = tegra210_emc_r21021_periodic_compensation, +}; diff --git a/drivers/memory/tegra/tegra210-emc.c b/drivers/memory/tegra/tegra210-emc.c index 4ea8fb70a4fd..67f8c4519eb8 100644 --- a/drivers/memory/tegra/tegra210-emc.c +++ b/drivers/memory/tegra/tegra210-emc.c @@ -80,6 +80,7 @@ _OB_DDLL_LONG_DQ_RANK ## rank ## _BYTE ## byte2 ## _MASK)) static const struct tegra210_emc_sequence *tegra210_emc_sequences[] = { + &tegra210_emc_r21021, }; static const struct tegra210_emc_table_register_offsets diff --git a/drivers/memory/tegra/tegra210-emc.h b/drivers/memory/tegra/tegra210-emc.h index e022c319f37f..8020974c2102 100644 --- a/drivers/memory/tegra/tegra210-emc.h +++ b/drivers/memory/tegra/tegra210-emc.h @@ -82,7 +82,16 @@ #define EMC_INTSTATUS_CLKCHANGE_COMPLETE BIT(4) #define EMC_DBG 0x8 #define EMC_DBG_WRITE_MUX_ACTIVE BIT(1) +#define EMC_DBG_WRITE_ACTIVE_ONLY BIT(30) #define EMC_CFG 0xc +#define EMC_CFG_DRAM_CLKSTOP_PD BIT(31) +#define EMC_CFG_DRAM_CLKSTOP_SR BIT(30) +#define EMC_CFG_DRAM_ACPD BIT(29) +#define EMC_CFG_DYN_SELF_REF BIT(28) +#define EMC_PIN 0x24 +#define EMC_PIN_PIN_CKE BIT(0) +#define EMC_PIN_PIN_CKEB BIT(1) +#define EMC_PIN_PIN_CKE_PER_DEV BIT(2) #define EMC_TIMING_CONTROL 0x28 #define EMC_RC 0x2c #define EMC_RFC 0x30 @@ -122,6 +131,8 @@ #define EMC_WEXT 0xb8 #define EMC_RFC_SLR 0xc0 #define EMC_MRS_WAIT_CNT2 0xc4 +#define EMC_MRS_WAIT_CNT2_MRS_EXT2_WAIT_CNT_SHIFT 16 +#define EMC_MRS_WAIT_CNT2_MRS_EXT1_WAIT_CNT_SHIFT 0 #define EMC_MRS_WAIT_CNT 0xc8 #define EMC_MRS_WAIT_CNT_SHORT_WAIT_SHIFT 0 #define EMC_MRS_WAIT_CNT_SHORT_WAIT_MASK \ @@ -158,14 +169,34 @@ #define EMC_PDEX2CKE 0x118 #define EMC_CKE2PDEN 0x11c #define EMC_MPC 0x128 +#define EMC_EMRS2 0x12c +#define EMC_EMRS2_USE_EMRS2_LONG_CNT BIT(26) +#define EMC_MRW2 0x134 +#define EMC_MRW3 0x138 +#define EMC_MRW4 0x13c #define EMC_R2R 0x144 #define EMC_EINPUT 0x14c #define EMC_EINPUT_DURATION 0x150 #define EMC_PUTERM_EXTRA 0x154 #define EMC_TCKESR 0x158 #define EMC_TPD 0x15c +#define EMC_AUTO_CAL_CONFIG 0x2a4 +#define EMC_AUTO_CAL_CONFIG_AUTO_CAL_COMPUTE_START BIT(0) +#define EMC_AUTO_CAL_CONFIG_AUTO_CAL_MEASURE_STALL BIT(9) +#define EMC_AUTO_CAL_CONFIG_AUTO_CAL_UPDATE_STALL BIT(10) +#define EMC_AUTO_CAL_CONFIG_AUTO_CAL_ENABLE BIT(29) +#define EMC_AUTO_CAL_CONFIG_AUTO_CAL_START BIT(31) #define EMC_EMC_STATUS 0x2b4 +#define EMC_EMC_STATUS_MRR_DIVLD BIT(20) #define EMC_EMC_STATUS_TIMING_UPDATE_STALLED BIT(23) +#define EMC_EMC_STATUS_DRAM_IN_POWERDOWN_SHIFT 4 +#define EMC_EMC_STATUS_DRAM_IN_POWERDOWN_MASK \ + (0x3 << EMC_EMC_STATUS_DRAM_IN_POWERDOWN_SHIFT) +#define EMC_EMC_STATUS_DRAM_IN_SELF_REFRESH_SHIFT 8 +#define EMC_EMC_STATUS_DRAM_IN_SELF_REFRESH_MASK \ + (0x3 << EMC_EMC_STATUS_DRAM_IN_SELF_REFRESH_SHIFT) + +#define EMC_CFG_2 0x2b8 #define EMC_CFG_DIG_DLL 0x2bc #define EMC_CFG_DIG_DLL_CFG_DLL_EN BIT(0) #define EMC_CFG_DIG_DLL_CFG_DLL_STALL_ALL_UNTIL_LOCK BIT(1) @@ -191,8 +222,17 @@ #define EMC_WDV_MASK 0x2d0 #define EMC_RDV_EARLY_MASK 0x2d4 #define EMC_RDV_EARLY 0x2d8 +#define EMC_AUTO_CAL_CONFIG8 0x2dc #define EMC_ZCAL_INTERVAL 0x2e0 #define EMC_ZCAL_WAIT_CNT 0x2e4 +#define EMC_ZCAL_WAIT_CNT_ZCAL_WAIT_CNT_MASK 0x7ff +#define EMC_ZCAL_WAIT_CNT_ZCAL_WAIT_CNT_SHIFT 0 + +#define EMC_ZQ_CAL 0x2ec +#define EMC_ZQ_CAL_DEV_SEL_SHIFT 30 +#define EMC_ZQ_CAL_LONG BIT(4) +#define EMC_ZQ_CAL_ZQ_LATCH_CMD BIT(1) +#define EMC_ZQ_CAL_ZQ_CAL_CMD BIT(0) #define EMC_FDPD_CTRL_DQ 0x310 #define EMC_FDPD_CTRL_CMD 0x314 #define EMC_PMACRO_CMD_BRICK_CTRL_FDPD 0x318 @@ -202,6 +242,13 @@ #define EMC_TR_TIMING_0 0x3b4 #define EMC_TR_CTRL_1 0x3bc #define EMC_TR_RDV 0x3c4 +#define EMC_STALL_THEN_EXE_AFTER_CLKCHANGE 0x3cc +#define EMC_SEL_DPD_CTRL 0x3d8 +#define EMC_SEL_DPD_CTRL_DATA_SEL_DPD_EN BIT(8) +#define EMC_SEL_DPD_CTRL_ODT_SEL_DPD_EN BIT(5) +#define EMC_SEL_DPD_CTRL_RESET_SEL_DPD_EN BIT(4) +#define EMC_SEL_DPD_CTRL_CA_SEL_DPD_EN BIT(3) +#define EMC_SEL_DPD_CTRL_CLK_SEL_DPD_EN BIT(2) #define EMC_PRE_REFRESH_REQ_CNT 0x3dc #define EMC_DYN_SELF_REF_CONTROL 0x3e0 #define EMC_TXSRDLL 0x3e4 @@ -214,6 +261,9 @@ #define EMC_TR_RDV_MASK 0x3f8 #define EMC_TR_QSAFE 0x3fc #define EMC_TR_QRST 0x400 +#define EMC_ISSUE_QRST 0x428 +#define EMC_AUTO_CAL_CONFIG2 0x458 +#define EMC_AUTO_CAL_CONFIG3 0x45c #define EMC_TR_DVFS 0x460 #define EMC_AUTO_CAL_CHANNEL 0x464 #define EMC_IBDLY 0x468 @@ -227,19 +277,26 @@ #define EMC_MRW6 0x4a4 #define EMC_MRW7 0x4a8 #define EMC_MRW8 0x4ac +#define EMC_MRW9 0x4b0 #define EMC_MRW10 0x4b4 #define EMC_MRW11 0x4b8 #define EMC_MRW12 0x4bc #define EMC_MRW13 0x4c0 #define EMC_MRW14 0x4c4 #define EMC_MRW15 0x4d0 +#define EMC_CFG_SYNC 0x4d4 +#define EMC_FDPD_CTRL_CMD_NO_RAMP 0x4d8 +#define EMC_FDPD_CTRL_CMD_NO_RAMP_CMD_DPD_NO_RAMP_ENABLE BIT(0) #define EMC_WDV_CHK 0x4e0 #define EMC_CFG_PIPE_2 0x554 +#define EMC_CFG_PIPE_CLK 0x558 +#define EMC_CFG_PIPE_CLK_CLK_ALWAYS_ON BIT(0) #define EMC_CFG_PIPE_1 0x55c #define EMC_CFG_PIPE 0x560 #define EMC_QPOP 0x564 #define EMC_QUSE_WIDTH 0x568 #define EMC_PUTERM_WIDTH 0x56c +#define EMC_AUTO_CAL_CONFIG7 0x574 #define EMC_REFCTRL2 0x580 #define EMC_FBIO_CFG7 0x584 #define EMC_FBIO_CFG7_CH0_ENABLE BIT(1) @@ -304,10 +361,13 @@ #define EMC_CMD_BRLSHFT_2 0x5a4 #define EMC_CMD_BRLSHFT_3 0x5a8 #define EMC_QUSE_BRLSHFT_0 0x5ac +#define EMC_AUTO_CAL_CONFIG4 0x5b0 +#define EMC_AUTO_CAL_CONFIG5 0x5b4 #define EMC_QUSE_BRLSHFT_1 0x5b8 #define EMC_QUSE_BRLSHFT_2 0x5bc #define EMC_CCDMW 0x5c0 #define EMC_QUSE_BRLSHFT_3 0x5c4 +#define EMC_AUTO_CAL_CONFIG6 0x5cc #define EMC_DLL_CFG_0 0x5e4 #define EMC_DLL_CFG_1 0x5e8 #define EMC_DLL_CFG_1_DDLLCAL_CTRL_START_TRIM_SHIFT 10 @@ -315,6 +375,11 @@ (0x7ff << EMC_DLL_CFG_1_DDLLCAL_CTRL_START_TRIM_SHIFT) #define EMC_CONFIG_SAMPLE_DELAY 0x5f0 +#define EMC_CFG_UPDATE 0x5f4 +#define EMC_CFG_UPDATE_UPDATE_DLL_IN_UPDATE_SHIFT 9 +#define EMC_CFG_UPDATE_UPDATE_DLL_IN_UPDATE_MASK \ + (0x3 << EMC_CFG_UPDATE_UPDATE_DLL_IN_UPDATE_SHIFT) + #define EMC_PMACRO_QUSE_DDLL_RANK0_0 0x600 #define EMC_PMACRO_QUSE_DDLL_RANK0_1 0x604 #define EMC_PMACRO_QUSE_DDLL_RANK0_2 0x608 @@ -623,9 +688,20 @@ #define EMC_PMACRO_DDLL_SHORT_CMD_0 0xc20 #define EMC_PMACRO_DDLL_SHORT_CMD_1 0xc24 #define EMC_PMACRO_DDLL_SHORT_CMD_2 0xc28 +#define EMC_PMACRO_CFG_PM_GLOBAL_0 0xc30 +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE0 BIT(16) +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE1 BIT(17) +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE2 BIT(18) +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE3 BIT(19) +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE4 BIT(20) +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE5 BIT(21) +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE6 BIT(22) +#define EMC_PMACRO_CFG_PM_GLOBAL_0_DISABLE_CFG_BYTE7 BIT(23) #define EMC_PMACRO_VTTGEN_CTRL_0 0xc34 #define EMC_PMACRO_VTTGEN_CTRL_1 0xc38 #define EMC_PMACRO_BG_BIAS_CTRL_0 0xc3c +#define EMC_PMACRO_BG_BIAS_CTRL_0_BG_E_PWRD BIT(0) +#define EMC_PMACRO_BG_BIAS_CTRL_0_BGLP_E_PWRD BIT(2) #define EMC_PMACRO_PAD_CFG_CTRL 0xc40 #define EMC_PMACRO_ZCTRL 0xc44 #define EMC_PMACRO_CMD_PAD_RX_CTRL 0xc50 @@ -640,15 +716,22 @@ #define EMC_PMACRO_CMD_PAD_TX_CTRL_CMD_DQ_TX_DRVFORCEON BIT(26) #define EMC_PMACRO_DATA_PAD_TX_CTRL 0xc64 +#define EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQ_E_IVREF BIT(0) #define EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQ_TX_E_DCC BIT(1) +#define EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQS_E_IVREF BIT(8) #define EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQSP_TX_E_DCC BIT(9) #define EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_DQSN_TX_E_DCC BIT(16) #define EMC_PMACRO_DATA_PAD_TX_CTRL_DATA_CMD_TX_E_DCC BIT(24) #define EMC_PMACRO_COMMON_PAD_TX_CTRL 0xc68 #define EMC_PMACRO_AUTOCAL_CFG_COMMON 0xc78 +#define EMC_PMACRO_AUTOCAL_CFG_COMMON_E_CAL_BYPASS_DVFS BIT(16) #define EMC_PMACRO_VTTGEN_CTRL_2 0xcf0 #define EMC_PMACRO_IB_RXRT 0xcf4 +#define EMC_PMACRO_TRAINING_CTRL_0 0xcf8 +#define EMC_PMACRO_TRAINING_CTRL_0_CH0_TRAINING_E_WRPTR BIT(3) +#define EMC_PMACRO_TRAINING_CTRL_1 0xcfc +#define EMC_PMACRO_TRAINING_CTRL_1_CH1_TRAINING_E_WRPTR BIT(3) #define EMC_TRAINING_CTRL 0xe04 #define EMC_TRAINING_QUSE_CORS_CTRL 0xe0c #define EMC_TRAINING_QUSE_FINE_CTRL 0xe10 @@ -674,15 +757,31 @@ #define EMC_COPY_TABLE_PARAM_TRIM_REGS BIT(1) enum burst_regs_list { + EMC_RP_INDEX = 6, + EMC_R2P_INDEX = 9, + EMC_W2P_INDEX, + EMC_MRW6_INDEX = 31, EMC_REFRESH_INDEX = 41, EMC_PRE_REFRESH_REQ_CNT_INDEX = 43, + EMC_TRPAB_INDEX = 59, + EMC_MRW7_INDEX = 62, EMC_FBIO_CFG5_INDEX = 65, + EMC_FBIO_CFG7_INDEX, + EMC_CFG_DIG_DLL_INDEX, + EMC_ZCAL_INTERVAL_INDEX = 139, + EMC_ZCAL_WAIT_CNT_INDEX, + EMC_MRS_WAIT_CNT_INDEX = 141, EMC_DLL_CFG_0_INDEX = 144, + EMC_PMACRO_AUTOCAL_CFG_COMMON_INDEX = 146, + EMC_CFG_INDEX = 148, EMC_DYN_SELF_REF_CONTROL_INDEX = 150, EMC_PMACRO_CMD_PAD_TX_CTRL_INDEX = 161, EMC_PMACRO_DATA_PAD_TX_CTRL_INDEX, EMC_PMACRO_COMMON_PAD_TX_CTRL_INDEX, EMC_PMACRO_BRICK_CTRL_RFU1_INDEX = 167, + EMC_PMACRO_BG_BIAS_CTRL_0_INDEX = 171, + EMC_MRW14_INDEX = 199, + EMC_MRW15_INDEX = 220, }; enum trim_regs_list { @@ -935,6 +1034,9 @@ static inline u32 div_o3(u32 a, u32 b) return result; } +/* from tegra210-emc-r21021.c */ +extern const struct tegra210_emc_sequence tegra210_emc_r21021; + void tegra210_emc_do_clock_change(struct tegra210_emc *emc, u32 clksrc); void tegra210_emc_set_shadow_bypass(struct tegra210_emc *emc, int set); void tegra210_emc_timing_update(struct tegra210_emc *emc, From patchwork Tue Mar 10 15:20:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 11429629 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D2DB139A for ; Tue, 10 Mar 2020 15:20:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2BE4B21D56 for ; Tue, 10 Mar 2020 15:20:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LnPycMi9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727769AbgCJPU3 (ORCPT ); Tue, 10 Mar 2020 11:20:29 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:53579 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726605AbgCJPU2 (ORCPT ); Tue, 10 Mar 2020 11:20:28 -0400 Received: by mail-wm1-f66.google.com with SMTP id 25so1848799wmk.3; Tue, 10 Mar 2020 08:20:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ejespMMiPzZHhFOGLxs6wv6JPThWUWz2rdaye1ZX1xE=; b=LnPycMi9mVpe4/wyAd0bNohxb10FTZG9jnzxFevxgCgHPfdCGIgnkkAxDD/um2OBiK AzWW4Am9+08wylaO5SWwDJuSXRAJEyZSoLidL4bPzg/a37sFEIHHzDYZKeYf09hnTISp XxZTs2yndfERean3v/j59OSa51pdh5kLj4mky9AWwgXyNEjsL9szWzfU5fCXCppaTFna KhkI7gGvJ/Eq8nrRxJiYiXWxJeoN5u6RGrm7lJ8hbRL2IRtTGYuoWLJiqIa5/ha8krZZ tIAh+vxhZpmYc2jIhwF/UxYk25hNb3xACzACIhIfUgbrtTacCaaN51CULObvwoV6Qt4w oz2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ejespMMiPzZHhFOGLxs6wv6JPThWUWz2rdaye1ZX1xE=; b=cac1wJReMwpBuUMgGy/a3RW1/ENiZPAZC7sgo5keBgNCER3lfvs4ohPRhfM0RQ+d8G 613cnqgo0zoB4SZm8Ne1FAPuO4N//MMmSyvnkdyk1o+1/FQQl79FwpMWoerpnI0vZ5yU pMLgt6rbnTCou/dHyY5zDH0e6QfPr6dUZBf5kGccLWo46OMkkh7ZFYy6L11ykF+jUzA6 8tsbHbqY3XleqCcHxIgEsrmxcosFuNlYn3BQS9E++kN6zQ8NVbShZ0muA+RqTuX8Va7I CXu3J7irbYASFb1+wECtv89AEc5qvr1baHh5Pe4HcsY1sv29fXyNhnU71O34g0fhD+W2 wJ/w== X-Gm-Message-State: ANhLgQ0c2i6t4xuJgIuKkFwiosw9EyneoFAlzscEN88aDUKKHREutKiE kQZK16juv49HlVi8n4TUnis= X-Google-Smtp-Source: ADFU+vunebyxLiVsdW9gdZAEwa9042/kxrT+tUmYqJglzEE5qNmB4sB0o7H1+jzA747X4EKITn6DEQ== X-Received: by 2002:a05:600c:204a:: with SMTP id p10mr2630343wmg.55.1583853626226; Tue, 10 Mar 2020 08:20:26 -0700 (PDT) Received: from localhost (pD9E516A9.dip0.t-ipconnect.de. [217.229.22.169]) by smtp.gmail.com with ESMTPSA id o3sm4778931wme.36.2020.03.10.08.20.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 08:20:25 -0700 (PDT) From: Thierry Reding To: Thierry Reding Cc: Jon Hunter , Dmitry Osipenko , Rob Herring , Mark Rutland , Michael Turquette , Stephen Boyd , Joseph Lo , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 7/8] arm64: tegra: Add external memory controller node for Tegra210 Date: Tue, 10 Mar 2020 16:20:02 +0100 Message-Id: <20200310152003.2945170-8-thierry.reding@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310152003.2945170-1-thierry.reding@gmail.com> References: <20200310152003.2945170-1-thierry.reding@gmail.com> MIME-Version: 1.0 Sender: linux-clk-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org From: Joseph Lo Add external memory controller (EMC) node for Tegra210 Signed-off-by: Joseph Lo Signed-off-by: Thierry Reding --- Changes in v5: - drop list of clocks because we need very strict ordering that DT can't guarantee - drop hard-coded EMC table, bootloaders should add it dynamically arch/arm64/boot/dts/nvidia/tegra210.dtsi | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/arch/arm64/boot/dts/nvidia/tegra210.dtsi b/arch/arm64/boot/dts/nvidia/tegra210.dtsi index 64c46ce3849d..3fa92dd8350b 100644 --- a/arch/arm64/boot/dts/nvidia/tegra210.dtsi +++ b/arch/arm64/boot/dts/nvidia/tegra210.dtsi @@ -895,6 +895,17 @@ mc: memory-controller@70019000 { #iommu-cells = <1>; }; + external-memory-controller@7001b000 { + compatible = "nvidia,tegra210-emc"; + reg = <0x0 0x7001b000 0x0 0x1000>, + <0x0 0x7001e000 0x0 0x1000>, + <0x0 0x7001f000 0x0 0x1000>; + clocks = <&tegra_car TEGRA210_CLK_EMC>; + clock-names = "emc"; + interrupts = ; + nvidia,memory-controller = <&mc>; + }; + sata@70020000 { compatible = "nvidia,tegra210-ahci"; reg = <0x0 0x70027000 0x0 0x2000>, /* AHCI */ From patchwork Tue Mar 10 15:20:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 11429631 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8183F138D for ; Tue, 10 Mar 2020 15:20:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 614F020873 for ; Tue, 10 Mar 2020 15:20:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mg2P+mWI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726283AbgCJPUc (ORCPT ); Tue, 10 Mar 2020 11:20:32 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:34030 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726605AbgCJPUc (ORCPT ); Tue, 10 Mar 2020 11:20:32 -0400 Received: by mail-wr1-f68.google.com with SMTP id z15so16424549wrl.1; Tue, 10 Mar 2020 08:20:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ii2BrewVS3q4fYdTKcoF5y7GinLZa/uKgQ7LuFVDOiE=; b=mg2P+mWIwefea2Gb4MbVfGYZyYQ/wESleXFBAU4JH6aLt3sl8z0AHjHfh+G3NtU31g KomkkTDwg8i1XsIUw0klCGuKm+WEyRL+zw7v5GoA6yg7LiDR4xcTCd/f2T9wJ+DnwD39 6mHkCwo1Rut9S89gI+b3bwe4jUB5NI+f52D6HXft8JDw3Y8W68NYnLYf+m46yDFHIGX8 K4PXT7vjnudt0xVlGOfXB7jONl7ObhhavdF3Q+unKhWVDnWCqk4A7lKM1pgkLg6z2viA cUKnlyeiVmBN+0ixLZmksTGwJcGJ+NwLHSbt58y2qgzYFn6nZ/UdID6fOeyfL5j8M5L4 Ys1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ii2BrewVS3q4fYdTKcoF5y7GinLZa/uKgQ7LuFVDOiE=; b=g2Dkpci+WwxfQ1igumejYEC/NmgSzopSjoFVLuVeJxPE5Ux1PH5japDgvcDqOd1YtB mJp0tDB7Ah/PZ4rjSo7hmuHSe4//Z9vdaHL18uoai/pd2ujR3o4gx6fH+jzEOiEh9jPX 7+vIAVhhaRfvmL5Wr11nhvY5KhaELe3QD+VP60ZG9iHbfGrKrmsNyxgt2r/BdmWi02Sx M6vDr+OgkgmfzpOo+kJIgbW+t2veg5uT6latOG2+3sCFUQbif9i2dU5Vjf8LiwIUuhga wd4wjR+LyJOxZGIMVTURUelIfVz8dmml39CTlOEwyyymRTr68FEJmZNfkOBdy6SKpDFT yvFw== X-Gm-Message-State: ANhLgQ0d87z2niezE0gKh2cNyLMzYTwSVG2BtsYN31WVltmrWAXkXRd/ 6qfLHuxUu89CD8jCRlikFwA= X-Google-Smtp-Source: ADFU+vtqTs4uQrDAjryzWDvDnNDQaQ9//VXaPAg8U0G9Hcxghhfz7KMqFy3Tr5AdeGHNKXA3DcuEfw== X-Received: by 2002:a5d:6aca:: with SMTP id u10mr26376254wrw.99.1583853628261; Tue, 10 Mar 2020 08:20:28 -0700 (PDT) Received: from localhost (pD9E516A9.dip0.t-ipconnect.de. [217.229.22.169]) by smtp.gmail.com with ESMTPSA id t18sm4676558wml.17.2020.03.10.08.20.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2020 08:20:27 -0700 (PDT) From: Thierry Reding To: Thierry Reding Cc: Jon Hunter , Dmitry Osipenko , Rob Herring , Mark Rutland , Michael Turquette , Stephen Boyd , Joseph Lo , devicetree@vger.kernel.org, linux-tegra@vger.kernel.org, linux-clk@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 8/8] clk: tegra: Remove the old emc_mux clock for Tegra210 Date: Tue, 10 Mar 2020 16:20:03 +0100 Message-Id: <20200310152003.2945170-9-thierry.reding@gmail.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200310152003.2945170-1-thierry.reding@gmail.com> References: <20200310152003.2945170-1-thierry.reding@gmail.com> MIME-Version: 1.0 Sender: linux-clk-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-clk@vger.kernel.org From: Joseph Lo Remove the old emc_mux clock and don't use the common EMC clock definition. This will be replaced by a new clock defined in the EMC driver. Signed-off-by: Joseph Lo Signed-off-by: Thierry Reding --- drivers/clk/tegra/clk-tegra210.c | 46 ++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 17 deletions(-) diff --git a/drivers/clk/tegra/clk-tegra210.c b/drivers/clk/tegra/clk-tegra210.c index 0a5be781da60..53a570b1fc29 100644 --- a/drivers/clk/tegra/clk-tegra210.c +++ b/drivers/clk/tegra/clk-tegra210.c @@ -319,12 +319,6 @@ static unsigned long tegra210_input_freq[] = { [8] = 12000000, }; -static const char *mux_pllmcp_clkm[] = { - "pll_m", "pll_c", "pll_p", "clk_m", "pll_m_ud", "pll_mb", "pll_mb", - "pll_p", -}; -#define mux_pllmcp_clkm_idx NULL - #define PLL_ENABLE (1 << 30) #define PLLCX_MISC1_IDDQ (1 << 27) @@ -2336,7 +2330,6 @@ static struct tegra_clk tegra210_clks[tegra_clk_max] __initdata = { [tegra_clk_i2c2] = { .dt_id = TEGRA210_CLK_I2C2, .present = true }, [tegra_clk_uartc_8] = { .dt_id = TEGRA210_CLK_UARTC, .present = true }, [tegra_clk_mipi_cal] = { .dt_id = TEGRA210_CLK_MIPI_CAL, .present = true }, - [tegra_clk_emc] = { .dt_id = TEGRA210_CLK_EMC, .present = true }, [tegra_clk_usb2] = { .dt_id = TEGRA210_CLK_USB2, .present = true }, [tegra_clk_bsev] = { .dt_id = TEGRA210_CLK_BSEV, .present = true }, [tegra_clk_uartd_8] = { .dt_id = TEGRA210_CLK_UARTD, .present = true }, @@ -2987,6 +2980,27 @@ static const char * const sor1_parents[] = { static u32 sor1_parents_idx[] = { 0, 2, 5, 6 }; +static const struct clk_div_table mc_div_table_tegra210[] = { + { .val = 0, .div = 2 }, + { .val = 1, .div = 4 }, + { .val = 2, .div = 1 }, + { .val = 3, .div = 2 }, + { .val = 0, .div = 0 }, +}; + +static void tegra210_clk_register_mc(const char *name, + const char *parent_name) +{ + struct clk *clk; + + clk = clk_register_divider_table(NULL, name, parent_name, + CLK_IS_CRITICAL, + clk_base + CLK_SOURCE_EMC, + 15, 2, CLK_DIVIDER_READ_ONLY, + mc_div_table_tegra210, &emc_lock); + clks[TEGRA210_CLK_MC] = clk; +} + static const char * const sor1_out_parents[] = { /* * Bit 0 of the mux selects sor1_pad_clkout, irrespective of bit 1, so @@ -3029,7 +3043,8 @@ static const char * const la_parents[] = { static struct tegra_clk_periph tegra210_la = TEGRA_CLK_PERIPH(29, 7, 9, 0, 8, 1, TEGRA_DIVIDER_ROUND_UP, 76, 0, NULL, 0); -static __init void tegra210_periph_clk_init(void __iomem *clk_base, +static __init void tegra210_periph_clk_init(struct device_node *np, + void __iomem *clk_base, void __iomem *pmc_base) { struct clk *clk; @@ -3075,15 +3090,12 @@ static __init void tegra210_periph_clk_init(void __iomem *clk_base, CLK_SOURCE_LA, 0); clks[TEGRA210_CLK_LA] = clk; - /* emc mux */ - clk = clk_register_mux(NULL, "emc_mux", mux_pllmcp_clkm, - ARRAY_SIZE(mux_pllmcp_clkm), 0, - clk_base + CLK_SOURCE_EMC, - 29, 3, 0, &emc_lock); + /* emc */ + clk = tegra210_clk_register_emc(np, clk_base); + clks[TEGRA210_CLK_EMC] = clk; - clk = tegra_clk_register_mc("mc", "emc_mux", clk_base + CLK_SOURCE_EMC, - &emc_lock); - clks[TEGRA210_CLK_MC] = clk; + /* mc */ + tegra210_clk_register_mc("mc", "emc"); /* cml0 */ clk = clk_register_gate(NULL, "cml0", "pll_e", 0, clk_base + PLLE_AUX, @@ -3728,7 +3740,7 @@ static void __init tegra210_clock_init(struct device_node *np) tegra_fixed_clk_init(tegra210_clks); tegra210_pll_init(clk_base, pmc_base); - tegra210_periph_clk_init(clk_base, pmc_base); + tegra210_periph_clk_init(np, clk_base, pmc_base); tegra_audio_clk_init(clk_base, pmc_base, tegra210_clks, tegra210_audio_plls, ARRAY_SIZE(tegra210_audio_plls), 24576000);