From patchwork Thu Jan 11 22:22:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thierry Reding X-Patchwork-Id: 10158791 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E382460170 for ; Thu, 11 Jan 2018 22:23:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D1507285FD for ; Thu, 11 Jan 2018 22:23:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C59612869C; Thu, 11 Jan 2018 22:23:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AAA56286F4 for ; Thu, 11 Jan 2018 22:23:01 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 477636E49D; Thu, 11 Jan 2018 22:22:58 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-qk0-x244.google.com (mail-qk0-x244.google.com [IPv6:2607:f8b0:400d:c09::244]) by gabe.freedesktop.org (Postfix) with ESMTPS id C90836E49D; Thu, 11 Jan 2018 22:22:56 +0000 (UTC) Received: by mail-qk0-x244.google.com with SMTP id r8so6385380qke.6; Thu, 11 Jan 2018 14:22:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xUPuVJlDK7rgg9K1PCOa1AKIOKy5fMjAxPrx1pVuM+g=; b=XC0eOyyEReBhTTO11Lk9oWrzTp5nAjEMSA7EUP4Uw0heJWYGCoc7ysoetaZwJ9mmnI ijBv7M3L8v0FI7Q5aKbtgOHsxM7Hm0dwwDa2Bb7hdzi8ueckzGNEVApJ3xUGsqjN/GGm FBv8c7SuH+Y8xZmPosyk+InY+yH/sk/SpFJuRYA+rf65DYUF5Gx1Yx5/SjmNVpODUms1 X7CiX5p6v4S2piifs6fLLeJs8ule9/XlTpl0YksCa+9hSnO8AimD7P86bkM8HUIS7S/y Jhiy9UKmkAkVYYK7ee+RwsDRow3PZjyUQOhB2SehRTWy8RyKO0bZklnB8d+o+PAhlFxz e47Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xUPuVJlDK7rgg9K1PCOa1AKIOKy5fMjAxPrx1pVuM+g=; b=PiqY0l9izegvc8SPnOLf12aq1fMyYjAUWdDNGoLgenZg2vEwKUirjfiSdIa4Zh8ZJb 45Ko9WB9sAbszODxdPS0jkFh1sk1lHuUNjKTIioGOP0HJCEGLiTpnp/QIa+rLyJgnaNF wHl6jLot5+Azg1ds3+NhzS5ZxTl4SefsBgHCNr5Gn6ofSnRydUbVQMYpyMurx+9Wv7xh ds6PFDM4nK2KPqm7f2YAYoPR9yxl+k5e8F/D8eOAdhSPUXmu5kZsYG+SF4TaDyJIA13H 1638xgoxE7BDihQkHmXP/LtVMQBIPMs61b7Hg/rNYhj70wR9GJ6gcSYnaH/nJH+3TaIN r2MQ== X-Gm-Message-State: AKwxytcUelbeHTfPSZaH9sTMHF4ILkAryWAsWVA6Jis62VRnCQI4gg+G vO7bFrXglW8ZYnDp++4QDdk= X-Google-Smtp-Source: ACJfBouuS04iMC85qcyJcTu0pXPwlLCOBtwlIRsNlgtdZggFPSZ2QFC1U/mH2s/MH2VlfH0FsN9pdA== X-Received: by 10.55.52.201 with SMTP id b192mr13535794qka.188.1515709375347; Thu, 11 Jan 2018 14:22:55 -0800 (PST) Received: from localhost (p200300E41F21CF00227DB10B7BBDB590.dip0.t-ipconnect.de. [2003:e4:1f21:cf00:227d:b10b:7bbd:b590]) by smtp.gmail.com with ESMTPSA id e63sm12992795qkb.46.2018.01.11.14.22.53 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 11 Jan 2018 14:22:54 -0800 (PST) From: Thierry Reding To: Thierry Reding Subject: [PATCH 1/3] gpu: host1x: Add support for DMA fences Date: Thu, 11 Jan 2018 23:22:47 +0100 Message-Id: <20180111222249.29105-2-thierry.reding@gmail.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180111222249.29105-1-thierry.reding@gmail.com> References: <20180111222249.29105-1-thierry.reding@gmail.com> Cc: linux-tegra@vger.kernel.org, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, Mikko Perttunen X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Mikko Perttunen Add an implementation of DMA fences backed by Host1x syncpoints, an interface to specify a prefence for job submissions. Before submission, prefences containing only Host1x syncpoints are waited by pushing wait commands to CDMA, whereas other fences are CPU-waited. Signed-off-by: Mikko Perttunen Signed-off-by: Thierry Reding --- drivers/gpu/host1x/Kconfig | 1 + drivers/gpu/host1x/Makefile | 1 + drivers/gpu/host1x/dev.h | 12 ++- drivers/gpu/host1x/fence.c | 202 +++++++++++++++++++++++++++++++++++++ drivers/gpu/host1x/fence.h | 28 +++++ drivers/gpu/host1x/hw/channel_hw.c | 36 +++++-- drivers/gpu/host1x/intr.c | 11 +- drivers/gpu/host1x/intr.h | 8 +- drivers/gpu/host1x/syncpt.c | 2 + include/linux/host1x.h | 12 ++- 10 files changed, 302 insertions(+), 11 deletions(-) create mode 100644 drivers/gpu/host1x/fence.c create mode 100644 drivers/gpu/host1x/fence.h diff --git a/drivers/gpu/host1x/Kconfig b/drivers/gpu/host1x/Kconfig index 91916326957f..e41032ebf16d 100644 --- a/drivers/gpu/host1x/Kconfig +++ b/drivers/gpu/host1x/Kconfig @@ -1,6 +1,7 @@ config TEGRA_HOST1X tristate "NVIDIA Tegra host1x driver" depends on ARCH_TEGRA || (ARM && COMPILE_TEST) + select DMA_SHARED_BUFFER select IOMMU_IOVA if IOMMU_SUPPORT help Driver for the NVIDIA Tegra host1x hardware. diff --git a/drivers/gpu/host1x/Makefile b/drivers/gpu/host1x/Makefile index b92016ce09b7..ae3a6edf74b4 100644 --- a/drivers/gpu/host1x/Makefile +++ b/drivers/gpu/host1x/Makefile @@ -9,6 +9,7 @@ host1x-y = \ job.o \ debug.o \ mipi.o \ + fence.o \ hw/host1x01.o \ hw/host1x02.o \ hw/host1x04.o \ diff --git a/drivers/gpu/host1x/dev.h b/drivers/gpu/host1x/dev.h index 43e9fabb43a1..c9071c9c443e 100644 --- a/drivers/gpu/host1x/dev.h +++ b/drivers/gpu/host1x/dev.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2012-2015, NVIDIA Corporation. + * Copyright (C) 2012-2016 NVIDIA CORPORATION. All rights reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms and conditions of the GNU General Public License, @@ -42,6 +42,7 @@ struct host1x_channel_ops { int (*init)(struct host1x_channel *channel, struct host1x *host, unsigned int id); int (*submit)(struct host1x_job *job); + void (*push_wait)(struct host1x_channel *ch, u32 id, u32 thresh); }; struct host1x_cdma_ops { @@ -117,6 +118,8 @@ struct host1x { struct clk *clk; struct reset_control *rst; + u64 fence_ctx_base; + struct iommu_group *group; struct iommu_domain *domain; struct iova_domain iova; @@ -250,6 +253,13 @@ static inline int host1x_hw_channel_submit(struct host1x *host, return host->channel_op->submit(job); } +static inline void host1x_hw_channel_push_wait(struct host1x *host, + struct host1x_channel *channel, + u32 id, u32 thresh) +{ + host->channel_op->push_wait(channel, id, thresh); +} + static inline void host1x_hw_cdma_start(struct host1x *host, struct host1x_cdma *cdma) { diff --git a/drivers/gpu/host1x/fence.c b/drivers/gpu/host1x/fence.c new file mode 100644 index 000000000000..3b056623ea64 --- /dev/null +++ b/drivers/gpu/host1x/fence.c @@ -0,0 +1,202 @@ +/* + * Copyright (C) 2016 NVIDIA CORPORATION. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include +#include +#include + +#include "fence.h" +#include "intr.h" +#include "syncpt.h" +#include "cdma.h" +#include "channel.h" +#include "dev.h" + +struct host1x_fence { + struct dma_fence base; + spinlock_t lock; + + struct host1x_syncpt *syncpt; + u32 threshold; + + struct host1x *host; + void *waiter; + + char timeline_name[10]; +}; + +static inline struct host1x_fence *to_host1x_fence(struct dma_fence *fence) +{ + return (struct host1x_fence *)fence; +} + +static const char *host1x_fence_get_driver_name(struct dma_fence *fence) +{ + return "host1x"; +} + +static const char *host1x_fence_get_timeline_name(struct dma_fence *fence) +{ + struct host1x_fence *f = to_host1x_fence(fence); + + return f->timeline_name; +} + +static bool host1x_fence_enable_signaling(struct dma_fence *fence) +{ + struct host1x_fence *f = to_host1x_fence(fence); + + if (host1x_syncpt_is_expired(f->syncpt, f->threshold)) + return false; + + return true; +} + +static bool host1x_fence_signaled(struct dma_fence *fence) +{ + struct host1x_fence *f = to_host1x_fence(fence); + + return host1x_syncpt_is_expired(f->syncpt, f->threshold); +} + +static void host1x_fence_release(struct dma_fence *fence) +{ + struct host1x_fence *f = to_host1x_fence(fence); + + if (f->waiter) + host1x_intr_put_ref(f->host, f->syncpt->id, f->waiter); + + kfree(f); +} + +const struct dma_fence_ops host1x_fence_ops = { + .get_driver_name = host1x_fence_get_driver_name, + .get_timeline_name = host1x_fence_get_timeline_name, + .enable_signaling = host1x_fence_enable_signaling, + .signaled = host1x_fence_signaled, + .wait = dma_fence_default_wait, + .release = host1x_fence_release, +}; + +static void host1x_fence_wait_single(struct host1x_fence *f, + struct host1x *host, + struct host1x_channel *ch) +{ + if (host1x_syncpt_is_expired(f->syncpt, f->threshold)) + return; + + host1x_hw_channel_push_wait(host, ch, f->syncpt->id, f->threshold); +} + +/** + * host1x_fence_is_waitable() - Check if DMA fence can be waited by hardware + * @fence: DMA fence + * + * Check is @fence is only backed by Host1x syncpoints and can therefore be + * waited using only hardware. + */ +bool host1x_fence_is_waitable(struct dma_fence *fence) +{ + struct dma_fence_array *array; + int i; + + array = to_dma_fence_array(fence); + if (!array) + return fence->ops == &host1x_fence_ops; + + for (i = 0; i < array->num_fences; ++i) { + if (array->fences[i]->ops != &host1x_fence_ops) + return false; + } + + return true; +} + +/** + * host1x_fence_wait() - Insert waits for fence into channel + * @fence: DMA fence + * @host: Host1x + * @ch: Host1x channel + * + * Inserts wait commands into Host1x channel fences in @fence. + * in @fence. @fence must only consist of syncpoint-backed fences. + * + * Return: 0 on success, -errno otherwise. + */ +int host1x_fence_wait(struct dma_fence *fence, struct host1x *host, + struct host1x_channel *ch) +{ + struct dma_fence_array *array; + int i = 0; + + if (!host1x_fence_is_waitable(fence)) + return -EINVAL; + + array = to_dma_fence_array(fence); + if (!array) { + host1x_fence_wait_single(to_host1x_fence(fence), host, ch); + return 0; + } + + for (i = 0; i < array->num_fences; ++i) { + host1x_fence_wait_single(to_host1x_fence(array->fences[i]), + host, ch); + } + + return 0; +} + +struct dma_fence *host1x_fence_create(struct host1x *host, + struct host1x_syncpt *syncpt, + u32 threshold) +{ + struct host1x_waitlist *waiter; + struct host1x_fence *f; + int err; + + f = kzalloc(sizeof(*f), GFP_KERNEL); + if (!f) + return NULL; + + waiter = kzalloc(sizeof(*waiter), GFP_KERNEL); + if (!waiter) { + kfree(f); + return NULL; + } + + f->host = host; + f->syncpt = syncpt; + f->threshold = threshold; + f->waiter = NULL; + snprintf(f->timeline_name, ARRAY_SIZE(f->timeline_name), + "%d", syncpt->id); + + spin_lock_init(&f->lock); + dma_fence_init(&f->base, &host1x_fence_ops, &f->lock, + host->fence_ctx_base + syncpt->id, threshold); + + err = host1x_intr_add_action(f->host, f->syncpt->id, f->threshold, + HOST1X_INTR_ACTION_SIGNAL_FENCE, f, + waiter, &f->waiter); + if (err) { + kfree(waiter); + dma_fence_put((struct dma_fence *)f); + return NULL; + } + + return (struct dma_fence *)f; +} +EXPORT_SYMBOL(host1x_fence_create); diff --git a/drivers/gpu/host1x/fence.h b/drivers/gpu/host1x/fence.h new file mode 100644 index 000000000000..5725c95c0f1b --- /dev/null +++ b/drivers/gpu/host1x/fence.h @@ -0,0 +1,28 @@ +/* + * Copyright (C) 2016 NVIDIA CORPORATION. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#ifndef __HOST1X_FENCE_H +#define __HOST1X_FENCE_H + +struct host1x; +struct host1x_channel; +struct dma_fence; + +bool host1x_fence_is_waitable(struct dma_fence *fence); +int host1x_fence_wait(struct dma_fence *fence, struct host1x *host, + struct host1x_channel *ch); + +#endif diff --git a/drivers/gpu/host1x/hw/channel_hw.c b/drivers/gpu/host1x/hw/channel_hw.c index 9af758785a11..d43828902248 100644 --- a/drivers/gpu/host1x/hw/channel_hw.c +++ b/drivers/gpu/host1x/hw/channel_hw.c @@ -1,7 +1,7 @@ /* * Tegra host1x Channel * - * Copyright (c) 2010-2013, NVIDIA Corporation. + * Copyright (C) 2010-2016 NVIDIA CORPORATION. All rights reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms and conditions of the GNU General Public License, @@ -16,6 +16,7 @@ * along with this program. If not, see . */ +#include #include #include @@ -23,6 +24,7 @@ #include "../channel.h" #include "../dev.h" +#include "../fence.h" #include "../intr.h" #include "../job.h" @@ -68,11 +70,26 @@ static void submit_gathers(struct host1x_job *job) u32 op1 = host1x_opcode_gather(g->words); u32 op2 = g->base + g->offset; + /* add a setclass for modules that require it */ + if (job->class) + host1x_cdma_push(cdma, + host1x_opcode_setclass(job->class, 0, 0), + HOST1X_OPCODE_NOP); + trace_write_gather(cdma, g->bo, g->offset, op1 & 0xffff); host1x_cdma_push(cdma, op1, op2); } } +static void channel_push_wait(struct host1x_channel *channel, + u32 id, u32 thresh) +{ + host1x_cdma_push(&channel->cdma, + host1x_opcode_setclass(HOST1X_CLASS_HOST1X, + host1x_uclass_wait_syncpt_r(), 1), + host1x_class_host_wait_syncpt(id, thresh)); +} + static inline void synchronize_syncpt_base(struct host1x_job *job) { struct host1x *host = dev_get_drvdata(job->channel->dev->parent); @@ -110,6 +127,16 @@ static int channel_submit(struct host1x_job *job) /* before error checks, return current max */ prev_max = job->syncpt_end = host1x_syncpt_read_max(sp); + if (job->prefence) { + if (host1x_fence_is_waitable(job->prefence)) { + host1x_fence_wait(job->prefence, host, job->channel); + } else { + err = dma_fence_wait(job->prefence, true); + if (err) + goto error; + } + } + /* get submit lock */ err = mutex_lock_interruptible(&ch->submitlock); if (err) @@ -151,12 +178,6 @@ static int channel_submit(struct host1x_job *job) job->syncpt_end = syncval; - /* add a setclass for modules that require it */ - if (job->class) - host1x_cdma_push(&ch->cdma, - host1x_opcode_setclass(job->class, 0, 0), - HOST1X_OPCODE_NOP); - submit_gathers(job); /* end CDMA submit & stash pinned hMems into sync queue */ @@ -212,4 +233,5 @@ static int host1x_channel_init(struct host1x_channel *ch, struct host1x *dev, static const struct host1x_channel_ops host1x_channel_ops = { .init = host1x_channel_init, .submit = channel_submit, + .push_wait = channel_push_wait }; diff --git a/drivers/gpu/host1x/intr.c b/drivers/gpu/host1x/intr.c index 8b4fad0ab35d..b3d51288243d 100644 --- a/drivers/gpu/host1x/intr.c +++ b/drivers/gpu/host1x/intr.c @@ -1,7 +1,7 @@ /* * Tegra host1x Interrupt Management * - * Copyright (c) 2010-2013, NVIDIA Corporation. + * Copyright (C) 2010-2016 NVIDIA CORPORATION. All rights reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms and conditions of the GNU General Public License, @@ -17,6 +17,7 @@ */ #include +#include #include #include #include @@ -133,12 +134,20 @@ static void action_wakeup_interruptible(struct host1x_waitlist *waiter) wake_up_interruptible(wq); } +static void action_signal_fence(struct host1x_waitlist *waiter) +{ + struct dma_fence *fence = waiter->data; + + dma_fence_signal(fence); +} + typedef void (*action_handler)(struct host1x_waitlist *waiter); static const action_handler action_handlers[HOST1X_INTR_ACTION_COUNT] = { action_submit_complete, action_wakeup, action_wakeup_interruptible, + action_signal_fence }; static void run_handlers(struct list_head completed[HOST1X_INTR_ACTION_COUNT]) diff --git a/drivers/gpu/host1x/intr.h b/drivers/gpu/host1x/intr.h index 1370c2bb75b8..6b2c090fa91c 100644 --- a/drivers/gpu/host1x/intr.h +++ b/drivers/gpu/host1x/intr.h @@ -1,7 +1,7 @@ /* * Tegra host1x Interrupt Management * - * Copyright (c) 2010-2013, NVIDIA Corporation. + * Copyright (C) 2010-2016 NVIDIA CORPORATION. All rights reserved. * * This program is free software; you can redistribute it and/or modify it * under the terms and conditions of the GNU General Public License, @@ -43,6 +43,12 @@ enum host1x_intr_action { */ HOST1X_INTR_ACTION_WAKEUP_INTERRUPTIBLE, + /* + * Signal a dma fence. + * 'data' points to a host1x_fence + */ + HOST1X_INTR_ACTION_SIGNAL_FENCE, + HOST1X_INTR_ACTION_COUNT }; diff --git a/drivers/gpu/host1x/syncpt.c b/drivers/gpu/host1x/syncpt.c index a2a952adc136..652803b610b2 100644 --- a/drivers/gpu/host1x/syncpt.c +++ b/drivers/gpu/host1x/syncpt.c @@ -18,6 +18,7 @@ #include #include +#include #include #include @@ -413,6 +414,7 @@ int host1x_syncpt_init(struct host1x *host) mutex_init(&host->syncpt_mutex); host->syncpt = syncpt; host->bases = bases; + host->fence_ctx_base = dma_fence_context_alloc(host->info->nb_pts); host1x_syncpt_restore(host); host1x_hw_syncpt_enable_protection(host); diff --git a/include/linux/host1x.h b/include/linux/host1x.h index ddf7f9ca86cc..fd4daa78768f 100644 --- a/include/linux/host1x.h +++ b/include/linux/host1x.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2009-2013, NVIDIA Corporation. All rights reserved. + * Copyright (C) 2009-2016 NVIDIA CORPORATION. All rights reserved. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by @@ -71,6 +71,7 @@ struct host1x_client { * host1x buffer objects */ +struct dma_fence; struct host1x_bo; struct sg_table; @@ -258,6 +259,9 @@ struct host1x_job { /* Add a channel wait for previous ops to complete */ bool serialize; + + /* Wait for prefence to complete before submitting */ + struct dma_fence *prefence; }; struct host1x_job *host1x_job_alloc(struct host1x_channel *ch, @@ -343,4 +347,10 @@ int tegra_mipi_enable(struct tegra_mipi_device *device); int tegra_mipi_disable(struct tegra_mipi_device *device); int tegra_mipi_calibrate(struct tegra_mipi_device *device); +struct host1x_fence; + +struct dma_fence *host1x_fence_create(struct host1x *host, + struct host1x_syncpt *syncpt, + u32 threshold); + #endif