From patchwork Thu Jul 10 21:50:10 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 4528161 Return-Path: X-Original-To: patchwork-dri-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 17DD2BEEAA for ; Thu, 10 Jul 2014 23:33:30 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 235DD20204 for ; Thu, 10 Jul 2014 23:33:29 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.kernel.org (Postfix) with ESMTP id EF342201DE for ; Thu, 10 Jul 2014 23:33:27 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 32D1C6E779; Thu, 10 Jul 2014 16:33:14 -0700 (PDT) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wi0-f181.google.com (mail-wi0-f181.google.com [209.85.212.181]) by gabe.freedesktop.org (Postfix) with ESMTP id ECA096E01B for ; Thu, 10 Jul 2014 14:51:19 -0700 (PDT) Received: by mail-wi0-f181.google.com with SMTP id bs8so414248wib.8 for ; Thu, 10 Jul 2014 14:51:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rWshBRswsSUa/yGBkthvqxwXc6Ck8oVQ5MQQhD+6PJQ=; b=E5oR+wyXtbvhUWSRlUXyC3x5vXAgHWb92g7A4UTMs3oq+gOJV0rAxd+ZfNmAN426Nq NxpfGCXngAkpvZ+QVa7x9/gkrHG5TCbacY0TummIsiB1Tlo4olm5HfUTTcHXa+19Na8e Ggzmhw7Ujglbz2rK4a7AcIK6QHuErCOqU3c2OZaiMUit5ZYQ5DGfPGev2yxRs99vTSZA 1jgYoqzAt7gldAwQHxFRQBLS3eLv0NrlDhc7L8JRry/PDVwmfpxRAgex6448IdfZXgLo LdRnjK/3GRM6De5N2hpj2tgydyhQFReLHIUblHeMbor/G3POj65f1ZUQI+i6zl5+vRzj wdRA== X-Received: by 10.180.206.144 with SMTP id lo16mr312592wic.52.1405029079228; Thu, 10 Jul 2014 14:51:19 -0700 (PDT) Received: from localhost.localdomain ([77.127.59.49]) by mx.google.com with ESMTPSA id n2sm805353wjf.40.2014.07.10.14.51.17 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 10 Jul 2014 14:51:18 -0700 (PDT) From: Oded Gabbay X-Google-Original-From: Oded Gabbay To: David Airlie , Alex Deucher , Jerome Glisse Subject: [PATCH 10/83] hsa/radeon: Add initialization and unmapping of doorbell aperture Date: Fri, 11 Jul 2014 00:50:10 +0300 Message-Id: <1405029027-6085-9-git-send-email-oded.gabbay@amd.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1405029027-6085-1-git-send-email-oded.gabbay@amd.com> References: <1405029027-6085-1-git-send-email-oded.gabbay@amd.com> X-Mailman-Approved-At: Thu, 10 Jul 2014 16:33:12 -0700 Cc: Andrew Lewycky , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Spam-Status: No, score=-4.0 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_WEB, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds initialization of the doorbell aperture when initializing a kfd device. It also adds a call to unmap the doorbell when a process unbinds from the kfd Signed-off-by: Oded Gabbay --- drivers/gpu/hsa/radeon/Makefile | 3 +- drivers/gpu/hsa/radeon/kfd_device.c | 2 + drivers/gpu/hsa/radeon/kfd_doorbell.c | 72 +++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/hsa/radeon/kfd_doorbell.c diff --git a/drivers/gpu/hsa/radeon/Makefile b/drivers/gpu/hsa/radeon/Makefile index ba16a09..989518a 100644 --- a/drivers/gpu/hsa/radeon/Makefile +++ b/drivers/gpu/hsa/radeon/Makefile @@ -3,6 +3,7 @@ # radeon_kfd-y := kfd_module.o kfd_device.o kfd_chardev.o \ - kfd_pasid.o kfd_topology.o kfd_process.o + kfd_pasid.o kfd_topology.o kfd_process.o \ + kfd_doorbell.o obj-$(CONFIG_HSA_RADEON) += radeon_kfd.o diff --git a/drivers/gpu/hsa/radeon/kfd_device.c b/drivers/gpu/hsa/radeon/kfd_device.c index d122920..4e9fe6c 100644 --- a/drivers/gpu/hsa/radeon/kfd_device.c +++ b/drivers/gpu/hsa/radeon/kfd_device.c @@ -123,6 +123,8 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, kfd->regs = gpu_resources->mmio_registers; + radeon_kfd_doorbell_init(kfd); + if (!device_iommu_pasid_init(kfd)) return false; diff --git a/drivers/gpu/hsa/radeon/kfd_doorbell.c b/drivers/gpu/hsa/radeon/kfd_doorbell.c new file mode 100644 index 0000000..79a9d4b --- /dev/null +++ b/drivers/gpu/hsa/radeon/kfd_doorbell.c @@ -0,0 +1,72 @@ +/* + * Copyright 2014 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + */ + +#include "kfd_priv.h" +#include +#include + +/* + * Each device exposes a doorbell aperture, a PCI MMIO aperture that + * receives 32-bit writes that are passed to queues as wptr values. + * The doorbells are intended to be written by applications as part + * of queueing work on user-mode queues. + * We assign doorbells to applications in PAGE_SIZE-sized and aligned chunks. + * We map the doorbell address space into user-mode when a process creates + * its first queue on each device. + * Although the mapping is done by KFD, it is equivalent to an mmap of + * the /dev/kfd with the particular device encoded in the mmap offset. + * There will be other uses for mmap of /dev/kfd, so only a range of + * offsets (KFD_MMAP_DOORBELL_START-END) is used for doorbells. + */ + +/* # of doorbell bytes allocated for each process. */ +static inline size_t doorbell_process_allocation(void) +{ + return roundup(sizeof(doorbell_t) * MAX_PROCESS_QUEUES, PAGE_SIZE); +} + +/* Doorbell calculations for device init. */ +void radeon_kfd_doorbell_init(struct kfd_dev *kfd) +{ + size_t doorbell_start_offset; + size_t doorbell_aperture_size; + size_t doorbell_process_limit; + + /* We start with calculations in bytes because the input data might only be byte-aligned. + ** Only after we have done the rounding can we assume any alignment. */ + + doorbell_start_offset = roundup(kfd->shared_resources.doorbell_start_offset, + doorbell_process_allocation()); + doorbell_aperture_size = rounddown(kfd->shared_resources.doorbell_aperture_size, + doorbell_process_allocation()); + + if (doorbell_aperture_size > doorbell_start_offset) + doorbell_process_limit = + (doorbell_aperture_size - doorbell_start_offset) / doorbell_process_allocation(); + else + doorbell_process_limit = 0; + + kfd->doorbell_base = kfd->shared_resources.doorbell_physical_address + doorbell_start_offset; + kfd->doorbell_id_offset = doorbell_start_offset / sizeof(doorbell_t); + kfd->doorbell_process_limit = doorbell_process_limit; +} +