From patchwork Wed Aug 1 08:19:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10551839 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5FEB014E2 for ; Wed, 1 Aug 2018 08:20:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5194E2AC33 for ; Wed, 1 Aug 2018 08:20:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45DB12AEC1; Wed, 1 Aug 2018 08:20:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0376B2AC33 for ; Wed, 1 Aug 2018 08:20:34 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7466989A74; Wed, 1 Aug 2018 08:20:32 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by gabe.freedesktop.org (Postfix) with ESMTPS id B1B35898E8; Wed, 1 Aug 2018 08:20:30 +0000 (UTC) Received: by mail-pf1-x442.google.com with SMTP id k19-v6so7463004pfi.1; Wed, 01 Aug 2018 01:20:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=FfNZ3vkejcDdZ3RL+qvpToofYePjIk6qctM3r052JZ4=; b=sDX/kN0mnaGq0Vc7bTISI9c6Fj5Z79LVDGEYFNkCDwdfiNpldFmDAtM3e8eKxi1dS3 XfD9iGtXigKSOVwNBaj64Y9nIbguU/mjEocDvwUJToRNbuj8cTyXy7wpKnB+1AYW/qLE FxgjcojuoorOaz3pithLsDP8Jvl2rE2XALB7BRH6d+Py443dMO75CnuZMuDtZzTmva7C 6AQEDVC52GLogAhpVEK4ZviP8gC7XYPhQ4Ok75hjkzc62MzIvpiKgSb/oU/x7vFOvbPL uxq0MfJAAcU5HtJA5BmYbtnSIdPk2y9rTBzJraF1tg6UhHhWhzKpGoxkOaOgPKZoOUU+ v2GA== X-Gm-Message-State: AOUpUlGGaC8hGdYITnFV/s1BQrGgHs3r+fzdkIhKecMv1QgViyl6Qnny HVftyDyBjVMIkpLwXS/K0jfJPxuV X-Google-Smtp-Source: AAOMgpd1xUrEx0c7D1eIv+brKE76+aAjh2j2O3lq1vmvNXtKjM1g+lvrWxt+zPJ/s9DVghHhl6e0Yg== X-Received: by 2002:a65:52cc:: with SMTP id z12-v6mr23675781pgp.69.1533111630110; Wed, 01 Aug 2018 01:20:30 -0700 (PDT) Received: from localhost.localdomain ([47.247.93.221]) by smtp.gmail.com with ESMTPSA id c1-v6sm13329283pfg.25.2018.08.01.01.20.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 01 Aug 2018 01:20:29 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 1/4] drm/scheduler: add a list of run queues to the entity Date: Wed, 1 Aug 2018 13:49:59 +0530 Message-Id: <20180801082002.20696-1-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP These are the potential run queues on which the jobs from this entity can be scheduled. We will use this to do load balancing. Signed-off-by: Nayan Deshmukh Reviewed-by: Christian König for the whole --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 8 ++++++++ include/drm/gpu_scheduler.h | 7 ++++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index 3f2fc5e8242a..a3eacc35cf98 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -179,6 +179,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, unsigned int num_rq_list, atomic_t *guilty) { + int i; + if (!(entity && rq_list && num_rq_list > 0 && rq_list[0])) return -EINVAL; @@ -186,6 +188,11 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, INIT_LIST_HEAD(&entity->list); entity->rq = rq_list[0]; entity->guilty = guilty; + entity->num_rq_list = num_rq_list; + entity->rq_list = kcalloc(num_rq_list, sizeof(struct drm_sched_rq *), + GFP_KERNEL); + for (i = 0; i < num_rq_list; ++i) + entity->rq_list[i] = rq_list[i]; entity->last_scheduled = NULL; spin_lock_init(&entity->rq_lock); @@ -363,6 +370,7 @@ void drm_sched_entity_fini(struct drm_sched_entity *entity) dma_fence_put(entity->last_scheduled); entity->last_scheduled = NULL; + kfree(entity->rq_list); } EXPORT_SYMBOL(drm_sched_entity_fini); diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index 091b9afcd184..a60896222a3e 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -50,7 +50,10 @@ enum drm_sched_priority { * * @list: used to append this struct to the list of entities in the * runqueue. - * @rq: runqueue to which this entity belongs. + * @rq: runqueue on which this entity is currently scheduled. + * @rq_list: a list of run queues on which jobs from this entity can + * be scheduled + * @num_rq_list: number of run queues in the rq_list * @rq_lock: lock to modify the runqueue to which this entity belongs. * @job_queue: the list of jobs of this entity. * @fence_seq: a linearly increasing seqno incremented with each @@ -74,6 +77,8 @@ enum drm_sched_priority { struct drm_sched_entity { struct list_head list; struct drm_sched_rq *rq; + struct drm_sched_rq **rq_list; + unsigned int num_rq_list; spinlock_t rq_lock; struct spsc_queue job_queue; From patchwork Wed Aug 1 08:20:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10551841 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BD96014E2 for ; Wed, 1 Aug 2018 08:20:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF2772AC33 for ; Wed, 1 Aug 2018 08:20:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A11D62AEC1; Wed, 1 Aug 2018 08:20:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 510FE2AC33 for ; Wed, 1 Aug 2018 08:20:43 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 815A289B99; Wed, 1 Aug 2018 08:20:41 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by gabe.freedesktop.org (Postfix) with ESMTPS id 664FA89B99; Wed, 1 Aug 2018 08:20:40 +0000 (UTC) Received: by mail-pf1-x441.google.com with SMTP id u24-v6so7455293pfn.13; Wed, 01 Aug 2018 01:20:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5GlFr87IlwXBB0lwQWapQtux1m3YiQrIk6nkGRs6QqE=; b=HqKE8MCNn8Qg/EphEILRLQQr9PYYdR8oxqExA1bjbD6QX6REPz4YwjRCaZhBITVAUf kudKuPJPfKO61squyyrS7RoH3bH7dnuJZFtJkKsSOfov/3nSAlI1oZsYc7RfzQADF5fg bCrLLZ4q6X88EdsVxp8Qf2/GYDsnErlFIsL08FKYJckDBYMMD9tnp+KiiqGggDXmJu6X 7Kr5DO351N9Gq5dyhdfvOVyMdNd2MknPLFIf4Lhp5Sm0MyQD+Oph0BJczKpR5tCmkfx4 EZaep8NyvhDuKaTdgZrWi4r7jW1t/h7lMVAeVHXUzzcdTLeBX/Wb3oepvRPccR1g4um1 LyXw== X-Gm-Message-State: AOUpUlHSc+mUrG77VRdWaYD5+zJkDHHEnFm0XWVwB5d0dTH36VfAodOi KJJonUsHHZkthLDRpoCWlpkPzQXm X-Google-Smtp-Source: AAOMgpeFyruIqmIqp0+/EDC44FAQfATQL6ik3tCPC+W6ziAB8agHd0+j/Tsprz/t3pSKoyYedW1q0w== X-Received: by 2002:a62:8913:: with SMTP id v19-v6mr4374217pfd.127.1533111639745; Wed, 01 Aug 2018 01:20:39 -0700 (PDT) Received: from localhost.localdomain ([47.247.93.221]) by smtp.gmail.com with ESMTPSA id c1-v6sm13329283pfg.25.2018.08.01.01.20.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 01 Aug 2018 01:20:39 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 2/4] drm/scheduler: add counter for total jobs in scheduler Date: Wed, 1 Aug 2018 13:50:00 +0530 Message-Id: <20180801082002.20696-2-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180801082002.20696-1-nayan26deshmukh@gmail.com> References: <20180801082002.20696-1-nayan26deshmukh@gmail.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nayan Deshmukh --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 3 +++ include/drm/gpu_scheduler.h | 2 ++ 2 files changed, 5 insertions(+) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index a3eacc35cf98..375f6f7f6a93 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -549,6 +549,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job, trace_drm_sched_job(sched_job, entity); + atomic_inc(&entity->rq->sched->num_jobs); first = spsc_queue_push(&entity->job_queue, &sched_job->queue_node); /* first job wakes up scheduler */ @@ -836,6 +837,7 @@ static void drm_sched_process_job(struct dma_fence *f, struct dma_fence_cb *cb) dma_fence_get(&s_fence->finished); atomic_dec(&sched->hw_rq_count); + atomic_dec(&sched->num_jobs); drm_sched_fence_finished(s_fence); trace_drm_sched_process_job(s_fence); @@ -953,6 +955,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, INIT_LIST_HEAD(&sched->ring_mirror_list); spin_lock_init(&sched->job_list_lock); atomic_set(&sched->hw_rq_count, 0); + atomic_set(&sched->num_jobs, 0); atomic64_set(&sched->job_id_count, 0); /* Each scheduler will run on a seperate kernel thread */ diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index a60896222a3e..89881ce974a5 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -260,6 +260,7 @@ struct drm_sched_backend_ops { * @job_list_lock: lock to protect the ring_mirror_list. * @hang_limit: once the hangs by a job crosses this limit then it is marked * guilty and it will be considered for scheduling further. + * @num_jobs: the number of jobs in queue in the scheduler * * One scheduler is implemented for each hardware ring. */ @@ -277,6 +278,7 @@ struct drm_gpu_scheduler { struct list_head ring_mirror_list; spinlock_t job_list_lock; int hang_limit; + atomic_t num_jobs; }; int drm_sched_init(struct drm_gpu_scheduler *sched, From patchwork Wed Aug 1 08:20:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10551843 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1A7A69093 for ; Wed, 1 Aug 2018 08:20:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0DAF22AC33 for ; Wed, 1 Aug 2018 08:20:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 023BB2AEC4; Wed, 1 Aug 2018 08:20:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9C89F2AC33 for ; Wed, 1 Aug 2018 08:20:57 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 77AA189C60; Wed, 1 Aug 2018 08:20:56 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pl0-x241.google.com (mail-pl0-x241.google.com [IPv6:2607:f8b0:400e:c01::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id 64D7489C59; Wed, 1 Aug 2018 08:20:54 +0000 (UTC) Received: by mail-pl0-x241.google.com with SMTP id u11-v6so2897588plq.5; Wed, 01 Aug 2018 01:20:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=016Q9PkrAbG9/0CKW7BPjQEUD6O0co1Kz4mFwlEgvuM=; b=HvXK++U35+jxjoGuUsScHQcUNAt5iOP3zDoZ+MoS+3uCoRDslCG2Qs6zx7GGAl4kO7 lPoyDdKzJY3Z9AAYskFWq8c6Gu6HEyYyU/F5Tfen2yxyzApD/saOJNItYirpF8+giyyL ooqdRWacfxaY4KxUwX6SgdWSISfWqshY/vd4/k1HLaLsNjyxxA1k5n2nMiLXon/wSJf8 /FpivKq4Au4YEV1YEFWzqOQM6JQqjjL1kSqPUq95T4mL5ki/qb7guq0rNnFzDtxv3gUw QFDEsXNsjcAahG9YnyGgOHHCUMz7N/XEcamqANhJNCvfuBVda5aydyw3jF8f53LTSN+2 hl3Q== X-Gm-Message-State: AOUpUlHatY/traQCJh2hA8ubl6/FeTfnd45wf2Z8MeyIQyCphgdk9wXM o9fwhVfIIYwTA4XSz2NfVqWH9BWa X-Google-Smtp-Source: AAOMgpdxBBWWUFo2DRnux1maUocY8KfKcZIyFWWoeMq/xNqYb63iCHi/QxOyBe5AeEh0wmYElVh6Ng== X-Received: by 2002:a17:902:4d46:: with SMTP id o6-v6mr15807024plh.59.1533111653818; Wed, 01 Aug 2018 01:20:53 -0700 (PDT) Received: from localhost.localdomain ([47.247.93.221]) by smtp.gmail.com with ESMTPSA id c1-v6sm13329283pfg.25.2018.08.01.01.20.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 01 Aug 2018 01:20:53 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 3/4] drm/scheduler: add new function to get least loaded sched v2 Date: Wed, 1 Aug 2018 13:50:01 +0530 Message-Id: <20180801082002.20696-3-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180801082002.20696-1-nayan26deshmukh@gmail.com> References: <20180801082002.20696-1-nayan26deshmukh@gmail.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP The function selects the run queue from the rq_list with the least load. The load is decided by the number of jobs in a scheduler. v2: avoid using atomic read twice consecutively, instead store it locally Signed-off-by: Nayan Deshmukh --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index 375f6f7f6a93..fb4e542660b0 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -255,6 +255,31 @@ static bool drm_sched_entity_is_ready(struct drm_sched_entity *entity) return true; } +/** + * drm_sched_entity_get_free_sched - Get the rq from rq_list with least load + * + * @entity: scheduler entity + * + * Return the pointer to the rq with least load. + */ +static struct drm_sched_rq * +drm_sched_entity_get_free_sched(struct drm_sched_entity *entity) +{ + struct drm_sched_rq *rq = NULL; + unsigned int min_jobs = UINT_MAX, num_jobs; + int i; + + for (i = 0; i < entity->num_rq_list; ++i) { + num_jobs = atomic_read(&entity->rq_list[i]->sched->num_jobs); + if (num_jobs < min_jobs) { + min_jobs = num_jobs; + rq = entity->rq_list[i]; + } + } + + return rq; +} + static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f, struct dma_fence_cb *cb) { From patchwork Wed Aug 1 08:20:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10551845 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B278E9093 for ; Wed, 1 Aug 2018 08:21:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A517D2AC33 for ; Wed, 1 Aug 2018 08:21:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 996A92AEC1; Wed, 1 Aug 2018 08:21:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 569712AC33 for ; Wed, 1 Aug 2018 08:21:02 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EF20C88052; Wed, 1 Aug 2018 08:21:00 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6379889CB5; Wed, 1 Aug 2018 08:21:00 +0000 (UTC) Received: by mail-pf1-x441.google.com with SMTP id j8-v6so7460830pff.6; Wed, 01 Aug 2018 01:21:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dprZ81F77+rCkqJL0PmM8SUhfK1KuDE5/d12GO5ieNY=; b=EF1gn4r8VKDvWVtd3MQGLuoW9Zaj11l+Qq6vZCvnvB9thrnI5lNbdbjEqCmmym5f8F 5/buMe/rf/GGBKtJmOfAjud48qsfhhq29iX7v/J9pdmgx3L0v7fbhGFkkZIB6nLsGS8u JFCZ3ArdkezM77In967K0SA1h+BxKg42ROWmRrXKWJ1M8AcUUlHRKiWPUPtej4WyiKVV 6SxMBKehkkg0Apn8058B6SyILlX9Hu8eSVBeweF2RDIpVmVGc+RuOcsqG6pMBd1hXSiT OcjD9iovAxJSBE75tSSOSKKDPgWeKAX87vbqdZCtNg0H7aUh8iIxmiXvxmdU8Pt4ghqq i+Ew== X-Gm-Message-State: AOUpUlFy/QlEa5eMJ/eW9908SSmTCHupI+urvKkQPusy0tj7YWmi1AKc LI5430salun173TrjGaUdfwTTVIj X-Google-Smtp-Source: AAOMgpfronWTmVtfUAtI/gU8JTXB55971Yb2qP6e2HF1jfkF9WL1mNhBWNv0l5uaZs67n9kz/koG8w== X-Received: by 2002:a63:1403:: with SMTP id u3-v6mr23538274pgl.13.1533111659768; Wed, 01 Aug 2018 01:20:59 -0700 (PDT) Received: from localhost.localdomain ([47.247.93.221]) by smtp.gmail.com with ESMTPSA id c1-v6sm13329283pfg.25.2018.08.01.01.20.56 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 01 Aug 2018 01:20:59 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 4/4] drm/scheduler: move idle entities to scheduler with less load v2 Date: Wed, 1 Aug 2018 13:50:02 +0530 Message-Id: <20180801082002.20696-4-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180801082002.20696-1-nayan26deshmukh@gmail.com> References: <20180801082002.20696-1-nayan26deshmukh@gmail.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is the first attempt to move entities between schedulers to have dynamic load balancing. We just move entities with no jobs for now as moving the ones with jobs will lead to other compilcations like ensuring that the other scheduler does not remove a job from the current entity while we are moving. v2: remove unused variable and an unecessary check Signed-off-by: Nayan Deshmukh --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index fb4e542660b0..087fa479f7e0 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -539,6 +539,8 @@ drm_sched_entity_pop_job(struct drm_sched_entity *entity) if (!sched_job) return NULL; + sched_job->sched = sched; + sched_job->s_fence->sched = sched; while ((entity->dependency = sched->ops->dependency(sched_job, entity))) if (drm_sched_entity_add_dependency_cb(entity)) return NULL; @@ -569,11 +571,23 @@ drm_sched_entity_pop_job(struct drm_sched_entity *entity) void drm_sched_entity_push_job(struct drm_sched_job *sched_job, struct drm_sched_entity *entity) { - struct drm_gpu_scheduler *sched = sched_job->sched; - bool first = false; + struct drm_sched_rq *rq = entity->rq; + bool first = false, reschedule, idle; - trace_drm_sched_job(sched_job, entity); + idle = entity->last_scheduled == NULL || + dma_fence_is_signaled(entity->last_scheduled); + first = spsc_queue_count(&entity->job_queue) == 0; + reschedule = idle && first && (entity->num_rq_list > 1); + if (reschedule) { + rq = drm_sched_entity_get_free_sched(entity); + spin_lock(&entity->rq_lock); + drm_sched_rq_remove_entity(entity->rq, entity); + entity->rq = rq; + spin_unlock(&entity->rq_lock); + } + + trace_drm_sched_job(sched_job, entity); atomic_inc(&entity->rq->sched->num_jobs); first = spsc_queue_push(&entity->job_queue, &sched_job->queue_node); @@ -588,7 +602,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job, } drm_sched_rq_add_entity(entity->rq, entity); spin_unlock(&entity->rq_lock); - drm_sched_wakeup(sched); + drm_sched_wakeup(entity->rq->sched); } } EXPORT_SYMBOL(drm_sched_entity_push_job);