From patchwork Tue Jul 31 10:37:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10550615 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61193139A for ; Tue, 31 Jul 2018 10:38:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5053B2A8BE for ; Tue, 31 Jul 2018 10:38:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 448442A8C5; Tue, 31 Jul 2018 10:38:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 00D052A8BE for ; Tue, 31 Jul 2018 10:38:09 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E9E056E343; Tue, 31 Jul 2018 10:38:07 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1E0256E33F; Tue, 31 Jul 2018 10:38:04 +0000 (UTC) Received: by mail-pf1-x442.google.com with SMTP id x17-v6so5918457pfh.5; Tue, 31 Jul 2018 03:38:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=FfNZ3vkejcDdZ3RL+qvpToofYePjIk6qctM3r052JZ4=; b=czWCqt6Rt3WktxfepY6G0l0h9y1oFxnfSoxeZJBcRcxYzZUaGUzxSq/u0NacMuh/Gd R7ojrxwYTsA9/8r0zXSDj407d/PU+ksVo/t9uA8mEPpq0L34gca7AgLxHJhu4jN9hFHX 0kBU00ySPlmSpThEOU9a8ZYLdYy5pz8/LmyjxbTGPgmoYyYHCV6pK2E+eU9J+Pclj3AK cdvueiWwM4ztkFfr1GAj58kLSRqU+BNtv4LIiDk3utyqigQcUrw4n3mkv6jaN6aq2p4l pRLeAzaKV/xgvwIKxSt0Bpsvu1a5fI7XzK1Ez3NHzq7r/RPEMIZxmZpnVYNL1D02OZYa 6mhw== X-Gm-Message-State: AOUpUlFFRDkapZ5ERIeAI4UUjG4k7A1hkFCIfM5hGfcKX+Jlv1MS7wXp hJgLAEKy30ZMClEciJjs0I02ELJk4tA= X-Google-Smtp-Source: AAOMgpfEGedhAUzt4fZO9gnT+zP/D/vIF9nL0KY/BE4aQzvrdkA1ZL9H2zrRTbsHBmd45iMiuqDPdg== X-Received: by 2002:a63:be4a:: with SMTP id g10-v6mr19678338pgo.378.1533033483123; Tue, 31 Jul 2018 03:38:03 -0700 (PDT) Received: from localhost.localdomain ([47.247.27.231]) by smtp.gmail.com with ESMTPSA id u13-v6sm23196084pfg.79.2018.07.31.03.37.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 31 Jul 2018 03:38:02 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 1/4] drm/scheduler: add a list of run queues to the entity Date: Tue, 31 Jul 2018 16:07:33 +0530 Message-Id: <20180731103736.7813-2-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180731103736.7813-1-nayan26deshmukh@gmail.com> References: <20180731103736.7813-1-nayan26deshmukh@gmail.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP These are the potential run queues on which the jobs from this entity can be scheduled. We will use this to do load balancing. Signed-off-by: Nayan Deshmukh --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 8 ++++++++ include/drm/gpu_scheduler.h | 7 ++++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index 3f2fc5e8242a..a3eacc35cf98 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -179,6 +179,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, unsigned int num_rq_list, atomic_t *guilty) { + int i; + if (!(entity && rq_list && num_rq_list > 0 && rq_list[0])) return -EINVAL; @@ -186,6 +188,11 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, INIT_LIST_HEAD(&entity->list); entity->rq = rq_list[0]; entity->guilty = guilty; + entity->num_rq_list = num_rq_list; + entity->rq_list = kcalloc(num_rq_list, sizeof(struct drm_sched_rq *), + GFP_KERNEL); + for (i = 0; i < num_rq_list; ++i) + entity->rq_list[i] = rq_list[i]; entity->last_scheduled = NULL; spin_lock_init(&entity->rq_lock); @@ -363,6 +370,7 @@ void drm_sched_entity_fini(struct drm_sched_entity *entity) dma_fence_put(entity->last_scheduled); entity->last_scheduled = NULL; + kfree(entity->rq_list); } EXPORT_SYMBOL(drm_sched_entity_fini); diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index 091b9afcd184..a60896222a3e 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -50,7 +50,10 @@ enum drm_sched_priority { * * @list: used to append this struct to the list of entities in the * runqueue. - * @rq: runqueue to which this entity belongs. + * @rq: runqueue on which this entity is currently scheduled. + * @rq_list: a list of run queues on which jobs from this entity can + * be scheduled + * @num_rq_list: number of run queues in the rq_list * @rq_lock: lock to modify the runqueue to which this entity belongs. * @job_queue: the list of jobs of this entity. * @fence_seq: a linearly increasing seqno incremented with each @@ -74,6 +77,8 @@ enum drm_sched_priority { struct drm_sched_entity { struct list_head list; struct drm_sched_rq *rq; + struct drm_sched_rq **rq_list; + unsigned int num_rq_list; spinlock_t rq_lock; struct spsc_queue job_queue; From patchwork Tue Jul 31 10:37:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10550617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3773514E2 for ; Tue, 31 Jul 2018 10:38:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 280B92A8BE for ; Tue, 31 Jul 2018 10:38:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C45B2A8C5; Tue, 31 Jul 2018 10:38:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CE7722A8BE for ; Tue, 31 Jul 2018 10:38:12 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 444C36E344; Tue, 31 Jul 2018 10:38:11 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pl0-x243.google.com (mail-pl0-x243.google.com [IPv6:2607:f8b0:400e:c01::243]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4D1246E342; Tue, 31 Jul 2018 10:38:07 +0000 (UTC) Received: by mail-pl0-x243.google.com with SMTP id t17-v6so6951843ply.13; Tue, 31 Jul 2018 03:38:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5GlFr87IlwXBB0lwQWapQtux1m3YiQrIk6nkGRs6QqE=; b=bII9i5nBRDP+iOmmIocgLouF2V366RzZSXzUXnAw9jcm3nZp8hbqbmzdJuAIroA6T+ KWWM6LybxqU0yll0YWgYtW7lyM7SForTlLfX3AbvidRASuwNq1UIT3LLu5xsw1ARt7xT OW5dMwhNo54eCRRCvdayIrUHRuG18HL1YeWD8+cB0IOo+bIX+xi3WJvzDe16p4ZAMv8I JK3NnHBbmNW099yInGGQc294ZKbcZc+r7cRZ5McxnUKjd6i1hOb3EnNJM6JOE0hNuBqi AAhtdFTdLsWL4Kqbh4u9bUOqhdI2Ao7rX/84NYnY7SmQFmmlXPbXmm4bjBUk52ZzlZI/ 3RIA== X-Gm-Message-State: AOUpUlGSsxkqZf04++YSHoqzkHMewMyvYU6hEuZ2IwQd//MAJZ3EqOwV uf1ASxek9ad3pmto+FEftCHZRp5wGrk= X-Google-Smtp-Source: AAOMgpfdgCI2Ma+3Jn8oMgj4bijCPqz+t/yhRJuJT6/jytStYTTXoBBIYWxDVLtGuMbybFUBMZSrAg== X-Received: by 2002:a17:902:3225:: with SMTP id y34-v6mr11991748plb.206.1533033486675; Tue, 31 Jul 2018 03:38:06 -0700 (PDT) Received: from localhost.localdomain ([47.247.27.231]) by smtp.gmail.com with ESMTPSA id u13-v6sm23196084pfg.79.2018.07.31.03.38.03 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 31 Jul 2018 03:38:06 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 2/4] drm/scheduler: add counter for total jobs in scheduler Date: Tue, 31 Jul 2018 16:07:34 +0530 Message-Id: <20180731103736.7813-3-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180731103736.7813-1-nayan26deshmukh@gmail.com> References: <20180731103736.7813-1-nayan26deshmukh@gmail.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Nayan Deshmukh --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 3 +++ include/drm/gpu_scheduler.h | 2 ++ 2 files changed, 5 insertions(+) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index a3eacc35cf98..375f6f7f6a93 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -549,6 +549,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job, trace_drm_sched_job(sched_job, entity); + atomic_inc(&entity->rq->sched->num_jobs); first = spsc_queue_push(&entity->job_queue, &sched_job->queue_node); /* first job wakes up scheduler */ @@ -836,6 +837,7 @@ static void drm_sched_process_job(struct dma_fence *f, struct dma_fence_cb *cb) dma_fence_get(&s_fence->finished); atomic_dec(&sched->hw_rq_count); + atomic_dec(&sched->num_jobs); drm_sched_fence_finished(s_fence); trace_drm_sched_process_job(s_fence); @@ -953,6 +955,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, INIT_LIST_HEAD(&sched->ring_mirror_list); spin_lock_init(&sched->job_list_lock); atomic_set(&sched->hw_rq_count, 0); + atomic_set(&sched->num_jobs, 0); atomic64_set(&sched->job_id_count, 0); /* Each scheduler will run on a seperate kernel thread */ diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index a60896222a3e..89881ce974a5 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -260,6 +260,7 @@ struct drm_sched_backend_ops { * @job_list_lock: lock to protect the ring_mirror_list. * @hang_limit: once the hangs by a job crosses this limit then it is marked * guilty and it will be considered for scheduling further. + * @num_jobs: the number of jobs in queue in the scheduler * * One scheduler is implemented for each hardware ring. */ @@ -277,6 +278,7 @@ struct drm_gpu_scheduler { struct list_head ring_mirror_list; spinlock_t job_list_lock; int hang_limit; + atomic_t num_jobs; }; int drm_sched_init(struct drm_gpu_scheduler *sched, From patchwork Tue Jul 31 10:37:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10550619 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3550214E2 for ; Tue, 31 Jul 2018 10:38:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24A5F2A8BE for ; Tue, 31 Jul 2018 10:38:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 193FF2A8C5; Tue, 31 Jul 2018 10:38:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D608B2A8BE for ; Tue, 31 Jul 2018 10:38:14 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1C02B6E347; Tue, 31 Jul 2018 10:38:12 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pl0-x243.google.com (mail-pl0-x243.google.com [IPv6:2607:f8b0:400e:c01::243]) by gabe.freedesktop.org (Postfix) with ESMTPS id DE97F6E344; Tue, 31 Jul 2018 10:38:10 +0000 (UTC) Received: by mail-pl0-x243.google.com with SMTP id d5-v6so1423258pll.4; Tue, 31 Jul 2018 03:38:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kKW+vNRiqTQpuCj67tpVWIBPOjKE+lFDcZPGt8HuxMc=; b=kn5qyjFx9+wxAMhNbKhwZId5Ea32UMgHTthvilyHW64YKK5w1H6Ad/LS/Kab3w3Cy+ chmWMc1wUUBXXdMexnfG8Rfrxzl7avlikyuDeLTT6gKp937v638mc7ZG7Wum7EzQJlOF QrVJH+jbD+pw0PnYlO0B9R2c6TX7530nhSuWZ9774R0sDugPBRI6LiIWkccNSGxEz5qI ZBhHd5a45HxHBPWTJ6I0piGc+JCp3pY+WQ1B1qSd0BPafJE6+yD1TKkcQbu4W18LTPmS IG3/oNgYED4Z7uP/pYXgilxcb+PyQoLmjKTx5NDCRKBjh+Y94UV9l/xqqK6W6KijRnfV XX3Q== X-Gm-Message-State: AOUpUlG1leCKAVnlz+Gm3N8+jjY2zCi4jAK+dsZtJ+23G05i2QweoW71 8nK3iLzY1rnf1/KBuDqGmd5/Z0REtTk= X-Google-Smtp-Source: AAOMgpf6EjBriPyDmqpKhJJ1zF9K4jv/DxjUVr0mCuJLI+m+dYndl9pmjhCoEbgebMSclv8V5Ckw6A== X-Received: by 2002:a17:902:4a0c:: with SMTP id w12-v6mr20090157pld.289.1533033490275; Tue, 31 Jul 2018 03:38:10 -0700 (PDT) Received: from localhost.localdomain ([47.247.27.231]) by smtp.gmail.com with ESMTPSA id u13-v6sm23196084pfg.79.2018.07.31.03.38.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 31 Jul 2018 03:38:09 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 3/4] drm/scheduler: add new function to get least loaded sched Date: Tue, 31 Jul 2018 16:07:35 +0530 Message-Id: <20180731103736.7813-4-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180731103736.7813-1-nayan26deshmukh@gmail.com> References: <20180731103736.7813-1-nayan26deshmukh@gmail.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP The function selects the run queue from the rq_list with the least load. The load is decided by the number of jobs in a scheduler. Signed-off-by: Nayan Deshmukh --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index 375f6f7f6a93..c67f65ad8f15 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -255,6 +255,32 @@ static bool drm_sched_entity_is_ready(struct drm_sched_entity *entity) return true; } +/** + * drm_sched_entity_get_free_sched - Get the rq from rq_list with least load + * + * @entity: scheduler entity + * + * Return the pointer to the rq with least load. + */ +static struct drm_sched_rq * +drm_sched_entity_get_free_sched(struct drm_sched_entity *entity) +{ + struct drm_sched_rq *rq = NULL; + unsigned int min_jobs = UINT_MAX; + int i; + + for (i = 0; i < entity->num_rq_list; ++i) { + if (atomic_read(&entity->rq_list[i]->sched->num_jobs) < + min_jobs) { + min_jobs = atomic_read( + &entity->rq_list[i]->sched->num_jobs); + rq = entity->rq_list[i]; + } + } + + return rq; +} + static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f, struct dma_fence_cb *cb) { From patchwork Tue Jul 31 10:37:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nayan Deshmukh X-Patchwork-Id: 10550621 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D12C0139A for ; Tue, 31 Jul 2018 10:38:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C12492A8D2 for ; Tue, 31 Jul 2018 10:38:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B5B6F2A8FC; Tue, 31 Jul 2018 10:38:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6E1652A8D2 for ; Tue, 31 Jul 2018 10:38:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8E79D6E34E; Tue, 31 Jul 2018 10:38:21 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by gabe.freedesktop.org (Postfix) with ESMTPS id 888146E34D; Tue, 31 Jul 2018 10:38:14 +0000 (UTC) Received: by mail-pf1-x443.google.com with SMTP id e13-v6so5911193pff.7; Tue, 31 Jul 2018 03:38:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LmitqUhrCD9pBquGeg+oxP9HMjCjWuPR1LVxkf8tBso=; b=RgR/49cBi6mHoPUHRca6EYZiNaX/JNtiS1mewbCJb1JNwImiN9OWiOVvf5eKVQ8m17 hboBc7RJtrF2Zp/51WmhUAddyUM/GGBMEBYIc/2OtKob051b7XpEr0u6/HnYWquTCl2p rpaOZLwSxO1caoJZR2m7ybbXj3b4oQGnnHqghGIZC+Y3YSYyvuwbuFCZHUkVy1tAm78c A2gNSH1aB80ZtnXOTgUXJx53EkpdYt4dRpT4pSHcHWXzM4kBpG5rkprvMXZt5FrcSMGg 9eSsiYjZUAwT5fLU7a12D7K5y+btxKpIrSMg6E97Rzmn4hGVE8D6zbdVA5DmmcfKCsAQ 2DFg== X-Gm-Message-State: AOUpUlHKxb79uedgq5qY8zrK32gxJWuNKN9MOEDbmB0bZjScqj/BuePz o2VoCJoRJPzaRfbLgdJtla/fs2RLYcg= X-Google-Smtp-Source: AAOMgpdvx5VmVaAVZhwmXa2S8OAVuK3SYYyq/eKZ0IxnMR1hi2CL1VxJYMRVxouR0l9vxOQssCqxgw== X-Received: by 2002:a63:8749:: with SMTP id i70-v6mr20177963pge.325.1533033493855; Tue, 31 Jul 2018 03:38:13 -0700 (PDT) Received: from localhost.localdomain ([47.247.27.231]) by smtp.gmail.com with ESMTPSA id u13-v6sm23196084pfg.79.2018.07.31.03.38.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 31 Jul 2018 03:38:13 -0700 (PDT) From: Nayan Deshmukh To: dri-devel@lists.freedesktop.org Subject: [PATCH 4/4] drm/scheduler: move idle entities to scheduler with less load Date: Tue, 31 Jul 2018 16:07:36 +0530 Message-Id: <20180731103736.7813-5-nayan26deshmukh@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180731103736.7813-1-nayan26deshmukh@gmail.com> References: <20180731103736.7813-1-nayan26deshmukh@gmail.com> X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nayan Deshmukh , amd-gfx@lists.freedesktop.org, christian.koenig@amd.com MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This is the first attempt to move entities between schedulers to have dynamic load balancing. We just move entities with no jobs for now as moving the ones with jobs will lead to other compilcations like ensuring that the other scheduler does not remove a job from the current entity while we are moving. Signed-off-by: Nayan Deshmukh --- drivers/gpu/drm/scheduler/gpu_scheduler.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/scheduler/gpu_scheduler.c b/drivers/gpu/drm/scheduler/gpu_scheduler.c index c67f65ad8f15..f665a84d48ef 100644 --- a/drivers/gpu/drm/scheduler/gpu_scheduler.c +++ b/drivers/gpu/drm/scheduler/gpu_scheduler.c @@ -540,6 +540,8 @@ drm_sched_entity_pop_job(struct drm_sched_entity *entity) if (!sched_job) return NULL; + sched_job->sched = sched; + sched_job->s_fence->sched = sched; while ((entity->dependency = sched->ops->dependency(sched_job, entity))) if (drm_sched_entity_add_dependency_cb(entity)) return NULL; @@ -570,16 +572,29 @@ drm_sched_entity_pop_job(struct drm_sched_entity *entity) void drm_sched_entity_push_job(struct drm_sched_job *sched_job, struct drm_sched_entity *entity) { - struct drm_gpu_scheduler *sched = sched_job->sched; - bool first = false; + struct drm_gpu_scheduler *sched = entity->rq->sched; + struct drm_sched_rq *rq = entity->rq; + bool first = false, reschedule, idle; - trace_drm_sched_job(sched_job, entity); + idle = entity->last_scheduled == NULL || + dma_fence_is_signaled(entity->last_scheduled); + first = spsc_queue_count(&entity->job_queue) == 0; + reschedule = idle && first && (entity->num_rq_list > 1); + + if (reschedule) { + rq = drm_sched_entity_get_free_sched(entity); + spin_lock(&entity->rq_lock); + drm_sched_rq_remove_entity(entity->rq, entity); + entity->rq = rq; + spin_unlock(&entity->rq_lock); + } + trace_drm_sched_job(sched_job, entity); atomic_inc(&entity->rq->sched->num_jobs); first = spsc_queue_push(&entity->job_queue, &sched_job->queue_node); /* first job wakes up scheduler */ - if (first) { + if (first || reschedule) { /* Add the entity to the run queue */ spin_lock(&entity->rq_lock); if (!entity->rq) { @@ -589,7 +604,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job, } drm_sched_rq_add_entity(entity->rq, entity); spin_unlock(&entity->rq_lock); - drm_sched_wakeup(sched); + drm_sched_wakeup(entity->rq->sched); } } EXPORT_SYMBOL(drm_sched_entity_push_job);