From patchwork Wed Dec 12 13:06:57 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 1866281 Return-Path: X-Original-To: patchwork-dri-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by patchwork1.kernel.org (Postfix) with ESMTP id 517D44006E for ; Wed, 12 Dec 2012 13:43:33 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 34719E6585 for ; Wed, 12 Dec 2012 05:43:33 -0800 (PST) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-ea0-f177.google.com (mail-ea0-f177.google.com [209.85.215.177]) by gabe.freedesktop.org (Postfix) with ESMTP id 23A78E62D6 for ; Wed, 12 Dec 2012 05:07:41 -0800 (PST) Received: by mail-ea0-f177.google.com with SMTP id c10so219419eaa.36 for ; Wed, 12 Dec 2012 05:07:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references; bh=rpQiIqUw1fg/4VBpG9w44F0yUBYX/xGIqV8gViWd1HY=; b=UA3zaNH+IYvOwWl1Qnrce0vsGuMsjtPSrcLPF/auLWQllnZhdmwsVaAEekoenJWY5w TYdzar9pOw8H6c5u8Q6M9PLNaUm9g83/02NUnb8B7y2pJWAJ4FSNqC6sPewtupsoLjHR zoB8E9KJfzvRWb4EXXizhA/vPe7dGR4yrA1d0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references :x-gm-message-state; bh=rpQiIqUw1fg/4VBpG9w44F0yUBYX/xGIqV8gViWd1HY=; b=ktvn5p2eiiOroqj2hsVQeqCB5mdngUNC1X36OvWiiWf6BIGO08ilx8rAjNrvYQzk90 IDbPlKY5Uq6NTgbYpVkSOCXYa7Nb9yMC10uwFEgp/00JSeT5+HBWopYAwbihnA79WMNz DEk4CK5fMC5FCnUUscW8r4Ah7Ha2556L+RYvQZU2MJ0QhglQ6wk0zAW45V8KeNN8XBXU RJa4SN4RIRbqlVrVXJ+uauyJW20FYvY94LaLdm76Tdf+KxLMweTA3CXD1ya/ubR7u/3T x3b5fVvYyhGq+SYGHRdrV67V2rzETOsasWwXSBIxLVfcKKh5f3UxkVcZ6ysj+PvqPl+K xp+g== Received: by 10.14.225.194 with SMTP id z42mr2718527eep.22.1355317661026; Wed, 12 Dec 2012 05:07:41 -0800 (PST) Received: from biers.ffwll.local (178-83-130-250.dynamic.hispeed.ch. [178.83.130.250]) by mx.google.com with ESMTPS id r1sm55868541eeo.2.2012.12.12.05.07.40 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 12 Dec 2012 05:07:40 -0800 (PST) From: Daniel Vetter To: DRI Development Subject: [PATCH 17/37] drm: revamp locking around fb creation/destruction Date: Wed, 12 Dec 2012 14:06:57 +0100 Message-Id: <1355317637-16742-18-git-send-email-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1355317637-16742-1-git-send-email-daniel.vetter@ffwll.ch> References: <1355317637-16742-1-git-send-email-daniel.vetter@ffwll.ch> X-Gm-Message-State: ALoCoQkr1G2/HqNe9iYPxa0X2VRdwhUI0jB03uimmdN1PyQvLDI0hKGa/bSwyBkV0Gn2SStX9Evf Cc: Nouveau Dev , Intel Graphics Development , Radeon Dev , Daniel Vetter X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dri-devel-bounces+patchwork-dri-devel=patchwork.kernel.org@lists.freedesktop.org Errors-To: dri-devel-bounces+patchwork-dri-devel=patchwork.kernel.org@lists.freedesktop.org Well, at least step 1. The goal here is that framebuffer objects can survive outside of the mode_config lock, with just a reference held as protection. The first step to get there is to introduce a special fb_lock which protects fb lookup, creation and destruction, to make them appear atomic. This new fb_lock can nest within the mode_config lock. But the idea is (once the reference counting part is completed) that we only quickly take that fb_lock to lookup a framebuffer and grab a reference, without any other locks involved. vmwgfx is the only driver which does framebuffer lookups itself, also wrap those calls to drm_mode_object_find with the new lock. Also protect the fb_list walking in i915 and omapdrm with the new lock. As a slight complication there's also the list of user-created fbs attached to the file private. The problem now is that at fclose() time we need to walk that list, eventually do a modeset call to remove the fb from active usage (and are required to be able to take the mode_config lock), but in the end we need to grab the new fb_lock to remove the fb from the list. The easiest solution is to add another mutex to protect this per-file list. Currently that new fbs_lock nests within the modeset locks and so appears redudant. But later patches will switch around this sequence so that taking the modeset locks in the fb destruction path is optional in the fastpath. Ultimately the goal is that addfb and rmfb do not require the mode_config lock, since otherwise they have the potential to introduce stalls in the pageflip sequence of a compositor (if the compositor e.g. switches to a fullscreen client or if it enables a plane). But that requires a few more steps and hoops to jump through. Note that framebuffer creation/destruction is now double-protected - once by the fb_lock and in parts by the idr_lock. The later would be unnecessariy if framebuffers would have their own idr allocator. But that's material for another patch (series). v2: Properly initialize the fb->filp_head list in _init, otherwise the newly added WARN to check whether the fb isn't on a fpriv list any more will fail for driver-private objects. Signed-off-by: Daniel Vetter --- drivers/gpu/drm/drm_crtc.c | 113 ++++++++++++++++++++++---------- drivers/gpu/drm/drm_fops.c | 1 + drivers/gpu/drm/i915/i915_debugfs.c | 5 +- drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c | 4 ++ drivers/staging/omapdrm/omap_debugfs.c | 2 + include/drm/drmP.h | 8 +++ include/drm/drm_crtc.h | 14 ++++ 7 files changed, 111 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c index 3b2f25d..5a46ea1 100644 --- a/drivers/gpu/drm/drm_crtc.c +++ b/drivers/gpu/drm/drm_crtc.c @@ -262,15 +262,21 @@ again: mutex_lock(&dev->mode_config.idr_mutex); ret = idr_get_new_above(&dev->mode_config.crtc_idr, obj, 1, &new_id); + + if (!ret) { + /* + * Set up the object linking under the protection of the idr + * lock so that other users can't see inconsistent state. + */ + obj->id = new_id; + obj->type = obj_type; + } mutex_unlock(&dev->mode_config.idr_mutex); + if (ret == -EAGAIN) goto again; - else if (ret) - return ret; - obj->id = new_id; - obj->type = obj_type; - return 0; + return ret; } /** @@ -312,6 +318,12 @@ EXPORT_SYMBOL(drm_mode_object_find); * Allocates an ID for the framebuffer's parent mode object, sets its mode * functions & device file and adds it to the master fd list. * + * IMPORTANT: + * This functions publishes the fb and makes it available for concurrent access + * by other users. Which means by this point the fb _must_ be fully set up - + * since all the fb attributes are invariant over its lifetime, no further + * locking but only correct reference counting is required. + * * RETURNS: * Zero on success, error code on failure. */ @@ -320,16 +332,19 @@ int drm_framebuffer_init(struct drm_device *dev, struct drm_framebuffer *fb, { int ret; + mutex_lock(&dev->mode_config.fb_lock); kref_init(&fb->refcount); + INIT_LIST_HEAD(&fb->filp_head); + fb->dev = dev; + fb->funcs = funcs; ret = drm_mode_object_get(dev, &fb->base, DRM_MODE_OBJECT_FB); if (ret) return ret; - fb->dev = dev; - fb->funcs = funcs; dev->mode_config.num_fb++; list_add(&fb->head, &dev->mode_config.fb_list); + mutex_unlock(&dev->mode_config.fb_lock); return 0; } @@ -387,8 +402,10 @@ void drm_framebuffer_cleanup(struct drm_framebuffer *fb) * this.) */ drm_mode_object_put(dev, &fb->base); + mutex_lock(&dev->mode_config.fb_lock); list_del(&fb->head); dev->mode_config.num_fb--; + mutex_unlock(&dev->mode_config.fb_lock); } EXPORT_SYMBOL(drm_framebuffer_cleanup); @@ -408,6 +425,7 @@ void drm_framebuffer_remove(struct drm_framebuffer *fb) int ret; WARN_ON(!drm_modeset_is_locked(dev)); + WARN_ON(!list_empty(&fb->filp_head)); /* remove from any CRTC */ list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { @@ -434,8 +452,6 @@ void drm_framebuffer_remove(struct drm_framebuffer *fb) } } - list_del(&fb->filp_head); - drm_framebuffer_unreference(fb); } EXPORT_SYMBOL(drm_framebuffer_remove); @@ -991,6 +1007,7 @@ void drm_mode_config_init(struct drm_device *dev) { mutex_init(&dev->mode_config.mutex); mutex_init(&dev->mode_config.idr_mutex); + mutex_init(&dev->mode_config.fb_lock); INIT_LIST_HEAD(&dev->mode_config.fb_list); INIT_LIST_HEAD(&dev->mode_config.crtc_list); INIT_LIST_HEAD(&dev->mode_config.connector_list); @@ -1093,6 +1110,9 @@ void drm_mode_config_cleanup(struct drm_device *dev) drm_property_destroy(dev, property); } + /* Single-threaded teardown context, so it's not requied to grab the + * fb_lock to protect against concurrent fb_list access. Contrary, it + * would actually deadlock with the drm_framebuffer_cleanup function. */ list_for_each_entry_safe(fb, fbt, &dev->mode_config.fb_list, head) { drm_framebuffer_remove(fb); } @@ -1222,8 +1242,8 @@ int drm_mode_getresources(struct drm_device *dev, void *data, if (!drm_core_check_feature(dev, DRIVER_MODESET)) return -EINVAL; - drm_modeset_lock_all(dev); + mutex_lock(&file_priv->fbs_lock); /* * For the non-control nodes we need to limit the list of resources * by IDs in the group list for this node @@ -1231,6 +1251,23 @@ int drm_mode_getresources(struct drm_device *dev, void *data, list_for_each(lh, &file_priv->fbs) fb_count++; + /* handle this in 4 parts */ + /* FBs */ + if (card_res->count_fbs >= fb_count) { + copied = 0; + fb_id = (uint32_t __user *)(unsigned long)card_res->fb_id_ptr; + list_for_each_entry(fb, &file_priv->fbs, filp_head) { + if (put_user(fb->base.id, fb_id + copied)) { + ret = -EFAULT; + goto out; + } + copied++; + } + } + card_res->count_fbs = fb_count; + mutex_unlock(&file_priv->fbs_lock); + + drm_modeset_lock_all(dev); mode_group = &file_priv->master->minor->mode_group; if (file_priv->master->minor->type == DRM_MINOR_CONTROL) { @@ -1254,21 +1291,6 @@ int drm_mode_getresources(struct drm_device *dev, void *data, card_res->max_width = dev->mode_config.max_width; card_res->min_width = dev->mode_config.min_width; - /* handle this in 4 parts */ - /* FBs */ - if (card_res->count_fbs >= fb_count) { - copied = 0; - fb_id = (uint32_t __user *)(unsigned long)card_res->fb_id_ptr; - list_for_each_entry(fb, &file_priv->fbs, filp_head) { - if (put_user(fb->base.id, fb_id + copied)) { - ret = -EFAULT; - goto out; - } - copied++; - } - } - card_res->count_fbs = fb_count; - /* CRTCs */ if (card_res->count_crtcs >= crtc_count) { copied = 0; @@ -1767,8 +1789,10 @@ int drm_mode_setplane(struct drm_device *dev, void *data, } crtc = obj_to_crtc(obj); + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, plane_req->fb_id, DRM_MODE_OBJECT_FB); + mutex_unlock(&dev->mode_config.fb_lock); if (!obj) { DRM_DEBUG_KMS("Unknown framebuffer ID %d\n", plane_req->fb_id); @@ -1895,8 +1919,10 @@ int drm_mode_setcrtc(struct drm_device *dev, void *data, } fb = crtc->fb; } else { + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, crtc_req->fb_id, DRM_MODE_OBJECT_FB); + mutex_unlock(&dev->mode_config.fb_lock); if (!obj) { DRM_DEBUG_KMS("Unknown FB ID%d\n", crtc_req->fb_id); @@ -2138,16 +2164,17 @@ int drm_mode_addfb(struct drm_device *dev, fb = dev->mode_config.funcs->fb_create(dev, file_priv, &r); if (IS_ERR(fb)) { DRM_DEBUG_KMS("could not create framebuffer\n"); - ret = PTR_ERR(fb); - goto out; + drm_modeset_unlock_all(dev); + return PTR_ERR(fb); } + mutex_lock(&file_priv->fbs_lock); or->fb_id = fb->base.id; list_add(&fb->filp_head, &file_priv->fbs); DRM_DEBUG_KMS("[FB:%d]\n", fb->base.id); - -out: + mutex_unlock(&file_priv->fbs_lock); drm_modeset_unlock_all(dev); + return ret; } @@ -2320,16 +2347,18 @@ int drm_mode_addfb2(struct drm_device *dev, fb = dev->mode_config.funcs->fb_create(dev, file_priv, r); if (IS_ERR(fb)) { DRM_DEBUG_KMS("could not create framebuffer\n"); - ret = PTR_ERR(fb); - goto out; + drm_modeset_unlock_all(dev); + return PTR_ERR(fb); } + mutex_lock(&file_priv->fbs_lock); r->fb_id = fb->base.id; list_add(&fb->filp_head, &file_priv->fbs); DRM_DEBUG_KMS("[FB:%d]\n", fb->base.id); + mutex_unlock(&file_priv->fbs_lock); -out: drm_modeset_unlock_all(dev); + return ret; } @@ -2360,27 +2389,34 @@ int drm_mode_rmfb(struct drm_device *dev, return -EINVAL; drm_modeset_lock_all(dev); + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, *id, DRM_MODE_OBJECT_FB); /* TODO check that we really get a framebuffer back. */ if (!obj) { + mutex_unlock(&dev->mode_config.fb_lock); ret = -EINVAL; goto out; } fb = obj_to_fb(obj); + mutex_unlock(&dev->mode_config.fb_lock); + mutex_lock(&file_priv->fbs_lock); list_for_each_entry(fbl, &file_priv->fbs, filp_head) if (fb == fbl) found = 1; - if (!found) { ret = -EINVAL; + mutex_unlock(&file_priv->fbs_lock); goto out; } - drm_framebuffer_remove(fb); + list_del_init(&fb->filp_head); + mutex_unlock(&file_priv->fbs_lock); + drm_framebuffer_remove(fb); out: drm_modeset_unlock_all(dev); + return ret; } @@ -2409,7 +2445,9 @@ int drm_mode_getfb(struct drm_device *dev, return -EINVAL; drm_modeset_lock_all(dev); + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, r->fb_id, DRM_MODE_OBJECT_FB); + mutex_unlock(&dev->mode_config.fb_lock); if (!obj) { ret = -EINVAL; goto out; @@ -2444,7 +2482,9 @@ int drm_mode_dirtyfb_ioctl(struct drm_device *dev, return -EINVAL; drm_modeset_lock_all(dev); + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, r->fb_id, DRM_MODE_OBJECT_FB); + mutex_unlock(&dev->mode_config.fb_lock); if (!obj) { ret = -EINVAL; goto out_err1; @@ -2519,9 +2559,12 @@ void drm_fb_release(struct drm_file *priv) struct drm_framebuffer *fb, *tfb; drm_modeset_lock_all(dev); + mutex_lock(&priv->fbs_lock); list_for_each_entry_safe(fb, tfb, &priv->fbs, filp_head) { + list_del_init(&fb->filp_head); drm_framebuffer_remove(fb); } + mutex_unlock(&priv->fbs_lock); drm_modeset_unlock_all(dev); } @@ -3547,7 +3590,9 @@ int drm_mode_page_flip_ioctl(struct drm_device *dev, if (crtc->funcs->page_flip == NULL) goto out; + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, page_flip->fb_id, DRM_MODE_OBJECT_FB); + mutex_unlock(&dev->mode_config.fb_lock); if (!obj) goto out; fb = obj_to_fb(obj); diff --git a/drivers/gpu/drm/drm_fops.c b/drivers/gpu/drm/drm_fops.c index 7ef1b67..e06c492 100644 --- a/drivers/gpu/drm/drm_fops.c +++ b/drivers/gpu/drm/drm_fops.c @@ -260,6 +260,7 @@ static int drm_open_helper(struct inode *inode, struct file *filp, INIT_LIST_HEAD(&priv->lhead); INIT_LIST_HEAD(&priv->fbs); + mutex_init(&priv->fbs_lock); INIT_LIST_HEAD(&priv->event_list); init_waitqueue_head(&priv->event_wait); priv->event_space = 4096; /* set aside 4k for event buffer */ diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c index 35d2ace..da1c8b6 100644 --- a/drivers/gpu/drm/i915/i915_debugfs.c +++ b/drivers/gpu/drm/i915/i915_debugfs.c @@ -1376,7 +1376,9 @@ static int i915_gem_framebuffer_info(struct seq_file *m, void *data) fb->base.bits_per_pixel); describe_obj(m, fb->obj); seq_printf(m, "\n"); + mutex_unlock(&dev->mode_config.mutex); + mutex_lock(&dev->mode_config.fb_lock); list_for_each_entry(fb, &dev->mode_config.fb_list, base.head) { if (&fb->base == ifbdev->helper.fb) continue; @@ -1389,8 +1391,7 @@ static int i915_gem_framebuffer_info(struct seq_file *m, void *data) describe_obj(m, fb->obj); seq_printf(m, "\n"); } - - mutex_unlock(&dev->mode_config.mutex); + mutex_unlock(&dev->mode_config.fb_lock); return 0; } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c index 13e4371..84c2c44 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c @@ -163,7 +163,9 @@ int vmw_present_ioctl(struct drm_device *dev, void *data, drm_modeset_lock_all(dev); + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, arg->fb_id, DRM_MODE_OBJECT_FB); + mutex_unlock(&dev->mode_config.fb_lock); if (!obj) { DRM_ERROR("Invalid framebuffer id.\n"); ret = -EINVAL; @@ -246,7 +248,9 @@ int vmw_present_readback_ioctl(struct drm_device *dev, void *data, drm_modeset_lock_all(dev); + mutex_lock(&dev->mode_config.fb_lock); obj = drm_mode_object_find(dev, arg->fb_id, DRM_MODE_OBJECT_FB); + mutex_unlock(&dev->mode_config.fb_lock); if (!obj) { DRM_ERROR("Invalid framebuffer id.\n"); ret = -EINVAL; diff --git a/drivers/staging/omapdrm/omap_debugfs.c b/drivers/staging/omapdrm/omap_debugfs.c index 2f122e0..e95540b 100644 --- a/drivers/staging/omapdrm/omap_debugfs.c +++ b/drivers/staging/omapdrm/omap_debugfs.c @@ -72,6 +72,7 @@ static int fb_show(struct seq_file *m, void *arg) seq_printf(m, "fbcon "); omap_framebuffer_describe(priv->fbdev->fb, m); + mutex_lock(&dev->mode_config.fb_lock); list_for_each_entry(fb, &dev->mode_config.fb_list, head) { if (fb == priv->fbdev->fb) continue; @@ -79,6 +80,7 @@ static int fb_show(struct seq_file *m, void *arg) seq_printf(m, "user "); omap_framebuffer_describe(fb, m); } + mutex_unlock(&dev->mode_config.fb_lock); mutex_unlock(&dev->struct_mutex); mutex_unlock(&dev->mode_config.mutex); diff --git a/include/drm/drmP.h b/include/drm/drmP.h index 3c609ab..e74731c 100644 --- a/include/drm/drmP.h +++ b/include/drm/drmP.h @@ -446,7 +446,15 @@ struct drm_file { int is_master; /* this file private is a master for a minor */ struct drm_master *master; /* master this node is currently associated with N.B. not always minor->master */ + + /** + * fbs - List of framebuffers associated with this file. + * + * Protected by fbs_lock. Note that the fbs list holds a reference on + * the fb object to prevent it from untimely disappearing. + */ struct list_head fbs; + struct mutex fbs_lock; wait_queue_head_t event_wait; struct list_head event_list; diff --git a/include/drm/drm_crtc.h b/include/drm/drm_crtc.h index f1296ce..334765a 100644 --- a/include/drm/drm_crtc.h +++ b/include/drm/drm_crtc.h @@ -254,6 +254,10 @@ struct drm_framebuffer { * userspace perspective. */ struct kref refcount; + /* + * Place on the dev->mode_config.fb_list, access protected by + * dev->mode_config.fb_lock. + */ struct list_head head; struct drm_mode_object base; const struct drm_framebuffer_funcs *funcs; @@ -780,8 +784,18 @@ struct drm_mode_config { struct mutex idr_mutex; /* for IDR management */ struct idr crtc_idr; /* use this idr for all IDs, fb, crtc, connector, modes - just makes life easier */ /* this is limited to one for now */ + + + /** + * fb_lock - mutex to protect fb state + * + * Besides the global fb list his also protects the fbs list in the + * file_priv + */ + struct mutex fb_lock; int num_fb; struct list_head fb_list; + int num_connector; struct list_head connector_list; int num_encoder;