From patchwork Mon Oct 25 02:54:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12580605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30238C433F5 for ; Mon, 25 Oct 2021 02:54:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1128961040 for ; Mon, 25 Oct 2021 02:54:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232226AbhJYC5M (ORCPT ); Sun, 24 Oct 2021 22:57:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58571 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232250AbhJYC5L (ORCPT ); Sun, 24 Oct 2021 22:57:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635130489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E22BEZISqI+m8U8CL0bHRSqqXn2CunwBHAAxX2ofm5s=; b=Oeu6ZbqqU7h3uZotsumsn6PzxRAIGdPgK4asG66AZaZH8XS9EOQmzA3SOtC+lgE/illq2F qTFXBTzLeH6P+xrmXyTaVaB0og4YAZH7XYuRXrMXTLuwEt7HaV6TNpXroHvNzxWC9ixbnt VJox5/IiTeL65KIOqIOGijuZZ58FmEU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-441-E8ziSHdBMrqcvyFAWT3FAw-1; Sun, 24 Oct 2021 22:54:46 -0400 X-MC-Unique: E8ziSHdBMrqcvyFAWT3FAw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 51D561006AA2; Mon, 25 Oct 2021 02:54:45 +0000 (UTC) Received: from localhost (ovpn-8-18.pek2.redhat.com [10.72.8.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4AB1B5C1CF; Mon, 25 Oct 2021 02:54:40 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Luis Chamberlain , Minchan Kim , Greg Kroah-Hartman , linux-kernel@vger.kernel.org, Ming Lei Subject: [PATCH V3 1/4] zram: fix race between zram_reset_device() and disksize_store() Date: Mon, 25 Oct 2021 10:54:23 +0800 Message-Id: <20211025025426.2815424-2-ming.lei@redhat.com> In-Reply-To: <20211025025426.2815424-1-ming.lei@redhat.com> References: <20211025025426.2815424-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When the ->init_lock is released in zram_reset_device(), disksize_store() can come in and try to allocate meta, but zram_reset_device() is freeing free meta, so cause races. Link: https://lore.kernel.org/linux-block/20210927163805.808907-1-mcgrof@kernel.org/T/#mc617f865a3fa2778e40f317ddf48f6447c20c073 Reported-by: Luis Chamberlain Reviewed-by: Luis Chamberlain Signed-off-by: Ming Lei --- drivers/block/zram/zram_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index a68297fb51a2..25d781dc5fef 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1704,12 +1704,13 @@ static void zram_reset_device(struct zram *zram) set_capacity_and_notify(zram->disk, 0); part_stat_set_all(zram->disk->part0, 0); - up_write(&zram->init_lock); /* I/O operation under all of CPU are done so let's free */ zram_meta_free(zram, disksize); memset(&zram->stats, 0, sizeof(zram->stats)); zcomp_destroy(comp); reset_bdev(zram); + + up_write(&zram->init_lock); } static ssize_t disksize_store(struct device *dev, From patchwork Mon Oct 25 02:54:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12580607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 580D0C433F5 for ; Mon, 25 Oct 2021 02:54:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C55660F4F for ; Mon, 25 Oct 2021 02:54:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232268AbhJYC5T (ORCPT ); Sun, 24 Oct 2021 22:57:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29297 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232263AbhJYC5T (ORCPT ); Sun, 24 Oct 2021 22:57:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635130497; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QfPolJNm7P7L/JGB1152ao6XDhnZRZSDuWq/Pii30qo=; b=NBtQdbsaoUUgQtUlCKBYGdJ0cj8pgoZML5H41vg1hzQOaAIbY/XVsdImbdRsl7LKOkf5ef khtopJqos8MRmKM/t3qIezATrDl4DXWGBGu8RvgroUIq1wAzRYS1SzIE/nSXq8vHrpHmQe TkrkkpXZq9xDtoKLrf7N5RDKOAExTa0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-459-CFq_rGAsMemUjQabbYQLvw-1; Sun, 24 Oct 2021 22:54:54 -0400 X-MC-Unique: CFq_rGAsMemUjQabbYQLvw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D57C5362F8; Mon, 25 Oct 2021 02:54:52 +0000 (UTC) Received: from localhost (ovpn-8-18.pek2.redhat.com [10.72.8.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 148C85D9D5; Mon, 25 Oct 2021 02:54:47 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Luis Chamberlain , Minchan Kim , Greg Kroah-Hartman , linux-kernel@vger.kernel.org, Ming Lei Subject: [PATCH V3 2/4] zram: don't fail to remove zram during unloading module Date: Mon, 25 Oct 2021 10:54:24 +0800 Message-Id: <20211025025426.2815424-3-ming.lei@redhat.com> In-Reply-To: <20211025025426.2815424-1-ming.lei@redhat.com> References: <20211025025426.2815424-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When the zram module is being unloaded, no one should be using the zram disks. However even while being unloaded the zram module's sysfs attributes might be poked at to re-configure zram devices. This is expected, and kernfs ensures that these operations complete before device_del() completes. But reset_store() may set ->claim which will fail zram_remove(), when this happens, zram_reset_device() is bypassed, and zram->comp can't be destroyed, so the warning of 'Error: Removing state 63 which has instances left.' is triggered during unloading module, together with memory leak and sort of thing. Fixes the issue by not failing zram_remove() if ->claim is set, and we actually need to do nothing in case that zram_reset() is running since del_gendisk() will wait until zram_reset() is done. Reported-by: Luis Chamberlain Reviewed-by: Luis Chamberlain Signed-off-by: Ming Lei --- drivers/block/zram/zram_drv.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 25d781dc5fef..8883de7aa3d7 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1968,25 +1968,40 @@ static int zram_add(void) static int zram_remove(struct zram *zram) { struct block_device *bdev = zram->disk->part0; + bool claimed; mutex_lock(&bdev->bd_disk->open_mutex); - if (bdev->bd_openers || zram->claim) { + if (bdev->bd_openers) { mutex_unlock(&bdev->bd_disk->open_mutex); return -EBUSY; } - zram->claim = true; + claimed = zram->claim; + if (!claimed) + zram->claim = true; mutex_unlock(&bdev->bd_disk->open_mutex); zram_debugfs_unregister(zram); - /* Make sure all the pending I/O are finished */ - fsync_bdev(bdev); - zram_reset_device(zram); + if (claimed) { + /* + * If we were claimed by reset_store(), del_gendisk() will + * wait until reset_store() is done, so nothing need to do. + */ + ; + } else { + /* Make sure all the pending I/O are finished */ + fsync_bdev(bdev); + zram_reset_device(zram); + } pr_info("Removed device: %s\n", zram->disk->disk_name); del_gendisk(zram->disk); + + /* del_gendisk drains pending reset_store */ + WARN_ON_ONCE(claimed && zram->claim); + blk_cleanup_disk(zram->disk); kfree(zram); return 0; @@ -2063,7 +2078,7 @@ static struct class zram_control_class = { static int zram_remove_cb(int id, void *ptr, void *data) { - zram_remove(ptr); + WARN_ON_ONCE(zram_remove(ptr)); return 0; } From patchwork Mon Oct 25 02:54:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12580609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D33AAC433EF for ; Mon, 25 Oct 2021 02:55:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B169E60F4F for ; Mon, 25 Oct 2021 02:55:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232290AbhJYC50 (ORCPT ); Sun, 24 Oct 2021 22:57:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:38446 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232280AbhJYC5Z (ORCPT ); Sun, 24 Oct 2021 22:57:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635130503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AE0JY6csuXHuv4AM42+KRRY1fWa7BR0iAL/p8wT8I/8=; b=EbdNglyPahV46o18aOQ41jBNu6FBsvov7lQdrlC0AOVpeFdL7AExp//lwllU1Kl6MjCBoh riHmEjYY4K/m4ECVsQztaEZGleArOolBKV9qXlgJoqooJeXL3VYmDRdyJ3U9I2teKcBbt7 8KDx/1eUv2OF+gJ/fgoNtIzySHLC9h4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-373-gh3LXyuHM9el8AtAnQEg4Q-1; Sun, 24 Oct 2021 22:55:00 -0400 X-MC-Unique: gh3LXyuHM9el8AtAnQEg4Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0569D1808302; Mon, 25 Oct 2021 02:54:59 +0000 (UTC) Received: from localhost (ovpn-8-18.pek2.redhat.com [10.72.8.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id D39585D705; Mon, 25 Oct 2021 02:54:55 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Luis Chamberlain , Minchan Kim , Greg Kroah-Hartman , linux-kernel@vger.kernel.org, Ming Lei Subject: [PATCH V3 3/4] zram: avoid race between zram_remove and disksize_store Date: Mon, 25 Oct 2021 10:54:25 +0800 Message-Id: <20211025025426.2815424-4-ming.lei@redhat.com> In-Reply-To: <20211025025426.2815424-1-ming.lei@redhat.com> References: <20211025025426.2815424-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org After resetting device in zram_remove(), disksize_store still may come and allocate resources again before deleting gendisk, fix the race by resetting zram after del_gendisk() returns. At that time, disksize_store can't come any more. Reported-by: Luis Chamberlain Reviewed-by: Luis Chamberlain Signed-off-by: Ming Lei --- drivers/block/zram/zram_drv.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 8883de7aa3d7..dba93b8ce511 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2002,6 +2002,13 @@ static int zram_remove(struct zram *zram) /* del_gendisk drains pending reset_store */ WARN_ON_ONCE(claimed && zram->claim); + /* + * disksize_store() may be called in between zram_reset_device() + * and del_gendisk(), so run the last reset to avoid leaking + * anything allocated with disksize_store() + */ + zram_reset_device(zram); + blk_cleanup_disk(zram->disk); kfree(zram); return 0; From patchwork Mon Oct 25 02:54:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 12580611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0C3EC433F5 for ; Mon, 25 Oct 2021 02:55:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C90A160FDA for ; Mon, 25 Oct 2021 02:55:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232292AbhJYC5n (ORCPT ); Sun, 24 Oct 2021 22:57:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:51407 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232301AbhJYC5l (ORCPT ); Sun, 24 Oct 2021 22:57:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635130519; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nFan7nTdgfoFap6cQpEwBX1t4meF6S1iT44x4AnXqtA=; b=eq4K8P9N9LSweOVUfsFK5tIALgJDk+ZtTVwmmCVrn2Y4ZQXsWIQl0KPXzIjZ40W3Yc+vJu HbJQUtXWK4U0kx2ppKFwDn0Bmo4zF903edikCSTKIMmIObdk8FfwA1OZYKYjXCPRRRja0m /Xpkh9NNYwJhohxYMkklupv+4GlfrpE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-317-HOEtoLDpNR6cG_067Q44ZA-1; Sun, 24 Oct 2021 22:55:15 -0400 X-MC-Unique: HOEtoLDpNR6cG_067Q44ZA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 86B511006AA3; Mon, 25 Oct 2021 02:55:14 +0000 (UTC) Received: from localhost (ovpn-8-18.pek2.redhat.com [10.72.8.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF59360CA1; Mon, 25 Oct 2021 02:55:01 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, Luis Chamberlain , Minchan Kim , Greg Kroah-Hartman , linux-kernel@vger.kernel.org, Ming Lei Subject: [PATCH V3 4/4] zram: replace fsync_bdev with sync_blockdev Date: Mon, 25 Oct 2021 10:54:26 +0800 Message-Id: <20211025025426.2815424-5-ming.lei@redhat.com> In-Reply-To: <20211025025426.2815424-1-ming.lei@redhat.com> References: <20211025025426.2815424-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When calling fsync_bdev(), zram driver guarantees that the bdev won't be opened by anyone, then there can't be one active fs/superblock over the zram bdev, so replace fsync_bdev with sync_blockdev. Reviewed-by: Luis Chamberlain Signed-off-by: Ming Lei --- drivers/block/zram/zram_drv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index dba93b8ce511..9609e2b31d5a 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1790,7 +1790,7 @@ static ssize_t reset_store(struct device *dev, mutex_unlock(&bdev->bd_disk->open_mutex); /* Make sure all the pending I/O are finished */ - fsync_bdev(bdev); + sync_blockdev(bdev); zram_reset_device(zram); mutex_lock(&bdev->bd_disk->open_mutex); @@ -1991,7 +1991,7 @@ static int zram_remove(struct zram *zram) ; } else { /* Make sure all the pending I/O are finished */ - fsync_bdev(bdev); + sync_blockdev(bdev); zram_reset_device(zram); }