From patchwork Fri Jun 1 14:41:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matias Bjorling X-Patchwork-Id: 10443631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1D7FE602BD for ; Fri, 1 Jun 2018 14:44:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0DCF12860A for ; Fri, 1 Jun 2018 14:44:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 011D128944; Fri, 1 Jun 2018 14:44:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6525C2860A for ; Fri, 1 Jun 2018 14:44:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752552AbeFAOoS (ORCPT ); Fri, 1 Jun 2018 10:44:18 -0400 Received: from mail-lf0-f65.google.com ([209.85.215.65]:38048 "EHLO mail-lf0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752522AbeFAOoI (ORCPT ); Fri, 1 Jun 2018 10:44:08 -0400 Received: by mail-lf0-f65.google.com with SMTP id i83-v6so3035781lfh.5 for ; Fri, 01 Jun 2018 07:44:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lightnvm-io.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+whTR88LAuwZRtqrxdWv+/rGI2QyrfoJIIrZBNJ7qXI=; b=BHzAzOeesLcT5Ps6uPTnkXoksYtJOGjyDJddTxdQrLyxX+6d+sC3GdVcDckmeViGrT +kLcCJOj5ldhO7+q1QpG7/qmX5Hju8CTKtLfaiuobFZzNxrcV/KQdUGlEUTOkpwNC80Q HGkIeBipSHnrwV66WTSzgVEHsWcZkn0kzUISPv+PVOOcqwC5YXsMfZXxigIP36/KjK9G pesEdFKl3/5RnKtEyEt8lxhYoVm8s4oAnr9wHt+7ek3N28VclIt+wJYOOkFBY8Q0dI+4 BL8oxrBR14m9MOEuw70nXbi8DCxsrjjXmQdp4Y999oizTRTnZXJxr+WZ9d06wfcTgHOx vdCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+whTR88LAuwZRtqrxdWv+/rGI2QyrfoJIIrZBNJ7qXI=; b=VOHwkztnRUTw67u4xFGkLfFPJZMns49N+ThCQD7ORqZVNB+dBE9o78zg8yN5cVR4WJ vBlYnyMcWml4iU6NI8zK7k0WEIgf2BZII41Pur5H/ZnoxUF2W/LQvdNCDUO44ME7HOuI YC0Q0WCymUaLLWfEyH9Nqiu+TahZGNpzHkj6ARcZ0nGF3VqxrhwHj+jFZxxihM/BnTM7 PgXMumfo+rpr1uRR87MH3FEKIIbYXe8pO6cs4ga6k6HgH0mU9LQKyG/EYtAa4WPD+qO/ T4fkXt1Tzdnqcnkndv/C8xODy+0jUkauAuk3lSYU+EcJxKLFeCw2eHgtjfBFU0571LOO +Dxw== X-Gm-Message-State: ALKqPweLR9z23y4ANe/+N+A7BxxOEKNiKkf0jznTD1jol5J24EOrbEwi iJqF0TaJzKN++xpwByZo52Nrbg== X-Google-Smtp-Source: ADUXVKKvIvG07wl4fjLxkIHkdQDvT1bJ6USOe00VTRjzoTkghxJRcT5uKACOE8G13Oh67y4j2BZqpA== X-Received: by 2002:a2e:8350:: with SMTP id l16-v6mr8341823ljh.28.1527864246137; Fri, 01 Jun 2018 07:44:06 -0700 (PDT) Received: from Macroninja.cnexlabs.com (95-166-82-66-cable.dk.customer.tdc.net. [95.166.82.66]) by smtp.gmail.com with ESMTPSA id e65-v6sm356488lff.13.2018.06.01.07.44.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Jun 2018 07:44:05 -0700 (PDT) From: =?UTF-8?q?Matias=20Bj=C3=B8rling?= To: axboe@fb.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, javier@cnexlabs.com, igor.j.konopko@intel.com, marcin.dziegielewski@intel.com, =?UTF-8?q?Javier=20Gonz=C3=A1lez?= , =?UTF-8?q?Matias=20Bj=C3=B8rling?= Subject: [GIT PULL 10/10] lightnvm: pblk: take bitmap alloc. out of critical section Date: Fri, 1 Jun 2018 16:41:14 +0200 Message-Id: <20180601144114.17490-11-mb@lightnvm.io> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180601144114.17490-1-mb@lightnvm.io> References: <20180601144114.17490-1-mb@lightnvm.io> MIME-Version: 1.0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Javier González pblk allocates line bitmaps within the line lock unnecessarily. In order to take pressure out of the fast patch, allocate line bitmaps outside of this lock and refactor accordingly. Signed-off-by: Javier González Signed-off-by: Matias Bjørling --- drivers/lightnvm/pblk-core.c | 97 +++++++++++++++++++++++++------------------- 1 file changed, 56 insertions(+), 41 deletions(-) diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c index a5750534efed..ed9cc977c8b3 100644 --- a/drivers/lightnvm/pblk-core.c +++ b/drivers/lightnvm/pblk-core.c @@ -1067,6 +1067,25 @@ static int pblk_line_init_metadata(struct pblk *pblk, struct pblk_line *line, return 1; } +static int pblk_line_alloc_bitmaps(struct pblk *pblk, struct pblk_line *line) +{ + struct pblk_line_meta *lm = &pblk->lm; + + line->map_bitmap = kzalloc(lm->sec_bitmap_len, GFP_KERNEL); + if (!line->map_bitmap) + return -ENOMEM; + + /* will be initialized using bb info from map_bitmap */ + line->invalid_bitmap = kmalloc(lm->sec_bitmap_len, GFP_KERNEL); + if (!line->invalid_bitmap) { + kfree(line->map_bitmap); + line->map_bitmap = NULL; + return -ENOMEM; + } + + return 0; +} + /* For now lines are always assumed full lines. Thus, smeta former and current * lun bitmaps are omitted. */ @@ -1171,18 +1190,7 @@ static int pblk_line_prepare(struct pblk *pblk, struct pblk_line *line) { struct pblk_line_meta *lm = &pblk->lm; int blk_in_line = atomic_read(&line->blk_in_line); - int blk_to_erase, ret; - - line->map_bitmap = kzalloc(lm->sec_bitmap_len, GFP_ATOMIC); - if (!line->map_bitmap) - return -ENOMEM; - - /* will be initialized using bb info from map_bitmap */ - line->invalid_bitmap = kmalloc(lm->sec_bitmap_len, GFP_ATOMIC); - if (!line->invalid_bitmap) { - ret = -ENOMEM; - goto fail_free_map_bitmap; - } + int blk_to_erase; /* Bad blocks do not need to be erased */ bitmap_copy(line->erase_bitmap, line->blk_bitmap, lm->blk_per_line); @@ -1200,15 +1208,15 @@ static int pblk_line_prepare(struct pblk *pblk, struct pblk_line *line) } if (blk_in_line < lm->min_blk_line) { - ret = -EAGAIN; - goto fail_free_invalid_bitmap; + spin_unlock(&line->lock); + return -EAGAIN; } if (line->state != PBLK_LINESTATE_FREE) { WARN(1, "pblk: corrupted line %d, state %d\n", line->id, line->state); - ret = -EINTR; - goto fail_free_invalid_bitmap; + spin_unlock(&line->lock); + return -EINTR; } line->state = PBLK_LINESTATE_OPEN; @@ -1222,16 +1230,6 @@ static int pblk_line_prepare(struct pblk *pblk, struct pblk_line *line) kref_init(&line->ref); return 0; - -fail_free_invalid_bitmap: - spin_unlock(&line->lock); - kfree(line->invalid_bitmap); - line->invalid_bitmap = NULL; -fail_free_map_bitmap: - kfree(line->map_bitmap); - line->map_bitmap = NULL; - - return ret; } int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line) @@ -1251,13 +1249,16 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line) } spin_unlock(&l_mg->free_lock); - pblk_rl_free_lines_dec(&pblk->rl, line, true); + ret = pblk_line_alloc_bitmaps(pblk, line); + if (ret) + return ret; if (!pblk_line_init_bb(pblk, line, 0)) { list_add(&line->list, &l_mg->free_list); return -EINTR; } + pblk_rl_free_lines_dec(&pblk->rl, line, true); return 0; } @@ -1269,6 +1270,24 @@ void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line) line->emeta = NULL; } +static void pblk_line_reinit(struct pblk_line *line) +{ + *line->vsc = cpu_to_le32(EMPTY_ENTRY); + + line->map_bitmap = NULL; + line->invalid_bitmap = NULL; + line->smeta = NULL; + line->emeta = NULL; +} + +void pblk_line_free(struct pblk_line *line) +{ + kfree(line->map_bitmap); + kfree(line->invalid_bitmap); + + pblk_line_reinit(line); +} + struct pblk_line *pblk_line_get(struct pblk *pblk) { struct pblk_line_mgmt *l_mg = &pblk->l_mg; @@ -1335,11 +1354,14 @@ static struct pblk_line *pblk_line_retry(struct pblk *pblk, return NULL; } + retry_line->map_bitmap = line->map_bitmap; + retry_line->invalid_bitmap = line->invalid_bitmap; retry_line->smeta = line->smeta; retry_line->emeta = line->emeta; retry_line->meta_line = line->meta_line; - pblk_line_free(line); + pblk_line_reinit(line); + l_mg->data_line = retry_line; spin_unlock(&l_mg->free_lock); @@ -1392,6 +1414,9 @@ struct pblk_line *pblk_line_get_first_data(struct pblk *pblk) } spin_unlock(&l_mg->free_lock); + if (pblk_line_alloc_bitmaps(pblk, line)) + return NULL; + if (pblk_line_erase(pblk, line)) { line = pblk_line_retry(pblk, line); if (!line) @@ -1536,6 +1561,9 @@ struct pblk_line *pblk_line_replace_data(struct pblk *pblk) goto retry_erase; } + if (pblk_line_alloc_bitmaps(pblk, new)) + return NULL; + retry_setup: if (!pblk_line_init_metadata(pblk, new, cur)) { new = pblk_line_retry(pblk, new); @@ -1575,19 +1603,6 @@ struct pblk_line *pblk_line_replace_data(struct pblk *pblk) return new; } -void pblk_line_free(struct pblk_line *line) -{ - kfree(line->map_bitmap); - kfree(line->invalid_bitmap); - - *line->vsc = cpu_to_le32(EMPTY_ENTRY); - - line->map_bitmap = NULL; - line->invalid_bitmap = NULL; - line->smeta = NULL; - line->emeta = NULL; -} - static void __pblk_line_put(struct pblk *pblk, struct pblk_line *line) { struct pblk_line_mgmt *l_mg = &pblk->l_mg;