From patchwork Thu Nov 1 10:38:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Feng Li X-Patchwork-Id: 10663865 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D9B21109C for ; Thu, 1 Nov 2018 10:52:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C0AD42B7EE for ; Thu, 1 Nov 2018 10:52:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B33732B80F; Thu, 1 Nov 2018 10:52:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4C2722B7E2 for ; Thu, 1 Nov 2018 10:52:17 +0000 (UTC) Received: from localhost ([::1]:41283 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gIAaG-0007C5-09 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 01 Nov 2018 06:52:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33821) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gIAIQ-0000tP-NA for qemu-devel@nongnu.org; Thu, 01 Nov 2018 06:33:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gIAIP-0002SM-NQ for qemu-devel@nongnu.org; Thu, 01 Nov 2018 06:33:50 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:38377) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gIAIK-0002Jf-Ca; Thu, 01 Nov 2018 06:33:44 -0400 Received: by mail-pl1-x641.google.com with SMTP id p7-v6so8758232plk.5; Thu, 01 Nov 2018 03:33:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=jrBkmcSHxIlO2hFWAUQYYOY8kpCuX2gMgvZpALY0gWg=; b=NvHuLqwdcETqjT70YddIs3Mfr7Re2BN//3dybQEjxftqg8OJqLbbM83oLSP3s8isWm 1VOSrQ4SzFWLgsS+ZbE2806NHi3/K/HUCPQGcLMTfQjMJDV2B3Pcs9dbMKn44YjDr033 scZVBogAAf/V0nIa+T1Rwgxw1XUFPtVUMCYc4XEbWPHKDMNqc9PGJILiFNUAxu6byed7 yRiwMdLqighTpSJ0Z2mgdA0sEY+rtqul1ziWuTXhbt/oA3VuoQ+A8DP3n3mZvC+gG+W7 Jh7UJnS1xBrnl5z8zsOALe8agGxnbuRSaD9neF2PwQF1WqPMlxKi1MTRGz8rzbb1s+bJ +ZZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=jrBkmcSHxIlO2hFWAUQYYOY8kpCuX2gMgvZpALY0gWg=; b=MyHwAK31b1nXudqAiGSsMYTphIZxVhDbloxr8bQTfHvtnHVjB3XgnFNdZAtkiIdKsK kuYvxx9vPnBzn6wdA1z7e6Bk8hu+hENnLbX8fqfIGyHdhwnF+2UqSAPODPaRytOS/Db/ KRDjEMxSiW/1+irFJH+AaeDzhE2z5JlrCMPuZe/hCc41bm021q6pl9qesbDa7/u5MJPp Zs9o92ISxFMlRqpAryTnQfVrOvpHhXVwQWwpIjjpFE/RRbnHayN45yQorIZsJbkcdklS APYpUGLLehTYXAV4KCJ9YKdLDMbj9vXJwZTGobctTLxX6p1yKEIKKTAZ+wktOPRkndxK fBIQ== X-Gm-Message-State: AGRZ1gJ5VQL3hJBcT8g5ZTaPEQWZ6KwZ1ucaQcG2b8tc6USPKFLSV0an orqxt6uKe+QthfMSeqtoWNs= X-Google-Smtp-Source: AJdET5f86ttW1zaaqorfw4jxRLjyhWhK281kyA/ZQ8baPdcJbJ/lmQmwJ8OR1vYDXYNYcNmk8QDAmw== X-Received: by 2002:a17:902:166:: with SMTP id 93-v6mr7005244plb.68.1541068419506; Thu, 01 Nov 2018 03:33:39 -0700 (PDT) Received: from kvm.localdomain ([207.148.85.3]) by smtp.gmail.com with ESMTPSA id k72-v6sm66457688pfj.63.2018.11.01.03.33.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Nov 2018 03:33:38 -0700 (PDT) From: Li Feng To: fengli@smartx.com Date: Thu, 1 Nov 2018 18:38:07 +0800 Message-Id: <20181101103807.25862-1-lifeng1519@gmail.com> X-Mailer: git-send-email 2.11.0 MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::641 Subject: [Qemu-devel] [PATCH] block/nvme: optimize the performance of nvme driver based on vfio-pci X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Li Feng , Kevin Wolf , Fam Zheng , "open list:NVMe Block Driver" , "open list:All patches CC here" , Max Reitz Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP When the IO size is larger than 2 pages, we move the the pointer one by one in the pagelist, this is inefficient. This is a simple benchmark result: Before: $ qemu-io -c 'write 0 1G' nvme://0000:00:04.0/1 wrote 1073741824/1073741824 bytes at offset 0 1 GiB, 1 ops; 0:00:02.41 (424.504 MiB/sec and 0.4146 ops/sec) $ qemu-io -c 'read 0 1G' nvme://0000:00:04.0/1 read 1073741824/1073741824 bytes at offset 0 1 GiB, 1 ops; 0:00:02.03 (503.055 MiB/sec and 0.4913 ops/sec) After: $ qemu-io -c 'write 0 1G' nvme://0000:00:04.0/1 wrote 1073741824/1073741824 bytes at offset 0 1 GiB, 1 ops; 0:00:02.17 (471.517 MiB/sec and 0.4605 ops/sec) $ qemu-io -c 'read 0 1G' nvme://0000:00:04.0/1 1 ↵ read 1073741824/1073741824 bytes at offset 0 1 GiB, 1 ops; 0:00:01.94 (526.770 MiB/sec and 0.5144 ops/sec) Signed-off-by: Li Feng --- block/nvme.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 29294038fc..982097b5b1 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -837,7 +837,7 @@ try_map: } for (j = 0; j < qiov->iov[i].iov_len / s->page_size; j++) { - pagelist[entries++] = iova + j * s->page_size; + pagelist[entries++] = cpu_to_le64(iova + j * s->page_size); } trace_nvme_cmd_map_qiov_iov(s, i, qiov->iov[i].iov_base, qiov->iov[i].iov_len / s->page_size); @@ -850,20 +850,16 @@ try_map: case 0: abort(); case 1: - cmd->prp1 = cpu_to_le64(pagelist[0]); + cmd->prp1 = pagelist[0]; cmd->prp2 = 0; break; case 2: - cmd->prp1 = cpu_to_le64(pagelist[0]); - cmd->prp2 = cpu_to_le64(pagelist[1]);; + cmd->prp1 = pagelist[0]; + cmd->prp2 = pagelist[1]; break; default: - cmd->prp1 = cpu_to_le64(pagelist[0]); - cmd->prp2 = cpu_to_le64(req->prp_list_iova); - for (i = 0; i < entries - 1; ++i) { - pagelist[i] = cpu_to_le64(pagelist[i + 1]); - } - pagelist[entries - 1] = 0; + cmd->prp1 = pagelist[0]; + cmd->prp2 = cpu_to_le64(req->prp_list_iova + sizeof(uint64_t)); break; } trace_nvme_cmd_map_qiov(s, cmd, req, qiov, entries);