From patchwork Fri Apr 26 00:53:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10917883 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E3B0314D5 for ; Fri, 26 Apr 2019 00:53:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CFFCE28BE6 for ; Fri, 26 Apr 2019 00:53:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C1FB528D82; Fri, 26 Apr 2019 00:53:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6AF5F28D7D for ; Fri, 26 Apr 2019 00:53:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730438AbfDZAxy (ORCPT ); Thu, 25 Apr 2019 20:53:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:34582 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727734AbfDZAxy (ORCPT ); Thu, 25 Apr 2019 20:53:54 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E468680E6A; Fri, 26 Apr 2019 00:53:53 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id D0D53600C0; Fri, 26 Apr 2019 00:53:50 +0000 (UTC) From: Ming Lei To: James Bottomley , linux-scsi@vger.kernel.org, "Martin K . Petersen" Cc: linux-block@vger.kernel.org, Ming Lei , Christoph Hellwig , Bart Van Assche , "Ewan D . Milne" , Hannes Reinecke Subject: [PATCH V3 0/2] scsi: core: avoid big pre-allocation for sg list Date: Fri, 26 Apr 2019 08:53:43 +0800 Message-Id: <20190426005346.27962-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 26 Apr 2019 00:53:54 +0000 (UTC) Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi, Since supporting to blk-mq, big pre-allocation for sg list is introduced, this way is very unfriendly wrt. memory consumption. There were Red Hat internal reports that some scsi_debug based tests can't be run any more because of too big pre-allocation. Also lpfc users commplained that 1GB+ ram is pre-allocatd for single HBA. sg_alloc_table_chained() is improved to support variant size of 1st pre-allocated SGL in the 1st patch as suggested by Christoph. The other two patches try to address this issue by allocating sg list runtime, meantime pre-allocating one or two inline sg entries for small IO. This ways follows NVMe's approach wrt. sg list allocation. V3: - improve sg_alloc_table_chained() to accept variant size of the 1st pre-allocated SGL - applies the improved sg API to address the big pre-allocation issue V2: - move inline sg table initializetion into one helper - introduce new helper for getting inline sg - comment log fix Ming Lei (3): lib/sg_pool.c: improve APIs for allocating sg pool scsi: core: avoid to pre-allocate big chunk for protection meta data scsi: core: avoid to pre-allocate big chunk for sg list drivers/scsi/scsi_lib.c | 36 +++++++++++++++++++++++---------- include/linux/scatterlist.h | 27 ++++++++++++++++++++----- lib/scatterlist.c | 36 +++++++++++++++++++++------------ lib/sg_pool.c | 49 ++++++++++++++++++++++++++++++--------------- 4 files changed, 103 insertions(+), 45 deletions(-) Cc: Christoph Hellwig Cc: Bart Van Assche Cc: Ewan D. Milne Cc: Hannes Reinecke