From patchwork Fri Jun 14 18:24:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13699035 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF264137764 for ; Fri, 14 Jun 2024 18:24:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718389460; cv=none; b=PfyP+KQZr7FMmTrQr8nIz0HtxY/ak9wbE/WvJt1pgW+XjPfUQP5fI/t5RtCzO6cDPNrRYV/aaukcilWbgGahZQ0ScQASWtHfLfBZvN79mo1G034B5SsYIMuBHv5k/KJS86501S6hocdXxE1a5LdrNnuhEKujpmspW8O2mjtUd0A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718389460; c=relaxed/simple; bh=cG5WviVEhA217aEoAiRjRp/N+zIMjVMAk9zeWnEj/X4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ScGr/kK+/IE0m2d2sSF81LWECGwUmFtwUw1jnf7KL6S/tQIjPqTnirfc8D50f3IHuRUs/ZJ/E8IebEcu1cUtukYmu5lTw40RRI92rLX2ThrbOY1uKTAbmcY/8/XB2IqpKv/5v2ziyWDrYlrgq1NKto127iympw3pegujOarT/qs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UOunB/1P; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UOunB/1P" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1718389457; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SY+pt5tiuepK+rGrj5eIXyEZBofOvKo3/WbvPjYRnkU=; b=UOunB/1PxCNtgEF6w9hNqsKCDlxrlsKJiI2EbyqXGMGBfzCrrMMgbtHlMGJV7lqw2MGn0O Ff6oPJqWbBCt+BDN3yffCeo68oTMdp6LKo9Q2FmcprLcPweElsUJg6K8MupZNMq08C4Dab 2sYGY8NQLdTXd7K+rQCm/VFwZkJ7lc0= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-173-tvXXss_dNHWlMXWs_qkvQA-1; Fri, 14 Jun 2024 14:24:16 -0400 X-MC-Unique: tvXXss_dNHWlMXWs_qkvQA-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1E93D195609D; Fri, 14 Jun 2024 18:24:15 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A22953000220; Fri, 14 Jun 2024 18:24:14 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 45EIODS7952182 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 14:24:13 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 45EIODPs952181; Fri, 14 Jun 2024 14:24:13 -0400 From: Benjamin Marzinski To: Mikulas Patocka , Mike Snitzer Cc: dm-devel@lists.linux.dev Subject: [PATCH 1/3] dm io: bump num_bvecs to handle offset memory. Date: Fri, 14 Jun 2024 14:24:10 -0400 Message-ID: <20240614182412.952166-2-bmarzins@redhat.com> In-Reply-To: <20240614182412.952166-1-bmarzins@redhat.com> References: <20240614182412.952166-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com If dp->get_page() returns an non-zero offset, the bio might need an additional bvec to deal with the offset. For example, if remaining is exactly one page size, but there is an offset, the memory will span two pages. Signed-off-by: Benjamin Marzinski --- drivers/md/dm-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index 7409490259d1..3333943fe288 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -347,7 +347,7 @@ static void do_region(const blk_opf_t opf, unsigned int region, break; default: num_bvecs = bio_max_segs(dm_sector_div_up(remaining, - (PAGE_SIZE >> SECTOR_SHIFT))); + (PAGE_SIZE >> SECTOR_SHIFT)) + 1); } bio = bio_alloc_bioset(where->bdev, num_bvecs, opf, GFP_NOIO, From patchwork Fri Jun 14 18:24:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13699037 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A2EF7E76D for ; Fri, 14 Jun 2024 18:24:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718389465; cv=none; b=YtqvNNIyljbFewtMGbXBIKx+vkGJ8yZVN/4s5akOwqlvnqmL7K7Ab/11qYAis8niK0sAriCWVnelUYXxZIL2gyfPyBzCAh2+nody1j2ClU7mzvX5VhvAMyMg3oH2yic3E4t+nZXfUCm/bZW0DcYli7fr7lETDytlxSf24fsHEe0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718389465; c=relaxed/simple; bh=zJqzTGjjvqWpyyNQt6Ph+2r/LUHSjUwYrr7yVaaO4PY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aXq4VukZigckLaOaepED7FFs0dikFJUDxFHjI2e1qOpHludKmgj/3knPRjtzIvivinyCDJoXS7mKTX1qHWj64SQGvGPRNNI0txE2JX1AlwdimrnPWy0h9CyNGeYFG+FbwioZY4s93u6V+pzVRwiske6ku83Tne9Fo/fTlQHSDFw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=U2BvAk34; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="U2BvAk34" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1718389462; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Mp44ZIQ3eijZtwlEnZ8brdG51OGsA0T6XxB9AYc6pI=; b=U2BvAk34y3moq814U/Jnv6ipdbegJpeeNmuAZ2pRHlQdgLnrW4u2xI50Ut9PJaSCIcN7Jq LkQZruVlYUQOVksUymiO4jbzdFlNM0jAYMTv+6omzQYa8ru3absGdY4xqsY2UEa28sC4mh VQgN01E9rUW9MtMvxuqqDQjzhqVaHSU= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-XXmgAenlP2O6QfcPjYU2Dg-1; Fri, 14 Jun 2024 14:24:16 -0400 X-MC-Unique: XXmgAenlP2O6QfcPjYU2Dg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1AAB8195609E; Fri, 14 Jun 2024 18:24:15 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 867181954AC0; Fri, 14 Jun 2024 18:24:14 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 45EIODXK952186 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 14:24:13 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 45EIODk0952185; Fri, 14 Jun 2024 14:24:13 -0400 From: Benjamin Marzinski To: Mikulas Patocka , Mike Snitzer Cc: dm-devel@lists.linux.dev Subject: [PATCH 2/3] dm io: don't call the async_io notify.fn on invalid num_regions Date: Fri, 14 Jun 2024 14:24:11 -0400 Message-ID: <20240614182412.952166-3-bmarzins@redhat.com> In-Reply-To: <20240614182412.952166-1-bmarzins@redhat.com> References: <20240614182412.952166-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com If dm_io() returned an error, callers that set a notify.fn and wanted it called on an error need to check the return value and call notify.fn themselves if it was -EINVAL but not if it was -EIO. None of them do this (granted, all the existing async_io users of dm_io call it in a way that is guaranteed to not return an error). Simplify the interface by never calling the notify.fn if dm_io returns an error. This works with the existing dm_io callers which check for an error and handle it using the same methods as the notify.fn. This also allows us to move the now equivalent num_regions checks out of sync_io() and async_io() and into dm_io() itself. Additionally, change async_io() into a void function, since it can no longer fail. Signed-off-by: Benjamin Marzinski --- drivers/md/dm-io.c | 31 ++++++++++++------------------- 1 file changed, 12 insertions(+), 19 deletions(-) diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index 3333943fe288..329a85a12061 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -431,11 +431,6 @@ static int sync_io(struct dm_io_client *client, unsigned int num_regions, struct io *io; struct sync_io sio; - if (num_regions > 1 && !op_is_write(opf)) { - WARN_ON(1); - return -EIO; - } - init_completion(&sio.wait); io = mempool_alloc(&client->pool, GFP_NOIO); @@ -458,19 +453,13 @@ static int sync_io(struct dm_io_client *client, unsigned int num_regions, return sio.error_bits ? -EIO : 0; } -static int async_io(struct dm_io_client *client, unsigned int num_regions, - struct dm_io_region *where, blk_opf_t opf, - struct dpages *dp, io_notify_fn fn, void *context, - unsigned short ioprio) +static void async_io(struct dm_io_client *client, unsigned int num_regions, + struct dm_io_region *where, blk_opf_t opf, + struct dpages *dp, io_notify_fn fn, void *context, + unsigned short ioprio) { struct io *io; - if (num_regions > 1 && !op_is_write(opf)) { - WARN_ON(1); - fn(1, context); - return -EIO; - } - io = mempool_alloc(&client->pool, GFP_NOIO); io->error_bits = 0; atomic_set(&io->count, 1); /* see dispatch_io() */ @@ -482,7 +471,6 @@ static int async_io(struct dm_io_client *client, unsigned int num_regions, io->vma_invalidate_size = dp->vma_invalidate_size; dispatch_io(opf, num_regions, where, dp, io, 0, ioprio); - return 0; } static int dp_init(struct dm_io_request *io_req, struct dpages *dp, @@ -529,6 +517,11 @@ int dm_io(struct dm_io_request *io_req, unsigned int num_regions, int r; struct dpages dp; + if (num_regions > 1 && !op_is_write(io_req->bi_opf)) { + WARN_ON(1); + return -EIO; + } + r = dp_init(io_req, &dp, (unsigned long)where->count << SECTOR_SHIFT); if (r) return r; @@ -537,9 +530,9 @@ int dm_io(struct dm_io_request *io_req, unsigned int num_regions, return sync_io(io_req->client, num_regions, where, io_req->bi_opf, &dp, sync_error_bits, ioprio); - return async_io(io_req->client, num_regions, where, - io_req->bi_opf, &dp, io_req->notify.fn, - io_req->notify.context, ioprio); + async_io(io_req->client, num_regions, where, io_req->bi_opf, &dp, + io_req->notify.fn, io_req->notify.context, ioprio); + return 0; } EXPORT_SYMBOL(dm_io); From patchwork Fri Jun 14 18:24:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Marzinski X-Patchwork-Id: 13699036 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5930F137764 for ; Fri, 14 Jun 2024 18:24:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718389463; cv=none; b=tXpuH8wPv5LcOTfkDE6c6a94+7/ULBUOprhbWyHRn3BECwRrgw+svccZ+k/DA9GqC9XrnJCKoS/hula82zBLUv1Kg1Uh5NH/yZWXn1N9z9VN/RbhzNUmrDNtcdXiQXwxhOuXNuAHX1udtudFnp9wuytvcNxCNr0ZcnJ6OgErhdw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718389463; c=relaxed/simple; bh=eb8rq39XBo5WV5JSo7sMfdOqz5PpMQdZY8ny+Z1lnoc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LFQ2ISzK3iUdCUCRpU7pafaIgSpsnlCefETJPsQP5bcGIScCOJOFJSKV0MWex1AO/qx5+17/n3fsxy9RjAjq4DVmKuBQMuDRHiTHj6KLojD6+cxxaRf+/CtqanrPUb/GdlCDx8W+cgoBc8O+XVliXY/0Ua2BQSPb3P9RtsdUms0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=eOi4knHu; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eOi4knHu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1718389461; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wv9Re76v8f3o9GsziQl2zMysv+AIlkj3wtFLZpGru0Y=; b=eOi4knHuNw4brIr1BEBeLGeKbUvUu1CdnIr4ItT/tfQDqUQ2kmh9LHJSjlivce+wjL1QVa 1+SJlqXNdEPrw7PRszpWmCx8cXTWFtp/GEuk5Mo5NMNz1z0i+nWObBbZsHJahLmx+n82lv WXEHWHJ9vklSmmdh2Ld+60ellnB/XfI= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-467-nnjIlVkyPwqu5Ni363bqLQ-1; Fri, 14 Jun 2024 14:24:16 -0400 X-MC-Unique: nnjIlVkyPwqu5Ni363bqLQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 54036195609F; Fri, 14 Jun 2024 18:24:15 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (bmarzins-01.fast.eng.rdu2.dc.redhat.com [10.6.23.12]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B040519560AA; Fri, 14 Jun 2024 18:24:14 +0000 (UTC) Received: from bmarzins-01.fast.eng.rdu2.dc.redhat.com (localhost [127.0.0.1]) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.1) with ESMTPS id 45EIODnu952190 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Fri, 14 Jun 2024 14:24:13 -0400 Received: (from bmarzins@localhost) by bmarzins-01.fast.eng.rdu2.dc.redhat.com (8.17.2/8.17.2/Submit) id 45EIODvr952189; Fri, 14 Jun 2024 14:24:13 -0400 From: Benjamin Marzinski To: Mikulas Patocka , Mike Snitzer Cc: dm-devel@lists.linux.dev Subject: [PATCH 3/3] dm io: remove code duplication between sync_io and aysnc_io Date: Fri, 14 Jun 2024 14:24:12 -0400 Message-ID: <20240614182412.952166-4-bmarzins@redhat.com> In-Reply-To: <20240614182412.952166-1-bmarzins@redhat.com> References: <20240614182412.952166-1-bmarzins@redhat.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com The only difference between the code to setup and dispatch the io in sync_io() and async_io() is the sync argument to dispatch_io(), which is used to update the opf argument. Update the opf argument direcly in sync_io(), and remove the sync argument from dispatch_io(). Then, make sync_io() call async_io() instead of duplicting all of its code. Signed-off-by: Benjamin Marzinski --- drivers/md/dm-io.c | 59 ++++++++++++++++++---------------------------- 1 file changed, 23 insertions(+), 36 deletions(-) diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index 329a85a12061..d7a8e2f40db3 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -384,16 +384,13 @@ static void do_region(const blk_opf_t opf, unsigned int region, static void dispatch_io(blk_opf_t opf, unsigned int num_regions, struct dm_io_region *where, struct dpages *dp, - struct io *io, int sync, unsigned short ioprio) + struct io *io, unsigned short ioprio) { int i; struct dpages old_pages = *dp; BUG_ON(num_regions > DM_IO_MAX_REGIONS); - if (sync) - opf |= REQ_SYNC; - /* * For multiple regions we need to be careful to rewind * the dp object for each call to do_region. @@ -411,6 +408,26 @@ static void dispatch_io(blk_opf_t opf, unsigned int num_regions, dec_count(io, 0, 0); } +static void async_io(struct dm_io_client *client, unsigned int num_regions, + struct dm_io_region *where, blk_opf_t opf, + struct dpages *dp, io_notify_fn fn, void *context, + unsigned short ioprio) +{ + struct io *io; + + io = mempool_alloc(&client->pool, GFP_NOIO); + io->error_bits = 0; + atomic_set(&io->count, 1); /* see dispatch_io() */ + io->client = client; + io->callback = fn; + io->context = context; + + io->vma_invalidate_address = dp->vma_invalidate_address; + io->vma_invalidate_size = dp->vma_invalidate_size; + + dispatch_io(opf, num_regions, where, dp, io, ioprio); +} + struct sync_io { unsigned long error_bits; struct completion wait; @@ -428,22 +445,12 @@ static int sync_io(struct dm_io_client *client, unsigned int num_regions, struct dm_io_region *where, blk_opf_t opf, struct dpages *dp, unsigned long *error_bits, unsigned short ioprio) { - struct io *io; struct sync_io sio; init_completion(&sio.wait); - io = mempool_alloc(&client->pool, GFP_NOIO); - io->error_bits = 0; - atomic_set(&io->count, 1); /* see dispatch_io() */ - io->client = client; - io->callback = sync_io_complete; - io->context = &sio; - - io->vma_invalidate_address = dp->vma_invalidate_address; - io->vma_invalidate_size = dp->vma_invalidate_size; - - dispatch_io(opf, num_regions, where, dp, io, 1, ioprio); + async_io(client, num_regions, where, opf | REQ_SYNC, dp, + sync_io_complete, &sio, ioprio); wait_for_completion_io(&sio.wait); @@ -453,26 +460,6 @@ static int sync_io(struct dm_io_client *client, unsigned int num_regions, return sio.error_bits ? -EIO : 0; } -static void async_io(struct dm_io_client *client, unsigned int num_regions, - struct dm_io_region *where, blk_opf_t opf, - struct dpages *dp, io_notify_fn fn, void *context, - unsigned short ioprio) -{ - struct io *io; - - io = mempool_alloc(&client->pool, GFP_NOIO); - io->error_bits = 0; - atomic_set(&io->count, 1); /* see dispatch_io() */ - io->client = client; - io->callback = fn; - io->context = context; - - io->vma_invalidate_address = dp->vma_invalidate_address; - io->vma_invalidate_size = dp->vma_invalidate_size; - - dispatch_io(opf, num_regions, where, dp, io, 0, ioprio); -} - static int dp_init(struct dm_io_request *io_req, struct dpages *dp, unsigned long size) {