From patchwork Wed May 8 09:36:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Changqi Lu X-Patchwork-Id: 13658742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8C6D8C04FFE for ; Wed, 8 May 2024 13:13:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s4h5w-0005GB-46; Wed, 08 May 2024 09:12:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s4dl7-0007p3-PR for qemu-devel@nongnu.org; Wed, 08 May 2024 05:38:46 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s4dl5-0000cK-NG for qemu-devel@nongnu.org; Wed, 08 May 2024 05:38:45 -0400 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1ed012c1afbso5437905ad.1 for ; Wed, 08 May 2024 02:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1715161122; x=1715765922; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YzUXiaM2rF7jrfvYaToKOL09mhX+9O3YqGRC0KLKQCU=; b=N5wBxn+xXLp05eWpv7OrdY+rzi2bcuBEKStX5e3ICJAdXOqPr0biEXRDKhK/mQAJhA HG+d3a2qNXFyWnP9hQOhjRU8jlPkmaAFbHaundPMVF1/KSFzG6WTA2Yd6yrPxU65GjS4 c92FEmBBXxmPy0xjGQ5rRiBVDZdaPXmyFyPA3KFD7lIcB0Tbr3mx/NxIazT0Wt4Njy8U 29Qza7ebwbGHQb72XWJTe5BcrjZ1RmqpBhplef4gNGcV1smh4AX52+JCmUBcXg8lbplF MW7mLKI3ZllynHQ3gH5TuaAlkAmzucossqS7jDgHO9olVMOv4X/1kEtNvvhB45fHodcm RMfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715161122; x=1715765922; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YzUXiaM2rF7jrfvYaToKOL09mhX+9O3YqGRC0KLKQCU=; b=sQs4G9Yunq4jy6ikeudhql+uE+niuvEf/BvIQeJpKCP2ijDdQBNnGCAP0QJQ+c3iJB zvAtzQpVt8Dv8eTARlbueSteboECVmIVr/9rpRvPUMTDyMv0NaC2cSBJ599JCAIvfNg4 yYyUxdbZZMqQ7rW+bWydtZTx4Lg6EbpT95BS36UvCnH+58AEdECkEbvjqYKSJPF8u6cU +J+7udSDQXTiOGBZxYTWvMdxfou1em9bGYHOBhCnDtHKOaqow9s/IrPOxUrECbXtWn3g x9/nl3KUFyyWfNbyg2JaG6u3n1brk9LKuln83kMFaCXNEvq3a/zvZRHe9+cxBbsmtp2x 1FPQ== X-Forwarded-Encrypted: i=1; AJvYcCV0kRAT/+VNa9d6Xr19sZYIZBAey2J9Muoa2czTeYRCdd6ABrgkcOj217+1NRUN50iYLSfsvPG6IRvoK4rn6G8K7v1ZFtg= X-Gm-Message-State: AOJu0Yxcy0RcdjLA9dEFOAv/GXeuopR0AK3qP+OO8MOyhs7b8n27vsYi JmwP5dA4idd2pFOI+9sG0fVyPTXKauDvEYSf+FU1kkEX6hljTvJMw6QIonIfSYs= X-Google-Smtp-Source: AGHT+IEO7jIQCm0v2ZcSn30JiFDREb6feNyGHvVZcsweJZrubutWiazXJxq7E8kHYJUe3kEGa8DzNQ== X-Received: by 2002:a17:902:650a:b0:1ec:c6ba:f2c3 with SMTP id d9443c01a7336-1eeab69ebc2mr26315975ad.2.1715161122488; Wed, 08 May 2024 02:38:42 -0700 (PDT) Received: from n37-006-243.byted.org ([180.184.51.134]) by smtp.gmail.com with ESMTPSA id p16-20020a1709027ed000b001ee13eb2bedsm4992178plb.98.2024.05.08.02.38.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 02:38:42 -0700 (PDT) From: Changqi Lu To: qemu-block@nongnu.org, qemu-devel@nongnu.org Cc: kwolf@redhat.com, hreitz@redhat.com, stefanha@redhat.com, fam@euphon.net, ronniesahlberg@gmail.com, pbonzini@redhat.com, pl@dlhnet.de, kbusch@kernel.org, its@irrelevant.dk, foss@defmacro.it, philmd@linaro.org, Changqi Lu , zhenwei pi Subject: [PATCH 9/9] block/iscsi: add persistent reservation in/out driver Date: Wed, 8 May 2024 17:36:29 +0800 Message-Id: <20240508093629.441057-10-luchangqi.123@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240508093629.441057-1-luchangqi.123@bytedance.com> References: <20240508093629.441057-1-luchangqi.123@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=luchangqi.123@bytedance.com; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 08 May 2024 09:12:22 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add persistent reservation in/out operations for iscsi driver. The following methods are implemented: bdrv_co_pr_read_keys, bdrv_co_pr_read_reservation, bdrv_co_pr_register, bdrv_co_pr_reserve, bdrv_co_pr_release, bdrv_co_pr_clear and bdrv_co_pr_preempt. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- block/iscsi.c | 390 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 390 insertions(+) diff --git a/block/iscsi.c b/block/iscsi.c index 2ff14b7472..212fab53ad 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -280,6 +280,8 @@ iscsi_co_generic_cb(struct iscsi_context *iscsi, int status, iTask->err_code = -error; iTask->err_str = g_strdup(iscsi_get_error(iscsi)); } + } else if (status == SCSI_STATUS_RESERVATION_CONFLICT) { + iTask->err_code = -EBADE; } } } @@ -2408,6 +2410,386 @@ out_unlock: return r; } +static int coroutine_fn +iscsi_co_pr_read_keys(BlockDriverState *bs, uint32_t *generation, + uint32_t num_keys, uint64_t *keys) +{ + IscsiLun *iscsilun = bs->opaque; + QEMUIOVector qiov; + struct IscsiTask iTask; + int xferlen = sizeof(struct scsi_persistent_reserve_in_read_keys) + + sizeof(uint64_t) * num_keys; + uint8_t *buf = g_malloc0(xferlen); + int32_t num_collect_keys = 0; + int r = 0; + + qemu_iovec_init_buf(&qiov, buf, xferlen); + iscsi_co_init_iscsitask(iscsilun, &iTask); + qemu_mutex_lock(&iscsilun->mutex); +retry: + iTask.task = iscsi_persistent_reserve_in_task(iscsilun->iscsi, + iscsilun->lun, SCSI_PR_IN_READ_KEYS, xferlen, + iscsi_co_generic_cb, &iTask); + + if (iTask.task == NULL) { + qemu_mutex_unlock(&iscsilun->mutex); + return -ENOMEM; + } + + scsi_task_set_iov_in(iTask.task, (struct scsi_iovec *)qiov.iov, qiov.niov); + iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; + } + + if (iTask.do_retry) { + iTask.complete = 0; + goto retry; + } + + if (iTask.status != SCSI_STATUS_GOOD) { + error_report("iSCSI PERSISTENT_RESERVE_IN failed: %s", iTask.err_str); + r = iTask.err_code; + goto out; + } + + memcpy(generation, &buf[0], 4); + *generation = be32_to_cpu(*generation); + memcpy(&num_collect_keys, &buf[4], 4); + if (num_collect_keys > num_keys) { + r = -EINVAL; + goto out; + } + + for (int i = 0; i < num_collect_keys; i++) { + memcpy(&keys[i], &buf[8 + i * 8], 8); + keys[i] = be64_to_cpu(keys[i]); + } + +out: + qemu_mutex_unlock(&iscsilun->mutex); + g_free(iTask.err_str); + g_free(buf); + return r; +} + +static int coroutine_fn +iscsi_co_pr_read_reservation(BlockDriverState *bs, uint32_t *generation, + uint64_t *key, BlockPrType *type) +{ + IscsiLun *iscsilun = bs->opaque; + QEMUIOVector qiov; + struct IscsiTask iTask; + int xferlen = sizeof(struct scsi_persistent_reserve_in_read_reservation); + uint8_t *buf = g_malloc0(xferlen); + uint8_t scope_type = 0; + int32_t num_collect_keys = 0; + int r = 0; + + qemu_iovec_init_buf(&qiov, buf, xferlen); + iscsi_co_init_iscsitask(iscsilun, &iTask); + qemu_mutex_lock(&iscsilun->mutex); +retry: + iTask.task = iscsi_persistent_reserve_in_task(iscsilun->iscsi, + iscsilun->lun, SCSI_PR_IN_READ_RESERVATION, + xferlen, iscsi_co_generic_cb, &iTask); + + if (iTask.task == NULL) { + qemu_mutex_unlock(&iscsilun->mutex); + return -ENOMEM; + } + + scsi_task_set_iov_in(iTask.task, (struct scsi_iovec *)qiov.iov, qiov.niov); + iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; + } + + if (iTask.do_retry) { + iTask.complete = 0; + goto retry; + } + + if (iTask.status != SCSI_STATUS_GOOD) { + error_report("iSCSI PERSISTENT_RESERVE_IN failed: %s", iTask.err_str); + r = iTask.err_code; + goto out; + } + + memcpy(generation, &buf[0], 4); + *generation = be32_to_cpu(*generation); + memcpy(key, &buf[8], 8); + *key = be64_to_cpu(*key); + memcpy(&scope_type, &buf[21], 1); + *type = scsi_pr_type_to_block(scope_type & 0xf); + memcpy(&num_collect_keys, &buf[4], 4); + r = be32_to_cpu(num_collect_keys) / sizeof(uint64_t); +out: + qemu_mutex_unlock(&iscsilun->mutex); + g_free(iTask.err_str); + g_free(buf); + return r; +} + +static int coroutine_fn +iscsi_co_pr_register(BlockDriverState *bs, uint64_t old_key, + uint64_t new_key, BlockPrType type, + bool ignore_key) +{ + IscsiLun *iscsilun = bs->opaque; + struct IscsiTask iTask; + struct scsi_persistent_reserve_out_basic *basic; + SCSIPrOutAction action = ignore_key ? SCSI_PR_OUT_REG_AND_IGNORE_KEY : + SCSI_PR_OUT_REGISTER; + int r = 0; + + basic = g_new0(struct scsi_persistent_reserve_out_basic, 1); + /** + * spec_i_pt/all_tg_pt is currently not supported, + * aptpl is enable by default. + */ + basic->reservation_key = old_key; + basic->service_action_reservation_key = new_key; + basic->aptpl = 1; + + iscsi_co_init_iscsitask(iscsilun, &iTask); + qemu_mutex_lock(&iscsilun->mutex); +retry: + iTask.task = iscsi_persistent_reserve_out_task(iscsilun->iscsi, + iscsilun->lun, action, 0, block_pr_type_to_scsi(type), + basic, iscsi_co_generic_cb, &iTask); + + if (iTask.task == NULL) { + qemu_mutex_unlock(&iscsilun->mutex); + return -ENOMEM; + } + + iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; + } + + if (iTask.do_retry) { + iTask.complete = 0; + goto retry; + } + + if (iTask.status != SCSI_STATUS_GOOD) { + error_report("iSCSI PERSISTENT_RESERVE_OUT failed: %s", iTask.err_str); + r = iTask.err_code; + } + + qemu_mutex_unlock(&iscsilun->mutex); + + g_free(iTask.err_str); + g_free(basic); + return r; +} + +static int coroutine_fn +iscsi_co_pr_reserve(BlockDriverState *bs, uint64_t key, BlockPrType type) +{ + IscsiLun *iscsilun = bs->opaque; + struct IscsiTask iTask; + struct scsi_persistent_reserve_out_basic *basic; + int r = 0; + + basic = g_new0(struct scsi_persistent_reserve_out_basic, 1); + basic->reservation_key = key; + + iscsi_co_init_iscsitask(iscsilun, &iTask); + qemu_mutex_lock(&iscsilun->mutex); +retry: + iTask.task = iscsi_persistent_reserve_out_task(iscsilun->iscsi, + iscsilun->lun, SCSI_PR_OUT_RESERVE, 0, + block_pr_type_to_scsi(type), basic, + iscsi_co_generic_cb, &iTask); + + if (iTask.task == NULL) { + qemu_mutex_unlock(&iscsilun->mutex); + return -ENOMEM; + } + + + iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; + } + + if (iTask.do_retry) { + iTask.complete = 0; + goto retry; + } + + if (iTask.status != SCSI_STATUS_GOOD) { + error_report("iSCSI PERSISTENT_RESERVE_OUT failed: %s", iTask.err_str); + r = iTask.err_code; + } + + qemu_mutex_unlock(&iscsilun->mutex); + + g_free(iTask.err_str); + g_free(basic); + return r; +} + +static int coroutine_fn +iscsi_co_pr_release(BlockDriverState *bs, uint64_t key, BlockPrType type) +{ + IscsiLun *iscsilun = bs->opaque; + struct IscsiTask iTask; + struct scsi_persistent_reserve_out_basic *basic; + int r = 0; + + basic = g_new0(struct scsi_persistent_reserve_out_basic, 1); + basic->reservation_key = key; + + iscsi_co_init_iscsitask(iscsilun, &iTask); + qemu_mutex_lock(&iscsilun->mutex); +retry: + iTask.task = iscsi_persistent_reserve_out_task(iscsilun->iscsi, + iscsilun->lun, SCSI_PR_OUT_RELEASE, 0, + block_pr_type_to_scsi(type), basic, + iscsi_co_generic_cb, &iTask); + + if (iTask.task == NULL) { + qemu_mutex_unlock(&iscsilun->mutex); + return -ENOMEM; + } + + + iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; + } + + if (iTask.do_retry) { + iTask.complete = 0; + goto retry; + } + + if (iTask.status != SCSI_STATUS_GOOD) { + error_report("iSCSI PERSISTENT_RESERVE_OUT failed: %s", iTask.err_str); + r = iTask.err_code; + } + + qemu_mutex_unlock(&iscsilun->mutex); + + g_free(iTask.err_str); + g_free(basic); + return r; +} + +static int coroutine_fn +iscsi_co_pr_clear(BlockDriverState *bs, uint64_t key) +{ + IscsiLun *iscsilun = bs->opaque; + struct IscsiTask iTask; + struct scsi_persistent_reserve_out_basic *basic; + int r = 0; + + basic = g_new0(struct scsi_persistent_reserve_out_basic, 1); + basic->reservation_key = key; + + iscsi_co_init_iscsitask(iscsilun, &iTask); + qemu_mutex_lock(&iscsilun->mutex); +retry: + iTask.task = iscsi_persistent_reserve_out_task(iscsilun->iscsi, + iscsilun->lun, SCSI_PR_OUT_CLEAR, 0, 0, basic, + iscsi_co_generic_cb, &iTask); + + if (iTask.task == NULL) { + qemu_mutex_unlock(&iscsilun->mutex); + return -ENOMEM; + } + + + iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; + } + + if (iTask.do_retry) { + iTask.complete = 0; + goto retry; + } + + if (iTask.status != SCSI_STATUS_GOOD) { + error_report("iSCSI PERSISTENT_RESERVE_OUT failed: %s", iTask.err_str); + r = iTask.err_code; + } + + qemu_mutex_unlock(&iscsilun->mutex); + + g_free(iTask.err_str); + g_free(basic); + return r; +} + +static int coroutine_fn +iscsi_co_pr_preempt(BlockDriverState *bs, uint64_t old_key, + uint64_t new_key, BlockPrType type, bool abort) +{ + IscsiLun *iscsilun = bs->opaque; + struct IscsiTask iTask; + struct scsi_persistent_reserve_out_basic *basic; + SCSIPrOutAction action = abort ? SCSI_PR_OUT_PREEMPT_AND_ABORT : + SCSI_PR_OUT_PREEMPT; + int r = 0; + + basic = g_new0(struct scsi_persistent_reserve_out_basic, 1); + basic->reservation_key = old_key; + basic->service_action_reservation_key = new_key; + + iscsi_co_init_iscsitask(iscsilun, &iTask); + qemu_mutex_lock(&iscsilun->mutex); +retry: + iTask.task = iscsi_persistent_reserve_out_task(iscsilun->iscsi, + iscsilun->lun, action, 0, block_pr_type_to_scsi(type), + basic, iscsi_co_generic_cb, &iTask); + + if (iTask.task == NULL) { + qemu_mutex_unlock(&iscsilun->mutex); + return -ENOMEM; + } + + + iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; + } + + if (iTask.do_retry) { + iTask.complete = 0; + goto retry; + } + + if (iTask.status != SCSI_STATUS_GOOD) { + error_report("iSCSI PERSISTENT_RESERVE_OUT failed: %s", iTask.err_str); + r = iTask.err_code; + } + + qemu_mutex_unlock(&iscsilun->mutex); + + + g_free(iTask.err_str); + g_free(basic); + return r; +} static const char *const iscsi_strong_runtime_opts[] = { "transport", @@ -2451,6 +2833,14 @@ static BlockDriver bdrv_iscsi = { .bdrv_co_writev = iscsi_co_writev, .bdrv_co_flush_to_disk = iscsi_co_flush, + .bdrv_co_pr_read_keys = iscsi_co_pr_read_keys, + .bdrv_co_pr_read_reservation = iscsi_co_pr_read_reservation, + .bdrv_co_pr_register = iscsi_co_pr_register, + .bdrv_co_pr_reserve = iscsi_co_pr_reserve, + .bdrv_co_pr_release = iscsi_co_pr_release, + .bdrv_co_pr_clear = iscsi_co_pr_clear, + .bdrv_co_pr_preempt = iscsi_co_pr_preempt, + #ifdef __linux__ .bdrv_aio_ioctl = iscsi_aio_ioctl, #endif