| Message ID | 20250310141813.969325-1-cel@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | Block device provisioning for storage nodes | expand |
On Mon, Mar 10, 2025 at 10:18:00AM -0400, cel@kernel.org wrote: > From: Chuck Lever <chuck.lever@oracle.com> > > Hi - > > Sorry for the length of the series. I'm posting the series as a > whole to provide context for each of the individual changes. Feel > free to focus on whichever patch or patches in this series are most > interesting to you. All review comments are welcome. > > The high-level goal is to enable testing NFS / SMB / iSCSI with > kdevops in the cloud. These workflows are already operational for > guestfs. The basic challenge is each cloud provider has a distinct > way of provisioning and attaching block devices. > > This series can be considered in two sections: > > - the first four patches wrangle the terraform for some of the > cloud providers to make them provision a set of extra block > volumes, each of the same size, just as guestfs and AWS > currently do. > > - the second half of the series adds a new playbook that: > > - de-duplicates the scripting that creates an LVM volume group, > because three different roles each implement this the same way > > - replaces the "skip one device" method for determining which > extra block volume the /data partition should live on. AWS still > needs some work here because NVMe devices can get renamed after > every instance reboot > > - adds LVM support that handles the differences amongst the cloud > providers, tucked away in the new playbook > > > GCE and OpenStack are not yet updated, but they are in plan. Reviewed-by: Luis Chamberlain <mcgrof@kernel.org> Luis
From: Chuck Lever <chuck.lever@oracle.com> Hi - Sorry for the length of the series. I'm posting the series as a whole to provide context for each of the individual changes. Feel free to focus on whichever patch or patches in this series are most interesting to you. All review comments are welcome. The high-level goal is to enable testing NFS / SMB / iSCSI with kdevops in the cloud. These workflows are already operational for guestfs. The basic challenge is each cloud provider has a distinct way of provisioning and attaching block devices. This series can be considered in two sections: - the first four patches wrangle the terraform for some of the cloud providers to make them provision a set of extra block volumes, each of the same size, just as guestfs and AWS currently do. - the second half of the series adds a new playbook that: - de-duplicates the scripting that creates an LVM volume group, because three different roles each implement this the same way - replaces the "skip one device" method for determining which extra block volume the /data partition should live on. AWS still needs some work here because NVMe devices can get renamed after every instance reboot - adds LVM support that handles the differences amongst the cloud providers, tucked away in the new playbook GCE and OpenStack are not yet updated, but they are in plan. Chuck Lever (13): terraform/AWS: Upgrade the EBS volume type to "gp3" terraform/Azure: Remove managed_disk_type selection terraform/Azure: Create a set of multiple generic block devices terraform/OCI: Create a set of multiple generic block devices guestfs: Set storage options consistently playbooks: Add a role to create an LVM volume group volume_group: Detect the /data partition directly volume_group: Prepare to support cloud providers volume_group: Create volume group on terraform/AWS nodes volume_group: Create a volume group on Azure nodes volume_group: Create a volume group on GCE nodes volume_group: Create a volume group on OCI nodes volume_group: Create a volume group on OpenStack public clouds kconfigs/Kconfig.iscsi | 22 --- kconfigs/Kconfig.libvirt | 3 + kconfigs/Kconfig.nfsd | 27 +-- kconfigs/Kconfig.smbd | 17 -- playbooks/roles/gen_nodes/defaults/main.yml | 2 +- playbooks/roles/gen_tfvars/defaults/main.yml | 1 - .../templates/azure/terraform.tfvars.j2 | 4 +- .../templates/oci/terraform.tfvars.j2 | 6 + playbooks/roles/iscsi/defaults/main.yml | 2 - playbooks/roles/iscsi/tasks/main.yml | 23 +-- playbooks/roles/iscsi/vars/Debian.yml | 1 - playbooks/roles/iscsi/vars/RedHat.yml | 1 - playbooks/roles/iscsi/vars/Suse.yml | 1 - playbooks/roles/nfsd/defaults/main.yml | 3 - playbooks/roles/nfsd/tasks/main.yml | 21 +-- playbooks/roles/smbd/defaults/main.yml | 3 - playbooks/roles/smbd/tasks/main.yml | 21 +-- playbooks/roles/volume_group/README.md | 42 +++++ .../roles/volume_group/defaults/main.yml | 4 + .../roles/volume_group/tasks/guestfs.yml | 59 ++++++ playbooks/roles/volume_group/tasks/main.yml | 42 +++++ .../volume_group/tasks/terraform/aws.yml | 54 ++++++ .../volume_group/tasks/terraform/azure.yml | 40 +++++ .../volume_group/tasks/terraform/gce.yml | 4 + .../volume_group/tasks/terraform/oci.yml | 38 ++++ .../tasks/terraform/openstack.yml | 4 + scripts/gen-nodes.Makefile | 10 -- scripts/iscsi.Makefile | 2 - scripts/nfsd.Makefile | 2 - scripts/smbd.Makefile | 2 - scripts/terraform.Makefile | 10 +- terraform/aws/main.tf | 3 +- terraform/azure/Kconfig | 168 ++++++++++++++++-- terraform/azure/main.tf | 48 ++--- terraform/azure/managed_disks/main.tf | 20 +++ terraform/azure/managed_disks/vars.tf | 29 +++ terraform/azure/vars.tf | 17 +- terraform/oci/Kconfig | 153 ++++++++++++++++ terraform/oci/main.tf | 28 ++- terraform/oci/vars.tf | 17 ++ 40 files changed, 744 insertions(+), 210 deletions(-) create mode 100644 playbooks/roles/volume_group/README.md create mode 100644 playbooks/roles/volume_group/defaults/main.yml create mode 100644 playbooks/roles/volume_group/tasks/guestfs.yml create mode 100644 playbooks/roles/volume_group/tasks/main.yml create mode 100644 playbooks/roles/volume_group/tasks/terraform/aws.yml create mode 100644 playbooks/roles/volume_group/tasks/terraform/azure.yml create mode 100644 playbooks/roles/volume_group/tasks/terraform/gce.yml create mode 100644 playbooks/roles/volume_group/tasks/terraform/oci.yml create mode 100644 playbooks/roles/volume_group/tasks/terraform/openstack.yml create mode 100644 terraform/azure/managed_disks/main.tf create mode 100644 terraform/azure/managed_disks/vars.tf