'Add a new Cloud Volume' form is vendor specific

[jaywcarman]

I'm implementing Cloud Volume creation in the manageiq-providers-ibm_cloud (specifically for the PowerVirtualServers::StorageManager). The 'Add a new Cloud Volume' - form is specific to OpenStack Cinder and Amazon EBS.

Per the discussion in #7187 I would like to work towards using provider defined DDF schema for this form.

• Core Introducing supports cloud_volume_create manageiq#20761
• Amazon - Declared DDF schema for cloud volume creation and editing manageiq-providers-amazon#657
• IBM - Declared DDF schema for cloud volume creation and editing manageiq-providers-ibm_cloud#84
• OpenStack - Declared DDF schema for cloud volume creation and editing manageiq-providers-openstack#644
• Google - Add dummy DDF schema for CloudVolume editing manageiq-providers-google#169
• API endpoint - Expose the DDF schema for CloudVolume subclasses via OPTIONS manageiq-api#914
• UI - Rewriting Cloud Volume Form to React #7399

[skateman]

Just for the record, what providers have CloudVolume entities?

[jaywcarman]

I cloned ManageIQ/manageiq-providers-* and used grep/find (is there a better way to search through MIQ plugins?).

Only Amazon, OpenStack and IBM Cloud have implemented CloudVolume creation:

• Amazon EBS, manageiq-providers-amazon:
  https://github.com/ManageIQ/manageiq-providers-amazon/blob/master/app/models/manageiq/providers/amazon/storage_manager/ebs/cloud_volume.rb

• OpenStack Cinder, manageiq-providers-openstack:
  https://github.com/ManageIQ/manageiq-providers-openstack/blob/master/app/models/manageiq/providers/openstack/cloud_manager/cloud_volume.rb

• IBM Cloud PowerVS, manageiq-providers-ibm_cloud (create is still a local WIP):
  https://github.com/ManageIQ/manageiq-providers-ibm_cloud/blob/master/app/models/manageiq/providers/ibm_cloud/power_virtual_servers/storage_manager/cloud_volume.rb

---

• Google does store CloudVolumes in inventory, but doesn't seem to implement any operations on them so I don't think it would require any updates:
  https://github.com/ManageIQ/manageiq-providers-google/blob/master/app/models/manageiq/providers/google/inventory/parser/cloud_manager.rb#L247

[skateman]

Okay, so now I need to know the specifics of each CloudVolume form, i.e. the schemas for each provider.

[skateman]

I guess these fields will be common for each provider and they will be living in the JS schema:

commonFields = [
  {
    component: componentTypes.TEXT_FIELD,
    name: name,
    id: name,
    label: __('Volume Name'),
    isRequired: true,
    validate: [{ type: validatorTypes.REQUIRED }],
  },
  {
    component: componentTypes.SELECT_FIELD,
    name: ext_management_system_id,
    id: ext_management_system_id,
    label: __('Storage Manager'),
    loadOptions: () => API.get('/api/providers'), // TODO: needs filtering based on the supports mixin (?)
    isRequired: true,
    validate: [{ type: validatorTypes.REQUIRED }],
  },
]

These fields are amazon-specific and will be populated via the API and they have to be defined in the Ruby model:

[
  {
    :component  => 'select',
    :name       => 'availability_zone_id',
    :id         => 'availability_zone_id',
    :label      => _('Availability Zone'),
    :options    => AvailabilityZone.where(:ext_management_system_id => params[:ems_id]), # the URL param might change
    :isRequired => true,
    :validate   => [{ :type => 'required' }],
  },
  {
    :component  => 'select',
    :name       => 'volume_type',
    :id         => 'volume_type',
    :label      => _('Cloud Volume Type'),
    :options    => self.class.CLOUD_VOLUME_TYPES, # TODO: the types are in the angular controller, need to extract them
  }, # TODO: ask Adam about isRequired
  {
    :component  => 'text-field',
    :name       => 'size',
    :id         => 'size',
    :label      => _('Size (in gigabytes'), # TODO: this might be multiplied up in angular
    :type       => 'number',
    :isRequired => true,
    :validate   => [{ :type => 'required' }],
  }, # FIXME: I guess the size is a common parameter too, so it might be extractable
  {
    :component  => 'text-field',
    :name       => 'iops',
    :id         => 'iops',
    :label      => _('IOPS'),
    :type       => 'number', # FIXME: just a guess, ask Adam
    :condition  => {
      :when => 'volume_type',
      :is   => 'io1',
    }
  }, # FIXME: ask Adam if it's required
  {
    :component  => 'select',
    :name       => 'cloud_volume_snapshot_id',
    :id         => 'cloud_volume_snapshot_id',
    :label      => _('Base Snapshot'),
    :options    => CloudVolumeSnapshot.where(:ext_management_system_id => params[:ems_id]).map do |cvs|
      {
        :value => cvs.id,
        :label => cvs.name,
      }
    end
  },
  {
    :component  => 'switch',
    :name       => 'encrypted',
    :id         => 'encrypted',
    :label      => _('Encrypted'),
    :onText     => _('Yes'),
    :offText    => _('No'),
  }
]

@agrare I have some questions inside the schema as comments, can you please answer them?

[agrare]

Volume Types
Looks like the list is AWS specific, but OpenStack also collects CloudVolumeTypes. Probably should be required for aws with a default of gp2 and not required otherwise

IOPS - from AWS API Docs

The number of I/O operations per second (IOPS) to provision for an io1 or io2 volume, with a maximum ratio of 50 IOPS/GiB for io1, and 500 IOPS/GiB for io2. Range is 100 to 64,000 IOPS for volumes in most Regions. Maximum IOPS of 64,000 is guaranteed only on Nitro-based instances. Other instance families guarantee performance up to 32,000 IOPS. For more information, see Amazon EBS volume types in the Amazon Elastic Compute Cloud User Guide.

This parameter is valid only for Provisioned IOPS SSD (io1 and io2) volumes.

Type: Integer

Required: No

Not required and only applies to AWS with volume_type io1

[jaywcarman]

Here's a first draft of the IbmCloud::PowerVirtualServers schema. I still own you an OpenStack schema.

[
  {
    :component  => 'text-field',
    :name       => 'size',
    :id         => 'size',
    :label      => _('Size (in gigabytes'), # TODO: this might be multiplied up in angular
    :type       => 'number',
    :isRequired => true,
    :validate   => [{ :type => 'required' }]
  }, # FIXME: I guess the size is a common parameter too, so it might be extractable
  {
    :component  => 'switch',
    :name       => 'shareable',
    :id         => 'shareable',
    :label      => _('Shareable'),
    :onText     => _('Yes'),
    :offText    => _('No')
  },
  {
    :component    => 'select',
    :name         => 'affinity_policy',
    :id           => 'affinity_policy',
    :label        => _('Affinity Policy'),
    :initialValue => 'off',
    :options      => [
      {
        :label => 'Off',
        :value => 'off'
      },
      {
        :label => 'Affinity',
        :value => 'affinity'
      },
      {
        :label => 'Anti-affinity',
        :value => 'anti-affinity'
      }
    ]
  },
  {
    :component  => 'select',
    :name       => 'volume_type',
    :id         => 'volume_type',
    :label      => _('Cloud Volume Type'),
    :options    => self.class.CLOUD_VOLUME_TYPES, # TODO: the types are in the angular controller, need to extract them
    :condition  => {
      :when => 'affinity_policy',
      :is   => 'off'
    },
    :validate   => [{ :type => 'required' }]
  },
  {
    :component  => 'select',
    :name       => 'affinity_volume_id',
    :id         => 'affinity_volume_id',
    :label      => _('Affinity Volume'),
    :options    => CloudVolume.where(:ext_management_system_id => params[:ems_id]).map do |cv|
      {
        :value => cv.id,
        :label => cv.name,
      }
    end,
    :condition  => {
      not: {
        :when => 'affinity_policy',
        :is   => 'affinity'
      }
    },
    :validate   => [{ :type => 'required' }]
  }
]

[skateman]

Volume Types
Looks like the list is AWS specific, but OpenStack also collects CloudVolumeTypes. Probably should be required for aws with a default of gp2 and not required otherwise

Yeah, the ruby schema above is for AWS only, the rest will come later from @jaywcarman

IOPS - from AWS API Docs

The number of I/O operations per second (IOPS) to provision for an io1 or io2 volume, with a maximum ratio of 50 IOPS/GiB for io1, and 500 IOPS/GiB for io2. Range is 100 to 64,000 IOPS for volumes in most Regions. Maximum IOPS of 64,000 is guaranteed only on Nitro-based instances. Other instance families guarantee performance up to 32,000 IOPS. For more information, see Amazon EBS volume types in the Amazon Elastic Compute Cloud User Guide.

This parameter is valid only for Provisioned IOPS SSD (io1 and io2) volumes.

Type: Integer

Required: No

Not required and only applies to AWS with volume_type io1

I'm hiding it for everything else already

[skateman]

I'm curious if the size for each provider is in the same unit? We're storing an integer, but I am guessing that the angular form multiplies up the number for Amazon at least. I'm just worried about the collision of different units (megabytes, gigabytes, etc) among providers here.

I'd not want to do the multiplication on the frontend at all, but show the raw value and use a steps parameter on the input field set to 1024, so if the user is setting the size by using the up/down buttons, it goes up by 1024 for example.

[agrare]

The value of CloudVolume#size is bytes, ideally we would pass everything down in bytes and if a provider requires gigabytes they can convert it.

[skateman]

@DavidResende0 your ansible credential form also had dynamic schema, this one would be something similar. The only difference is that the current form is not dynamic. I guess this should be easy to do for you, wdyt?