Fix disable dr when VR failed validation

[nirs]

When deleting a primary VRG, we wait until the VR Completed condition is
met. However if a VR precondition failed, for example using a drpolicy
without flattening enabled when the PVC needs flattening, the VR will
never complete and the vrg and drpc deletion will never complete.

Since csi-addons 0.10.0 we have a new Validated VR condition, set to
true if pre conditions are met, and false if not. VR is can be deleted
safely in this state, since mirroring was not enabled.

This changes modifies deleted VRG processing to check the new VR
Validated status. If the condition exist and the condition status is
false, validateVRStatus() return true, signaling that the VR is in the
desired state, and ramen completes the delete flow.

If the VR does not report the Validated condition (e.g. old csi-addon
version) or the condition status is true (mirroring in progress), we
continue in the normal flow. The VR will be deleted only when the
Completed condition status is true.

Tested locally with discovered app using a pvc created from a volume
snapshot.

Status:

• Test with discovered apps
• Test vm created from golden image
• Tests added in Test VR Validated condition #1577

[nirs]

New logs during disabling dr:

2024-09-24T19:42:04.755Z    INFO    controllers.VolumeReplicationGroup.vrginstance  controller/vrg_volrep.go:1428   VolumeReplication restored-pvc/flatten failed validation and can be deleted {"VolumeReplicationGroup": {"name":"flatten-drpc","namespace":"ramen-ops"}, "rid": "12b2da9d-8c78-40a8-893f-574ed2ea26ca", "State": "primary", "Finalize": true}

2024-09-24T19:42:04.755Z    INFO    controllers.VolumeReplicationGroup.vrginstance  controller/vrg_volrep.go:1839   Deleting VolumeReplication resource %s/%s   {"VolumeReplicationGroup": {"name":"flatten-drpc","namespace":"ramen-ops"}, "rid": "12b2da9d-8c78-40a8-893f-574ed2ea26ca", "State": "primary", "Finalize": true, "flatten": "restored-pvc"}

2024-09-24T19:42:04.884Z    INFO    controllers.VolumeReplicationGroup.vrginstance  controller/vrg_volrep.go:449    Deleting ramen annotations from PersistentVolume    {"VolumeReplicationGroup": {"name":"flatten-drpc","namespace":"ramen-ops"}, "rid": "615ab075-dfbf-4ad4-bba8-b016104463f2", "State": "primary", "Finalize": true, "pvc": "flatten/restored-pvc", "pv": "pvc-94c88954-8764-4e88-a438-158efa0a6a3d"}                                                                                                                                

2024-09-24T19:42:04.888Z    INFO    controllers.VolumeReplicationGroup.vrginstance  controller/vrg_volrep.go:467    Deleting ramen annotations and finallizers from PersistentVolumeClaim   {"VolumeReplicationGroup": {"name":"flatten-drpc","namespace":"ramen-ops"}, "rid": "615ab075-dfbf-4ad4-bba8-b016104463f2", "State": "primary", "Finalize": true, "pvc": "flatten/restored-pvc", "owner removed": true, "finalizer removed": true}

Complete log:
disable-dr.log

[ShyamsundarR]

Isn't this a straight call to check the condition, do we need a wrapper function for this?

Suggested change

	validated, ok := v.validateVRValidatedStatus(volRep)
	conditionMet, errorMsg := isVRConditionMet(volRep, volrep.ConditionValidated, metav1.ConditionTrue)
	if !conditionMet && errorMsg == "" {

[nirs]

Here we can update the DataReady and Protected conditions since we have more specific error message. I remvoed this update since many tests failed, I think the issue is wrong validation in the test and it is not related to this update, but I focused on solving the main problem. We can improve this in the next step.

[nirs]

Also, v.validateVRValidatedStatus is a good place to log errors about missing, stale, or unknown condition, similar to validateCompletedStatus and the other helper for secondary vrg.