Skip to content

[Core] [Azure] - Some azure resources are not cleaned up during ray down #55392

@marosset

Description

@marosset

What happened + What you expected to happen

When cleaning up a ray cluster by running ray down with the Azure provider, some resources are not cleaned up.

specifically, there are managed identity, virtual network, and network security group object left around

ex:
Image

These objects have cluster-specific identifiers in them so I'm pretty sure they should get cleaned up along with the cluster

Ex: from the cached cluster config I see this

on/ray$ cat /tmp/ray-config-01cb007290805aeeac3694abb554fb46eae36aeb | jq
{
  "_version": 1,
  "provider_log_info": null,
  "config": {
    "cluster_name": "nightly-cpu-minimal-1754603045",
    "max_workers": 2,
    "upscaling_speed": 1.0,
    "docker": {},
    "idle_timeout_minutes": 5,
    "provider": {
      "cache_stopped_nodes": false,
      "location": "westus2",
      "resource_group": "ray-nightly-cpu-minimal",
      "type": "azure",
      "subscription_id": "xxx",
      "unique_id": "dff1",
      "msi": "/subscriptions/xxx/resourceGroups/ray-nightly-cpu-minimal/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ray-nightly-cpu-minimal-1754603045-dff1-msi",
      "nsg": "/subscriptions/xxx/resourceGroups/ray-nightly-cpu-minimal/providers/Microsoft.Network/networkSecurityGroups/ray-nightly-cpu-minimal-1754603045-dff1-nsg",
      "subnet": "/subscriptions/xxx/resourceGroups/ray-nightly-cpu-minimal/providers/Microsoft.Network/virtualNetworks/ray-nightly-cpu-minimal-1754603045-dff1-vnet/subnets/ray-nightly-cpu-minimal-1754603045-dff1-subnet"
    },
    "auth": {
      "ssh_private_key": "xxx",
      "ssh_public_key": "xxx",
      "ssh_user": "ubuntu"
    },

Versions / Dependencies

ray @ main

Reproduction script

run ray up with an azure cluster config then run ray down

Issue Severity

Low: It annoys or frustrates me.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1Issue that should be fixed within a few weeksazurebugSomething that is supposed to be working; but isn'tcommunity-backlogcoreIssues that should be addressed in Ray Corestability

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions