Skip to content

cannot run planner in simulated environments #8221

@davepacheco

Description

@davepacheco

I tried to run omdb nexus blueprints regenerate against omicron-dev run-all, but it failed:

$ omdb --destructive nexus blueprints regenerate
note: Nexus URL not specified.  Will pick one from DNS.
note: using Nexus URL http://[::1]:12221
Error: generating blueprint

Caused by:
    Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9fdc761c-6e35-4097-aa39-995e96c523e1", "content-length": "124", "date": "Tue, 27 May 2025 17:40:02 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9fdc761c-6e35-4097-aa39-995e96c523e1" }

Here are the logs for that request (excluding authz-related ones):

$ grep 9fdc761c-6e35-4097-aa39-995e96c523e1 "/dangerzone/omicron_tmp/omicron-dev-omicron-dev.16474.0.log" | grep -v '"authorize result"' | grep -v '"loading roles"' | grep -v '"roles"' | grep -v '"authorize begin"' | looker
17:40:02.295Z TRAC e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (dropshot_internal): incoming request
    local_addr = [::1]:12221
    method = POST
    remote_addr = [::1]:58174
    req_id = 9fdc761c-6e35-4097-aa39-995e96c523e1
    uri = /deployment/blueprints/regenerate
17:40:02.451Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (dropshot_internal): altered physical disks
    actor_id = 001de000-05e4-4000-8000-000000000002
    authenticated = true
    local_addr = [::1]:12221
    method = POST
    remote_addr = [::1]:58174
    req_id = 9fdc761c-6e35-4097-aa39-995e96c523e1
    sled_edits = SledEditCounts { disks: EditCounts { added: 3, updated: 0, expunged: 0, removed: 0 }, datasets: EditCounts { added: 6, updated: 0, expunged: 0, removed: 0 }, zones: EditCounts { added: 0, updated: 0, expunged: 0, removed: 0 } }
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /deployment/blueprints/regenerate
17:40:02.452Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (dropshot_internal): found sled missing NTP zone (will add one)
    actor_id = 001de000-05e4-4000-8000-000000000002
    authenticated = true
    local_addr = [::1]:12221
    method = POST
    remote_addr = [::1]:58174
    req_id = 9fdc761c-6e35-4097-aa39-995e96c523e1
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /deployment/blueprints/regenerate
17:40:02.452Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (dropshot_internal): sled getting NTP zone has other services already; considering it eligible for discretionary zones
    actor_id = 001de000-05e4-4000-8000-000000000002
    authenticated = true
    local_addr = [::1]:12221
    method = POST
    remote_addr = [::1]:58174
    req_id = 9fdc761c-6e35-4097-aa39-995e96c523e1
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /deployment/blueprints/regenerate
17:40:02.452Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (dropshot_internal): parent blueprint contains NTP zone, but it's not in inventory yet
    actor_id = 001de000-05e4-4000-8000-000000000002
    authenticated = true
    local_addr = [::1]:12221
    method = POST
    remote_addr = [::1]:58174
    req_id = 9fdc761c-6e35-4097-aa39-995e96c523e1
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /deployment/blueprints/regenerate
17:40:02.452Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (dropshot_internal): request completed
    error_message_external = Internal Server Error
    error_message_internal = error generating blueprint: no Boundary NTP zones exist in parent blueprint
    latency_us = 156832
    local_addr = [::1]:12221
    method = POST
    remote_addr = [::1]:58174
    req_id = 9fdc761c-6e35-4097-aa39-995e96c523e1
    response_code = 500
    uri = /deployment/blueprints/regenerate

I think this used to work so my guess is we added something to the planner that expects boundary NTP zones (reasonable) but never added these to the simulated blueprint?

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueIssues that are good for learning the codebase

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions