feat: deploy Agentic Observatory

[Svaag]

Summary

• add loop-host Agentic Observatory role, systemd service, Vault template/policy, and bootstrap docs
• expose observatory.servify.network through Caddy and allow proxy/mon firewall access to loop:8780
• add health monitoring plus app pin updates for enriched trace producers and the observatory service

Validation

• uv run pytest tests/iac/test_vault_and_runner_contracts.py tests/iac/test_app_version_pins.py -q
• uv run --with jinja2 pytest tests/iac -q
• ansible-playbook playbooks/engineering-loop.yml --tags validate --connection=local --limit loop --skip-tags=snapshot --syntax-check
• ansible-playbook playbooks/engineering-loop.yml --tags validate --connection=local --limit loop --skip-tags=snapshot

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🏅 Score: 85

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Missing render-check gate The generated icinga_host.conf and nftables.conf are manually updated in this PR but there is no automated CI check (render-check) that validates these generated files remain in sync with their Jinja2 templates. Future template changes could silently produce drift unless regenerated manually. The PR should either add a render-check step to the workflow that compares generated output against ansible/generated/ or document the regeneration procedure as a required CI gate. object Service "agentic-observatory-health" { host_name = "loop" check_command = "http" check_interval = 1m retry_interval = 30s max_check_attempts = 3 notes = "Agentic Observatory internal operator console health endpoint" vars.http_address = "2a0c:b641:b50:2::f0" vars.http_port = 8780 vars.http_uri = "/healthz" vars.http_vhost = "observatory.servify.network" vars.http_ssl = false vars.http_ipv6 = true vars.http_timeout = 5 vars.http_expect_body_regex = "\"status\":\"ok\"" } Missing render-check gate Same as above: the new nftables rule on line 41 is manually committed without an automated render-check. Without a CI step that validates the generated rules match the template, inadvertent template edits could break the firewall configuration. ip6 saddr { 2a0c:b641:b50:2::40, 2a0c:b641:b50:2::50 } tcp dport 8780 counter accept comment "agentic observatory"

[github-actions]

PR Code Suggestions ✨

No code suggestions found for the PR.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 92a849343e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a644db5fe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Svaag]

CI coverage note: the PR includes generated ansible/generated/loop/{icinga_host.conf,nftables.conf} updates from the validation render, and the branch now has passing render, static-iac, ansible-idempotency, and iac-gate checks on head 99c8ea0. No additional render-check workflow change was needed for this deployment PR.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99c8ea0699

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 20e8bc463d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 24fe20dd35

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".