Skip to content

Foremanctl networks#6

Open
pablomh wants to merge 12 commits intomasterfrom
foremanctl_networks
Open

Foremanctl networks#6
pablomh wants to merge 12 commits intomasterfrom
foremanctl_networks

Conversation

@pablomh
Copy link
Owner

@pablomh pablomh commented Mar 4, 2026

No description provided.

pablomh and others added 4 commits March 4, 2026 16:07
Introduces deploy_network as the canonical way to create networks,
with a Podman implementation mapping directly to the podman_network
Ansible module.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace network: host on all containers with dedicated bridge networks:
- foreman-db (internal, isolated): postgresql and all DB clients
- foreman-cache (internal, isolated): redis and all cache/queue clients
- foreman-app: candlepin, foreman, pulp-*, dynflow workers (internet access
  retained for content sync)
- foreman-proxy-net: foreman and foreman-proxy

Update all localhost connection strings to use container DNS names
(postgresql, redis, candlepin). Publish only ports required by the host
httpd process (127.0.0.1:3000, :24816, :24817) and external smart proxy
clients (0.0.0.0:8443). PostgreSQL retains 127.0.0.1:5432 for Ansible
setup tasks. Redis no longer publishes 6379 externally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Expose Candlepin (23443) and Artemis STOMP (61613) on 127.0.0.1 so
that host-side tooling and tests can reach them directly. Both ports
are bound to loopback only, so they remain inaccessible from outside
the host.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Redis no longer publishes port 6379 to the host since it is an
internal-only service. Verify it is reachable by running redis-cli
ping inside the container.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@pablomh pablomh force-pushed the foremanctl_networks branch from 64f5ba6 to c53663b Compare March 4, 2026 15:08
pablomh and others added 6 commits March 4, 2026 16:09
Adds a "Container Networking" section to the deployment docs covering
the four named Podman bridge networks, the rationale for internal and
isolate flags, container membership per network, and the ports
published to the host.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The postgresql role now defaults to postgresql_network: foreman-db, but
the foreman-db network is only created in deploy.yaml (targeting quadlet).
On the external database VM, the network never exists, so the PostgreSQL
container fails to attach to it and systemctl reports a control process
failure.

For the external database, host networking is correct: the VM is
standalone (no bridge networks are set up there), and PostgreSQL must
bind to all interfaces so the quadlet host can reach it over the
network. The loopback-only port mapping used on the quadlet VM would
block that cross-host access.

- Extract postgresql_ports as a variable (default: 127.0.0.1:5432:5432)
- Use postgresql_ports in the quadlet task instead of hardcoded value
- Override to network: host and postgresql_ports: [] in remote-database

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
One-shot migration containers (foreman-db-migrate, pulpcore-manager-migrate,
pulpcore-manager-admin-password) were only attached to the foreman-db network,
which is internal and isolated. When database_mode is external, the database
host is a remote IP unreachable from that network.

Add a *_migration_networks variable to the foreman and pulp roles that includes
foreman-app when database_mode is external, giving the containers outbound
routing to reach the external database host.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
deploy-dev.yaml was missing the deploy_network roles that deploy.yaml
runs before postgresql. Without the foreman-db network pre-created,
the postgresql container restart (triggered by the scram-sha-256
handler at end of play) fails because Podman cannot attach to a
non-existent network.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Candlepin's network list was hardcoded to [foreman-db, foreman-app].
In external database mode, foreman-db is an internal+isolated bridge
with no PostgreSQL container, so attaching Candlepin to it serves no
purpose and may interfere with outbound routing to the external DB host.

Introduce candlepin_networks that follows the same pattern as
foreman_migration_networks: include foreman-db only in internal mode
(to reach the local PostgreSQL container), always include foreman-app
for communication with Foreman and the external database.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
With bridge networks, Podman port mapping uses DNAT to the container's
bridge IP. If Tomcat binds to address="localhost" (127.0.0.1 only),
neither port mappings from the host nor container-to-container
connections via the bridge IP can reach it -- even though the container
itself is healthy (the healthcheck runs inside and hits loopback).

Bind Tomcat and Artemis STOMP to 0.0.0.0 so all container interfaces
are covered. Security is provided by the bridge network isolation
(foreman-app is not exposed externally), not by the listener address.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@pablomh pablomh force-pushed the foremanctl_networks branch from 836e3ec to 659d740 Compare March 4, 2026 23:03
The foreman-db network uses isolate: true which adds iptables rules
blocking all host-to-container traffic, including DNAT'd port-mapped
connections. The community.postgresql.* modules use login_host: localhost
(TCP to 127.0.0.1:5432) which hangs indefinitely as packets are dropped.

Replace with podman exec --user postgres to run psql/createdb inside the
container via Unix socket peer auth, avoiding TCP entirely. Proper
idempotency is handled by checking the current value before changing it
(password_encryption) and tolerating "already exists" errors for users
and databases.

A flush_handlers is added between the scram-sha-256 change and user
creation to ensure PostgreSQL restarts with the new encryption setting
before passwords are stored.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@pablomh pablomh force-pushed the foremanctl_networks branch from 77706ec to ca705f2 Compare March 4, 2026 23:42
With network: host, containers shared the host network namespace so
localhost inside any container correctly reached PostgreSQL on the
host loopback. With bridge networks, each container has its own
loopback, making localhost unreachable for cross-container connections.

Use the PostgreSQL container name as the database host so that services
on the foreman-db bridge network can resolve it via Podman DNS.

For external database deployments, database_host is passed explicitly
at deploy time and is unaffected by this change.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant