Spanner CDC connection using Debezium - unable to dial: dial tcp 127.0.0.1:9093: connect: connection refused

[abhijitpratinidhi]

Version & Environment

Redpanda version: (use rpk version):rpk version v24.1.9

Please also give versions of other components:

• Operating System (e.g. from /etc/os-release): MAC ARM64
• Any Kafka client libraries in use (e.g. rdkafka, franz-go) - Kafka connect
• Docker (use docker info): Yes
• Kubernetes (use kubectl version): No

Description - Using the dockerfile(given below) to spin up a redpanda with kafka connect for subscribing events out of a spanner CDC.Using a debezium connector to connect to spanner CDC to poll messages and publish to a redpanda topic.

Issue:Once the redpanda and kafka connect are running,the rpk commands are not working.They give a 'connection refused' as shown below ->
Command - rpk topic list
Error - unable to request metadata: unable to dial: dial tcp 127.0.0.1:9093: connect: connection refused
Tried changing advertise-kafka-addr PLAINTEXT://redpanda:9092 to advertise-kafka-addr PLAINTEXT://machine -ip-address:9092 and few other stuffs but didnt work.Any help please?

Dockerfile -

version: '3.8'
services:
redpanda:
image: redpandadata/redpanda:latest
command: >
redpanda start
--overprovisioned
--smp 1
--memory 1G
--reserve-memory 0M
--node-id 0
--check=false
--kafka-addr PLAINTEXT://0.0.0.0:9092
--advertise-kafka-addr PLAINTEXT://redpanda:9092
ports:
- "9094:9094"
- "9644:9644"
- "9092:9092"
networks:
- redpanda-network
kafka-connect:
image: debezium/connect:2.7.1.Final
environment:
- BOOTSTRAP_SERVERS=redpanda:9092
- CONNECT_BOOTSTRAP_SERVERS=redpanda:9092
- GROUP_ID=1
- CONFIG_STORAGE_TOPIC=my-configs
- OFFSET_STORAGE_TOPIC=my-offsets
- STATUS_STORAGE_TOPIC=my-status
- CONNECT_PLUGIN_PATH=/kafka/connect
- CLASSPATH=/kafka/connect/*:$CLASSPATH
ports:
- "8083:8083"
volumes:
- ./connectors:/kafka/connect
depends_on:
- redpanda
networks:
- redpanda-network

networks:
redpanda-network:
driver: bridge

JIRA Link: CORE-7047