Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamline cluster configuration #292

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

tjsturos
Copy link
Contributor

There are two parts to this.

Starting control process with minimal config for clusters

The aim is that a Node Runner can add to their cluster by the following config changes:

...
engine:
  ...
  cluster:
    enabled: true
    machineConfigs:
      - ipAddress: 127.0.0.1 # required
        dataWorkerProcessesCount: 127 # required
        dataWorkerBaseListenPort: 40100 # default = 40000
    - ipAddress: 192.168.0.110
      dataWorkerProcessesCount: 128
   # ... etc.

this will tell the node's control process to create clients for addresses /ip4//tcp/<dataWorkerBaseListenPort + index>

So, the above config will create clients for:

  • 127.0.0.1:40100-40227
  • 192.168.0.110:40000-40128

Starting a data worker with no config

To start workers, a script just needs to start the data-workers:

For 127.0.0.1 config

~/ceremonyclient/node/node-1.4.21.1-amd-linux --core 1 --core-base-listen-port 40100

For 182.168.0.110

~/ceremonyclient/node/node-1.4.21.1-amd-linux --core 1 // defaults listen port to 40000

This could be scripted, such that:

DEFAULT_CORES=$(nproc)
## could find if there are already 'node' processes running and subtract, but for simplicities sake
DATA_WORKER_COUNT="${1:-$DEFAULT_CORES}"
BASE_LISTEN_PORT=${2:-40000}

CORE=1
while CORE <= $DATA_WORKER_COUNT; do
  ~/ceremonyclient/node/node-1.4.21.1-amd-linux --core $CORE --core-base-listen-port $BASE_LISTEN_PORT
   CORE=$CORE + 1
done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant