Skip to content

Streamr 1.0 #219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,9 +205,9 @@ streamr-docker-dev start --wait
## Included services

### Streamr services
- 3 x [Broker](https://github.com/streamr-dev/broker) nodes
- 3 x [Broker](https://github.com/streamr-dev/network) nodes
- 2 broker nodes + 1 storage Streamr network nodes. This creates a local and private Streamr Network.
- 3 x [Tracker](https://github.com/streamr-dev/broker)
- 1 x [Entry point](https://github.com/streamr-dev/network)
- Helps node discovery in the Strearm Network
- 1 x [Hub frontend](https://github.com/streamr-dev/streamr-platform/app)
- See more detailed build instructions in the streamr-platform repo
Expand Down
24 changes: 0 additions & 24 deletions docker-compose-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,30 +33,6 @@ services:
memory: 500M
reservations:
memory: 6M
tracker-1:
deploy:
resources:
limits:
cpus: '2.0'
memory: 200M
reservations:
memory: 150M
tracker-2:
deploy:
resources:
limits:
cpus: '2.0'
memory: 200M
reservations:
memory: 150M
tracker-3:
deploy:
resources:
limits:
cpus: '2.0'
memory: 200M
reservations:
memory: 150M
broker-node-storage-1:
deploy:
resources:
Expand Down
132 changes: 28 additions & 104 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,84 +98,11 @@ services:
interval: 5s
timeout: 10s
retries: 10
tracker-1:
container_name: streamr-dev-tracker-1
image: streamr/tracker:dev
platform: linux/x86_64
init: true
networks:
- streamr-network
restart: unless-stopped
ports:
- "30301:30301"
volumes:
- type: bind
source: ./certs
target: /certs
read_only: true
bind:
propagation: rprivate
command: npm exec -- streamr-tracker 0xe5abc5ee43b8830e7b0f98d03efff5d6cae574d52a43204528eab7b52cd6408d tracker-1 --port=30301 ${TRACKER_ARGS}
healthcheck:
test: ["CMD", "curl", "--fail", "--silent", "--show-error", "--max-time", "9", "--insecure", "${STREAMR_BASE_URL}:30301/topology/"]
interval: 30s
timeout: 10s
retries: 20
tracker-2:
container_name: streamr-dev-tracker-2
image: streamr/tracker:dev
platform: linux/x86_64
init: true
networks:
- streamr-network
restart: unless-stopped
ports:
- "30302:30302"
volumes:
- type: bind
source: ./certs
target: /certs
read_only: true
bind:
propagation: rprivate
command: npm exec -- streamr-tracker 0x96de9d06f9e409119a2cd9b57dfc326f66d953a0418f3937b92c8930f930893c tracker-2 --port=30302 ${TRACKER_ARGS}
healthcheck:
test: ["CMD", "curl", "--fail", "--silent", "--show-error", "--max-time", "9", "--insecure", "${STREAMR_BASE_URL}:30302/topology/"]
interval: 30s
timeout: 10s
retries: 20
tracker-3:
container_name: streamr-dev-tracker-3
image: streamr/tracker:dev
platform: linux/x86_64
init: true
networks:
- streamr-network
restart: unless-stopped
ports:
- "30303:30303"
volumes:
- type: bind
source: ./certs
target: /certs
read_only: true
bind:
propagation: rprivate
command: npm exec -- streamr-tracker 0x6117b7a7cb8f3c8d40e3b7e87823c11af7f401515bc4fdf2bfdda70f1b833027 tracker-3 --port=30303 ${TRACKER_ARGS}
healthcheck:
test: ["CMD", "curl", "--fail", "--silent", "--show-error", "--max-time", "9", "--insecure", "${STREAMR_BASE_URL}:30303/topology/"]
interval: 30s
timeout: 10s
retries: 20
entry-point:
# NOTE THIS IS NOT USABLE YET as streamr/broker-node:dev images are in brubeck era, not in streamr-1.0 era
# - we currently publish images only from main, not from streamr-1.0
# - i.e. in the current streamr/broker-node:dev image there is no "entry-point" script at all
# TODO could move entry-point.ts from broker/bin to dht/bin then create a dht docker image and then use it here
# - also remove "entry-point" from broker's package.json bin script list
# TODO add this as a dependency for all brokers
container_name: streamr-dev-entry-point
image: streamr/broker-node:dev
image: streamr/broker-node:dev-tatum
init: true
networks:
- streamr-network
Expand All @@ -190,7 +117,7 @@ services:
retries: 10
broker-node-storage-1:
container_name: streamr-dev-broker-node-storage-1
image: streamr/broker-node:dev
image: streamr/broker-node:dev-tatum
init: true
networks:
- streamr-network
Expand All @@ -203,9 +130,9 @@ services:
depends_on:
- init-keyspace
- cassandra
- tracker-1
- tracker-2
- tracker-3
- entry-point
- dev-chain-fast
- deploy-network-subgraphs-fastchain
environment:
STREAMR_URL: "${STREAMR_BASE_URL}"
CASSANDRA_HOST: 10.200.10.1:9042
Expand All @@ -217,7 +144,7 @@ services:
retries: 20
broker-node-no-storage-1:
container_name: streamr-dev-broker-node-no-storage-1
image: streamr/broker-node:dev
image: streamr/broker-node:dev-tatum
init: true
networks:
- streamr-network
Expand All @@ -228,9 +155,9 @@ services:
- "9100:9100"
- "30316:30316"
depends_on:
- tracker-1
- tracker-2
- tracker-3
- entry-point
- dev-chain-fast
- deploy-network-subgraphs-fastchain
environment:
STREAMR_URL: "${STREAMR_BASE_URL}"
command: npm exec -- streamr-broker configs/docker-2.env.json
Expand All @@ -241,7 +168,7 @@ services:
retries: 20
broker-node-no-storage-2:
container_name: streamr-dev-broker-node-no-storage-2
image: streamr/broker-node:dev
image: streamr/broker-node:dev-tatum
init: true
networks:
- streamr-network
Expand All @@ -252,9 +179,9 @@ services:
- "9200:9200"
- "30317:30317"
depends_on:
- tracker-1
- tracker-2
- tracker-3
- entry-point
- dev-chain-fast
- deploy-network-subgraphs-fastchain
environment:
STREAMR_URL: "${STREAMR_BASE_URL}"
command: npm exec -- streamr-broker configs/docker-3.env.json
Expand Down Expand Up @@ -287,9 +214,7 @@ services:
ports:
- "3334:80"
depends_on:
- tracker-1
- tracker-2
- tracker-3
- entry-point
healthcheck:
test: ["CMD", "curl", "--fail", "--silent", "--show-error", "--max-time", "9", "http://localhost"]
interval: 10s
Expand Down Expand Up @@ -493,7 +418,7 @@ services:
retries: 3
graph-node-fastchain:
container_name: streamr-dev-thegraph-node-fastchain
image: graphprotocol/graph-node:v0.30.0
image: graphprotocol/graph-node:v0.32.0
restart: unless-stopped
networks:
- streamr-network
Expand Down Expand Up @@ -585,20 +510,19 @@ services:
interval: 5s
timeout: 10s
retries: 10
stream-metrics-index:
container_name: streamr-dev-stream-metrics-index
image: streamr/stream-metrics-index
networks:
- streamr-network
restart: unless-stopped
ports:
- "4001:4001"
depends_on:
- mysql
- graph-deploy-streamregistry-subgraph
- tracker-1
- tracker-2
- tracker-3
# TODO enable when upstream-metrics-index application updated to use streamr-1.0 network
# stream-metrics-index:
# container_name: streamr-dev-stream-metrics-index
# image: streamr/stream-metrics-index
# networks:
# - streamr-network
# restart: unless-stopped
# ports:
# - "4001:4001"
# depends_on:
# - mysql
# - graph-deploy-streamregistry-subgraph
# - entry-point
ens-sync-script:
container_name: ens-sync-script
image: streamr/ens-sync-script:dev
Expand Down
4 changes: 1 addition & 3 deletions streamr-docker-dev/bin.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,8 @@ EXCEPT_SERVICES_DEFAULT=() # array of string e.g. ("a" "b")
NODE_NO_STORAGE='broker-node-no-storage-1 broker-node-no-storage-2'
NODE_STORAGE='broker-node-storage-1'
NODES="$NODE_NO_STORAGE $NODE_STORAGE"
TRACKERS='tracker-1 tracker-2 tracker-3'

# swap aliases for full names e.g. trackers = tracker-1 tracker-2 tracker-3
# swap aliases for full names
# feel free to add more, just make sure you don't end up using actual service
# names, or part-thereof as alias names
expandServiceAliases() {
Expand All @@ -40,7 +39,6 @@ expandServiceAliases() {
names="${names//storage-nodes/$NODE_STORAGE}"
names="${names//brokers/$NODES}"
names="${names//nodes/$NODES}" # brokers/nodes sort of interchangeable
names="${names//trackers/$TRACKERS}"
echo "$names"
}

Expand Down
26 changes: 13 additions & 13 deletions streamr-docker-dev/help_scripts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ Commands:
not just Streamr-related ones!)
Examples:
streamr-docker-dev start
streamr-docker-dev stop tracker-1
streamr-docker-dev start --except tracker-1 --wait
streamr-docker-dev log -f tracker-1
streamr-docker-dev shell tracker-1
streamr-docker-dev stop entry-point
streamr-docker-dev start --except entry-point --wait
streamr-docker-dev log -f entry-point
streamr-docker-dev shell entry-point
streamr-docker-dev pull

Options:
Expand All @@ -47,8 +47,8 @@ Usage: streamr-docker-dev start [--] <service>...

Examples:
streamr-docker-dev start
streamr-docker-dev start tracker-1 parity-node0
streamr-docker-dev start --except tracker-1 --wait
streamr-docker-dev start entry-point parity-node0
streamr-docker-dev start --except entry-point --wait

Options:
--except [service] start services except the one given
Expand All @@ -65,7 +65,7 @@ Usage: streamr-docker-dev stop [options] [--] <service>...

Examples:
streamr-docker-dev stop
streamr-docker-dev stop tracker-1 parity-node0
streamr-docker-dev stop entry-point parity-node0
"
}

Expand All @@ -77,7 +77,7 @@ Usage: streamr-docker-dev restart [options] [--] <service>...

Examples:
streamr-docker-dev restart
streamr-docker-dev restart tracker-1 parity-node0
streamr-docker-dev restart entry-point parity-node0
"
}

Expand All @@ -104,7 +104,7 @@ Usage: streamr-docker-dev ps [<service>...]

Examples:
streamr-docker-dev ps
streamr-docker-dev ps tracker-1
streamr-docker-dev ps entry-point
"
}

Expand All @@ -116,8 +116,8 @@ Usage: streamr-docker-dev log [[options] [--] <service>...]

Examples:
streamr-docker-dev log
streamr-docker-dev log tracker-1 parity-node0
streamr-docker-dev log -f tracker-1 parity-node0
streamr-docker-dev log entry-point parity-node0
streamr-docker-dev log -f entry-point parity-node0

Options:
-f --follow follow log in realtime
Expand All @@ -131,7 +131,7 @@ Opens an interactive shell into the target container.
Usage: streamr-docker-dev shell <service>

Examples:
streamr-docker-dev shell tracker-1
streamr-docker-dev shell entry-point
"
}

Expand All @@ -143,7 +143,7 @@ Usage: streamr-docker-dev pull [ [--] <service>...]

Examples:
streamr-docker-dev pull
streamr-docker-dev pull tracker-1 parity-node0
streamr-docker-dev pull entry-point parity-node0
"
}

Expand Down