resolve merge conflict

Author: nonsense

.dockerignore

@@ -0,0 +1,10 @@
+docker/devnet/data
+react/node_modules
+# filecoin-ffi installation
+build/.filecoin-install
+extern/filecoin-ffi/.install-filcrypto
+extern/filecoin-ffi/filcrypto.*
+extern/filecoin-ffi/libfilcrypto.a
+# local binaries
+boost*
+devnet

README.md

@@ -4,79 +4,290 @@ Boost is a tool for Filecoin storage providers to manage data storage and retrie
 
 See the docs at [https://boost.filecoin.io](https://boost.filecoin.io/getting-started) to get started.
 
-## For development:
+## Table of Contents
 
+- [Building and Installing Boost](#building-and-installing-boost)
+- [Running Boost for development](#running-boost-for-development)
+- [Running Boost devnet in Docker](#running-boost-devnet-in-docker)
+- [License](#license)
 
-1. Install using instructions in the building and installation section in [the docs](https://boost.filecoin.io/getting-started#building-and-installing).
+## Building and Installing Boost
 
-2. Make sure you have a local Lotus fullnode and miner running and listening to `localhost:1234` and `localhost:2345` respectively, for example with a devnet:
+Compile and install using the instructions at the `Building and installing` section in [the docs](https://boost.filecoin.io/getting-started#building-and-installing).
 
+## Running Boost for development
+
+To run Boost on your development machine, you will need to set up a devnet:
+
+1. Remove any existing Lotus and Boost repositories
 ```
-devnet
+rm -rf ~/.lotusmarkets ~/.lotus ~/.lotusminer ~/.genesis_sectors
+rm -rf ~/.boost
 ```
 
-Note that currently `devnet` is using the default paths that `lotus` and `lotus-miner` use for their repositories, and you should make sure these directories are empty:
+2. Build Lotus in debug mode
 
+The version of lotus needs to match the version in Boost's go.mod
 ```
-LOTUS_PATH=~/.lotus
-LOTUS_MINER_PATH=~/.lotusminer
+cd lotus
+git checkout <tag>
+make debug
+```
+
+3. Install Lotus
 
-rm -rf ~/.lotus ~/.lotusminer
+The devnet script uses the installed `lotus` and `lotus-miner` binaries to run the miner and daemon.
+```
+make install
+install -C ./lotus-seed /usr/local/bin/lotus-seed
 ```
 
+4. Build Boost in debug mode
 
-3. Create Boost repository
+Double check if environment variables are set:
+```
+export LIBRARY_PATH=/opt/homebrew/lib
+export PATH="$(brew --prefix coreutils)/libexec/gnubin:/usr/local/bin:$PATH"
+```
 
+Build and install
 ```
-export $(lotus auth api-info --perm=admin)
-export $(lotus-miner auth api-info --perm=admin)
+cd boost
+make debug
+make install
+```
+
+5. Start the devnet
 
-boostd --vv init \
-       --api-sealer=`lotus-miner auth api-info --perm=admin` \
-       --api-sector-index=`lotus-miner auth api-info --perm=admin` \
-       --wallet-publish-storage-deals=`lotus wallet new bls` \
-       --wallet-deal-collateral=`lotus wallet new bls` \
-       --max-staging-deals-bytes=50000000000
+The following command will use the binaries that you built and installed above, and will run `lotus`, `lotus-miner` and `lotus-seed`. The `lotus` version must match the version in Boost's go.mod.
+```
+cd boost
+./devnet
 ```
 
-4. Run the Boost daemon service
+The first time you run it, it will download the Filecoin proof parameters. It will take at least 10 minutes depending on your connection speed. You may need to restart the command multiple times as your terminal will probably timeout before it finishes downloading everything.
 
+The devnet isn't designed to be restartable. After it has been successfully run once, you'll have to clear out the previous data before re-running `./devnet`:
+```
+rm -rf ~/.lotusmarkets && rm -rf ~/.lotus && rm -rf ~/.lotusminer && rm -rf ~/.genesis_sectors
 ```
-export $(lotus auth api-info --perm=admin)
 
-boostd --vv run
+6. Wait for `lotus-miner` to come up (through the command above)
+
+Unset these variables as they interfere with the `lotus-miner` command.
+```
+unset MINER_API_INFO
+unset FULLNODE_API_INFO
 ```
 
-5. Interact with Boost
+Then repeatedly run this command until it succeeds:
+```
+lotus-miner auth api-info --perm=admin
+```
 
-Pass the client address (wallet) and the provider address to the `dummydeal` command.
-Note that
-- the client address is the address of a wallet with funds in `lotus wallet list`
-- you can find the provider address in `~/.boost/config.toml` under the config key `Wallets.Miner`
+7. Get the authentication tokens to connect to the lotus daemon and miner:
 
 ```
-boostd dummydeal <client address> <provider address>
+export ENV_MINER_API_INFO=`lotus-miner auth api-info --perm=admin`
+export ENV_FULLNODE_API_INFO=`lotus auth api-info --perm=admin`
+
+export MINER_API_INFO=`echo $ENV_MINER_API_INFO | awk '{split($0,a,"="); print a[2]}'`
+export FULLNODE_API_INFO=`echo $ENV_FULLNODE_API_INFO | awk '{split($0,a,"="); print a[2]}'`
+
+echo MINER_API_INFO=$MINER_API_INFO
+echo FULLNODE_API_INFO=$FULLNODE_API_INFO
 ```
 
-## Running the UI in Development Mode:
+8. Create the wallets needed for Boost
 
-1. Run the server
+```
+export DEFAULT_WALLET=`lotus wallet list | tail -1 | awk '{print $1}'`
+export COLLAT_WALLET=`lotus wallet new bls`
+export PUBMSG_WALLET=`lotus wallet new bls`
+export CLIENT_WALLET=`lotus wallet new bls`
+```
+
+9. Add funds to the wallets
 
 ```
-cd react
-npm install
-npm start
+lotus send --from $DEFAULT_WALLET $COLLAT_WALLET 10
+lotus send --from $DEFAULT_WALLET $PUBMSG_WALLET 10
+lotus send --from $DEFAULT_WALLET $CLIENT_WALLET 10
 ```
 
-2. Open UI
+Run this command repeatedly until each wallet you created has 10 FIL:
+```
+lotus wallet list
+```
+
+This should take about 10 seconds.
+
+10. Set the Publish Message Wallet as a control address on the miner
 
 ```
-http://localhost:3000
+lotus-miner actor control set --really-do-it $PUBMSG_WALLET
 ```
 
-## Running a devnet:
+11. Add funds into the Market Actor escrow for the client and Collateral wallets
+
+```
+lotus wallet market add --from $DEFAULT_WALLET --address $CLIENT_WALLET 5
+lotus wallet market add --address $COLLAT_WALLET 5
+```
 
-Follow the instructions in the [devnet guide](./documentation/devnet.md)
+12. Initialize Boost / Create Boost repository
+
+```
+boostd -vv init \
+  --api-sealer=$MINER_API_INFO \
+  --api-sector-index=$MINER_API_INFO \
+  --wallet-publish-storage-deals=$PUBMSG_WALLET \
+  --wallet-deal-collateral=$COLLAT_WALLET \
+  --max-staging-deals-bytes=2000000000
+```
+
+13. Build the Web UI
+```
+make react
+```
+
+14. Edit config to set a fixed listen address
+
+Edit `~/.boost/config.toml`
+
+Set the port in the `ListenAddresses` key to `50000`
+```
+[Libp2p]
+  ListenAddresses = ["/ip4/0.0.0.0/tcp/50000", "/ip6/::/tcp/0"]
+```
+
+15. Run Boost
+```
+boostd -vv run
+```
+
+Note the peer ID of the boost instance:
+```
+2022-06-10T09:32:28.819Z        INFO    boostd  boostd/run.go:114       Boost libp2p node listening     {"maddr": "{12D3KooWQNNWNiJ1mieEk9EHjDVF2qBc1FSjJGEzwjnMJzteApaW: [/ip4/172.17.0.2/tcp/50000 /ip4/127.0.0.1/tcp/50000]}"}
+```
+In this example: `12D3KooWQNNWNiJ1mieEk9EHjDVF2qBc1FSjJGEzwjnMJzteApaW`
+
+14. Set the peer ID and multi-address of the miner on chain
+```
+lotus-miner actor set-peer-id <peer id>
+lotus-miner actor set-addrs /ip4/127.0.0.1/tcp/50000
+```
+
+16. Open the Web UI
+
+Open http://localhost:8080 to see the Boost UI
+
+### Make a deal with Boost
+
+1. Initialize the Boost client
+```
+boost init
+```
+
+This will output the address of the wallet (it's safe to run the init command repeatedly).
+
+2. Send funds to the client wallet
+```
+lotus send --from=$DEFAULT_WALLET <client wallet> 10
+```
+
+3. Follow the guide at https://boost.filecoin.io/tutorials/how-to-store-files-with-boost-on-filecoin
+
+Note that above you already ran a command to export FULLNODE_API (and point it to your local devnet lotus daemon).
+
+Note also that the provider address is `t01000` and you will need to supply an appropriate `--storage-price` when using `boost deal` since the devnet has a minimum price. Alternatively, using "Settings" in the Boost web UI to set the deal price to zero.
+
+## Running Boost devnet in Docker
+
+### Building Docker images
+
+1. Select Lotus version, for example: `lotus_version=1.17.1-rc2`. It must be the tag name of [the Lotus git repo](https://github.com/filecoin-project/lotus/tags) without `v` prefix.
+
+2. Select Boost version, for example: `boost_version=1.3.0-rc1`.
+
+3. Build images
+
+```
+cd docker/devnet
+make build/all
+```
+
+If you need to build a different version, edit the `.env` file.
+
+### Start devnet docker stack
+
+1. Run
+
+```
+cd docker/devnet
+docker compose up -d
+```
+
+It will spin up `lotus`, `lotus-miner`, `boost`, `booster-http` and `demo-http-server` containers. All temporary data will be saved in `./data` folder.
+
+The initial setup could take up to 20 min or more as it needs to download Filecoin proof parameters. During the initial setup, it is normal to see error messages in the log. Containers are waiting for the lotus to be ready. It may timeout several times. Restart is expected to be managed by `docker`.
+
+2. Try opening the Boost GUI http://localhost:8080 . Devnet is ready to operate when the URL opens and indicates no errors on the startup page.
+
+You can inspect the status using `docker compose logs -f`.
+
+### Start monitoring docker stack
+
+```
+docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
+
+cd docker/monitoring
+docker compose up -d
+```
+
+### Connect monitoring stack to devnet stack
+
+```
+docker network connect devnet tempo
+```
+
+### Explore Grafana / Tempo and search for traces
+
+http://localhost:3333
+
+### Making a deal
+
+The `boost` container is packed with `boost` and `lotus` clients. You can connect to the container with the command `docker compose exec boost /bin/bash` and follow instructions for [storing files with Boost guide](https://boost.filecoin.io/tutorials/how-to-store-files-with-boost-on-filecoin). But the recommended startup is to follow the semi-interactive demo first:
+
+```
+# Attach to a running boost container
+docker compose exec boost /bin/bash
+
+# Execute the demo script /app/sample/make-a-deal.sh
+root@83260455bbd2:/app# ./sample/make-a-deal.sh
+```
+
+### Accessing Lotus from localhost
+
+By default the [docker-compose.yaml](./docker-compose.yaml) does not expose any port of the `lotus` container. To access the `lotus` from a local machine:
+1. You can either expose `1234` in [docker-compose.yaml](./docker-compose.yaml) or find the IP of the `lotus` container using `docker inspect lotus | grep IPAddress` command.
+2. Get the `FULLNODE_API_INFO`
+```
+docker exec -it lotus lotus auth api-info --perm=admin
+FULLNODE_API_INFO=eyJ...ms4:/dns/lotus/tcp/1234/http
+```
+3. Change the `dns/lotus/tcp/1234/http` to `ip4/<127.0.0.1 or container's IP>/tcp/1234/http` for the use in `FULLNODE_API_INFO`.
+
+### Cleaning up
+
+To stop containers and drop everything:
+```
+docker compose down --rmi local
+
+rm -rf ./data
+
+rm -rf /var/tmp/filecoin-proof-parameters
+```
 
 ## License
 

build/version.go

@@ -2,7 +2,7 @@ package build
 
 var CurrentCommit string
 
-const BuildVersion = "1.4.0-rc1"
+const BuildVersion = "1.4.0-rc2"
 
 func UserVersion() string {
 	return BuildVersion + CurrentCommit