Multi pool support

README.md

@@ -1,6 +1,6 @@
 # Auctioneer Bot
 
-The auctioneer bot monitors a Blend pool to create and bid on auctions. This includes user liquidation auctions, bad debt auctions, and interest auctions. The auctioneer focuses on completeness and pool safety over profit, but the existing code can be modified to meet any use case.
+The auctioneer bot monitors Blend V1 pools to create and bid on auctions. This includes user liquidation auctions, bad debt auctions, and interest auctions. The auctioneer focuses on completeness and pool safety over profit, but the existing code can be modified to meet any use case.
 
 For more information please see [the Blend docs](https://docs.blend.capital/tech-docs/core-contracts/lending-pool).
 
@@ -33,6 +33,8 @@ Auctions filled by the bot will have an entry populated in the `filled_auctions`
 
 ## Important Info
 
+This version only support V1 Blend pools.
+
 This bot does not automatically unwind all positions it bids on. It is recommended to manually adjust your positions as necessary as auctions get filled. 
 
 Certain auctions cause your filler to take on liabilities, and if these assets are not cleared in a timely manner, could result in the filler also getting liquidated.
@@ -41,19 +43,25 @@ Certain auctions cause your filler to take on liabilities, and if these assets a
 
 For an example config file that is configured to interact with [Blend v1 mainnet protocol](https://docs.blend.capital/), please see [example.config.json](https://github.com/script3/auctioneer-bot/blob/main/example.config.json).
 
+### Database Migration
+
+**It is highly recommended to create a backup of the database file before attempting any migration.**
+
+For auctioneers that were started before multi-pool functionality a db migration will be neccessary. On startup of the updated auctioneer bot a command line argument will be required to be inputed with `-p` or `--prev-pool-id` followed by the pool id that the single pool auctioneer bot was using. The migration will update the database to the new schema and will populate the pool id column with the one provided.
+
 #### General Settings
 
 | Field | Description |
 |-------|-------------|
 | `name` | A descriptive name for your bot instance. |
 | `rpcURL` | The URL of the Soroban RPC endpoint. |
 | `networkPassphrase` | The network passphrase identifying the Stellar network. |
-| `poolAddress` | The address of the Blend pool contract this bot will be tracking. |
 | `backstopAddress` | The address of the Blend backstop contract. |
 | `backstopTokenAddress` | The address of the Blend backstop token contract. |
 | `usdcAddress` | The address of the USDC token contract. |
 | `blndAddress` | The address of the BLND token contract. |
 | `keypair` | The secret key for the bot's auction creating account. This should be different from the fillers as auction creation and auction bidding can happen simultaneously. **Keep this secret and secure!** |
+| `pools` | A list of pool addresses that dicate what pools are monitored |
 | `fillers` | A list of accounts that will bid and fill on auctions. |
 | `priceSources` | (Optional) A list of assets that will have prices sourced from exchanges instead of the pool oracle. |
 | `profits` | (Optional) A list of auction profits to define different profit percentages used for matching auctions.
@@ -69,12 +77,21 @@ The `fillers` array contains configurations for individual filler accounts. The
 | `keypair` | The secret key for this filler account. **Keep this secret and secure!** |
 | `primaryAsset` | The primary asset the filler will use as collateral in the pool. | 
 | `defaultProfitPct` | The default profit percentage required for the filler to bid on an auction, as a decimal. (e.g. 0.08 = 8%) |
-| `minHealthFactor` | The minimum health factor the filler will take on during liquidation and bad debt auctions, as calculated by `collateral / liabilities`. |
-| `minPrimaryCollateral` | The minimum amount of the primary asset the Filler will maintain as collateral in the pool. |
-| `forceFill` | Boolean flag to indicate if the bot should force fill auctions even if profit expectations aren't met to ensure pool health. |
+| `supportedPools` | An array of configs that control what pools the filler can interact |
 | `supportedBid` | An array of asset addresses that this filler bot is allowed to bid with. Bids are taken as additional liabilities (dTokens) for liquidation and bad debt auctions, and tokens for interest auctions. Must include the `backstopTokenAddress` to bid on interest auctions. |
 | `supportedLot` | An array of asset addresses that this filler bot is allowed to receive. Lots are given as collateral (bTokens) for liquidation auctions and tokens for interest and bad debt auctions. The filler should have trustlines to all assets that are Stellar assets. Must include `backstopTokenAddress` to bid on bad debt auctions. |
 
+#### Pool Filler Configs
+The `PoolFillerConfig` array contains configurations for pools that are to be monitored.
+
+| Field | Description |
+|-------|-------------|
+| `poolAddress` | The address of the pool |
+| `primaryAsset` | The primary asset that will be used as collateral in the pool. | 
+| `minPrimaryCollateral` | The minimum amount of the primary asset that is maintained as collateral in the pool. |
+| `minHealthFactor` | The minimum health factor the filler will take on during liquidation and bad debt auctions, as calculated by `collateral / liabilities`. |
+| `forceFill` | Boolean flag to indicate if the bot should force fill auctions even if profit expectations aren't met to ensure pool health. |
+
 #### Price Sources
 
 The `priceSources` array defines the additional sources for price data. If an asset has a price source, the oracle prices will not be used when calculating profit, and instead the price fetched from the price source will be. 
@@ -133,3 +150,5 @@ The bot can also be run locally with node, but you will need to invoke `start.sh
 
 
 
+
+

init_db.sql → db/init_db.sql

@@ -6,13 +6,22 @@ CREATE TABLE IF NOT EXISTS status (
     latest_ledger INTEGER NOT NULL
 );
 
+-- Table to store the version of the database
+CREATE TABLE IF NOT EXISTS db_version (
+    version INTEGER PRIMARY KEY NOT NULL,
+    description TEXT NOT NULL,
+    applied_at INTEGER NOT NULL
+);
+
 -- Table to store the user's that have positions in the pool
 CREATE TABLE IF NOT EXISTS users (
-    user_id TEXT PRIMARY KEY NOT NULL,
+    pool_id TEXT NOT NULL,
+    user_id TEXT NOT NULL,
     health_factor REAL NOT NULL,
     collateral JSON NOT NULL,
     liabilities JSON,
-    updated INTEGER NOT NULL
+    updated INTEGER NOT NULL,
+    PRIMARY KEY (pool_id, user_id)
 );
 CREATE INDEX IF NOT EXISTS idx_health_factor ON users(health_factor);
 
@@ -25,18 +34,20 @@ CREATE TABLE IF NOT EXISTS prices (
 
 -- Table to store ongoing auctions
 CREATE TABLE IF NOT EXISTS auctions (
+    pool_id TEXT NOT NULL,
     user_id TEXT NOT NULL,
     auction_type INTEGER NOT NULL,
     filler TEXT NOT NULL,
     start_block INTEGER NOT NULL,
     fill_block INTEGER NOT NULL,
     updated INTEGER NOT NULL,
-    PRIMARY KEY (user_id, auction_type)
+    PRIMARY KEY (pool_id, user_id, auction_type)
 );
 
 -- Table to store filled auctions
 CREATE TABLE IF NOT EXISTS filled_auctions (
     tx_hash TEXT PRIMARY KEY,
+    pool_id TEXT NOT NULL,
     filler TEXT NOT NULL,
     user_id TEXT NOT NULL,
     auction_type INTEGER NOT NULL,
@@ -49,3 +60,4 @@ CREATE TABLE IF NOT EXISTS filled_auctions (
     timestamp INTEGER NOT NULL
 );
 CREATE INDEX IF NOT EXISTS idx_filler ON filled_auctions(filler);
+

db/migrate_db_v1_to_v2.sql

@@ -0,0 +1,152 @@
+-- Migration steps
+
+-- Migrate users table
+
+-- Create new table with pool_id
+CREATE TABLE users_new (
+    pool_id TEXT NOT NULL,
+    user_id TEXT NOT NULL,
+    health_factor REAL NOT NULL,
+    collateral JSON NOT NULL,
+    liabilities JSON,
+    updated INTEGER NOT NULL,
+    PRIMARY KEY (pool_id, user_id)
+);
+
+-- Copy data to new table with default pool_id
+INSERT INTO users_new (
+    pool_id, 
+    user_id, 
+    health_factor, 
+    collateral, 
+    liabilities, 
+    updated
+)
+SELECT 
+    'default_pool', 
+    user_id, 
+    health_factor, 
+    collateral, 
+    liabilities, 
+    updated 
+FROM users;
+
+-- Drop old table
+DROP TABLE users;
+
+-- Rename new table
+ALTER TABLE users_new RENAME TO users;
+
+-- Recreate indexes
+CREATE INDEX IF NOT EXISTS idx_health_factor ON users(health_factor);
+
+-- Migrate auctions table
+-- Check if pool_id column exists
+
+-- Create new table with pool_id
+CREATE TABLE auctions_new (
+    pool_id TEXT NOT NULL,
+    user_id TEXT NOT NULL,
+    auction_type INTEGER NOT NULL,
+    filler TEXT NOT NULL,
+    start_block INTEGER NOT NULL,
+    fill_block INTEGER NOT NULL,
+    updated INTEGER NOT NULL,
+    PRIMARY KEY (pool_id, user_id, auction_type)
+);
+
+-- Copy data to new table with default pool_id
+INSERT INTO auctions_new (
+    pool_id, 
+    user_id, 
+    auction_type, 
+    filler, 
+    start_block, 
+    fill_block, 
+    updated
+)
+SELECT 
+    'default_pool', 
+    user_id, 
+    auction_type, 
+    filler, 
+    start_block, 
+    fill_block, 
+    updated 
+FROM auctions;
+
+-- Drop old table
+DROP TABLE auctions;
+
+-- Rename new table
+ALTER TABLE auctions_new RENAME TO auctions;
+
+-- Migrate filled_auctions table
+-- Check if pool_id column exists
+
+-- Create new table with pool_id
+CREATE TABLE filled_auctions_new (
+    tx_hash TEXT PRIMARY KEY,
+    pool_id TEXT NOT NULL,
+    filler TEXT NOT NULL,
+    user_id TEXT NOT NULL,
+    auction_type INTEGER NOT NULL,
+    bid JSON NOT NULL,
+    bid_total REAL NOT NULL,
+    lot JSON NOT NULL,
+    lot_total REAL NOT NULL,
+    est_profit REAL NOT NULL,
+    fill_block INTEGER NOT NULL,
+    timestamp INTEGER NOT NULL
+);
+
+-- Copy data to new table with default pool_id
+INSERT INTO filled_auctions_new (
+    tx_hash, 
+    pool_id, 
+    filler, 
+    user_id, 
+    auction_type, 
+    bid, 
+    bid_total, 
+    lot, 
+    lot_total, 
+    est_profit, 
+    fill_block, 
+    timestamp
+)
+SELECT 
+    tx_hash, 
+    'default_pool', 
+    filler, 
+    user_id, 
+    auction_type, 
+    bid, 
+    bid_total, 
+    lot, 
+    lot_total, 
+    est_profit, 
+    fill_block, 
+    timestamp 
+FROM filled_auctions;
+
+-- Drop old table
+DROP TABLE filled_auctions;
+
+-- Rename new table
+ALTER TABLE filled_auctions_new RENAME TO filled_auctions;
+
+-- Recreate index
+CREATE INDEX IF NOT EXISTS idx_filler ON filled_auctions(filler);
+
+-- Update version information
+INSERT INTO db_version (
+    version, 
+    description, 
+    applied_at
+) VALUES (
+    2, 
+    'Add pool_id column to users, auctions, and filled_auctions tables', 
+    unixepoch()
+);
+

db/setup_db.sh

@@ -0,0 +1,119 @@
+#!/bin/bash
+
+# Set variables
+DB_PATH="./data/auctioneer.sqlite"
+V2_MIGRATION_SCRIPT="./db/migrate_db_v1_to_v2.sql"
+
+PREV_POOL_ID=""
+DOES_DB_EXIST=1
+CURRENT_DB_VERSION=2
+
+# Function to display usage information
+show_usage() {
+    echo "Usage: $0 [options]"
+    echo "Options:"
+    echo "  -h, --help                   Show this help message"
+    echo "  -p, --prev-pool-id ID        Previous pool ID (required for v2 migration)"
+}
+# Parse command line arguments
+while [[ $# -gt 0 ]]; do
+    case $1 in
+        -h|--help)
+            show_usage
+            exit 0
+            ;;
+        -p|--prev-pool-id)
+            if [ -z "$2" ]; then
+                shift
+            fi
+            PREV_POOL_ID="$2"
+            shift 2
+            ;;
+        *)
+            echo "Error: Unknown option $1"
+            show_usage
+            exit 1
+            ;;
+    esac
+done
+
+# Check if sqlite3 is installed
+if ! command -v sqlite3 &> /dev/null; then
+    echo "Error: sqlite3 is not installed. Please install SQLite3."
+    exit 1
+fi
+
+# Check if database file exists
+if ! test -f $DB_PATH; then
+    # Initialize the database
+    DOES_DB_EXIST=0
+fi
+
+# Check that all tables exist or create them
+sqlite3 ./data/auctioneer.sqlite < ./db/init_db.sql
+
+if [ "$DOES_DB_EXIST" -eq 0 ]; then
+    sqlite3 "$DB_PATH" "
+        INSERT INTO db_version (version, description, applied_at)
+        SELECT 2, 'init db', unixepoch();
+    " 
+    echo "Database initialized."
+fi
+
+# Check if migration script exists
+if [ ! -f "$V2_MIGRATION_SCRIPT" ]; then
+    echo "Error: Migration script $V2_MIGRATION_SCRIPT does not exist."
+    exit 1
+fi
+
+# Function to get current database version
+get_current_version() {
+    sqlite3 "$DB_PATH" "
+        CREATE TABLE IF NOT EXISTS db_version (
+            version INTEGER PRIMARY KEY NOT NULL,
+            description TEXT NOT NULL,
+            applied_at INTEGER NOT NULL
+        );
+        SELECT COALESCE(MAX(version), 0) FROM db_version;
+    "
+}
+
+# Get current version
+CURRENT_VERSION=$(get_current_version)
+
+# Check if migration should be applied
+if [ "$CURRENT_VERSION" -lt 2 ]; then
+  # Check if previous pool ID is provided for v2 migration
+    if [ -z "$PREV_POOL_ID" ]; then
+        echo "Error: Previous pool ID is required for v2 migration."
+        echo "Please provide it using the -p or --prev-pool-id option."
+        exit 1
+    fi
+    echo "Applying migration to version 2..."
+
+ # First run the migration script to create the new tables
+    sqlite3 "$DB_PATH" < "$V2_MIGRATION_SCRIPT"
+    MIGRATION_RESULT=$?
+
+    # Check if migration was successful
+    if [ $MIGRATION_RESULT -eq 0 ]; then
+        echo "Migration to version 2 completed successfully."
+
+        # Update the pool_id in all tables
+        echo "Updating pool_id to '$PREV_POOL_ID' in all tables..."
+
+        sqlite3 "$DB_PATH" "
+            UPDATE users SET pool_id = '$PREV_POOL_ID' WHERE pool_id = 'default_pool';
+            UPDATE auctions SET pool_id = '$PREV_POOL_ID' WHERE pool_id = 'default_pool';
+            UPDATE filled_auctions SET pool_id = '$PREV_POOL_ID' WHERE pool_id = 'default_pool';
+        " 
+
+        # Check if migration was successful
+        if [ $? -eq 0 ]; then
+            echo "Successfully updated pool id to $PREV_POOL_ID."
+        else
+            echo "Error: Failed to update pool id."
+            exit 1
+        fi
+    fi
+fi