Skip to content

Commit

Permalink
# This is a combination of commits.
Browse files Browse the repository at this point in the history
# This is the commit message chainbase-labs#1:

✨ feat: New manuscript GUI client (chainbase-labs#54)

* ✨ feat: New manuscript GUI client
# This is the commit message chainbase-labs#2:

✨ feat: New manuscript GUI client

# This is the commit message chainbase-labs#3:

 ✨ feat: modify the chain display page to add ticker (chainbase-labs#57)

* ✨ feat: Modify the chain display page to add ticker
# This is the commit message chainbase-labs#4:

✨ feat: Modify the chain display page to add ticker

# This is the commit message chainbase-labs#5:

🔧 Fix: modify install script for broader compatibility (chainbase-labs#59)

* 🔧 Fix: modify install script for broader compatibility

I updated the bash install script so that it would work in my Fedora 41 environment. It still likely needs testing in other environments, but the key differences will be outlined in the PR related to this commit. The key differences include permissions handling, tmp folder checking, and more robust error handling.

* 🔧 Fix: fixed error handling on /tmp/ for install.sh

This commit adds additional error handling, ensuring that the script exits gracefully even when /tmp/ dir clearance is not allowed due to permissions errors. This is particularly relevant  in distros using SELinux which prevents users from deleting directories they don't own, even with sudo. Now, clean up is handled or user is handed an error message and the script continues to run as intended.
# This is the commit message chainbase-labs#6:

docs(contributor): contrib-readme-action has updated readme
# This is the commit message chainbase-labs#7:

🔧 [fix] Fix Port Mapping Issue (chainbase-labs#62)

* 🐛 Fix: Add port initializiation for Docker Compose

This commit adds significant logging for debugging as well as port initialization if a specified port is not found in the existing config file.

* 🚑️ Add port validation before manuscript deployment

This commit validates and assigns ports before deploying manuscripts, ensuring that ports for graphQL , the database, and the manuscript itself are all properly assigned.

* ♻️ [chore] Move Port Verification to Pkg

This commit moves the port verification out of deploy manuscript and into the common_command.go file in the pkg package. The idea would be to gradually migrate common methods to a single source so that the script is easier to follow and debug.

* 🐛 [fix] Fix overlapping ports upon manuscript init

This commit modifies GetListeningPorts() in common_command.go so as to make it more robust. Instead of just looking at  system ports. It evaluates docker container ports and system ports comprehensively to ensure that port conflicts don't occur. It also continues the migration of system functionality to the common_command.go inside pkg package for better encapsulation.

* 🎨 DRY out createDockerComposeFile function

Applied principle of D.R.Y. - don't repeat yourself - to init_manuscripts.go. Now, instead of reusing similar logic to assign ports, it uses the common pkg.InitialzePorts function on the manuscript  to assign new ports. This is a significantly cleaner implementation as it means that we'll only need to change the logic  in one location going forward.

* 🎨 Clean up duplicate FindAvailablePort definition

Due to previous commmits, the FindAvailablePort() is now defined in pkg/common_command.go and no longer needs to be put here in init_manuscript.go. It is now being removed for clarity.
# This is the commit message chainbase-labs#8:

✨feat: Increase Solana data capability (chainbase-labs#64)

* ✨feat: Increase Solana data capability
# This is the commit message chainbase-labs#9:

docs(contributor): contrib-readme-action has updated readme
# This is the commit message chainbase-labs#10:

Refactoring flink code logic and adding batch mode (chainbase-labs#56)

* feat:Refactoring flink code logic and adding batch mode

* docs(contributor): contrib-readme-action has updated readme

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
# This is the commit message chainbase-labs#11:

docs(contributor): contrib-readme-action has updated readme
# This is the commit message chainbase-labs#12:

✨ feat: render sql with params

docs(contributor): contrib-readme-action has updated readme

✨ feat: UI optimization (chainbase-labs#68)

✨feat: ui Optimization

docs(contributor): contrib-readme-action has updated readme

:wrench: fix: Revamped `help <cmd>` and `--help` flag  (chainbase-labs#69)

* ✨ Feat: Modify help command to display version

Version was previously omitted from the list of available commands. Having it explicitly listed is an ease-of-use thing for developers.

* 🚚 [chore] Rename commanderr.go to commander.go

This commit fixes a typo where commander had two trailing r's instead of a single one. This is a purely cosmetic fix and shouldn't meaningfully affect the functionality of the manuscript-cli.

* 🎨 [chore] group initialized cmds into logical groups

Reorganizes the body of the init() function in commander.go to put a logical order where commands go so that it is more easily followed. Also adds comments to better describe what's happening.

* 📝 [docs] Add Help Context to all Cmds

This commit adds Long text and Example text to each command which more adequately describes their functions and capabilities.

* ♻️ [refactor] Reorganize help system using cobra templates

This change is designed to make the help system more maintainable and consistent. I replaced custom help function with native cobra templates. I created a separate `help_template.go` to keep separation of concerns. And I removed the fragile manual command listing and formatting.

* 🎨 [chore] Beautify help output

This commit makes help output more concise and more beautiful - adding emojis and simplifying  some of the text in help so as to be more accurate.

* 💄 [fix] Disable alphabetical command sorting

This commit ensures that commands are displayed by the help message in the logical order they were added as opposed to in alphabetical order sorted by Cobra.

docs(contributor): contrib-readme-action has updated readme

📝 doc: adjust the readme regarding gui content (chainbase-labs#73)

📝 doc: adjust the readme regarding gui content (chainbase-labs#73)

📝 doc: adjust the readme regarding gui content

Fix: Failed to parse JSON response: reqwest::E┃ror { kind: Decode, source: Error("missing field transactionLogs", line: 1, column: 10452) } (chainbase-labs#74)

docs(contributor): contrib-readme-action has updated readme

🔧 fix: fix the issue of not finding the configuration file. (chainbase-labs#75)

:sparkles: feat: enhance cli version cmd (chainbase-labs#76)

* ✨ Feat: introduce new version --verbose flag

This commit introduces a `--verbose`/`-v` flag for the version command and the top-level logic for the version command to handle verbosity requests.

* ✨ Feat: add version verbose logic

This commit adds `version_manuscript.go` as a new file to the repo. Within, new structs are defined for handling version information for manuscript-core, docker, docker desktop,as well as the build time environment of rust, and go. Additionally, versioning information is  pretty-printed to the console & emojified!

* 🔖 [chore] bump version number to 1.1.0

The CLI was still displaying the old version number (1.0.4) ; this commit bumps the version var in the commander.go package so that the version command will display the proper version.

* 💡 [chore] update build information about ldflags

There was an error previously. Wording  did not contain the correct path for the `go build`instructions in the source code comments of `version_manuscript.go`

:hammer: fix: repair broken init_manuscript table selection (chainbase-labs#82)

* 🐛 Fix: fix database table selection

 This commit reworks and simplifies the selectTable() function in `init_manuscript.go` - this function was previously demonstrating problematic behavior by showing the same table set over and over regardless of which chain was selected.

* ⚰️ [chore] remove version from docker compose template

Per the repeated warning, docker compose does not require version specification in the template. I have removed that to get rid of the pesky warning.

:hammer: feat: add Makefile for Unified Build System (chainbase-labs#83)

* 🔨 Feat: add Makefile for CLI & GUI

This commit adds a makefile which can be used to install either the CLI, the GUI, or both. It also allows for a single interface to interact with both rust and go commands respectively.

* 🔨 Fix: remove build dependency for make install

Removed where install automatically built both binaries. Separate install and build steps. Running build commands with sudo causes problems because sudo has a different environment

* 🔨 Feat: add sophisticated error handling in Makefile

This adds significant error handling in each step of the Makefile. Install verifies that binaries are in the right location. Build verifies that binaries are created.  Permissions are checked to prevent permissions errorsand most failures are handled gracefully. This can also help with debugging any issues during builds by giving this detailed user feedback

Fix: modify createKafkaSink (chainbase-labs#87)

* Fix: modify createKafkaSink

* Fix: remove annotation

🔨 fix: GUI runtime panic caused by window size (chainbase-labs#89)

* 🔨 fix: GUI runtime panic caused by window size

docs(contributor): contrib-readme-action has updated readme

🔧 fix: initial creation of ms no status alerts (chainbase-labs#90)

* 🔧 fix: initial creation of ms no status alerts

✏️ doc: new makefile supports local compilation and installation of the client (chainbase-labs#91)

✏️ doc: add configuration file and data directory description (chainbase-labs#92)

✏️ doc: change the color and style of the homepage image. (chainbase-labs#93)

✏️ doc: change the color and style of the homepage image

* ✏️ doc: change the color and style of the homepage image.

* ✏️ doc: change the color and style of the homepage image.

✏️ doc: modify installation document prompts and scripts

✏️ doc: modify installation document prompts and scripts

✏️ doc: modify installation document prompts and scripts

:bug: fix: eliminate zombie jobs (chainbase-labs#96)

* 🎨 [chore] create chainEndpoint constant

This commit moves the hard-coded network chains endpoint into a `chainEndpoint` constant for better visibility and configurability.

* ✨ Feat: add manuscript state machine

Added a Manuscript State Detector which can be instanciated in the ListJobs() function for sake of identifying manuscript state with accuracy and finality.  The state machine executes a series of elaborate looks at container status and flink logs to  update to one  of five states: RUNNING, INITIALIZING, FAILED, STOPPED, and UNKNOWN (rare). This is the grounds for more advanced manuscript detection from the CLI.

* ♻️ [chore] refactor listjobs to accomodate state machine

This commit adds new logic to jobs_manuscipt that allows for the detection of manuscripts by using `manuscript_state.go`

* 💡 [chore] add comments to state machine

Recognizing that there is a lot of complicated logic in the state detector, I added comments explaining its general operation throughout.

* 🐛 Fix: enhance manuscript state detection

Within the DetectState method, manuscripts previously defaulted to initializing state, despite surviving all previous checks. Now, if a manuscript survives all checks it will display as `RUNNING` in the jobs list.

* 🦺 [chore] add graphQL check to state detector

This adds as an additional state check - the checkGraphQLEndpoint function which hits the local endpoint to determine healthiness and ensure it is live before returning RUNNING. If it is determined not to be live, but survive all other checks, the manuscript will return INITIALIZING status.

* 📝 [chore] add additional help documentation for cli `list` cmd

This commit adds additional status indicators to the long help text of the list command.

* ✨ [feat] add alias `ls` to cli `list` command

This commit adds an alias, `ls` to the list command.

* 🦺 [chore] add guard rails for cli init

This commit adds additional guard rails to fix broken functionality of `init` command that was affecting the jobs list. Previously, a person could init and partially overwrite an existing manuscript. Now, extensive checks are in place to prevent that from happening: manuscript is looked for in the directory of the manuscript path, the manuscript is already running, the manuscript is in the config file.

* ✨ [feat] enhance `manuscript-cli list` cmd

This commit adds new functionality to manuscript-cli list. It now accepts a directory as an argument. If no directory is specified it looks in the default directory for users. It also shows STOPPED jobs by looking at the manuscript yamls in the subdirectories of the specified directory and showing ALL manuscripts. This makes it more informative and usable.

✏️ doc: create --verbose or --debug flag for version command

✏️ doc: create --verbose or --debug flag for version command

🔧 fix: fix port occupancy issue (chainbase-labs#102)

✨ feat: improve repo organization & separate cli and gui (chainbase-labs#103)

✨ feat: improve repo organization & separate cli and gui

🔧 fix: fix network port conflicts caused by multiple jobs (chainbase-labs#104)

🔧 fix: graphQL endpoint is not picking up data (chainbase-labs#105)

🔧 fix: modify the release build directory

🔧 fix: modify the release build directory

🔧 fix: modify the release build directory

🔧 fix: modify the release build directory

🔧 fix: modify the release build directory

✏️ doc: update the latest version number

🔖 fix: bump Makefile version tag (chainbase-labs#106)

Update the version tag in the Makefile to be commensurate with the version number in both CLI and GUI files.

🔨 fix: index outside of buffer (chainbase-labs#107)

* 🔨 fix: index outside of buffer

📝 chore: fix issue template environment text (chainbase-labs#109)

This updates the environment text to be more clearly reflective of the desired input.

✨ feat: add `config` command to CLI (chainbase-labs#108)

* 🚨 Fix: changed Println to Printf in jobListCmd

Eliminate simple linter warning about returning value using `Println` function.

* ✨ Feat: add new `config` command to CLI

Added a new `config` command to CLI - with following sub-commands: `config show` , `config clean` . It is the basis for a user to look at and alter settings of their configuration from the command line without having to navigate the OS.

* ✨ Feat: add new functionality to `config clean` CLI cmd

This commit adds multiple quality of life enhancements for manuscript removal from the config.ini.  the clean command can now specify specific manuscripts, bypass interactive mode, remove all manuscripts from a config file, etc.

* ✨ Feat: add `--summary` flag to CLI `config show` cmd

This additional flag prints out a beautified summary of the config file which shares where manuscripts are located - what they're configrued to/some basic information output.  Implemented as a useful debugging tool.

* 🎨 Chore: clean up commander.go

This commit adds additional structure to commander.go to keep it clean as the number of commands increases. In particular, it adds comments and breaks things into helper functions that clarify things throughout.

* 🎨 [chore] Add comments and fix nomenclature

This commit renames ShowConfigSummary to ConfigShowSummary for consistency of the config_manuscript.go file. IT also adds notations and comments throughout so that the logic being used is more apparent to first time readers.

✨ feat: add non-interactive manuscript `init` CLI command (chainbase-labs#110)

* ✨ Feat: add non-interactive CLI `init` logic

This commit adds the basic functionality to generate a manuscript non-interactively to `init_manuscript.go.` It is a preliminary action and violates some DRY
principles for ease of use and simple implementation. In future versions, it will be wise to
refactor both `init` paths to use common helper funcs.

* ✨ Feat: add `init` non-interactive and flags to CLI

This commit adds flags and pre-run validation to the non-interactive version of the `init` command. It greatly expands the functionality of the command to be able to take arguments directly.

* 🐛 Fix: manuscripts create in proper directory

This commit fixes the `init` non-interactive to ensure manuscripts are created in the proper place.

✨ feat[flink-etl]: Add support for configurable savepoint and checkpoint directories
✨ feat[flink-UDF]: new UDF GetTokenMeta
🔧 fix[flink-etl]: Resolve dependency conflict
🎨 docs[example]: add readme
🔧 fix[manusctipt-cli]: icp's manuscript
  • Loading branch information
fishTsai20 committed Jan 27, 2025
1 parent a9746bf commit fa21d7c
Show file tree
Hide file tree
Showing 91 changed files with 11,923 additions and 715 deletions.
9 changes: 7 additions & 2 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,12 @@ body:
label: Environment
description: Tell us about your environment
placeholder: >
Tell us what you're using it: Docker, Flink, Manuscript etc.
Please provide:
- manuscript-core version
- Docker Desktop Version
- System OS Information
If manuscript-cli is installed, run `manuscript cli version -v` and paste the output here.
validations:
required: true

Expand All @@ -44,4 +49,4 @@ body:
label: ❌ Actual Behavior
placeholder: What happened instead?
validations:
required: false
required: false
17 changes: 0 additions & 17 deletions .github/workflows/codespell.yml

This file was deleted.

9 changes: 2 additions & 7 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,19 +27,13 @@ jobs:
go-version: 1.22
- name: Get Manuscript CLI Version
run: |
cd cli && \
MANUSCRIPT_CLI_VERSION='v'$(go run main.go version | tr -s ' ' | awk '{print $NF}')
echo "MANUSCRIPT_CLI_VERSION=$MANUSCRIPT_CLI_VERSION" >> $GITHUB_ENV
- name: Get Git Tag Version
run: |
Git_TAG_VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')
echo "Git_TAG_VERSION=$Git_TAG_VERSION" >> $GITHUB_ENV
- name: Check if versions match
run: |
if [ "${{ env.MANUSCRIPT_CLI_VERSION }}" != "${{ env.Git_TAG_VERSION }}" ]; then
echo "Versions do not match!"
exit 1
fi
echo "Versions match!"
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
with:
Expand All @@ -48,5 +42,6 @@ jobs:
# 'latest', 'nightly', or a semver
version: "~> v2"
args: release --clean
workdir: cli
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
41 changes: 0 additions & 41 deletions .github/workflows/test.yml

This file was deleted.

9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
.idea
.Ds_Store
flink/runner/target
flink/runner/libs
flink/runner/checkpoint
examples/*/data/
manuscript/
dist/
flink/data/
flink/runner/runner.iml
flink/lib
flink/checkpoint
flink/data
flink/lib
flink/flink-1.18.1
156 changes: 156 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Variables
VERSION ?= 1.1.1
GIT_COMMIT=$(shell git rev-parse --short HEAD)
BUILD_DATE=$(shell date -u '+%Y-%m-%d:%H:%M:%S')
GO_VERSION=$(shell go version | cut -d' ' -f3)
RUST_VERSION=$(shell rustc --version | cut -d' ' -f2)
BINARY_NAME_CLI=manuscript-cli
BINARY_NAME_GUI=manuscript-gui

# Go build flags matching version_manuscript.go
GO_LDFLAGS=-ldflags "\
-X manuscript-core/commands.gitCommit=$(GIT_COMMIT) \
-X manuscript-core/commands.buildDate=$(BUILD_DATE) \
-X manuscript-core/commands.goVersion=$(GO_VERSION) \
-X manuscript-core/commands.rustVersion=$(RUST_VERSION)"

# Directory Locations
CLI_DIR=cli
GUI_DIR=gui

# Default to help when nothing is specified by user
.DEFAULT_GOAL := help

# Validate version format
VERSION_PATTERN := ^[0-9]+\.[0-9]+\.[0-9]+$$
$(if $(shell echo "$(VERSION)" | grep -E "$(VERSION_PATTERN)"),,\
$(error "💥 Error: Invalid version format. Format should conform to X.Y.Z"))

# Build both binaries with `make all`
.PHONY: all
all: cli gui
@if [ ! -f "$(CLI_DIR)/$(BINARY_NAME_CLI)" ] || [ ! -f "$(GUI_DIR)/target/release/$(BINARY_NAME_GUI)" ]; then \
echo "💥 Error: One or more binaries failed to build"; \
exit 1; \
fi
@echo "🎉 All builds succeeded"
@echo "📦 CLI binary: $(CLI_DIR)/$(BINARY_NAME_CLI)"
@echo "📦 GUI binary: $(GUI_DIR)/target/release/$(BINARY_NAME_GUI)"

# Build only CLI with `make cli`
.PHONY: cli
cli:
@echo "🚀 Building CLI..."
@echo "🔖 Git Commit: $(GIT_COMMIT)"
@echo "📅 Build Date: $(BUILD_DATE)"
@echo "🐹 Go Version: $(GO_VERSION)"
@echo "🦀 Rust Version: $(RUST_VERSION)"
@cd $(CLI_DIR) && go build $(GO_LDFLAGS) -o $(BINARY_NAME_CLI)
@if [ $$? -ne 0 ]; then \
echo "💥 Error: CLI build failed"; \
exit 1; \
else \
echo "✅ CLI build succeeded"; \
fi

# Build only GUI with `make gui`
.PHONY: gui
gui:
@echo "🚀 Building GUI..."
@cd $(GUI_DIR) && cargo build --release
@if [ $$? -ne 0 ]; then \
echo "💥 Error: GUI build failed"; \
exit 1; \
else \
echo "✅ GUI build succeeded"; \
fi

# Clean build artifacts with `make clean`
.PHONY: clean
clean:
@echo "🧹 Cleaning build artifacts..."
@cd $(CLI_DIR) && go clean || { echo "💥 Error: go clean failed"; exit 1; }
@rm -f $(CLI_DIR)/$(BINARY_NAME_CLI) || { echo "💥 Error: Failed to remove CLI binary"; exit 1; }
@cd $(GUI_DIR) && cargo clean || { echo "💥 Error: cargo clean failed"; exit 1; }
@echo "✅ Clean succeeded"

# Run respective tests with `make test`
.PHONY: test
test:
@echo "🧪 Running tests..."
@cd $(CLI_DIR) && go test ./...
@if [ $$? -ne 0 ]; then \
echo "💥 Error: CLI tests failed"; \
exit 1; \
fi
@cd $(GUI_DIR) && cargo test
@if [ $$? -ne 0 ]; then \
echo "💥 Error: GUI tests failed"; \
exit 1; \
else \
echo "✅ All tests passed"; \
fi

# Install binaries to /usr/local/bin with `make install`
.PHONY: install
install: install-cli install-gui
@if [ ! -f "/usr/local/bin/$(BINARY_NAME_CLI)" ] || [ ! -f "/usr/local/bin/$(BINARY_NAME_GUI)" ]; then \
echo "💥 Error: Installation failed - one or more binaries not installed"; \
exit 1; \
fi
@echo "🎉 All installs succeeded"
@echo "📦 CLI binary: /usr/local/bin/$(BINARY_NAME_CLI)"
@echo "📦 GUI binary: /usr/local/bin/$(BINARY_NAME_GUI)"

# Install only CLI binary to /usr/local/bin with `make install-cli`
.PHONY: install-cli
install-cli:
@echo "📦 Installing CLI binary..."
@if [ ! -w "/usr/local/bin" ]; then \
echo "💥 Error: Insufficient permissions. Try using sudo."; \
exit 1; \
fi
@if [ ! -f $(CLI_DIR)/$(BINARY_NAME_CLI) ]; then \
echo "💥 Error: CLI binary not found. Please build first."; \
exit 1; \
fi
@cp $(CLI_DIR)/$(BINARY_NAME_CLI) /usr/local/bin/
@if [ $$? -ne 0 ]; then \
echo "💥 Error: CLI install failed"; \
exit 1; \
else \
echo "✅ CLI install succeeded"; \
fi

# Install only GUI binary to /usr/local/bin with `make install-gui`
.PHONY: install-gui
install-gui:
@echo "📦 Installing GUI binary..."
@if [ ! -w "/usr/local/bin" ]; then \
echo "💥 Error: Insufficient permissions. Try using sudo."; \
exit 1; \
fi
@if [ ! -f $(GUI_DIR)/target/release/$(BINARY_NAME_GUI) ]; then \
echo "💥 Error: GUI binary not found. Please build first."; \
exit 1; \
fi
@cp $(GUI_DIR)/target/release/$(BINARY_NAME_GUI) /usr/local/bin/
@if [ $$? -ne 0 ]; then \
echo "💥 Error: GUI install failed"; \
exit 1; \
else \
echo "✅ GUI install succeeded"; \
fi

# Help target
.PHONY: help
help:
@echo "Available targets:"
@echo " 🔧 all - Build both CLI and GUI"
@echo " 🐹 cli - Build only the CLI"
@echo " 🦀 gui - Build only the GUI"
@echo " 🧹 clean - Clean build artifacts"
@echo " 🧪 test - Run tests for both projects"
@echo " 📦 install - Install both binaries to /usr/local/bin"
@echo " 📦 install-cli - Install only the CLI binary to /usr/local/bin"
@echo " 📦 install-gui - Install only the GUI binary to /usr/local/bin"
Loading

0 comments on commit fa21d7c

Please sign in to comment.